Reviewing Peer Review: A Quantitative Analysis of Peer Review

Date Added: Feb 2010
Format: PDF

In this paper, the authors focus on the analysis of peer reviews and reviewers behavior in a number of different review processes. More specifically, they report on the development, definition and rationale of a theoretical model for peer review processes to support the identification of appropriate metrics to assess the processes main properties. They then apply the proposed model and analysis framework to data set from conference evaluation processes and they discuss the results implications and their eventual use toward improving the analyzed peer review processes. A number of unexpected results were found, in particular: the low correlation between peer review outcome and impact in time of the accepted contributions and the presence of a high level of randomness in the analyzed peer review processes.