History, Research, and Challenges: A Systematic Analysis of Peer Review for Journals, Grants, and Faculty Appointments

Peer review is a self-regulation mechanism for scientific inquiry. Institutionalized and incorporated into the structure and operation of science, it has received considerable support in the academic setting. The legitimacy of peer review is based on trust and integrity. In various ways, it allocates scarce resources such as journal space, research funding, faculty recruitment, recognition, and rewards for academic achievements. But there are growing indications that peer review has yet to fulfill its potential functions, leading to negative assessments as to whether it is effective, efficient, or reliable. Many studies have found links between potential sources of bias and judgments in peer review and expressed reservations over the fairness of the process. It is, therefore, important that the peer review process should be subjected to serious scrutiny and regular evaluation that would lead to better quality and greater fairness. This study presents a systematic review of the empirical literature on peer review of journal manuscripts, grant applications, and faculty appointments and promotions. Historical and contextual information is provided as a basis for interpreting this review. Finally, the authors discuss international recommendations for good practice in peer review and the potential and problems of peer review and bibliometrics. [Article content in Chinese