Reviewers perform extraordinarily important work for our journal. They evaluate the rigor and relevance of scholarly papers under consideration by Academic Psychiatry. They craft thoughtful, timely critiques offering guidance to authors and editors. They ensure that the scholarship of colleagues reflects the scientific and ethical standards of the profession of psychiatry. Most important, they give the gifts of time, honesty, and mentorship to authors who take on the hard task of sharing their work. For these reasons, peer reviewers assume substantive responsibilities in the publication process, and their role involves knowledge, judgment, and generosity.

The criteria and practices used in reviewing manuscripts in medical education are patterned on those across the fields of medicine and science. For conceptual and literature reviews, the criteria include clarity and relevance of the topic, fairness and diligence in representing the key concerns and questions in the area, quality of the presentation, and value of the work in advancing our understanding within the field. For research reports, review criteria encompass the following: problem statement, conceptual framework, and research question; references to the literature and documentation; relevance; research design; instrumentation, data collection, and quality control; population and sample; data analysis and statistics; reporting of statistical analyses; presentation of results; discussion and conclusion; interpretation; title, authors, and abstract; presentation and documentation; and scientific conduct (+1).

Professional practices of peer reviewers of medical education literature were recently articulated as having six essential features (+2). Reviewers are to treat the existence and substance of a manuscript as confidential. They are to evaluate only those manuscripts that fall within their scope of expertise. Reviews should be collegial and constructive in intent and in tone. The reviewer cannot plagiarize the manuscript. The reviewer must "disclose, avoid, or appropriately manage" conflicts of interest. Finally, the reviewer is expected to avoid biases that influence the scientific basis for evaluating the paper.

So how does this work in reality? To answer this question, Bordage (+3) recently performed a qualitative content analysis of the peer review process for research reports in medical education. He examined reviewers' positive and negative comments on 151 research manuscripts submitted in 1997 and 1998 to the Research in Medical Education (RIME) proceedings. On average, 4 reviewers evaluated each manuscript, and the assigned reviewers were unanimous in recommending acceptance for only 28 of the 151 papers (19%). Reviewers proffered far fewer positive comments (n=252) than negative comments (n=1,053), and, interestingly, nearly 40% of reviewers recommended rejection of manuscripts but gave no "unsatisfactory" ratings on the reviewer form checklist. The top four negative comments in this data set pertained to "scientific" considerations, i.e., statistical flaws, overinterpretation of results, poor instrumentation, and small or biased sampling. The top three positive comments in this data set related to significance, caliber of the writing, and study design.

For Academic Psychiatry, my sense is that the review process reflects a genuine commitment to nurturing inexperienced, tentative, or even awkward writers. The rejection rate for manuscripts has remained stable at approximately 50%, and it is clear that our reviewers make a point of providing at least some favorable comments in every critique. We often receive mixed recommendations on papers, and we often require multiple revisions and re-review prior to publication. In research reports, frequent problems pertain to flawed or modest studies with results that are difficult to interpret. In conceptual papers, key obstacles that arise are the presentation of opinion as fact and the lack of diligence in reviewing the relevant literature. With our early-career reviewer program, in which a resident or junior faculty person is included as a reviewer on every manuscript we consider, there may be remarkable variability in the critiques. (Usually our young reviewers are more thorough but also more generous in their assessments!) As with the Academic Medicine study, the overall impression of the acceptability of a paper for our journal is not always reflected in the responses on the reviewer checklist. Occasionally, the comments to the authors will be very benign and superficial while the narrative for the editors is quite negative and the quantitative rating very weak. On the other hand, it is extremely rare that we receive reviews that are professionally inappropriate or that include gratuitous or ad hominem comments.

In coming years reviewers will take on new challenges, ranging from navigating Web-based critique procedures to offering assessments of the adequacy of regulatory compliance evidenced in data-based papers. Reviewers will also be held to a new level of accountability in the process of evaluating manuscripts. For instance, many journals have begun to ask explicitly about potential conflicts of interest in reviewing papers. A few journals have introduced a practice in which reviewers are offered the option of disclosing their identity to manuscript authors. These new approaches will place a greater burden squarely on the shoulders of reviewers, who will fulfill an ever more important role in scholarship. For their service, past, present, and future, we at Academic Psychiatry thank them.