One key rating of importance of an academic medical journal is the impact factor. Journals are judged both by the impact factor score and by how it compares with closely related (competitive) journals. The size of the impact factor for Academic Psychiatry has been gradually increasing over recent years (Figure 1
). The latest available score of 0.914 (which was published in 2006) was the highest achieved so far. In 2005 the impact factor was 0.808, and in 2004 it was 0.657. When compared to other education journals, Academic Psychiatry compares similarly to Teaching and Learning in Medicine and to Medical Teacher but performs considerably less well than Academic Medicine and Medical Education (Figure 2
).
It could be argued that comparing the impact factor of Academic Psychiatry to those of other journals is not valid because Academic Psychiatry is the only international journal serving clinician educators and leaders in psychiatric education. Articles of interest to our readership, however, are sometimes published in other education journals or general psychiatric journals. Because impact factors can determine where authors choose to publish, editors must seek to boost the impact factor to service readers by publishing the best available research.
To this end, our goals for this editorial are to describe briefly the impact factor, including its strengths and limitations in relation to other indices of a journal’s performance. We will discuss what Academic Psychiatry might reasonably do to improve its recognition and standing. Although our comments will be limited to the role of the impact factor, the overall goal, of course, is to achieve standards of excellence in every facet of the Journal’s operations.
The impact factor is published annually by the Institute of Scientific Information in its Journal Citation Reports. It provides a measure of the frequency with which the “average article” from two preceding years is cited in a more recent year. It is determined by the number of cites to articles in a particular (current) year divided by the number of substantive articles published over the two preceding years (1). Thus, the 2006 impact factor for Academic Psychiatry was determined by the number of cites (in 2006) across journals divided by the articles published in Academic Psychiatry in 2004 and 2005. The numerator was 96 (number of citations of articles published), and the denominator was 105 (number of articles published). The resulting impact factor of 0.914 indicates that articles published in 2004 and 2005 were cited on average about once each in 2006.
Of note is that Academic Psychiatry published many more articles of all types than the 105 constituting the denominator of the impact factor. Although it is not altogether clear how decisions are made on what is included as substantive, or citable, articles and whether decisions are consistently applied (2), presumably editorials, correspondence, commentaries, and articles without abstracts are excluded from consideration in the denominator. Journals are thus able to manipulate the impact factor when case studies, commentaries, and other reports are published without abstracts. For interested readers, the rank orders of the most frequently cited and electronically accessed articles are available at the Journal’s web site (http://ap.psychiatryonline.org/); these are derived from all articles as opposed to only citable articles.
It should be appreciated that the impact factor is not an absolute measure and does not by itself confer meaning (3). The impact factor has utility only in comparing journals from similar fields as opposed to different fields (3). The impact factor has been attacked as flawed because the number of citations of a particular article does not necessarily reflect the quality of that same article (4), and citation analysis does not necessarily concur with peer judgment of quality (5). Indeed, some commentators have noted that articles with obvious deficiencies are more likely to be cited because researchers will want to show how their own research improves on the identified deficiencies of that earlier research (4, 5). Citation measures can also be skewed by the publication of one article with an exceptionally high citation rate (6). Other shortcomings include a liability to manipulation of the impact factor and misapplication by a presumption that readers, promotion committees, or grant review committees will be biased in their judgment of quality of research by the impact factor of the journal within which the research is published (4). The impact factor, in fact, does not permit quality assessment of a single article (5). In one study of a cohort of published research, commonly used measures of study methodology and design did not predict the frequency of citations or the importance of citing journals (7). These same authors noted, however, that the finding may have reflected a lack of accuracy of quality assessments among other possible limitations so that the findings should be interpreted with caution.
We recognize that there are many other considerations in the “success” of a scientific journal. These might include the extent to which the relevant audience reads the journal and incorporates the best evidence into practice. The impact factor does not measure the utility or effect of articles on educational leadership, educational interventions, or related clinical outcomes. These considerations are obviously weighted more toward the clinician educator than the researcher or scholar. There are, indeed, a number of alternative measures to the impact factor, including the Journal Performance Index, which is like the impact factor but excludes citations to nonscholarly articles (1, 3). A very recent proposal recommended creating a standardized and multidimensional measure of merit that includes the impact factor and new indicators such as the impact on clinical practice, medical education, and health policy (8). Nevertheless, the impact factor continues to be the most established and widely used metric and, as such, demands our sustained attention.
As indicated in Table 1
, there are a number of ways for the Journal to improve the score of the impact factor. For one, the likelihood of citation of articles surely relates to their ease of readability and to quality. Quality, in turn, is dependent on the adequacy of description of methods and findings in addressing the identified educational problem. As we previously argued (9), improving the standards of conduct and communication of educational research with a particular emphasis on the methods should promote the integrity of the scientific basis of academic psychiatry. Another method for improving the impact factor is to be proactive in addressing salient issues in medical education and to communicate and collaborate with authors, reviewers, and editors of other education journals to cultivate interest in academic psychiatry. In this vein we hope to encourage our colleagues with whom we collaborate on interdisciplinary teaching projects to consider publishing in Academic Psychiatry.
One current initiative is to broaden the international status and impact of the Journal (10, 11). As Balon et al. (11) argued, information about medical education is beginning to flow in many directions, including toward the United States, and we must embrace this process. We have introduced an international column and an international advisory board in part to foster international relationships. We intend for the Journal to occupy a more prominent leadership role in international settings.
The readership has also been expanded by the publication of an electronic, web-based version of the Journal. In addition, published articles have been incorporated into MEDLINE since 2001, markedly increasing their visibility. Expanding the readership both nationally and internationally in these ways should contribute to the influence of individual articles and be reflected by the impact factor in due course.
More specifically, we encourage all authors to assiduously ensure that all relevant articles are accurately and appropriately cited, especially those that are published in Academic Psychiatry. Citing all relevant work is important in assisting readers’ appreciation of the full context of the research, especially how it adds to that which is already published. In addition, we will continue to examine decisions about when to include abstracts. Generally abstracts for commentaries, educational case reports, and resource columns have not been included, and this policy should continue.
We also encourage systematically collected reviews on topics of interest or importance; these may be cited more frequently than research papers (3, 6). We will continue to seek to develop theme issues serving to foster research and scholarship and to remedy identified deficits in the literature. Another option for improving the impact factor includes reducing the number of articles published (thereby reducing the denominator). This is not, however, a policy that we will adopt during this transitional period of growth and expansion. Nor will we include an article’s potential to be cited as a publication criterion, as others have done (12), because we recognize that some educational problems and methods, even if more specialized or less well-appreciated, are nevertheless equally worthy of attention.
Furthermore, we are committed to improving the editorial processes of the Journal by reducing the review and editorial decision-making and publication times and by improving authors’ experiences. To this end we welcome any comments on how these processes can be improved, and we might seek to gather information by a formal survey of authors in the future. Currently, there is too long a delay from acceptance of a manuscript to publication, and we are working on remedying this problem. We want to be sure that authors feel valued and receive high-quality feedback (13). As such we hope that all those who submit will want to continue to resubmit.
In conclusion, the editorial processes of the Journal will be governed, in part, by our determination to improve the impact factor of the Journal. Our commitment extends to improving all facets of review and editorial decision-making processes. We believe that we are accountable to our readership and that strengthening the Journal’s impact factor is one potentially valuable objective measure. We aim to improve the recognition of the Journal both nationally and internationally. We are also committed to assisting authors to communicate effectively through the peer review and editorial processes.
We seek a balanced process that does not undermine the priority of our primary missions of publishing scholarly work focused on innovative psychiatric education, professional development in academic psychiatry, and leadership. We will continue to grow the Journal, both nationally and internationally, by increasing the number of publications. We will also continue to publish articles on less well-known or appreciated topic areas by cultivating a richness and progressively more rigorous diversity of educational research and scholarship. As exemplified by the wonderful collection of articles published in this issue, our future seems very bright indeed. Again, we welcome readers’ counsel as to how we might achieve all of these goals.