Each year, residency training directors rely on the information contained within the Electronic Residency Application Service (ERAS) application to provide a summary of each applicant’s academic accomplishments. ERAS provides a useful conduit of information between medical school graduates hoping to find graduate medical education in their specialty of choice, and residency training programs hoping to match the best qualified candidates available. Given the overwhelming volume of data associated with each applicant (educational history, grades, board scores, letters of recommendation, publications, volunteer experience, etc.), ERAS provides a valuable service by organizing this data into a single, relatively manageable, electronic packet; however, ERAS does not, and realistically cannot, verify the data that it reports. While some of the data reported in the ERAS application is provided (and thus verified) by a third party, a large portion of the information found in the ERAS application is simple self-report by the applicant. United States Medical Licensing Examination scores are independently reported by the National Board of Medical Examiners and measures of medical school performance (including grades, class rank, Dean’s letter, and individual letters of recommendation) are independently uploaded to ERAS by the applicant’s medical school. The remainder of the application is entirely based on applicant self-report, and its veracity is based solely on “the honor system.” Some residency training directors may assume that all information provided is accurate. Others, faced with hundreds of applications, each containing a multitude of data points, may feel resigned to relying on the honesty of the applicant, since it would be impossible for them to verify each reported activity or accomplishment across the total field of applicants.
One applicant-reported data-point that is not difficult to verify is peer-reviewed publications. With the advent of online search engines that can efficiently search the expanse of medical literature, an applicant’s list of publications can be verified in a matter of minutes with only a few computer keystrokes. If detected, misrepresentation of the publication record on the ERAS application, whether due to deliberate malfeasance or poor attention to detail, may provide the residency training director with important information regarding the applicant.
Perhaps sparked by increasing concerns over honesty and transparency in academic medicine, a burgeoning literature has recently developed documenting surprisingly high rates of misrepresentation of publications by applicants for training in a wide variety of different fields. The first of these studies by Sekas and Hutson (1) reported that 30.2% (16/53) of applicants to a gastroenterology fellowship who reported published articles had misrepresented them on their application. A number of other studies across disciplines including dermatology, radiology, pediatrics, orthopedic surgery, neurosurgery, internal medicine, emergency medicine, and radiation oncology have followed, with the results summarized in Table 1
(2–11). To our knowledge, no such study has yet been conducted examining applicants to psychiatry residency.
We retrospectively reviewed all applications that were submitted for the 2004–2005 and 2005–2006 academic years to the Harvard Longwood Psychiatry Residency Training Program, a large, multi-institution residency program sponsored by the Departments of Psychiatry at Beth Israel Deaconess Medical Center, Brigham and Women’s/Faulkner Hospitals, and Massachusetts Mental Health Center, all in Boston. Since we do not have access to the data of applicants who did not apply to our program, we are unable to state whether these applicants are representative of the applicant pool as a whole. Since these programs are known to draw applicants with an interest in academic psychiatry, it is possible that these applicants are more likely to have engaged in academic activities (including writing for publication) during medical school. Institutional review board approval was obtained for this study, but, at their insistence, applicants who eventually matched at the Harvard Longwood program, or at the Massachusetts General Hospital/McLean Residency Training Program in Psychiatry (MGH/McLean), another training program with which we have significant contact, were excluded from verification of listed publications in our data analysis for reasons of confidentiality (although their number of publications is reported).
Applications were reviewed for the citation of articles published in peer-reviewed journals. Publications listed in foreign journals that were not available for verification were excluded from the study. Misrepresentation was cited based on: listed articles that did not exist in the literature, articles listed in publications other than the one the article actually appeared in, the inclusion of the applicant’s name as an author of an article when he or she was not so listed on the article itself, and the movement of the applicant’s name further up the list of authors than it actually appeared in the published paper.
Articles listed were verified using a stratified approach. First, the article was searched for using the online search engine PubMed. If the PubMed search failed to locate the article, a repeat search was conducted using the online search engine Google Scholar. If this again failed to locate the article, the cited journal itself was reviewed, either electronically or in paper form. When an article could not be found, all issues of the journal in question dated 12 months before or after the questioned citation were searched. Only when these steps were completed was an article noted as nonexistent. Articles that were listed by the applicants as “submitted” or “in press” were not reviewed, to allow for benefit of the doubt around changes that may have been made to the article or author list in the time between the submission of the ERAS residency application and publication of the article. Abstracts were not counted as publications for the purposes of this study since they are not individually listed in PubMed and the resultant difficulty inherent in verifying each one by hand. Judgments regarding authorship were made by simple comparison of the author list presented by the applicant and that listed on the article itself.
The proportions of publications found to have misrepresentations by United States medical graduates (USMGs) versus those of international medical graduates were compared using either the chi-square test statistic or the two-tailed Fisher Exact Probability Test when an expected cell value was less than five. Since four prior studies provided separate data for rates of publication misrepresentation by USMGs and international medical graduates, the rates were similarly compared within each study. A composite comparison was also made by summing up the numbers of publication misrepresentations by type of medical school graduate.
Of the 697 total applicants to the Harvard Longwood program in the 2 years studied, 486 (69.7%) were USMGs, and 211 (30.3%) were international medical graduates.
Published peer-reviewed articles were reported by 224 applicants (32.1% of the total applicant pool), with the total number of articles per applicant ranging from one to 32 (mean=2.63). This number includes the 56 applicants who ultimately matched at either the Harvard Longwood program or MGH/McLean; as noted above, these applications were not reviewed for publication veracity, but 28 (50%) reported published peer reviewed articles, with a range of 1–9 articles per applicant (mean=2.75). Once these were excluded, 196 (30.6%) of the remaining 641 applicants reported publications in peer-reviewed journals (range=1–32, mean=2.61). Of the 196 reporting publications, 145 (74.0%) of the applicants were USMGs (range=1–32, mean=2.60), while 51 (26.0%) were international medical graduates (range=1–17, mean=2.65).
Review of the 196 applicants citing publications revealed a total of 18 (9.2%) with clear examples of misrepresentation. Of those 18, eight (44.4%) were USMGs, and 10 (55.6%) were international medical graduates. Thus, 5.5% of USMGs citing publications and 19.6% of international medical graduates citing publications were found to have at least one example of misrepresentation. Fourteen applicants (two USMGs, 12 international medical graduates) listed publications in foreign journals that were not available for online verification and, as above, these publications were excluded from the study and not included in the data either as listed publications or misrepresentations.
Of the 18 applicants who misrepresented their publications, seven (38.9%) did so by listing their name further up the author list than it actually appeared in the published article (with the largest elevation being from seventh author to second author), seven (38.9%) did so by citing articles that were not found to exist in the published literature, two (11.1%) did so by adding their name to the author list of articles actually published (in both cases listing themselves as first author), and two (11.1%) did so by reporting an article as published in a more prestigious journal than it actually was (as measured by Journal Impact Factor). Of note, no applicants listing publications were found to have reported themselves lower down the author list than they actually were, and none cited articles as published in less prestigious journals than they actually were. Of the 18 applicants found to have misrepresented their publications, two (one USMG, one international medical graduate) were found to have multiple examples of elevating their name in the list of authors.
Table 2
summarizes the comparisons of publication misrepresentation from this study and the other studies already described. The p values for the two-tailed Fisher Exact Probability Test results are reported for all the individual studies which each had expected cells less than 5. The difference in higher rates of publication misrepresentation by international medical graduates compared to USMGs were statistically significant for applicants in our study of psychiatry residency (p=0.008) and in a prior study of emergency medicine residency (p=0.028). When results for all five programs were combined, the higher rate of publication misrepresentation by international medical graduates (22.2%) compared to USMGs (11.7%) was statistically significant (χ2=4.6, df=1, p=0.033).
The issue of honesty in academic medicine has recently become more prominent in both the professional literature and general media. While most medical schools include some teaching of clinical ethics, fewer place the same focus on ethics in the academic sphere. Our study of psychiatry residency applicants supports the finding of studies in other specialties that a small but substantial number of physicians-in-training have already begun to engage in dishonest academic practices before they complete medical school. When compared to studies from other specialties, the 9.2% misrepresentation rate found in this study of applicants to psychiatry residency falls toward the lower end of the spectrum, although meaningful statistical comparison is confounded by broad differences in methodology between studies (inclusion of abstracts, articles “in press” or “submitted for publication,” etc.).
In their original study of applicants to gastroenterology fellowship, Sekas and Hutson (1) postulated several hypotheses to explain why candidates might misrepresent publications on their applications: to gain a competitive advantage over other applicants, low probability of detection, material gain, the perception that “everyone does it,” innocent error, and “mental aberration.” Attitudes toward the societal acceptability of exaggerating accomplishments may be shifting, with a recent article in the Journal of Business Ethics going so far as to argue for the ethical permissibility of resume embellishment on the basis of relativity (since it is almost certain that some resumes are embellished, it is better that all resumes are embellished to allow for any comparison) (12). Another study categorized lying as a process of everyday social interaction “infused with little cognitive or emotional baggage” (13).
While gaining a competitive advantage seems the most obvious, and perhaps likely, explanation for misrepresentation, the other factors should not be ignored. Concern about being at a competitive disadvantage with USMGs and the opportunities opened up by the chance to train and ultimately practice in the United States may be reasons that international medical graduates have a statistically significant higher rate of misrepresentation than USMGs in our study and over the five studies in which international medical graduates and USMGs have been compared.
Sekas and Hutson give the “mental aberration” hypothesis short shrift as an explanation for misrepresentation, stating only that, “misrepresentation may reflect an underlying psychiatric problem.” Since we are unaware of any published data regarding the prevalence of various psychiatric diagnoses in the population of applicants to psychiatry residency, we can only speculate as to possible relationships. Knowing misrepresentation may be the product of antisocial or narcissistic personality disorders, or the sense of invulnerability associated with mania, while accidental misrepresentation could be attributed to attention deficit/hyperactivity disorder or the poor concentration seen in major depression. Various medical conditions and prescribed medications can also cause accidental misrepresentation. Cast in this light, multiple misrepresentations may not necessarily be due to Machiavellian intentions, but may signify the presence of a psychiatric or other medical disorder that could impair both training and practice.
The findings of this study emphasize that the data contained in the ERAS application, while a useful triage tool, is not sufficient in the evaluation of potential residents. Other studies have highlighted the frequently unreliable (and sometimes deceitful) aspects of elements of the ERAS application, including the dean’s letter (14), letters or recommendation (15), and core clerkship grades (16). To this list we can now add reported publications which, since they are readily verifiable, provide the opportunity in an interview with the applicant to explore any misrepresentation about publications as well as the reliability of other ERAS data.
Authors from other disciplines have called for changes in the methods in which applicants report their publications in an attempt to eliminate misrepresentation on residency application, and some report requesting a copy of each cited article from their applicants (1–8). Given the above possible causes of publication record misrepresentation, it may be prudent for residency training directors to do an online check of the publications listed in residency applications and explore any discrepancies with the applicant prior to finalizing their match lists.