Full Text (PDF) Alert me when this article is cited Alert me if a correction is posted Email this article to a Colleague Similar articles in this journal Similar articles in PubMed Alert me to new issues of the journal Add to My Articles & Searches Download to citation manager Citing Articles via Google Scholar Articles by Lehrmann, J. A. Articles by Walaszek, A. Search for Related Content PubMed Citation Articles by Lehrmann, J. A. Articles by Walaszek, A. Education, Psychiatrists Other Administrative Issues

Assessing the Quality of Residency Applicants in Psychiatry

Received January 29, 2008; accepted February 13, 2008. Dr. Lehrmann is affiliated with Zablocki VA Medical Center, Mental Health Division, in Milwaukee, Wis.; Dr. Walaszek is affiliated with the Department of Psychiatry at the University of Wisconsin School of Medicine and Public Health in Milwaukee. Address correspondence to Jon A. Lehrmann, M.D, Mental Health Division, Zablocki VA Medical Center, 5000 W. National Ave., Milwaukee, WI 53295; jon.lehrmann{at}va.gov (e-mail).

Perhaps the most important role of the residency training director is the recruitment of quality residents. Compassionate, smart, and motivated residents contribute to a robust educational environment at the center of all academic programs. Strong residents help other residents develop into better residents. Strong residents foster the development of the faculty, inspiring, motivating, and sustaining their careers. Residents help dismantle stigma about the psychiatric profession. These same residents may join our faculty as well and thereby become our colleagues and collaborators. This is the stuff of program directors’ dreams.

On the other hand, residents with significant professionalism, interpersonal, or academic difficulties take an inordinate amount of time from the faculty, program coordinator, and training director. This can take valuable time away from the rest of the residents. These "problem" residents can also negatively affect recruitment at one’s own institution (students interact with these residents directly on psychiatry and medical services) and from outside (the interviewing candidates may interact with them at lunches, dinners, on tours, or on second visits). These same residents may affect or add to negative stigmas at medical institutions such as, "psychiatrists are not real doctors," "medical students go into psychiatry because they have their own mental health problems," and "medical students go into psychiatry because they are ineffective medical doctors."

As educators, we hope that through excellent training we can strengthen the quality of our residents. High-quality education, modeling, supervision, remediation, and mentoring can go a long way in helping our residents and are essential in a residency program. Yet residents do not arrive at our doorsteps as blank slates—and this is where a rigorous recruitment process is critical.

For most training directors, recruitment is a yearlong process. Even during the "quiet" months between Match Day and October (when applications officially begin to pour in), almost daily e-mails arrive asking about our programs. Training directors meet with medical student interest groups, attend medical school career days, or American Medical Student Association residency days. Training directors often meet with the students from their own medical school who are interested in going into psychiatry to discuss what a career in psychiatry is like or what to look for in a psychiatry residency program. Letters of support are often requested and written. When October rolls around again, training directors review several hundred applications, and based upon each director’s or recruitment committee’s unique (and often secret) formula, the limited number of candidates to be interviewed are determined, and invitations are sent.

So, how can we be sure we are recruiting quality residents? Assessing the quality of a residency applicant is a very complicated and time-consuming process. Can we assume medical schools have screened out the students who have had academic problems or are poor residency candidates? Though quality medical schools may help weed out, remediate, and strengthen residency candidates, those of us who have been in academic roles for a while realize that we cannot assume all of the graduates are highest, or even adequate, quality candidates. Can we assume the Electronic Residency Application Service (ERAS) confirms the accuracy and source of every letter of recommendation and document submitted? Though ERAS does require and document a complete history of the United States Medical Licensing Examination (USMLE) results for each candidate, it cannot and does not ensure that every other document is accurate. However, ERAS does have a system of notifying programs when a candidate has been found to falsify data.

A recent general "formula" at one of the author’s (JL) institutions includes the following:

• Applications must come through ERAS
  
• USMLE scores are reviewed. If a candidate failed two or more times, the application goes on a "no interview" pile; however, if something else stands out positively on the application, it may end up on a "maybe" pile. Applications showing scores of 200 or higher go on a "yes" pile. Applications with passing scores less than 200 go on the "maybe" pile.
  
• Other factors that could move a candidate’s application out of the "yes" pile include problems completing medical school; poor grades or evaluations; too much unaccounted-for time between medical school and the application; no U.S., U.K., Canadian, or Australian clinical experience; a personal statement with unusual or disconcerting content; evidence of language difficulties; and a letter of support that voices significant concerns.
  
• Factors that can move an application into the "yes" pile include glowing letters of recommendation or letters of support from people the training director knows and trusts; an indication that the candidate has a social conscience (such as significant voluntarism); significant evidence of a commitment and passion for psychiatry; evidence of research experience (publications); and other related educational experience (masters or doctorate degrees).
  

The process at the other author’s (AW) institution is similar, though with additional criteria for international medical graduates, including higher USMLE scores (a minimum of 220) and direct clinical experience in the previous 2 years.

Being naive, trusting, and fairly new Midwestern training directors, we were shocked last year when we saw a "personal statement development" web site for U.S. and international medical students which displayed example personal statements, and actually recognized some of them! We had received several nearly identical personal statements that had been plagiarized. We saw another web site (1) that offered ghost writing for personal statements. One of us (AW) recently received a "personal" statement whose "author" was in fact a chapter from a textbook on selecting a career in psychiatry. Although neither author recalls interviewing an applicant with an apparently plagiarized personal statement, we wonder if this is because of effective screening or simply dumb luck. In other words, how effective are we in evaluating applications to our residencies?

Now training programs have to worry about misrepresentations of publications listed in applications (2). Caplan et al., having found that 9% of applicants reporting publications misrepresented them, suggest that program directors perform an online review of the publications. Reviewing the candidates’ publications may be helpful, but whatever happened to being able to trust someone? Training directors have significant time constraints, especially with new requirements to assess competency. We may instead suggest that applicants submit a copy of each referenced manuscript as verification.

It may be cold comfort to point out that misrepresentation of one’s credentials is by no means limited to graduate medical education. Automatic Data Processing, Inc., a firm specializing in human resources, each year releases statistics on hiring and screening. In 2007, out of 445,000 reference verifications, 41% had discrepancies between what the applicant provided and what the reference reported (3). When analyzed by business sector, health care had a higher-than-average rate of 51%. Perhaps reassuringly, applicants to health care positions had the lowest rate of criminal records, at 3%. Not so reassuringly, formal background checks, at least at our institutions, do not occur until after the Match.

A survey by the web site CareerBuilder.com found that 57% of hiring managers discovered inaccuracies in candidates’ applications (4). The most common categories of misrepresentations were stretched dates to cover employment gaps, past employers, academic degrees and institutions, technical skills, and certifications and accomplishments.

A British preemployment screening firm, PowerChex, found that 31% of job applications submitted to financial institutions in 2005–2006 included "embellishments and false information" such as employment dates (22%), academic and professional qualifications (11%), and employment titles or duties (4%), though a follow-up survey in 2006–2007 revealed lower numbers (5). Although these surveys are not terribly rigorous, the results are consistent enough to support the conclusion that a substantial number of job applications include lies.

While Caplan et al. (2) found that international medical graduates are more likely to misrepresent their publications than U.S. medical graduates, human resources surveys have been inconsistent in identifying other risk factors. Thus, the human resources literature recommends that employers require candidates to complete application forms thoroughly and consistently check on every aspect of the application (6). Although this would be a daunting task for an already beleaguered program director, it appears to be typical of other business sectors.

Perhaps the most important screening tool we have is the candidate visit and interview. Having several psychiatrists and psychiatry chief residents interview the candidates provides extremely valuable information. Being rude to the residency coordinator or making off-color statements when no one seems to be listening are significant cues that may lead to rapid exclusion from the rank order list. But does observing someone on her or his best behavior for several hours suffice? And should we be spending more time Googling our candidates?

REFERENCES

TOP REFERENCES

 

1. The Doctor Job: For Medical Students. Available at http://www.thedoctorjob.com/medicalstudents.html
2. Caplan J, Borus J, Chang G, et al: Poor intentions or poor attention: Misrepresentation by applicants to psychiatry residency. Acad Psychiatry 2008; 32:225–229[Abstract/Free Full Text]
3. ADP Screening Index: Background Screening Results. Available at http://www.adphire.com/screeningIndex.html
4. CareerBuilder.com: Job Seekers are Stretching the Truth. Available at http://www.careerbuilder.com/JobSeeker/Careerbytes/CBArticle.aspx?articleID=574&pf=true
5. PowerCheck: Pre-employment screening. Available at http://www.powerchex.co.uk/preemploymentscreening-1.htm
6. Andler EC, Herbst D: The Complete Reference Checking Handbook, 2nd ed. New York, AMACOM/American Management Association, 2003

Full Text (PDF) Alert me when this article is cited Alert me if a correction is posted Email this article to a Colleague Similar articles in this journal Similar articles in PubMed Alert me to new issues of the journal Add to My Articles & Searches Download to citation manager Citing Articles via Google Scholar Articles by Lehrmann, J. A. Articles by Walaszek, A. Search for Related Content PubMed Citation Articles by Lehrmann, J. A. Articles by Walaszek, A. Education, Psychiatrists Other Administrative Issues

Get information about faster international access.

Privacy Policy

Copyright © 2008 Academic Psychiatry. All rights reserved.

Home | Search | Current Issue | Past Issues | Subscribe | All APPI Journals | Help | Contact Us