In psychiatric residency training in the United States, there are three major categories of residents, as based on the country of undergraduate medical education. These categories are 1) foreign international medical graduates (F-IMGs), who are citizens of foreign nations where they received their medical education and who constitute 80% of all IMGs; 2) United States (U.S.) international medical graduates (US-IMGs), who are citizens of the United States, but received their medical education abroad and who constitute 20% of all IMGs; and 3) U.S. medical graduates (USMGs), who were born in and received their medical education in the United States. In this paper, the term “IMGs” denotes both IMG groups. In psychiatry, 37% of residency positions in the United States are filled by IMGs, and there have been controversies over their workforce contributions as well as the quality of care that they deliver (1, 2). They are seen as having significant educational needs that are not being met by the training programs (2).
Against this backdrop, the American Board of Psychiatry and Neurology (ABPN) has changed the format of the Board Examination by eliminating the live patient interview, in favor of an in-training assessment conducted by residency programs (3, 4). The new format consists of Clinical Skills Verification (CSV), a process by which training directors credential residents’ competencies before the end of training. The CSV requires that training programs document residents’ competency in 1) the physician–patient relationship; 2) conducting an interview and mental status examination; and 3) case presentation, through a series of observed clinical encounters. This assessment has been endorsed by the Psychiatry Residency Review Committee of the Accreditation Council for Graduate Medical Education (5, 6).
According to the American Association of Directors of Psychiatry Residency Training Programs (AADPRT) CSV Task Force, some programs may elect to offer the assessments frequently and in a variety of settings (e.g., during the course of every rotation), whereas other programs might offer very few opportunities for assessment (6). Consequently, the exam process may be variably implemented across a broad spectrum of training programs. Factors that might affect the administration of the CSV are twofold: 1) training program issues; and 2) resident issues. As for the former, the program size and type, work-hours, patient loads, availability of supervisors, protected time for core didactics, and the balance of service versus academics in the training are particularly relevant. As for the latter, residents’ age, gender, pre-residency training, primary language, location of undergraduate medical education, and familiarity with U.S. subcultures and medical cultures are essential.
The goal of the CSV is to assess residents’ clinical skills and offer formative feedback, which represents “information communicated to the learner that is intended to modify the learner’s thinking or behavior for the purpose of improving learning” (7) (p1). Ironically, such a constructive educational approach might itself be a stressor for residents unaccustomed to formative feedback. This process might be especially difficult for F-IMGs because of differences in the medical education systems and cultures between their home countries and the United States (8).
We hypothesized that all three groups of residents would find the CSV to be a valuable educational tool, but that these groups would differ significantly in their experience of anxiety with its implementation. Second, residents in a later year of postgraduate training (Postgraduate Year 3 [PGY-3]) would have more positive attitudes toward the exam than their junior counterparts (PGY-2). Third, there would be significant differences in perceptions between IMGs and USMGs about the CSV and its implementation. Specifically, F-IMGs would have less-positive attitudes toward the assessment process than the USMGs or US-IMGs because of their lack of previous exposure, as well as their cultural and linguistic deficits. Finally, the type of training program—academic or service-oriented—would also be a key contributor to residents’ satisfaction.
The first major objective of this pilot study was to clarify how all residents respond to the CSV. Our second objective was to bring to light the commonalities and differences in perceptions among various groups of residents (i.e., F-IMGs, US-IMGs, and USMGs) regarding the CSV process. Finally, this study was designed to assess how various demographic and program-related factors affect residents’ perceptions and attitudes toward the changes in the ABPN assessment process.
This was a cross-sectional, Internet-based survey of PGY-2 and PGY-3 residents enrolled in general-psychiatry programs. The Institutional Review Board of the University of Michigan, Ann Arbor, approved the protocol, the script for e-mail communications, and the survey questionnaire.
The 51-item survey questionnaire was designed by the authors, including training directors (NRR and MDJ), residents (RK and UR), and other faculty (AM). The questionnaire was designed to gather demographic data and information about three principal content areas: 1) the residents’ views on the effectiveness of the program (i.e., how well the program prepared them for the CSV and how well the CSV was conducted by the program); 2) the residents’ experiences in the CSV; and 3) evaluation and feedback that the residents received. Survey items used a 5-point Likert scale (1: strongly disagree; 2: disagree; 3: neither agree nor disagree; 4: agree; and 5: strongly agree). The survey questionnaire is available online at http://vtigp.org/Tools/CSV_Questionnaire.pdf.
The survey was uploaded to the online survey website http://www.surveymonkey.com. A convenience sample of the authors’ programs and one or two nearby residency programs included the following nine accredited general-psychiatry programs in the United States: 1) Nassau University Medical Center, East Meadow, New York; 2) University of Michigan, Ann Arbor; 3) Baylor College of Medicine, Houston, Texas; 4) Washington University, Saint Louis, Missouri; 5) Wayne State University, Detroit, Michigan; 6) Albert Einstein Medical College, Bronx, New York; 7) Temple University, Philadelphia, Pennsylvania; 8) Cooper University Hospital, Camden, New Jersey; and 9) Henry Ford Hospital, Detroit, Michigan.
Program directors were contacted at each site and given information regarding the study. They gave permission for the survey to be conducted and agreed to forward the direct link to the survey website to their residents. Residents received a description of the study, a brief note regarding its importance, a statement of its voluntary nature, and assurance of anonymity. Data collected through the survey website from February 2010 through March 2010 were used in the analysis.
Only data-points with no ambiguity in the responses were included in the analysis. The data for all the continuous variables and Likert-scale scores are expressed as mean (standard deviation [SD]). The frequencies for the categorical variables are reported as numbers and percentages of total responses. Data analyses were performed with Microsoft Excel 2007 (Microsoft Corporation, Redmond, WA), SAS Version 9.1 (SAS Institute Inc., Cary, NC), and SPSS Version 15 (SPSS Inc., Chicago, IL). For most survey questions, responses were compared between graduates of U.S. medical schools and graduates of international schools (i.e., F-IMG and US-IMG responses were combined) with a two-tailed t-test. For survey questions on which the three groups of graduates (F-IMGs versus USMGs versus US-IMGs) were expected or observed to differ, and for comparison of responses from different types of programs, a one-way analysis of variance (ANOVA) was performed. For comparison of high- and low-percentage IMG programs, all programs were included and categorized by whether the majority of their respondents were IMGs or USMGs. For comparisons of residents in IMG/USMG–balanced programs, only those with at least 25% of respondents in each category were included. Categorical variables were compared with a χ2 test. In all analyses, a p value of <0.05 was considered significant.
All contacted program directors agreed to request their residents to participate in the study. The survey was completed by 63 of the 123 eligible residents, for a response rate of 51.2%. The make-up of the programs and distribution of residents are as follows: 1) three USMG-dominated programs, with 11 USMGs, 0 US-IMGs, and 1 F-IMG; 2) five balanced programs of USMGs/non-USMGs, with 19 USMGs, 6 US-IMGs, and 9 F-IMGs; and 3) one IMG-dominated program, with 2 USMGs, 4 US-IMGs, and 10 F-IMGs. The racial/ethnic composition of the sample of residents (N [%]) is as follows: Caucasian: 23 (36.5%), African American: 8 (12.7%), Hispanic: 5 (7.9%), Asian: 7 (11.1%), South Asian: 12 (19.0%), and Other: 8 (12.7%). The demographic characteristics of the respondents are shown in Table 1. Approximately one-half of the respondents were graduates of U.S. medical schools (32 [50.8%]), and the other half were graduates of international medical schools (31 [49.2%]). Age, year of training, and program type were comparable among USMG, F-IMG, and US-IMG residents. Gender distribution was skewed toward more women in the F-IMG group than the USMG and US-IMG groups (p=0.03). The likelihood of affiliation with a medical school program did not differ among the groups (p=0.70). The IMG residents were more likely to have had previous clinical training than the other residents (p=0.0013).
There were no differences in the residents’ responses regarding their work-hours, patient loads, availability of supervision, or frequency of CSV assessments when compared by resident background, residency program, or year of training. Table 2 compares these data between predominantly IMG and USMG programs. These residencies differed only in the time available for didactics, with programs composed primarily of IMG residents providing 2 more hours per week than programs composed mostly of U.S. graduates. Residency programs composed mostly of IMGs did not otherwise differ from the predominantly USMG programs.
Results on all survey questions were similar when the analysis was limited to programs with a balance of USMG and IMG residents; in those programs; residents had an average of 2.42 CSV experiences and a pass-rate of 85%. Text Box 1 shows the specific issues on which all groups of residents were in agreement, and those on which IMG residents differed from their USMG counterparts. Also, Table 3 lists the comparison of the mean Likert scores on key survey items: a) between USMG and non-USMG (IMG + US-IMG) residents in all programs; b) between USMG and non-USMG residents in balanced programs;, and c) among IMG, US-IMG, and USMG residents in all programs.
+
BOX 1. Commonalities and Differences Between USMG and IMG Residents
+
Commonalities Among All Three Groups
Felt well informed about CSV, although many were not formally trained in how to do it.
Felt that they had good training in clinical interviewing.
Were comfortable with the format, although they were anxious in front of faculty.
Felt they were given sufficient time for the interview and presentation.
Reported that they received constructive, formative feedback, including positive and negative observations.
Felt that CSV helped them improve their clinical skills.
Felt that CSV was useful.
There was wide disparity among all residents regarding the opportunity to observe faculty doing interviews.
+
IMGs in comparison to USMGs:
+
F-IMGs in comparison to US-IMGs and USMGs:
Were less likely to have the CSV as part of an organized teaching experience.
Had fewer supervised interview experiences in medical school.
Were less comfortable with cultural expression (non-US background only).
Were less likely to be aware of CSV before their residency.
The primary limitations of the study were the nonrandomized selection of programs surveyed and the small sample size. First, although a convenience sample was surveyed, a broad range of program size and composition was represented. Second, additional analyses of the data by program type showed comparable results to the larger survey. Finally, a subgroup of programs, with a balance between IMG and USMG residents, showed equivalent results. At a minimum, we expect that these findings should stand as a useful pilot study for further research.
Even though the success rates on the exam between both of the IMG groups and the USMGs were not significantly different, the study results (see Text Box 1) indicate that the IMG residents differ from the USMGs in other key aspects of the CSV exam process. The exposure to psychiatry that students receive in foreign schools may be meager or nil, and there is little prestige in a career in psychiatry, with many schools even lacking formal departments of psychiatry (9, 10). The rotations tend to be short, and psychiatry is treated as a subspecialty of internal medicine. There is no written or clinical examination in psychiatry other than an occasional essay question that appears in the internal medicine examination. Clinical training is poorly organized, and it includes a brief rotation on a psychiatric ward supervised by general internists. Students have few opportunities to observe a senior faculty member conduct interviews or be supervised while interviewing in medical school. The psychiatric interview is poorly differentiated from the conventional medical history-taking interview. The IMG residents, therefore, lack the opportunity to develop and practice their interviewing skills.
Furthermore, IMG residents are less likely to be aware of CSV before their residency. The majority of IMGs may be unaware of policies and procedures of residency training because graduate medical education in the United States has significant differences from many other nations, even though there is some movement toward adopting a North American style of training in certain countries (8, 11).
These issues could be addressed if training programs would show demonstration tapes of the CSV during residency interviews to drive home the critical importance of this process to the resident’s success in the program. Role-playing exercises, with observation and feedback, may also be useful. Interestingly, the survey results indicated that IMG residents want more supervised interviews during residency training. The IMG residents may, therefore, be offered more intensive psychiatric interview training than they are currently experiencing in their PGY-1.
The survey indicates that IMG residents felt that “the feedback after the exam was excessively critical and painful—would not like to go through a similar experience again.” Foreign medical schools’ emphasis on impersonal summative evaluations and numerical grades anesthetize the evaluation process for the IMGs; the experience of receiving detailed, individual performance evaluation, consequently, is both unfamiliar and stressful. Narcissistic vulnerability caused by viewing feedback as an attack also plays a role. Fear of losing the esteem of one’s training director and the unrealistic anticipation of being deported are also contributing factors. Faculty need to be sensitive to the cultural influences on self-esteem regulation, to provide feedback by focusing on actions rather than on individuals, and offer clear goals and objectives that can be operationalized. In this respect, faculty development programs for teachers of IMGs are helpful (12).
Our concern that IMG residents would be more likely to find themselves in resource-poor, predominantly IMG programs and that this would result in less-satisfactory performance on the CSV and lower satisfaction with the process was not supported. Although it was true that many of the IMGs who participated in our survey were from programs with IMG majorities, all measures of residency quality that we examined were comparable or superior in these programs in comparison with programs with USMG majorities. Measures of resident satisfaction with the CSV process were likewise similar between programs.
The comparison of USMG and IMG responses in programs balanced in their resident demographics provided an additional probe into the impact of residency type on survey results. Despite the small sample size available, we found significant differences, paralleling those of the larger sample. The samples differed only on the item regarding whether the CSV was too painful to repeat.
Finally, the results yielded valuable findings regarding differences among F-IMGs, US-IMGs, and USMGs, thus providing support for one of the hypotheses of the study. It was shown that F-IMGs felt less comfortable than their US-IMG and USMG counterparts while talking to a patient before an audience. For F-IMGs in contrast to US-IMGs, public-speaking anxiety, lack of training in interviewing in medical school (13, 14), lack of proficiency in spoken English—especially the “familiar language,” (15, 16) and lack of familiarity with local cultural norms may contribute to discomfort in conducting interviews. It must be noted that the public psychiatric interview, which involves conducting an interview in front of a group or an interview that is taped, is considered to be stressful even for those who are assimilated in the culture in which they practice (12). The public interview is seen as affecting the concentration of the interviewer, while changing the patient’s social behavior. Also, being observed in a public interview makes it more difficult for the therapist to be empathic with his or her patient because the therapist is too busy monitoring his or her own experiential reactions to the observers (12). It is essential that training programs address this potential vulnerability through English-as-second-language (ESL) courses and accent-reduction training and thus help residents with public-speaking strategies by providing such courses at annual meetings (17).
In conclusion, our study demonstrates a high level of satisfaction among residents with the CSV process, but it has also brought to light several noteworthy differences in the experiences and perceptions of IMG and USMG residents. Training programs seem to be implementing the CSV in a manner that is devoted to the expectations of regulatory bodies. Our study has identified the receptivity of and difficulties faced by F-IMG residents in comparison with and in contrast to the USMGs with regard to the CSV process. Although the causes for these contrasting experiences and perceptions may be multifaceted, addressing these concerns may be essential for the optimal performance of the residents as well as for the success of the training programs themselves.
The authors have no competing interests relative to the subject of this study.