Previous investigations have identified a number of factors that affect measures of medical student academic achievement across several disciplines during third- and fourth-year clinical clerkships, including subjectivity of preceptors, student and preceptor gender, clerkship site, clerkship duration, clerkship order and variations in perceived quality of clinical teaching, and stringency of grading (1–13). In these studies, dependent variables were typically subjective “final” grades, performance on objective structured clinical exams, and/or scores from nationally standardized subject examinations such as those produced by the National Board of Medical Examiners (e.g., the “shelf” exams). For psychiatry clerkships, these variables, as well as the learners’ personality style (14) have also been shown to influence performance on the final examination given at the end of their clinical rotations (15, 16).

Because medical care is being delivered in predominately outpatient environments, medical training is occurring in ambulatory settings at an increasing rate (17, 18). Although a small literature supports the notion that core educational goals of a psychiatry clerkship can be met using a variety of clinical settings (19–21), there has been little systematic investigation concerning the effects of psychiatry clerkship setting (ambulatory versus hospital based) on measures of academic performance, motivation to pursue psychiatry as a career, and attitudes about psychiatry as a professional discipline. In order to better describe this relationship, we conducted a longitudinal prospective study on third-year medical students during their required psychiatric rotation.

This study was performed using a sample of third-year medical students during their psychiatry clerkship rotation to assess the influence and interaction of psychiatric rotation type, academic performance, and choice of residency in psychiatry. Following institutional review board approval, enrollment information was reviewed on all medical students who were registered for the third-year psychiatry clinical clerkship between academic years 1997 and 2001, inclusive. Demographic information obtained on all students included gender and age at matriculation to medical school. The types of clinical rotation, location, and time of rotation were also examined.

Students were administratively assigned to one of several clinical settings: an outpatient clinic, a hospital based consultation-liaison service, or an acute inpatient ward. Changes in rotation site or setting were not allowed. Each rotation lasted 6 weeks, and the duration of the clerkship did not change over the 5-year period of this study. Students assigned to the inpatient or consultation-liaison services were exposed to a wide variety of psychiatric diagnoses, including psychotic disorders, mood disorders with and without psychotic features, severe anxiety disorders, substance-related conditions, adjustment disorders, somatoform disorders, and personality disorders.

Performance data consisted of standardized National Board of Medical Examiners (NBME) exam scores from tests administered at the end of the rotation (e.g., “shelf” examinations). Because the NBME scoring system changed over the course of the project, standard scores that were reported in the three-digit range prior to 1999–2000 were standardized to two-digit subject scores using the z-scores of the NBME standard score and the mean and standard deviations (SD) of scores provided by the NBME. Other outcome data consisted of eventual choice of residency, dichotomized into psychiatric versus nonpsychiatric residency. Individuals who chose combined family practice/psychiatry or internal medicine/psychiatry programs were treated as having chosen psychiatric residencies for purposes of this study. Attitudes about psychiatry as a discipline were assessed using two anonymous surveys that employed a five-point Likert scale (Appendix 1 ).

Statistical calculations were performed using Stata^TM 8.0 (Stata Corporation, College Station, Tex.) software, at the α=0.05 level of significance. Descriptive statistics were used to summarize characteristics of our sample. Among categorical variables, we performed comparisons across rotation types using Pearson’s chi-square test. Due to the moderate non-normality of grade point average (GPA) and age distribution, we used the Kruskal-Wallis test to compare these data across rotation types. Linear regression models were used to predict NBME score as a function of rotation type, while multivariate logistic regression was used to identify variables associated with later selection of psychiatric residency. We examined the relationship between rotation type and choice of specialty using the Pearson’s chi-square test followed by univariate logistic regression. Finally, we analyzed changes in responses on paired pre- and postrotation survey questions using the Mann-Whitney test.

Study Population

There were a total of 801 students for whom enrollment information was obtained between 1997 and 2001. Of these, a total of 674 students completed a rotation at the designated study sites. Complete performance data (GPA and NBME shelf exam score) and residency choice data were available for 647 of these students, thus comprising the study population (Table 1 ). A majority were assigned to inpatient rotations. There were no significant differences between students assigned to inpatient versus outpatient rotations with respect to age, gender, or other baseline characteristics.

Performance Data

Using linear regression modeling, rotation type did not predict NBME “shelf” exam scores. NBME scores were 1.4 points higher [p=0.113, 95% confidence interval (CI)=−0.3–3.0] among students rotating in inpatient settings relative to students on consultation-liaison rotations, and were 0.8 points higher (p=0.406, 95% CI=−1.1–2.6) than students rotating in outpatient settings.

In order to assess the independent effect of rotation setting on student performance, it was necessary to examine the data for possible confounding effects. NBME scores and GPA were weakly correlated in the study population (r=0.19). NBME scores were also weakly correlated with academic year (r=0.01) and time of rotation (r=0.02). There was no statistically significant correlation between NBME scores and age, although we noted a trend toward rising NBME score by age in females and declining performance by age in males.

After adjusting for GPA, age and gender interaction, time of rotation and rotation year, there were no significant differences in NBME scores associated with rotation type (Table 2 ). Exam scores were 1.8 points higher (p=0.02, 95% CI=0.3–3.3) among students who rotated on inpatient services relative to those on consultation-liaison services (Table 2 ); however, this small difference was regarded as academically insignificant. Although NBME scores for students from inpatient services were also higher than those rotating in outpatient settings (0.4 points higher, p=0.6, 95% CI=−1.2–2.0), this difference was not statistically significant.

Specialty Choice

Among the study population, 26 students (4.02%) selected a psychiatric residency. There was no relationship between type of psychiatry clerkship rotation and specialty choice (χ²=4.35, p=0.11) (Table 3 ). Among those who selected a psychiatric residency, the mean GPA was 2.82, while the mean NBME score was 67.35. Among those selecting other residencies, the mean GPA was 3.14, while the mean NBME score was 70.40. In univariate logistic regression, selection of psychiatric residency was not associated with gender or age; however, there was a strong inverse association between third-year GPA and choice of psychiatric residency (odds ratio 0.21, p=0.001, 95% CI=0.08–0.52), and an inverse association between NBME score, and choice of psychiatric residency (odds ratio 0.94, p=0.04, 95% CI=0.89–0.998). A multivariate logistic regression model that included gender and age interaction, NBME score, and GPA showed that only GPA was found to retain statistical significance in terms of selection of psychiatric residency (odds ratio 0.24, p=0.005, 95% CI=0.09–0.65).

Survey Data

A total of 183 students completed Survey A and 185 students completed Survey B (Appendix 1 ). Both surveys were completed by 85 students. No statistically meaningful patterns emerged between responses in either survey and rotation type. Neither the responses to questions from Survey A nor the pre-/postitems on Survey B (questions 1 and 2) were found to be associated with choice of psychiatric residency. These items aimed to assess, specifically, subjective motivation to pursue psychiatry as a career.

Prior studies have documented the effect of psychiatry clerkship rotation setting on measures of academic performance, student attitudes, and other learning outcomes. Griffin et al. (22) reported that students assigned to a consultation-liaison setting achieved similar scores on a postclerkship departmental examination and evidenced similar attitudes toward psychiatry when compared to a group of students assigned to a traditional inpatient site. Studies by Yano et al. (23) and Harsch and Young (24) documented similar results among students assigned to a consultation-liaison or inpatient service. No significant differences on NBME part 2 psychiatry examination scores or on student evaluations of the clerkships were reported by Weddington et al. (25) in a study comparing students on a consultation-liaison clerkship with students in other inpatient and outpatient psychiatry settings. Alternatively, students assigned to an emergency room psychiatry clerkship consistently achieved lower final examination scores than students assigned to other clerkships in one study (26).

Our study adds to this scant literature by documenting the following key findings: rotation type was not associated with differences in NBME psychiatry “shelf” examination scores; rotation type appeared to have no significant effect on eventual specialty choice, though a trend level relationship was observed linking selection of psychiatric residency by rotation type; and student attitudes about psychiatry and psychiatric patients, motivation to pursue psychiatry as a career, and subjective level of psychiatric knowledge acquired did not differ as a function of rotation type. To our knowledge, this is the first study to directly compare three rotation types.

Our results, particularly for the outpatient group, were encouraging considering the challenges inherent to teaching and learning in an evolving managed health care system. These challenges include the need for specialized curricula; need for faculty training in supervision, education and evaluation; site availability; physical space limitations; pressure on faculty to remain clinically efficient while simultaneously providing good supervision and education; and possible patient disinclination against having medical students involved in their care (27–31). We are aware of only one published study (32) and one unpublished study (Bobo, Carr-Malone, Lacy, unpublished data) that investigated the effect of medical student presence on patient satisfaction with care. In both studies, patient satisfaction was unaltered by the presence of medical students.

In our study, the timing of the clerkship did not appear to affect postrotation testing scores. This result was consistent with data published by Case et al. (33) and Newton et al. (34), but not Hoge et al. (35). In their large retrospective analysis, small but significant differences in NBME subject exam scores favoring those who were assigned psychiatry clerkships later in the academic year were reported.

In our study, about 4% of participants selected a psychiatric residency. Similar figures for career choice have been reported by others (34) and appear to be in keeping with career choice trends by U.S. medical school graduates (36). Negative views about the specialty and the patients it serves held by medical students, both before and after entry into medical school (37, 38), have been documented. Negative attitudes about the specialty have been associated with a decline over the past several years in psychiatry as a career choice among medical students (39–42). The existing literature offers only mixed results when considering the effect of rotation type or quality on positive change in attitudes about psychiatry, its patients or increased recruitment into the field (43–48); however, high quality supervision and having a more active role in the care of patients appears to correlate with positive change in attitude and recruitment numbers (48–50). On the other hand, it is not clear that change in attitudes alone would be sufficient to increase the number of students likely to choose psychiatry as a career. This may be especially true given a reemerging interest in increasing the number of generalist physicians in the United States (51).

In this study, we found an inverse relationship between GPA and choice of psychiatric residency; however, in absence of more specific information about the relationship between GPA and choice of specific nonpsychiatric residencies, this finding is difficult to interpret. Based on our data, we cannot conclude that less academically strong students were more likely to choose a psychiatric residency. The mean GPA in this group equated with a B average in terms of letter grade, a level of performance that would not be considered marginal or poor.

For performance outcome measures, we reported only standardized examination results. We felt that the other available outcome measures of academic performance (e.g., nonstandardized departmental examination scores, grades on a required end-rotation essay, grades of formal clinical write-ups, and performance evaluations by clinical preceptors) were too subjective or did not lend themselves to statistical analysis, even though their inclusion may have provided more information in “skill-based” performance. Performance data using standardized patients were not available for this sample, despite reports of their successful implementation (52, 53). The surveys used in this study were not themselves standardized instruments.

Notwithstanding these limitations, the findings from our study indicate that knowledge-based educational objectives for a psychiatry clerkship may be met regardless of rotation type. Eventual choice of specialty, student attitudes about psychiatry as a discipline or psychiatric patients, motivation to pursue psychiatry as a career, and subjective ability to acquire knowledge also appear to be unrelated to rotation type.

We are grateful to Michaela Nevin for her valuable administrative support and assistance with collecting data for this project.

Dr. Bobo has been a member of the Speaker's Bureau for Janssen and Pfizer in the last 12 months. Drs. Nevin, Greene, and Lacy disclosed no competing interests.