Advanced Search

REGULAR | March 01, 2009

The Effect of Psychiatric Third-Year Rotation Setting on Academic Performance, Student Attitudes, and Specialty Choice

William V. Bobo; Remington Nevin; Elizabeth Greene; Timothy J. Lacy

Academic Psychiatry 2009;33:105-111.

View Article Information

Received May 6, 2007; revised August 20 and December 17, 2007; accepted January 14, 2008. Dr. Bobo is affiliated with the Department of Psychiatry at Vanderbilt University School of Medicine, Vanderbilt Psychiatric Hospital, in Nashville, Tenn.; Dr. Nevin is affiliated with the Army Medical Surveillance Activity in Silver Spring, Md.; Dr. Greene is affiliated with the Department of Psychiatry at Wilford Hall USAF Medical Center, San Antonio, Tex.; Dr. Lacy is affiliated with the Department of Psychiatry at Malcolm Grow USAF Medical Center, Andrews AFB. Address correspondence to William V. Bobo, M.D., Vanderbilt University School of Medicine, Department of Psychiatry, Vanderbilt Psychiatric Hospital, 1601 23rd Ave. South, Suite 3035, Nashville, TN 37212; william.v.bobo@vanderbilt.edu (e-mail).

Article

Tables

References

text A A A

Abstract

Objective: Few studies have directly compared the effects of third-year clerkship rotation type on measures of academic performance, student attitudes about psychiatry and psychiatric patients, and level of interest in psychiatry as a career. The goal of this study was to assess the extent to which rotation type influenced these outcome variables among third-year medical students. Methods: The authors conducted a prospective study of 647 third-year medical students administratively assigned to one of three clinical settings: an acute inpatient ward, a hospital-based consultation-liaison service, and an outpatient mental health care clinic. Academic performance was estimated using scores from a nationally standardized examination provided by the National Board of Medical Examiners (NBME), while responses to an anonymous survey developed by the investigators were used as indicators of student attitudes about and interest in psychiatry as a potential career field. Administrative residency match data were collected on all participants. Results: Rotation type had no effect on NBME exam scores after controlling for grade point average, age, gender, rotation order, and rotation year. Although individuals who rotated on the inpatient service scored an average of 1.8 points higher on the examination relative to individuals who rotated on the consultation service, this small difference was not considered academically meaningful. Similarly, there were no statistically meaningful patterns that emerged between survey responses and rotation type. Approximately 4% of our sample matched into psychiatric residencies after graduating from medical school. Rotation type and survey responses were not statistically correlated with specialty choice. Conclusion: Rotation type does not appear to affect acquisition of psychiatric knowledge as estimated by standardized examination scores, nor does it appear to influence students’ perceptions of psychiatry or specialty choice.

Figures in this Article

Methods

Previous investigations have identified a number of factors that affect measures of medical student academic achievement across several disciplines during third- and fourth-year clinical clerkships, including subjectivity of preceptors, student and preceptor gender, clerkship site, clerkship duration, clerkship order and variations in perceived quality of clinical teaching, and stringency of grading (¹—¹³). In these studies, dependent variables were typically subjective "final" grades, performance on objective structured clinical exams, and/or scores from nationally standardized subject examinations such as those produced by the National Board of Medical Examiners (e.g., the "shelf" exams). For psychiatry clerkships, these variables, as well as the learners’ personality style (¹⁴) have also been shown to influence performance on the final examination given at the end of their clinical rotations (¹⁵, ¹⁶).

Because medical care is being delivered in predominately outpatient environments, medical training is occurring in ambulatory settings at an increasing rate (¹⁷, ¹⁸). Although a small literature supports the notion that core educational goals of a psychiatry clerkship can be met using a variety of clinical settings (¹⁹—²¹), there has been little systematic investigation concerning the effects of psychiatry clerkship setting (ambulatory versus hospital based) on measures of academic performance, motivation to pursue psychiatry as a career, and attitudes about psychiatry as a professional discipline. In order to better describe this relationship, we conducted a longitudinal prospective study on third-year medical students during their required psychiatric rotation.

This study was performed using a sample of third-year medical students during their psychiatry clerkship rotation to assess the influence and interaction of psychiatric rotation type, academic performance, and choice of residency in psychiatry. Following institutional review board approval, enrollment information was reviewed on all medical students who were registered for the third-year psychiatry clinical clerkship between academic years 1997 and 2001, inclusive. Demographic information obtained on all students included gender and age at matriculation to medical school. The types of clinical rotation, location, and time of rotation were also examined.

Students were administratively assigned to one of several clinical settings: an outpatient clinic, a hospital based consultation-liaison service, or an acute inpatient ward. Changes in rotation site or setting were not allowed. Each rotation lasted 6 weeks, and the duration of the clerkship did not change over the 5-year period of this study. Students assigned to the inpatient or consultation-liaison services were exposed to a wide variety of psychiatric diagnoses, including psychotic disorders, mood disorders with and without psychotic features, severe anxiety disorders, substance-related conditions, adjustment disorders, somatoform disorders, and personality disorders.

Performance data consisted of standardized National Board of Medical Examiners (NBME) exam scores from tests administered at the end of the rotation (e.g., "shelf" examinations). Because the NBME scoring system changed over the course of the project, standard scores that were reported in the three-digit range prior to 1999—2000 were standardized to two-digit subject scores using the z-scores of the NBME standard score and the mean and standard deviations (SD) of scores provided by the NBME. Other outcome data consisted of eventual choice of residency, dichotomized into psychiatric versus nonpsychiatric residency. Individuals who chose combined family practice/psychiatry or internal medicine/psychiatry programs were treated as having chosen psychiatric residencies for purposes of this study. Attitudes about psychiatry as a discipline were assessed using two anonymous surveys that employed a five-point Likert scale (Appendix 1 ).

Statistical calculations were performed using StataTM 8.0 (Stata Corporation, College Station, Tex.) software, at the α=0.05 level of significance. Descriptive statistics were used to summarize characteristics of our sample. Among categorical variables, we performed comparisons across rotation types using Pearson’s chi-square test. Due to the moderate non-normality of grade point average (GPA) and age distribution, we used the Kruskal-Wallis test to compare these data across rotation types. Linear regression models were used to predict NBME score as a function of rotation type, while multivariate logistic regression was used to identify variables associated with later selection of psychiatric residency. We examined the relationship between rotation type and choice of specialty using the Pearson’s chi-square test followed by univariate logistic regression. Finally, we analyzed changes in responses on paired pre- and postrotation survey questions using the Mann-Whitney test.

Results

Study Population

There were a total of 801 students for whom enrollment information was obtained between 1997 and 2001. Of these, a total of 674 students completed a rotation at the designated study sites. Complete performance data (GPA and NBME shelf exam score) and residency choice data were available for 647 of these students, thus comprising the study population (Table 1 ). A majority were assigned to inpatient rotations. There were no significant differences between students assigned to inpatient versus outpatient rotations with respect to age, gender, or other baseline characteristics.

Performance Data

Using linear regression modeling, rotation type did not predict NBME "shelf" exam scores. NBME scores were 1.4 points higher [p=0.113, 95% confidence interval (CI)=−0.3—3.0] among students rotating in inpatient settings relative to students on consultation-liaison rotations, and were 0.8 points higher (p=0.406, 95% CI=−1.1—2.6) than students rotating in outpatient settings.

In order to assess the independent effect of rotation setting on student performance, it was necessary to examine the data for possible confounding effects. NBME scores and GPA were weakly correlated in the study population (r=0.19). NBME scores were also weakly correlated with academic year (r=0.01) and time of rotation (r=0.02). There was no statistically significant correlation between NBME scores and age, although we noted a trend toward rising NBME score by age in females and declining performance by age in males.

After adjusting for GPA, age and gender interaction, time of rotation and rotation year, there were no significant differences in NBME scores associated with rotation type (Table 2 ). Exam scores were 1.8 points higher (p=0.02, 95% CI=0.3—3.3) among students who rotated on inpatient services relative to those on consultation-liaison services (Table 2 ); however, this small difference was regarded as academically insignificant. Although NBME scores for students from inpatient services were also higher than those rotating in outpatient settings (0.4 points higher, p=0.6, 95% CI=−1.2—2.0), this difference was not statistically significant.

Specialty Choice

Among the study population, 26 students (4.02%) selected a psychiatric residency. There was no relationship between type of psychiatry clerkship rotation and specialty choice (χ2=4.35, p=0.11) (Table 3 ). Among those who selected a psychiatric residency, the mean GPA was 2.82, while the mean NBME score was 67.35. Among those selecting other residencies, the mean GPA was 3.14, while the mean NBME score was 70.40. In univariate logistic regression, selection of psychiatric residency was not associated with gender or age; however, there was a strong inverse association between third-year GPA and choice of psychiatric residency (odds ratio 0.21, p=0.001, 95% CI=0.08—0.52), and an inverse association between NBME score, and choice of psychiatric residency (odds ratio 0.94, p=0.04, 95% CI=0.89—0.998). A multivariate logistic regression model that included gender and age interaction, NBME score, and GPA showed that only GPA was found to retain statistical significance in terms of selection of psychiatric residency (odds ratio 0.24, p=0.005, 95% CI=0.09—0.65).

Survey Data

A total of 183 students completed Survey A and 185 students completed Survey B (Appendix 1 ). Both surveys were completed by 85 students. No statistically meaningful patterns emerged between responses in either survey and rotation type. Neither the responses to questions from Survey A nor the pre-/postitems on Survey B (questions 1 and 2) were found to be associated with choice of psychiatric residency. These items aimed to assess, specifically, subjective motivation to pursue psychiatry as a career.

Discussion

Prior studies have documented the effect of psychiatry clerkship rotation setting on measures of academic performance, student attitudes, and other learning outcomes. Griffin et al. (²²) reported that students assigned to a consultation-liaison setting achieved similar scores on a postclerkship departmental examination and evidenced similar attitudes toward psychiatry when compared to a group of students assigned to a traditional inpatient site. Studies by Yano et al. (²³) and Harsch and Young (²⁴) documented similar results among students assigned to a consultation-liaison or inpatient service. No significant differences on NBME part 2 psychiatry examination scores or on student evaluations of the clerkships were reported by Weddington et al. (²⁵) in a study comparing students on a consultation-liaison clerkship with students in other inpatient and outpatient psychiatry settings. Alternatively, students assigned to an emergency room psychiatry clerkship consistently achieved lower final examination scores than students assigned to other clerkships in one study (²⁶).

Our study adds to this scant literature by documenting the following key findings: rotation type was not associated with differences in NBME psychiatry "shelf" examination scores; rotation type appeared to have no significant effect on eventual specialty choice, though a trend level relationship was observed linking selection of psychiatric residency by rotation type; and student attitudes about psychiatry and psychiatric patients, motivation to pursue psychiatry as a career, and subjective level of psychiatric knowledge acquired did not differ as a function of rotation type. To our knowledge, this is the first study to directly compare three rotation types.

Our results, particularly for the outpatient group, were encouraging considering the challenges inherent to teaching and learning in an evolving managed health care system. These challenges include the need for specialized curricula; need for faculty training in supervision, education and evaluation; site availability; physical space limitations; pressure on faculty to remain clinically efficient while simultaneously providing good supervision and education; and possible patient disinclination against having medical students involved in their care (²⁷—³¹). We are aware of only one published study (³²) and one unpublished study (Bobo, Carr-Malone, Lacy, unpublished data) that investigated the effect of medical student presence on patient satisfaction with care. In both studies, patient satisfaction was unaltered by the presence of medical students.

In our study, the timing of the clerkship did not appear to affect postrotation testing scores. This result was consistent with data published by Case et al. (³³) and Newton et al. (³⁴), but not Hoge et al. (³⁵). In their large retrospective analysis, small but significant differences in NBME subject exam scores favoring those who were assigned psychiatry clerkships later in the academic year were reported.

In our study, about 4% of participants selected a psychiatric residency. Similar figures for career choice have been reported by others (³⁴) and appear to be in keeping with career choice trends by U.S. medical school graduates (³⁶). Negative views about the specialty and the patients it serves held by medical students, both before and after entry into medical school (³⁷, ³⁸), have been documented. Negative attitudes about the specialty have been associated with a decline over the past several years in psychiatry as a career choice among medical students (³⁹—⁴²). The existing literature offers only mixed results when considering the effect of rotation type or quality on positive change in attitudes about psychiatry, its patients or increased recruitment into the field (⁴³—⁴⁸); however, high quality supervision and having a more active role in the care of patients appears to correlate with positive change in attitude and recruitment numbers (⁴⁸—⁵⁰). On the other hand, it is not clear that change in attitudes alone would be sufficient to increase the number of students likely to choose psychiatry as a career. This may be especially true given a reemerging interest in increasing the number of generalist physicians in the United States (⁵¹).

In this study, we found an inverse relationship between GPA and choice of psychiatric residency; however, in absence of more specific information about the relationship between GPA and choice of specific nonpsychiatric residencies, this finding is difficult to interpret. Based on our data, we cannot conclude that less academically strong students were more likely to choose a psychiatric residency. The mean GPA in this group equated with a B average in terms of letter grade, a level of performance that would not be considered marginal or poor.

For performance outcome measures, we reported only standardized examination results. We felt that the other available outcome measures of academic performance (e.g., nonstandardized departmental examination scores, grades on a required end-rotation essay, grades of formal clinical write-ups, and performance evaluations by clinical preceptors) were too subjective or did not lend themselves to statistical analysis, even though their inclusion may have provided more information in "skill-based" performance. Performance data using standardized patients were not available for this sample, despite reports of their successful implementation (⁵², ⁵³). The surveys used in this study were not themselves standardized instruments.

Notwithstanding these limitations, the findings from our study indicate that knowledge-based educational objectives for a psychiatry clerkship may be met regardless of rotation type. Eventual choice of specialty, student attitudes about psychiatry as a discipline or psychiatric patients, motivation to pursue psychiatry as a career, and subjective ability to acquire knowledge also appear to be unrelated to rotation type.

View Large | Download Slide(.ppt)
Add to My POL

TABLE 1. Demographic and Enrollment Information of Study Participants

View Large | Download Slide(.ppt)
Add to My POL

TABLE 2. NBME "Shelf" Exam Scores by Rotation and Rotation Setting Using Adjusted Linear Regression*

View Large | Download Slide(.ppt)
Add to My POL

TABLE 3. Selection of Psychiatry Residency by Rotation Type

View Large | Download Slide(.ppt)
Add to My POL

APPENDIX 1. Surveys A and B

Acknowledgments

We are grateful to Michaela Nevin for her valuable administrative support and assistance with collecting data for this project.

Dr. Bobo has been a member of the Speaker's Bureau for Janssen and Pfizer in the last 12 months. Drs. Nevin, Greene, and Lacy disclosed no competing interests.

References

Campos-Outealt D, Witzke DB, et al: Correlations of family medicine clerkship evaluations with scores on standard measures of academic achievement. Fam Med 1994; 26:85—88

Dolmans DH, Wolfhagen IH, Essed GG, et al: The impacts of supervision, patient mix, and numbers of students on the effectiveness of clinical rotations. Acad Med 2002; 77:332—335

Edwards RK, Davis JD, Kellner KR: Effect of obstetrics-gynecology clerkship duration on medical student examination performance. Obstet Gynecol 2000; 95:160—162

Hampton HL, Collins JB, Perry KG Jr, et al: Order of rotation in third-year clerkships: influence on academic performance. J Reprod Med 1996; 41:337—340

Holmes FF, Holmes GE, Hassanein R: Performance of male and female medical students in a medicine clerkship. JAMA 1978; 239:2259—2262

Jacobson MH, Sherman L, Perlman I, et al: Clerkship site and duration: do they influence student performance? Surgery 1986; 100:306—311

Lind DS, Marum T, Ledbetter D, et al: The effect of the duration and structure of a surgery clerkship on student performance. J Surg Res 1999; 84:106—111

Manyon A, Shipengrover J, McGuigan D, et al: A Defining differences in the instructional styles of community preceptors Fam Med 2003; 35:181—186

Pritchard DJ: Effects of sex and alphabetical listing on examination performance of medical students. Med Educ 1988; 22:205—210

Reteguiz JA, Crosson J: Clerkship order and performance on family medicine and internal medicine national board of medical examiners exams. Fam Med 2002; 34:604—608

Wang-Cheng RM, Fulkerson PK, Barnas GP, et al: Effect of student and preceptor gender on clinical grades in an ambulatory clerkship. Acad Med 1995; 70:324—326

Whalen JP, Moses VK: The effect on grades of the timing and site of third-year internal medicine clerkships. Acad Med 1990; 65:708—709

White CB, Thomas AM: Students assigned to community practices for their pediatric clerkship perform as well or better on written examinations as students assigned to academic medical centers. Teach Learn Med 2004; 16:250—254

Pollock DC, Byrne PN, Shanley DF: Personality and performance in psychiatric education. Med Educ 1982; 16:44—47

Golubic R, Golubic K: What do grades in clinical subjects depend on? Case study of the Zagreb University School of Medicine. Croat Med J 2004; 45:67—71

Keitner GI, Baldwin LM, McKendall MJ: Gender and psychiatric clerkship performance. Can J Psychiatry 1984; 29:327—329

Irby D: Teaching and learning in ambulatory care settings: a thematic review of the literature. Acad Med 1995; 70:898—931

O’Sullivan PS, Weinberg E, Boll AG, et al: Students’ educational activities during clerkship. Acad Med 1997; 72:308—313

Briones DF, Johnson DT: The community mental health center as a setting for a psychiatry junior clerkship. J Med Educ 1978; 53:759—763

Frank D, Propst A, Goldhamer P: The effects of teaching medical students psychotherapy skills in the outpatient department. Can J Psychiatry 1987; 32:185—189

Reiser LW, Sledge WH, Edelson M: Four-year evaluation of a psychiatric clerkship: 1982—1986. Am J Psychiatry 1988; 145:1122—1126

Griffin P, et al: Consultation-liaison and inpatient psychiatry clerkships: comparison of learning outcomes and student attitudes. J Psychiatr Education 1982; 6:200—206

Yano B, Markoff R, Alexander L, et al: An alternative model for the psychiatric clerkship. Med Educ 1981; 15:116—121

Harsch HH, Young LD: The psychiatric rotation: traditional inpatient or consultation clerkship. Psychosomatics 1985; 26:942—945

Weddington WW Jr, Hine FR, Houpt HL, et al: Consultation-liaison versus other psychiatry clerkships: a comparison of learning outcomes and student reactions. Am J Psychiatry 1978; 135:1509—1512

Puryear DA, Miller DA: The effect of clerkship site on medical student examination grades. Med Educ 1990; 24:239—242

Bowen J, Stearns JA, Dohner C, et al: Defining and evaluating quality for ambulatory care educational programs. Acad Med 1997; 72:506—510

Gary N: Barriers to medical student education in ambulatory settings (editorial). J Med Educ 1987; 62:527—529

Lesky LG, Hershman WY: Practical approaches to a major educational challenge: training students in the ambulatory setting. Arch Intern Med 1995; 155:897—904

Pessar LF: Ambulatory care teaching and the psychiatric clerkship. Acad Psychiatry 2000; 24:61—67

Skeff KM, Bowen JL, Irby DM: Protecting time for teaching in the ambulatory care setting. Acad Med 1997; 72:694—697

Frank SH, Stange KC, Langa D, et al: Direct observation of community-based ambulatory encounters involving medical students. JAMA 1997; 278:712—716

Case SM, Ripkey DR, Swanson DB: The effects of psychiatry clerkship timing and length on measures of performance. Acad Med 1997; 72(suppl 1):S34—S36

Newton DA, Grayson MS: Trends in career choice by US medical school graduates. JAMA 2003; 290:1179—1182

Hoge C, Castro CA, Messer SC, et al: Combat duty in Iraq and Afghanistan, mental health problems, and barriers to care. N Engl J Med 2004; 351:13—22

Feifel D, Moutier CY, Swerdlow NR: Attitudes toward psychiatry as a prospective career among students entering medical school. Am J Psychiatry 1999; 156:1397—1402

Zinny GH, Sata LS: Influence of factors before and during medical school on choice of psychiatry as a specialty. Am J Psychiatry 1986; 143:77—80

Sierles FS, Taylor MA: Decline of US medical student career choice of psychiatry and what to do about it. Am J Psychiatry 1995; 152:1416—1426

Brockington I, Mumford D: Recruitment into psychiatry. Br J Psychiatry 2002; 180:307—312

Hamilton P: Planning and implementing a positive psychiatric rotation. J Psychiatr Education 1982; 6:166—173

Rajagopal S, Rehill KS, Godfrey E: Psychiatry as a career choice compared with other specialties: a survey of medical students. Psychiatric Bulletin 2004; 28:444—446

Eagle PF, Marcos LR: Impact of the outpatient clerkship on medical students. Am J Psychiatry 1980; 137:1599—1602

Galletly CA, Schrader GD, Chesterman HM, et al: Medical student attitudes to psychiatry: lack of effect of psychiatric hospital experience. Med Educ 1995; 29:449—451

Maurice WL, Klonoff H, Miles JE, et al: Medical student change during a psychiatric clerkship: evaluation of a program. J Med Educ 1975; 50:181—189

Wilkinson DG, Toone BK, Greer S: Medical students’ attitudes to psychiatry at the end of the clinical curriculum. Psychol Med 1983; 13:655—658

Yager J, Lamotte K, Nielsen A, et al: Medical students’ evaluation of psychiatry: a cross-country comparison. Am J Psychiatry 1982; 139:1003—1009

Fabrega H: Does a clerkship affect students’ views of psychiatric patients? J Nerv Ment Dis 1995; 183:736—742

Fox HA, McGahee CL, Bluestone H: Quality in a psychiatric clerkship. Hillside J Clin Psychiatry 1981; 3:177—185

Linet LS: Challenging the medical student in a psychiatry clerkship. Am J Psychiatry 1977; 134:53—55

Scher M: Comparison of two formats for a psychiatry clerkship. J Med Education 1988; 63:140—143

Schwartz MD, Basco WT, Grey MR, et al: Rekindling student interest in generalist careers. Ann Intern Med 2005; 142:715—724

Hall MJ, Adamo G, McCurry L, et al: Use of standardized patients to enhance a psychiatry clerkship. Acad Med 2004; 79:28—31

Bennett AJ, Arnold LM, Welge JA: Use of standardized patients during a psychiatry clerkship. Acad Psychiatry 2006; 30:185—190