Since World War II, university-based medical training programs have relied increasingly on U.S. Department of Veterans Affairs (VA) Medical Center programs and staff to teach medical students and residents (1–3). Few studies have assessed the educational effectiveness of these programs. A large educational satisfaction survey of psychiatry, medicine, surgery, and subspecialty residents assigned to VA facilities found high satisfaction with VA faculty (87%, n=1,597) and the learning environment (78%, n=1,120) (4). Only a few studies have measured outcomes of VA-based education; these include a comparison of surgical outcomes between teaching and nonteaching VA hospitals (5) and an assessment of the impact of psychopharmacology education of residents and medical students on prescribing practices (6). We could identify no studies assessing educational outcomes for VA compared with non-VA clinical sites for psychiatry or other medical clerkships.

Ensuring “comparable educational experiences and equivalent methods of evaluation across all alternative instructional sites” is a required medical educational objective (ED-8) among accreditation standards of the Liaison Committee on Medical Education (7). An important tool for assessing medical students’ clinical knowledge gained from clinical clerkships is a standardized exam, such as the National Board of Medical Examiners (NBME) subject exam (8, 9). Researchers have had varying results in examining effects of clinical sites on final exam scores within psychiatry, primary care, and obstetrics and gynecology clerkships (10–12).

Our study answered the following research question: Within a multisite psychiatry clerkship, do students assigned to VA inpatient or outpatient clinical sites perform as well on the NBME subject exam for psychiatry as students assigned to non-VA sites, controlling for the preclerkship knowledge base? We conducted this study in a third-year psychiatry clerkship at a southwestern university medical school.

After obtaining approval from the Institutional Review Board of the University of Oklahoma Health Sciences Center, we obtained the following data for three consecutive classes of third-year medical students from 2001 to 2004:

To estimate students’ baseline knowledge about psychiatry, we used the final grade from the Human Behavior II course, which covers diagnosis, etiology, and treatment of psychiatric disorders. To measure postclerkship knowledge, we used the NBME subject exam, which was administered to students on the last day of the 6-week rotation.

Out of a total of 440 students who completed basic sciences education at the Oklahoma City campus during the study years and who had Human Behavior II course scores, we eliminated 153 students who completed their clinical training at the Tulsa campus and for whom we did not have NBME scores and 21 students from inpatient and five students from outpatient group sampling who alternated between multiple part-time sites. Thus, the present study is limited to full-time psychiatry clerkship sites at the Oklahoma City campus.

The inpatient sites included the VA inpatient psychiatry unit and the non-VA University (general adult), Integris (community child/adolescent), Presbyterian (geropsychiatry) hospitals, and the Oklahoma Youth Center (community child/adolescent). Full-time outpatient sites included the VA posttraumatic stress unit, the outpatient and substance abuse clinics, and the non-VA Decisions Day Treatment (a community-based program) and consultation-liaison sites (university hospital and clinics). Students were assigned to sites according to their preference; they could have been assigned to two VA sites, two non-VA sites, or one VA and one non-VA site.

A stratified random sample (N=140) was created for each of the inpatient and outpatient site comparisons through the following processes. The 440 students for whom Human Behavior II course scores were available was classified into the first stratum, which consisted of four quartile groups of Human Behavior II course scores to control for preclerkship clinical knowledge. The 287 students with reported NBME scores were classified into the second stratum, which consisted of VA and non-VA sites. After excluding data for students in part-time rotations, 266 students were identified for inpatient and 152 students for outpatient sampling processes. In each of the inpatient and outpatient groups, we conducted random sampling until we could achieve the condition in which classified quartile groups were sampled 35 times each and assigned to the VA or non-VA group. The numbers were 17, 18, 17, 18 for the VA group, and 18, 17, 18, 17 for the non-VA group (totaling 70 each). With Levene’s test for equality of variances, independent sample t tests were conducted to test the equality of group means. Significance was set at 5%, with 80% power for the two-tailed t tests (13).

Table 1 shows inpatient site comparison results for group mean differences between VA and non-VA sites on NBME psychiatry subject scores. There were no significant differences between groups for inpatient sites (p=0.84). The non-VA inpatient group was the aggregated group of four inpatient sites.

Table 2 depicts the outpatient site comparison results for group mean differences between VA and non-VA sites on NBME psychiatry subject scores. As with inpatient group results, we found no significant differences between groups on NBME scores (p=0.88). The VA outpatient group was aggregated from three sites and the non-VA outpatient group was aggregated from two sites.

Our study determined that for psychiatry clerkship students from three successive medical student classes who were included in the analyses, those rotating at VA inpatient and outpatient sites performed equally as well as those assigned to non-VA sites on the standardized outcome measure of NBME psychiatry subject examinations. Preclerkship knowledge base was controlled for using Human Behavior II course grades. Establishing intersite consistency is an important part of educators’ development, implementation, and revision of clerkship programs. Because VA clinical programs teach a large proportion of medical students in our psychiatry clerkship and in many other medical school programs, it is essential to demonstrate their educational value through objective, standardized measures.

VA programs offer important, unique mixes of patients from whom medical students learn under the supervision of experienced clinicians and residents. As the U.S. population ages, older veterans suffer a wide range of combined general medical and psychiatric illnesses, especially those related to military trauma. More recently, this patient mix includes younger veterans returning from war who are experiencing recent-onset posttraumatic stress disorder and traumatic brain and other injuries, as well as increasing numbers of women veterans with families.

An inherent limitation of our study is that it is unknown whether NBME scores correlate with clinical competence. Future assessments of students’ learning at VA sites could examine “hands-on” clinical skills through standardized measures of communication skills and written medical formulations, such as objective structured clinical evaluations (OSCEs). Other outcome measures could be United States Medical Licensing Examination Step 2 Clinical Skills and Clinical Knowledge components, if available, or more challenging assessments of clinical competence later in careers. Additionally, future studies might assess qualities of patient care that are more difficult to quantify, such as compassion, empathy, and persistence of caretakers.

Other limitations include the lack of NBME data for students assigned to the smaller Tulsa campus and to more than one outpatient site, leading to a smaller sample size. Also, our sample is limited to three consecutive class years within a single institution, so our results may not be generalizable to other medical schools. However, our study may serve as a catalyst for additional studies at other medical schools; if this occurred, an alternative to using preclerkship Human Behavior II course scores would be to give a preclerkship exam as a standardized way to assess baseline knowledge. We did not examine other factors that might affect student performance, such as timing of clerkships during the academic year, potential differences in teaching effectiveness between site supervisors, or differences in patients’ diagnoses across sites. We also did not examine the effects of the standardized, required weekly clerkship lecture and teaching conference series. (However, all students attended the same didactics lectures in a central site.) Future studies could assess these factors in comparing VA and non-VA clerkship sites. A final improvement in our study could be to consider only students assigned to one VA and one non-VA site to reduce possible differences in exam performance due to variability in student inpatient and outpatient assignments.

Nonetheless, our study accomplished an important goal in using one standardized educational measure to demonstrate comparable clinical learning at VA sites compared with non-VA sites. Continuing assessments of VA programs’ educational value are important to strengthen collaboration between academic and VA-based clinical centers in medical training.

The opinions contained above are those of the authors and do not reflect the opinion of the Department of Defense or the U.S. Department of Veterans Affairs.

At the time of submission, the authors disclosed no competing interests.