Acad Psychiatry 30:185-190, May-June 2006
doi: 10.1176/appi.ap.30.3.185
© 2006 Academic Psychiatry
Use of Standardized Patients During a Psychiatry Clerkship
Aurora J. Bennett, M.D.,
Lesley M. Arnold, M.D. and
Jeffrey A. Welge, Ph.D.
Received May 26, 2005; revised October 6, 2005; accepted October 12, 2005. Drs. Bennett and Arnold are affiliated with the University of Cincinnati, Cincinnati, Ohio. Address correspondence to Dr. Bennett, Univ. of Cincinnati, 222 Piedmont Ave., Medical Arts Building, Ste. 8500, Cincinnati, OH 45219; aurora.bennett{at}uc.edu (E-mail). Copyright © 2006 Academic Psychiatry.
|
ABSTRACT
|
OBJECTIVE: Standardized patients are used in teaching medical students and evaluating their clinical skills during the psychiatric clerkship. The purpose of this study was to evaluate whether the addition of a Psychiatry Clinical Standardized Patient Examination (PCX) during the third-year clerkship improved students’ performances on the psychiatry component of the Clinical Competency Examination (CCX) that is used at our institution to prepare fourth-year students for the National Board of Medical Examiners Step 2 Clinical Skills Exam (Step 2 CSA).METHOD: Scores on the CCX were analyzed by two-by-two tables (yes/no response to each clinical competency item versus took/did not take PCX) using Fisher’s exact test. RESULTS: Of the 469 students who took the CCX between 2002 and 2004, 246 students also took the PCX. Compared to students who did not take the exam, students who took the PCX were significantly more likely (p<0.05) to receive positive ratings on the CCX items for professionalism, addressing patients’ stressors at work and home and inquiring about thoughts of harm to self or others. CONCLUSIONS: The implementation of the PCX during the psychiatry clerkship significantly improved student performance on several important components of the CCX, which is used to prepare students for the Step 2 CSA. This suggests that the use of standardized patients during the clerkship may help students improve their interviewing and interpersonal skills.
|
INTRODUCTION
|
There is consensus within the medical field that empathic, professional, and patient-centered communication between the physician and patient improves patient satisfaction, adherence to treatment, and clinical outcomes (1–5). Despite the importance of communication skills in optimizing patient care and outcome, medical educators express concerns about deficiencies in trainees’ interpersonal skills (6–9). As a result, medical school education has focused increasingly on the development of clinical and interpersonal skills, in addition to the traditional emphasis on basic sciences. In 1991, the Liaison Committee on Medical Education (LCME) required that medical institutions determine, through direct observation, that students acquire core clinical skills (10, 11).
In 1964, Barrows and Abrahamson (11) introduced the concept of using standardized patients in medical education to provide opportunities for students to develop clinical skills. Standardized patients are individuals specifically trained to portray clinical scenarios in a valid and reliable manner. In 1992, the American Association of Medical Colleges (AAMC) supported the use of standardized patients in teaching and evaluating students’ clinical skills, which cannot be assessed adequately through written examinations (12, 13). More than 90% of LCME-accredited medical schools now use standardized patients in their educational curriculum (14). The Accreditation Council for Graduate Medical Education (ACGME) has also encouraged residency programs to utilize standardized patients for assessing and teaching competence in interviewing, patient education, physical examination, interpersonal communication, and professionalism (15). As of the graduating medical student class of 2005, the National Board of Medical Examiners (NBME) has instituted the Step 2 Clinical Skills Exam (Step 2 CSA), which uses standardized patients in several of its stations to determine whether students have achieved the minimum clinical competencies necessary to enter their residency training (16).
We recently developed the Psychiatry Clinical Standardized Patient Examination (PCX) which is administered to third-year students during the psychiatry clerkship. The goal of this study was to determine the impact of the addition of this interview on the development of students’ psychiatric clinical interviewing skills. Our hypothesis was that the PCX would improve the students’ performance on the psychiatry component of the Clinical Competency Examination (CCX), which is given at our institution during the fourth year of medical school to prepare students for the Step 2 CSA. This study was approved by the Institutional Review Board at the University of Cincinnati.
|
Method
|
Standardized Patients
Standardized patients are recruited locally by the director of the Center for Competency Development and Assessment (CCDA) at our institution, a center that is funded by the medical college to design and implement all standardized patient examinations and teaching sessions. The standardized patients are trained by the director of the CCDA and the psychiatry clerkship director for their portrayals in the PCX. There are three standardized patients trained for each of the three psychiatry case vignettes. The training focuses on standardizing both the portrayals of the vignettes and the completion of the grading checklists.
Psychiatry Clinical Vignettes
The psychiatry vignettes and standardized patient grading checklists are written by the psychiatry clerkship director. The vignettes contain all components of a comprehensive interview, including identifying information, chief complaint, history of present illness, past psychiatric and medical histories, family history, social history, substance use history, and mental status examination. The standardized patient grading checklists consist of 20 items that are deemed to be the most important to elicit the history and mental status, and six items that cover the conduct of the interview. Each item is assigned a point value by the clerkship director, with 50 points being the maximum number that can be attained.
Psychiatry Examination (PCX)
The PCX was first implemented as a pilot examination that was given to all third-year psychiatry clerkship students from January 2003 to June 2003. During the pilot phase, the clerkship director made minor changes to the exam and examination process based on student feedback, but there were no changes to the scripts or checklists. Since July 2003, the PCX has constituted 15% of the final clerkship grade. No other changes were made in the clerkship curriculum. It is given at the end of the fourth week of the 6-week psychiatry clerkship and consists of the student performing a 30-minute interview of a standardized patient. The interview is videotaped for quality and educational review purposes. The student has the option of documenting the clinical data on blank paper, but is not allowed to bring screening instruments or other written material into the examination room. Each student is then given 2 hours to write a complete history, mental status examination, differential diagnosis, and comprehensive treatment plan, which are e-mailed to the clerkship director. The clerkship director grades all write-ups using a grading system that assigns point values to each component of the write-up as is done for the standardized patient checklist. The maximum possible score for the write-up is 50 points. The student is e-mailed a grade sheet that provides the total number of points earned on each component of the write-up, the total points earned on the standardized patient grade sheet, and narrative comments describing strengths and deficits that resulted in the deduction of points by the standardized patient and the clerkship director. The clerkship director views selected videotapes of interviews for the following reasons: a discrepancy between the standardized patient grade sheet and the student write-up, a poor performance on the write-up or standardized checklist; random review of tapes to monitor standardization of the portrayals of the vignettes; and review of tapes of students who are known by the clerkship director to have academic problems or previously identified deficits in professionalism.
Students who score < 80 out of the 100 points possible on the examination (mean = 88.23 [SD = 5.6]) are required to remediate by reviewing their tape and write-up with the clerkship director. Approximately 10% of students fall within this category. Reexaminations are not offered. Although all other students are invited to contact the clerkship director to review their videotapes and write-ups individually, less than 1% elect to do so.
Clinical Competency Examination (CCX)
The CCX, implemented at our institution for the graduating class of 2002, is conducted in the fourth year of medical school, after completion of all of the clinical clerkships. There are seven stations that use standardized patients. The students are informed that all of the major clerkships are represented in the examination but are not told which clerkship is represented by which station. The standardized patient vignettes are written by the Clinical Competency Examination Committee, which consists of faculty from primary care and specialty fields representative of the major third-year clerkships. The standardized patients are trained for the CCX by the CCDA director. The duration of each station is 15 minutes. Most of the stations provide minimal clinical data in written format prior to the student performing the 15-minute exercise, which may be a history and/or physical examination followed by an interstation exercise. One of the stations involves the student performing a 15-minute interview of a standardized patient portraying a depressed patient. The student is provided only with the chief complaint of "fatigue" and a list of normal vital signs before entering the room to perform the interview. The student is observed conducting the interview in real time by a faculty member via a monitor. The student then gives a 15-minute presentation of the case, including differential diagnosis and proposed treatment plan, to the same faculty member.
Comparison of the PCX and CCX
Both the PCX and the psychiatry case for the CCX include assessments of mood and anxiety symptoms, environmental stressors, pertinent medical symptoms, support systems, psychiatric and medical history, substance use, family history, dangerousness to self or others, perceptual disturbances, and cognitive function. The conduct of both interviews is also examined and includes a determination of the student’s effective use of open- and close-ended questions, eye contact, display of a nonjudgmental attitude, professional demeanor, and empathy. As in the PCX, students taking the CCX are graded by the standardized patient. Furthermore, in the CCX, the faculty member viewing the interview via the video monitor completes the same checklist of items as the standardized patient. The faculty checklist has a few additional items evaluating the student’s presentation, differential diagnosis, and treatment plan for the patient. The interrater reliability between the standardized patients and the faculty on the CCX is 0.9 for the content areas and 0.4 for the process of the interview, with the standardized patients typically grading the students higher in the latter area than the faculty. All interviews in the CCX are viewed by faculty via a monitor, whereas in the PCX only specific student interviews are viewed by the clerkship director who grades the write-ups.
The two differ somewhat in their design and purpose. The CCX provides 15 minutes per station to assess each student’s ability to discern the most important issues within the context of brief patient contact, as may be typical of a primary care practice. In contrast, the 30 minutes allotted to the PCX allows the student a more focused and in-depth examination of the patient, which provides the material for judging each student’s ability to obtain a comprehensive history and perform a complete mental status examination from which is derived a differential diagnosis and treatment plan. In the CCX, students are graded by a different faculty member at each station, whereas in the PCX, all of the students are graded by the clerkship director.
Statistical Analysis
Each CCX item was scored as positive (student appropriately addressed the content of the item in the interview) or negative. For both faculty evaluations and standardized patient evaluations, these data were cross-tabulated against whether or not the student had taken the PCX. The resulting two-by-two tables were analyzed with Fisher’s exact test. A significant effect of the PCX on the rate of positive endorsement of a CCX item was inferred when p<0.05.
|
Results
|
Of the 469 students who took the CCX in their fourth year of medical school between 2002 and 2004, 246 students also took the third-year PCX during their psychiatry clerkship when it was implemented in January 2003. The students who did not take the PCX included all third-year psychiatry clerkship students in academic year 2001–2002 (July 2001 to June 2002) and the first half of academic year 2002–2003 (July 2002 to December 2002). The students who took the PCX included all third-year psychiatry clerkship students in the second half of academic year 2002–2003 (January 2003 to June 2003) and all of the third-year psychiatry clerkship students in academic year 2003–2004 (July 2003 to June 2004). Table 1 summarizes the demographic characteristics of the students, which remained stable over the 3 years of this study. Table 2 shows the significance levels observed for each CCX item rated by the standardized patient and the faculty evaluators. Significant results in all cases favored the student group that had taken the PCX. Students who took this clerkship examination were significantly more likely to receive positive ratings on items reflecting professionalism from both the standardized patient and the clinical evaluators. For example, both evaluators rated these students as more often treating the patient with respect, using language the patient could understand, and behaving in a professional manner. The students who had completed the PCX were also significantly more likely than students who had not had this clerkship examination to address other symptoms and concerns, such as patients’ stressors at work and home and thoughts of harming themselves or others.
View this table:
[in this window]
[in a new window]
|
TABLE 2. Comparison of CCX Items Between the Groups That Did and Did Not Take the PCX. All Significant Values Favor the Student Group That Had Taken the PCX*
|
|
Discussion
|
The development of medical students’ communication skills is an important goal of medical education. These skills include the ability to gather pertinent clinical data from patients and educate and counsel patients in a professional and empathic manner. The clinical biennium provides students with opportunities to learn these clinical skills. Unfortunately, competing responsibilities of the faculty members and financial pressures may limit their availability to provide direct observation of students interviewing a patient.
Standardized patients have emerged as a means to supplement the teaching and evaluation of students’ clinical skills. This study was designed to determine whether the use of standardized patients in a third-year psychiatry clerkship had an effect on students’ subsequent performances on a clinical skills examination in their fourth year of medical school.
We found that students who previously had conducted a comprehensive psychiatric interview of a standardized patient during their psychiatry clerkship demonstrated significant improvement in data gathering, safety assessment, and professional demeanor during the psychiatric component of the fourth-year standardized patient examination (CCX), which is similar in design to the Step 2 CSA. During the psychiatry component of the CCX, the standardized patients and the faculty evaluators found that the students who had taken the PCX in their third-year clerkship were significantly more likely than those students who did not take the clerkship examination to obtain data from the standardized patients about environmental stressors, support systems, and suicidal/homicidal ideation. There was also a significant improvement in their ability to conduct the interview in a respectful and professional manner. These results differ from those of a previous study in which there was no correlation between students’ scores on a third-year family medicine objective structured clinical examination (OSCE) and the fourth-year examination that was also designed similarly to the Step 2 Clinical Skills Exam (17). The lack of an association may have been a result of different clinical tasks evaluated in the two examinations in contrast to the similarities between the psychiatric stations in the CCX and the PCX.
Student feedback has been positive about the usefulness of the PCX in improving their interviewing skills and helping them learn how to gather pertinent patient data and perform a comprehensive mental status examination. The requirement to submit a written diagnostic evaluation with a differential diagnosis and treatment plan within 2 hours of completing the interview is seen by the students as comparable to what will be expected of them at their clinical sites. They have welcomed the opportunity to have the clerkship director provide detailed feedback about their interviewing skills. All students who have had to remediate the PCX expressed gratitude for the opportunity to view their interview and have a detailed discussion regarding their strengths and relative weaknesses. The CCDA director has received similar positive student feedback about the usefulness of the CCX experience in preparing them for the Step 2 CSA.
There are several limitations that must be considered when evaluating the results of this study. There are no data currently available regarding the validity and reliability of the PCX and CCX. Both examinations can provide only a cross-sectional view of students’ clinical skills in contrast to having a longitudinal view of students’ performances on their clinical rotations. The CCX provides data on their interactions in a variety of clinical situations with actual patients as opposed to the portrayals offered by standardized patients in the PCX. No analyses were performed to look for a correlation between students’ performances on the standardized patient examinations and their clinical rotation evaluations or psychiatry written exam scores. Previous studies have found conflicting results regarding correlations among the various graded components of the psychiatry clerkship, possibly due, in part, to some variation in the skills tested by each of the components (18). Future studies should examine whether students who complete the PCX also demonstrate improved interviewing skills across all of the CCX patient presentations. It would have been interesting to obtain student feedback regarding their perceptions of the impact of the psychiatry examination on their performance in the psychiatry portion of the CCX. Further data are also needed on the reliability and validity of the PCX and CCX, as recommended in previous reports (19).
Medical educators have mandated that students demonstrate a minimum level of competency in the delivery of patient care and in their acquisition of scientific knowledge before they can obtain their medical license. The use of standardized patients to supplement the training and evaluation of students’ clinical skills has provided a standardized method of assessing students’ communication and problem-solving skills. Our study demonstrated that the use of a standardized patient examination during a third-year psychiatry clerkship provides a potentially valuable experience that may improve the students’ interviewing skills. Further studies are needed to evaluate whether standardized patient exercises during medical school have a positive impact on the physician-patient interactions in the residency years and beyond.
|
ACKNOWLEDGMENTS
|
Supported by a medical education grant from the Office of the Dean
|
REFERENCES
|
- Blue AV, Chessman AW, Gilbert GE, et al: Responding to patients’ emotions: important for standardized patient satisfaction. Fam Med 2000; 32:326–330[Medline]
- Stewart MA: Effective physician-patient communication and health outcomes: a review. Can Med Assoc J 1995; 152:1423–1433[Abstract]
- Roter DL, Stewart M, Putnam SM, et al: Communication patterns of primary care physicians. J Am Med Assoc 1997; 277:350–356[Abstract/Free Full Text]
- Holmboe ES: Faculty and the observation of trainees’ clinical skills: problems and opportunities. Acad Med 2004; 79:16–22[CrossRef][Medline]
- Kernan WN, Holmboe E, O’Connor PG: Assessing the teaching behaviors of ambulatory care preceptors. Acad Med 2004; 79:1088–1094[CrossRef][Medline]
- Pfeiffer C, Madray H, Ardolino A, et al: The rise and fall of students’ skill in obtaining a medical history. Med Educ 1998; 32:283–288[CrossRef][Medline]
- Nutter D, Whitcomb M: The AAMC Project on the Clinical Education of Medical Students. Washington, DC, Association of American Medical Colleges, 2002
- Hall MJ, Graceanne A, McCurry L, et al: Use of standardized patients to enhance a psychiatry clerkship. Acad Med 2004; 79:28–31[CrossRef][Medline]
- Kassebaum DG, Eaglen RH: Shortcomings in the evaluation of students’ clinical skills and behaviors in medical school. Acad Med 1999; 74:842–849[Medline]
- Functions and Structure of a Medical School: Accreditation and the Liaison Committee on Medical Education; Standards for Accreditation of Medical Education Programs Leading to the M.D. Degree. Washington, DC, and Chicago, IL: Liaison Committee on Medical Education, 1991
- Barrows HS, Abrahamson S: The programmed patient: a technique for appraising student performance in clinical neurology. J Med Educ 1964; 39:802–805[Medline]
- Anderson MB: Educating medical students. Assessing change in medical education—the road to implementation. Acad Med 1993;68(6Suppl):S1-46
- Yudkowsky R: Should we use standardized patients instead of real patients for high-stakes exams in psychiatry? Acad Psychiatry 2002; 26:187–192[Free Full Text]
- LCME annual medical school questionnaire, 1999-2000. Data available from the Association of American Medical Colleges, www.aamc.org
- Accreditation Council for Graduate Medical Education: The outcome project. Available at www.acgme.org
- United States Medical Licensing Examination. An analysis of U.S. student field trial and international medical graduate certification test results for the proposed USMLE clinical skills examination, 2003. Available at www.usmle.org/news/cse/ceftresults
- Chessman AW, Blue AV, Gilbert GE, et al: Assessing students’ communication and interpersonal skills across evaluation settings. Fam Med 2003; 35:643–648[Medline]
- McLay RN, Rodenhauser P, Anderson DS, et al: Simulating a full-length psychiatric interview with a complex patient: an OSCE for medical students. Acad Psychiatry 2002; 26:162–167[Abstract/Free Full Text]
- Hodges B, Hanson M, McNaughton BA, et al: Creating, monitoring, and improving a psychiatry OSCE. Acad Psychiatry 2002; 26:134–161[Free Full Text]
This article has been cited by other articles:
|
|
|
|
 
W. V. Bobo, R. Nevin, E. Greene, and T. J. Lacy
The Effect of Psychiatric Third-Year Rotation Setting on Academic Performance, Student Attitudes, and Specialty Choice
Acad Psychiatry,
March 1, 2009;
33(2):
105 - 111.
[Abstract]
[Full Text]
[PDF]
|
|
|
|
|
|
|
A. M. Brenner
Uses and Limitations of Simulated Patients in Psychiatric Education
Acad Psychiatry,
March 1, 2009;
33(2):
112 - 119.
[Abstract]
[Full Text]
[PDF]
|
|
|
|
|
|
|
N. A. Vaidya
Psychiatry Clerkship Objective Structured Clinical Examination is Here to Stay
Acad Psychiatry,
May 1, 2008;
32(3):
177 - 179.
[Full Text]
[PDF]
|
|
|
Get information about faster international access.
Privacy Policy
Copyright © 2006
Academic Psychiatry.
All rights reserved.
Home
| Search
| Current Issue
| Past Issues
| Subscribe
| All APPI Journals
| Help
| Contact Us
|
TOP
ABSTRACT
INTRODUCTION
