Medical education, in general, and psychiatry education, in particular, has been changing dramatically. Clerkships now include goals and objectives and have well-organized clinical experiences and didactic activities. It is no longer acceptable that trainees be evaluated by vague impressions and global rating scales, and take written exams largely detached from the clinical setting. As the formal curriculum evolved, the demand for change in evaluation of students’ and residents’ performance has increased.

One of the major changes in evaluation has been the development of the Objective Structured Clinical Examination (OSCE). Hardin and Gleeson (1) outlined the elements of a performance-based examination called the Objective Structured Clinical Examination. This examination consisted of various stations that required students to perform certain activities determined by previously stated objectives for each of the stations. Students were evaluated on their ability to carry out these tasks. In this examination format, typically each station was of short duration and focused on a specific skill or activity. Students were rated by faculty based on direct observation and/or rating scales.

This concept was developed further by other investigators by increasing the length of each station and requiring fourth-year medical students to complete more complex and comprehensive activities (2) and the reliability of the examination process has been established at acceptable levels (3).

Application of the OSCE to the psychiatric clerkship has been slow. Hodges et al. (4) described the use of the OSCE in evaluating psychiatry clerkship students and importantly established some of the costs associated with this format of testing. The same group later published a study of validity, comparing student and resident performance in psychiatry that supported the utility of this examination format in psychiatry (5). The use of OSCE in U.S. medical schools is increasing. Psychiatric educators agree that standardized patient-based assessment offers the opportunity to significantly upgrade the validity, reliability, and fairness of clinical examinations of psychiatry (6–9).

Not everyone is in favor of the use of the OSCE examination. Ramchandani (10), in the preceding article, raises the question of OSCE’s suitability in evaluating third-year medical students. My disagreement with Dr. Ramchandani stems from two assumptions he has made: psychiatry is not like any other specialty, and despite the efficacy and validity of the OSCE in other specialties, somehow results of psychiatric education cannot be tested by OSCE; and third-year students are not advanced enough to synthesize the psychiatric information in a meaningful manner. I would attempt to address both of these issues.

The Diagnostic Process in Psychiatry is Unique

Guze (11) has long argued against the special nature of psychiatry. He asserted that if psychiatric disorders are to receive the same stature of other illnesses, psychiatrists need to treat psychiatry as a branch of medicine. We must view the mind as an organ of the brain and treat psychiatric illnesses as disorders of the brain. The process of diagnosis in psychiatry is no different from diagnosing any other medical disorder. When we encounter a patient who comes to us with a sad mood and sleep disturbance, we are required to get additional information to determine if this person indeed has a syndrome of depression. Once we determine that it is depression, we need to identify the etiology. Once we recognize that the depression is idiopathic, we try to assess the factors that precipitate, exacerbate, and ameliorate the symptoms. Based on the patient’s general medical condition, we pick a treatment. We also assess the impact of the patient’s illness on him and his family. How different is our approach from an internist who is dealing with diabetes or a neurologist who is dealing with headache? Insisting that psychiatry is somehow unique and different from other branches of medicine has not served us well in the past and definitely will not serve us well in the future. Park et al. (12) have demonstrated that a psychiatry OSCE that incorporates both checklist scores and global ratings in the evaluation of psychiatry clerks tests clinical skills that include history taking, interpersonal skills, and physical examination which are common to all disciplines and a requirement for all medical students.

Third-Year Students are Not Able to Synthesize the Necessary Information

The second issue Ramchandani has raised is that of timing. There are several studies which show the validity of the OSCE as an examination and assert the timing of the clerkship is not relevant (12, 13, 14). In another study, Park et al. (12) studied the effect of clerkship timing and specialty preference on OSCE performance and found the timing of a student’s psychiatry clerkship during the third year of medical school is not related to that student’s performance during psychiatry clerkship OSCE. The experience of third-year psychiatry OSCE actually improves students’ clinical and interpersonal abilities. Bennett et al. (14) found that students who previously had taken their psychiatric clerkship OSCE demonstrated significant improvement in data gathering, safety assessment, and professional demeanor during the psychiatric component of the fourth-year standardized patient examination, which is similar in design to the Step 2 Clinical Skills Assessment. During the psychiatry component of the fourth-year examination, the standardized patients and the faculty evaluators in the Bennett study found that the students who had taken the psychiatry clerkship OSCE in their third-year clerkship were significantly more likely than those students who did not take it to obtain data from the standardized patients about environmental stressors, support systems, and suicidal/homicidal ideation. There was also a significant improvement in their ability to conduct the interview in a respectful and professional manner.

It is argued that clinical evaluations by attendings are superior because they offer a thorough and comprehensive evaluation of the student’s skill, and the time spent by students on psychiatry clerkship OSCE stations is too brief for the evaluation to be meaningful. McLay et al. (15) designed a psychiatry clerkship OSCE that simulated a full length interview with a complex psychiatric patient and found it to be as effective and no more costly.

The following example of a possible OSCE illustrates the similarities between a medical and psychiatric assessment.

Vignette

A 58-year-old woman is hospitalized because of a dramatic loss in function over the past year. She can no longer work or care for herself at home. She is dirty and disheveled, has lost 40 pounds during the past year, and looks emaciated. She is agitated. She is disoriented to date and day, but her level of arousal is not reduced (i.e., she is not delirious). She is fearful, and has paucity of speech. When she does speak, she says she is confused, frightened, and dead. She says she has not slept for 6 months. She performs poorly on numerous bedside cognitive tests and scores within the demented range. Her admitting diagnosis is dementia.

Question: What is the first step in trying to diagnose this patient?

Possible Answer: “Many things can cause this. But she is apprehensive, fearful, and agitated. She has experienced severe weight loss, sleeps poorly, and has diffuse and substantial cognitive impairment in clear consciousness. Some would say this is dementia. Others would say this is depression. Still others would say both. Because she appears to have elements of both, we’re stuck with an odd duck. We now must follow the diagnostic rule of common things occur commonly and go with the diagnosis with the best prognosis. It is unlikely that a 58-year-old woman would have dementia without a family history or other risk factors such as hypertension and it is very likely that a 58-year-old would have primary or secondary depression.”

Question: What is the differential diagnosis? Why?

Possible Answer: “She could have a drug reaction; thyroid or parathyroid disease; an inflammatory disease, such as lupus; depression; Alzheimer’s disease; vascular dementia; or malignancy.”

Question: What is your next step?

Possible Answer: “Getting additional history to rule out drug reactions and ordering necessary lab tests such as thyroid function tests, a Lupus panel to rule out thyroid disease and lupus, and imaging to rule out vascular changes and malignancy.”

Question: How would you treat her and what is your rationale for that particular choice?

Possible Answer: “ECT, because of the severity of her condition, followed by antidepressant treatment for maintenance.”

Additional exercises, such as telling the family and getting their consent can test the student’s communication and professional skills. Evaluation of this patient does not require any additional psychotherapeutic skills on a medical student’s part, and clinicians often encounter patients like this one in their practice. The psychiatric clerkship OSCE requires students to interact with live people who, even if standardized, are able to portray problems in realistic human terms. These exams can test students’ ability to pick up on nuances of communication and interpersonal cues. Careful construction of vignettes and intensive training of standardized patients can help us in making the psychiatric clerkship OSCE an invaluable tool.

Certainly psychiatry clerkship OSCE is not an answer to everything. Psychiatric educators understand the limitation of an OSCE examination. Hodges et al. (4) have identified that using a binary checklist to score student performance does not accurately reflect student competence. In their study, they found that when evaluating a student’s performance on a checklist, an experienced clinician’s ability to assess the student was worse than that of a trainee. Long-term studies of this method have not yet been conducted. To my knowledge, there are no long outcome studies that have compared students who were trained using OSCE and students who were trained in the traditional manner. The psychiatric OSCE involving standardized patients is expensive and resource-intensive and should be reserved for skills that can be assessed only through the vehicle of a doctor-patient interaction, and not to assess skills, such as clinical reasoning, that can be assessed as well or better by written or computer-based exams (8). In summary, I believe the psychiatry clerkship OSCE is here to stay. We can study it more, correct any further problems, and adjust it to the needs of psychiatric education. But let us not throw out the baby with the bath water.