Over the past two decades, psychiatric education has changed dramatically. Clerkships and residencies now routinely base the curriculum on stated goals and objectives that are met by means of well-organized clinical experiences and didactic activities. Gone are the days when the trainee served an unstructured apprenticeship, was evaluated by global rating scales, and sat for written exams largely detached from the clinical setting. As the formal curriculum has evolved, evaluation of students' and residents' performance has likewise had to change.

One of the major changes in evaluation has been the development of performance-based testing. The landmark article by Hardin and Gleeson (+1) outlined the elements of a performance-based examination called the objective structured clinical examination (OSCE). This examination consisted of various stations that required students to perform certain activities determined by previously stated objectives for each of the stations. Students' performance was evaluated on the basis of their ability to carry out these tasks as judged by rating scales, observation by faculty, multiple-choice questions, or performance of other activities that could be evaluated by an external person. In this examination format, typically each station was of short duration and focused on a specific skill or activity.

Barrows and colleagues (+2) developed this concept further by increasing the length of each station and requiring fourth-year medical students to complete more complex and comprehensive activities. Each station was now evaluated by a more comprehensive set of evaluation instruments, including rating scales completed by the standardized patient in the station, rating scales completed by faculty observers, and completion of a more comprehensive set of multiple-choice and short-answer questions at the end of each station, with an emphasis on pathophysiology and basic mechanisms of disease. What is noteworthy about this evolutionary testing paradigm was not that there was any fundamental change in the format of the OSCE, but rather that it was an extension of the paradigm to sample student knowledge and performance more completely. Later work at Southern Illinois University School of Medicine has focused on establishing the reliability and validity of the evaluation format (+3), and indeed the reliability of the examination process has been established at acceptable levels.

Application of the OSCE to psychiatric education, although theoretically very attractive, has been slow to occur. Loschen (+4) described the application of the OSCE format to one element of yearly evaluation of resident performance. Hodges et al. (+5) described the use of the OSCE in evaluating psychiatry clerkship students and importantly established some of the costs associated with this format of testing. The same group later published a study of validity, comparing student and resident performance in psychiatry, that supported the utility of this examination format in psychiatry (+6).

A recently added requirement for residency accreditation is that residents be evaluated yearly for their clinical skills in some sort of formal testing situation. At Southern Illinois University School of Medicine (SIU), the Department of Psychiatry has continued to use the OSCE to meet this requirement and also as a major component of the yearly evaluation of resident performance. This examination continues to be composed of 5 to 6 stations, but over the years the stations have evolved to include more comprehensive activities than those reported earlier (+4). For example, one station routinely now requires the resident to complete a brief psychiatric evaluation (reminiscent of the Part II of the American Board of Psychiatry and Neurology examination). Other stations continue to resemble earlier stations in asking residents to perform circumscribed specific activities like completing a mental status examination.

As has been found in other settings, mounting such an examination requires sufficient resources to provide a test with enough authenticity to convince both trainees and faculty that it is a fair measure on which to base educational decisions. In our experience one of the critical resources needed for examination development is the availability of standardized patients who can provide an accurate portrayal of the disease process in question as well as doing so in a reliable manner, that is to say, in the same way to each successive trainee evaluated on that station. At SIU, we are fortunate to have a large and well-developed standardized patient program with experienced trainers who readily provide this resource. Other psychiatry training programs would need to assure themselves of the availability of such a resource before committing the time and resources to implement an OSCE that used live standardized patients. If the training program does not have access to appropriate standardized patients, then the OSCE should concentrate on use of recorded patients, specific discrete activities that do not change substantially with the patient involved (for example, completion of a Mini-Mental State Examination on a normal person), or even computer simulations.

Other costs of conducting an OSCE are predictable and can in most instances be handled within the context of the education budget. Each station requires faculty time for development of the objectives for the station, identification of the examination activity, development of the rating scales and questions and probes, and the usual expenses of mounting any examination, such as proctoring and coverage costs. In our experience these costs are not excessive and, given the broad acceptance of the authenticity of the OSCE by both faculty and residents, well worth it. We have now used this format of resident evaluation for over a decade. It continues to have wide acceptance by both faculty and residents.

Southern Illinois University School of Medicine continues to use the format developed by Barrows et al. (+2) for testing senior medical students for clinical skills competency. However, many departments now use a similar format for at least one part of the end-of-clerkship examination. The Department of Psychiatry has used a 3-station exam for the past 4 years as a major component of the end-of-clerkship examination. Each station presently consists of a standardized psychiatric patient for whom the student is expected to provide a brief (20- to 30-minute) psychiatric evaluation in a hypothetical outpatient setting. Each standardized patient rates the student on "people skills" such as being empathetic and providing appropriate patient education. A faculty member observes the examination through a one-way mirror and rates the student's gathering of clinical information and conduct of the patient examination. This phase is followed by a 15- to 20-minute period in which the student answers a series of short-answer and multiple-choice questions about the findings of the patient, differential diagnosis, pathophysiology, and treatment options. This part of the station is completed entirely on a computer, and the program used scores the student's performance against preestablished objectives for this portion of the exam. This method allows for rapid scoring and numerous comparisons of performance related to the objectives of the clerkship and the examination station. Given that students are assigned to a variety of clinical settings throughout central and southern Illinois and that there is sometimes substantial variation in clinical ratings of student performance by various faculty, this examination allows for a reasonable evaluation of student skills at the end of the clerkship that allows for comparison of student performance across sites.

Performance-based assessment evaluates the clinical skills of trainees in ways that cannot be duplicated by the usual paper-and-pencil test. The usual test of knowledge, whether it be essay, multiple-choice, or another method, may sample knowledge very effectively. However, this type of testing does not tell the evaluator if the candidate can use the knowledge in a clinical setting. Performance-based assessment attempts to address this by requiring the trainee to demonstrate clinical skills in settings that are similar to the usual clinical setting where the individual will be practicing in the future. The OSCE is one method of performance-based assessment that allows this to be done in a cost-effective manner and with enough face validity to gain the acceptance of faculty and trainees.