Residents training in Canada must pass the specialty examination set by the Royal College of Physicians and Surgeons of Canada (RCPSC) to practice as psychiatrists. The RCPSC instituted phenomenology, diagnosis, and management (PDM) stations in addition to the diagnostic long case several years ago for the second or oral component of this examination. The PDM stations eventually replaced the diagnostic long case as the sole oral component of the specialty exam. Criticisms of the previous oral examination format and of using real, nonstandardized patients (¹—³) were part of the impetus for this change. The RCPSC then amended the name of the exam stations to the Objective Structured Clinical Examination (OSCE). The description of the exam content and process from the RCPSC web site (⁴) is as follows:

The oral component consists of several OSCE stations, phenomenology, diagnosis, and management, each of which will be approximately 20 minutes long. These stations may include videos; interpretation of laboratory results or imaging results; or role play with family physicians, health professionals, patients, or family members. The stations will be designed to evaluate multiple CanMEDS Roles across the lifespan. Instructions will be given for each station outlining the expectations of the candidates. The total duration of the oral component will be approximately 4 hours.

Residents who have early exposure to practice examinations in the format of their qualifying examination are more accustomed to the examination process, which can help decrease the anxiety that may impair performance. Residents’ preparation can also be more effective as a result of knowing the range and complexity of information required to pass the OSCE stations. A training and examination program is described here that functions well in the university where it was developed and can be adopted in other programs in preparation for Part 2 of the American Board of Psychiatry and Neurology (ABPN) exam.

Postgraduate-year 1 (PGY-1) residents are given an introductory lecture in their first month of training that identifies the CanMEDS roles (⁵), which are similar to the Accreditation Council for Graduate Medical Education (ACGME) core competencies (⁶). The format of the written and oral components of the specialty examination is detailed, along with a mock master exam schedule that includes testing all of the roles and competencies in different formats across the lifespan.

Residents from PGY-2 to PGY-5 attend a monthly 2-hour training session for OSCE stations. Each month, new stations are authored by residents, who then act as examiners for their own stations, which is an excellent opportunity for gaining an understanding of the exam process. Assigning the OSCE station topics to the residents prevents duplication within the same training year. A separate list of examinees (candidates) is also prepared. The examinees are not made aware of the topic of the station nor with which examiner they are to be paired. The rest of the residents observe the OSCE stations. Although this is anxiety provoking for examinees, this form of exposure can ultimately become beneficial as a means of exam preparation. Each resident from PGY-2 to PGY-5 authors one station and acts as a candidate once per academic year. Because the OSCE stations are prepared at a level of difficulty suitable for senior residents (⁷), PGY-1 residents are invited to observe but are not expected to participate, though participation can be offered to them based on individual program expectations.

The OSCE stations are conducted in a small amphitheater, with the examiner and candidate sitting at the front of the room. The examiner faces a pull-down screen, while the examinee faces the audience, so that the screen is out of view. The station contents are shown on the screen to the examiner and audience using a computer and LCD projector or overhead transparencies.

The author/examiner reads from printed sheets containing the questions and marking scheme for his or her OSCE station. Showing overhead transparencies containing the same information as the examiner’s sheets is an effective educational practice because answers can be revealed in stages as the candidate progresses through the station. It has been helpful to show the question posed to the candidate, but then reveal the answer in sections to give audience members a chance to think about how they would respond.

The training sessions are moderated by a consultant psychiatrist who ideally functions as a departmental examiner and who is knowledgeable about the specialty exam. The moderator also reviews the stations ahead of time to assess their appropriateness as teaching tools and to determine if the questions are of suitable complexity and of an appropriate length to last for 20 minutes. During the station, the moderator helps the author/examiner simulate actual test conditions. Participating in this role is a valuable opportunity for faculty educational development, particularly for junior faculty members who have recently completed their board exams and can impart an accurate description of the process.

When the student is finished with an OSCE station, the moderator provides feedback to the examiner on the construction of the station, including an appraisal of how the station set about testing its specified roles or competencies and if these were actually achieved (⁸, ⁹). The moderator also provides feedback to the examinee about his or her performance and, ultimately, if this would be a pass or fail under actual exam conditions (¹⁰). It has been very helpful to then turn to the rest of the residents to comment on both the exam process and content, which makes for a rich discussion. The enhancements thus obtained are then incorporated into an updated version and circulated to all residents to build a library of referenced OSCE stations that have been used in a simulated exam situation.

Residents participate in an official OSCE examination held on a department-wide level twice per year, the results of which are recorded on their permanent training file. The number of stations that can be offered depends on program resources. A formula that has worked for our program has residents doing the same number of stations as their year of training (i.e., a PGY-4 resident does four stations per exam). An examination for all residents can usually be scheduled in a half day of testing.

In keeping with the protocol of the specialty examination, brief instructions for the residents are posted on the outside of the exam room door. This includes information about the format of the station (such as a role play or video clip). A timer keeps track of the progress of the station. A schedule is distributed to residents ahead of the exam, detailing the time and location of the stations. Contiguous offices help reduce travel time between stations. No feedback is given to candidates at the time of the examination, which also simulates actual conditions.

In the exam setting, candidates finishing the OSCE stations are required to remove all communication devices and wait in a room until the last cohort of examinees has started. This eliminates any opportunity to pass along information, such as the content of the stations. In departmental-level exams, this protocol would also have to be followed to ensure that the results are as reflective as possible of a resident’s performance.

Examiners for the stations are consultant psychiatrists who, having completed the specialty exam, are in an excellent position to assess what would constitute a passing performance. OSCE stations can be generated by a group of examiners and reviewed by the exam coordinator before being implemented. If there is a conflict of interest, such as a consultant being a National Exam Board member, the exam coordinator can author the stations to help circumvent this issue.

The stations are scored out of 10, with 7 being a pass. Results are tabulated for each station so that residents know how they performed compared with others at their level of training and all residents completing the exam. Table 1 shows a sample exam result letter. For stations where residents obtain a passing grade, only numerical results are given. Examiners are encouraged to write comments that can be included in a letter detailing the results, should it become necessary, such as in the case of a failing performance. Scores of 5 or less require that examiners provide written feedback for candidates.

The bank of exam OSCE stations is not shared with the residents. They do not receive the marking scheme, nor do they get to discuss the results with the individual examiners, modeling the process for both the Royal College of Physicians and Surgeons of Canada (RCPSC) and American Board of Psychiatry and Neurology (ABPN). This also confers the advantage to the exam coordinator of being able to use the stations again.

In the 5 years since this program’s inception, 26 residents have completed their training at University of Western Ontario and were eligible for the RCPSC examination. All 26, 12 of whom are international medical graduates, were successful on their first attempt. The RCPSC does not publish detailed data on examination results for comparison (¹¹). In the 4 years prior to the implementation of this training program, University of Western Ontario had 11 residents complete the psychiatric residency, and eight were successful on their first attempt on the RCPSC exam.

This OSCE training program has been cited by incoming residents as a principal reason for choosing our university. The reputation of this OSCE training program is emphasized to senior medical students who come for interviews for residency placement. Since this program’s inception, our residency program has matched all of its places on the first iteration. In contrast, during the prior 4 years, our university matched 75% of its places on the first round.

At the time of submission, Dr. Robinson disclosed no competing interests.