Residents have an influential role in shaping medical students’ attitudes toward the profession and in teaching the requisite knowledge and skills for clinical practice. One early survey suggested that most residents in psychiatry spend about 1 or 2 hours daily teaching students (1). It had also been estimated that about 62% of psychiatric residency programs offered training in teaching skills (2). Residency programs therefore require valid and reliable tools for evaluating residents as teachers and for evaluating the utility of their curriculum in training residents as teachers. Valid and reliable tools are also required to advance medical education research in this field, especially given the dearth of rigorously conducted studies of residents as teachers in psychiatry (3).

In order to facilitate evaluations of residents as teachers, and in order to promote medical education research, our goal for this article is to review existing assessment tools related to residents’ teaching skills and teaching effectiveness. We identified assessment instruments that have been used to assess residents’ teaching skills, attitudes and knowledge and summarized their important features, potential strengths and weaknesses, and psychometric properties in order to guide program directors and prospective researchers in their choice of instruments.

A search was conducted of PubMed and PsycInfo databases and the web pages of the Association of American Medical Colleges’ (AAMC) selected bibliography on residents as teachers and the residents’ teaching skills project group (4) using combinations of the key search terms “residents,” “registrars,” “residents as teachers,” “teaching skills,” and “assessments” or “rating scales.” In addition, primary publications were searched for additional references. We focused on identifying the instruments used in descriptive or experimental research studies concerning residents as teachers. Instruments were selected for inclusion in this report when the full instrument was accessible; one was excluded because it concerned academic preparation to teach without assessing teaching knowledge or skills (5). In addition we did not include the Teacher Identity Scale (6) because this concerned elements of teacher identity and not teaching in specific settings.

All instruments were described, when applicable, in relation to their domains of evaluation and number of questions; availability of information on validity and reliability; areas of assessment (knowledge, attitudes, and skills); identity of raters (self, learner, or faculty); and format (direct evaluation, observed structured teaching evaluation, videotape, audiotape). All relevant articles were read by at least three members of the team and consensus was achieved for all of the findings described here.

Table 1 depicts the 11 evaluation tools that utilized self-reports, learner evaluations, or an observed structured teaching evaluation (OSTE) (7–21). Methods that involved direct audio or video recordings of teaching are discussed separately (22–24). One particular tool, the Stanford Faculty Development Program, appears twice in Table 1 because it was first published as a 58-item instrument (11, 12) and was then shortened to a 26-item instrument (13) while maintaining reliability and validity. The length of all instruments varied from one item on which students were asked to rate their experience with the faculty member (15, 16) to 58 items (11, 12). Eight of the 11 evaluation tools depicted in Table 1 had been judged to be valid and reliable.

The four longer scales, including the Stanford Faculty Development Program (13), the Wisconsin Inventory of Clinical Teaching (10), the Resident Evaluation Form (20), and Self Perceptions of Teaching (9), divided the assessed items into different categories or domains. The Stanford Faculty Development Program was divided into the following categories: learning climate, control of the teaching session, communication of goals, promoting understanding and retention, evaluation, feedback, promoting self-directed learning, and teacher’s knowledge. The Wisconsin Inventory of Clinical Teaching evaluated outpatient teaching and assessed attending doctors as a clinical role model, mentor, supervisor, instructor, evaluator, and consultant. The Resident Evaluation Form assessed resident teachers in categories of teaching effectiveness, instructor knowledge, initiative, relationship with students, feedback, judgment, and overall. The Self Perceptions of Teaching scale assessed resident perceptions of teaching in the areas of teaching as a responsibility, role as teachers, teaching methods, styles and skills, and improvement of clinical teaching.

Most of the tools depicted in Table 1 were learner evaluated; only two self-assessment tools (9, 14) were found, and the OSTE was evaluated by faculty (21). All 11 of these evaluation tools used some form of a rating scale (mostly a 5-point scale) for assessment. Four of the 11 evaluation tools assessed knowledge, attitudes, and skills (8, 10, 12, 20). The tool that utilized one question only was not considered to formally evaluate each of these components even though it was a validated tool (15, 16).

An OSTE is a direct observation of a research participant’s teaching ability as judged by predetermined observable criteria. In one study (21), resident teachers were rated on eight 15-minute stations that reflected residents’ learning needs for teaching skills development. The stations included orienting a learner, outpatient precepting, bedside teaching, giving feedback, inpatient teaching, teaching charting, teaching a procedure, and giving a mini-lecture. One or two medical students enacted each case while another watched by camera, and evaluations were completed on 5-point rating scales. The OSTE was found to be a valid and reliable measure of residents’ clinical teaching skills (21).

Evaluation methods that utilized direct observations rather than self-reports, learner evaluations, or OSTEs included video- and audiotaped observations. In one study, videotapes were made of each resident teaching in the context of a case presentation, and these sessions were evaluated by trained raters in eight domains (22). Similarly, videotapes were used to identify key behaviors taught at a Teaching Improvement Project System Workshop with items of assessment relating to use of instructional objectives, providing a motivational “set,” and use of an active “body” such as an appropriate pace of presentation and variety of teaching techniques and activities (23). A method for assessment by audiotape entitled the low inference self-assessment measure (LISAM) focused on verbal interactions and coded specific categories of teaching (24). The coding instrument assesses questioning skills and the balance of factual to higher cognitive questions, the ratio of teacher and student talk, the identification of motivating set and closure, wait time, number of positive statements, and the number of times the teacher used student ideas (24).

These results show that there is a substantial number of instruments available that also vary considerably in their methods of assessment (learner-completed and self-completed questionnaires, videotapes, audiotapes, OSTEs), domains of inquiry, and length. The majority of the instruments described here were assessed for validity and reliability. These tools should assist program directors and others to better assess resident teaching skills and to conduct research on residents as teachers. Because tools vary and no one tool fits all programs or research projects, program directors and researchers should choose tests that best suit their needs.

As noted in a recent review of residents-as-teachers curricula (25), the most commonly used evaluation methods were learner-completed questionnaires. Learner-completed questionnaires are relatively easy to administer, but lack the potential to capture the same richness of interplay between teachers and learners. Self-evaluation is most often used as a secondary outcomes measure, whereas OSTEs and videotapes have also been commonly used (25).

The most rigorous evaluations are the OSTEs. Observed structured teaching evaluations require manpower, time, an assessment site (clinical space or simulation lab), tools (standardized patient and cases), and significant financial resources. These, along with audiotapes and videotapes, allow for direct, comprehensive, and independent observations of teaching skills and therefore constitute perhaps the gold standard for assessing a resident’s ability to teach. These are limited, however, by their potential financial costs and by the amount of work required. The estimated annual expense of an OSTE including getting standardized patients, faculty time, space, and food amounts to $424 per resident (26). Ways to reduce costs and time for OSTEs may include using standardized students over trained actors and combining direct and videotaped observed teachings at various times.

Limitations of the available studies and evaluation tools include a lack of tools that evaluate all three domains (knowledge, attitudes, and skills). Moreover the available assessments by learners, as far as we could find, did not incorporate information on motivating and closing sets, wait times, and the balance of types of questions asked of learners (24). Users will need to determine which tools are best used at various times in training. For example, tools utilizing self-rating scales may be more accurate if a resident has been through a teaching program first.

There are several limitations to this review. For one, our search was not necessarily complete in identifying all potentially useful and validated tools. In particular, we did not search the ERIC database or MedEd Portal which might contribute information on assessments used in nonmedical or medical education not in medical journals. Nor did we write to authors to request full scales when these were not provided in published articles. Third, we did not provide the specifics of the validity and reliability of the individual instruments (27). Nevertheless, we anticipate that the information provided here will serve as a starting point in facilitating the evaluation of residents as teachers in training programs and in promoting future research.

Disclosures of editors are published in each January issue. At the time of submission, Drs. Ismail, Mian, and Dewey reported no competing interests.