Academic Psychiatry 23:225-226, December 1999
© 1999 Academic Psychiatry
Educational Abstracts
Dorthea Juul, Ph.D.
Key Words: Educational Abstracts
Kassebaum DG, Eaglen RH: Shortcomings in the evaluation of students' clinical skills and behaviors in medical school. Academic Medicine 1999; 74:841–849
Kassebaum and Eaglen analyzed two data sets from the Liaison Committee on Medical Education (LCME) to identify trends in the assessment of medical students. The first set was information about the 97 medical schools that were site-visited for accreditation purposes between July 1993 and June 1998. The second set was data collected on the annual LCME Part II Medical School Questionnaire. In 1992–1993 and 1993–1994, the questionnaire was completed by 126 schools, and in 1997–1998, after the merger of the Medical College of Pennsylvania and Hahnemann University, the questionaire was completed by 125 schools.
For basic science courses, paper-and-pencil tests, particularly multiple-choice questions, were uniformly used to measure knowledge. The use of National Board of Medical Examiners (NBME) subject examinations remained fairly stable between 1992–1993 and 1997–1998. In the most recent school year, the use of NBME tests ranged from 10% of the behavioral science courses to 30% of the pathology courses.
Assessment of other behaviors, such as participation in small-group sessions, conducting literature reviews and research projects, and communication skills, occurred in about 25% of the courses, and these evaluations determined about 10% of the student's grade. The authors also noted that the schools with robust learning objectives (stated in terms of measurable learning outcomes) were significantly more likely to evaluate noncognitive abilities (36% vs. 22%) than were the schools that did not have robust learning objectives. There was no significant difference between the schools with appreciable problem-based learning (PBL) in place, compared with those with little or no PBL.
The authors argue that the almost total reliance on written tests for the assessment of students in the basic sciences is inappropriate, given the broader range of objectives for these courses.
Faculty and resident ratings were collected in almost all of the core clerkships and accounted for 50%–70% of a student's grade. Kassebaum and Eaglen report that their extensive review of clerkship evaluation forms indicated that the ratings "hardly accumulate findings of sufficient specificity and discrimination to judge the spectrum of students' clinical skills (p. 848)." In psychiatry, the ratings were used in 97% of the clerkships and contributed a median of 60% to a student's final grade.
Written tests (either the NBME subject examinations or internally developed examinations) were also administered in most (96%) of the psychiatry clerkships, and the scores accounted for a median of 33% of the final grade. The use of NBME subject examinations increased continuously between 1992–1993 and 1997–1998, and were used in 64% of the psychiatry clerkships in the most recent school year.
Other formats used to evaluate clinical clerks included oral examinations, objective structured clinical examinations (OSCEs), and standardized patients (SPs). In psychiatry, oral examinations were used in 31% of the clerkships and OSCE/SP examinations were used in 12% of them. They contributed a median of 20% and 23% , respectively, to the final grade. Other methods, such as papers, projects, and presentations, were used in 6% of the psychiatry clerkships and contributed a median of 10% to the final grade.
The authors conclude that "the results described in this article explain why accreditors are paying close attention to how well schools provide measured assurances that students learn what the faculties set out to teach;" (p. 849). The authors argue for clearer specification of learning outcomes, assessment of a broader range of attributes, and more use of structured observations to assess clinical skills.
McNaughton N, Tiberius R, Hodges B: Effects of portraying psychologically and emotionally complex standardized patient roles. Teaching and Learning in Medicine 1999; 11:135–141
As reported by Kassebaum and Eaglen, standardized patients (SPs) are being increasingly used to test the clinical skills of medical students during their clerkships. McNaughton and her colleagues describe an examination developed to assess the content knowledge and patient management skills of third-year psychiatry clerks. The examination consisted of eight SPs and two written stations, each of which was 12 minutes in length. The SPs portrayed a range of psychiatric conditions, and the purpose of this study was to determine the consequences of enacting these stressful roles.
These SPs had an average of 5 years of experience and ranged in age from early 20s to late 60s. There were equal percentages of men and women, and 75% of the SPs were professional actors. Open-ended questionnaires were administered to 16 SPs over the course of an examination day, and 4 focus groups were held with a total of 36 SPs. These discussions were transcribed, and the comments were categorized by two raters.
The SPs reported that the most difficult roles were portraying someone with dementia and someone with mania and that the parent of a person with schizophrenia was the easiest. Several physical and mental effects were identified, including exhaustion, stiffness, headache, psychological irritations, and sleep problems. Most of these effects continued beyond the test day.
Compared with nonpsychiatric roles, the SPs unanimously felt that the psychiatric roles were more exhausting, and they felt that they could perform the same role three or four times in a row and then needed an equivalent amount of down time.
On the positive side, the SPs were strongly motivated to do this work. In addition to the money, they felt they were making a valuable contribution to medical education and hence to society. They also enjoyed a sense of camaraderie with the other SPs and the opportunity to hone their acting skills.
The authors note that this work is at an early and descriptive stage, but that these data can nonetheless serve as the basis for developing strategies to help SPs cope with portraying these emotionally intense roles.
Elstein AS: Heuristics and biases: selected errors in clinical reasoning. Academic Medicine 1999; 74:791–794
Elstein begins his discussion of common errors in diagnostic reasoning and treatment selection by defining some of the concepts of behavioral decision theory. Two of these are heuristics, which are mental shortcuts commonly used in decision making, and biases, which are faulty beliefs. He also distinguishes between the normative and the descriptive components of decision making. The first addresses how we should make decisions, for example, by applying Bayes theorem, and the second how we actually make decisions.
The process of clinical reasoning begins with the formulation of an opinion about the probability of various diagnoses based on the clinical evidence. As more evidence becomes available, the probability of each diagnosis is updated. That is, each posttest probability becomes the pretest probability for the next stage of the inference process (p. 791). Two of the most common errors in clinical reasoning are faulty beliefs about pretest probabilities and inaccurate assessment of the strength of the evidence.
More specific errors include overestimating the frequency of very unusual but very memorable events, overestimating the prevalences of rare conditions, collecting more data than are needed to revise diagnostic probabilities, collecting redundant data, and inappropriately attaching more weight to data obtained late in a case than to data obtained earlier. Two common errors in treatment selection are omission bias and outcome bias. It is considered preferable to have a bad outcome that results from inaction, compared with one that is perceived to be causally linked to a choice that was made. For example, studies have demonstrated that physicians were reluctant to prescribe hormonal replacement therapy for postmenopausal women because the physicians felt that a death resulting from prescribing, for example, due to uterine or breast cancer, was worse than a death resulting from not prescribing, for example, due to complications from fractures.
Outcome bias means that the quality of decisions is evaluated more positively when there are good results rather than bad results. Since the outcome cannot be known at the time of the decision, it is the logic of the decision process that should be evaluated.
Elstein argues that increasing physician awareness of these cognitive tendencies can lead to more thoughtful deliberation, which in turn will enhance the quality of decision making (p. 793). For example, evidence-based medicine and decision analysis both offer techniques to minimize the errors just described.
Get information about faster international access.
Privacy Policy
Copyright © 1999
Academic Psychiatry.
All rights reserved.
Home
| Search
| Current Issue
| Past Issues
| Subscribe
| All APPI Journals
| Help
| Contact Us
| 
