Academic Psychiatry 23:105-106, June 1999
© 1999 Academic Psychiatry
Educational Abstracts
Dorthea Juul, Ph.D.
Key Words: Educational Literature, Abstracted • Juul
Anderson MB, Cohen JJ, Hallock JE, et al: Learning objectives for medical student education-guidelines for medical schools: Report I of the Medical Schools Objectives Project. Academic Medicine 1999; 74:13–18
This is the first report from the Medical Schools Objectives Project, an initiative undertaken by the Association of American Medical Colleges in 1996. The goals for the first phase of the project were to "develop a consensus within the medical education community on the attributes that medical students should possess at the time of graduation, and to set forth learning objectives for the medical school curriculum derived from these attributes."
The authors argue that society's expectations of physicians have changed from a focus on curing disease to concern for all aspects of health care, and the four attributes that were identified reflect this broad-based conception of physician competence. The attributes are altruism, knowledge, skill, and dutifulness, and several learning objectives are identified for each of these. A learning objective for altruism is that before graduation, a medical student will have demonstrated, to the satisfaction of the faculty, "compassionate treatment of patients, and respect for their privacy and dignity." One of the objectives for knowledge is "an understanding of the need to engage in lifelong learning to stay abreast of relevant scientific advances, especially in the disciplines of genetics and molecular biology." Skillfulness includes "the ability to perform both a complete and an organ-system-specific examination, including a mental status examination." An example of an objective in the dutiful category is "a commitment to provide care to patients who are unable to pay and to be advocates for access to health care for members of traditionally underserved populations."
Medical schools are encouraged to review their curricula within this framework and, if necessary, make changes in their educational programs. The authors acknowledge that a major difficulty in making changes is the lack of suitable outcomes for measuring many of these attributes/objectives. Subsequent reports will further address implementation of these recommendations.
Hodges B, Regehr G, Hanson M, et al: An objective structured clinical examination for evaluating psychiatric clinical clerks. Academic Medicine 1997; 72:715–721
Hodges B, Regehr G, Hanson M, et al: Validation of an objective structured clinical examination in psychiatry. Academic Medicine 1998; 73:910–912
Hodges and his colleagues describe the development and implementation of an objective structured clinical examination (OSCE) for psychiatry clerks. The examination consisted of eight standardized patient (SP) stations and two short-answer stations. The SP stations required a 12-minute interview of a psychiatric patient. While SPs have been used to assess physicians across various levels of training and content areas, they have not often been used to portray psychiatric patients. Two forms of the test were developed that were administered alternately throughout the academic year in the fifth week of the 6-week clerkship.
Faculty observers used both checklists and global rating scales to assess skill in diagnosis, management, and patient education and process skills such as rapport, interviewing technique, and emotional control. An overall judgment of each student (pass, borderline, or fail) was also made. Results for 192 medical students are reported in the 1997 paper. The internal consistency reliability ranged from 0.49 to 0.69 across the test administrations, and most of the students and faculty rated the format favorably. The total cost per student was estimated at $77. Incidents that occurred during the examination in which a student experienced excessive and unexpected affect or demonstrated unusual or inappropriate behavior were recorded by the examiners, and a number of such events did occur. The examples given are the assaultive restraint of an SP portraying schizophrenia and inappropriate responses to the scripted sexual advances of an SP portraying mania. The authors suggest that because of the highly stressful emotional nature of these stations, "this examination may be the first to allow for the formal documentation of potential problems in the domain of professional conduct." The effect of portraying these cases on the SPs is also discussed. Focus groups with the patients suggested that there are some negative effects that varied in severity and duration. Examples are mild exhaustion; euphoria; sleep disturbances; and heightened levels of sadness, anger, and anxiety. The authors argue for the careful selection, monitoring, and debriefing of these "patients."
The 1998 study addressed the validity of this OSCE. In addition to 33 third-year medical students, 17 psychiatry residents were also tested. The residents scored significantly higher on the global ratings than the medical students, but there were no differences in the checklist scores.
Concurrent validity was assessed by correlating OSCE scores with faculty ratings of the medical students' overall interviewing skills and with predictions of each clerk's relative success on each of the eight SP stations. Both of these ratings correlated modestly with OSCE performance. Eighty percent (12/15) of the residents described the SPs as real or very real and agreed that the situations accurately reflected emergency psychiatry situations. Based on their analyses, the authors conclude that SPs can be used to validly and reliably assess complex psychiatric skills.
Goodwin LD: Relations between observed item difficulty levels and Angoff minimum passing levels for a group of borderline examinees. Applied Measurement in Education 1999; 12:13–28
The most widely used technique for setting criterion-referenced standards for multiple-choice tests is the Angoff method, in which judges (content experts) are asked to estimate the performance of minimally competent, or borderline, candidates on each item. These percentages are averaged across judges to produce a minimum passing level (MPL) for each item. These, in turn, are averaged to determine the pass/fail level for the test. This study used data from one administration of the Certified Financial Planner Examination that consisted of 310 items taken by 972 examinees. A group of borderline examinees was identified, and their performance on each item, as well as that of the total group, was correlated with the MPLs derived from 14 judges representing different specialty areas. The borderline group consisted of 115 examinees (12%) who scored within one standard error of measurement of the pass score. The interrater reliability of the judges' ratings was 0.81. The correlations between the MPLs for each item and the P-values (percent correct) for the borderline group and the total group were both 0.55, which was statistically significant. For the borderline group, 61% of the estimates were classified as accurate, 28% as overestimates, and 11% as underestimates. The author points out that this study is one of the few to identify a group of borderline candidates and hence provides particularly useful information about the validity of the pass/fail standard yielded by the Angoff method.
Get information about faster international access.
Privacy Policy
Copyright © 1999
Academic Psychiatry.
All rights reserved.
Home
| Search
| Current Issue
| Past Issues
| Subscribe
| All APPI Journals
| Help
| Contact Us
|
