Academic Psychiatry 22:56-58, March 1998
© 1998 Academic Psychiatry
Educational Abstracts
Dorthea Juul, Ph.D.
Key Words: Educational Abstracts • Dorthea Juul
Tierney WG: The parameters of affirmative action: equity and excellence in the academy. Review of Educational Research 1997; 67:165–196
Tierney contributes to the debate on affirmative action (AA) by reviewing its history in public higher education and criticisms that have been made and outlining alternatives. He notes that "Affirmative action came about because our campuses were White, male centers of learning. The faculty, administration, and students were overwhelmingly White men." (p. 169) Tierney also points out that access to public higher education has been seen as a primary vehicle for increasing social equity.
Three reasons are given for the development of AA policies— compensation for previous discrimination, correction of present discrimination, and diversification or creation of a more pluralistic society. The first intent, compensation, has been rejected by the courts in such rulings as the Bakke and Hopwood decisions. Correction, the second, only applies to specific instances of injustice, while diversification has usually been deemed acceptable. Tierney suggests that AA has not had remarkably high rates of success, although it has contributed to increased numbers of men of color and of women in the academy.
In recent years, AA has been harshly criticized as being unfair to groups not protected by the policy (reverse discrimination) and possibly harmful to those who are protected by it and as having led to the dilution of standards in the academy because of the admittance and employment of unqualified persons. Tierney states, "the objections to affirmative action assume that all individuals are equal and that they should be judged strictly by a neutral standard." (p. 190) He goes on to question the heavy reliance on standardized admissions tests (e.g., ACT, SAT) to define merit.
Tierney ends by emphasizing his earlier point about the role of higher education in enabling citizens to participate in our democratic society. He argues that we need to develop "more proactive policies that help academe serve the public good by advancing diversity and fostering the public culture so that everyone is able to participate." (p. 193)
Clauser BE, Margolis MJ, Clyman SG, et al: Development of automated scoring algorithms for complex performance assessments: a comparison of two approaches. Journal of Educational Measurement 1997; 34:141–161
The National Board of Medical Examiners has engaged in a lengthy research and development project to create realistic clinical cases that can be administered and scored via computer for licensure testing. This article discusses two approaches to scoring these problems—a regression-based method and a rule-based method. Data were obtained from a sample of 200 senior medical students randomly selected from a larger group that had taken an eight-case examination on a voluntary basis.
For the first method, one committee of content experts categorized all actions into one of three levels of potential benefit to the patient or one of three levels of potential risk.
A second committee reviewed transaction lists (a record of all actions taken by the examinees along with the time at which the actions occurred) and rated the performance on a one-to-nine scale. A regression equation was then produced for each case in which the mean rating from the second committee was the dependent variable, and the independent variables were the six counts of helpful and harmful options and a variable representing the timeliness of the most essential beneficial actions.
The second scoring method used rules to assign scores to transaction lists based on specific combinations of beneficial actions, with points subtracted for risky actions. These rules were derived by asking the committee members who rated the overall performance based on transaction lists to articulate their scoring criteria.
The scores of the 5 judges who rated the transaction lists yielded high intercase reliability estimates (0.93–0.99). Both the regression-based and the rule-based scores were moderately to highly correlated with the expert ratings and moderately to highly correlated with each other. The correlations between the scores for each case and the mean rating on the other 7 cases ranged from 0.25 to 0.55.
Misfits, that is, discrepancies between scores and ratings, were reviewed, and it was found that many were due to a small number of actions that the raters deemed to be substantially more dangerous than their classifications would suggest.
The authors conclude that "both systems provided an effective means of approximating expert ratings of the same performances." (p. 157) However, the authors argue that the regression-based procedure was superior to the rule-based procedure because of somewhat higher correlations with the expert ratings, and they emphasize the need for such scoring systems if these computer-based patient-management problems are to be used in large-scale testing contexts.
Swartz MH, Colliver JA, Bardes CL, et al: Validating the standardized-patient assessment administered to medical students in the New York City Consortium. Academic Medicine 1997; 72:619–626
Swartz and his colleagues report on a study undertaken to explore the use of global ratings to score performance on standardized patient (SP) examinations. Five clinicians from the consortium's steering committee independently observed and rated the videotaped performances of 44 fourth-year medical students who had encountered 7 SPs (100+ hours of viewing time per rater). For each of the seven cases, the students were rated on a four-point scale (fail, low pass, pass, and high pass) for two dimensions (clinical competence and interpersonal and communication skills), and an overall rating for each examinee was also obtained. Mean scores from the five judges were used in the subsequent analyses.
These ratings were correlated with three scores produced by summing the number of correct actions taken in three areas: history and physical examination, history and physical examination plus postencounter actions, and interpersonal and communication skills.
The interrater reliabilities for the 5 judges were generally in the 0.70 to 0.80 range for both the overall ratings and the case ratings. The correlations between the ratings and the 3 scores ranged from 0.60 to 0.70. The researchers also found low correlations between the overall clinical competence ratings and student scores on Parts 1 (0.16) and 2 (0.30) of the United States Medical Licensure Examination, suggesting that the SP test measured something different than the multiple-choice examinations in the basic and clinical sciences.
Swartz et al. conclude that global ratings by content experts should be used "as the gold-standard criterion for SP assessment." (p. 625) They also suggest that the ratings be used as guideposts to improve the content of SP cases as well as the scoring of SP-based assessments.
McLeod PJ, Meagher T, Tamblyn RM, et al: Are ambulatory care-based learning experiences different from those on hospital clinical teaching units? Teaching and Learning in Medicine 1997; 9:125–130
McLeod and his colleagues studied case mix and medical students' and residents' learning experiences in an internal medicine ambulatory care clinic and an inpatient unit. For 6 months, 1 of the authors regularly observed in both settings and conducted semistructured interviews with residents and clerks after patient encounters. There were 3 parts to the interview: a log of the patient's presenting diagnosis or diagnoses, an indication of which of 16 clinical skills could be learned and practiced during that patient encounter, and a closed-ended questionnaire about the patient. In addition, a questionnaire was sent to the 14 internists who had supervised residents and medical students in the ambulatory setting at some time during the past 10 years. The questionnaire obtained information about the patients seen in that setting and the relevance of various clinical skills to their care.
Forty residents and 29 medical students volunteered to participate in the study. The residents reported on an average of 11.4 patient encounters in the ambulatory setting and 5.2 encounters in the inpatient setting. For the clerks, the averages were 4.2 outpatient encounters and 8.9 inpatient encounters. There were a total of 495 patient encounters in the ambulatory setting and 444 in the inpatient setting.
As might be expected, the case mix was rather different for the two settings, with more diversity in the clinic and an overrepresentation of cancer, AIDS, and end-organ failure in the hospitalized patients. The mean relevance rating for the inpatient encounters was 3.38 points higher (on a scale of 0 to 16) than for the outpatient encounters. Fifty-eight percent of the problems encountered in the hospital patients were new, whereas only 29% of the problems encountered in the ambulatory clinic were new. Fifteen of the 16 clinical skills were judged to be relevant in a greater percentage of the inpatient encounters than they were in the outpatient encounters.
The faculty survey revealed that these tutors rated the educational relevance of the outpatient encounters higher than did the trainees. Their perceptions of the outpatients also differed from those of the trainees—they felt they posed more of a challenge in terms of patient management than did the clerks and residents.
The authors conclude that while a broad diversity of health problems is encountered in the ambulatory setting, the hospital encounters are better for learning because they more often present new problems for the trainees and require the application of a broader range of clinical skills. They state, "If our findings are confirmed by other investigators and if the learning experiences really are superior during interactions with more seriously ill hospitalized patients, we may have to reconsider our love affair with ambulatory care training." (p. 130)
Get information about faster international access.
Privacy Policy
Copyright © 1998
Academic Psychiatry.
All rights reserved.
Home
| Search
| Current Issue
| Past Issues
| Subscribe
| All APPI Journals
| Help
| Contact Us
| 
