Academic Psychiatry 27:227-228, December 2003
© 2003 Academic Psychiatry
Evidence, Methods, and Psychiatric Education
Laura Weiss Roberts, M.D.,
John H. Coverdale, M.D, FRANZCP and
Alan K. Louie, M.D.
Laura Weiss Roberts, M.D., Editor-in-Chief
John H. Coverdale, M.D., FRANZCP, Associate Editor
Alan K. Louie, M.D., Associate Editor
"Best evidence medical education" is the implementation by teachers of methods and approaches to the education of physicians and physicians-in-training based on the best evidence available (1), and it is the emerging standard in the United States. Best evidence medical education includes the following four steps: 1. asking a focused question based on a particular teaching scenario, 2. searching the relevant literature, 3. critically appraising the selected paper, and crucially 4. applying the results of the critical appraisal in order to improve teaching within the constraints of the teaching scenario. Reliant upon scientific evidence, it is an intentional and rigorous teaching method that merits empirical study for its value and effectiveness.
An intriguing consideration in the discussion on best evidence medical education is how this form of teaching might influence clinician behaviors and clinical outcomes. Very few studies link medical education to the conduct or clinical results of doctors, however (2). Indeed, our understanding of the impact of medical education has been informed primarily by experience and, when there are data, by observational studies. When observational studies include a comparison group, they are referred to as analytical rather than descriptive. Examples of analytical studies include cohort studies that follow subjects forward in time, cross-sectional studies that assess prevalence at a particular point in time, and case-control studies that look backward from a particular outcome.
An important strength of analytical studies as opposed to descriptive studies is that they allow assessment of possible causal associations. The most rigorous forms of evidence are derived from experimental studies with randomized controlled trials (RCTs) (3,4), which are highly valuable from a scientific perspective but are resource-intensive and may be ethically problematic in education settings. Qualitative studies that adhere to well-defined standards (5,6) can provide an in-depth examination of the experiences of learners and teachers and help education scholars generate new theories. All of these diverse forms of educational research provide valuable insights and guidance in our young field of psychiatric educational research.
Recently, we reviewed all of the titles and abstracts of articles published in Academic Psychiatry from 1999 to the present (except editorials, commentaries, media columns, and book reviews). If a study had a comparison or control group, we read the method section. Out of 73 articles reviewed, we found two RCTs (7,8) and many more analytical studies. We were positively impressed with this result, given the limited number of experimental studies in some areas of medical education research (9–13). Nevertheless, methodological considerations appear to be very important to expert reviewers who are entrusted with evaluating the educational literature during the publication process. In a sample of manuscripts in medical education, soundness of study design was a main strength of accepted manuscripts whereas rejections commonly had a weak or biased design, samples that were too small, inappropriate or suboptimal instrumentation, insufficient data, or problematic analyses (14).
In reality, one is faced with numerous barriers to conducting experimental studies in psychiatric education research. Quantitative methods can be impractical. For example, important outcome measures such as clinical behaviors or patient outcomes are far more difficult to obtain than subjective measures such as learner satisfaction and may be more difficult to validate and assess reliably. Programs may have insufficient numbers of learners to adequately power a study. Quantitative methods may be insufficient to capture the complexity of an educational system (9). Quantitative and qualitative studies occurring in single institutions lack demonstrable generalizability. Financial barriers include the hesitancy of some institutions to pay students as research subjects or the tentativeness of some granting agencies to give priority to educational interventions. In addition, the key role of the RCT in medical education does not yet possess full "buy-in" by the crucial stakeholders: administrators, faculty, and students. Finally, randomized teaching interventions with appropriate ethical safeguards (e.g., International Review Board approval, confidentiality protections, consent, and debriefing mechanisms) may be both more complicated and intimidating than most educational research projects ordinarily undertaken in our medical schools and residencies.
To be increasingly meaningful, future work in the field of psychiatric education will necessitate closer attention to methodological issues. Increasingly rigorous quantitative and qualitative studies will enhance their validity and value and provide teachers with better tools for preparing physicians for their clinical responsibilities. Over time, psychiatric education will benefit from comparison and controlled trials, especially well-conducted RCTs. In addition, it is critically important that we explore more sophisticated statistical and interpretive approaches to help translate the data we have obtained, even from methodologically modest studies, into findings that may be useful in improving educational approaches. A salient, long-term goal for our profession is to overcome the financial and administrative barriers to conducting adequately designed and appropriately powered experimental studies. These goals are congruent with best evidence medical education. Now, we simply need more evidence.
To this end, we now call for papers on the theme of best evidence medical education. Examples of topics related to this theme include methodological issues in educational research in psychiatry, curriculum design and teaching methods in relation to evidence, and linking psychiatric education, learner behaviors, and clinical outcomes. The deadline for submissions is February 1, 2005. We look forward with enthusiasm to receiving your contributions.
REFERENCES
- Harden RM, Grant J, Buckley G, et al: BEME guide No. 1: Best evidence medical education. Med Teacher 1999; 21:553–562[CrossRef]
- Whitcomb ME: Research in medical education: what do we know about the link between what doctors are taught and what they do? Acad Med 2002; 77:1067–1068[Medline]
- Guyatt GH, Sackett DL, Cook DJ: Users' guides to the medical literature II. How to use an article about therapy or prevention. A. Are the results of the study valid? Evidence-Based Medicine Working Group. JAMA 1993; 270:2598–2601[Abstract/Free Full Text]
- Guyatt GH, Sackett DL, Cook DJ: Users' guides to the medical literature II. How to use an article about therapy or prevention. B. What were the results and will they help me in caring for my parents? Evidence-Based Medicine Working Group. JAMA 1994; 271:59–63[Abstract/Free Full Text]
- Giacomini MK, Cook DJ for the Evidence-Based Medicine Working Group: Users' guides to the medical literature XXIII. Qualitative research in health care. B. What are the results and how do they help me care for my patients? JAMA 2000; 284:478–482[Abstract/Free Full Text]
- Giacomini MK, Cook DJ for the Evidence-Based Medicine Working Group: Users' guides to the medical literature XXIII. Qualitative research in health care. A. Are the results of the study valid? JAMA 2000; 284:357–362[Abstract/Free Full Text]
- Eells TD, Strauss GD, Teller D, et al: Problem-oriented instructions as a predictor of success in early psychiatric interviewing. Acad Psychiatry 2002: 26:82–89
- Musick DW, Cheever TR, Quinlivan S, et al: Spirituality in medicine: a comparison of medical students' attitudes and clinical performance. Acad Psychiatry 2003; 27: 67–73
- Hatala R, Guyatt G: Evaluating the teaching of evidence-based medicine. JAMA 2002; 288:1110–1112.[Free Full Text]
- Bowen JL, Irby DM: Assessing quality and costs of education in the ambulatory setting: a review of the literature. Acad Med 2002; 77:621–680[Medline]
- Ogrinc G, Mutha S, Irby DM: Evidence for longitudinal ambulatory care rotations: a review of the literature. Acad Med 2002; 77:688–693[Medline]
- Davies P: Approaches to evidence-based teaching. MedTeacher 2000; 22:14–20
- Colliver JA: Educational theory and medical education practice: a cautionary note for medical school faculty. Acad Med 2002; 77:1217–1220[Medline]
- Bordage G: Reasons reviewers reject and accept manuscripts: the strengths and weaknesses in medical education reports. Acad Med 2001; 76:889–896[Medline]
This article has been cited by other articles:
|
|
|
|
 
V. Srihari
Evidence-Based Medicine in the Education of Psychiatrists
Acad Psychiatry,
November 1, 2008;
32(6):
463 - 469.
[Abstract]
[Full Text]
[PDF]
|
|
|
|
|
|
|
J. Coverdale, L. Roberts, A. Louie, and E. Beresin
Writing the Methods
Acad Psychiatry,
October 1, 2006;
30(5):
361 - 364.
[Full Text]
[PDF]
|
|
|
|
|
|
|
L. W. Roberts, J. Coverdale, A. Louie, and E. V. Beresin
Academic Psychiatry: Coming of Age
Acad Psychiatry,
August 1, 2005;
29(3):
237 - 239.
[Full Text]
[PDF]
|
|
|
Get information about faster international access.
Privacy Policy
Copyright © 2003
Academic Psychiatry.
All rights reserved.
Home
| Search
| Current Issue
| Past Issues
| Subscribe
| All APPI Journals
| Help
| Contact Us
|
