Acad Psychiatry 30:150-157, April 2006
doi: 10.1176/appi.ap.30.2.150
© 2006 Academic Psychiatry
Use of Clerkship Learning Objectives by Members of the Association of Directors of Medical Student Education in Psychiatry
Amy C. Brodkey, M.D.,
Frederick S. Sierles, M.D. and
John L. Woodard, Ph.D.
Received July 12, 2005; revised November 26, 2005; accepted December 6, 2005. Dr. Brodkey is affiliated with the University of Pennsylvania School of Medicine, Department of Psychiatry, Philadelphia. Drs. Sierles and Woodard are affiliated with the Rosalind Franklin University of Medicine and Science, Department of Psychiatry and Behavioral Science, North Chicago. Address correspondence to Dr. Brodkey; brodkeya{at}verizon.net (E-mail). Copyright © 2006 Academic Psychiatry.
|
ABSTRACT
|
OBJECTIVE: The authors aimed to determine the extent and use of the 1995 psychiatry clerkship goals and objectives published by the Association of Directors of Medical Student Education in Psychiatry (ADMSEP) and to obtain members’ guidance regarding their proposed revision. METHODS: ADMSEP members were surveyed regarding their awareness and current use of the objectives, their advantages and disadvantages, and suggestions for revision. RESULTS: Fifty-four of approximately 130 members returned surveys, including 60% of members who identified as clerkship directors. Ninety percent of respondents were aware of the objectives and 48% used them. Use was significantly related to years of ADMSEP membership. Those who used the objectives did so moderately or extensively, and 84% found them moderately to very useful. Reasons for nonuse were the employment of other objectives, unawareness of them, too lengthy/not user-friendly, and lack of resources to implement them. Comprehensiveness and specificity were cited as both their most useful aspects and their least useful aspects. The most frequently suggested revisions included prioritization, adding more clinical competencies, decreasing the total number, and developing supporting documents such as clinical cases, references, sample examinations, and a resource manual. Respondents held strongly favorable attitudes regarding the utility of learning objectives. CONCLUSION: The ADMSEP psychiatry clerkship learning objectives were widely but not uniformly used among members surveyed, and use is significantly related to years of membership. Respondents strongly favor using learning objectives and desire that a new iteration of the document be more user-friendly, oriented to competencies, and supported by resource materials.
|
INTRODUCTION
|
All major national clerkship organizations specify educational objectives to guide expectations for student learning in their clerkships (1). In 1995, the Association of Directors of Medical Student Education in Psychiatry (ADMSEP) developed and published psychiatry clerkship goals and objectives using a consensus process among its members (2). Given that the clerkship may be a future physician’s last comprehensive educational experience in psychiatry, the objectives were explicitly based upon an assessment of the knowledge, skills and attitudes needed by generalists to recognize and treat or refer the common and emergent psychiatric disorders they are likely to encounter. The document consists of 23 sections, including topics encompassing clinical skills (e.g., interviewing); diagnostic categories (e.g., mood disorders); therapies (e.g., psychopharmacology); clinically relevant groupings (e.g., geriatric psychiatry); and professional development (e.g., collaboration). Each section includes an overall goal and specific behaviorally defined learning objectives encompassing knowledge, skills and attitudes. The document concludes with "Sample Psychiatric Screens," which are suggested sets of questions for diagnostic screening. The extent and types of use of the ADMSEP objectives remain unknown.
Moreover, concern among medical student educators and organizations has grown regarding the clinical competency of medical school graduates (3, 4, 5) due to adverse developments in academic and clinical medicine. Decreased availability of faculty and resident time for teaching (6, 7), increased specialization, and regulatory and payer exclusion of students from active participation in patient care have led to diminished opportunities for students to perform basic clinical tasks under faculty supervision (8). Alarm over high rates of medical errors (9), as well as the recognition of the limitations of written knowledge-based examinations (4), has also fueled a growing emphasis on measuring trainees’ behavioral performance outcomes. This paradigm shift away from knowledge mastery toward measurable competencies (10) is evidenced by such recent developments as the Accreditation Council on Graduate Medical Education (ACGME) competencies (11), the Association of American Medical Colleges Task Force on Clinical Skills Education of Medical Students (8), the United States Medical Licensing Examination Step 2 Clinical Skills Exam (12), and the widespread adoption of clinical skills centers and objective structured clinical examinations (13). In 2004 the Liaison Committee on Medical Education (LCME) promulgated regulations including documentation of students’ achievement of selected clinical objectives supporting the overall articulated goals of the medical school and the specific objectives of clerkships (14).
In light of these developments, in 2002 the ADMSEP council established a task force charged with reviewing, revising and updating the 1995 objectives. To facilitate this task, we surveyed the ADMSEP membership to determine the current extent and types of usage of the objectives and the members’ preferences for the proposed new iteration.
|
Methods
|
Survey Instrument
In 2000, the Council on Medical Student Education in Pediatrics (COMSEP) published a survey of their membership’s use of COMSEP’s national pediatrics clerkship curriculum (15), which we adapted into a 29-item instrument. The descriptive items included number of years as an ADMSEP member, current educational administrative positions, number of clerkship geographic sites, use of the same objectives at all sites, and perceived extent of control of the curriculum. Respondents were asked whether they were aware of the ADMSEP objectives, whether they had read them, and the formats in which they were read.
Respondents were queried whether they use the objectives. If not, they were asked to check all reasons from a list provided (see Appendix 1). If "yes," they were asked the extent of their use (minimal, moderate, extensive); how useful they were (not at all, somewhat, moderately, very); their most and least useful aspects; and with whom they reviewed them.
View this table:
[in this window]
[in a new window]
|
APPENDIX 1. Reasons Given by Directors of Medical Student Education in Psychiatry for Not Using ADMSEP Goals and Objectives (N=31, 56 total responses*)
|
Respondents who used the objectives were asked to check all ways they used them from a list provided (see Appendix 2) and to indicate whether they modified them, as well as to check all sections they used from a list provided (see Appendix 3). Section XXI, Psychotherapies, was inadvertently left off the final survey form. All respondents were asked to select items from a list of suggestions for revision of the objectives (see Appendix 4).
View this table:
[in this window]
[in a new window]
|
APPENDIX 4. Respondents’ Suggestions for Improving the Psychiatry Clerkship Goals and Objectives (N=44, 146 total responses*)
|
Respondents’ opinions regarding the general use of educational objectives were assessed with eight items using a four-point Likert scale (Figure 1). A final optional item asked them to list up to 10 content and skill areas they believed were most essential for students to learn in the psychiatry clerkship (see Appendix 5).
View larger version (50K):
[in this window]
[in a new window]
|
FIGURE 1. Perceptions About Clerkship Objectives Among Directors of Medical Student Education in Psychiatry
|
Sample Selection
ADMSEP membership is individual, not institutional. Some of the 125 U.S. and 16 Canadian LCME-accredited medical schools were represented by more than one member, while others had none. In April 2003, we mailed surveys to 130 ADMSEP members representing 90 (72%) U.S. and 7 (44%) Canadian medical schools as an attachment to the ADMSEP annual newsletter. Of these, 10 were returned due to inability to deliver to the addressee. In June 2003 surveys were again distributed to the 94 attendees at the ADMSEP annual meeting in Jackson Hole, Wyoming, with the request that only members respond and that those who had previously returned surveys refrain from doing so a second time. Respondents were instructed to avoid appending any identifying data.
Statistical Analysis
Data were entered and analyzed on SPSSx (SPSS Inc, Chicago). Most statistics were descriptive. Pearson chi-square (2-sided) was used to ascertain the extent to which length of ADMSEP membership was associated with awareness and use of ADMSEP objectives. Written answers to open-ended questions were compiled into similar content groups and tabulated according to frequencies.
|
Results
|
Fifty-four of 120 (45%) surveys were returned. The total number of respondents who described themselves as clerkship directors (N=39, 72%), either alone or in combination with other positions, totaled 60% of the 65 ADMSEP members identified as clerkship directors in our database. The mean number of years of membership in ADMSEP was 6.3 (range=0–29). The mean number of clerkship sites directed was 4.7 (range=1–23), and most respondents (89.6%) used the same educational objectives at all sites. Respondents perceived they had moderate (13.5%) or extensive (69.2%) control of the curriculum.
Forty-three (89.6%) respondents reported awareness of the objectives, and 38 (76%) reported having read them. Of these, 79.2% had read them in Academic Psychiatry (2), 47.3% on the ADMSEP website (www.admsep.org), 36.8% in the original drafts sent to members in 1994–5, and 34.2% in the Handbook of Psychiatric Education and Faculty Development (16).
Twenty-five (48%) respondents indicated that they currently used the ADMSEP objectives. Of those who did not use them, reasons for nonuse are listed in Appendix 1. A majority had developed their own objectives or used another system mandated by their school. Of those respondents who used the objectives, 10.5% indicated minimal, 52.6% moderate, and 36.8% extensive use. Use was significantly associated with 5 or more years of membership in ADMSEP (p=0.026, Pearson chi-square test). The middle two pages of the survey contained questions for those who currently used the objectives; response to these questions was lower than that of the rest of the survey, ranging from N=16 to N=44. The objectives were utilized in a variety of ways (see Appendix 2); the mean number of uses was 3.7 (range=1–7, SD=1.6). We found a significant correlation between reported extent of use and the number of uses listed (Spearman’s r=0.653, p=0.006). Eighty-four percent of respondents found the objectives to be moderately or very useful, and two thirds reported modifying them to fit their own needs. Respondents had reviewed the document with their psychiatry department faculty (64.7%), education committee (64.7%), chair (47%), and their school curriculum committee (47%).
Of the sections used by respondents, use ranged from a high of 100% to a low of 47%. Appendix 3 lists the extent of use of each section. The mean use of all 23 sections by respondents was 80.0%, indicating that they used most sections of the document.
In response to open-ended questions, 16 respondents listed what they felt were the objectives’ most and least useful aspects. Most frequently cited positive aspects included their comprehensiveness, relevance, and thoroughness; their standardization and endorsement by ADMSEP; and their provision of an outline to guide students’ and faculty members’ learning expectations. Conversely, the extensiveness, level of detail, comprehensiveness and length of the objectives were also cited most frequently as their least useful qualities. A list of ideas for revision of the ADMSEP objectives was provided to respondents to endorse; results are summarized in Appendix 4.
Figure 1 shows members’ responses to a series of statements about the value and utility of learning objectives. We combined strongly agree/agree responses and strongly disagree/disagree responses for purposes of simplifying and clarifying the results represented in the figure. Overall, respondents had strongly favorable attitudes toward the use of objectives; however, most (N=36, 70.6%) agreed that students pay little attention to learning objectives.
Twenty-three respondents completed an optional open-ended request to list up to 10 content and skill areas they believed to be most essential for students to learn in the psychiatry clerkship. These responses are summarized in approximate order of frequency in Appendix 5.
|
Discussion
|
This survey constitutes a "snapshot" of the current use by respondents of the 1995 ADMSEP clerkship objectives, their attitudes about using learning objectives generally, and their ideas for updating and elaborating the document. Eight years after their initial publication, the objectives were widely but not uniformly used by our respondents. Those who used them found them moderately to very useful, and used them relatively extensively and in a variety of ways. Use was significantly associated with years of ADMSEP membership; this may reflect earlier participation in the consensus process of their formulation, resulting in having "bought into" them, as well as increased familiarity with them (2). Members strongly endorsed the utility of educational objectives generally.
To our knowledge, only one preexisting report evaluated members’ use of a clerkship organization’s national curriculum (15). From 1993 to 1995, pediatric clerkship directors from COMSEP, assisted by a grant from the Health Resources and Services Administration, developed competencies and objectives for attitudes, skills, and knowledge in general pediatrics, as well as short clinical cases and an accompanying resource manual. As was the case with the ADMSEP document, the pediatrics curriculum was developed using an iterative process of review and input by clerkship directors to build broad support and consensus. However, pediatric department chairs, national pediatric organizations, a multidisciplinary advisory committee, and students also participated in this process. Furthermore, workshops and other activities supporting implementation of the curriculum continued after its adoption. Three consecutive national surveys assessing aspects of its implementation in U.S. medical schools demonstrated increasing use, with the last survey reporting 90% of schools using some part of the curriculum. Inadequate time, faculty acceptance, and financial resources were cited most often as the barriers to implementation (15).
The higher level of use of the COMSEP, as compared to the ADMSEP, curriculum is possibly due to several factors, including 1) its conscious development as a national curriculum; 2) a more elaborate development and dissemination process that emphasized buy-in by a large number of stakeholders; 3) ongoing efforts at implementation; 4) the development of supporting materials; and 5) the use of a framework of presenting problems and symptoms rather than specific diseases. Finally, external financial support is crucial due to the extensive time and resources required for the multiple revisions necessary to achieve widespread consensus, the development of resource materials, and continued efforts at implementation (1).
The major weakness of this study is its relatively low response rate (45%). There are several possible reasons for this. First, at the time of this survey, the ADMSEP membership database was not routinely updated, resulting in a return of 10 (8%) mailed surveys; most probably, an even larger number were sent to persons who were no longer members. Second, conversations with members indicated that in some cases it was assumed that only one member per medical school, or only clerkship directors, were asked to respond. In 2002, approximately 65 of 130 members were listed as clerkship directors in our database, and 30 schools were represented by more than one member. Finally, members may have been discouraged due to the length of the survey and the number of open-ended questions, which had the lowest response rates. The lower response rates for items on pages 2 and 3 of the survey may also have been due in part to members unknowingly skipping those pages by flipping from the front to the back pages.
Unfortunately, our data cannot definitively tell if bias was introduced because of this. The fact that a higher percentage of clerkship directors than others responded may have increased the validity of some findings, since clerkship directors would be more likely to use the clerkship objectives. Conversely, it is likely that more active ADMSEP members were more likely to respond, both because of the effort involved in completing the survey and because of its distribution to attendees at the annual meeting. If true, this could result in a finding of higher use of the objectives by respondents than by other organization members.
A National Curriculum?
Concern over the importance of the local medical school environment caused ADMSEP to eschew the idea of developing a "national curriculum" when creating the 1995 objectives (2). Their revision may provide an incentive to reconsider that decision. Pediatric clerkship directors cited the ability to collaborate on creative teaching and evaluation approaches based on their nationally shared curriculum, and various educational resources (e.g., a textbook, written and web-based educational materials) were developed based upon it (15). Such collaborations could create better materials, reduce educational costs and influence the content of national examinations.
In our survey, almost 80% of respondents disagreed with the statement that "national clerkship objectives are troublesome in that they ignore the particular circumstances at my school." In addition, a number of respondents cited as one of its most useful aspects that the ADMSEP document was a standardized curriculum endorsed by a national organization. A national curriculum may lend itself more readily to the achievement of general clinical competencies since these may be less dependent on specific types of patients and diseases and more reflective of the general methods and skills of a particular discipline. Certainly, an evident desire for more uniformity in experiences and skills of graduating medical students exists (8, 13, 14). However, care must be taken to protect and utilize the idiosyncratic resources and experiences of individual departments. Emphasizing outcomes rather than prescribing process should help increase acceptance of a "national" document, but a careful process of inclusive collaborative development, wide dissemination, and continued efforts at implementation are most important.
Essential Knowledge and Skills
A comparison of Appendix 3 and 5 reveals similarities between the sections of the objectives currently used by respondents and those spontaneously described as the most important knowledge and skill areas to be learned in the psychiatry clerkship. These topics have remained remarkably consistent as constituents of psychiatry clerkship curricula among ADMSEP members (1, 17, 18). The small numbers responding to these questions (N=17, N=23), the regrettable omission from the survey of the psychotherapies section, and the small differences in use among items in Appendix 3 should evoke caution about drawing unduly specific conclusions from these findings.
Competencies
The term "competency" is used to mean both synthetic, complex, overarching goals (as in the ACGME competencies), as well as measurable performance expectations related to learning objectives that support those goals. In addition, the term often infers an observable act of care (also known as a clinical skill). Many of the current objectives are essentially competencies (e.g., interviewing skills, risk assessments). Twenty-one (48%) respondents endorsed the item "add more clinical competencies/skills" for the next iteration. This finding provides some support for the linkage of learning objectives to performance standards and assessment strategies (1). However, the most "competency-oriented" section of the current objectives, the Sample Psychiatric Screens, is the one whose use was endorsed least by respondents. In addition, the other two sections that were cited as least used, Attitudes, Perspectives, and Personal Development and Collaboration, address two of the six ACGME competencies. This apparent discrepancy may bear further investigation.
Level of Detail and Complexity
In developing the initial set of objectives (2), a choice was made to abjure a minimalist standard for the level of knowledge and skill expected of students. In debating this question, we based this decision on several facts. First, the number and level of detail of the ADMSEP objectives were no greater than those of other national clerkship directors’ organizations. Second, there was additional concern that psychiatry was frequently misapprehended as a less substantive field. Third, each objective was felt to be clinically important, covered in standard textbooks, and testable. Fourth, the literature demonstrating the need for more sophisticated psychiatric practice in primary care settings resulted in a desire among educators to attain a higher standard of competency in their clerkship students. Finally, to some extent our learning objectives represented a 4 year psychiatry curriculum; not all of the learning would be expected to be achieved in the clerkship.
The tension between more simple, and possibly more achievable, learning objectives and a more complex, detailed and demanding standard is not unique to this document. It is likely reflected in the finding that our respondents cited the document’s length, comprehensiveness, and level of detail as among both its most and least useful aspects; suggestions for improvement included an equal number of requests to make them more and less specific. It should be noted that number, depth of learning, and level of detail and of complexity of learning objectives, while possibly related, are not conceptually equivalent.
In addition to the length of the current document, other developments may contribute to a desire expressed by many respondents to shorten (45%) and prioritize (75%) its next iteration. These include the increased emphasis on clinical competencies, the widespread national reduction in the time allotted to the psychiatry clerkship (19), the new LCME ED-2 requirement for specific types and numbers of patient experiences, and continued concern over the inadequate level of psychiatric care in primary settings. In our opinion, we can and should maintain a relatively high standard of expectations, while endeavoring to make the revised objectives more user-friendly by using prerequisites (e.g., to encompass some of the knowledge objectives learned in the preclinical years) and by prioritizing them (e.g., depth of learning, essential versus desirable). Both of these actions would support an effort to define learning expectations as a developmental process.
Mandate for Supporting Materials
Besides requesting the addition of clinical competencies, and shortening and prioritizing the objectives, respondents also endorsed the development of additional supporting and elaborative documents (see Appendix 4). Certain of these materials (e.g., glossary, prerequisites) may be relatively simple and straightforward to develop; others (references, clinical cases, examination questions) would require greater effort. The most requested addition was that of a resource manual. The resource manual developed by COMSEP included three major sections: implementation strategies, teaching strategies, and evaluation strategies. The ADMSEP document could be tailored to fit the needs of members; for example, an electronic version might contain links to online educational resources. The development of these materials will require substantial time and effort, necessitating external support.
|
Conclusion
|
Our study supports the importance and widespread use of learning objectives as an essential aspect of teaching in the clinical psychiatry clerkship. Developments in medical education since the 1995 publication of ADMSEP’s objectives, especially the emphasis on measurable clinical competencies, have resulted in an expressed need for their revision and elaboration. Our finding of a significant relationship between years of ADMSEP membership and use of the ADMSEP objectives supports the importance of active participation by members in the development of a new iteration. In addition, it is likely that external funding will be necessary in order to produce the types of supporting materials requested by our survey’s respondents. The fact that inadequate time and resources were cited as major barriers to implementation of both the COMSEP and ADMSEP curricula highlights the importance of ensuring these resources. No document, however thoughtfully and collaboratively designed, can substitute for excellent and dedicated teachers in the achievement of quality clinical education.
|
REFERENCES
|
- Burke M, Brodkey A: Trends in undergraduate medical education: clinical clerkship learning objectives. Acad Psychiatry 2006; 30:158–165[Abstract/Free Full Text]
- Brodkey A, Van Zant K, Sierles F: Educational objectives for a junior psychiatry clerkship. Acad Psychiatry 1997; 21:179–204[Abstract]
- Nutter D, Whitcomb M: The AAMC Project on the clinical education of medical students. Washington, DC, Association of American Medical Colleges, 2000
- Kassebaum D, Eaglen R: Shortcomings in the evaluation of students’ clinical skills and behaviors in medical school. Acad Med 1999; 74:842–849[Medline]
- Howley LD, Wilson WG: Direct observation of students during clerkship rotations: A multiyear descriptive study. Acad Med 2004;79: 276-280
- Brodkey A, Sierles F, Weiner C, et al: Clerkship directors’ perceptions of the effects of managed care on medical students’ education. Acad Med 2002; 77:40–48[Medline]
- Ludmerer KM, Time To Heal: American Medical Education from the Turn of the Century to the Era of Managed Care. The Forgotten Medical Student:196–218, Oxford University Press,1999
- The AAMC Project on the Clinical Education of Medical Students: Recommendations for Clinical Skills Curricula for Undergraduate Medical Education AAMC Task Force on the Clinical Skills Education of Medical Students. Washington DC, Association of American Medical Colleges, 2005
- Kohn L, Corrigan J, Donaldson M: (Committee on Quality of Health Care in America of the Institute of Medicine of the National Academy of Sciences, Eds): To Err is Human: Building a Safer Health System. Washington, DC, National Academy Press, 2000
- Carraccio C, Wolfsthal S, Englander R, et al: Shifting paradigms: From Flexner to competencies. Acad Med 2002; 77:361–367[Medline]
- ACGME Outcome Project. www.acgme.org/outcome/comp/refList.asp. Accessed July 5, 2005.
- Papadakis M: The Step 2 Clinical Skills Examination. N Engl J Med 2004;350;17:1703-1705
- Corbett E, Whitcomb M: The AAMC Project on the Clinical Education of Medical Students: Clinical Skills Education. Washington, DC: Association of American Medical Colleges, 2004
- Liaison Committee on Medical Education website: http://www.lcme.org/standard.htm. Accessed July 5, 2005
- Olson A: A national general pediatric clerkship curriculum: The process of development and implementation. Pediatrics 2000; 106:216–222[Abstract/Free Full Text]
- Brodkey A, Sierles F: Psychiatric Clerkships. In Kay J, Silberman E, Pessar L, Eds. Handbook of Psychiatric Education and Faculty Development; pp 255-304. Washington, DC: American Psychiatric Press, Inc. 1999
- Lamdan R: ADMSEP LCME Task Force report. Presented at the ADMSEP 31stAnnual Meeting, June 25, 2005, Monterey CA.
- Thornhill J, Tong, L: From Yoda to Sackett: The Future of Psychiatry Medical Student Education. Acad Psychiatry 2006; 30:1:23-28.[Free Full Text]
- Rosenthal R. Levine R, Carlson D, et al: The "shrinking" clerkship: characteristics and length of clerkships in psychiatry undergraduate education. Acad Psychiatry 2005; 29:47–51[Abstract/Free Full Text]
This article has been cited by other articles:
|
|
|
|
 
L. W. Roberts, J. Coverdale, and A. Louie
Committing to Medical Student Education
Acad Psychiatry,
April 1, 2006;
30(2):
93 - 94.
[Full Text]
[PDF]
|
|
|
|
|
|
|
J. L. Cutler
Psychiatric Education for Medical Students: Challenges and Solutions
Acad Psychiatry,
April 1, 2006;
30(2):
95 - 97.
[Full Text]
[PDF]
|
|
|
Get information about faster international access.
Privacy Policy
Copyright © 2006
Academic Psychiatry.
All rights reserved.
Home
| Search
| Current Issue
| Past Issues
| Subscribe
| All APPI Journals
| Help
| Contact Us
|
TOP
ABSTRACT
INTRODUCTION
