As long as medical schools, residency programs and continuing medical education (CME) activities exist, educators will need to select good students, teach them, develop new curriculum materials and teaching methods, and evaluate and test the results of their efforts. These educational efforts will take place in systems undergoing constant change that are contending with ongoing political pressures regarding selection of trainees; concerns about the pertinence and quality of educational offerings; technological innovations; and financial pressures for increased efficiency, effectiveness, and accountability of medical education and clinical care. There will never be enough resources to support all the educational efforts that teachers and administrators believe to be desirable.
In adapting to this world, medical educators need to be increasingly inventive, incorporating rapid changes in pedagogy fueled by advances in informatics and the internet revolution; they will have to meet learners' rising expectations for increasingly sophisticated teaching materials and methods, evident in all college, technical, professional and adult learning. Pressures on medical educators, as on all other producers of goods and services, will be to continually do it "better, faster, cheaper."
Those privileged and challenged to create this educational future require the skills necessary to effect and study these changes. For the purposes of this article, we use the broadest definitions of educational research: systematic efforts that span inventive, inductive advances; rigorous quantitative hypothesis-testing; and rich, qualitatively descriptive and theory-enriching scholarship; and starting from this perspective, I review the various domains of educational research, peer into potential future developments in psychiatric education, and address how today's and tomorrow's educators may best be prepared to conduct the studies they are likely to originate.
+
Why Be Concerned About EducationalResearch in Psychiatry?
As in any profession, devoted medical educators want to advance their fields, innovate, develop teaching methods of greater elegance, and, ultimately, do things better than their forebears. And, as in any complex system, medical school and training-program decision-makers allocate teaching faculty, curriculum time, money, and associated resources on the basis of Darwinian evolutionary principles. The best cooperators and competitors are likeliest to survive and prosper. Proof of adaptive success has to be demonstrated in the operating currency of the milieu's quantitative and qualitative measures of effort: papers and presentations documenting successes and innovations that enhance the reputation of schools and departments, helping to put them on the educational map. Deans and department chairs constantly look at each other's performances—striving for more successful, less expensive, and more contemporary ways of educating students. They like to develop ideas that others adopt. No one wants to be the last one to implement worthwhile, innovative methods. As in other fields, innovations and new concepts may achieve the status of "memes," novel ways of doing business that catch on like fads, some of which may ultimately be proven to be beneficial, and some of which may not be worth the effort or costs. Those who develop and disseminate the best memes achieve higher status.
For individual faculty members, engaging in educational innovation and research gratifies many of our higher needs and values, including generativity, creativity, novelty, and stimulating affiliations, thereby increasing personal and job satisfaction. If you've gone to the trouble of creating a new curriculum or educational-delivery system, you're bound to be curious as to how good your innovations will be. You'll at least want feedback to help you modify or tweak your program locally. If you really think you've developed a "hot product," you're going to be more interested in showing that you've accomplished something worthwhile, to inform others about your work. If you and others think that your achievements are worth expanding and exporting, you'll want to set up multi-institutional programs to demonstrate and evaluate your new developments. For those who find themselves in work situations where convivial collaborators share educational interests, developing teaching programs and research programs ensures a way of spending enjoyable time with people they like. Remember, academic research is largely a form of adult play. Playmates can come from within one's own department or, sometimes more fruitfully, from across several departments in medical schools and universities at large. Where collaborators from several disciplines join forces, the scope of educational research projects undertaken may be broader, more complex, and more impact-loaded. For those stimulated by novel technology, the thrill of being able to purchase and play with new technological wonders to be used in psychiatric education—that is, "toys"—may be a greater incentive to involvement.
Finally, conducting research in psychiatric education may also address some of our baser needs and values. Faculty members on clinical-educator tracks may find that scholarly activity in education is exactly what's expected at their medical schools for promotion. Developing, testing, and demonstrating the superior value of new education materials and methods and creating products attractive enough to merit use in other institutions are tried-and-true paths to academic advancement (
+1,
+2).
+
Which Research Questions in Medical and Psychiatric Education Merit Investigation? What Are the Good Questions?
So many areas can grab our attention—from how learners are selected to how they learn; how individual characteristics influence learning; how learners are best exposed to what educational content via what sorts of teaching tools, materials, environments, and situations; how the systems, facilities, and environments in which learning is conducted affect outcomes; and how the achievements of learners and programs are assessed. Questions often erupt in curious minds as faculty and learners contend on a daily basis with rounds, morning reports, chart reviews, sign-outs, staffing, and various entrance and exit interviews associated with courses, rotations, and semi-annual reviews of training. "How did we get such thick-headed medical students?" "What exactly should we be teaching our students, anyway?" "How much do these students really know about how to conduct a mental status exam?" "Why do so few of our students go into psychiatry?" "How can we ‘spiff up’ our seminars?" "How can we teach all the students the same material using fewer faculty hours?" "What do our psychiatry residents really know about psychopharmacology?" "Why can't our primary care house-staff tell delirium from psychosis?" These questions are often embedded in clear targets of opportunity—clinical and classroom situations directly and easily available to the potential researcher.
Questions meriting attention should, first of all, be those about which researchers feel most strongly and those that capture imaginations. Good research questions are hypothesis-driven and are based on some sort of theoretical model; they are not atheoretical data-quests, so-called "blind fishing expeditions." Good research questions address important gaps in the literature and generate meaningful responses to the "So what?" question. ("So what if you find the anticipated [or unanticipated] result?") What difference will the answers make, one way or the other? Is the problem sufficiently important to make studying it worth the effort? Investigations for which the imagined or anticipated findings are likely to result in important "next steps"—changes in policy, procedures, programs, resource allocation, and so forth, or open up new and important research areas—are worthier than efforts unlikely to have such impact. In evaluating potential educational research questions, investigators are well served to think through the long-term intended and unintended consequences of their studies with regard to "better, faster, cheaper." Are the studies feasible—realistic to consider within the constraints of the time, energy, talent, abilities, and financial resources available to tackle them?
Consultations with statisticians at early stages are very important in assessing whether the anticipated effect size of an intervention, as well as the sample size of learners, incidents, outcomes, etc., are large enough for the study results potentially to achieve the statistical power necessary for the investigators to be able to make a meaningful statement based on the results. If it's clear from the beginning that a good idea will never have a chance of actually being proved or disproved, is it still worth investigating?
Finally, as a very practical matter, it's always appropriate to evaluate potential research questions with respect to likely sources and extent of research funding available to support the work. In the absence of funding, how likely is it that the work can be carried out and sustained to completion simply by faculty volunteerism and scrounging up necessary resources? At the very start of planning, it's important to estimate realistically just how much time and effort will be required, so that the researchers aren't caught short or blind-sided in the middle of the project. Too many good ideas are dropped in midstream because time, energy, or enthusiasm run out and because competing interests and concerns emerge to take center stage. Projects are much more likely to be dropped in the middle if they are not sustained by the resources, demands, and external deadlines of funded grants.
Several ways of categorizing educational issues have been suggested.
+Table 1 integrates and lists some of the non-exhaustive domains suggested in the education research literature. This grouping offers a wide array of independent, mediating, moderating, and dependent variables.
+Table 2A,
+Table 2B, and
+Table 2C list illustrative research issues involving psychiatric education associated with each of these domains.
+
What Study Designs and Methods Are Used in Psychiatric Education Research?
The designs and methods available to the psychiatric education researcher vary widely in sophistication and complexity, ranging from relatively simple cross-sectional surveys ("Here's what students/ faculty think, know, do about…"), single case studies with pre—post evaluations of courses, programs, and institutions ("Here's what we did and here's what happened as a result.") to complex controlled interventions (introducing new courses or methods in a carefully controlled manner, using non-intervention comparison groups across settings for long periods of time). Most successful research programs start small, gathering experience and pilot data, and then evolve into larger, multi-institutional efforts. As educational assessments and interventions examine clinical practice outcomes and tackle more complex relationships of education and practice, they dissolve into the domains of health services research (
+3).
Surveys of programs elsewhere in your school, region, or around the country are always useful to help assess how your own efforts are working in comparison to others and to identify the potentially best practices (
+4). Because most educators are curious about how well things work elsewhere, surveying may help satisfy these curiosities and uncover helpful ideas. For any significant undertaking of this sort, it's useful to begin with some in-depth conversations or focus groups with individuals who are likely to represent targets of the survey, to learn as much as possible about the details of the issue to be studied. Preliminary questionnaires and interview instruments should always be pilot-tested to assess clarity, user-friendliness, and respondent burden before the instruments are finalized. Quantitative information should, whenever possible, be complemented by qualitative and narrative information, to learn about the dynamics, nuances, and contextual realities of what's behind certain developments. To paraphrase Einstein: not everything that counts can be counted, and not everything that can be counted counts.
Developing and testing new methods in education closely resembles research involving clinical interventions, and similar design principles hold (
+5). Educational research designs may be based on cohort studies, case-control studies, and randomized clinical trials, in which appropriate control and comparison groups are established and for which pre- and post-testing pilot data are used to plan larger studies with adequate sample size and power.
+
How Can Proficiencies Needed by the Psychiatric Educational Researcher Be Acquired?
Which required research proficiencies will be required will depend on the general scope and nature of the projects being undertaken and the questions being asked. At minimum, novice researchers should be familiar with principles of clinical and epidemiological research design and especially with methods associated with survey and clinical interventions research. Although some statistical background is helpful, in all but the most elementary of projects, most faculty will collaborate with biostatisticians to help with initial design and data management. Beyond the basics, necessary proficiencies will depend on the specific technical nature and complexities of the educational interventions and research being proposed; and it always helps to be proficient in knowing how to acquire resources, including funding, to accomplish the project.
These proficiencies can be acquired in a several ways. The exceptionally talented, independent-minded person can self-instruct with the help of the many excellent available sources (see, for example, references
+1—
+6). The bright self-instructor will carefully read recent issues of the major journals in medical and psychiatric education, particularly
JAMA,
Academic Medicine, and
Academic Psychiatry, as well as the general internal medicine and family practice journals, and become familiar with the questions, designs, methods, and resources needed to conduct the sort of research that makes its way into high-impact journals.
A more usual method for acquiring knowledge and experience involves finding mentors and collaborators at one's local health science center and university who possess the necessary research and statistical backgrounds and who find the proposed project sufficiently interesting to work on. The novice may be lucky enough to find appropriate mentors nearby, in the department, in other departments in the school, in other schools within the University, in other local colleges and universities, or at a distance. Mentors come in different types (varying in technical specificity and general wisdom), and they also vary in their altruism and availability.
Collaboration is the rule rather than exception. A perusal of the tables of contents of recent volumes of Academic Medicine reveals that the large number of research reports are multi-authored and that many of the authors have Ph.D. and Master's degrees rather than M.D.s. Many are affiliated with educational centers and dean's offices in medical schools, academic departments of psychology, and other affiliated research institutes. Through closely working with these collaborators, the new researcher may, via apprenticeship, learn many of the issues to be considered, techniques to be applied, and important tricks of the trade. However, the skills learned from mentors via apprenticeship may not be sufficient for subsequent projects, and the investigator who hopes to develop a career in educational research may require more formal lessons. Often, formal courses in research design, statistics, and other specific content areas pertinent to educational research are locally available in universities.
Over the past decade, an increasing number of medical schools have developed special "mini-fellowships" for academic junior faculty interested in education and educational research. Organized primarily through the dean's office, these programs, usually known as Medical Scholars' or Medical Education Scholars' programs, are usually about a year in duration, requiring a half-day to 1 day per week of formal activities. In the course of these programs, most participants focus primarily on teaching methods, but many are also expected to complete a supervised educational evaluation. Although these programs are generally too short and provide less instruction and experience than is necessary for conducting a research career, they are helpful for getting started. Some typical programs may be visited at the websites listed in
+Table 3.
For the faculty member seriously preparing for an academic career in which educational research will play a major role, certain types of post-residency fellowships offer attractive opportunities. The prototype is the Robert Wood Johnson Clinical Scholars program, now hosted at seven universities around the nation. These fellowships usually last 2 years, require course work often leading to advanced degrees such as the M.P.H., and include strongly-mentored independent research on the part of the scholars. Much of the research conducted through this program concerns health services or related social science research; virtually all the acquired skills are useful in education-related research. The goal of the training is to help develop scholars into independent researchers capable of writing their own grant proposals and getting started in performing independent research. Interested readers can visit the appropriate websites at http://www.rwjf.org/nation/jnation.htm.
Other NIMH- and NIH-sponsored postdoctoral research training fellowships are available through many universities, and institutional program directors may be able to provide faculty mentors for educational research and/or health services research. Good programs provide trainees with start-up funds, computers, access to laboratories needed for the research, access to research subjects, and access to support personnel, including research assistants and statistical consults. Their didactic programs usually include or provide access to courses and seminars in research design, statistics, the applicants' specific areas of research interest, general research in psychiatry and ethics, and opportunities to earn academic degrees such as the Ph.D. or M.P.H. (
+7). The APA (American Psychiatric Association) Office of Research ordinarily can provide up-to-date listings of available research fellowships in psychiatry. Additional opportunities are available to minority residents and junior faculty via the Program for Minority Research Training in Psychiatry (PMRTP) program sponsored by the APA (
http://www.psyc.org).
Finally, increasing numbers of universities are providing sophisticated distance-learning programs for physicians, some of which involve degree-granting programs such as the M.P.H., which may entail a few weeks spent on-site at the sponsoring university, followed up by courses and projects completed locally and worked on by mail, e-mail, and web-based instruction. These distance-learning opportunities mean that virtually any young faculty member anywhere can now access expertise and mentoring in ways that were not possible a decade ago. Although this type of education and training is not necessarily as comprehensive or convenient as having a slew of mentors available down the hall, nevertheless, current possibilities ensure that the faculty member who is motivated to learn educational research methods will have many opportunities to do so.
Funding opportunities to support research in medical education may be sought in a variety of directions. For small grant proposals, especially for research that might provide pilot data leading to larger proposals in the future, many university medical schools provide in-house research funding of amounts in the range of about $10,000, whose purpose is to provide seed-money for junior faculty research. For grant proposals involving demonstrations and evaluations of innovative programs in which physician behaviors, system change, and patient outcomes are included, funding from NIMH, the Agency for Healthcare Research and Quality (AHRQ, at
http://www.ahcpr.gov/fund/training/trainix.htm), and the Substance Abuse and Mental Health Services Administration (SAMSHA, at
http://www.samsha.gov) also offer assistance. Other federal agencies that are not often considered for funding but which may be sympathetic to research pertinent to psychiatric education include the Department of Education and the National Library of Medicine (the latter particularly concerning evaluating education projects associated with informatics and newer methods of electronic communication). Private foundations at the national level that have supported educational innovations include the Robert Woods Johnson Foundation, the Kellogg Foundation, and the Josiah Macey Foundation, among others. Funding sources for psychiatric research in general may be found in the volume edited by Pincus (
+7). In many regions, smaller, private family foundations make funds available for purposes meeting their specific interests, including aspects of medical education. University development officers are usually aware of these opportunities for junior faculty. Even pharmaceutical companies may provide unrestricted grants for medical education and research, particularly when the projects touch on topics such as developing and evaluating media presentations of educational material, physician—patient communications, teaching learners to provide informed consent concerning medications, and other areas in which industry has even a peripheral interest.
Helping to foster research in medical education, the Association of American Medical Colleges (AAMC) developed the Group on Educational Affairs (GEA), devoted to promoting teaching and learning opportunities to all medical-education constituents and sponsoring mini-workshops, small-group discussions, and meetings of GEA Special Interest Groups (SIGs). Among the 11 current SIGs, several may be of special interest to readers of this article, including Computers in Medical Education (http://www.med.sc.edu:108), Faculty Development (http://www.aamc.org/about/gea/sigs/faculty.htm), and Quality Improvement in Medical Education (http:// www.aamc.org/about/gea/sigs/cqisighp.htm). The GEA's Research in Medical Education (RIME) Conference, also held in conjunction with the AAMC Annual Meeting, includes RIME papers, symposia and abstracts sessions, and Innovations in Medical Education exhibits. Titles of presentations at the most recent RIME conference can be found at http://www.aamc.org/ meetings/annual/2000/sponsor/rimeover.htm. The GEA has four regions: Southern, Central, Northeastern, and Western, corresponding to AAMC regions. Each holds an annual Spring Meeting for faculty, educators, and administrators from the member institutions within their respective regions. Regional GEA meetings include activities analogous to those offered at the national meeting, including meetings of regional SIGs. GEA also publishes a twice-yearly newsletter, "The GEA Correspondent," and sponsors four list-serves focusing on undergraduate medical education, graduate medical education, continuing medical education, and research in medical education. For further information, including information on how to be included on these list-serves, visit http:// www.aamc.org/about/gea.
Recognizing that opportunities for formal training in educational research and evaluation were limited, the GEA established the Fellowship in Medical Education Research (FMER), a 1-year program in which teachers and course directors interested in exploring questions that arise from their work are matched with an experienced medical-education researcher who guides and assists the fellow in completing a medical-education research project. The aims of the fellowship have been to focus on skill-building in research and to foster communication and socialization related to the norms and values of medical-education research. The program has grown considerably in scope and complexity since its inception in the mid-1990s and is currently being evaluated and revised accordingly. FMER focuses on skill-building in research, communication, and socialization related to the norms and values of medical-education research. During the course of the program, fellows are expected to design an educational research project using appropriate methodologies; establish a network of medical-education research colleagues; attend three Core Curriculum Sessions, AAMC Annual Meetings, and a GEA Regional Meeting; and participate in the fellowship's list-serve by posing or answering questions with other fellows, mentors, and instructors. Those chosen as mentors in this program have been expected to contact their Fellows at least twice each month; review the Fellow's project, proposal, and abstract at various stages to provide feedback; and provide general support, collegiality, and encouragement, among other functions. Costs, including travel costs to several meetings per year and a $1,000 Fellowship fee, are incurred by the sponsoring department or dean's office. Information may be obtained from http://www.aamc.org/meded/research/ fmer.htm.
Finally, a great deal of small-scale and even multisite collaborative educational research is conducted from resources derived from creative scrounging and volunteerism. The perusal of research reports in recent issues and volumes of Academic Medicine, mentioned earlier, also revealed that relatively few of the projects acknowledged sponsorship by a funding agency. This may reflect the fact that for some of the smaller projects reported on, the scope of the project did not merit the time and effort required for writing a proposal or the fact that that many educational researchers are not schooled in grant-writing and acquisition, and/or that many opportunities exist to develop funding to support the field of research in medical and psychiatric education. It is highly likely that, in the future, research concerning psychiatric education will be better, and perhaps faster, but probably not cheaper, given that a great deal of current educational research often seems to be funded "out of thin air." We are unaware of any studies on the number of medical-education grants submitted, funded, and turned down, and reasons for success and/or failure. Come to think of it, wouldn't that be the subject of an interesting research study?
It should be evident that more important questions pertinent to medical education in general and psychiatric education in particular can be posed than will ever be addressed in the real world. One task for the field as a whole is to determine priorities in order to combine and marshal resources so that the most important of these questions can systematically be addressed. Currently, some consortium arrangements of educational researchers exist, such as those organized under the AAMC by the GEA, as described above. These groups meet at regular intervals, share their ongoing work, and sometimes develop collaborative relationships and projects.
For psychiatry, consortiums of interested educational researchers might be established through any of several education-oriented professional societies, including the Association for Academic Psychiatry (AAP), the American Association of Directors of Psychiatric Training (AADPRT), and the Association of Directors of Medical Student Education in Psychiatry (ADMSEP). Such consortia could determine priorities for the field, collaborate on research proposals and funding applications, and link with related groups through GEA activities. The stronger the proposal and the sponsoring base, the more likely funding requests will be successful.
For the individual faculty member considering investments of time, energy, and cathexis: if passion, curiosity, and resolve to study medical and psychiatric education are present, it's very likely that the field—and the funding agencies—will respond.