Academic Psychiatry 29:14-21, March 2005
© 2005 Academic Psychiatry
Getting Started in Educational Research
John Coverdale, M.D., M.Ed., F.R.A.N.Z.C.P.,
Alan Louie, M.D. and
Laura Weiss Roberts, M.D.
|
INTRODUCTION
|
For many of us, the idea of becoming an educational researcher feels, at best, awkward and burdensome and, at worst, intimidating and overwhelming. We at Academic Psychiatry often receive queries from prospective authors about how to get started in educational research. These can take the form of how to choose a specific topic, what would be of interest to readers, and what scientific design to use. Academic Psychiatry aims to promote original research in our developing field and, especially, to support new researchers. We hope to persuade our readers that doing educational research is possible and enriching. To this end, we offer some comments on how to initiate and sustain educational research projects.
Research is an expression of our curiosity. For academically-committed psychiatrists who serve as clinician educators, educational research represents a way of answering questions of specific interest to us. Research is systematic inquiry that begins with a question that may arise from a particular observation, problem, assumption, or "intuition." The underlying question becomes reframed as a "hypothesis," the process of examining the question becomes the "methodology," and the "results" are the findings (or "data") as interpreted in light of the context, strengths, and limitations of the approach undertaken to explore the question.
The basis of an empirical study can be found in the everyday ideas and activities of the clinician-educator. The key to getting started in research is being vigilant for opportunities to generate and pursue questions: take the time to notice the potential value of an idea worth exploring! It is valuable to maintain a log of interesting ideas, hypotheses, or problems to resolve. Educational research ideas can arise from comments from colleagues, clinical events, or from any educational experience. For example, educational research can arise from the need to make a decision concerning the merit of an educational intervention or program, in relation to its specific goals and objectives. Ideas for educational research can also follow from reading the work of others that is viewed as interesting or compelling. Educational journals constitute a rich resource, with some prominent examples in medicine, including Academic Medicine and Teaching and Learning in Medicine, which are published in North America, and Medical Education and Medical Teacher, published in the United Kingdom. Educational articles also appear in general and specialty medical journals. These papers can help provide models for early researchers who are seeking to understand how to take a core concept or observation and translate it into a project which is then reported in a clear, concise format.
While the academic clinical and teaching environment is rich with ideas and activities to inspire our curiosity, how do we know which to study? What are the next steps? Crucial aspects to successfully launching and completing an educational research project are: selecting a topic; literature searching; mentoring and teamwork; and perseverance. We will work through these considerations in turn.
|
SELECTING A TOPIC
|
There are several factors that might be considered as influential in the process of prioritization and selection of a research question. These include the level of personal enthusiasm for the topic area (1, 2), the importance of the topic (1–3), and the achievability of the goals within administrative and time restraints (1–3).
Enthusiasm
The research topic should be of interest to the investigator for the following reasons. First, an interest or enthusiasm of the investigator will likely enhance the interests of others. Further, relevance of a topic to the investigators’ own activities and efforts will likely relate to it having authenticity and value to others. Finally, genuine understanding of the complexity of an issue helps establish one’s expertise as the right person to explore and then write about a topic area. Confidence that the investigator is the right person to write on the topic supports the enthusiasm which is necessary to sustain the drive to see the project through to completion.
Significance
The most vital questions that editors ask reviewers is whether a given manuscript addresses an issue that is important and whether it contains important new ideas or findings. This is vital to the rationale for doing the work, and it is also critical in the decisions whether and when to publish the work. Significance pertains to the prevalence and/or seriousness of an issue. Further, significance is understood in part by how the research question and methodology will contribute to current theory or relevant research.
The status of the literature on a particular topic dictates how rigorous a study has to be in order to be a helpful contribution to the field. A descriptive study may be of value when there are few studies on a topic that is timely, whereas more substantive data will be needed when a topic area has been relatively well addressed. Importance is determined in part, therefore, by how a study improves on earlier methodologies, although replication studies have merit by enabling a reassessment of the confidence attributable to the findings. Importance is also judged by the likelihood that the results will benefit teaching as judged by learner outcomes and any associated clinical outcomes. Findings that have wider generalizability as determined by the sample size, inclusion and exclusion criteria, might be construed as more important than studies with limited generalizability.
One helpful strategy after deciding on the research question or goals, and after finding all relevant articles, is to draft the Introduction of the paper. The Introduction includes comments about what has been achieved before on the topic, any limitations of that earlier work, and how, if at all, the proposed research will attempt to rectify these. Drafting the Introduction enables reclarification of the merits of the study for the researcher and fine-tuning of the question or goals when indicated which, in turn, prescribes the research design. It is also helpful to note that there is no such thing as a "perfect" study. All studies have strengths and limitations, which should be accurately understood so that interpretation of the findings is sound.
Since the scientific literature is not a unitary entity, knowing which field or aspect of the literature one hopes to advance is a strategic point for early educational researchers to consider. For example, in the mainstream education journals, much has been written about competencies and about performance-based examinations. Although the editors, reviewers, and readers of these journals will understand the significance of the work, the "bar" for methodological rigor for publication will be higher in these journals. On the other hand, editors, reviewers, and readers of academic journals that are more clinically-oriented may be very interested in papers that are more fundamental (e.g., effective ways of working with competencies and performance examinations in clinical curricula for various learners so long as the relevance is clearly explained). For these reasons, the investigator should have some ideas about the peer-reviewed journals to which prospective manuscripts might be submitted. This list may shape plans since each journal has different ideas about which research questions are important and what methodologies are rigorous enough for publication.
Achievable Goals
It is imperative to choose a topic and to define goals and objectives that are realistic for the time frame and resources available and that are within existing ethical parameters. This can be the most difficult part of getting launched in educational research. Choosing a topic and framing a core research question that can be tested scientifically with a simple survey is a widely used, relatively low tech and low cost technique (4). Similarly, selecting an issue that can be examined through the analysis of existing data (i.e., information collected for other purposes, such as test performance data) is another key technique for educational research on a "shoe string." Both of these approaches require oversight (i.e., either approval or formal exemption) by the institutional review board (see companion article in this issue of Academic Psychiatry, pp 1–5.) In the early stages of deciding on a topic, it is helpful to canvass what support, including administrative and financial support, will be available. By a realistic estimation of how much time and effort will be required, researchers should not be caught short or blind-sided in the middle of the project (3).
Keeping it simple facilitates the achievement of goals and objectives. The investigator should anticipate that a research interest will evolve and hopefully blossom. Starting out modestly with a small or pilot study tests the investigator’s interest in a particular topic area, stimulates creativity and develops a track record which in turn enhances confidence. This is not to say, however, that more challenging and resource-dependent projects of potentially higher impact should be avoided, but these need to be approached in a stepwise manner.
Last, details of the study design should be outlined to be sure they are practical and achievable. Flaws in design are best discovered early and need to be anticipated to ensure that the final data will be publishable. The statistical soundness of the study should also be assessed. For instance, a power analysis will help determine the minimum number of subjects necessary to avoid a type II statistical error. Then the investigator may assess whether that many subjects and the total study duration are realistic given available resources.
|
LITERATURE SEARCHING
|
For the purpose of searching the educational literature, research questions can be focused on three parts. These include the identification of the population from which the learners are sampled (e.g. medical students), the educational component or issue to be studied (e.g. an intervention), and specific outcomes. This is similar to the construction of the focused question based on a clinical scenario in evidence-based medicine (5), which in turn consists of four parts when a control group is involved. Each of the key words or terms identified in the question serves as an entranceway to the literature.
There are two excellent papers that describe the iterations of comprehensive searching in medical education (6, 7). These describe methods for searching including the use of Boolean operators, combining terms and limiting data sets. These processes are illustrated with search screen images and by numerous examples.
With respect to educational research, it is worth appreciating that there are no databases constructed for medical education literature alone. PubMed does incorporate educational articles from medicine, but these might be difficult to retrieve because of inadequate subject headings (6). Searching should therefore be supplemented by checking citations in recent publications on a topic, and by use of more than one database. For the readers of our journal, please note that Academic Psychiatry was only incorporated into Medline in 2002. Prior to then, Academic Psychiatry was indexed in EMBASE, Social Sciences Citation Index and other databases. This is a resource that might not be tapped by searching PubMed alone. We found two such excellent papers of relevance to this piece that were not retrievable from PubMed (3,11).
|
MENTORING, NETWORKING, AND TEAMWORK
|
For many early career scholars, seeking mentoring, engaging in "networking," and fostering teamwork will be especially helpful. Mentoring is an important factor in promoting research in psychiatry (8–12), and presumably this holds true for educational research in psychiatry also. By selecting a mentor with whom ideas can be expressed, approaches planned and frustrations expressed, successes can be reinforced and negative forces buffered (1). Psychiatry departments should also appreciate that a milieu that promotes curiosity about research, provides research training and encourages networking can be influential (12). Teamwork is often an essential, and very rewarding, element in getting started in research. In a recent study, collaboration across a variety of settings was viewed as an influential factor in promoting the productivity of a sample of highly successful medical education research groups (13). Yager (3) reminds us that academic research is largely a form of adult play in that enjoyable time is spent in collaborating with people who are liked.
Mentors and colleagues may assist in various ways. For instance, they can assist by asking around about an idea, asking for constructive criticism about a method or an early draft, getting help with an IRB application, getting advice about what journal to go to, interpreting data, interpreting rejection letters from journals that are really "fix-and-come-hither" letters and so on.
Few research articles appearing in these pages are single-authored. Factors to consider in picking a mentor (who may or may not serve eventually as a co-author), networking, and developing a team are many. For instance, mentors at other institutions can facilitate the extension of a study to other sites, which, by increasing the recruitment of subjects, can aid generalizability and publishability. It is also important to consider prospective members’ excitement for the topic, willingness to contribute meaningfully, and compatibility with other members. There may be a need to fill a niche in a team in terms of a requirement for expertise in a special area. Alternatively, some individuals might be chosen for a team because they have the drive, resources or administrative leadership to make the research happen.
Though the writing of a manuscript may seem far off into the future, the investigator should begin to think about the members of the research team, their anticipated contributions to the research, and everyone’s expectations with regards to whether they will be an author and the order of authors in the byline. Open discussion about this will prevent future misunderstandings and allow the investigator to later provide an ethical listing of authors, giving credit where credit is due.
|
PERSEVERANCE
|
Perseverance is relevant to all research, but it requires particular emphasis regarding educational research. Medical education research remains relatively uncharted territory and pioneers always face extra obstacles. Funding streams for medical education research are not well established and doing research with medical students is not considered standard procedure. Additionally, there are few clinician-educator role models and peers in comparison to other fields of research. A great deal of patience is required to plan and complete the research and to write a through paper. In this regard, it is important to maintain a focused attitude and avoid becoming overwhelmed by taking on too many other projects (1)or to abandon promising ideas. It is also important to have fun with the process, in brainstorming ideas, in teamwork, in writing, and in anticipation of the final product. Finally, anticipate how to build on the accomplishments of one project for future projects (1).
|
CONCLUSION
|
The primary intent of this article was to encourage those who are thinking about getting started in educational research toward doing so. Factors deemed to relate to success in this process include having a low threshold for appreciating the potential value of ideas, selecting a topic that is important, maintaining enthusiasm, setting achievable goals, mentoring, networking, teamwork, maintaining focus, and not giving up! At Academic Psychiatry, we welcome your inquiries about how to get started, and we are delighted to offer feedback (and words of encouragement) as you pursue this important work for our field and for our present and future learners. The spirit we wish to foster is one of collegiality and support in all of your endeavors in educational research.
|
REFERENCES
|
- Kahn CR: Picking a research problem: the critical decision. N Engl J Med 1994; 330:1530–1533[Free Full Text]
- Gall MD, Gall JP, Borg WR: Educational Research: An Introduction. White Plains, New York, Longman, 2003
- Yager J: Preparing psychiatrists to do educational research. Acad Psychiatry 2001; 25:17–27[Abstract/Free Full Text]
- Sierles FS: How to do research with self-administered surveys. Acad Psychiatry 2003; 27:104–113[Abstract/Free Full Text]
- Sackett DC, Richardson WS, Rosenberg W, et al: Evidence-Based Medicine: How to Practice and Teach EBM. Churchill Livingstone, New York, 1998
- Haig A, Dozier M: BEME Guide No. 3: Systematic searching for evidence in medical education, part 1: sources of information. Med Teach 2003; 25:352–363[Medline]
- Haig A, Dozier M: BEME Guide No. 3: Systematic searching for evidence in medical education, part 2: constructing searches. Med Teach 2003; 25:463–484[CrossRef][Medline]
- Shapiro T, Mrazek D, Pincus HA: Current status of research activity in American Child and Adolescent Psychiatry, part I. J Am Acad Child Adolesc Psychiatry 1991; 30:443–448[Medline]
- Mrazek D, Shapiro T, Pincus HA: Current status of research activity in American Child and Adolescent Psychiatry, part II: a developmental analysis by age cohorts. J Am Acad Child Adolesc Psychiatry 1991; 30:1003–1008[Medline]
- Pincus HA, Haviland MG, Dial TH, et al: The relationship of postdoctoral research training to current research activities of faculty in academic departments of psychiatry. Am J Psychiatry 1995; 152:596–601[Abstract/Free Full Text]
- Lambert MT, Gaver DL: Mentoring psychiatric trainees first paper for publication. Acad Psychiatry 1998; 22:47–55[Abstract/Free Full Text]
- Faville A, Bloch S: Women psychiatrists and research: so far, no further? An Australasian perspective on factors that encourage and hinder women psychiatrists’ involvement in research. Aust N Z J Psychiatry 2004; 38:470–476[CrossRef][Medline]
- Arnold L: Preface: Case studies of medical education research groups. Acad Med 2004; 79:966–968[CrossRef][Medline]
This article has been cited by other articles:
|
|
|
|
 
K. Edenharder
Practical Knowledge for Academic Psychiatrists
Acad Psychiatry,
June 1, 2006;
30(3):
238 - 238.
[Full Text]
[PDF]
|
|
|
|
|
|
|
L. W. Roberts, J. Coverdale, A. Louie, and E. V. Beresin
Academic Psychiatry: Coming of Age
Acad Psychiatry,
August 1, 2005;
29(3):
237 - 239.
[Full Text]
[PDF]
|
|
|
Get information about faster international access.
Privacy Policy
Copyright © 2005
Academic Psychiatry.
All rights reserved.
Home
| Search
| Current Issue
| Past Issues
| Subscribe
| All APPI Journals
| Help
| Contact Us
|
TOP
INTRODUCTION
SELECTING A TOPIC
