Most psychiatry training directors would not consider themselves research scholars or even broadly well-schooled in evidence-based practice. Many are “gun-shy” and perhaps even avoidant of anything that smacks too directly of research methodology, design, and, especially, statistics. Yet they are the front-line educators in preparing tomorrow’s psychiatrists to keep up with, critically evaluate, and, in some cases, actually participate in the discovery of new and emerging psychiatric knowledge. Thus, in each of the past 7 years, the American Association of Directors of Psychiatric Residency Training (AADPRT) has held an NIMH-funded (5R13MH74298) daylong conference aimed at increasing psychiatry training and associate training directors’ competence in evidence-based medicine, enhancing their research and neuroscience literacy, and aiding them in the acquisition of tools to deliver that knowledge to their home programs. The overarching rationale is that by helping psychiatry training leaders become more enthusiastic, knowledgeable, and pedagogically prepared to create research-friendly environments at their home institutions, more trainees will, in turn, become research-literate, practice evidence-based psychiatry, and enter into research fellowships and careers.

Results of the first 5 years have been reported elsewhere (1). In brief, these first five conferences were well-attended, well-received and, on the basis of annual evaluations, seemingly meeting their objectives. However, a change in the NIMH’s policy on funding annual conference grants created new challenges. The initial grant (Michele Pato, P.I.) provided funding in each of the first 5 years, so that training and associate training directors could attend the meeting at no cost beyond a single night’s lodging. Attendance was excellent and relatively stable, ranging from 98 to 169 participants per year (1). But, by the time of the competitive renewal (Sidney Zisook, P.I.), the NIMH was no longer willing to fund annual conferences. Thus, AADPRT felt fortunate to receive NIMH support for 3 of the 5 years (Years 6, 8, and 10) during the second 5-year cycle. To meet the challenge of providing a quality meeting that directors would attend in the absence of external funding in Years 7 and 9, AADPRT was required to cut down conference costs and use carry-over funds supplemented by a conference fee. Given the financial hardships many departments of psychiatry were facing in 2012, it was not at all clear that this new strategy would succeed. This report highlights the evolution of the program from the first 5 years to Years 6 and 7; it details how the funding challenge was met, discusses the strengths and weaknesses of the revised format, and presents plans for the future.

To maximize attendance at these conferences, the sessions occur on the day preceding AADPRT’s annual meeting. As described in more detail elsewhere by Pato et al., (1), the overall design of each year’s meeting is a series of plenary sessions introducing participants to new information pertaining to the core theme of that year’s meeting, integrated with highly interactive small-group teaching sessions designed to consolidate knowledge and provide pragmatic teaching tools appropriate for residents at various levels of training. The theme of each meeting, selected to be a compelling and contemporary clinical problem, serves as a vehicle to capture training directors’ attention while teaching relevant brain science, research literacy, and effective pedagogy. With the additional funding in the first 5 years, Dr. Pato had been able to send each year’s participants back to their home institutions with a textbook relevant to the pre-meeting’s particular theme and a CD with all of the day’s presentations and background literature. Although we no longer have funding to provide hard copies of these resources to all participants, we have made all the materials from each conference available for members on the AADPRT website. Each year, written evaluations assess whether and how the previous meeting’s content was incorporated into training programs, satisfaction with the corresponding conference, and suggestions for the following year’s program.

One key addition to the Year 6 meeting (2011) was the inclusion of a recent graduate who had published results of a study closely related to the meeting’s theme (2) and five research residents who had been selected by a “research scholar” committee. They were provided funds for travel and expenses for the conference and were invited to extend their stay for the entire AADPRT meeting. They also were invited to provide a lunchtime workshop: “What Works for Resident Research?” which focused on how they managed to find time and resources to do research as residents and their recommendation to training directors interested in facilitating residents’ research scholarship. The resident scholars also were invited to help co-lead the afternoon workshops.

Since the Year 7 meeting (2012) was one of the unfunded years, a number of new features were instituted: 1) we asked the NIMH to allow carry-over funds to help support continuing invitation of resident research scholars; 2) we saved travel money on research scholars by selecting three scholars from a national pool of applicants nominated by training directors or department chairs and two from the pool of applicants who had already received one of AADPRT’s other fellowships; 3) we charged a modest fee for each participant, to cover meeting expenses; 4) we invited only AADPRT members or local faculty who agreed to speak without honoraria and who did not require travel or hotel reimbursement as plenary speakers and group leaders; and 5) we limited the number of attendees to 150.

Data collection for evaluation of the conferences utilized anonymous questionnaires, which were distributed and collected on-site at the end of each conference. In previous years (Pato et al., in press) data pertaining to utilization of the previous meeting’s content and pedagogy had been collected by an online, between-meeting survey asking attendees whether various teaching methods or skill-areas learned during the previous conference were being employed within their home programs. Because of the relatively low return on these online surveys, they were discontinued after the 2011 meeting, but key items were incorporated into the on-site questionnaires for the 2012 meeting. All evaluations materials were submitted to the Institutional Review Board (IRB) of the University of California San Diego for consent approval before the 2011 conference and were granted an exemption from participants’ consent because they were being used to assess an educational program.

Based on a Likert scale of 1–7, conference evaluations asked all respondents to rate their satisfaction with the content of each component of the conference (1: Not Effective, to 7: Outstanding), and report the likelihood of their using the material introduced (1: Definitely Won’t Use, to 7: Will Definitely Use). Results are presented as relative frequencies (percentages) of specific ratings on the 1–7 Likert scale. Ratings of 5–7 are collapsed in order to present percentages of respondents who rated items above average. Also, those attendees who had been to the previous year’s meeting were asked to comment on whether and how (e.g., in didactics, individual supervision, group supervision, or on clinical rounds) their programs had incorporated more teaching about evidence-based medicine (EBM) since the last conference. For these latter questions, we counted only one survey response per institution, taking the higher-ranking respondent (e.g., training director over associate training director) if multiple people from the same program answered the survey; therefore, the number of respondents for each question equals the number of unique programs answering the question.

The 2011 Annual Conference (Year 6 of the grant), “Evidence-Based Approaches to Suicide Risk-Assessment and Prevention: Insights from the Neurosciences and Behavioral Sciences for Use in Psychiatry Residency Training,” featured three morning plenary lectures on suicide neurobiology, risk and prevention, and assessment; a lunchtime session on the practicalities of residents’ carving out time and acquiring resources for research; and three afternoon workshops (see Table 1). Participant satisfaction was quite high, with 95% of respondents rating the conference above average (5–7 on a scale of 7, where 7: Outstanding); and the majority of respondents stated that they planned to use the material covered in most of the sessions that day (see Table 2). The most poorly-rated session was the lunchtime workshop on “What Works for Resident Research?” Training directors liked the idea of the workshop, but the room selected for this event was not ideal, and technical difficulties made it almost impossible for many participants to hear the presentations. Plans were made to repeat the resident session the following year, but to ensure appropriate facilities and resources. The five research scholars were unanimously pleased with the conference, but some felt that they were not as well prepared as they might have liked for their roles in the afternoon workshops.

The 2012 conference (Year 7), “Evidence-Based Approaches to PTSD Assessment, Prevention, and Treatment: Insights From the Neurosciences and Behavioral Sciences for Use in Psychiatry Residency Training,” followed the same overall design of the previous year (see Table 1). Based on the previous year’s feedback, the noontime session on “What Works for Resident Research?” was held in a large lecture room with adequate audio-visual support. As the summary ratings of each year’s evaluations show (see Table 2), this session was much better received than the previous year’s resident session. However, we also received feedback suggesting we vary the format in the following year’s conference and include training directors from the residents’ programs in the session. One other change to the Year 7 program also involved the resident scholars: instead of having the resident scholars participate in each afternoon workshop, they all were given roles in one workshop (repeated three times), titled “Using Team-Based Learning to Teach Evidence-Based Treatment for PTSD,” and they were provided teaching materials and training before the meeting. Residents were very positive about this experience and felt that this focus on their helping as team-based learning (TBL) facilitators should be continued in the future. Of note: the small-group session that the resident scholars co-led had the highest ratings of all teaching sessions, with 96% (103 of 107 responders) rating the content as Above Average to Outstanding, and 97% (103 of 106 responders) rating it as Will Probably Use to Will Definitely Use.

As illustrated in Figure 1, 84% of responding training programs (46 of 55) between the 2010 and 2011 conferences and 78% (40 of 51) between the 2011 and 2012 conferences integrated more EBM teaching in their didactic curricula. Between the 2010 and 2011 conferences, another 53% of training programs incorporated more teaching on EBM in individual supervision, 27% in group teaching and 42% during clinical rounds. Similarly, between the 2011 and 2012 conferences, 43% of the training programs incorporated more teaching on EBM in individual supervision, 14% in group teaching, and 45% during clinical rounds.

Attendance was excellent at both conferences, with 170 participants in Year 6 (2011) and 150 in Year 7 (2012). In Year 7, some training and associate training directors were turned away because of the “cap” of 150. Since the 2012 conference was held in San Diego, the UC San Diego, San Diego VA, and San Diego State University (SDSU) faculty provided all of the plenary speakers and a few small group leaders in order to supplement the resident research scholars and AADPRT members who also co-led small groups. In each year, five outstanding research scholars were selected, each of whom had outstanding research accomplishments, generally in the areas of the conference’s clinical focus. In each year, two of the research scholars also were AADPRT International Graduate Medical (IMG) Fellows, adding considerable ethnic diversity to the meeting.

This report describes two in a series of NIMH-supported conferences aimed at enhancing the ability of leaders of psychiatry residency training to provide training in research literacy and produce both clinician–scholars and physician–scientists in their home programs. Both the excellent attendance, even after a modest conference fee was instituted, and the evaluations, suggest that training directors recognize the importance of these goals and that the conferences are helping to fill a critical gap in their preparedness. As in previous years (1) attendees appreciated the content provided, planned to implement at least some of the material in their training programs, and, in fact, programs did incorporate increased teaching on EBM after each conference.

Major limitations are the lack of information on exactly how attendance at the annual meetings alters training and whether any subsequent changes in research training translate into the desired results: better-trained clinical scholars and more physician–scientists launching successful academic careers. Pato et al. (1) documented a modest increase in the number of hours dedicated to research-related training over the first 5 years of these conferences, but, without a control group, it is impossible to know the degree to which such increases can be attributed to these annual meetings; nor do we know whether there have been any changes in the quality or quantity of research presentations, manuscripts, or grants, or whether more graduates are entering research fellowships or receiving academic appointments as a result of their training and as associate training directors attending these conferences. To address these limitations, Drs. Pato and Zisook have modified the immediate post-meeting survey to anonymously track the implementation of research and training ideas from previous conferences and to gain more specific information on the career paths of residents within each training program. Also, we will begin obtaining formal written evaluations from each of the scholars and solicit their permissions to track their academic paths.

These limitations aside, the annual conferences are a relatively modest investment of NIMH funds to enhance research literacy and provide meaningful pedagogical guidelines to the leaders of American psychiatric training, perhaps the most influential group of psychiatric academicians for shaping the knowledge, attitudes, behaviors, and practices of tomorrow’s psychiatrists (3–6).