According to the Accreditation Council for Graduate Medical Education (ACGME), residents are expected to “systematically analyze practice, using quality improvement methods, and implement changes with the goal of practice improvement” (1). Quality Improvement (QI) approaches have become widely recognized as an effective means for improving patient care and clinical outcomes (2). In the setting of healthcare reform, training the next generation of physicians in QI will be increasingly essential, as payment for services becomes linked to performance reporting and compliance with high-quality practice standards (3).

Within psychiatry, reports describing residents' involvement in QI are beginning to emerge (4, 5). Although departmental and hospital-wide QI initiatives have inherently included residents, they have generally not engaged residents as significant collaborators in the QI process. Although various approaches to QI training within residency have been reported across virtually all medical specialties, adoption of systematic and structured curricula has not been widespread (6, 7).

Based on the model for improvement described by the Institute for Healthcare Improvement (8), we created a year-long curriculum to train and engage all PGY3 psychiatry residents in QI. Following adult learning models that favor the immediate application of concepts, our curriculum utilized didactic material while focusing on experiential learning. With support and guidance from the course director, our curriculum allowed residents to work as a group to develop and implement their own QI proposals. This process also allowed residents to consolidate their learning and experience the impact of change in-vivo.

The year-long curriculum was broken up into two phases (Figure 1). During Phase I, residents participated in three 1-hour training sessions focused on the foundations of QI. Each session consisted of a didactic portion, which explained a major QI concept, and an activity designed to consolidate learning of the concept. Residents practiced setting aims and establishing measures by brainstorming about clinical problems they would like to address (e.g., tracking weight changes in patients taking medication, screening patients for depression, monitoring depressive symptoms over time, and systematically tracking intervals when patients need prescription refills to avoid gaps in pharmacologic treatment). Residents were encouraged to consider treatment guidelines, consensus standards of care, and quality indicators to identify potential gaps in care and inform project aims (9). They then proposed small changes that might help them move toward their goals (e.g., incorporating a standardized depression-screening form into the intake packet in the outpatient clinic). Along with the course director, residents reviewed the strengths and weaknesses of various proposals and identified projects that appeared to be feasible. By the end of the third session, residents selected two group projects to pursue over the course of the year.

Through the second phase of this curriculum, residents attended regularly scheduled meetings every 2 months in order to follow up on their set goals in “real time.” Each session in Phase II was structured around the Plan-Do-Study-Act (PDSA) model, as described by the Institute for Healthcare Improvement (8) and included a review of overall group data and individual resident performance measures extracted monthly from the electronic medical record. Residents shared their own perspectives about the value and impact of the most recent changes and developed changes to the plan as indicated.

We determined the success of the curriculum by measuring improvements in knowledge and attitudes about QI in addition to successful engagement in a group QI project. We hypothesized that, if the course was successful, by its end, the residents should develop specific knowledge, skills, and attitudes, including the following:

• Knowledge: Basic strategies for developing QI initiatives, as evidenced by class participation and performance on the Quality Improvement Knowledge Assessment Tool (QIKAT). The QIKAT, developed by Ogrinc et al., has demonstrated discriminate validity in assessing knowledge in QI concepts and consists of three clinical vignettes (10, 11). Residents must identify specific aims, methods for measurement, and potential changes as if they were developing a program to investigate and improve the hypothetical scenario described. Each scenario is graded on a scale of 0–5, for a maximum total score of 15. In our curriculum, we adapted the QIKAT scenarios for relevance to psychiatry.

• Skills: How to apply the PDSA model to QI initiatives, as evidenced by the ability to develop and implement a group QI project.

• Attitudes: Confidence in applying QI techniques, as evidenced by the QI Self-Assessment survey (10, 11). The QI Self-Assessment survey asks residents to rate their level of comfort in 12 QI skills on a scale of 1–4, with 1 indicating “No Confidence” and 4 indicating they are “Extremely Comfortable.”

All PGY3 residents (N=12) completed the QIKAT and QI Self-Assessment Survey before and after completion of the course. In order to eliminate rater bias, the instructor scored all QIKAT exams anonymously using pre-established scoring anchors (11). We compared QIKAT scores between groups by use of paired t-tests for participants’ pre- and post- test scores, and applied Wilcoxon signed-rank tests (S) to compare the pre-test and post-test scores on the QI Self-Assessment survey. Based on our hypothesis that the course would improve knowledge and perceived self-efficacy, one-tailed p values were calculated. The Institutional Review Board at the New York State Psychiatric Institute determined that this project was not subject to IRB review.

During the 2010–2011 academic year, residents collectively developed two QI initiatives aimed to increasing the use of the Quick Inventory of Depressive Symptoms-Self Report (QIDS-SR) scale in the outpatient clinic to 1) screen at least 80% of new patients for depression; and 2) to monitor depression symptoms systematically at least once every 6 weeks in at least 70% of patients diagnosed with major depressive disorder (MDD). Over the course of the program, residents identified methods to incorporate the QIDS-SR into the clinic system (e.g., including the QIDS-SR among the initial admission forms that patients must complete during their first visit and making the forms accessible to patients at the front desk) and their own individual work (e.g., individual reminders, giving the QIDS-SR form to the patients in advance).

As with many QI initiatives, there were substantial differences in performance among residents, with the maximum individual monthly rates of screening and monitoring for depression ranging between 40% and 100% over the course of the year. However, all residents improved rates of screening and monitoring within their own caseloads, and, within 9 months of the academic year, residents collectively increased patient screening for depression in the clinic from 3% to 88%. Regular monitoring of depression symptoms in patients with a diagnosis of MDD increased from 3% to a peak of 61% within 7 months. Subsequently, rates of monitoring averaged about 50%.

Residents showed gains in QI knowledge based on their ability to evaluate hypothetical scenarios and create a proposal for change on the QIKAT. QIKAT scores were significantly higher for residents after participating in the curriculum (6.54; standard deviation [SD]: 2.42) when compared with pretest scores (3.67, SD: 0.96; t=4.1, p<0.001). Individual gains in knowledge varied for course participants. Among the 12 residents, 83% (N=10) showed gains on the QIKAT.

Residents’ attitudes and confidence in constructing QI projects improved significantly over the course of the project according to the QI Self-Assessment survey. Residents’ ratings of self-efficacy in QI methodology at completion of the year-long QI project were significantly higher across each domain when compared with the pre-assessment (p<0.05). Residents reported the highest gains in their ability to use the PDSA model as a systematic framework for trial and learning (S=22.5; p=0.002) and to identify how data are linked to specific processes (S=27.5; p=0.001). Data from all 12 items are available upon request.

This study was limited by the small sample size and its focus on only one residency training program. Although programs designed to train psychiatry residents systematically in QI methodology are beginning to emerge (12, 13), there remain few published reports. Outcomes of such initiatives have shown positive changes in residents’ perceptions of QI and gains in QI knowledge. The success of this curriculum was demonstrated by measured improvements in residents' knowledge as well as gains in confidence and QI skills.

This curriculum engaged residents in the QI process by allowing them to identify and develop goals that are meaningful to them. Teaching QI is more effective when the students are actively involved in a QI project of their own design (6). Although including key stakeholders early in the planning is critical for those initiating any QI project, this is particularly true for residents who often do not have the authority and resources necessary to implement change (6, 7). In this approach, the course director often acted as a liaison with clinic administrators and staff members and helped residents select projects that were feasible, given the time and resources of the group.

The primary focus of this curriculum was the acquisition of QI skills. Efforts to teach practice-based learning and improvement often focus on expanding up-to-date scientific knowledge and familiarity with practice guidelines (14), with relatively little attention to building the skills necessary to implement change. Choosing goals early in the year and allowing residents to meet during protected class time provided a structure that validated QI as a training priority, reminded everyone of the goals at hand, and built momentum.

Few programs allow residents an opportunity to track the outcomes of their own interventions (7). Although the outcomes we shared with the group were de-identified, individual residents who participated in this curriculum were able to track their own performances and see how they compared with the group. This allowed residents to think about their individual responsibility and contribution to the project (9). Comparative performance data can be a strong motivator for change.

QI projects can be labor-intensive, and improvements often occur over time. Having the course run concurrently with the year-long outpatient experience (in which all psychiatry residents engage) allowed time for residents to complete several PDSA cycles and see the impact of their efforts on the QI goal. Other investigators have found that relatively short experiences, such as 1-month block rotations, may not be sufficient for residents to gain an understanding of how to make or sustain changes (10).

We designed this course to be conducted with an entire class of residents over the course of a year. Before instituting a formalized curriculum, some programs may choose to start by developing an elective in QI, thus capitalizing on the enthusiasm of interested residents and staff (7, 8). QI is about change—and all systems can benefit from some sort of change. Thought of in this way, this curriculum is eminently adaptable to other psychiatry training programs seeking to teach and engage residents in QI. Possibilities for future studies include dissemination efforts and an attempt to better understand differences in individual resident outcomes (such as the variation in individual gains in knowledge and varied uptake in screening and monitoring for depression, as noted in this study).

Dr. Arbuckle received financial support for this work through the New York State Office of Mental Health Policy Scholars Program.

At the time of submission, the authors reported no competing interests.