Nicotine dependence is the most prevalent substance abuse disorder among individuals with mental illness, with the prevalence of tobacco use two to four times that reported in the general population (1, 2). Individuals with mental illness and addictive disorders are estimated to account for 44% to 46% of cigarettes sold in the United States (1, 3), equating to the consumption of 187 billion cigarettes annually (4). Tobacco use adversely affects the duration and quality of life for patients with mental illness, is predictive of future suicidal behavior, and can reduce the therapeutic blood levels of some psychiatric medications (5–11). In terms of lives saved, quality of life, and cost efficacy, treating smoking is considered one of the most important activities a clinician can do (12).
Treatment guidelines recommend integration of smoking cessation efforts within psychiatric care (13–15). APA recommends that psychiatrists assess the smoking status of all patients, including readiness to quit, level of nicotine dependence, and previous quitting history, and provide explicit advice to motivate patients to stop smoking (16). Although long-term abstinence from all nicotine-containing products is the ultimate goal of treatment, “initial goals include moving smokers from not contemplating smoking cessation, to contemplating cessation, to initiating a quit attempt for a short period” (13). Evidence-based tobacco treatments include nicotine replacement, sustained-release bupropion, varenicline, nortriptyline, clonidine, and psychosocial therapies. Combined therapies (counseling plus pharmacotherapy) for treating nicotine dependence are emphasized (16). An organizational framework for tobacco cessation treatment is the 5As intervention for clinicians—to ask all patients about tobacco use, advise smokers to quit, assess readiness to quit, assist with quit attempts, and arrange follow-up care (17, 18).
Few psychiatrists routinely address patients’ tobacco use, and psychiatry residents report lack of tobacco treatment training in medical school and psychiatry residency training (19–21). Nationally, only half of psychiatry residency programs provide training for treating tobacco use and dependence, with a median reported duration of 1 hour (22). Covered content areas vary greatly. Programs without tobacco training report low levels of confidence in residents’ skills for treating nicotine dependence and specify lack of faculty expertise as a barrier to training. Most training directors (89%) report interest in evaluating a model tobacco treatment curriculum for psychiatry and state that they would, on average, dedicate 4 hours of curriculum time. A variety of medical curricula have been developed to address tobacco use, and a recent systematic review suggested that training health professionals had a measurable effect on professional performance, including offering counseling, setting quit dates and follow-up visits, distributing self-help materials, and recommending nicotine replacement. Of the 10 trials identified, however, none were conducted with mental health professionals (23). Dissemination of an evidence-based tobacco treatment curriculum has the potential of dramatically increasing the proportion of smokers with mental illness who receive assistance with quitting.
Built upon the Rx for Change curriculum, which was developed for schools of pharmacy and informed by a comprehensive literature review, consultation with an expert advisory group, interviews with psychiatry residency training faculty, and a focus group with psychiatry residents, we developed a 4-hour evidence-based tobacco treatment curriculum for psychiatry residency training programs (24–26). To our knowledge, this is the first curriculum specifically tailored to the clinical issues unique to treating smoking in individuals with mental illness.
Here we summarize our evaluation of the Rx for Change in Psychiatry curriculum in three psychiatry residency training programs in northern California. We hypothesized that the curriculum would be associated with significant improvements in psychiatry residents’ knowledge, attitudes, confidence, and counseling behaviors for supporting cessation among their patients who use tobacco, with initial changes from pre- to posttraining sustained at 3-months’ follow up. Residents’ tobacco cessation practices were operationalized in terms of the 5As framework (18). A 6-month chart review was conducted at one of the training sites to verify changes in residents’ self-reported tobacco cessation counseling activities. Additionally, the per-resident costs associated with curriculum implementation were determined.
Curriculum Development and Description
We applied Kern and colleagues’ (27) framework for medical education curriculum development. Surveys conducted with psychiatry residents (n=105) and national residency training directors (n=114) documented training needs and informed the curriculum’s educational goals and learning objectives (21, 22). We created a fully referenced curriculum outline, with didactic and interactive learning strategies, which we sent to our expert advisory group. We incorporated their feedback and developed curriculum slides, which were evaluated in a focus group with psychiatry residents and informed by individual interviews with psychiatry residency training faculty (25). The interviews aimed to delineate effective methods for integrating tobacco treatment strategies within psychiatric training and practice.
The Rx for Change in Psychiatry curriculum includes faculty lecture slides and a resident resource binder. The training emphasizes evidence-based, patient-oriented tobacco treatments relevant for all tobacco users, including those not yet ready to quit. Key topics include the epidemiology of tobacco use in patients with mental illness and health effects, drug interactions with smoking (with a focus on psychiatric medications), clinical practice guidelines and the role of psychiatry, counseling, and pharmacological tobacco treatments. The techniques, which can be applied within a variety of treatment orientations (e.g., cognitive behavior, psychodynamic, medication management) and clinical settings (e.g., outpatient, inpatient, addictions treatment), are fully supported by evidence-based research, and the teaching slides are heavily referenced with current literature citations. Interactive training components include the opportunity for residents to handle actual (nonprescription) or placebo (prescription) nicotine replacement products while learning key counseling messages; a clinical practice exercise for residents to practice the 5As with an actual patient in their caseload; and case-based treatment planning with eight fully developed cases differing in demographics, diagnostic details, and stage of change. The resident resource binder includes all teaching slides; ancillary handouts on the 5As, strategies for managing nicotine withdrawal, a pharmaceutical product guide, and a booklet for patient tracking of mood and tobacco use; recommended readings; a referral resource list; patient cessation brochures; and stickers to streamline charting of the 5As. Abbreviated materials and shortened (1-hour) trainings were provided to clinical faculty and attending physicians at two of the three programs to support residents’ efforts to address tobacco with their patients.
The current study summarizes curriculum implementation and evaluation using a quasi-experimental design. A clinical psychologist (JJP) and psychiatrist (SCF) team taught the curriculum over two to four sessions at the three participating sites. Four separate trainings were held with psychiatry residents at the University of California, San Francisco (UCSF, 2005 and 2006), San Mateo County (2006), and California Pacific Medical Center (CPMC, 2006). Participants were UCSF third- and fourth-year residents and residents across all 4 years of training at San Mateo and CPMC. The training was required at San Mateo, CPMC, and for third-year residents at UCSF. Training attendance rates were higher when the training was required (84%) rather than voluntary (30%). Attendees were asked to complete a voluntary pretest survey prior to the training, a posttest survey immediately after the training, and a 3-month follow-up survey. Respondents were paid $25 for each completed survey. One resident opted to attend the training sessions but not complete the study evaluations. All other attendees (n=55) participated in the pretraining evaluation, with 96% survey completion at posttraining and 98% at the 3-month follow up. The study procedures were approved by the UCSF institutional review board for conduct of research with human subjects, and informed consent was obtained from residents prior to completing the survey evaluation.
In the first year, a systematic chart review (N=1,204 medical charts) was conducted covering the timeframe 3-months prior to and following the training to evaluate documented clinician practices in assessing and treating patients’ tobacco use. The chart review was conducted for all adult individual outpatient visits for UCSF third- and fourth-year residents regardless of the residents’ training attendance. The first author (JJP) developed a codebook detailing definitions and coding rules and trained study staff in chart review procedures. Three staff completed the chart reviews. For each staff member, the first 10 charts were dual-coded to ensure fidelity with the coding scheme. Additionally, a random sample of 10% of the charts were selected for assessment of errors in coding (28). Because of the time-intensive nature of chart reviews and the inherent difficulty in gaining access to medical records due to Health Insurance Portability and Accountability Act (HIPAA) regulations and institutional access restrictions, the chart review was conducted only with UCSF residents participating in year 1 of the curriculum evaluation.
Psychiatry Resident Self-Report Survey
Many of the items were drawn from existing measures for the Rx for Change curriculum and had been extensively pilot tested (with more than 3,000 students) (24). The knowledge, confidence, and attitude scales were included at all three time points, while the behavioral items were assessed only at pretraining and the 3-month follow-up to allow time for changes in clinical practices. Resident descriptive characteristics were assessed at baseline. A variety of curriculum satisfaction and evaluation items were included in the posttraining and 3-month follow-up surveys.
Knowledge was assessed with nine multiple choice and three true-false items that addressed the epidemiology, health effects, and treatment (pharmacological and psychosocial) of smoking among psychiatric patients. Scores were calculated as the percent correct out of 12 items. Three sets of knowledge items (36 items in total) were developed and administered at different time points among the different residents, so that changes over time would not be due to memory effects.
Attitudes regarding potential barriers to addressing tobacco use in psychiatric practice were assessed with 10 items rated on a 5-point scale (1=strongly disagree, 5=strongly agree). The items were averaged into a single scale score, Cronbach alpha=0.83 at baseline. Two additional items assessed general attitudes concerning the role of psychiatry in prevention and treatment of tobacco use among patients with mental illness.
Confidence for treating tobacco use and dependence in psychiatric practice was assessed with seven items rated on a 5-point scale (1=not at all confident, 5=extremely confident). The items were averaged into a single scale score, Cronbach alpha=0.75 at baseline.
Behaviors were assessed as the proportion of patients in a resident’s caseload who were asked about tobacco use and, among tobacco users, provided each of the remaining “4As” (advise, assess, assist, arrange follow-up), ranging from 0% to 100%.
Descriptive items assessed residents’ age, sex, race/ethnicity, residency program and training year, and tobacco use history. Curriculum satisfaction items assessed residents’ impressions of curriculum content and recommendations for future use and dissemination.
A structured form facilitated systematic coding of patients’ tobacco use status and, for patients who used tobacco, resident cessation advice, assessment of readiness to quit, assistance, and arranging follow-up (5As). “Advise” was defined as notation of advice or discussion about quitting smoking; “assess” as reference to patients’ readiness or motivation for quitting smoking; “assist” as giving patients cessation materials, referrals, recommendations or prescriptions for medications to aid cessation, or providing cessation counseling or problem solving; and “arrange” as arranging follow-up visits or phone calls (29). The names of the residents providing the treatment and the number of outpatient contacts during the chart review period were coded. All visits for a patient scheduled within the 3-month identified period were reviewed, and documentation of the 5As at any point during that period was coded as five dichotomous variables (occurring or not). Performance measures were calculated as the ratio of reviewed patient charts with documentation for each of the 5As over the total number of patient charts reviewed, per resident, expressed as percentages.
Determination of the Cost of the Curriculum
Data were collected on all curriculum-related costs of materials and services, such as training manuals, nicotine replacement samples, and faculty teaching and preparation time. Curriculum-related costs were differentiated from research-related costs of study recruitment, assessment, evaluation, start up, and development. The cost estimate reflects all costs of activities needed to replicate the curriculum in an academic setting, including contacting residents and providing an initial introduction of the curriculum to program training directors.
The primary interest concerned change from pretest in residents’ knowledge, attitudes, confidence, and behaviors posttraining and at 3-months’ follow-up. Linear regression models were run evaluating change scores in the primary outcome variables with the intercept providing a test of the significance of changes reported over time. Because of nesting of residents within training programs, an effect for program was included in all models. Additionally, because of the quasi-experimental study design, potential covariates of resident training year, smoking status, and interest in tobacco treatment training at pretest were evaluated for inclusion in model testing. Cost-identification analysis estimated the incremental cost that would be incurred by psychiatry residency training programs, per resident trained, in adopting the tobacco cessation curriculum.
Sample Characteristics and Program Attendance
The resident sample was 58% female (n=32). Ethnicity was 53% Caucasian (n=29), 24% Asian American/Pacific Islander (n=13), 7% Hispanic (n=4), 4% African American (n=2), and 12% other (n=7). Just over half the sample (54.5%) was from UCSF (n=30), 20% from San Mateo (n=11), and 25.5% from CPMC (n=14). The residents were in their first (12.7%, n=7), second (14.5%, n=8), third (60%, n=33), and fourth (12.7%, n=7) years of training. The residents’ tobacco use status was 0% current, daily use; 9% current, nondaily use (n=5); 18% former use (n=10); 44% former experimentation (n=24) (defined as none currently and less than 100 cigarettes in one’s lifetime); and 29% never use (n=16). At baseline, 98% of the sample (n=54) reported being moderately to extremely interested in learning how to help their patients quit smoking.
The residents attended a mean of 3.4 hours (SD=0.9) of the 4-hour program, with 60% attending the complete program. The training was delivered in two, three, or four sessions. Greater attendance was associated with scheduling the training in fewer sessions (r=−0.28, p=0.048), being in an earlier year of residency training (r=−0.35, p=0.008), and greater interest in tobacco treatment training at pretest (r=0.36, p=0.007).
Change in knowledge scores was significant from pre- to posttraining, averaging an increase of 18 percentage points (F=66.14, df=1, 51, p<0.001). Both attendance (r=0.36, p=0.010) and initial interest level (r=0.28, p=0.043) were positively correlated with knowledge gains from pre- to posttraining. That is, the more hours of the training the residents attended and the greater their initial interest in the training, the greater their gains in knowledge. A significant decline was observed in knowledge scores—a decrease of 11 percentage points—from posttraining to 3-months’ follow up (F=37.44, df=1, 51, p<0.001). However, the 3-month scores remained significantly higher than the pretest scores (F=8.72, df=1, 53, p=0.005). Changes in knowledge did not differ by resident smoking status or year of training.
Statistically significant reductions were observed for each of the attitudinal barrier items and the total score (p<0.05 in both cases) (Table 1
). The change in the attitude total score was unrelated to resident smoking status, year of training, or reported interest. At posttraining, all residents (100%) indicated that psychiatry should be more active in helping patients with mental illness quit smoking, and 89% believed that psychiatry should be more active in preventing patients from starting to smoke.
Confidence for Counseling
Significant gains were observed from pretest to posttraining and sustained at the 3-month follow up for all seven confidence items and the total score (p<0.005 in all cases) (Figure 1
). Confidence items with the greatest gains from pretest to 3-months follow up were having the skills to monitor and assist patients through their quit attempt (meanΔ=+0.74); addressing patients’ smoking adequately when time is limited (meanΔ=+0.76); assisting recent quitters with relapse prevention skills (meanΔ=+0.87); and referring patients to appropriate smoking cessation programs (meanΔ=+1.09 on a 5-point scale). Change in the confidence total score was unrelated to resident smoking status, year of training, or reported interest.
The percent of patients that residents reported asking about smoking remained unchanged from pretraining to 3-month follow-up: 39% and 41% respectively, p=0.967. However, significant increases were found in residents’ reports of advising their patients who smoke to quit (28% to 55%, p=0.002), assessing their patients’ readiness to quit (19% to 48%, p<0.001), providing assistance with quitting (26% to 50%, p=0.004), arranging follow up (8% to 28%, p=0.010), and providing cessation referrals (3% to 19%, p=0.003).
A systematic chart review was conducted on all resident outpatient contacts at UCSF in the 3 months prior to and following the Rx for Change in Psychiatry training during year 1 of the curriculum evaluation. We reviewed 570 medical records at baseline and 634 at follow-up, representing a total of 5,862 outpatient contacts. Reliability analysis indicated strict adherence to the coding protocol with a 1% error rate at pretest and 3-month follow-up. Patient contacts were collapsed at the level of the resident to evaluate changes in resident behaviors. Comparisons were made over time for the 18 residents who attended the training in year 1 and the 7 residents who did not (Figure 2
). No significant changes were found over time among the untrained residents. Among the residents who were trained, consistent with their self-reports, significant improvements were found in their advising smokers to quit, assessing readiness to quit, assisting with quitting, and arranging follow up (p<0.05 in all cases); asking about tobacco use was high at pretest due to a tobacco-use item on the intake form.
Perceptions of the Training Program
On average, residents reported that 53% of the curriculum material taught was completely new to them and 44% had been taught before but needed to be reviewed (3% reported that it was an unnecessary review). The residents estimated that they would use 67% of the program information when working with patients. When asked if they believed participating in the program would increase the number of patients they would counsel to quit smoking, 40% said “probably yes” (n=21) and 60% said “definitely yes” (n=31). Three participants did not complete the posttraining assessment. When asked if participating in the program would increase the quality of counseling they would provide, 4% said “not sure” (n=2), 27% “probably yes” (n=14), and 69% “definitely yes” (n=36). On a 6-point scale (1=poor, 6=outstanding, top 5%), the residents gave the training an average score of 5.4 (SD=0.6), with 6% rating the training as good (n=3), 48% as excellent (n=25), and 46% as outstanding (n=24). All of the residents (100%) recommended that students at other psychiatry residency training programs would benefit from the tobacco cessation training.
The residents rated the usefulness of the different curriculum components on a 5-point scale (1=not at all useful, 5=extremely useful). All curriculum components received a mean of 3 or greater (3=useful) (Table 2
Curriculum costs were tracked and summed to estimate the costs to deliver the curriculum in a residency training program. Our program development costs for article copyright approvals, program graphics, and logo were not included ($550 total). Costs for resident materials for the resource binder ($26 per resident), nicotine replacement samples ($2.60 per resident), and the costs for faculty time for course preparation (4 hours), providing the training (4 hours), and responding to residents’ questions after the training (3 hours) were included. Salary rates were based on 2001 data for physician faculty salaries in psychiatry departments, adjusted for inflation to 2006 dollars (30). A fringe benefit rate of 25% was used for estimation of fully absorbed faculty salary costs for faculty-related expenses. The total cost averaged $139 per resident trained. Additional costs for training clinical faculty and attending physicians would be $11 to $29 per individual trained, based on the level of materials provided, number of faculty participating in the training, plus any costs for providing continuing education credits.
Our previous research documented a need for formal instruction in treating tobacco use and dependence in psychiatry residency training programs (21, 22). To address this need, we developed and evaluated an evidence-based 4-hour curriculum with 55 residents in three northern California psychiatry residency training programs. The curriculum resulted in significant changes in residents’ knowledge, attitudes, confidence, and behaviors for treating tobacco dependence among patients with mental illness. Changes pre- to posttraining were sustained at 3-month follow-up. To our knowledge, this is the first comprehensive curriculum that addresses tobacco use among smokers with mental illness, a group that is estimated to account for nearly one in two cigarettes sold in the United States (1, 3).
The uniformity of the curriculum’s effects with residents representing different residency programs, training years, smoking histories, and initial levels of interest in tobacco treatment supports the generalizability of the program. The effect of the training also can be conceptualized by the increase in the number of patients seen by residents who received attention to their tobacco use. On average, the 55 residents reported clinical encounters with 36 outpatients during the 3-month follow-up assessment window (1,980 patients in total). Based on the residents’ reported changes in the 5As, if a third of these patients were cigarette smokers (a conservative estimate), we estimate that the training resulted in 177 additional patients being advised to quit smoking, 189 being assessed for their readiness to quit, 157 receiving assistance with quitting, 131 receiving follow-up attention on tobacco, and 105 receiving tobacco treatment referrals. Changes in documented practices mirrored self-reported practice behavior. The chart review, though systematic, comprehensive, and evaluated for coding accuracy, was limited to UCSF because of the intensive nature of the chart review and restricted access to patient records at the other two sites. Generalizability of the study findings beyond residency programs in northern California and when taught by faculty other than study investigators is unknown and needs to be tested.
Dissemination of the curriculum to psychiatry residency training programs at a state or national level provides opportunity for even greater reach and impact. The costs of the training, which averaged $139 per resident, are nominal in light of the financial burden of tobacco use and the costs of psychiatry residency training (31). The cost of 4 years of psychiatry residency training, inflated to 2006 dollars, is estimated at $400,000 per resident (32). Based on this estimate, the tobacco treatment training would represent a 0.03% portion of total training costs. To reduce the potential costs of the training for dissemination, the resident and faculty materials are available on the internet as downloadable and printable PDFs (http://rxforchange.ucsf.edu). Identifying faculty available to provide the 4-hour training as part of their teaching commitments can lower the costs even further.
In the current study, the 4-hour training was provided in two to four sessions, providing flexibility in curriculum delivery across programs. Of note, better attendance was obtained when the curriculum was delivered in fewer sessions and when it was required. Attendance also was higher among residents in an earlier year of training and among those who endorsed greater interest in tobacco treatment training at pretest.
Although the curriculum resulted in significant changes in residents’ clinical practices, room for improvement remains. In particular, residents did not ask all of their patients about tobacco use. We recommend offering the training each year and working more toward implementing system-level changes to recognize, prioritize, and reinforce tobacco treatment practices at all levels of patient care. The curriculum was very highly rated, and all participants recommended that students at other psychiatry residency training programs would benefit from the tobacco cessation training.
Lastly, in addition to its primary application to tobacco treatment, the current study is offered as a potential model for developing other shared, evidence-based curricular materials for residency training programs. Several benefits of a shared tobacco education program are noteworthy: all programmatic materials can be maintained at a central location, with ease of access to the most current versions via the internet; resource sharing eliminates the need for duplication of efforts in developing and updating lecture materials, thereby saving faculty resources and providing administrative and economic incentive for adoption of tobacco education in general; and the materials benefit from the collective feedback from faculty who utilize the program. Because enhanced training yields increased delivery of cessation interventions in clinical practice (23) and because clinicians have a proven positive influence on patients’ ability to quit (17), we believe that efforts toward enhancing the tobacco cessation counseling skills of psychiatric residents will ultimately translate into reductions in tobacco use and improvements in public health among patients with mental illness.
FIGURE 1. Changes in Residents’ Confidence in Treating Tobacco Use and Dependence in Psychiatry Settings from Pretest to Posttraining and 3-Month Follow-Up
7-item scale, pretest Cronbach alpha =0.75.
*Pretest to posttraining (F=126.57, df=1, 51, p<0.001); **pretest to 3-month follow-up (F=79.80, df=1, 53, p<0.001); posttraining to 3-month follow-up (F=1.54, df=1, 51, p=0.220)
1=not at all confident, 5=extremely confident
FIGURE 2. Changes in Charted Tobacco Treatment Practices from Pretest to 3-Month Follow-Up among Trained and Untrained Residents
Data were collected from systematic chart review of 1,204 medical records.
Trained residents attended the Rx for Change in Psychiatry in year 1 (n=18), and untrained residents (n=7) did not.
Manuscripts authored by an editor of Academic Psychiatry or by a member of its editorial board undergo the same editorial review process, including blinded peer review, applied to all manuscripts. Additionally, the Editor is recused from any editorial decision-making.
This work was supported by the State of California Tobacco Related Disease Research Program (#13KT-0152) and the National Institute on Drug Abuse (#K23 DA018691 and #P50 DA09253). We acknowledge our expert advisory group members Neal Benowitz, M.D., Stuart Eisendrath, M.D., Mark Myers, Ph.D., Victor Reus, M.D., Steven Schroeder, M.D., and Doug Ziedonis, M.D., who provided valuable input on identification of the curriculum’s content. We thank David Goldberg, M.D., for the opportunity to work with psychiatry residents training at the California Pacific Medical Center and thank Kevin Delucchi, Ph.D., for his statistical consultation.