Alcohol and drug dependence-related disorders are increasingly recognized as major public health problems. Alcohol abuse and dependence are well-documented causes of serious social, legal, economic, and health complications. One in every 10 deaths in the United States is related to alcohol, and 20% of the total national health expenditure for hospital care is spent on alcohol-related illnesses (+1). In addition, alcohol and drug abuse have been identified as common factors in marital conflicts and family dysfunction and may account for an estimated $70 billion annually in time lost from work (+2). Hypertension, cancer, and gastrointestinal, liver, cardiac, and psychiatric diseases have been linked to alcohol and drug abuse (+3). Estimates suggest that as many as 60% of psychiatric patients may have coexisting alcohol and drug abuse problems (+4). Further, most addictive disorders are frequently underdiagnosed (+5). While several factors influence this, one may be residents' perceived lack of preparedness to treat these disorders (+6).

Therefore, assessing and improving psychiatrists' level of preparedness to diagnose and treat alcohol and drug abuse might have an impact on their ability to provide adequate medical and psychiatric care to this patient population.

While several educational and training programs have been developed to better teach psychiatrists and prepare them to treat patients with alcohol and drug related problems, studies suggest that their long-term impact may be limited (+7). Further, while junior staff may be justified in reporting a lack of preparedness to diagnose and treat substance related disorders (+8), graduating psychiatry residents reporting a lack of preparedness to treat these disorders is a cause for concern (+6).

The Accreditation Council for Graduate Medical Education (ACGME) currently requires that psychiatry residents receive at least 1 month of full-time, supervised evaluation and clinical management of patients, either in an inpatient and/or outpatient setting and familiarity with rehabilitation and self-help groups. This may occur as part of an inpatient or outpatient requirement (+8). Although most programs meet this 1-month mandatory training requirement, most programs offer residents 2 months of addictions training during residency, with diversity in timings and settings (+8). During this period, however, most programs rely on one key supervisor to educate and train residents and evaluate their training (+8). Current methods of evaluating residents' training are limited to faculty observations of clinical performance (+9). This single method may not be an optimal gauge of residents' competency to treat substance abuse disorders (+10). The Psychiatry Residents in Training Examination (PRITE) is a standardized testing tool that has been used to assess competencies in psychiatry training. The PRITE is a standardized examination taken each year by all residents. Besides addressing general psychiatric disorders and treatment of these disorders, the PRITE contains a variable number of questions on addictive disorders. However, this examination and other similar standardized tests may have limited ability to consistently and accurately identify deficits in residents' substance abuse training. A recent report suggesting that graduating residents, after four years of training, felt less than prepared to treat addictive disorders is an example of how the PRITE may have failed to identify this issue for these residents during training (+6). This implies that current evaluation measures might not be successful in identifying residents' lack of confidence in their addiction psychiatry training. There are few other options available to clinical faculty to determine residents' competence in this area.

In a recent survey of graduating psychiatry residents, Blumenthal et al. (+6) reported that psychiatry residents expressed preparedness to treat most psychiatric disorders. However, when asked about their preparedness to treat substance abuse disorders, the number of residents expressing preparedness decreased significantly (+6). Blumenthal et al.'s report (+6) of this discrepancy was based on one survey question on substance abuse training. Even though the residents' response to this one question suggested deficits in substance abuse training, a single question cannot provide a complete picture of the residents' preparedness to treat this complex group of disorders. Substance abuse training includes education about a spectrum of disorders ranging from those associated with legal drugs, such as alcohol and nicotine, to illegal drugs, such as heroin, cocaine, and marijuana, as well as abuse of prescription drugs. Each drug requires unique knowledge that may prove critical for physicians in providing appropriate management and treatment. Further, treating substance abuse involves education of and training in other complex issues regarding a patient's social structure and support, coexisting medical and psychiatric conditions, pharmacological and nonpharmacological therapies, and an understanding of structured living and self-help groups.

In an effort to provide us with another mechanism to assess residents' preparedness to treat addictive disorders, we developed the Addiction Training Scale (ATS). This is a self-reported questionnaire, which can be completed by residents in a few minutes and will help identify areas where the residents themselves believe improvement is necessary. We also conducted a pilot study of the ATS to assess its validity.

The authors developed the ATS as a self-report tool to yield more detailed information about residents' preparedness to treat substance abuse disorders. Training faculty with expertise in the substance abuse field (comprised of six faculty members with fellowship training and/or certification by the American Society of Addiction Medicine or American Board of Psychiatry and Neurology in addiction psychiatry) proposed 16 items that evaluated confidence in both general knowledge and in practice areas believed to be relevant to successful clinical work with this population. The goal of the research team (consisting of three addiction training faculty members and two psychologists, including a biostatistician) was to develop a questionnaire that was internally consistent and similar to the general question used in the survey by Blumenthal et al. (+6) The 16 items were clustered together by content, and subjects were asked to measure each item on a 5-point Likert scale ("no knowledge" = 1; "very knowledgeable" = 5). The items asked specific questions about residents' knowledge of self-help groups, pharmacotherapies for addictive disorders, alternatives to pharmacotherapies for addictive disorders, various psychotherapies for addictive disorders, and structured living programs. The questionnaire also asked residents to rate their knowledge ("no knowledge" = 1; "very knowledgeable" = 5) of treating patients with alcohol, benzodiazepine, barbiturate, amphetamine, and cocaine addiction. Residents were also asked about their preparedness to work with, treat, and provide nonpharmacologic therapy to patients with active substance abuse or dual diagnosis.

By using the 16-item questionnaire, the ATS is not only multifaceted in providing more information, but also provides more validity when compared to the single question method used by Blumenthal et al. (+6) The ATS yielded more detailed information about the comfort level of residents treating patients with substance abuse disorders.

Three experimental hypotheses were tested: 1) whether items on the new questionnaire would correlate with each other and with Blumenthal et al.'s (+6) general question regarding comfort in diagnosing and treating substance abuse and dependency (internal consistency), 2) whether factor analysis of the questionnaire would yield more than one factor (multidimensionality), and 3) whether residents with more training in substance abuse or more years of general psychiatry training would score higher on the new questionnaire than residents without substance abuse training or those with fewer years of general psychiatry training (criterion validity).

The subjects were 21 psychiatry residents who agreed to participate according to the procedures of the Creighton University Institutional Review Board. Questionnaires were completed anonymously. Nine residents had completed a specific rotation in the treatment of substance abuse and alcohol related disorders. Residents ranged from years 1 to 5 in their overall training experience. All residents completed the ATSS and the 27 items of the original Blumenthal et al. questionnaire (+7).

Internal consistency was evaluated by Cronbach's alpha. For the 16 items on the new scale, α = 0.97. The mean interitem correlation was 0.69. If the general substance abuse and delirium items from the Blumenthal et al. survey (+7) were included in the analysis, Cronbach's alpha remained at 0.97, and the mean interitem correlation was 0.67. These values are consistent with the hypothesis that the new questionnaire would show appropriate internal consistency.

Dimensionality of the ATS was evaluated using factor analysis, which is a method that examines a correlation matrix among a set of variables to determine whether one or more variables can be constructed to represent the intercorrelations among the original variables. It is then possible, using factor loadings, to examine the correlation of each variable in the original data set with these constructed variables, the factors, and to interpret what each factor means and what the original variables have in common that produces a particular factor. Principal components factoring of the 16 items yielded two factors, with Eigenvalues exceeding 1.00 and accounting for 77.5% of the total variance of the 16 items. Factor loadings, after application of a normalized varimax rotation, are presented in +Table 1. Only the highest loadings of each factor were used for interpretation. While some factor analysts will interpret loadings down to 0.35 or even 0.30, this does not seem reasonable in a pilot study that factors data from such a small sample. The highest loading items on the first factor related to confidence in withdrawing patients from addictive substances. Items reflecting familiarity with nonmedical approaches to treatment loaded most highly on the second factor (i.e. psychotherapies, structured living programs, and 12-step meetings). We entitled the first factor "Medical Approaches to Addiction Treatment," while the second factor was entitled "Non-Medical Approaches to Addiction Treatment."

Multivariate analysis of variance (MANOVA) was conducted using factor scores measuring the two dimensions of the new questionnaire. First, MANOVA showed a significant effect for years of residency, F (8, 30) = 3.60, p = 0.004. Univariate values indicated that the differences among years of residency were significant only for "Medical Approaches to Addiction Treatment Factor," F (4, 16) = 4.13, p = 0.017, with more advanced residents showing higher scores. Second, MANOVA showed a significant effect of participating in a specific training program for substance abuse, F (2, 16) = 6.94, p = 0.005. Univariate results indicated a significant difference for "Non-Medical Approaches to Addiction Treatment Factor," F (1, 19) = 10.98, p = 0.004, with residents who had completed a specific substance abuse rotation achieving higher scores.

Our results suggest that the ATS is related to the confidence and preparedness that residents express in their ability to treat substance abuse problems. Additionally, the ATS appears to represent two factors, "Medical Approaches to Addiction Treatment Factor," reflecting medical knowledge, and "Non-Medical Approaches To Addiction Treatment Factor," reflecting psychosocial interventions. Our results indicate adequate internal consistency and suggest that general medical knowledge and psychosocial knowledge, as measured by the two factors, are independent components of substance abuse treatment confidence. The validity of the questionnaire and the interpretation of the two factors are supported by the association between 1) the "Medical Approaches to Addiction Treatment Factor" and years of psychiatry residency training and 2) the "Non-Medical Approaches To Addiction Treatment Factor" and specific substance abuse training. Thus, the "Medical Approaches to Addiction Training Factor" appears to be heavily influenced by the extent of training that residents have had within the discipline of psychiatry, while the "Non-Medical Approaches to Addiction Treatment Factor" is most highly correlated with specific exposure to substance abuse training.

We recognize that the generalizability of these results is limited by the small sample size. The present study needs to be replicated with a larger number of subjects and in different areas of the country. The conclusion regarding the factor structure of the ATS is clearly preliminary. The small number of cases in this pilot study is inadequate to support assumptions of a stable factor structure and there is strong cross loading of items on the factors derived in this study. Factor methods used in the preset study are relatively simple and direct, chosen out of recognition of the limitations imposed by our small sample size. Replication with a larger sample size will allow application of more complex factor analytic techniques, including examination of the possibility that the factors identified may be correlated. We note that some items that intuitively belong with Factor I, medical approaches, do not show a clear association with that factor when compared with Factor II. It is possible that a larger sample size may also clarify the relationship of specific items to the factors and improve factor stability. Additionally, future studies should correlate the ATS with the individual subject's PRITE results to further validate this questionnaire. Unfortunately, since participants completed the questionnaires for this study anonymously, there was no mechanism for matching PRITE and ATS scores.

Despite these limitations, initial results are promising and somewhat remarkable in their strength, in view of the small sample size. The results suggest that using the ATS may be of benefit in evaluating residents' self-reported preparedness to treat substance abuse disorders.

Even though assessing residents' competence in treating clinical disorders should not be limited to simply asking them about their confidence (+7), self-report scales such as the ATS can provide valuable feedback about residents' view of their training. Some residents will inaccurately estimate their abilities to treat the disorders listed on the ATS, but even these data can provide important information to the training directors, especially if residents identify weakness in certain areas of substance abuse training (+8). We propose that the ATS be used to complement other strategies for evaluating the competency of residents in substance abuse treatment and not as a substitute for clinical supervision and appropriate examination of their knowledge base. Completing the ATS at the beginning of the rotation can identify areas where residents perceive themselves to be unprepared. This will help faculty and residents focus on specific areas of substance abuse training. Completing the ATS at the end of the rotation will measure the progress residents think they have made in addressing their perceived deficits in preparedness to treat substance abuse disorders. For residents to continue to identify areas of weakness suggests that extra work is needed to address deficiencies.

Our results show that the ATS could be beneficial in evaluating psychiatry residents' substance abuse training. To verify these findings, this study should be replicated with a larger sample size. If supported by future research, the ATS could be a useful self-report evaluation tool for psychiatry residents' substance abuse training. We would like to reinforce that the ATS is not a substitute for traditional methods of resident evaluation. However, using the ATS may aid faculty in developing an individualized addiction psychiatry training program for residents and provide a measure of progress toward addressing identified shortcomings.