The Accreditation Council on Graduate Medical Education (ACGME) recently mandated that all psychiatric residency programs demonstrate resident competency in five different forms of psychotherapy: brief, supportive, psychodynamic, cognitive-behavioral, and combined psychotherapy and psychopharmacology (1). Various conceptual issues, such as the definition of therapies, and practical issues such as how to determine competence, need to be addressed before responses can be made to these new requirements. For example, brief therapy, supportive therapy, psychodynamic therapy, and combined psychotherapy and psychopharmacology need to be clearly defined so that the specific skills in these therapies are differentiated from each other. Defining psychotherapies has been advanced by the development of manuals, many of which are available for different therapies and disorders.

Perhaps a more critical issue centers around the definition of competence. In psychotherapy, competence usually refers to the therapist's level of skill in delivering a particular treatment, whereas adherence refers to the extent to which techniques that are used in therapy match those described in a manual (2). Competence assumes adherence to specific techniques and a good working alliance. It is important to note, however, that rigid adherence to protocols that do not consider patient factors is negatively related to competence in many therapies (3). Competence in psychotherapy implies a good understanding of theory, the treatment outcome literature, and the maintenance of professional behaviors.

When considering residents, should competence refer to demonstration of beginning skills or to those of a master therapist? It is unlikely that residents will attain the latter, but many may achieve the former. Master therapists are proficient in the delivery of a particular therapy, whereas residents may attain varying degrees of competence. Defining an acceptable level of competence for a resident will be challenging.

On a practical level, how will we know when competence has been achieved? Therapy process notes discussed in supervision tend to accentuate positive aspects of therapy and minimize those that contribute to ruptures in the alliance (4). Assessment of competence should therefore be based on multiple sources and multiple methods. Supervisors need to be presented with session material they can see or at least hear, so that competence can be clearly assessed and feedback can be given. An "objective" evaluation is also essential whereby an unbiased evaluator assesses performance based on clearly defined criteria of competence. These "criteria" have been established by therapist competency rating scales for several different therapies (4,5,6,7). These instruments can be used in training institutions, keeping in mind that the standard expectation of a resident will certainly differ from that of a therapist following a research protocol. These scales however, do not take into account contextual variables, such as patient characteristics, therapist characteristics, presenting problem, and stage of therapy, all important when assessing therapists competence (3). Despite such difficulties, these scales provide us with some objective means to assess specific skills and can be used in conjunction with other more qualitative evaluation methods to assess overall competency in psychotherapy.

The purpose of this paper is to present a competency-based psychotherapy program in the hope that it may assist program directors to meet the RRC's new requirements. Preliminary evaluative data on the program will also be discussed.

In 1995 McMaster University implemented a competency-based empirically oriented psychotherapy training program that now trains residents in seven different forms of psychotherapy: client-centered (supportive), cognitive-behavioral (CBT), long-term psychodynamic, interpersonal (IPT), family, couple, and group therapies (8). These therapies were selected as they have been demonstrated to teach fundamental and specific psychotherapeutic skills, are effective in the treatment of psychiatric disorders, and provide learning experiences in long- and short-term interventions. Pharmacotherapy, if indicated, is integrated with all these therapies, with residents learning to manage both aspects of treatment. Residents are also trained in more than one form of therapy for the same clinical problem (e.g. both IPT and CBT for depression), given patient resistance (i.e., homework compliance in CBT), and therapist affinity toward certain forms of treatments.

It is not the teaching of multiple therapies that differentiates our program from others, but rather the standardized approach to training and attention given to the assessment of competence. With the exception of long-term psychodynamic therapy, all therapies are clearly defined and implemented through the use of manuals. All therapy sessions are audio taped or in some cases videotaped or screened and assessed weekly by the supervisor, who has been specifically trained in that module. Residents are required to purchase a tape recorder, while video equipment is available in most placements. Audio/videotaped material is played back under supervision, permitting supervisors to give moment-to-moment feedback to the resident. Therefore, competency assessment is based on ongoing evaluation of segments of resident's performance data, rather than solely on the resident's impression of the session. More importantly, for those therapies with established therapist rating scales, residents submit early and late session tapes for evaluation by an alternate supervisor, who blindly rates these tapes utilizing an empirically validated therapist competency rating scale. A minimum of two raters per module is required for this purpose. Given the expansion of the training requirements and the rigorous evaluation process, competency assessment using therapist rating scales is carried out for one case per module. Failure to meet the competency requirement of a module necessitates the resident's repeating the module. Skills learned in the "test case" are generalized to other cases throughout the resident's training, and for these cases competency is assessed by the supervisor. Many residents carry several cases throughout their training in different placements and have the opportunity to "specialize" in one or two areas in the senior or fellowship year.

It is important to mention that competency assessment is not solely based on performance on therapist rating scales, but it includes faculty evaluation of the resident's performance in supervision and seminars. Using our own evaluation forms, we assess a resident's general and specific psychotherapy skills, knowledge base, and professional behaviors (available from senior author). The relationship between performance on the therapist competency rating scales and our own evaluation forms have yet to be investigated.

Faculty is assessed for their supervisory and seminar presentation skills (knowledge base, facilitative skills, professional behaviors and attitudes, and use of learning resources). Module coordinators, supervisors, and evaluators are either full time or part time faculty comprised of psychiatrists, psychologists, and social workers. We initially began with two supervisors per module, except in the case of psychodynamic psychotherapy where we had a large group of supervisors. Supervision was initially offered in groups, and still is in some cases, and has been favorably received by the residents. Since the inception of the Program we have increased the complement of supervisors through in-house training and recruitment to supervise a complement of about 30 residents.

Over the past seven years, residents have consistently rated supervision as excellent in five out of seven modules and very good in the remaining two modules. In the seminars, the overall rating across all modules has been above average. The response rate in both cases has been about 90%. Of significance is that both supervision and seminar satisfaction ratings have improved over time. We would like to attribute this to the annual changes that are made as a result of our ongoing evaluation process and resident input.

Formal approval to carry out this review was not required by the McMaster University's Research Ethics Board, as the data collected were part of an ongoing evaluation of the Psychotherapy Training Program of the Department of Psychiatry and Behavioral Neurosciences.

1. Client-Centered Therapy (Supportive therapy)

Residents begin their training with Client-Centered therapy, a "supportive"-exploratory therapy that places heavy emphasis on the alliance and empathic attunement (9). Seminars introduce the resident to theoretical and research issues and model specific interventions such as accurate empathic reflections, active listening, and showing support. These interventions are illustrated through videotape presentations and role playing exercises. In this brief therapy, supervision is facilitated by established manuals (10), with therapy being offered to patients with mild to moderate depression and in cases where supportive interventions are used to supplement medication treatments. This treatment is not offered to patients with Axis II diagnoses, since it has not been investigated in this population. Microcounselling techniques, the moment-to-moment analysis of taped session material with feedback, are used to assist learners in the difficult task of delivering accurate empathic responses that facilitate deeper exploration (11).

Competence is assessed using our own evaluation forms (therapy specific skills-empathic attunement, support, alliance) and the Truax Accurate Empathy Scale (TAES). The TAES is a 9-point anchored rating scale that has shown good construct validity and adequate reliability (7). Supervisors uninvolved in the resident's training and blind to session order, rate three randomly selected 10-minute segments from an early (#3) and late (#15) tape, representing early, middle, and late periods in the session. These ratings are then averaged so as to arrive at a score of 1 to 9 for the early and late sessions. Competence is achieved with scores equal to or greater than 5. Preliminary analysis with 26 residents shows significant gain in competence from early (M = 4.38, SD = 2.07) to late sessions (M = 6.06, SD = 2.9), t(25) = 4.60, p = .0001. Luborsky's brief supportive-expressive therapy is an alternate therapy that may be used for this purpose but it integrates interpretive and supportive interventions (12).

2. Cognitive-Behavioral Therapy (CBT)-Depression

CBT for depression represents the next module, and is the first exposure to cognitive-behavioral strategies that residents receive. The seminars combine an academic and practical focus with selected readings, videotapes, and role playing exercises that model specific CBT skills. The final seminar teaches residents formulation and conceptualization based on the cognitive model and the interventions they have previously learned.

Trained supervisors use specific manuals to assist residents in treating depressed patients (13,14). Competence and adherence are assessed by the primary supervisor and a second independent rater. For this purpose, the Cognitive Therapy Scale (CTS) is used; this scale has been shown to have adequate psychometric properties for rating adherence and competence to CBT (5). This instrument assesses general therapeutic skills (e.g. setting an agenda, providing feedback), conceptualization, strategy, and techniques used (e.g. guided discovery, focusing on key cognitions and behaviors). When rated by the supervisor, the CTS is completed in a largely qualitative fashion. Ratings on items are shared and discussed with the resident, and there is no overall score. Use of the CTS in this way can point to particular areas of strength and also to areas where further work is needed. Additionally, residents are required to submit tapes from an early session and a late session for assessment by an independent rater who uses the CTS to derive a quantitative score. Scores of 30—39 (out of 66) on the CTS denote competency for novices. The information gathered from the supervisor and these independent ratings are collated to help ascertain resident competency and determine successful completion of the module. Preliminary data with respect to changes in resident performance from early to late sessions is being collected and will be reported in a future paper.

3. Cognitive-Behavioral Therapy-Anxiety

The CBT-anxiety module focuses on training residents in short-term, structured interventions for adults with anxiety disorders. Seminars provide an introduction to the assessment and CBT treatment for the full range of anxiety disorders and discuss the cognitive, exposure-based, and relaxation training strategies for a particular anxiety disorder. Expert supervisors use established manuals (15,16) to help residents deliver this brief treatment in a structured yet nonrigid manner.

Competence is assessed by the supervisor who listens to audiotapes on a weekly basis, and by an independent evaluator using the CTS (5). The assessment of resident competency focuses more on competence in cognitive therapy than in behavior therapy. For example, competency is rated based on taped sessions, which can almost never include in vivo exposure sessions conducted outside of the office. In addition, the CTS was designed to measure competence in cognitive therapy, rather than behavior therapy. Although the scale is fairly generic, there are no items that specifically rate competency at administering particular behavioral techniques (e.g., exposure, social skills training, relaxation training). However, for now we are using the CTS and developing our own instrument to assess therapist competence in conducting the behavioral component of CBT for anxiety disorders.

4. Psychodynamic Therapy

Following the CBT modules, residents begin their long-term psychodynamic case. The goal of this module is for residents to gain an understanding of the theoretical complexities and clinical techniques of psychodynamic psychotherapy through didactic lectures, small group discussions, readings, plus weekly supervision. Selected readings are assigned to assist residents in carrying out therapy with at least one case for 1—2 years of therapy. A minimum requirement however of completing at least 75% of one year of therapy (40 sessions) was instituted, as long as the supervisor deemed the resident had demonstrated reasonable competence with this case. Most residents continue much longer with their patients (over 2 years) and carry several patients throughout their training. However, at this time formal evaluation including the formulation submission is carried out on only one case. Subsequent cases are evaluated by supervisors but do not require other formal assessment procedures. Patients selected for treatment suffer from personality disorders, history of abuse and other traumas, loss, and interpersonal and identity problems. However, in the case of patients with borderline personality disorder, residents are encouraged to pursue training in dialectical behavior therapy offered in the group module.

Competence is assessed weekly by listening to audio taped session material. Joint assessments, whereby the resident assesses the patient for suitability for psychodynamic therapy in the presence of the supervisor, and screenings, where the supervisor sits behind a two-way mirror, also permit supervisors to assess resident competence. Residents are also required to submit a written psychodynamic formulation on their patient and their treatment process, and this is assessed by their supervisor. In the seminars, residents also complete a formulation of a videotaped case and submit this for evaluation. At present we do not have an objective method of assessing competency but are considering using measures established in brief dynamic therapy (12) such as the Penn-Adherence Scale for Supportive-Expressive Therapy (unpublished, 1988; obtained from author). It will be interesting to compare the effects of training in this module with the others, given the lack of adherence to a manual in this case. We hope to look at these issues in the future, once competency measures have been established in this module.

5. Family Therapy

The family therapy module occurs concurrently with the psychodynamic module and is integrated into the child psychiatry rotation. The major goal of this module is for residents to learn a method of family assessment, formulation, and treatment through seminars and case supervision utilizing the McMaster Model of Family Functioning (17). As in other modules, videotapes are used to model specific skills and test resident knowledge base. Expert supervisors screen or view weekly videotapes of resident sessions to assess competence and provide appropriate feedback.

Competence is also assessed by the supervisor and an objective evaluator who, rate early and late tapes using the Family Therapist Rating Scale (FTRS:6). This scale assesses global therapist behaviors from diverse theoretical orientations. There are five categories (10 items in each) of therapists behaviors that are rated on a 7-point Likert scale (0—6) for a maximum score of 300. The categories include: structuring, relationship, historical, structural/process and experiential behaviors. The FTRS demonstrates good psychometric properties, with good interrater reliability (0.61 to 0.87) and validity, with experienced family therapists scoring higher than inexperienced therapists on each of the five subscales (M = 162.4 vs. M = 93.2, p<0.001) (6). A study correlating the five subscales with expert ratings of therapists' effectiveness, found three of the five scales to be significantly correlated (6). A score of 100 indicates beginning skills in family therapy. Preliminary data is only available on five residents who showed gains in competence from early (M = 121.3, SD = 8.08) to late sessions (M = 161.9, SD = 28.2), t(4) = 3.12, p = .01. The small sample makes it difficult to interpret these results at this time. Hopefully, as our sample grows we can assess the utility of this scale in our Program.

6. Interpersonal Therapy

Interpersonal psychotherapy (IPT) is a brief therapy that utilizes specific techniques to address four interpersonal problem areas associated with the onset of depression: complicated bereavement, interpersonal role disputes, role transition, and interpersonal deficits. Training in IPT follows the psychodynamic module, considered essential for effective delivery of IPT. Seminars expose trainees to the underlying theoretical model of IPT and videotapes model specific techniques. Supervision is carried out with the use of well-established manuals (18) by supervisors who have received formal training in this area.

To ensure adherence to IPT early in supervision, the supervisor rates an early tape using the Therapy Strategy Rating Form (TSRF; 4, unpublished, obtained from author), and provides appropriate feedback to the resident. Competency assessment involves obtaining objective ratings on the TSRF and the Process Rating Form (PRF; Weissman, unpublished 1979), of early (#3) and late (#12) tapes. The TSRF assesses IPT specific interventions as well as nonspecific interventions associated with a positive therapeutic alliance. The PRF is a measure of the application of IPT techniques. Overall competence is determined for both the TSRF and PRF by a total mean score of 4 or less on a 7-point qualitative scale where 1 is excellent and 7 is poor. A score of 4 or less is deemed to indicate therapist competence in conducting IPT (4). TSRF interrater reliability is high with a Pearson r = 0.88 (p<0.001) (4). Unfortunately we have not yet been able to obtain sufficient pairs of tapes to look at progress over time; however, analysis of 12 residents' tapes at the end of therapy indicate residents have attained competence (M = 2.7, SD = 0.78). We are collecting more early/late tape pairs and plan to discuss these results in a future paper.

7. Couple and Group Therapy

Following successful completion of the training described above, residents select from either couple or group therapy modules. The couple therapy module is brief and is applied to distressed couples where one spouse is suffering from a psychiatric disorder. We have chosen emotionally-focused couples therapy (19). This modality is effective in couples with relational problems and integrates client-centered techniques learned in the first module with other systemic interventions. Supervision is carried out during live screening of couples so that the resident is able to implement specific interventions discussed in that setting during therapy. Although we do not presently utilize any specific competency measure in this module, we are considering utilizing established couple alliance measures as these are predictive of outcome.

Group therapy training opportunities include CBT groups (mood, anxiety and childhood behavioral disorders), IPT groups (mood disorders), groups for patients with borderline personality disorders, and experiential groups. Residents initially participate as cotherapists, and later as sole therapists in some placements, utilizing manuals to deliver treatment. At present we are investigating various competency measures that can be used in this module (20).

Since medications are integrated with all therapies, we have not developed a "formal" combined psychotherapy and psychopharmacology module. However, supervisors do attend to important issues related to integrating these two treatments, paying attention to attributions, transferences and other issues. Assessing competency in the delivery of an integrated treatment is even more difficult. One can use specific psychotherapy rating scales, as discussed above, and checklists that attend to the administration of medications. The latter should focus on educating patients about dosage, side effects, adverse effects, and compliance issues. Many pharmacotherapy research protocols address these issues. For example, in the Collaborative Study of Depression (21), at least 15 items of the Hollon scale (unpublished) are devoted to pharmacotherapy treatment issues. This scale can be obtained and used as a tool for assessing pharmacotherapy competence in specific sessions. A manual is also available for the integration of medications and "minimal" supportive therapy in the treatment of depression (22). What is needed, however, is a measure of competence that assesses a therapist's skill at "integrating" these two types of therapies, not simply the separate evaluation of each treatment component. Assessing resident competency to deliver both psychotherapy and medications is one of our most important challenges, for it is this modality that is truly unique to psychiatry.

Assessing competency in psychotherapy is a difficult and complex task. Supervisor evaluations and therapist rating scales by themselves do not provide all the answers. A skillful therapist must not only deliver the active ingredients dictated by manuals but also form effective alliances and tailor treatments to deal with unique patient characteristics. Integrating therapies is also essential in some cases, as many patients do not benefit from "pure" therapies alone. Integrative models that can assist residents with this process need to be included in training (23). At present, our residents demonstrate their integrative skills in couple, group, and family therapies.

In order to assist educators with the daunting task of developing new psychotherapy training curricula, we presented a competency-based psychotherapy program that can be used as a template, in that it is portable and has already been adopted at other institutions with minor modifications. Many issues, however, need to be considered before adopting such a program. Departmental support is essential to ensure that this program be mandatory. Minimally, funding is needed for a psychotherapy coordinator (2 days/week) and an administrative assistant (2 days/week). Obtaining adequate funding took considerable time, but with increasing recognition through awards, invited presentations, and publications, funding has improved. Full time medical faculty are funded through the department compensation plan. Non-MD, full-time faculty receive salaries from their clinical placements, while part-time faculty earn their department appointment through providing 100 hours/year of teaching time in any program.

Despite this support and infrastructure, considerable time and energy is required to maintain the Program. Obtaining adherence to utilization of tapes in sessions, and submission of tapes and other evaluation material is difficult. This was especially problematic in the early phases of the Program as philosophical differences concerning psychotherapy training had to be resolved. Recruitment and retention of skilled supervisors is also a problem. To deal with this, our faculty development program promotes in-house training and minimizes costs.

The major limitation of the program is that with a shift to broad-based training, less emphasis has been given to one type of therapy, thereby sacrificing proficiency in any one form. Proficiency in psychotherapy, however, takes years of practice over time. Our program has focused on establishing an adequate level of competence first, with the hope that subsequent experience will direct the resident toward specialization and later proficiency. It is also important to mention that the competency data presented in this paper was based on a within subjects design, which does not take into account improvements in general skills that develop over time. We are hoping to refine our evaluative methods as our program grows.

Although challenging, the RRC's mandate for training programs to demonstrate resident competency in psychotherapy is timely. Psychotherapy as a field has flourished, and extensive research has demonstrated the efficacy of numerous psychotherapies in the treatment of psychiatric disorders (24). From our perspective, the expectation that psychiatrists be competent in some of these therapies is reasonable and necessary, especially if they are to treat a wide range of patients with state-of-the-art treatment techniques (25). If we accept this professional mandate, then it is our obligation to ensure that trainees demonstrate some level of competency in psychotherapy before they graduate. Hopefully, this will encourage us to develop more reliable and valid measures of competence, to the benefit of our field and our patients.