Acad Psychiatry 33:67-70, January-February 2009
doi: 10.1176/appi.ap.33.1.67
© 2009 Academic Psychiatry
Insights about Psychotherapy Training and Curricular Sequencing: Portal of Discovery
K. Ramsey McGowen, Ph.D.,
Merry Noel Miller, M.D.,
Michael Floyd, Ed.D.,
Barney Miller, Ph.D. and
Brent Coyle, M.D.
Received June 1, 2007; revised August 3, 2007; accepted August 22, 2007. Drs. McGowen, M. Miller, and B. Miller are affiliated with the Department of Psychiatry and Behavioral Sciences at East Tennessee State University in Johnson City, Tenn.; Dr. Floyd is affiliated with the Department of Family Medicine at East Tennessee State University; Dr. Coyle is affiliated with Blount Memorial Hospital in Maryville, Tenn. Address correspondence to K. Ramsey McGowen, East Tennessee State University, Psychiatry and Behavioral Sciences, Box 70567, Johnson City, TN 37614; mcgowen{at}etsu.edu (e-mail).
|
ABSTRACT
|
OBJECTIVE: The authors discuss the curricular implications of a research project originally designed to evaluate the instructional strategy of using standardized patients in a psychotherapy training seminar. METHODS: The original project included second-year residents enrolled in an introductory psychotherapy seminar that employed sequential meetings with standardized patients. Residents were videotaped at baseline and at 6 week intervals; these sessions were rated by outside raters, standardized patients, and the residents themselves using two rating scales designed to assess psychotherapy skill. Results of the ratings were used to assess whether the instructional strategy was effective in teaching psychotherapy. RESULTS: Data were analyzed for group and individual effects. Results of unpaired t tests revealed that as a group resident performance did not improve. Individual effects were examined using regression analysis of individual learning plots. This analysis revealed that residents differed widely in their individual responses to this instructional technique. CONCLUSION: These results precipitated a realization about the curriculum. Despite initial disappointment about the apparent limitations of the technique, thoughtful analysis prompted a reinterpretation that led to residency curriculum modification.
|
INTRODUCTION
|
How to best teach psychotherapy is one of the most pressing issues in psychiatric education (1, 2). Although psychotherapy training has long been a core topic in psychiatric education, recent developments focusing on competency-based curricula have raised the bar considerably and resulted in a scramble to develop programs with demonstrated efficacy for the task (3–5). In an attempt to address this issue, we devised a course for psychiatric residents using standardized patients to teach psychotherapy skills (6). After using this instructional format for several years, we developed a research project to assess the effectiveness of this training approach. We wanted to learn whether a resident course employing sequential meetings with a standardized patient resulted in the acquisition of demonstrable competence in psychotherapy. We developed a research project to answer this question. Although the results obtained initially suggested one conclusion, exploration of the data resulted in an entirely different conclusion. This article describes the original research, data analysis, and the ultimate conclusions that took us far from the evaluation of one specific psychotherapy training course.
|
Methods
|
Participants were second-year psychiatric residents enrolled in a required psychotherapy seminar (three women and three men). The Institutional Review Board of East Tennessee State University approved the project. After complete description of the study, residents provided written informed consent. A baseline evaluation of therapy skills was obtained prior to the seminar by videotaping residents in a 30-minute interview/brief counseling session with a standardized patient. The seminar included initial didactic presentations on the principles of psychotherapy, followed by a series of weekly, 1-hour simulated therapy meetings for 18 weeks. Scripts for all the standardized patients (baseline and seminar) were structured to provide specific learning opportunities related to psychotherapy skills (e.g., identifying feelings, challenging irrational ideas), but were flexible enough to allow for logical development of the case. The 18-week course was broken into 6-week portions, which included three different resident dyads for three separate cases.
Two residents acted as cotherapists throughout a six-session treatment course with each standardized patient. After completion of the six-session simulated therapy, a different resident dyad began acting as the therapist in a different simulated therapy sequence with a different standardized patient. The residents who acted as therapists were observed by faculty and other resident participants, who offered feedback, posed questions, and elaborated on the processes and psychotherapeutic techniques being taught. At the end of each 6-week case, all residents were individually videotaped interacting with a standardized patient who was novel to the resident.
Instruments employed to evaluate residents’ simulated therapy and the seminar’s effectiveness included the Working Alliance Inventory, Bond scale (observer, client, and therapist formats) (6), the Common Ground rating scale (7), and an eight-item, individually administered follow-up questionnaire designed by the experimenters that examined residents’ perceptions of the seminar’s strengths and weaknesses. Evaluation of seminar effectiveness for teaching psychotherapy was based on ratings of resident interactions with a standardized patient by outside raters, the resident, and the standardized patient.
|
Results
|
Interrater reliability for the outside raters was confirmed for three of the four raters (t test, p>0.06); data from the fourth rater (who lacked a clinical background) were dropped from further analyses because they were significantly different from all other raters (p<0.02). Additionally, each of the clinician raters was significantly different on both scales from both the standardized patient ratings and the resident raters (p<0.02).
Data were analyzed for group and individual effects. Group results were examined by comparing ratings on the Bond and the Common Ground rating scales for each video segment across time (baseline and weeks 6, 12, and 18). Unpaired t tests were used to evaluate whether residents as a group demonstrated changes in therapy outcomes over the course of the seminar. No variable showed significant change.
Individual effects were explored by linear regression analysis of learning plots for each resident for the rating scales. The Common Ground scoring is expressed as the percent of maximum for that scale. Figure 1 shows the plots of "session number" against the ratings of the outside clinician raters (CLIN) score for both scales.
View larger version (36K):
[in this window]
[in a new window]
|
FIGURE 1. Individual Learning Curves Rated by Outside Clinicians (CLIN)
The rating score by the CLIN group for the Common Ground (panel A) and the Bond scales (Panel B) are shown plotted for each of the four training sessions. Regression (linear) analysis p-values are shown for each subject. WAI=Working Alliance Inventory
|
Residents differed widely in their response to this teaching technique. Resident E improved dramatically on both scale ratings, while Resident D was very poor at the onset of the course and worsened further. Residents A and F demonstrated no change in Bond scale ratings but improved in ratings on Common Ground. Resident B’s performance deteriorated slightly, more so on the Bond scale. Resident C began the seminar with high preexisting skills, which worsened over the course of the seminar, but ended the seminar with skills in the acceptable range. As previously mentioned, resident self-ratings and standardized patient ratings were all significantly different from those of the outside raters. These ratings were also plotted and examined through regression analysis. No clearly identifiable trends emerged from comparison between these curves or from inspection within them.
Residents were asked to evaluate the seminar at its conclusion. They reported very positive impressions, with an average numerical rating of 8.25 for usefulness and 8.75 for effectiveness on a 10-point scale.
|
Discussion
|
These results initially puzzled us. The course format was congruent with educational models that are rapidly infusing medical education (8–10). The format and sequence continued to strike us as logically consistent, with an optimal structure for acquiring psychotherapy skills. Finally, the residents’ feedback on the seminar effectiveness questionnaire indicated that they found the seminar both useful and effective. Nonetheless, the absence of clear improvement in the measures of the therapeutic relationship was not congruent with these impressions. What happened? Why did they not learn what we expected?
The search for an explanation to these questions led us to examine our original question from a different perspective. We looked closely at the data available from the instruments and discovered that what we had actually assessed was not the effectiveness of a specific instructional technique, but something more fundamental: communication and relationship building.
The Working Alliance Inventory Bond scale is a widely used psychotherapy research instrument (11). Items on this scale assess the patient’s confidence in the therapist’s ability to help, sense of comfort and understanding, and sense of working together effectively. Our choice to use it as a measure of psychotherapy outcome is supported by research findings documenting that therapy outcome is largely attributable to and correlated with the strength of the alliance (12). We assumed that if residents improved in psychotherapy skills, the improvements would be reflected in improved ratings on the Bond scale. Likewise, we expected the Common Ground rating items to document therapeutic success, because it assesses whether the resident encourages the patient to identify everything the patient wants to address, explores for determinants of the patient’s perceptions, and reaches a mutually agreeable treatment approach.
The wide variability in individual responses to the seminar suggests that many variables affected the outcome. Residents may have become preoccupied with the application of specific psychotherapy techniques and neglected more fundamental alliance-building skills, which undermined their ultimate performance. Such deterioration of basic communication and therapeutic alliance skills in the face of competing priorities has been documented in medical students (7, 13). The resident whose therapy performance was weakest (Resident D) had globally inadequate skills and left the program at the end of the academic term. Finally, factors such as a resident’s readiness to learn material possibly affected the performance of some residents.
An epiphany occurred after reviewing these results from the perspective of communication and therapeutic alliance skills. An assumption undergirding our seminar had been that, in learning psychotherapy, residents would automatically employ the necessary communication and therapeutic alliance skills. In essence, we assumed that acquiring psychotherapy skills subsumed relationship and communication skills. When we looked at the data available from our evaluation instruments, we found that these two issues (communication/relationship and specific psychotherapy skills) were, in fact, separate issues. The findings were entirely consistent with concerns repeatedly voiced by faculty about our residents’ abilities in these fundamental skills, but had not informed our curricular planning. We determined that what our results confirmed was a curriculum sequencing issue, not a course effectiveness issue. Our findings were still informative and helpful for evaluating the psychotherapy training program, but in a much more basic way. The data pushed us to acknowledge that we had placed residents in a course to acquire psychotherapy skills without confirming their readiness for such a course. It was indeed a portal to discovery. It led us to restructure the curriculum to include a first-year resident seminar, called "The Therapeutic Experience," which focuses on the communication and relationship skills necessary for conducting psychotherapy. The ability to reconceptualize our results into a more fundamental issue ultimately allowed us to use data to improve the curriculum, which has benefited all.
|
ACKNOWLEDGMENTS
|
The authors wish to acknowledge our research assistants Christine L. Newell, M.Ed., Ph.D., and Zachary A. Hammons, B.S., who provided invaluable assistance on this project.
At the time of submission, Drs. McGowen, M. Miller, Floyd, and B. Miller disclosed no competing interests. Dr. Coyle is on the Speakers’ Bureau for Wyeth and AstraZeneca.
|
REFERENCES
|
- Yager J, Mellman L, Rubin E, et al: The RRC mandate for residency programs to demonstrate psychodynamic psychotherapy competency among residents: a debate. Acad Psychiatry 2005; 29:339–349[Abstract/Free Full Text]
- Ravitz P, Silver I: Advances in psychotherapy education. Can J Psychiatry 2004; 49:230–237[Medline]
- ACGME: ACGME Common Program Requirements in BOLD: Program Requirements for Residency Training in Psychiatry. Available at http://www.acgme.org/acWebsite/downloads/RRC_progReq/400pr1104.pdf
- Mellman LA, Beresin E: Psychotherapy competencies: development and implementation. Acad Psychiatry 2003; 27:149–153[Abstract/Free Full Text]
- Beitman B, Yue D: Learning Psychotherapy, 2nd ed. New York, Norton, 2005
- Horvath AO, Greenberg LS: Development and validation of the working alliance inventory. J Couns Psychol 1989; 36:223–233[CrossRef]
- Lang F, Harvill L, McCord R, et al: Communication assessment using the Common Ground instrument: psychometric properties. Fam Med 2004; 36:189–198[Medline]
- Coyle B, Miller M, McGowen KR: Using standardized patients to teach and learn psychotherapy. Acad Med 1998; 73:591–592[Medline]
- Klamen DL, Yudkowsky R: Using standardized patients for formative feedback in an introduction to psychotherapy course. Acad Psychiatry 2002; 26:168–172[Abstract/Free Full Text]
- Swanson DB, Stillman PL: Use of standardized patients for teaching and assessing clinical skills. Eval Health Prof 1990; 13:79–103[Abstract/Free Full Text]
- Hovarth AO, Bedi RP: The alliance, in Psychotherapy Relationships That Work: Therapist Contributions and Responsiveness to Patients. Edited by Norcross JC. New York, Oxford University Press, 2002
- Lambert MJ, Barkey DE: Research summary on the therapeutic relationship and psychotherapy outcome, in Psychotherapy Relationships That Work: Therapist Contributions and Responsiveness to Patients. Edited by Norcross JC. New York, Oxford University Press, 2002
- Prislin MD, Giglio M, Lewis EM, et al: Assessing the acquisition of core clinical skills through the use of serial standardized patient assessments. Acad Med 2000; 75:480–483[Medline]
Get information about faster international access.
Privacy Policy
Copyright © 2009
Academic Psychiatry.
All rights reserved.
Home
| Search
| Current Issue
| Past Issues
| Subscribe
| All APPI Journals
| Help
| Contact Us
|
TOP
ABSTRACT
INTRODUCTION
