Psychiatric interviewing is a complex and demanding activity. It includes a variety of separate skills, including establishing and maintaining rapport, good listening and observing, communicating empathy, and sensitive structuring of the interview. It also requires using open-ended and closed-ended questions effectively, maintaining a professional stance toward the patient, and eliciting the necessary information to establish a diagnosis and formulate a treatment plan (1—4).
Although most would agree that skillful interviewing involves a complex set of abilities, few studies have addressed how individuals learn to become skilled interviewers or how well existing training models work in building these skills. A review of the existing literature, however, suggests that there is significant room for improvement in how this important skill is taught (5,6).
The work of cognitive psychologists provides insights into how to better train students in the mental health professions to become competent interviewers. The problem of inert knowledge is particularly relevant (7—10). Bransford et al. (10) describe inert knowledge as information an individual has learned, but tends not to apply when it is relevant. To illustrate, Bereiter (11) describes a teacher who gave her students a long, difficult article and told them they had 10 minutes to learn as much from it as they could. Almost without exception, the students began with the first sentence and read straight through as far as they could until time was up. When strategies were later discussed, many students recalled that they had knowledge of better ways to learn material. These included searching for main ideas, reviewing headings, and formulating questions and then searching for answers. But these students did not spontaneously use this knowledge when it would have helped them. Other studies show that individuals often do not apply problem-solving strategies that they have learned, unless explicitly prompted to do so (12,13).
Sherwood et al. (14) showed that one can facilitate students' spontaneous access to successful problem-solving strategies by creating problem-oriented instruction (POI) environments, rather than merely presenting factual information. They emphasize the importance of enhancing the similarity between the problem-solving environments used during training and those that will be necessary at the time of transfer to a new problem. Specifically, students who learn interviewing skills in a problem-solving context may transfer that learning more readily to actual patients in clinical settings. We examined this possibility in the current study.
Problem-Oriented vs. Problem-Based Learning
Since the 1960s, problem-based learning (PBL) has gained considerable influence in United States medical schools. PBL is an instructional method that uses patient problems as a context for students to learn problem-solving skills and acquire knowledge about the basic and clinical sciences (15). Typically, students work in groups and are presented with critical signs and symptoms of a hypothetical patient. The students' task is to discuss these problems, research relevant references, and produce a tentative explanation for the phenomena, describing each in terms of some underlying process, principle, or mechanism. Students may also attempt to solve the problem and suggest ways to diagnose and treat the case (16).
Although similar, POI differs from PBL in important ways. First, POI is based primarily on simulation and role-play activity, and it has more immediacy in its assessment. Second, in POI there is no research assignment. Third, POI emphasizes the affective and experiential recall of the learner. And fourth, participation in POI is continuous; the process of interaction is monitored in real time with emphasis on affective and subjective evaluation.
Two hypotheses were tested. First, medical students trained in psychiatric interviewing with problem-oriented instruction were expected to report being better prepared for their interview than students receiving the same content but in a lecture format, or than those receiving no formal instruction. Second, we expected that students receiving problem-oriented instruction about psychiatric interviewing would conduct better interviews than students who receive the same information in a traditional lecture format or who conducted their interviews prior to attending a lecture on interviewing skills.
For several years, students enrolled in the Behavioral Sciences course were required to interview a psychiatric patient under supervision. The current study was integrated into that exercise. We randomly assigned second-year medical students enrolled in the course to one of three interview training conditions prior to their interview: problem-oriented instruction, lecture, or syllabus materials only. The students subsequently conducted a supervised, 20- to 30-minute interview of a psychiatric patient. Interviews were observed and evaluated by experienced clinicians.
Medical Student Interviewers
Second-year medical students enrolled in the Behavioral Sciences course at the University of Louisville School of Medicine for the 1996-1997 academic year were invited to participate in the study. Of the 136 enrolled students who completed the class, 104 agreed to participate and were randomized to one of the three instruction conditions. Of these, 27 were excluded from the analyses for one or more of the following reasons: more than one year of mental health experience that included interviewing patients (n=4); student was assigned to interview a child or adolescent during the course (n=8); or the student's interview supervisor did not return sufficient data (n=15).
Of the 77 students included in the analyses, 27 were in the POI group, 27 in the lecture group, and 23 in the self-study group. The mean age of the students was 24.5 years (SD=3.0). There were 41 males and 36 females.
Thirty-eight psychiatrists, five clinical psychologists, five social workers, two doctoral pastoral counselors, and one art therapist supervised the interviews. Of the 38 psychiatrists, 12 were psychiatry residents in their first (n=1), second (n=2), third (n=4), or fourth (n=5) year. The most frequently cited primary practice settings of the supervisors were a university outpatient psychiatry clinic (n=18) and a hospital setting (n=13). Most supervisors described themselves as experienced in treating adults in psychotherapy. On a nine-point scale (1=very little experience, 9=highly experienced) the mean rating was 6.8 (SD=2.34). The mean number of years doing psychotherapy was 11.4 (SD=9.2), indicating a wide range of years of experience. The mean number of years providing psychotherapy supervision was 7.5 (SD=8.3), also indicating a wide range of experience. The supervisors were blind to the student learning condition. Of the 51 supervisors, 29 attended a one-hour training session in which they viewed interviews and practiced scoring the questionnaires.
Interviewed patients were drawn from multiple sites in the Louisville community where the supervisors work. These locations included the University of Louisville Hospital and Outpatient Psychiatry Clinic and other inpatient and outpatient settings in private, university-based, and community-based facilities. For the 77 patient interviews included in the analyses, the mean patient age was 42.2 years (SD=14.1); the youngest was 20 years old and the oldest was 77. There were 39 males and 38 females. The mean Global Assessment of Functioning (GAF) scale rating was 48.8 (SD=16.6), indicating serious symptoms or serious impairment in social, occupational, or school functioning (17). The most frequent Axis I diagnoses were major depression (n=26), bipolar disorder (n=10), and schizophrenia (n=10). In addition, 4 patients each were diagnosed with schizoaffective disorder, organic mood disorder, adjustment disorder, or substance abuse or dependence, and 3 patients each were diagnosed with psychotic disorder not otherwise specified or obsessive-compulsive disorder. The following diagnoses occurred in no more than 2 patients: depressive disorder not otherwise specified, anxiety disorder, hypomania, attention deficit disorder, dementia, and delirium. One patient received no Axis I diagnosis. Seventeen of the 77 patients received personality disorder diagnoses: borderline (n=5), personality disorder not otherwise specified (n=5), histrionic (n=2), antisocial (n=2), avoidant (n=2), and dependent (n=1). One patient was diagnosed with mental retardation and another with borderline intellectual functioning.
All sophomore medical students were invited to participate in the study, which was described as an attempt to improve understanding of how to train medical students in interviewing. Students who agreed to participate signed a consent form and were randomly assigned to one of three groups: problem-oriented instruction, lecture-only control, or self-study control. The time allotted for both problem-oriented instruction and the lectures was three hours.
We conducted two sessions of POI: a 2-hour session followed 2 weeks later with a 1-hour session. The principal activity in each session involved students role-playing interview situations. These situations were hierarchically structured beginning with basic interviewing skills and building toward more complex skills. The sequence was based on Ivey's interviewer-training model (2), which is highly compatible with problem-oriented instruction. One of the authors (G.D.S.) gave a 10-minute didactic instruction on Attending Behaviors (e.g., eye contact, body language, vocal qualities, verbal tracking) to the 36 students who attended the first session. The students then engaged in the first round of role-play. A second round of role-play was similarly preceded by a brief didactic presentation about Asking Questions (e.g., open-ended, closed-ended, asking for specifics, sustaining a line of inquiry). Following a short break, there was a brief introduction to Reflecting Feelings and Paraphrasing and Summarizing, followed by the third round of role-play. Following each round of role-play there was an opportunity for students to ask questions and reflect briefly on their experiences.
For each round of role-playing, the students were arranged in groups of three. Each round consisted of a series of three 3-minute vignettes in which each student shifted among three roles: "patient," "interviewer," and "observer." Each "interviewer" was expected to practice the specific skills emphasized in each round, but each was given a different topic or line of inquiry with which to question the "patient." Within each role-play, the "patient's" instructions varied: they were instructed to be more or less cooperative in response to the "interviewer's" behavior. For each role-play, one of the students served as an observer and was expected to provide feedback to the other two and in the process to heighten his or her own awareness of interviewer behavior. Faculty supervisors moved among the student trios, observing, listening, and making comments when appropriate.
The second POI session was attended by 32 of the original 36 students. (Only 27 of these students' data were included in the analysis, as explained above.) After a brief review of the interview elements that had been taught and practiced previously, the students again broke into groups of three where they engaged in a longer (7-minute) role-play series. Once again each had an opportunity to play each of the three roles, but this time we structured greater emotional and interpersonal complexity into the "patient" and "interviewer/doctor" roles. In one role-play, for example, the "patient" had waited two and a half hours for an appointment to discuss a painful lump in his or her neck; the "interviewer/doctor" in this vignette was told that he or she was running late because another patient had died in the hospital during rounds that morning and the doctor took time to talk with distraught out-of-town relatives. Students in the POI group conducted their actual patient interviews 1 to 3 weeks after the second POI session.
Lecture-Only Control Group:
These students listened to 3 hours of lecture material on psychiatric interviewing. The lectures reviewed the same topics and concepts presented to the POI group, but with further elaboration and examples. Students in the lecture-only group conducted their patient interviews 1 to 3 weeks after the lectures.
Self-Study Control Group:
These students conducted their patient interviews prior to attending the class lectures on interviewing. Detailed information about interviewing skills was available for their review and study in the course syllabus. This information was available to all students.
Interview Evaluation Questionnaire (IEQ)
The IEQ contains 18 statements that address four basic categories of interviewing activity identified by Ivey (2). These categories are 1) focus on nonverbal aspects of interview ("Nonverbal Focus"), 2) use of questions/communication skills ("Use of Questions"), 3) use of techniques that encourage patient to continue productive talk ("Facilitate Productive Talk"), and 4) exploring the patient's feelings ("Explore Feelings"). Specific item content is indicated in t1. The IEQ also contains an item rating the overall quality of the interview ("Overall Quality"). Students and supervisors rated the IEQ items on a scale ranging from 1 (below my expectations) to 10 (truly exceptional.) A category for "does not apply" was also included. Ratings were made immediately after the interview was completed.
A source of unwanted variability is that attributable to the difficulty of interviewing an individual patient. We anticipated that patients with certain traits or conditions (e.g., personality disorders, poor verbal skills, little psychological mindedness) would be more difficult to interview than others. Supervisors provided a DSM-IV diagnosis and rated the patient's overall level of functioning using the DSM-IV Global Assessment of Functioning scale (17). Supervisors also rated how difficult the patient was to interview, using a nine-point scale (1=much easier than most psychiatric patients, 9=much more difficult than most psychiatric patients).
After conducting their interview, students rated their preparation by answering the question, "How well did your training experience prepare you for this interview?" (1=not well, 7=quite well).
The first hypothesis we tested was that students trained with POI would feel better prepared for their psychiatric interview than would the lecture-trained students. (The self-study group was dropped from this analysis because they did not have a training experience.) An independent group t-test indicated that the POI students rated their training experience as providing better preparation (mean=5.2), compared with the lecture-trained students (mean=3.6; t=3.61, df=52, P<0.001).
The second hypothesis was that POI-trained students would conduct better interviews than students in the other two instructional conditions. For this hypothesis, we analyzed student and supervisor evaluations separately.
Our first step in evaluating the second hypothesis was to assess how much the supervisors' GAF ratings and ratings of the difficulty of the interview were associated with the four major IEQ interview categories and the overall interview quality rating. Our purpose for including these covariates was to statistically control for patient variability.
The first covariate, supervisor ratings of the difficulty of the patient to be interviewed ("HowDiff"), significantly predicted ratings on four of the five IEQ dependent variables, as rated by the students: Overall Quality (F=7.00, df=1,73, P<0.01), Nonverbal Focus (F=4.39, df=1,73, P=0.04), Use of Questions (F=8.68, df=1,73, P=0.004), and Explore Feelings (F=5.01, df=1,73, P=0.03). The dependent variable, Facilitate Productive Talk, was not significantly predicted by student ratings of the difficulty of the interview (F=0.84, df=1,73, P=0.36). Only Explore Feelings was significantly predicted by the second covariate, GAF (F=9.42, df=1,73, P<0.01). For this reason, GAF as a covariate was dropped from the analyses.
After adjusting for differences in group means based on the covariate HowDiff, we conducted two sets of planned comparisons. First, we compared the mean of the POI-trained group on each of the IEQ dependent variables with the combined means of the two control groups as well as with the Lecture group alone. As shown in t2, the POI-trained students rated themselves as performing better interviews as measured by all dependent variables except Facilitate Productive Talk, and, when the POI group is compared with the Lecture group alone, Explore Feelings.
To further investigate which interview skills differentiated the groups, we analyzed the specific IEQ items. t1 shows that the IEQ items at the simpler levels of Ivey's structure (Nonverbal Focus and Use of Questions/Communication Skills) tended to differentiate the POI and Lecture students better than did the items measuring higher-level skills (e.g., Facilitate Productive Talk.) The skills reflected in these lower-level items were also practiced more by the POI-trained students across the role-plays.
The second set of planned analyses compared the supervisors' ratings of the POI-trained students with the other two groups. No significant differences were found on any supervisor IEQ variables. Further, neither covariate (HowDiff or GAF) significantly predicted IEQ variables.
Our results support the first hypothesis but provide little support for the second. Students given problem-oriented instruction (POI) felt better prepared to conduct psychiatric interviews than students taught by traditional classroom lecture or those given no training beyond a detailed written syllabus. POI-trained students also rated their own interview performance more highly than did students in the other groups. This higher rating of their own performance applied to the overall assessment of quality as well as to the nonverbal aspects of performance, use of questions, and exploration of feelings. The interview supervisors did not evaluate the interviews of the POI-trained students as superior to those of the other students.
The discrepancy between the student and supervisor ratings raises the question of whether real differences existed between the interviews conducted by the POI-trained students and the other students. One possibility is that the differences in the student ratings were due to demand characteristics. Specifically, the students may have evaluated themselves in a manner intended to confirm what they conjectured were our hypotheses. Based on this reasoning, the supervisor ratings, because they were blind to the instruction condition of the student, may be the better measure. In addition, the greater interviewing experience of the supervisors may qualify their judgments as more valid than those of the less experienced students.
Despite these possibilities, we do not believe one should discount the student ratings entirely. The differences in the two sets of ratings may reflect different frames of reference. The students were trained to focus on a relatively narrow set of very basic interviewing techniques. On the other hand, the supervisors did not receive explicit training in the Ivey method and may have based their judgments on a much broader set of interviewing skills. Therefore, what appeared to be significant to the students may have appeared slight to more experienced supervisors. Given the same frame of reference on interviewing skills, the supervisors might have rated the interviews in the same direction as the students.
Methodological considerations also limit the reliability of the supervisor ratings. This was an empirical trial conducted in the medical school curriculum, not a funded research project. The large number of supervisors (51 as compared to 77 students) introduced rating variability due to factors other than the specific skills we intended to measure. A poll of a subgroup of supervisors showed that some ratings were motivated by factors other than those focused on in this study, such as providing a success experience for the student. Further, the performance expectations of the supervisors may have varied more than those of the students. Many of the clinicians had served as supervisors in previous years when their task and rating forms were somewhat different. Indeed, two supervisors commented that they rated a student lower because that student either did not conduct a mental status evaluation or a review of systems, neither of which was intended to be part of this interviewing exercise. Statistical support for this argument is provided by a review of the mean square error (MSE) terms for the univariate analyses of the major IEQ categories as rated by students and by supervisors. In all but one case, the MSEs for the supervisor ratings were greater than those for the student ratings, indicating greater variability among the ratings.
Although we have attempted to bring elements of good research design to this study of an educational intervention (e.g., randomized assignment to POI versus control conditions), a number of methodological improvements should occur if this intervention is the focus of further study. These include training all of the supervisors in the use of the IEQ and establishing their reliability with that instrument; ensuring that all supervisors return their ratings; limiting the number of supervisors as well as the range of professional and supervisorial experience; and seeking a less heterogeneous cohort of patients for the student interviews. (We did not have a chance to do this ourselves because of a school-mandated curriculum change the year after our study that eliminated the Behavioral Science course. Interviewing is now taught in a multidisciplinary Introduction to Medical Practice course headed by the Department of Family Medicine.)
Nevertheless, there are implications from this study for the education of clinician interviewers. Problem-oriented instruction is a more active form of learning and results in greater subjective sense of mastery. On this basis our results suggest that POI should be considered over traditional classroom approaches to teaching interviewing. Because POI is a more labor-intensive form of teaching, the lack of faculty corroboration for the POI-trained students' appraisal of their performance may temper this recommendation. However, from the student ratings of performance it also appears that POI facilitated the transfer of information and skills from classroom to practical application in a manner that suggests less inert knowledge than with traditional lecture format. Finally, interviewing is a complex skill in which years of experience are often required to achieve high levels of competence. That a three-hour intervention such as POI produced the results it did is promising, but future studies should explore the impact of longer instructional interventions for both basic and more advanced interviewing skills.