Advanced Search

REGULAR | September 01, 2009

Adolescent Depression: Evaluating Pediatric Residents’ Knowledge, Confidence, and Interpersonal Skills Using Standardized Patients

Colleen Lewy; C. Wayne Sells; Jennifer Gilhooly; Robert McKelvey

Academic Psychiatry 2009;33:389-393.

View Article Information

Received January 21, 2008; revised May 30, September 4, and November 26, 2008; accepted December 11, 2008. Drs. Lewy and McKelvey are affiliated with Psychiatry at Oregon Health & Sciences University in Portland, OR; Drs. Sells and Gilhooly are affiliated with Adolescent Health at Oregon Health & Science University. Address correspondence to Colleen Lewy, OR Health & Sciences University, Psychiatry, 3181 SW Sam Jackson Park Rd., GH249, Portland, OR 97239-3098; lewyc@ohsu.edu (e-mail).

Article

Tables

References

text A A A

Abstract

Objective: The authors aim to determine whether pediatric residents used DSM-IV criteria to diagnose major depressive disorder and how this related to residents’ confidence in diagnosis and treatment skills before and after clinical training with depressed adolescents. Methods: Pediatric residents evaluated two different standardized patients portraying major depressive disorder before and after learning about adolescent depression. Residents’ interactions with standardized patients were observed, and checklists containing DSM-IV criteria for major depressive disorder and other pertinent information, such as comorbidities, were completed for each interaction. After the encounter, residents completed a survey asking about the "patient’s" diagnosis and the residents’ confidence in their clinical skills. Results: Residents assessed significantly more checklist items in the second encounter with a standardized patient. Residents’ confidence in treatment was significantly higher after the rotation, while confidence in diagnostic skills was unchanged, remaining high. Even after the rotation, residents did not use all DSM-IV criteria for major depressive disorder in their diagnoses. Conclusion: Major depressive disorder is a common adolescent psychiatric disorder. Pediatricians must be equipped with appropriate interpersonal and diagnostic skills to detect this and other psychiatric disorders. Standardized patients represent one useful way to teach and assess these skills. This study suggests that residents’ interpersonal and diagnostic skills can improve with practice. Although resident scores improved, post-encounter checklists showed that residents were still not asking all the necessary questions for a DSM-IV diagnosis, concluding prematurely that the standardized patients had major depressive disorder before satisfying all diagnostic criteria. The majority did not consider other depressive conditions or comorbid disorders.

Figures in this Article

Suicide is the third leading cause of death in the United States for persons 15—24 years old. Indeed, from 2003 to 2004, for some age groups (e.g., females, 10—14 and 15—19 years old; males, 15—19 years old), the suicide rate has increased significantly (¹). In the 2005 national youth risk survey of high school students, almost 17% had considered suicide, 13% had made a plan, 8% had attempted suicide, and more than 2% had required medical intervention.

Adolescents who commit suicide typically meet the criteria for at least one psychiatric disorder, most commonly major depressive disorder (²). In a retrospective study of adolescents who committed suicide, Shaffer et al. (³) found that 90% had at least one DSM-III diagnosis, the most common being major depressive disorder, anxiety, and/or substance and alcohol abuse. Half had reported symptoms for at least 1 year.

The potential impact of undiagnosed or undertreated psychiatric conditions is significant. Children with untreated mental health problems frequently experience mental health issues as adults (⁴). Studies suggest that lack of treatment leads to school failure, social withdrawal, increased economic costs for treatment, and decreased quality of life (⁵).

Pediatricians are often the first physicians to assess children’s mental health problems (⁵). Despite this, resident education may be lacking. According to the majority of primary care course directors surveyed, education about suicide prevention is inadequate (⁶). In a survey of 280 randomly selected pediatricians, 90% responded that it was their responsibility to diagnose depression in children and adolescents. However, almost half did not feel confident in their ability to do so (⁷). Further, Gardner et al. (⁸) found that when clinicians did identify mental health problems, they only based their diagnoses on DSM criteria 23% of the time. Just half of psychotropic medication prescriptions were based on full DSM diagnostic criteria (⁸). Clearly, training pediatricians to recognize and treat common mental health disorders is a priority. At Oregon Health and Science University, pediatric residents have multiple exposures to children and adolescents with mental health problems. All first-year residents participate in the community-based adolescent and child health block rotation, where residents are integrated into a variety of community agencies’ daily activities. For example, residents participate in clinical care in a daycare center, schools, and a homeless clinic and work with child and adolescent psychiatrists. During their third year, residents are exposed to community-based services and lectures and are mentored by adolescent medicine specialists, child and adolescent psychiatrists, drug and alcohol counselors, and nurse practitioners. Additionally, residents work with homeless youth and patients with eating disorders.

Despite broad exposure to adolescents with mental health disorders, it is not clear how much pediatric residents learn or retain about accurate, criteria-based DSM diagnoses, comorbid disorders, and evidence-based treatments. In this study, we hypothesized that residents would correctly diagnose depression and that their use of DSM criteria would improve with training. We further posited that residents’ confidence would increase with training.

Methods

Pediatric residents were asked to assess a case of adolescent major depressive disorder prior to and immediately after one of their rotations (assessments were 4—6 weeks apart). Residents were also given a CD (after the pretest) about teen depression and suicide designed for this project. The CD content included the DSM-IV criteria for the diagnosis of major depressive disorder, dysthymic disorder, bipolar disorder, and medical causes of depression. Treatment options for depressed youth were reviewed, and printable materials, as well as self-quizzes, were also available. Finally, the CD included interactive case presentations with videos of a clinician interviewing a patient, illustrating evaluation and treatment issues.

We developed the clinical cases for this study using standardized patients, actors who follow scripted roles to portray the behavioral, emotional, and physical symptoms of people with particular illnesses. This permitted us to tailor each case to the mental and physical symptoms of interest. Further, it allowed for uniform presentation of the same case to multiple residents so that trends in resident approaches to diagnosis and treatment could be examined. Standardized patients have been used extensively at the majority of U.S. medical schools (⁹) and are now required as part of the United States Medical Licensing Examination, Step 2 (¹⁰). They also have been used successfully in other studies to assess physicians’ diagnostic skills with major depressive disorder (¹¹, ¹²).

Two cases were developed (Table 1) that were loosely based on cases found in the DSM-IV Case Book (¹³). Each case was designed to have major depressive disorder as the primary diagnosis but also included a comorbid mental illness (e.g., substance abuse), which is common in the adolescent population (¹⁴). Two standardized patients were hired. Both had prior acting experience and no history of depression or drug abuse. Each received approximately 12 hours of training by the first author (CL). Consistent affect was more problematic than role content. Only when the standardized patients delivered their lines consistently were they used with residents. Their performances with residents were also randomly screened, and they received ongoing feedback.

Over a period of 2 years, first-year residents who rotated through the community-based adolescent and child health rotation and third-year residents starting their adolescent health rotation were recruited (residents were not allowed to participate twice). All residents signed informed consent forms, and the study was approved by the university’s institutional review board. Residents who participated had one standardized patient encounter prior to beginning their rotation and the other at the conclusion. The case order was randomized.

Resident encounters with the standardized patients were videotaped, and the performance was scored based on a checklist developed by an adolescent medicine specialist and a child and adolescent psychiatrist (CWS, RM) to cover DSM-IV criteria for major depressive disorder and to screen for bipolar disorder, substance abuse, and suicidality. Residents were scored independently (CSL) and knew that the patients were standardized patients. At the end of the second case, residents completed a post-encounter survey assessing their confidence in their diagnosis and treatment recommendations. For the post-encounter survey, residents were asked to respond to individual items assessing their confidence in diagnosis and treatment on a 5-point Likert-type scale (1=no confidence, 5=extremely confident).

Results

Participants

A total of 34 pediatric residents (27 women) evaluated at least one of the standardized patients (27 residents evaluated both; the seven missed interviews were due to illness or scheduling conflicts). Of the residents who evaluated both standardized patients, 14 assessed Mary first (51.9%). There were no significant differences in checklist scores between residents who saw Mary before Shelly. Residents were either in their first or third year (76.5% in their first year, n=26). The majority of residents reported that the exercise was useful, while 69% reported that it was "very" or "extremely helpful."

Pre- Versus Postrotation Checklist Scores

Checklist scores (3=fairly confident, 5=extremely confident) were derived from the total number of "yes" or "conditional" items. Of those who interviewed both cases, significantly more items were completed postrotation. Residents’ confidence in treatment was significantly higher postrotation (prerotation: mean=20.41, SD=4.17; postrotation: mean=23.22, SD=3.61; t=3.14, df=26, p<0.04).

Based on the 34 postrotation surveys completed by the residents, confidence in treatment was significantly higher postrotation (prerotation: mean=3.43, SD=0.74; postrotation: mean=4.0, SD=0.72; t=3.44, df=27, p<0.00). Residents’ confidence in diagnosis was high both pre- and postrotation and did not change significantly (prerotation: 3.91, postrotation: 4.1).

Neither year of training nor gender accounted for a significant amount of the variance in how much residents’ checklist scores changed from pre- to postrotation (R2=0.02, F=0.27, df=27, p<0.77).

Diagnoses

The percentage of residents who diagnosed major depressive disorder only (versus major depressive disorder and a comorbid diagnosis such as substance abuse) decreased overall before compared with after the rotation (prerotation: 65.6%; postrotation: 58.6%; McNemar test of dependent proportions was nonsignificant, p=1.0).

Table 2 reviews the proportion of residents who asked questions concerning the criteria needed to make the diagnosis of major depressive disorder. In general, residents asked more questions postrotation. The only significant difference between pre- and postrotation questions was an increase in the proportion of residents who asked about the patient’s prior history of depression and alcohol use.

Discussion

The majority of pediatric residents correctly diagnosed major depressive disorder in the adolescent standardized patients. Of concern is that most residents did not use full DSM-IV criteria to make the diagnosis. Instead, residents seemed to rely on partial criteria, such as affect and duration of sadness. In addition, residents frequently failed to consider comorbidities such as drug or alcohol use. This supports findings from previous studies. For example, Carney et al. (¹¹), in a study of primary care physicians, found that although the majority diagnosed major depressive disorder in unannounced standardized patients, they did not cover all DSM criteria. Further, only a little over one-quarter of physicians asked about substance abuse.

In our study there were no significant changes pre- to postrotation in the percentage of residents who diagnosed only major depressive disorder compared with major depressive disorder and a comorbid disorder. It appears that very few considered alternative medical causes of depression or inquired about symptoms that would rule out other conditions with depression, such as bipolar disorder. Despite this, resident confidence levels in their abilities were high. This concerns us because it may indicate that residents are unaware of their deficits. Misdiagnosis and pharmacologic treatment of the depressive phase of bipolar disorder can result in treatment-induced mania (¹⁵). Further, important comorbidities, such as substance abuse or attention-deficit/hyperactivity disorder, that are not recognized but left untreated can diminish the effectiveness of treatment for major depressive disorder.

With training, resident checklist scores improved significantly. In particular, residents were more likely to ask about history of depression and alcohol use. One explanation for residents missing some major depressive disorder criteria is that many symptoms may have been so obvious that residents did not believe that they needed to ask about them. Interestingly, there were no significant differences in diagnostic performance between first- and third-year residents. This may be due to the small sample size. Alternatively, this could suggest that prior educational experiences did not impact diagnostic skills or have a sustained influence. Learman et al. (¹⁶) found that the effects of a depression education program led to improvements 3 months later for resident diagnosis of depression in adults represented by unannounced standardized patients. However, there do not appear to be any longer longitudinal studies (including the current study, which only covered a period of 6 weeks). Additional studies evaluating clinician behavior over longer periods of time are needed. It is likely that clinicians require repeated exposure to training and practice in diagnosing and treating adolescent psychiatric disorders. However, the most effective type of training is still unclear.

There are a number of limitations to the current study. Resident responses could have been biased by the artificiality of the test conditions (e.g., the standardized patients were not depressed adolescents and may have behaved in ways that were subtly different; there was a video camera in the room). However, if anything, the results would probably be biased toward residents asking more mental health questions than they would in practice because they knew they were being assessed. Given the incompleteness of their DSM-IV diagnostic criteria for major depressive disorder, one wonders if their diagnostic accuracy is even lower in "real life" situations. Another limitation is that the residents’ assessments were based on only two exams. Other studies have found that multiple exams are needed for satisfactory reliability levels (¹⁷).

It is important for future pediatricians to acquire the knowledge, skills, and confidence to identify common psychiatric disorders. It is clear from the present study that more needs to be done to help pediatricians improve their skills. Finding well-supervised clinical opportunities for primary care residents to develop their psychiatric diagnostic and therapeutic skills remains a challenge.

View Large | Download Slide(.ppt)
Add to My POL

TABLE 1. Cases Portrayed by Standardized Patients

View Large | Download Slide(.ppt)
Add to My POL

TABLE 2. Residents Who Asked About Specific Criteria for the Diagnosis of Major Depressive Disorder

Acknowledgments

At the time of submission, the authors declared no competing interests.

References

Lubell KM, Kegler SR, Crosby AE, et al: Suicide trends among youths and young adults aged 10—24 years, United States, 1990—2004. MMWR Morb Mortal Wkly Rep 2007; 56:905—908

Fleischmann A, Bertolote JM, Belfer M, et al: Completed suicide and psychiatric diagnoses in young people: a critical examination of the evidence. Am J Orthopsychiatry 2005; 75:676—683

Shaffer D, Gould MS, Fisher P, et al: Psychiatric diagnosis in child and adolescent suicide. Arch Gen Psychiatry 1996; 53:339—348

Kessler RC, Berglund P, Demler O, et al: Lifetime prevalence and age-of-onset distributions of DSM-IV disorders in the National Comorbidity Survey Replication. Arch Gen Psychiatry 2005; 62:593—602

Koppelman J: Children with mental disorders: making sense of their needs and the systems that help them. NHPF Issue Brief 2004; 4:1—24

Sudak D, Roy A, Sudak H, et al: Deficiencies in suicide training in primary care specialties: a survey of training directors. Acad Psychiatry 2007; 31:345—349

Olson AL, Kelleher KJ, Kemper KJ, et al: Primary care pediatricians’ roles and perceived responsibilities in the identification and management of depression in children and adolescents. Ambul Pediatr 2001; 1:91—98

Gardner W, Kelleher KJ, Pajer KA, et al: Primary care clinicians’ use of standardized psychiatric diagnoses. Child Care Health Dev 2004; 30:401—412

Barzansky B, Etzel SI: Educational programs in US medical schools, 2003—2004. JAMA 2004; 292:1025—1031

United States Medical Licensing Examining: USMLE Bulletin Examination Content, 2007. Available at www.usmle.org/Examinations/step2/step2cs_content.html

Carney PA, Dietrich AJ, Eliassen MS, et al: Recognizing and managing depression in primary care: a standardized patient study. J Fam Pract 1999; 48:965—972

Gerrity M, Cole SA, Dietrich AJ, et al: Improving the recognition and management of depression: is there a role for physician education? J Fam Pract 1999; 48:949—957

Spitzer RL, Gibbon M, Skodol AE, et al: DSM-IV Case Book. Washington, DC, American Psychiatric Press, 1994

Armstrong TD, Costello EJ: Community studies on adolescent substance use, abuse, or dependence and psychiatric comorbidity. J Consult Clin Psychol 2002; 70:1224—1239

Perlis RH: Misdiagnosis of bipolar disorder. Am J Manag Care 2005; 11:S271—274

Learman LA, Gerrity MS, Field DR, et al: Effects of a depression education program on residents’ knowledge, attitudes, and clinical skills. Obstet Gynecol 2003; 101:167—174

Colliver J, Swartz M: Assessing clinical performance with standardized patients. JAMA 1997; 278:790—791