Academic Psychiatry 27:12-18, March 2003
© 2003 Academic Psychiatry
Complications Associated With Surveying Medical Student Depression
The Importance of Anonymity
Ruth E. Levine, M.D.,
Carmen Radecki Breitkopf, Ph.D.,
Frederick S. Sierles, M.D. and
Gwendie Camp, Ph.D.
Drs. Levine, Breitkopf, and Camp are affiliated with the University of Texas Medical Branch, Galveston, TX. Dr. Sierles is affiliated with FUHS/The Chicago Medical School. Address correspondence to Dr. Levine, The University of Texas Medical Branch, Department of Psychiatry and Behavioral Sciences, Rebecca Sealy Room 6.220, Galveston, TX 77555-0193. E-mail: rlevine{at}utmb.edu
|
ABSTRACT
|
OBJECTIVE: To evaluate students' attitudes and concerns regarding potential repercussions of completing the Beck Depression Inventory (BDI). METHODS: A survey based on focus group data was developed and distributed. RESULTS: One hundred and ninety-one of 400 surveys (48%) were returned. Of 160 students who remembered completing the BDI, 31 (19%) admitted to concerns about the research, and nearly 10% admitted to recording dishonest answers. CONCLUSIONS: These findings emphasize the importance of conducting anonymous assessments of medical students, particularly when assessing sensitive psychological states.
Key Words: Medical Students • Survey Methods • Confidentiality
|
Editorial Note
|
In this issue of Academic Psychiatry, we have given special attention to methodological issues relevant to educational and mental health research. In the first paper that follows, Ruth Levine and colleagues describe the importance of anonymity in survey studies that inquire about the personal mental health issues of medical students. This work is innovative and empirically derived, making it an unusually valuable contribution to the psychiatric education literature. In his companion commentary, Michael Myers of the University of British Columbia reminds us of the genuine humanity that we each bring to the practice of medicine. Together this paper and commentary underscore the imperative to pursue research on important but neglected medical student health care issues. They further demonstrate the need to do such work with sensitivity and with an awareness of intersecting ethical and scientific issues. The second paper, by Judith Meinert and colleagues, focuses on the recruitment of African-American women as subjects in mental health research. Although it is a nontraditional topic for our journal, the reviewers enthusiastically endorsed the importance of this paper for our readers who wish to engage in educational research efforts with students and residents of culturally and ethnically distinct backgrounds. The companion commentary by Lauren Bonner of the University of Washington gives emphasis to shared history and emerging collaborative opportunities in human research. Attention by academic psychiatrists to the considerations presented in this set of manuscripts may help ensure that educational research in our field is respectful, attuned, and methodologically rigorous. —Ed.
|
Introduction
|
Depression among medical students is a serious problem. There is considerable evidence that rates of depression and suicide are higher in medical students than in students involved in other graduate studies and that these rates continue to remain elevated when these students become physicians (1). A recent review reported higher rates of suicide in physicians compared with rates in both the general population and other professionals (2). Several studies have also documented high rates of depression in medical students compared with the general population. For instance, Clark and Zedlow (3) reported that at least 12% of medical students evidenced significant symptoms of depression on the Beck Depression Inventory (BDI) at three measuring points during their first three years compared with a rate of 3% to 4% observed among the general population (4). Similarly, Zoccolillo et al. (5) using clinical interviews of students, found a 12% prevalence of major depression in the first two years of medical school. Camp et al. (6), gathering self-assessments of depressed mood on the Zung inventory, reported scores associated with mild to moderate depression in 20% of medical students in a traditional curriculum compared with 10% of students in a problem-based curriculum. Wolf et al. (7) reported significant depression based on BDI scores in 18% of a medical school class at the end of their first year.
In 1998 we began research that focused on comparing depressed mood in medical students from different curricula (traditional, hybrid, and problem based) at the University of Texas Medical Branch in Galveston. Previous studies have demonstrated lower rates of depression (6) and higher rates of emotional and intellectual satisfaction (8) in a problem-based curriculum compared with a traditional curriculum. Our own problem-based curriculum subjectively appeared to provide greater social support from peers and faculty and a less competitive and more caring environment. In order to objectively assess these observations, we decided to test the hypothesis that rates of depression might be lower in our problem-based or hybrid curriculum compared with the rates in our outgoing traditional curriculum. The BDI was administered with a number of other measures upon matriculation, at the end of year 1, at the end of year 2, and during year 4. The students were given a separate consent form attached to the BDI. This document informed students that if they acknowledged suicidal ideation or scored in a clinically significant range, they would be contacted by the Dean of Students.
The initial administration of the BDI to the Class of 2001, at the end of the first year, yielded a lower than expected rate of abnormal scores. Only 2% to 3% of students scored 14 or higher, in contrast to 12% to 25% of students in published research in which the BDI had been administered at similar intervals (3). In response to this discrepancy, we modified the consent form. Specifically, we removed the name of the Dean of Students, although we continued to inform students that they would be contacted "to provide guidance regarding appropriate services" if they had significantly elevated scores.
Our consent form modification had little effect. Arguments arose in our research group between those who felt we had an obligation to act if we identified a depressed student and others who believed that telling the students that they would be contacted was interfering with the collection of valid data. We modified our consent form further, emphasizing that scores would not be a part of a student's academic record. Again, little difference was found in the students' low scores on the BDI. Because of our doubts over the results, we did not submit these data for publication.
The purpose of this follow-up study was to gain an understanding of the nature of the unexpectedly low observed scores and to evaluate the hypothesis that student responses were adversely influenced by concerns over anonymity and potential negative repercussions associated with a high score on the BDI. Specifically, this research was designed to discern students' concerns about the BDI administration, determine whether students intentionally distorted their responses, and identify the reasons for dishonesty among those who reported distortion.
|
METHODS
|
Preliminary Study
In the fall of 1999, two focus group sessions with approximately 15 students per group were conducted for the Class of 2001. Students were recruited to participate in the focus groups while on their psychiatry rotation and were offered a small number of extra-credit points to participate. The groups were conducted by faculty who had no relationship to the psychiatry department and whose impressions had no impact on the students' psychiatry grades. Minutes of the groups were taken by psychiatry staff members. The purpose of these sessions was to elicit students' perceptions of the depression assessment and the different consent forms and to discern reasons for dishonesty in filling out the assessment.
The students were asked their opinions about the study, whether any of them believed scores had been underreported, and if so, why this might have happened. Several themes emerged from these groups. Students were skeptical about our claims of confidentiality, particularly since their packets were numbered in order to correlate measures over time. Students described conflicted feelings: they were afraid to fill out the measure honestly and afraid not to comply with our request. Students reported that they did not want to be contacted if they had a high score because they wanted to choose for themselves how or when to seek help.
Primary Study
Based on the focus group responses, a brief, anonymous survey was developed to examine frankness and honesty among medical students completing the BDI. Survey questions addressed students' recall of, and opinions and feelings about, filling out the BDI. Specifically, the items assessed whether students could recall completing the BDI, whether they remembered the consent forms, and whether they were concerned about negative consequences of 1) being fully frank and honest while completing the BDI or 2) skipping the BDI and turning it in blank. Responses to these questions were in a yes/no format. A copy of the BDI and the consent forms was attached to the survey to facilitate recall. Overall frankness and honesty were assessed by using the question "Were you completely frank and honest when filling out the questionnaire?", to which students could answer yes or no. Students who admitted to dishonesty were offered a checklist of four reasons (derived from the focus group sessions) as possible explanations and were asked to mark all that applied to them. The choices offered were 1) the information was private but I felt compelled to fill out something; 2) I feared the information would become part of my academic record; 3) I was concerned that the information might stigmatize me; and 4) I did not want to complete the questionnaire next to my peers because this was a personal matter.
Initially, the survey was distributed to students via campus mail. A second distribution was conducted by a psychiatry staff member during sessions in which the students were either filling out psychiatry clerkship evaluations (Class of 2002) or being oriented to their fourth-year surgery rotation (Class of 2001). No members of the faculty were present during the administration or collection of the surveys. Students were specifically instructed to exclude names and identifying information and to return completed surveys to the staff member present in the room.
Using the Statistical Package for the Social Sciences (SPSS), we evaluated group differences by using chi-square or Fisher's exact test as appropriate. A two-sided significance level of 0.05 was used to determine the statistical significance of observed differences.
|
RESULTS
|
A total of 191 students out of approximately 400 returned the questionnaire: 57 from the Class of 2001 (29% of the class) and 133 from the Class of 2002 (68% of the class). Fifty-five percent of those returning the questionnaire were women. Twenty-six students reported that they did not remember filling out the BDI, and 4 students failed to answer this question. No difference in recall of the BDI was observed by class ( 2=2.14, df=1, P=0.17) or gender (2=3.98, df=1, P=0.06). Responses from the 161 students who indicated that they remembered completing the depression assessment (84% of respondents) were retained for analysis.
Sixteen students (9.9%) reported that they had not been completely frank and honest when filling out the questionnaire, and 1 student did not respond to that question. Nineteen students (11.8%) reported being concerned that there could be negative consequences for being fully frank and honest on the questionnaire, and 22 students (13.7%) were concerned that they would be affected in a negative way if they skipped the questionnaire or turned it in blank. Ten of these (6.2% of respondents) admitted to both concerns. Thus, 31 students (19.2%) admitted to one or both of these concerns. Reporting dishonesty on the BDI was unrelated to gender (2=0.70, df=1, P=0.43) or class (2=0.74, df=1, P=0.39) but was significantly related to beliefs about negative consequences of responding honestly (2=33.45, df=1, P<0.0001) and of turning in a blank depression inventory (2=8.36, df=1, P=0.011).
Odds ratios and 95% confidence intervals were calculated. Results are reported in Table 1. Students were 17 times more likely to be dishonest if they believed that being honest might have negative consequences than if they did not hold this belief. The estimated odds of being dishonest among students who were concerned about turning in a blank inventory were over four times the estimated odds for those who did not share this concern. Holding both beliefs increased the odds of being dishonest, but not in an apparent additive fashion. Table 2 depicts the numbers of students who were honest or dishonest as a function of whether they reported neither, either, or both concerns. The final column presents the percentage of students who reported lying as a function of neither, one, or both concerns. For example, of the students holding both concerns, 40% reported dishonesty; of students who held neither concern, 3.9% reported being dishonest on the BDI.
View this table:
[in this window]
[in a new window]
|
TABLE 2. Frequency of concerns as a function of reported honesty and percentage of students who reported being dishonest among respondents with neither, one, or both concerns
|
Half of the 16 students who reported not being honest indicated that they "felt the information was private but that they were compelled to fill out something." The students' responses were divided among the reasons presented on the survey for not being honest. (Students were allowed to indicate more than one reason.) Figure 1 presents the percentage of students indicating each of the available reasons for not being honest. The modal number of reasons checked was one (n=8), although 5 students checked two reasons. No student reported all four reasons, and only 2 students reported three of the four available reasons for dishonesty.
|
DISCUSSION
|
In recent years, there has been a growing concern about the importance of protecting research subjects, including medical students (9). At the same time, there are fears on the part of the public about privacy and confidentiality, particularly in light of increased access to personal information created by burgeoning technologies such as electronic medical records and the Internet. The importance of protecting individuals from the harm created by illness (e.g., previously undiagnosed depression) and the importance of protecting research subjects from invasions of privacy (e.g., the dean would be informed if a respondent was depressed) may not seem like contradictory values, but in our attempts to survey medical students about depression, a clear conflict became evident. Members of our research team made passionate arguments about the "ethical necessity" of identifying students who acknowledged suicidal ideation so that potential tragedy could be averted. Other researchers expressed shock and disbelief that such a sensitive subject was surveyed without the clear promise of anonymity. When our first consent form was presented at a plenary session of an organization of psychiatric educators, there was an audible gasp of incredulity. How could we expect students to be honest about a sensitive subject like suicide when being called into the dean's office was a potential consequence?
As with any value-laden controversy, there are no absolute solutions. Nevertheless, our findings illustrate that subjects will be swayed by these issues. Medical students and physicians are highly concerned about the documentation of any potentially "negative" data about themselves. With the fierce competition for desirable residencies, students are very sensitive regarding how they are portrayed in their records. Any perceived "weakness," whether academic, general medical, or psychiatric, might prevent them from attaining a desired position. As the data from our survey indicate, research that lacks anonymity creates a serious dilemma for students. They are divided between fears that they will be hurt by refusing to participate in the research and fears that honest responses will have negative repercussions. For some students, dishonestly filling out the questionnaire seemed to be the only "safe" solution to this conflict.
For psychiatric educators and other researchers, recognition of this conflict is vital. A strong stigma remains attached to mental illness. Any individual who acknowledges symptoms of psychopathology risks being stigmatized. Medical students and physicians are particularly sensitive about this risk, since there can in fact be professional repercussions associated with "impairment" due to physical or mental illness (10). In some instances, the stigma associated with mental illness may be even higher for the medical professional than for the general population (11). Not only do mentally ill medical students and physicians live with immense internalized stigma, they also often associate with colleagues who are not very understanding, sensitive, or forgiving. Research has demonstrated that physicians are reluctant to acknowledge mental illness and reluctant to seek help (12). Part of this reluctance is associated with the knowledge that annual medical license renewal forms routinely screen for impairment due to psychiatric illness. Students are also aware that being officially diagnosed and treated for depression or a similar condition could create future difficulties in acquiring health or disability insurance. A more pressing concern is fear that a diagnosis of psychiatric illness could interfere with academic standing and acquisition of competitive residency positions. In conducting research with medical students, it is important to recognize these fears. Although we expect our medical students to be honest, the risk associated with being honest on a questionnaire surveying mental illness far outweighs the benefits to them associated with our research. Forcing students into a position where many feel compelled to record dishonest answers compromises our research and impairs our relationship with them. Our attempt to assure them of "confidentiality" but not "anonymity" was an ineffective solution. The public in general has a low rate of confidence in the confidentiality of survey data (13). It is plausible that students may assume that "researchers" who are also their course directors, attending physicians, and administrators may be biased by their dual roles.
Of course, we must continue to study medical student populations. Medical students endure highly stressful conditions, and evidence suggests they have significant problems with mood compared with students in other professions (14). The stressors of medical school and residency may contribute to mood problems. As medical educators modify curricula, it is essential that the psychological impact of these modifications be monitored to prevent the unwitting increase in stress and depression.
Surveying students in sensitive areas requires the utmost precautions to ensure anonymity. Making anonymity a top priority can be problematic in a variety of ways. For instance, it renders the increasingly popular use of Internet-based surveys difficult or even impossible; students log on using a password, and therefore there is no assurance that responses cannot be traced back through the computer system. Additionally, anonymity complicates following students longitudinally. Moreover, employing a completely anonymous administration compromises the ability to answer other research questions, such as how responses on the BDI correlate with student demographic information, undergraduate grade point average, and MCAT scores.
Nevertheless, it is possible to survey sensitive information in an anonymous way and not entirely relinquish longitudinal follow-up. In one study surveying "cheating" behavior, students were given randomly numbered answer sheets and were asked to keep a private record of the code number, which could be entered in a follow-up questionnaire (15). Variations on this method could be accomplished by coding surveys with personal identification numbers the students could hold (if numbers were given to them) or remember (if they chose their own numbers). This method could result in decreased response rates, but the tradeoff might be an increase in the validity of the collected data.
There are several limitations to be noted in this study. First, our modest response rate of 47.8% suggests that the students who responded to our survey may have reflected a biased and nonrepresentative sample; therefore, our conclusions may not be generalizable to the entire class of medical students. On the other hand, it is likely that nonrespondents were at least as likely, if not more so, to have answered the original questionnaire inaccurately. The survey literature consistently identifies an underreporting bias for socially unacceptable phenomena (16). Based on what we know about underreporting bias, one might infer that a higher response rate would have yielded a higher rate of reported deception. A second limitation is that only 84% of respondents reportedly remembered the BDI, thus limiting the number of respondents we could use for analysis.
Despite these shortcomings, our research suggests that anonymity is important in identifying realistic levels of depressed mood. Additional research in this area is warranted, particularly that which develops innovative techniques for assessing sensitive issues such as psychological well-being among medical students. Comparisons between future anonymous data and the data we derived under conditions of "confidentiality but not anonymity" will assist clinicians, researchers, and educators in further appreciating the impact of measurement conditions on the validity of the results and the conclusions that can be drawn.
|
ACKNOWLEDGMENTS
|
The authors thank Anne Frye, Ph.D., Steven Lieberman, M.D., and James Hokanson, Ph.D., for their assistance with this research and Paula Levine for her assistance in preparation of the manuscript. This research was presented during a poster session at the 27th annual meeting of the Association of Directors of Medical Student Education in Psychiatry, Whistler, BC, June 21–23, 2001.
|
REFERENCES
|
- Levine RE, Bryant SG: The depressed physician: a different kind of impairment. Hosp Physician 2000; 36:67-73
- Lindeman S, Laara E, Hakko H, et al: A systematic review on gender-specific suicide mortality in medical doctors. Br J Psychiatry 1997; 168:274-279[Abstract/Free Full Text]
- Clark DC, Zedlow PB: Vicissitudes of depressed mood during four years of medical school. JAMA 1988; 260:2521-2528[Abstract/Free Full Text]
- Barrett J, Hurst MW, DiScala C, et al: Prevalence of depression over a 12-month period in a nonpatient population. Arch Gen Psychiatry 1978; 35:741-744[Abstract/Free Full Text]
- Zoccolillo M, Murphy GE, Wetzel RD: Depression among medical students. J Affect Disord 1986; 11:91-96[CrossRef][Medline]
- Camp DL, Hollingsworth ME, Zaccaro J, et al: Does a problem-based learning curriculum affect depression in medical students? Acad Med 1994; 69(10, suppl):S25-S27
- Wolf TM, Scurria PL, Webster M: A four-year study of anxiety, depression, loneliness, social support and perceived mistreatment in medical students. J Health Psychol 1998; 3:125-136[Abstract]
- Lieberman SA, Stroup-Benham CA, Peel JL, et al: Medical student perception of the academic environment: a prospective comparison of traditional and problem-based curricula. Acad Med 1991; 72(10, suppl):S13-S15
- Roberts LW, Geppert C, Connor R, et al: An invitation for medical educators to focus on ethical and policy issues in research and scholarly practice. Acad Med 2001; 76: 876-885
- Boisaubin E, Levine R: Identifying and assisting the impaired physician. Am J Med Sci 2001; 322:31-36[CrossRef][Medline]
- Dickstein LJ: Medical students and residents: issues and needs, in The Handbook of Physician Health. Edited by Goldman LS, Myers M, Dickstein LJ. Chicago, American Medical Association, 2000, pp 161-179
- Bittker TE: Reaching out to the depressed physician. JAMA 1976; 236:1713-1716[Abstract/Free Full Text]
- Martin E: Surveys as social indicators: problems in monitoring trends, in Handbook of Survey Research. Edited by Rossi PH, Wright JD, Anderson AB. San Diego, CA, Academic Press, 1985, pp 677-736
- Helmers KF, Danoff D, Steinert Y, et al: Stress and depressed mood in medical students, law students, and graduate students at McGill University. Acad Med 1997; 72:708-714[Medline]
- Sierles FS, Kushner BD, Krause PM: A controlled experiment with a medical student honor system. J Med Educ 1988; 63:705-712[Medline]
- Bradburn NM: Response effects, in Handbook of Survey Research. Edited by Rossi PH, Wright JD, Anderson AB. San Diego, CA, Academic Press, 1985, pp 289-318
This article has been cited by other articles:
|
|
|
|
 
M. F. Myers
Physician Impairment: Is It Relevant To Academic Psychiatry?
Acad Psychiatry,
February 1, 2008;
32(1):
39 - 43.
[Abstract]
[Full Text]
[PDF]
|
|
|
|
|
|
|
A. M. J. Reisbig, M. Hafen Jr., M. B. White, and B. R. Rush
Improving Response Rates: Introducing an Anonymous Longitudinal Survey Research Protocol for Veterinary Medical Students
J Vet Med Educ,
January 1, 2007;
34(2):
194 - 201.
[Abstract]
[Full Text]
[PDF]
|
|
|
|
|
|
|
S. Martini, C. L. Arfken, A. Churchill, and R. Balon
Burnout Comparison Among Residents in Different Medical Specialties
Acad Psychiatry,
September 1, 2004;
28(3):
240 - 242.
[Abstract]
[Full Text]
[PDF]
|
|
|
|
|
|
|
M. Myers
On the Importance of Anonymity in Surveying Medical Student Depression
Acad Psychiatry,
March 1, 2003;
27(1):
19 - 20.
[Full Text]
[PDF]
|
|
|
Get information about faster international access.
Privacy Policy
Copyright © 2003
Academic Psychiatry.
All rights reserved.
Home
| Search
| Current Issue
| Past Issues
| Subscribe
| All APPI Journals
| Help
| Contact Us
|
TOP
ABSTRACT
Editorial Note
