Academic Psychiatry 27:39-43, March 2003
© 2003 Academic Psychiatry
A Comparison of Evaluations of Male and Female Psychiatry Supervisors
Janet de Groot, M.D.,
Aileen Brunet, M.D.,
Allan S. Kaplan, M.D. and
Michael Bagby, Ph.D.
This work was completed at the University of Toronto Department of Psychiatry, Toronto, Ontario, Canada. Address correspondence to Dr. de Groot, Department of Psychiatry, University Health Network, 200 Elizabeth Street, Suite 8 EN-219A, Toronto, Ontario, Canada M5G 2C4. E-mail: janet.degroot{at}uhn.on.ca
|
ABSTRACT
|
OBJECTIVE: To assess whether male and female psychiatry supervisors are evaluated differently by psychiatry residents. METHODS: The University of Toronto Department of Psychiatry compiled anonymous supervisor evaluations completed semiannually by psychiatry residents over a period of 3 years. Male and female psychiatry supervisors' ratings were compared by using t-tests, effect estimates, and chi-square analyses. Results from these ratings were discussed in a resident focus group. RESULTS: Female psychiatry supervisors (n=76) were rated significantly lower than male supervisors (n=222), both overall (P<0.05) and in the areas of enthusiasm (P<0.05), clarity (P<0.05), and knowledge (P<0.001). CONCLUSIONS: Future studies comparing evaluations of supervision by male and female psychiatrists must control for academic rank, numbers of publications, and hours of teaching. Comparing evaluations of the various male–female supervisory pairs will be useful to assess for gender biases.
Key Words: Evaluation • Psychiatry Supervision • Gendered Differences
|
INTRODUCTION
|
Academic medicine is increasingly recognizing the important contributions of educators through promotion and through the granting of awards for teaching excellence. It has been suggested that achievements in teaching are more difficult to measure than numbers of publications and grants and are not as likely to contribute to promotion as research productivity (1). The most commonly evaluated area of teaching in academia is supervision, which is also considered the primary professional training model for mental health clinicians.
Supervision is a complex activity that includes mentoring, acting as a role model, teaching theoretical knowledge, guiding the resident in the use of specific techniques as applications of theory, and supporting the resident in dealing with the intensity and ambiguity of clinical experience (2). In this regard, psychiatry residents report that they value supervision and the activity of assessing patients with their supervisor more highly than they value didactic teaching, both for current educational needs and for their future professional development (3).
Although there are concerns regarding the subjectivity of student ratings of supervision, one study (4) and a survey of the literature (5) indicate that the dimensions that characterize the best supervisors include enthusiasm, organization and clarity, knowledge, clinical supervision, availability, and modeling. Thus, a supervisor evaluation form that includes and defines these dimensions has been in use at the University of Toronto Department of Psychiatry since 1993.
The literature reveals that women in academic psychiatry spend more time teaching than their male contemporaries (6). In the specialty with the highest proportion of female faculty, pediatrics, women also spend more time in teaching than men do, particularly at the lower academic ranks (7). Although women do not differ from men in how rewarding they perceive teaching to be, lower academic productivity and more time spent in teaching were associated with women's lower academic ranks (7). This balance of activities may contribute to understanding that although women medical school graduates are more likely to enter academia than men, they advance more slowly (8). A cohort study that controlled for work schedule, specialty, and productivity among those in academic medicine in the United States found that women remained substantially less likely to achieve promotion (9), even though male and female faculty value promotion equally (10). Given women's greater involvement in teaching in some fields of medicine, the difficulty of measuring achievement in teaching, and the tendency for women to progress more slowly in academia, it will be useful to determine whether male and female supervisors are evaluated differently.
Anecdotal experience in the Supervisor Evaluation Committee at the University of Toronto Department of Psychiatry suggested that more female than male supervisors had negative supervisor evaluations. To evaluate this observation, members of the committee undertook a retrospective survey comparing evaluations of male and female supervisors by their supervisees.
|
METHODS
|
Anonymous supervisor evaluation forms are completed semiannually by psychiatry residents in the University of Toronto Department of Psychiatry. In order to reduce the bias of reactivity, residents are asked to indicate that they have completed the supervisor evaluation prior to receiving their own evaluation. On the Supervisor Evaluation form, psychiatry residents evaluate their supervisors by using a five-point Likert-type scale with the following descriptive anchors: 1=poor, 2=below average, 3=average, 4=above average, 5=excellent. This scale was used to provide an overall rating and a rating in each of six areas: enthusiasm, organization and clarity, knowledge, clinical supervision, availability, and modeling. At the request of the Supervisor Evaluation Committee, the Postgraduate Education Office compiled the evaluations retrospectively over a 3-year period within the past decade. Because these evaluations are completed as part of routine evaluation of our education program, the Research Ethics Board does not need to approve the collection of such data under the Tri-Council Statement Guidelines. However, the Research Ethics Board reviewed and approved submission of this manuscript.
In addition, we assessed whether male and female psychiatry supervisors had been honored with a prestigious academic award at a rate proportional to their respective numbers in the department. This evaluation was determined by assessing for the gender split in winners of an annual postgraduate teaching award presented to a supervisor on the basis of resident nominations and voting from 1986 through 2001.
The second author, while a resident, led a one-hour informal focus group with approximately 10 residents including equal numbers of male and female residents. Consent was implied by involvement. Notes were taken during the session. The residents were provided with the quantitative data described below and were asked two questions: 1) "What do you think of the findings?" and 2) "How do you understand the findings?"
The continuously rated questions from the evaluation form were assessed for supervisor gender differences by using two-tailed, independent-groups t-tests. Because large sample sizes such as the one employed in this study typically produce significant t values, we supplemented these between-groups results with effect size estimates, including Cohen's d (11) and corresponding Pearson r values. Although Cohen's d is arguably the best known and most widely used effect size estimate, we also provide r values in order to provide the reader with information about rule-of-thumb amount of variance explained (r value squared). Cohen's d>0.20 but  0.49 is considered a small effect size, >0.50 but 0.79 a medium effect size, and >0.80 a large effect size (11). It is important to note that most significant results in the behavioral sciences and nonbiologically based psychiatry research typically produce small to medium effect sizes (12).
|
RESULTS
|
A total of 1,765 evaluations were received for 298 psychiatry supervisors who provide training in all areas of psychiatry, including inpatient, outpatient, consultation-liaison, chronic care, child, geriatric, and psychotherapy. Of this group, 76 (25.5%) were women. The mean overall score for female supervisors was 4.15 (SD=0.88), which was significantly lower (P<0.05) than that for male supervisors, 4.35 (SD=0.76).
Comparison of composite ratings found that female supervisors were rated significantly lower than male supervisors by overall scores (P<0.05) and in the areas of enthusiasm (P<0.05), clarity (P<0.05), and knowledge (P<0.001). Effect sizes (Cohen's d) and r values were calculated to determine the relative magnitude of these differences and amount of variance accounted for by these variables (11). A medium effect size was found for knowledge (d=0.41), and small to medium effect sizes were found in the areas of enthusiasm (d=0.30) and clarity (d=0.28). The corresponding r values were, respectively, 0.20, 0.15, and 0.14, suggesting that these variables were accounting for less than 5% of the variance. Effect sizes in the areas of clinical supervision, availability, and modeling and in overall scores were small.
To determine if these differences were more significant at the low end of ratings, we compared female (n=15) and male supervisors (n=23) who received ratings that were two or more standard deviations below the mean in any area. A chi-square analysis found a higher than expected number of women in this group ( 2=4.47, df=1, P<0.045, two-tailed). Although significant differences were not found in any area by t-tests, a medium effect size was found, revealing that ratings of male supervisors' knowledge (d=0.53) and clarity (d=0.36) subscale scores in this group were higher than those of female supervisors, but again this accounted for only a small amount of variance (6%, r=0.25, and 1%, r=0.18, respectively).
The annual postgraduate teaching award, which is selected by residents, was used to assess for gender differences at the high end of evaluation. For the first 10 years that this award was presented, all awards went to male supervisors. In the past 7 years, two of the seven awards (29%) were given to female supervisors and five (71%) to male supervisors. This percentage closely approximates the percentage of women supervisors in the faculty.
In the focus group, male residents were more vocal than female residents were. The male residents tended to express support for the findings as reflecting male and female supervisors' actual performance, and they reported a preference for academic supervisors—who, in their experience, were less frequently female. More female residents than male residents expressed the belief that the differences between female and male supervisors were not "real" but instead were due to residents evaluating male and female supervisors according to different criteria. Some residents also suggested that a larger proportion of the female faculty may hold junior faculty positions and that this might contribute to the differences.
|
DISCUSSION
|
This preliminary retrospective pilot study of anonymous supervisor evaluations indicates that psychiatry residents endorse high levels of satisfaction with their supervisors, irrespective of supervisor gender. This finding is consistent with our previous study of psychiatry resident training activities (3) and a study by others (13) that found supervision was the most highly valued educational experience during residency. To our knowledge, this is the first study to examine gender differences in the evaluation of supervisors. Direct comparison of composite evaluations for a three-year period in the past decade found that female supervisors were assessed somewhat less favorably in the dimensions of knowledge, clarity, enthusiasm, and overall scores. Further, a larger than expected proportion of female supervisors was at the low end of evaluations. By contrast, female supervisors in recent years have been awarded the psychiatry department's most significant teaching award at a rate more consistent with their numbers in the department.
Supervision occurs in a relational context, and it is known that men and women approach relationships differently: men are generally found to be more goal oriented and women to focus more on responsibilities within the relationship (14). Nadelson et al. (15) suggest that in supervision, the supervisor's communication conveys a preexisting, self-ascribed gender role. With four possible supervisory pairs (male supervisor–male resident, female supervisor–female resident, male supervisor–female resident, and female supervisor–male resident), the potentially subtle and complex influence of gender in supervision (16) may not be captured by a quantitative analysis of evaluations of supervisors. Because this study used anonymous supervisor evaluations, we were unable to compare evaluations by supervisor pair. A microanalytic study of the four possible supervisory pairs found that most interactions had a hierarchical nature, in which the supervisor assumed an "expert role" that superseded a gender influence (17). Thus, it is not surprising that the significant difference found between evaluations of male and female supervisors is small.
As of 2001, in the Department of Psychiatry at the University of Toronto, 34% of the faculty were women, but women hold a greater proportion of entry-level academic positions (44% of lecturers and 35% of assistant professors) than do men. In contrast, nearly three-quarters of the associate professors and more than 80% of the full professors are men, which is greater than their overall proportion of the faculty (66%). The higher ranks are typically associated with greater research productivity, and this variable may contribute to full and associate professors' appearing more knowledgeable when supervising in their field of research. Further, Leibenluft et al. (18) found that among American full-time psychiatric faculty, women were less likely to have research training, to be principal investigators on peer-reviewed grants, or to be currently involved in research activity. Similarly, Reiser and colleagues' study (6) of Yale psychiatry graduates reported that women had fewer peer-reviewed publications than men. This is not surprising, since female academics tend to work fewer total hours than do male academics (19).
In the context of supervision, an active teaching style and a focus on the central therapeutic problem (20,21) are valued more than the supervisor's knowledge. The fact that residents most value those aspects of supervision that do not necessarily have to do with knowledge may explain why supervisees' rating of their female supervisors "overall" was not as negative as their rating of their knowledge compared with male supervisors. There have been no previous studies of a gender influence in the areas of enthusiasm, organization, or clarity in supervision.
It may be most useful to consider the influence of same-gender and cross-gender supervision in various areas. With respect to mentoring, women trainees may look to women supervisors to provide models of "illustrative success," a psychosocial function. It has also been suggested that the lower numbers of females than males in senior positions in academic medicine contributes to "surplus visibility" (22). That is, there may be greater critical scrutiny of female faculty behavior and treatment. Male faculty tend to be valued by male students for their capacity to identify with the protégé and for providing access to the organization (23). One study found that psychology Ph.D. students receiving same-gender supervision had greater productivity than those who were supervised by an opposite-gender faculty member (24). It has also been suggested that some particular supervisory pairs may result in a more negative evaluation of women. Nadelson et al. (15) reported that residents given a choice of supervisors would choose on the basis of reputation and perceived power; however, if a male and a female supervisor were equal in this regard, male residents preferred male supervisors, whereas female residents would look for a balance of male and female supervisors.
|
CONCLUSIONS AND FUTURE DIRECTIONS
|
To our knowledge, this preliminary retrospective study is the first to assess for differences in the evaluation of male and female psychiatry supervisors. Clearly, residents place a high value on the supervision provided by both male and female faculty. The finding of a somewhat less favorable evaluation of female supervisors and a higher-than-expected frequency of poor evaluations of female supervisors for three consecutive years in the past decade requires greater scrutiny. However, this difference may be diminishing, as evidenced by new recognition of female faculty members for excellence in teaching.
These findings must be considered preliminary. Further study will be required that controls for the gender and year of the resident as well as for supervisor variables such as field of supervision, academic seniority, and time devoted to teaching. More research will be needed to determine whether our findings can be replicated and may be applicable in other areas of medicine, including those where there are smaller proportions of women.
It may be of greater interest to look beyond evaluation of supervisors and instead to apply the lens of gender to investigating the qualitative factors in supervision. Relational variables such as feedback, support, power and rivalry, and role modeling have been found to influence supervisory effectiveness (25). Examination of these variables and how different gender pairings of supervisor and resident influence them is fertile ground for further study. For example, qualitative research with focus groups could be used to investigate the experience of same-gender and cross-gender supervision. Results from such an investigation may increase our understanding of which factors contribute to a "good fit" and the influence of those factors on the effectiveness of supervision.
|
ACKNOWLEDGMENTS
|
This work was previously presented at the 154th annual meeting of the American Psychiatric Association, New Orleans, LA, May 8, 2001.
|
REFERENCES
|
- Tesch BJ, Nattinger AB: Career advancement and gender in academic medicine. Journal of the Irish Colleges of Physicians and Surgeons 1997; 26:172-176
- Alonso A: The Quiet Profession: Supervisors of Psychotherapy. New York, Macmillan, 1985
- de Groot J, Tiberius R, Sinai J, et al: Psychiatric residency: an analysis of training activities with recommendations. Acad Psychiatry 2000; 24:139-146[Abstract/Free Full Text]
- Irby D, Rakestraw P: Evaluating clinical teaching in medicine. J Med Educ 1981; 56:181-186[Medline]
- Herrmann N: Supervisor evaluation: from theory to implementation. Acad Psychiatry 1996; 20:205-211[Abstract]
- Reiser LW, Sledge WH, Fenton W, et al: Beginning careers in academic psychiatry for women: "Bermuda Triangle"? Am J Psychiatry 1993; 150:1392-1397[Abstract/Free Full Text]
- Kaplan SH, Sullivan LM, Dukes KA, et al: Sex differences in academic advancement: results of a national study of pediatricians. N Engl J Med 1996; 335:1282-1289[Abstract/Free Full Text]
- Nonnemaker L: Women physicians in academic medicine: new insights from cohort studies. N Engl J Med 2000; 342:399-405[Abstract/Free Full Text]
- Tesch BJ, Wood HM, Helwig AL, et al: Promotion of women physicians in academic medicine: glass ceiling or sticky floor? JAMA 1995; 273:1022-1025[Abstract/Free Full Text]
- Buckley LM, Sanders K, Shih M, et al: Obstacles to promotion? Values of women faculty about career success and recognition. Acad Med 2000; 75:283-288[Medline]
- Rosenthal RD, Rosnaw, RL: Essentials of Behavioral Research: Methods and Data Analysis. New York, McGraw-Hill, 1984
- Cohen J: Statistical Power Analysis for the Behavioral Sciences, 2nd ed. Hillsdale, NJ, Lawrence Erlbaum, 1988
- Leichner P, Harper D: Profile of the ideal residency program in psychiatry: the Canadian resident's viewpoint. Journal of Psychiatric Education 1981; 5:269-277
- Gilligan C: In a Different Voice. Cambridge, MA, Harvard University Press, 1982
- Nadelson CC, Belitsky C, Seeman MV, et al: Gender issues in supervision, in Clinical Perspectives on Psychotherapy Supervision. Edited by Greben S, Ruskin R. Washington, DC, American Psychiatric Press, 1994, pp 41-51
- Scher M: Gender issues in psychiatric supervision. Compr Psychiatry 1981; 22:179-183[CrossRef][Medline]
- Nelson ML, Holloway EL: Relation of gender to power and involvement in supervision. Journal of Counseling Psychology 1990; 37:473-481[CrossRef]
- Leibenluft E, Dial TH, Haviland MG, et al: Sex differences in rank attainment and research activities among academic psychiatrists. Arch Gen Psychiatry 1993; 50:896-904[Abstract/Free Full Text]
- Barer ML, Kazanjian A, Pagliccia N, et al: A profile of academic physicians in British Columbia. Acad Med 1989; 64:524-532[Medline]
- Kline F, Goin MK, Zimmeran W: You can be a better supervisor! Journal of Psychiatric Education 1977; 1:174-179
- Shanfield SB, Matthews KL, Hetherly V: What do excellent psychotherapy supervisors do? Am J Psychiatry 1993; 150:1081-1084[Abstract/Free Full Text]
- Bickel J: Women in academic medicine. J Am Med Womens Assoc 2000; 55:10-19
- Gilbert LA, Rossman KM: Gender and the mentoring process for women: implications for professional development. Professional Psychology: Research and Practice 1992; 23:233-238
- Goldstein E: Effects of same-sex and cross-sex role models on the subsequent academic productivity of scholars. Am Psychol 1979; 34:407-410[CrossRef]
- Sinai J, Tiberius RG, de Groot J, et al: Developing a training program to improve supervisor-resident relationships, step one: defining the types of issues. Teach Learn Med 2001; 13:80-85[CrossRef][Medline]
This article has been cited by other articles:
|
|
|
|
 
P. E. Garfinkel, R. M. Bagby, D. R. Schuller, S. E. Dickens, F. S. Schulte, and L. Fitzgerald
Gender Differences in the Practice Characteristics and Career Satisfaction of Psychiatrists in Ontario
Acad Psychiatry,
December 1, 2004;
28(4):
310 - 320.
[Abstract]
[Full Text]
[PDF]
|
|
|
Get information about faster international access.
Privacy Policy
Copyright © 2003
Academic Psychiatry.
All rights reserved.
Home
| Search
| Current Issue
| Past Issues
| Subscribe
| All APPI Journals
| Help
| Contact Us
|
TOP
ABSTRACT
INTRODUCTION
