Academic medicine is increasingly recognizing the important contributions of educators through promotion and through the granting of awards for teaching excellence. It has been suggested that achievements in teaching are more difficult to measure than numbers of publications and grants and are not as likely to contribute to promotion as research productivity (1). The most commonly evaluated area of teaching in academia is supervision, which is also considered the primary professional training model for mental health clinicians.

Supervision is a complex activity that includes mentoring, acting as a role model, teaching theoretical knowledge, guiding the resident in the use of specific techniques as applications of theory, and supporting the resident in dealing with the intensity and ambiguity of clinical experience (2). In this regard, psychiatry residents report that they value supervision and the activity of assessing patients with their supervisor more highly than they value didactic teaching, both for current educational needs and for their future professional development (3).

Although there are concerns regarding the subjectivity of student ratings of supervision, one study (4) and a survey of the literature (5) indicate that the dimensions that characterize the best supervisors include enthusiasm, organization and clarity, knowledge, clinical supervision, availability, and modeling. Thus, a supervisor evaluation form that includes and defines these dimensions has been in use at the University of Toronto Department of Psychiatry since 1993.

The literature reveals that women in academic psychiatry spend more time teaching than their male contemporaries (6). In the specialty with the highest proportion of female faculty, pediatrics, women also spend more time in teaching than men do, particularly at the lower academic ranks (7). Although women do not differ from men in how rewarding they perceive teaching to be, lower academic productivity and more time spent in teaching were associated with women's lower academic ranks (7). This balance of activities may contribute to understanding that although women medical school graduates are more likely to enter academia than men, they advance more slowly (8). A cohort study that controlled for work schedule, specialty, and productivity among those in academic medicine in the United States found that women remained substantially less likely to achieve promotion (9), even though male and female faculty value promotion equally (10). Given women's greater involvement in teaching in some fields of medicine, the difficulty of measuring achievement in teaching, and the tendency for women to progress more slowly in academia, it will be useful to determine whether male and female supervisors are evaluated differently.

Anecdotal experience in the Supervisor Evaluation Committee at the University of Toronto Department of Psychiatry suggested that more female than male supervisors had negative supervisor evaluations. To evaluate this observation, members of the committee undertook a retrospective survey comparing evaluations of male and female supervisors by their supervisees.

Anonymous supervisor evaluation forms are completed semiannually by psychiatry residents in the University of Toronto Department of Psychiatry. In order to reduce the bias of reactivity, residents are asked to indicate that they have completed the supervisor evaluation prior to receiving their own evaluation. On the Supervisor Evaluation form, psychiatry residents evaluate their supervisors by using a five-point Likert-type scale with the following descriptive anchors: 1=poor, 2=below average, 3=average, 4=above average, 5=excellent. This scale was used to provide an overall rating and a rating in each of six areas: enthusiasm, organization and clarity, knowledge, clinical supervision, availability, and modeling. At the request of the Supervisor Evaluation Committee, the Postgraduate Education Office compiled the evaluations retrospectively over a 3-year period within the past decade. Because these evaluations are completed as part of routine evaluation of our education program, the Research Ethics Board does not need to approve the collection of such data under the Tri-Council Statement Guidelines. However, the Research Ethics Board reviewed and approved submission of this manuscript.

In addition, we assessed whether male and female psychiatry supervisors had been honored with a prestigious academic award at a rate proportional to their respective numbers in the department. This evaluation was determined by assessing for the gender split in winners of an annual postgraduate teaching award presented to a supervisor on the basis of resident nominations and voting from 1986 through 2001.

The second author, while a resident, led a one-hour informal focus group with approximately 10 residents including equal numbers of male and female residents. Consent was implied by involvement. Notes were taken during the session. The residents were provided with the quantitative data described below and were asked two questions: 1) "What do you think of the findings?" and 2) "How do you understand the findings?"

The continuously rated questions from the evaluation form were assessed for supervisor gender differences by using two-tailed, independent-groups t-tests. Because large sample sizes such as the one employed in this study typically produce significant t values, we supplemented these between-groups results with effect size estimates, including Cohen's d (11) and corresponding Pearson r values. Although Cohen's d is arguably the best known and most widely used effect size estimate, we also provide r values in order to provide the reader with information about rule-of-thumb amount of variance explained (r value squared). Cohen's d>0.20 but ≤0.49 is considered a small effect size, >0.50 but ≤0.79 a medium effect size, and >0.80 a large effect size (11). It is important to note that most significant results in the behavioral sciences and nonbiologically based psychiatry research typically produce small to medium effect sizes (12).

A total of 1,765 evaluations were received for 298 psychiatry supervisors who provide training in all areas of psychiatry, including inpatient, outpatient, consultation-liaison, chronic care, child, geriatric, and psychotherapy. Of this group, 76 (25.5%) were women. The mean overall score for female supervisors was 4.15 (SD=0.88), which was significantly lower (P<0.05) than that for male supervisors, 4.35 (SD=0.76).

Comparison of composite ratings found that female supervisors were rated significantly lower than male supervisors by overall scores (P<0.05) and in the areas of enthusiasm (P<0.05), clarity (P<0.05), and knowledge (P<0.001). Effect sizes (Cohen's d) and r values were calculated to determine the relative magnitude of these differences and amount of variance accounted for by these variables (11). A medium effect size was found for knowledge (d=0.41), and small to medium effect sizes were found in the areas of enthusiasm (d=0.30) and clarity (d=0.28). The corresponding r values were, respectively, 0.20, 0.15, and 0.14, suggesting that these variables were accounting for less than 5% of the variance. Effect sizes in the areas of clinical supervision, availability, and modeling and in overall scores were small.

To determine if these differences were more significant at the low end of ratings, we compared female (n=15) and male supervisors (n=23) who received ratings that were two or more standard deviations below the mean in any area. A chi-square analysis found a higher than expected number of women in this group (χ²=4.47, df=1, P<0.045, two-tailed). Although significant differences were not found in any area by t-tests, a medium effect size was found, revealing that ratings of male supervisors' knowledge (d=0.53) and clarity (d=0.36) subscale scores in this group were higher than those of female supervisors, but again this accounted for only a small amount of variance (6%, r=0.25, and 1%, r=0.18, respectively).

The annual postgraduate teaching award, which is selected by residents, was used to assess for gender differences at the high end of evaluation. For the first 10 years that this award was presented, all awards went to male supervisors. In the past 7 years, two of the seven awards (29%) were given to female supervisors and five (71%) to male supervisors. This percentage closely approximates the percentage of women supervisors in the faculty.

In the focus group, male residents were more vocal than female residents were. The male residents tended to express support for the findings as reflecting male and female supervisors' actual performance, and they reported a preference for academic supervisors—who, in their experience, were less frequently female. More female residents than male residents expressed the belief that the differences between female and male supervisors were not "real" but instead were due to residents evaluating male and female supervisors according to different criteria. Some residents also suggested that a larger proportion of the female faculty may hold junior faculty positions and that this might contribute to the differences.

This preliminary retrospective pilot study of anonymous supervisor evaluations indicates that psychiatry residents endorse high levels of satisfaction with their supervisors, irrespective of supervisor gender. This finding is consistent with our previous study of psychiatry resident training activities (3) and a study by others (13) that found supervision was the most highly valued educational experience during residency. To our knowledge, this is the first study to examine gender differences in the evaluation of supervisors. Direct comparison of composite evaluations for a three-year period in the past decade found that female supervisors were assessed somewhat less favorably in the dimensions of knowledge, clarity, enthusiasm, and overall scores. Further, a larger than expected proportion of female supervisors was at the low end of evaluations. By contrast, female supervisors in recent years have been awarded the psychiatry department's most significant teaching award at a rate more consistent with their numbers in the department.

Supervision occurs in a relational context, and it is known that men and women approach relationships differently: men are generally found to be more goal oriented and women to focus more on responsibilities within the relationship (14). Nadelson et al. (15) suggest that in supervision, the supervisor's communication conveys a preexisting, self-ascribed gender role. With four possible supervisory pairs (male supervisor—male resident, female supervisor—female resident, male supervisor—female resident, and female supervisor—male resident), the potentially subtle and complex influence of gender in supervision (16) may not be captured by a quantitative analysis of evaluations of supervisors. Because this study used anonymous supervisor evaluations, we were unable to compare evaluations by supervisor pair. A microanalytic study of the four possible supervisory pairs found that most interactions had a hierarchical nature, in which the supervisor assumed an "expert role" that superseded a gender influence (17). Thus, it is not surprising that the significant difference found between evaluations of male and female supervisors is small.

As of 2001, in the Department of Psychiatry at the University of Toronto, 34% of the faculty were women, but women hold a greater proportion of entry-level academic positions (44% of lecturers and 35% of assistant professors) than do men. In contrast, nearly three-quarters of the associate professors and more than 80% of the full professors are men, which is greater than their overall proportion of the faculty (66%). The higher ranks are typically associated with greater research productivity, and this variable may contribute to full and associate professors' appearing more knowledgeable when supervising in their field of research. Further, Leibenluft et al. (18) found that among American full-time psychiatric faculty, women were less likely to have research training, to be principal investigators on peer-reviewed grants, or to be currently involved in research activity. Similarly, Reiser and colleagues' study (6) of Yale psychiatry graduates reported that women had fewer peer-reviewed publications than men. This is not surprising, since female academics tend to work fewer total hours than do male academics (19).

In the context of supervision, an active teaching style and a focus on the central therapeutic problem (20,21) are valued more than the supervisor's knowledge. The fact that residents most value those aspects of supervision that do not necessarily have to do with knowledge may explain why supervisees' rating of their female supervisors "overall" was not as negative as their rating of their knowledge compared with male supervisors. There have been no previous studies of a gender influence in the areas of enthusiasm, organization, or clarity in supervision.

It may be most useful to consider the influence of same-gender and cross-gender supervision in various areas. With respect to mentoring, women trainees may look to women supervisors to provide models of "illustrative success," a psychosocial function. It has also been suggested that the lower numbers of females than males in senior positions in academic medicine contributes to "surplus visibility" (22). That is, there may be greater critical scrutiny of female faculty behavior and treatment. Male faculty tend to be valued by male students for their capacity to identify with the protégé and for providing access to the organization (23). One study found that psychology Ph.D. students receiving same-gender supervision had greater productivity than those who were supervised by an opposite-gender faculty member (24). It has also been suggested that some particular supervisory pairs may result in a more negative evaluation of women. Nadelson et al. (15) reported that residents given a choice of supervisors would choose on the basis of reputation and perceived power; however, if a male and a female supervisor were equal in this regard, male residents preferred male supervisors, whereas female residents would look for a balance of male and female supervisors.

To our knowledge, this preliminary retrospective study is the first to assess for differences in the evaluation of male and female psychiatry supervisors. Clearly, residents place a high value on the supervision provided by both male and female faculty. The finding of a somewhat less favorable evaluation of female supervisors and a higher-than-expected frequency of poor evaluations of female supervisors for three consecutive years in the past decade requires greater scrutiny. However, this difference may be diminishing, as evidenced by new recognition of female faculty members for excellence in teaching.

These findings must be considered preliminary. Further study will be required that controls for the gender and year of the resident as well as for supervisor variables such as field of supervision, academic seniority, and time devoted to teaching. More research will be needed to determine whether our findings can be replicated and may be applicable in other areas of medicine, including those where there are smaller proportions of women.

It may be of greater interest to look beyond evaluation of supervisors and instead to apply the lens of gender to investigating the qualitative factors in supervision. Relational variables such as feedback, support, power and rivalry, and role modeling have been found to influence supervisory effectiveness (25). Examination of these variables and how different gender pairings of supervisor and resident influence them is fertile ground for further study. For example, qualitative research with focus groups could be used to investigate the experience of same-gender and cross-gender supervision. Results from such an investigation may increase our understanding of which factors contribute to a "good fit" and the influence of those factors on the effectiveness of supervision.

This work was previously presented at the 154th annual meeting of the American Psychiatric Association, New Orleans, LA, May 8, 2001.