Medical student evaluation is an important function of a clinical educator. It consists of a clinical component, which typically includes an assessment of a student's knowledge, his or her clinical skills, and personal and interpersonal attributes (Table 1), and a score on a written subject examination. The latter is usually a standardized National Board of Medical Education subject examination (shelf-examination). These evaluations are used to identify students who fail to achieve the desired level of competence. However, there is only mixed evidence that the clinical component of such evaluations is objective (1–5). Myles found that students' subjective clinical evaluation scores linearly correlated with performance on standardized examinations (1). However, other studies report contrary findings. Davis and Banken found that clinical supervisors gave higher clinical grades to "extraverted" medical students but that these grades did not correlate with the shelf examination (2). Coulson et al. reported that mentors scored their mentored students statistically higher than non-mentors (3). Lee et al. found that students reporting greater reticence and lower quality of clerkship experience reported lower grades in various clerkships (4). Nahum reported that the shelf-examination score was the only normally-distributed performance indicator and was the most highly correlated measure with overall clinical clerkship performance (5). It is likely, therefore, that clerkships may use the standardized shelf-examination to balance the subjectivity of clinical assessments (6). In order to explore the vexing question of the ability of medical educators to effectively and objectively evaluate their students, we will report and discuss the findings of a study that was undertaken to correlate the different components of the clinical grade of third-year medical students in their psychiatry clerkship with their shelf-examination scores.

The subject protocol for this study was reviewed by the Institutional Review Board and was approved as exempt research.

Setting

The students are required to do a 6-week clerkship in psychiatry and are placed in eight academic or community hospital-based clinical sites. Each student is evaluated on four parameters (Table 1) that are graded on a 5-point Likert-type scale. A composite grade is obtained by averaging the component grades from each supervisor. The final clerkship grade is a combination of the clinical grade and the shelf-examination score. For each grade of Satisfactory, Highly Satisfactory, and Honors-level achievement, the student must achieve a threshold level on the composite clinical grade and the shelf-examination score. The evaluating faculty supervisors receive no specific training, but, at the end of each academic year, each faculty member is provided with a summary chart of his or her medical student grades and an anonymous chart that compares his or her grades with those of other supervisors.

Protocol/Analysis

During the first 6 months of 2008, the clinical grades of 141 students on the four parameters (Table 1), the overall clinical grade, and the shelf-examination score were compared.

The data were analyzed by a statistician to clarify the relationship between the four parameters of clinical evaluation, the total clinical grade, and the shelf-examination score. On the raw data, each variable had at least three common values, so we performed a correlation analysis (Table 2). The data on medical knowledge and clinical reasoning and synthesis skills were not skewed, so we used a Pearson correlation statistic; however, the data on history-taking skills, personal and interpersonal attributes (IP), and the total clinical grades were skewed, so Spearman correlations were used (7).

Grades received by the students in clinical reasoning and data-synthesis (correlation: 0.056; NS), history-taking skills (correlation: –0.041; NS), and the total clinical grade (correlation: 0.074; NS) did not predict the shelf-examination score (Table 2). Knowledge-base scores correlated weakly with shelf-examination scores (0.154; p<0.034). However, the IP grade was more strongly correlated with the shelf-examination score (0.194; p<0.011).

The total clinical grade did not correlate with the shelf-examination. This is consistent with the Nahum study (5). It is of interest, however, that the shelf-examination scores showed a stronger correlation with the IP component of the clinical grade than with the faculty assessment of the students' medical knowledge, history-taking skills, or the clinical skills/synthesizing abilities. This curious finding may have various explanations. As shown in Table 1, the IP component of the clinical evaluation is based on student attributes such as courtesy, effective communication, respectfulness, integrity, cooperation, punctuality, and maturity, in addition to good writing skills and motivation, Such characteristics may endear the students to the faculty supervisors and promote a mutually positive relationship that allows for greater interaction and more opportunity for feedback, both qualitative and quantitative. This may spur such students to study harder and, therefore, do better on the shelf-examination. In 2007, Brar et al. (8) had, indeed, found that the introduction of regular quantitative feedback in their program significantly improved student performance on the National Board of Medical Examiners examination. Alternatively, such likable and "cooperative" students might be more diligent and industrious to begin with and, therefore, more apt examination-takers. However, since the magnitude of the correlation is small for all parameters, it is also possible that the shelf-examination scores are quite independent of the clinical evaluations of the students; this was suggested by Nahum and others (5, 6, 9). This study is a small one and represents the experience of only one institution, but its findings, in light of a review of the existing literature on the evaluation measures used in clerkships, lead to some interesting conclusions that would have to be borne out by continuing investigation. It is heartening to note that there is a positive, albeit slight, correlation between the faculty assessment of the students' knowledge-base and the shelf-examination. More importantly, it appears that desirable personal attributes are identifiable and appear to be associated with the students' success, at least as measured by the shelf-examination.

At the time of submission, the author reported no competing interests.