The provision of feedback is one of the most profoundly effective educational tools we have for both improvement of performance and for guiding learning. This is true for the junior medical student or the senior physician, regardless of when or where they trained. Ende defined feedback as “the formative process by which the teacher provides the learner with information about his/her performance, for the purpose of improving that performance” (1). Without specific and focused feedback from teachers, learners tend to determine for themselves the quality of their work, either by self-assessment or by monitoring. Neither of these imparts accurate information to the learner. Self-assessment is often unreliable, and, without reinforcement by teachers, learners may extinguish desired behaviors or establish undesirable ones. Often, a learner does something especially positive without being aware of it. Monitoring involves the interpretation of perceived cues or feedback from other people’s behaviors (2). Monitored feedback is received through the individual’s own filter or self-perception, and it is prone to misinterpretation. For example, a supervisor may grimace because of the twinge of an arthritic knee during a learner’s case presentation. A learner who is feedback monitoring may well notice that and incorrectly infer a negative assessment of his/her performance.
By definition, accurate feedback is going to include corrective (negative) as well as reinforcing (positive) items. Human nature being what it is, teachers are often uncomfortable giving what feels like critical feedback, and learners are often reluctant to seek it. This can be addressed in part by training and skill-building for feedback-givers and by creating a culture of a reciprocal feedback for all. Because LCME-accredited medical schools have specific standards for feedback, most U.S. medical graduates enter residency training with a certain amount of familiarity, if not comfort, with the feedback process. Many international medical graduates (IMGs) enter United States residency training with either limited experience with feedback or negative perceptions and experiences with feedback. The lack of comfort and familiarity with feedback has been postulated to be an associated factor in IMGs with poor academic performance (3). In 2009–2010, IMGs made up 27.4% of U.S. residents (4). The percentage of IMGs in psychiatric residency/fellowship positions was slightly higher, at 33% (5). Of these, about three-quarters are non-U.S. IMGs. In this article, the term “IMG” will refer to non-U.S. IMGs. Schools in India/Nepal and Pakistan educated the largest group of new IMGs entering United States residency in 2009 (6).
In this article, we will outline basic tenets of good feedback for all learners and address some of the issues that can present barriers to the delivery and receipt of effective feedback for IMGs. The authors are a current and former psychiatry program director, with 27 combined years of experience working with United States-trained and IMG residents. In preparation for this research, we reviewed pertinent literature regarding effective feedback, pulled from personal observations, reviewed narrative accounts of IMGs training in the U.S. and Canada (7–9), and interviewed a small group of internal medicine and/or psychiatry residents (4 men, 1 woman) from India, Nepal, and Nigeria about their experiences with feedback. Their comments have been used with permission to illustrate salient points.
+
Delivery of Effective Feedback
Compared with North American schools, many international medical schools have a much steeper power-differential between teachers and learners (7). Formative feedback is often not a uniform component of the curriculum. For many IMGs, feedback is tantamount to criticism and is interpreted as a serious failure on the part of the learner. This can make it difficult to actively seek feedback or engage in self-assessment, as it is seen as exposing weaknesses:
We called the doctors “Ma’am” and “Sir” and stood up when they walked into the room. Mistakes were pointed out publicly–seldom constructively.
Feedback was given during rounds in front of the patients, nurses and peers … It was very embarrassing. The belief is that we will study more to avoid shame. I still get anxious when I am called in for feedback.
The only way to do it (give feedback) was by making you embarrassed … yelling at you in front of the patient and family. You only got feedback if you were in the limelight, presenting or something.
This group also described a culture of extreme competition and pressure to be the best:
Everyone has to excel; otherwise it reflects badly on the professor. If you weren’t among the top scorers, it was regarded as failure. Self-reflection was not encouraged or even tolerated, as no student wanted to expose weaknesses.
Multiple authors have contemplated and reported on learner and teacher factors that contribute to an effective feedback experience (1, 10–13). Brinko (10) organized them into a variation of journalism’s “five ‘W’s:”
It comes from a number of sources (peers, supervisors, patients, etc.)
It comes from self as well as others.
The source is perceived by a learner as credible and well-intentioned.
Any well done feedback session should begin with self-reflection. (“How do you think you did?” or “Tell me what you did well and what you would like to improve upon.”) This necessary step is often challenging for learners and teachers. An honest self-assessment will often feel like either bragging or exposing weaknesses, both of which can generate anxiety in the learner. For IMGs, the fear of exposing weakness and the pressure to excel, both for the sake of the learner and the teacher, can make this vital step of the process more anxiety-provoking. Searight and Gafford (8) and Dorgan et al (9) interviewed family- and community-medicine residents from a variety of international medical schools regarding their perceptions of doctor–patient communication and their own experiences with behavioral-science education. In these interviews, a number of the IMG residents felt that they were being scrutinized by their patients and faculty more closely than their U.S.-trained peers. This could clearly influence how credible the feedback is perceived to be. These residents also echoed the feeling that they needed to prove themselves to faculty. Also, a number described a high level of insecurity and anxiety about being terminated. This reluctance to self-assess is often interpreted by the supervisor as a lack of interest or lack of self-reflection. A predictable vicious cycle ensues.
It contains concrete, specific information.
It contains accurate, irrefutable data/observations.
It is focused on behaviors (not personality).
It is limited to actions (not presumed intentions).
It is limited to behaviors that are remediable.
It creates cognitive dissonance (between the learners’ self-perception of performance and where he/she would like to be).
It contains suggestions for improvement (must end with an action plan).
It is reciprocal.
In terms of performance-improvement, immediate feedback based on direct observation of specific behaviors is the ideal. However, it is not uncommon in training programs for the lines between feedback and evaluation to become blurred. It is very important for both learners and teachers to be clear on the differences between feedback and evaluation. Feedback can be seen as a form of coaching or continuous performance-monitoring and correction. Evaluation, on the other hand, is a summative assessment, which involves a judgment or rating for a particular performance or time-period. It is useful to see these as points on an educational continuum.
Pure formative feedback will often take place in the context of a specific clinical encounter or direct observation. It’s useful to conceptualize this as an educational gift for learners. Most feedback will never appear on a summative evaluation unless previously-identified deficiencies are not corrected. Formative evaluation refers to that blending of the two. Formative evaluation typically takes the form of a mid-rotation feedback session and often includes information not just from one encounter, but from a variety of encounters and even second-hand input from different sources (i.e., nursing, etc.) over the course of the first half of the rotation. The summative evaluation is what we typically consider the formal end-of-rotation evaluation, which may be delivered face-to-face and in written form. If feedback is occurring on a regular basis, there should be no surprises for the learner in the formative evaluation/mid-rotation feedback. Likewise, if formative evaluation is taking place, there should be no surprises for the learner in the final or summative evaluation. This is true for all learners. For IMGs, it can be helpful to clearly and specifically identify an encounter as feedback versus mid-rotation formative evaluation, etc., to temper some of the previously-described anxieties.
In a well-functioning system, feedback is reciprocal. Although most residents recognize that faculty need feedback to make their teaching experiences the best, many IMGs have difficulty with this. In discussion of reciprocal feedback in our focus group, one senior resident stated: “There is no way we can do that!” However, our group reported that the fear factor associated with feedback, both receiving and giving, did dissipate somewhat with time in U.S. training, and all reported that it was important for faculty to keep asking.
A safe learning climate is of paramount importance for all learners. Because of the association of feedback with shame and embarrassment that has been reported by many IMGs, this assumes a greater level of importance. One resident in our focus-group described postponing a scheduled feedback session with a very warm and caring faculty member three times because of anxiety, even though he knew the faculty involved would not belittle him in any way. As with self-assessment, the potential exists here for faculty to interpret this as a resident who is unreceptive to feedback or uninterested in his/her education.
It is approached with teacher and learner working together as allies with a common goal.
It is relevant to the learner and based on clear expectations.
It reduces uncertainty for the learner.
It allows for response and interaction.
It is phrased in descriptive, common, nonjudgmental language.
It blends a moderate amount of positive feedback with selected negative feedback.
+
Communication and Language
In a well done feedback encounter, the learner should walk out of the experience with a clear understanding of his/her supervisors’ expectations to either build on what the learner is doing well or change what is not being done well. In many cases, the learner and teacher will be in agreement about what the deficiencies are. There will be some situations where the learner does not perceive the deficiency or agree with the faculty. In these instances, it is doubly important to be clear on the expectations and what follow-up will be expected. The way feedback is phrased is very important in how it is received. The language should be nonjudgmental and descriptive. It is sometimes helpful to sandwich corrective (negative) feedback between reinforcing (positive) feedback. This approach can be highly valuable for IMGs who may have remembered only negative feedback in their earlier careers. When positive or reinforcing feedback is given, it is more effective when it is attributed to internal causes (hard work, effort) and when it is given in the second person (i.e., “You handled that patient’s angry outburst very calmly and kindly.”). Negative feedback, on the other hand, tends to be better received when it is delivered in the first person (i.e., “I had trouble following you when you presented the HPI,” as opposed to “You were very disorganized in your HPI.”). The presentation of negative feedback is also better received when it is self-referenced, comparing the learner’s performance to measures of his/her ability and capabilities, rather than another learner’s performance (“If you make these changes, I’m confident that you will perform a much more accurate mental status exam.”) (10, 14).
Giving feedback to residents on interpersonal aspects of their patient encounters is often harder for faculty than giving feedback on more neutral aspects of the physical or mental status examination, yet it is a vitally important part of clinical care and professional development for residents in any specialty. For IMGs, physician–patient communication is sometimes made more complicated by language difficulties. IMGs enter U.S. training with widely varying levels of English speaking skills and exposure. Some have trained in and functioned in English for years; others have just recently mastered the language upon their decision to train in the United States. Most IMGs have a good-to-excellent command of formal English. However, they may have had little exposure to the regional dialects, accents, or the colloquialisms of their U.S. training sites. Because of factors previously described, IMGs may be reluctant to ask people for repetition, for fear of exposing weaknesses. One member of our focus group reported:
I have difficulty in understanding Dr. ____ (a U.S. native) about half the time. When I don’t understand what he is saying, I tend to interpret it negatively.
When giving feedback to IMGs, it is doubly important therefore, for faculty to have the learner paraphrase what they said and make sure that he/she is clear on the behaviors in question and action plan. A learner who comes to premature diagnostic closure and receives feedback about “putting the cart before the horse” may well walk out of the session perplexed about what to do differently next time. There is some evidence that IMGs may underestimate the impact that their accent or language skills have in physician–patient communication. In their qualitative analysis of interviews with IMGs from multiple countries, Dorgan, et al (9) reported that IMGs readily identified their patients’ accents or dialect as a barrier to communication, but generally did not identify their own accent or non-colloquial English as a barrier. When language-related barriers are identified in clinical encounters, it is tempting for supervisors to avoid addressing this in feedback in the mistaken belief that language skill is a nonremediable behavior. This is a misconception, and residents functioning in their second (or third!) language can benefit greatly from accent-reduction classes, education in regional colloquialisms, slang, etc., or increased opportunities to converse in English in non-medical terminology. When communication factors are the subject of feedback, it is doubly important to ascertain the resident’s awareness of the language issues and to address them in an observationally-based, nonjudgmental way.
+
The Role of Program Director
In any training program, although feedback should come from a number of sources, the program director is an important source of formative feedback, summative evaluation, and multiple points in-between. There is some evidence that the program director may assume a greater importance to IMGs than to their U.S.-trained counterparts. Elliott et al. (15) surveyed 180 psychiatry residents (38% IMGs) and asked them to rate in order of importance the factors associated with their level of satisfaction with their training. The top five factors were identical for USMGs and IMGs except for one: The “personal qualities of the program director” was rated in the “top five” by IMGs, but not USMGs. Ellencweig et al (16) replicated this with psychiatry trainees in Israeli programs. Their sample had about 40% IMGs. In their survey, IMGs rated the “professional abilities of the program director” significantly higher in importance than did the Israeli residents. The authors theorized quite plausibly that the IMGs, as immigrants, had a higher need for role models. This information suggests that when feedback is given to IMGs from the program director, it may assume a higher level of importance—and have a greater potential for both positive impact and cultural challenges. Although we are aware of no literature exploring the effect of the supervisors’ country of training on the delivery of feedback, residents in our focus group reported no difference in feedback-associated anxiety when receiving it from U.S.-trained versus international faculty. Brinko (10) reported, in the non-medical educational literature, that feedback is more effective when the source of feedback is lower or equal in status to the recipient. To our knowledge, this has not been formally studied in medical education with either USMGs or IMGs.
+
Feedback and Supervision
From the standpoint of performance-improvement, the most effective feedback comes from direct observation of resident–patient encounters. There is a large degree of overlap between feedback and supervision. A resident’s willingness to actively seek feedback regarding his/her clinical decisions is vital for adequate supervision and patient safety. The July 2011 ACGME Common Program Requirements (CPRs) mandate that residents know when they can act independently and when they cannot. Each program will set its own guidelines for when residents must contact their supervisor and when they can exercise discretion (17). As medical educators, our understanding of feedback is going to have to expand from a technical understanding of effective delivery of feedback to a greater appreciation of what leads residents to be active seekers of feedback or supervision. Pulling from research in social and organizational psychology, Teunissen et al. (2) examined variables associated with the active seeking of feedback among a group of OB/GYN residents on a night shift. They divided feedback-seeking behavior into that of inquiry and monitoring. Feedback-inquiry is the active seeking of or asking for feedback. Feedback-monitoring is taking in information from the environment by observing the behavior of others. Feedback-monitoring can be problematic because it requires people to interpret and derive meaning themselves from presented information. This information is likely to be interpreted in a manner consistent with the residents’ self-perception—which may or may not be accurate. An individual’s propensity to seek feedback is directly influenced by an assessment of the costs and benefits associated with feedback. If an individual feels that asking for feedback may make him/her look incompetent, he/she then would tend to refrain from active inquiry and engage more in feedback-monitoring. All of the residents in the study engaged actively in feedback-monitoring. However, the residents who perceived more feedback benefits also engaged in feedback-inquiry. The residents who mainly perceived the costs of feedback relied more exclusively on monitoring. This group also identified some supervisor characteristics that were positively associated with active feedback-inquiry: approachable, supportive attending physicians who were considerate of the residents’ needs were associated with more perceived feedback benefits and fewer costs. A supervisory style with clear expectations and active efforts to help structure and organize the residents’ workflow was also associated with more active feedback-inquiry. It is reasonable to expect that IMGs will have a higher assessment of the costs of feedback and will benefit from routinely scheduled feedback, and very clear expectations and instructions regarding when to call a senior or attending for feedback/supervision.
Timely, specific, behaviorally-based feedback is a cornerstone of medical education. The ability of any physician (whether trainee or attending) to receive, integrate, and participate in feedback is vital. The ability to actively seek out faculty for supervision and feedback is assuming an even greater importance with the onset of new accreditation requirements. Many IMGs were trained in a hierarchical system where active interaction with attending physicians was not encouraged or tolerated, and the delivery of feedback was often done publicly, in a manner associated with shame and embarrassment. These experiences, combined with the challenge of functioning in a second language and anxieties over exposing weaknesses, present some barriers that make it more difficult for IMGs to participate in feedback inquiry, self-reflection, and reciprocal feedback. Because it is such a crucial part of training, it is in everyone’s best interest to address these barriers wherever we can. These challenges can be mitigated by acknowledging the anxieties learners may have, fostering a learning culture that values feedback as an expected and important part of all learning, ensuring that all (learners and supervisors) are trained in feedback skills, and clear setting of expectations. There is still much we have to learn about how culture, past experience, developmental stage, and delivery technique interact to influence the feedback experience for all learners.