Residency training programs in the United States are entrusted with the responsibility to educate and graduate competent physicians. This responsibility is receiving increasing national attention both within both residency training regulatory bodies and residency training programs (1). The Work Group on Training and Education of the American Academy of Child and Adolescent Psychiatry has responded by proposing a set of sample core competencies for residents in child and adolescent psychiatry training that are consistent with the goals of the Accreditation Council for Graduate Medical Education (2). The multifaceted nature of these competency expectations adds significant complexity to the always challenging task of assessing the knowledge, skill, and attitude attainment of trainees. The purpose of this paper is to set forth sample guidelines for assessment of residents’ knowledge, skills, and attitudes in each core competency.
Similar to previous discussions of this issue (3), we believe assessment methods should be:
Residency training is a unique adult learning experience in which the trainee is a student, an apprentice, and an employee all at the same time. A resident is asked to balance the demands of pursuing an education while providing clinical services. Assessment of educational achievement and clinical performance thus requires a collaborative dialogue between the resident and the faculty, which is mediated by the residency training director (4). The following recommendations for assessment are designed to foster the necessary agreement among faculty and residents about what to learn and the commitment to ensure that learning takes place.
The Resident Should Share Responsibility for Identification and Remediation of Deficiencies in Knowledge, Skills, and Attitudes With the Training Director and Faculty
Residents must establish a trajectory of self-learning and the desire to continually assess their abilities in residency that will persist over the resident’s career. Traditionally, training directors and faculty have assumed the primary responsibility for monitoring the progress of trainees. However, this approach does not require residents to actively demonstrate to themselves their mastery of competencies and reflect on areas of relative weakness. While training directors have an ethical and administrative obligation to ensure that graduates of residency training programs are competent when they complete their training, residents are responsible for developing their knowledge, skills, and attitudes throughout their careers to ensure the public trust and the safe, effective treatment of children and their families.
The Assessment of Residents Should Be an Ongoing, Open, and Predictable Process
Collaborative assessment in which the resident is an active participant should occur in an atmosphere of openness (3). Together, the resident and training director should document mastery of the core competencies and meet regularly to discuss and compare impressions of the resident’s progress toward achieving these competencies. In addition, faculty evaluators should be expected to review and discuss feedback they submit to the training director with the resident. To facilitate this process, the training director should assist faculty in developing skills that are necessary in order to concisely identify and constructively communicate areas of strength and weakness. Identification of strengths helps the resident appreciate the skills, knowledge, and attitudes that they will build on through training and practice. Recognition of weaknesses helps the resident understand areas on which to focus and build skills. This process should occur regardless of the overall competency attainment of any individual resident. Sometimes the faculty is uncomfortable giving negative feedback. Faculty members may need assistance in mastering this ability and will need the support of their training director to ensure that constructive criticism is appreciated by the resident.
A Wide Range of Evaluators Should Be Utilized to Ensure the Most Accurate View of Resident Competency
Efforts should be made to assess competency through direct observation of clinical performance by traditional as well as nontraditional evaluators. In each training program there are certain individuals who are easily identifiable as appropriate evaluators of residents’ performance, such as ward and service attending physicians, supervisors, and didactic seminar leaders. However, traditional evaluators often must base their assessments on a resident’s self-report of a clinical encounter or other second-hand means. Direct observation of the resident’s clinical work, through live observation of interviews or video taped review of clinical encounters should be included in any resident’s assessment process. Additional assessments from other individuals whose roles in a resident’s education are significant should also be obtained. These individuals may be supervising attending physicians during on-call situations, research mentors, nonphysician members of care teams (i.e., psychologists, social workers, nurses, and teachers) as well as patients and their families.
Residents Should Be Encouraged to Demonstrate Competency Through a Variety of Formats
Individuals learn knowledge, skills, and attitudes through a variety of educational methods such as assigned reading, lectures, clinical demonstrations, discussion, interactive demonstrations, direct clinical experience, and feedback on their performance. There are a variety of formats through which residents can demonstrate competency, including written multiple-choice or essay examination, oral examination, written or oral presentation, didactic discussion participation, and actual clinical performance. Although residents should have a basic ability to communicate learning through all of these formats, multiple assessment formats should be provided so that an accurate evaluation of competency can be based on either a composite of several assessments or the resident’s best performance among these formats. This will ensure that residents are able to demonstrate competency through both universal assessment measures and through formats that best reflect their abilities.
The Goal Is for 100% of Residents To Achieve the Core Competencies
Core competencies within any program are written to be comprehensive but achievable and represent a flexible minimum standard for child and adolescent psychiatrists graduating from training programs. Nonetheless, mastery of the core competencies requires an active partnership between resident and training director to ensure success. The assessment process is crafted not as a method of stratifying residents into a hierarchy of performance capabilities, but rather as a method of enabling all residents to emerge from training as competent child and adolescent psychiatrists.
The following tables describe sample methods of assessment and their use in child and adolescent psychiatry residency training. A multi-method, multi-informant method is likely to be most effective for training programs. Each method will need to be modified to fit the unique culture of each training program (5). While some are readily quantifiable, others are more subjective. Some depend upon the specifics of rotations and training sites, and others depend upon the individual interests, goals, and qualities of each resident. A number of assessment methods (t1) reflect a formal evaluation process. Such methods are oriented toward directly evaluating residents’ competency in a reproducible fashion. Some examination formats (e.g., objective structured clinical examinations, simulated patients, computer based simulations) are methods that will benefit from pilot project development efforts, efforts of several programs working together, and collaborations between programs and national professional organizations. Other methods (t2) require a program to encourage independent resident activity and assess the competencies demonstrated. The final group of assessment methods (t3) makes use of a check list format and expands the range of evaluations to many of those who work with a particular resident whose evaluation can assist in the formation of a comprehensive portrait of resident competency.
Training programs will need to develop their own compendium of assessment methods in order to achieve the most practical, comprehensive, and equitable process for their trainees and faculty. Each training program must evaluate their own resources and utilize methods that work. Most programs will appreciate that the use of several methods with assessments by many differing evaluators will be most effective.
Regardless of the methods of assessment utilized by a training program, each program will need to address the following key issues to ensure a successful assessment experience for each trainee: