New residency training directors are often faced with multiple competing tasks, such as meeting Accreditation Council for Graduate Medical Education (ACGME) Psychiatry Program Requirements and achieving successful completion of residency review committee (RRC) site visits while managing operational aspects of the program. Preparation for a site visit requires exacting compilation of comprehensive and accurate information about all aspects of a training program. The effort entailed in this important academic function can be very time consuming, taxing, and anxiety provoking.
The purpose of a site visit is to verify information provided by a program as offered in its completion of the Program Information Form (PIF). This verification by the site visitor requires a review of compliance with the program requirements and the ascertainment of any inconsistencies between the description in the PIF and feedback from trainees, faculty and observations made in visitation to training sites. As one residency training director remarked, "It is like an internal revenue service audit attesting to the accuracy of your returns." The "returns" here are, of course, such things as the comprehensive program description, formal training records and briefings with the "witnesses," i.e., residents and faculty.
The following information is based on the authors’ experience, notably multiyear workshops at the annual meetings of the American Association of Directors of Psychiatry Residency Training (AADPRT) and comments from attendees. They also share insights from their careful review of the essentials of psychiatry residency training and Program Director’s Reference Guide to ACGME and Residency Review Committees as well as general insights from their combined 25 years experience as residency training directors. Perhaps the most essential offering is a practical guideline for completing a PIF. This must reflect a well-maintained and comprehensive program with formal trainee and faculty records relevant to the general residency training program. For more details and specific information including that for psychiatry subspecialty training requirements, see the ACGME website.
Where to begin? At the beginning! Here is a list of steps:
Review Current Program Requirements for Residency Training by the RRC for Psychiatry (+1)
Make certain that program requirements will be in effect on the date of the site visit. Also review revised requirements that are due to go into effect within the next year.
Learn the Meaning of Action Terms in the Description of Program Requirements
Action terms provide guidance for adherence to essentials of training. This is especially important for new directors of training, but even experienced hands can be surprised by new terminology.
1. The terms in this program requirement document designated as "must," "shall" or "essential" indicate that something is required or mandatory or to be done without fail; i.e., an absolute requirement (
+2). For example, here is a description in the program requirements (
+1): "A psychiatric first postgraduate year must include at least 4 months in internal medicine, family practice, and/or pediatrics. This training must be in a clinical setting that provides comprehensive and continuous patient care." This is a mandatory requirement for which anything at all less than described invites a negative citation.
2. Parts of the program requirement are described as "so important that their absence must be justified" (
+2). Thus a program operated without certain key elements can be accredited but only if such absence is plainly justified. For example the RRC requires that "Programs should be conducted under the sponsorship of an institution that meets the Institutional Requirements" (
+1). Your program technically need not be so conducted, but if not, anything less than a very clear explanation invites negative citation.
3. Parts of the program requirement which are designated as "desirable" or "highly desirable" are "aspects of educational programs that are not mandatory but are considered to be very important" (
+2). Thus there is a stipulation that "a program may be cited for failing to do something that is "desirable" or "highly desirable" even if it is not actually mandatory (
+2). For example the essentials state that "It is desirable that residents have didactic learning and supervised experiences in the delivery of psychiatric services in public sector and in managed care health system" (
+1). If this requirement is not met, the program may be cited, particularly if it readily could be included in the range of training experiences.
4. Other descriptions in the program requirement such as "suggested" or "highly suggested" are "used to indicate that something that is distinctly urged rather than required." A program will not be cited for failing to do something that is only "suggested" or even "strongly suggested" (
+2).
Carefully Review the Prior Letter of Notification by the RRC About the Status of Your Program
The prior letter of notification by the RRC should include information about accreditation status (full, probation or provisional), length of training, maximum number of approved residency positions and residency positions approved for each year of training (
+2). If there was an adverse accreditation action, the justification for that decision is also provided in this letter. This justification includes applicable elements of noncompliance with the program requirements as well as noncompliance with institutional requirements (
+2).
This letter should also contain approximate time of the next site visit of the program (usually 5 years for full accreditation but could be shorter even without an adverse action). However, presence of citations frequently shortens this period. If there was any deficiency cited, note the description of these deficiencies with "must," "should," "desirable" or "highly desirable" or "suggested" "highly suggested," as these terms have similar connotations as in the program requirement and should be dealt with accordingly. Clearly document how these deficiencies are being addressed. If the RRC was notified of an action plan, note both the RRC response and the outcome of your efforts. It is valuable to document feedback from residents and from their annual retreat report to support the positive impact of any changes on the quality of residency education and training experiences.
Use Previously Completed PIF, If Not Different From the Current PIF, and Program Requirements as Working Documents and Update Regularly
If the current PIF and program requirements are different from the previous one, complete the current PIF and use that as a working document.
Review of the "Handbook of Psychiatry Residency Training" (
+3), books on "Core Competencies for Psychiatric Education"(
+4) and Core Competencies for Psychiatric Practice (
+5) can provide guidance as to residency administration, competency-based curricula development, implementation and monitoring, resident supervision and evaluation, program needs, role of faculty and other training-related matters.
Carefully update the Program Information Form (PIF) for Internal Review—the formal process undertaken, usually in mid-cycle, by the Graduate Medical Education Committee (GMEC) of the sponsoring institution. The GMEC typically appoints a subcommittee, which reviews the PIF in light of program requirements.
For accurate completion of PIF, each clinical site supervisor, course director and other responsible faculty should provide information for their area of participation, e.g., participating institution (an institution where rotations are required or most residents opt to take electives), rotations, educational experiences, supervision, learning goals and objectives. Have the clinical site directors describe site-specific clinical experiences, breadth of patient population, supervision and the evaluation process at the middle and at the end of rotation and how core competencies (
+4—
+6) are evaluated. Prior to transmitting information for internal review, the rough draft of PIF should be brought to the Residency Training Committee (RTC) for review and feedback. Optimally, residents from each year of training are also given opportunity for feedback. It works well if the document is reviewed section by section by the members of the training committee.
1. It should be noted that most of the site visitors are not psychiatrists and that psychiatrist special site visitors are used generally for either new programs or may signal that the program has significant issues.
2. RRC in their letter specifies the date of the visit and the professional background of the site visitor. The training director may make a request for particular time for site visit based on when most residents and key faculty are present.
3. As soon as actual date for site visit is confirmed, send a memo to key residents and faculty notifying that no one should be away or take vacation at that time.
Have an organizational structure to support successful site visit.
1. Organize a site visit task force chaired by the training director and supported by the division director(s), associate training director, subspecialty training directors, training site directors and other key faculty member(s), a resident from each year of training, and the chief and associate chief resident. It is suggested to also include resident(s) who are complainers and often critical of the program, listen to and address their concerns and to help them buy- in to the site visit process.
2. The training director should develop a timeline for meetings well within the deadline for PIF completion.
3. All task force members should be provided a copy of the psychiatry RRC Program Requirement applicable to the date of site visit, a copy of the updated (requirements if these are going in to effect within a year), a copy of the previous RRC letter of notification about status of the program, a copy of the PIF which was submitted for the Internal Review and updated.
4. The program director should orient task force members to documents related to program requirements, updated PIF and the last RRC letter and share information about their role and responsibility in preparation and successful execution of the site visit.
5. Each member of the task force should review the content of the PIF, recommendations of internal reviewers and RRC letter. This will assist the task force members in providing an informed input.
6. If any deficiency was cited in a previous RRC communication or internal review, it is essential that the training director should have documentation that this has been remedied with a proper plan of correction.
7. An impending site visit can often and appropriately be used as leverage to garner resources needed for program compliance.
The PIF is the most important document completed by the program director. All RRC actions are based on verification (or lack thereof) of information in this document at the time of the site visit.
The PIF must contain an accurate and complete description of the educational program and residency training experiences. The site visitor basically verifies and clarifies information in the PIF and that the PIF reflects reality.
Upon appointment as a training director, one should review the previous PIF completed at the time of previous site visit, compare the current and previous PIF and program requirements. If there are no differences between the two, use previous PIF as your working document. Share the latest update of the most current PIF for the upcoming site visit for the review by site visit task force.
Access the updated Part 1 of the PIF from the ACGME website (www.acgme.org). This Web Accreditation Data System (Web ADS) requires an assigned user name and password. The three sections of this data system include the general information about accreditation, program information, as well as the information about participating institutions and resident complement.
Also from this website under the section RRC for psychiatry Part 2 (Initial Accreditation), a template PIF can be downloaded as a either a Word or WordPerfect document. Answer all questions fully and avoid redundancy. However, working documents should be based on current PIF and program requirement format. If any requested information is not applicable or is unavailable, provide a clear explanation. Any changes as to overall program, faculty, staff, affiliations, rotations, clinical educational and training experiences, faculty complement and other elements of the program changes should be reflected in this working document. Under the direction of residency training director, a residency training coordinator can collate updated information and any changes about any part of the PIF and assure adequate secretarial support for transcribing these descriptions utilizing uniform format and style.
The following suggestions for completion of the PIF are tried and true.
1. Do write a program description without any ambiguity and in a way that a site visitor who does not have any firsthand knowledge of the program will get a good understanding of the program structure and function.
2. Do not consider a site visitor as a conduit of information between you and the RRC—communicate with the RRC directly and in an ongoing manner.
3. Do not attach to PIF unneeded materials like residency training brochures, annual resident retreat reports, etc.
4. Do have a PIF reflect the current reality about the education and training experiences. Include all rotations on the block diagram. If there are multiple tracks in the program, include block diagram for each track.
5. Do review and proofread the document and have all associate training directors, a senior colleague and the chief resident in the program do the same to assure completeness and accuracy. Assure all required signatures are entered, that pagination is correct, and strive for clarity, currency and consistency. Any inconsistencies raise a red flag and may trigger more in-depth review.
6. Do provide the following documents as attachment to the PIF: a copy of the outcome of the institutional site visit review, if it transpired after the last psychiatry residency review; a letter of commitment from sponsoring institution; all letters pertaining to interinstitutional agreements; a table of content of the resident permanent record file; a statement of program goals and objectives; a copy of the due process document and a narrative description of any due process action since the last accreditation survey; a blank sample page of patient and supervision log; and a statement of JCAHO accreditation status of each affiliated institution and an explanation if any participating institution is not JCAHO-accredited.
7. Do complete the PIF description so it demonstrates compliance with the ACGME psychiatry residency program requirements as well as compliance with remediation of any deficiency cited in the previous RRC letter.
The resident records should include data from application to graduation as well as communications after graduation. This data provides validation of resident’s education and training experiences. Comprehensive and complete training records may include application and admission documents, documentation as to clinical rotations, evaluations, biannual letters of evaluation by the training director and content, completed supervision and patient logs from all rotations including elective and selective rotations, psychotherapy supervision, examinations scores from Psychiatry Residents In Training Examination (PRITE) or other in-house examinations including "mock boards," contract and benefits, vacations and sick leaves including maternity and paternity leaves, intra and extramural letters of recommendation/commendations, any local and national honors/awards and letter of program completion, due process proceedings (if any) and letter(s) to and from ABPN. This documentation in the training records will be extremely helpful in the future when responding to the enquiries from American Board of Psychiatry and Neurology, hospital credentialing and privileging coordinators. This file should contain the section below.
1. Application documents should include: completed application blanks; curriculum vita; personal statement; copy of the medical school; transcription; and letters of support. For residents transferring from another training program the file must have a letter from previous residency training director attesting to what required rotations have been successfully completed.
2. Admission documents should include: interview feedback and recommendations from the faculty and resident interviewers; recommendation of the admission committee; residency training director’s letter of acceptance; letter of acceptance by the trainee; and a copy of the signed contract.
3. Clinical rotations and evaluations must include: comprehensive monthly log of rotations for each year of training; resident-specific block diagram of all rotations including research and clinical electives for all years of training; confidential midrotation evaluation of core competencies of the resident for each rotation with areas of strengths and weakness identified and the action plan to remedy those weaknesses (if any); end of the rotation evaluation of core competencies; evaluation of case conference presentation skills; evaluation of journal club presentation skills; evaluation of continuous case conference presentation skills; and evaluation of resident’s performance on "mock boards."
4. Training directors meeting with the residents should take place every 6 months; document residents’ progress; include discussion as to strength and weaknesses and remediation strategies (if needed); copy of the PRITE and other internal examination scores are reviewed; and letter of summary of progress by the training director.
5. Resident supervision and patient logs:
For supervision log, residents should be made responsible to obtain in writing documentation of their weekly clinical, psychotherapy, research and elective supervision from the supervising faculty. Making residents responsible to seek supervision documentation gives a clear message as to its importance as well as gives them ownership about this process. A copy of this document is included in the trainee’s file. This information is discussed during resident training director six monthly meeting assuring adequacy of supervision.
Resident’s patient logs containing information about age, sex, race, socioeconomic status, type of treatment intervention, frequency of contacts with patient, duration of contact, number of sessions and name of supervisor is placed in trainees’ file. This provides information about trainee’s depth, breadth and diversity of clinical experiences.
6. Copy of the Essentials of Psychiatry Residency training in force at the time of resident admission.
7. Licensure information should include: a copy of the Medical License (educational or permanent); and a copy of the Drug Enforcement Administration (DEA) certificate.
8. Immunization data, e.g., Hepatitis B vaccination data.
9. Copy of Basic Cardio-Pulmonary Resuscitation and Advance Cardiac Life Support (ACLS) certifications.
10. Special achievements include: letters of recommendations and awards (1. research accomplishments: poster presentation, publications, grants, awards, etc. 2. Local or national honors, e.g., by APA, American Association of Directors of Psychiatry Residency Training [AADPRT], and American College of Psychiatrists [ACP] honors, e.g., PRITE or Laughlin Fellows).
11. Insurance certificates for each year in the program.
12. Vacation and sick leave utilization data sheet: This data will help with accurate accounting of the training time and Medicare audit that determines the direct and indirect Medicare support for the teaching hospitals for resident services.
13. Approved moonlighting activities. In our program moonlighting is allowed for residents with permanent (not educational) medical license and only after approval by residency training committee and may require consideration toward work hours.
14. Residents are asked to compute their weekly work hours and report any deviations from the resident work hour policy. Also describe if there were any fatigue factors.
15. Resident-related miscellaneous correspondence, e.g., with American Board of Psychiatry and Neurology.
16. Due process proceedings (if any).
17. Letters of advancement.
18. Letter of graduation attesting to resident’s competency and respect for ethical boundaries.
1. Updated Curriculum Vita.
2. Copy of the Mission-Based Management documentation by the faculty. Some medical schools and departments of psychiatry use Mission-Based Management tool to track clinical, research and education faculty productivity and performance, for rewarding the productive faculty and to plan about the future. Among others this tool is currently being used at Creighton University School of Medicine and University of California-Davis School of Medicine (
+7).
3. Responsibilities of the faculty for the training program: training site(s) direction; course(s) directed, which includes curriculum design, arranging speakers for the course, e.g., "course on neurosciences"; lectures(s) taught, e.g., Advances in Psychobiology of Bipolar Disorders; and supervisory responsibilities for specific rotations, e.g., emergency psychiatry or morning report.
4. Evaluations by residents of the faculty as to: affiliated site direction; clinical rotation supervision; psychotherapy supervision; research supervision; elective(s) supervision; course direction, e.g., neurosciences; teaching skills relating to specific lecture presented by the faculty; and annual resident retreat feedback pertaining to the faculty.
A copy of these evaluations should be provided to the Chairperson of the Department. This clearly supports faculty member’s commitment to the education mission of the medical school and may help the faculty member in the annual evaluations and decision about promotion and tenure.
The program records file may contain the following information and make it available to the site visitor on the day of site visit: a letter from RRC regarding last site visit; interim correspondence to and from residency review committee; copy of the current (and yet to be implemented within the year) program requirements; yearly program evaluations through resident retreat reports; interinstitutional affiliation agreements; program goals and objective and goals and objectives for each rotation on the block diagram; residency training policies and procedure; graduate medical education institutional policies and procedures; sample of evaluation forms; master residency training schedules; data from the American Board of Psychiatry and Neurology as to performance of the graduates from the program; program-related correspondence with the office of the dean of graduate medical education; correspondence with affiliates and training sites, training committee agendas and minutes; and miscellaneous correspondence.
The training director should meet with residents a few additional times to share the site visit review process, information about the content of PIF, and their role as consumers of education and training during the site visit. It is important to listen to their concerns and address those prior to the site visit. One of the training directors remarked "tell the residents it is a time for program improvement and not a time to air dirty laundry."
Prior to site visit, address all issues related to resident morale. Often call schedule is one of those issues. If there are major morale issues within the program, it is advisable to brief the department chair and seek alternate channels of resolution.
As a host, the training director must assure smooth execution of this site visit as per schedule. The following recommendations adapted from ACGME Program Director’s Guide (2) may be helpful.
1. Assign the residency training coordinator to ensure smooth execution of the site visit on the scheduled date.
2. Contact all individuals (faculty and residents) scheduled to meet with the site visitor the day before the site visit to assure their availability for meetings and interviews and monitor the schedule for the day.
3. Arrange a parking space for the site visitor.
4. Have suitable space and telephone equipment available for the site visitor. This will allow the site visitor privacy for meetings, interviews, personal calls, to write notes and review materials, as well as have lunch with the residents.
5. Assign a secretary on a stand-by basis for clerical help.
6. Leave a telephone number to call for resource in case any special need arises for the site visitor.
7. Have a copy of the updated Program Records file available for review.
8. Have permanent records files of residents available for inspection, which should include current residents, residents who have graduated from the program and residents who may have been subjected to due process action.
9. Have a sample patient record available showing resident involvement. To comply with HIPPA, patient consent should be obtained.