The Accreditation Council on Graduate Medical Education (ACGME's) outcome-based accreditation approach began by endorsing six general competencies, then establishing a skill-acquisition continuum, creating measurement tools, and initiating a support network (1). Implementing the competencies and matching training objectives to levels of competence requires creativity in developing programs, strategies, and assessment mechanisms (2–4). In recognition of the distinct challenges posed by the practice-based learning and improvement (PBL) competency, Ogrinc et al. (5) developed a program that emphasizes the use of knowledge in daily practice, the importance of patients' experience in clinical encounters, and skills identified by Epstein and Hundert (6). These skills include clinical reasoning, judgment, and management of ambiguity. Although Ogrinc's program originated in general and community medicine, the emphasis on patient experience resonates with a well-established line of discourse in psychiatry (7, 8). The link between patients' perceptions of illness, concurrence with treatment plans, and outcomes has been amply documented (9–11); current policy statements emphasize that patients' values, perceptions, and goals are integral to clinical practice (12, 13).

Ironically, as the ACGME is encouraging adaptation of clinical practice to the needs of patients, initiatives to standardize clinical care are gaining momentum (14–16), and their impact on residency programs is likely to increase. PBL programs are particularly vulnerable to initiatives that gauge quality of care by adherence to standards, policies, and evidence-based practices (17, 18). The implication, that lack of adherence demonstrates biased clinical judgment and leads to poor quality of care, is actually embedded within the logic of evidence-based practices (19). Efforts to encourage or even mandate conformance to normative standards are supported by a myriad of studies whose conclusions suggest that when clinicians are left to their own discretion—whether as students or expert practitioners—they exhibit biased or “suboptimal” judgment (20–23).

The approach described in this article responds to the challenge posed by current initiatives. It adapts Orginc's PBL framework and accommodates Eddy's (24) Evidence-Based Decision-Making (EBDM) proposal to formalize clinical decision-making practices, invoke treatment guidelines as representations of general clinical knowledge, and incorporate patient-specific factors into treatment decisions. The value of guidelines as aids to clinical judgment has not been fully appreciated, and these are commonly misconstrued as standards of care (25–27). Along with recommendations about which treatments have the greatest overall likelihood of producing a good outcome, guidelines include assessment procedures and decision-points; they propose treatment alternatives and emphasize that procedures and algorithms be adapted to the needs of patients (28, 29). Key features of the PBL framework that are presented and tested in this article include knowledge about treatment guidelines, knowledge about patients' perceptions of illness, and skills of judgment and reasoning that are requisite to incorporating patient-specific information into treatment recommendations under a variety of conditions, including ambiguous cases (30).

The approach uses the Texas Implementation of Medical Algorithms Schizophrenia Module (TMAP), a flexible, mature, and widely-disseminated switching guideline (29, 31). Case material, in the form of vignettes, is presented to third, fourth, and fifth year psychiatric trainees, who represent different levels of training. The cases focus on the roughly 15% to 25% of patients who have not attained a positive response after two full courses of antipsychotic treatment (32). The TMAP gives clinicians three switching options for this treatment-resistant subset, assuming that they have been treated unsuccessfully with two second-generation agents: 1) introducing a first-generation (i.e., “typical”) antipsychotic agent; 2) switching to a third second-generation (or “atypical”) agent; or 3) clozapine therapy. Continuing the current treatment, the principal alternative to these options, is contrary to the guideline.

Case Vignettes

Study participants reviewed six vignettes that fell within the schizophrenia spectrum, either paranoid type or chronic-undifferentiated. To simplify, the cases involved no comorbid conditions or Axis II diagnoses. The vignettes contained the following information:

PBL Study Design and Data Collection

Following the progressive scheme proposed by Dreyfus and Dreyfus (36), Orginc et al. used a four-level framework, with each level having distinct objectives and evaluation measures. The framework, with training objectives and assessment procedures adapted for the pilot study, is summarized in Table 1. The first level was assessed by summary questions about the treatment guideline and the importance of patient perception factors. The second level was assessed by questions about the influence of the guideline and patient perception factors on specific treatment recommendations. The third level was assessed by examining how case complications influenced treatment recommendations. The fourth level, which requires well-integrated knowledge about the guideline and patient-specific factors, was assessed by examining how recommendations were influenced by incongruence, or inconsistency, between clinical facts and patient experience. Congruence was crossed on complications, which enabled congruence and complications to be assessed independently.

Study Participants

A group of 22 trainees at one psychiatry residency program who had previous experience in treating persons with schizophrenia volunteered to participate in the study. They were paid $100 for participating. The study was approved by three different Human Investigation Committees, representing the residency program, the site where the study was conducted, and the funding source. These committees required convenience sampling to ensure that the decision to participate would be truly voluntary. The study was not integrated with the training program so as to ensure that responses to the questionnaire items would have no influence on residents' status or progress. At the time of the study, the residency program did not include didactic training in schizophrenia treatment guidelines, clinical decision-making, or patient perception of illness.

The 22 participants consisted of 9 women and 13 men, with mean ages of 33.5 (standard deviation [SD]: 7.7) and 31.8 (SD: 6.4), respectively. Of the 22, 9 were 3rd-year residents; 9 were 4th-year residents, and 4 were fellows; 16 residents (9 men, 7 women) were Caucasian; 6 residents (4 men, 2 women) were Asian, and 1 Caucasian man identified himself as Hispanic.

A limited number of assessment findings are reported for each training objective. Inferential tests reported below are based on generalized estimating equation (GEE) models (37), which are appropriate for analyzing small samples consisting of continuous or categorical dependent variables in repeated-measure designs.

First-Level Objective

Because participants had no previous experience with the TMAP, summary questions inquired into their opinions about its value as a decision aid; 18 of 22 rated it somewhat or very helpful in making treatment decisions, and 11 rated it as moderately or very important in deciding whether to recommend a switch. (Both ratings were 3+ on a 4-point scale.) Positive symptoms, the guideline's principal progress assessment criterion, was listed as a top three influence on the treatment decisions of 18 participants; 10 listed a patient perception factor as a top-three influence, and 8 listed both.

Second-Level Objective

Treatment switches were recommended at an 84% rate, with 37% of the switches recommending a first-generation agent and 40% recommending clozapine. However, the switch rate was only 33% when the importance of positive symptoms was rated 4-or-less, the importance of the clinician's progress assessment was 3-or-less (both on a 5-point scale), and patient's progress rating was the most important patient experience factor.

Third-Level Objective

Subjective assessments of case complexity were affected by case complications (Wald χ² [2]=26.45; p<0.001), and complications had a significant effect on the endorsement pattern (Wald χ² [2]=6.36; p=0.036). Compared with stressors alone, stressors plus negative coping led to more switch recommendations (91% versus 75%), more second-generation recommendations (30% versus 11%), and fewer clozapine recommendations (25% versus 34%).

Fourth-Level Objective

In the most complex cases, where negative coping is combined with incongruence, there were proportionately more first-generation recommendations and fewer clozapine recommendations (Wald χ² [1]=11.463; p=0.001); 7 of the 22 participants recognized that the patient's perception of illness directly influences the clinical outcome; these 7 had substantially higher clozapine recommendation rates in congruent cases (80%, versus 22% for the other 14), but the difference in incongruent cases was relatively small (20% versus 13%).

This pilot study introduced a practice-based learning approach, based on the framework developed by Orginc et al. (5), that responds to the dilemma posed by initiatives to standardize care and limit discretion in clinical practice. Training objectives were inspired by Eddy's EBDM (24). They emphasize requisite skills of incorporating patient experience into clinical encounters and linking general knowledge to patient-specific data. Even though the participants had not received formal training in either guidelines or patient perceptions of illness, they understood the guideline as a decision aid and factored patient experience into treatment recommendations. However, even advanced residents were inclined to trade-off rather than incorporate clinical and patient perception data, suggesting that they may benefit from a program that begins with didactic training in connecting disparate sources of information and culminates with treatment recommendations in complex cases. Recognition, a crucial skill in decision-making (38, 39), is fundamental to connecting clinical facts with patient experience. Especially in incongruent cases, this skill may be integral to PBL (40).

The procedure described in this report can be modified in a variety of ways. It can adapted for online presentation, used for various disorders, and applied to various decision points. General knowledge can be represented by decision trees, policies, or evidence-based practices, as well as guidelines. Actual cases can be substituted for vignettes to evaluate higher levels of competence. Cases can be expanded to incorporate clinical data, such as laboratory findings or negative symptoms, and to address medical or psychiatric comorbidities. Vignettes can be modified to depict a deteriorating condition or progress at the threshold of a positive response. Treatment options can include evidence-based psychosocial treatments; they can emphasize under-prescribed alternatives and enable treatment decisions to be postponed for a specific period (e.g., 1 week, or until the patient's housing problem has been resolved).

The study has several limitations that can be addressed in future projects. First-level assessment did not address knowledge of treatment-resistant schizophrenia; efficacy of alternative therapies, including clozapine; the nature, construction, and implementation of treatment guidelines; and procedures for assessing patient perception of illness. The participant population was drawn from a single residency program that may not be typical of other training programs and facilities. Mean subjective complexity assessment for the most complex cases (maladaptive coping with incongruency) was 3.37 (SD: 0.144) on 5-point scale, suggesting that complexity can be increased by including comorbidities and medical complications, information pertinent to engagement and treatment alliance, and availability of community and programmatic support. The TMAP emphasizes clinical discretion in selecting alternatives to the current treatment, but the study did not compare participants' first and second treatment choices. Finally, ambiguity can be assessed more precisely by targeting inconsistency between specific aspects of the patient's perception of illness (for instance, between treatment goals and progress assessments), and by factoring in insufficient or unreliable information.

Implications for Education and Practice

A focus on specific decisions, however important, should not overlook the unfolding nature of treatment for chronic conditions. Clinical judgment is required in recommending a course of treatment, and also in determining whether to monitor the situation, modify the current regimen, or change it altogether, refer to another provider, postpone until a stressor clears or a support system is put in place, or defer, because recommending a treatment change may lead the patient to disengage. The claim that expert judgment is compromised by sundry sources of bias draws principally from studies that originate in the decision sciences and compare clinical decisions against logical and mathematical norms (20). That expertise more aptly resembles practical problem-solving than logical optimizing, and that a crucial skill is to recognize when a decision is required, has been noted by advocates of naturalistic decision-making (41, 42). The investigators' current work focuses on recognition in chronic illness decision-making.

Perhaps the most penetrating commentaries emanate from works that bring a historical and cultural perspective to bear on the relationship between expertise and decision-making. Prominent among them is a paper published by Dreyfus in the same year that his skill-acquisition model was introduced. This article gives his model a broad, holistic sweep and portrays practical understanding as something that cannot be spelled out theoretically or represented by factual knowledge (43). The model, as integral to ACGME's initiative, is guided by the principle that skills are the foundation of competence, and competence is the product of a lifelong commitment to learning through practice. This principle is also apparent in EBDM, which depicts patient-specific information as the product of a shared background between patient and physician and the catalyst for knowledgeable decisions. In affirming the primacy of practice, the Dreyfus model serves as a cogent reminder that general representations of knowledge are derived ultimately from practical understanding.