Since Engel’s 1960 publication of "A Unified Concept of Health and Disease" (+1) and his 1977 publication entitled "The Need for a New Medical Model: A Challenge for Biomedicine" (+2), psychiatry and other medical specialties have recognized the need to approach patient care in an integrative rather than reductionistic fashion, now known as the biopsychosocial model. The biopsychosocial model is a widely accepted style of synthesizing information to develop a case formulation. Over the years, numerous authors have discussed issues related to the model (+3—+5), teaching the model (+6—+10), and the model’s application (+11). Some authors have debated whether an actual model exists (+12, +13), as opposed to a theory or an idea. To our knowledge, however, there are no publications that rebuff the concept of incorporating biological, psychological, and sociocultural factors to formulate cases and promote better health outcomes for individual patients as well as to gain better understanding of patients and more insight into the disease process. Indeed, few have or would dispute Len Sperry’s assertion that "the ability to conceptualize and write succinct case formulations is considered basic to daily clinical practice" (+14).

In addition to the benefits that competent case formulation brings to clinical practice, the American Board of Psychiatry and Neurology (ABPN) includes case formulation as part of the grading criteria for board certification of psychiatrists (+15). McDermott identified the inability to integrate "bits of data gathered into a coherent, multidimensional formulation and management plan that tests a higher level of thinking that goes beyond interview technique or observational capacities" (biopsychosocial formulation) as a key factor for candidates who fail the ABPN oral board examinations (+16).

Despite the near universal acceptance of the importance of the biopsychosocial approach, little has been documented regarding the ability of psychiatrists in general or resident psychiatrists in particular to generate a competent biopsychosocial formulation. Likewise, there is very little information available addressing the question of how to evaluate formulations. The purpose of this study was to examine the ability of psychiatric residents in multiple programs to competently formulate a case using the biopsychosocial model as measured by portfolio entries. Secondarily, we determined whether a specific intervention could improve residents’ biopsychosocial formulations.

One part of the study was a nonexperimental component describing residents’ ability to generate formulations. The second part was a pre-post design for one institution to evaluate the effects of an intervention. Residents developed a portfolio entry for biopsychosocial formulation according to specific guidelines (see + Figure 1) that were developed by psychiatric faculty, as previously published (+17, +18). The study received approval from the Institutional Review Board, and those entries from institutions outside the researchers home institution were anonymous.

Psychiatric residents in postgraduate years 1—4 were asked to participate. The residents came from three different programs in the 2000—2001 academic year. In the 2001—2002 academic year, residents from a fourth program also participated. The residency programs ranged in size from 10 to 24 residents and were geographically dispersed in the United States. One residency program was from the Northeast, and one was from the Midwest. Two were from the South. Of the four participating sites, only one program described itself as "dynamically oriented," with the other three institutions self-identified as "balanced" or "biologically oriented."

Guidelines for developing a portfolio entry for biopsychosocial formulation were provided and are shown in +Figure 1. Each portfolio entry consists of the formulation itself, a self-reflective cover letter describing the residents’ rationale for selecting the particular case, and any relevant supporting documents such as the initial psychiatric assessment, laboratory data, radiographic data, psychological testing, etc. Residents were encouraged to select a case that they managed within the last academic year that best demonstrates their ability to develop a biopsychosocial formulation.

Using the scoring rubric described by O’Sullivan et al. (+17, +18) and illustrated in +Figure 2, entries were scored by two trained raters who were board-certified psychiatrists and not on the clinical faculty of any of the participating programs. The raters were blinded to the patient name, resident name, year in training, and training institution. Our previous research with this data indicated that two raters were needed to obtain a reliability of 0.8 (generalizability coefficient) for relative decisions about residents (+18).

In one institution, an intervention was implemented after the first year of the study. The intervention involved grand rounds discussions aimed at increasing faculty awareness of residents’ poor performance on portfolio entries. The residency program director met with individual faculty members to discuss the importance of increasing attention to teaching this skill. The program director met with residents in both group and individual settings to stress the value of being able to formulate cases using the biopsychosocial model. These interactions resulted in increased attention to teaching and learning biopsychosocial formulation skills. Some faculty actually incorporated the biopsychosocial formulation into the curriculum and devoted didactic time to case formulation within clinical rotations. Providing appropriate feedback regarding resident performance in developing biopsychosocial formulations was also a focus of the faculty.

Residents outside the University of Arkansas for Medical Sciences (UAMS) were provided with entry guidelines and scoring rubric as well as examples of entries rated at different levels of competency and an informed consent form approved by the Institutional Review Board at UAMS. The study was considered exempt at UAMS and no informed consent was required from UAMS residents. The guidelines provided a description of a portfolio, a definition of a biopsychosocial entry, and instructions for assembling an entry including specific information that was essential for rating (+Figure 1). After assembling their entry, residents were asked to complete a short questionnaire regarding their experience in developing a portfolio entry. Prior to submission of the entries to the principal investigator, all patient and resident identifiers were obscured. Residency program directors were asked to collect the entries, assign a unique resident identifier (code number) to each entry, and to complete a cover sheet indicating the resident’s level of training, to be used in data analysis, and the identifier code. The unique identifier could be matched to individual residents only by the residency program directors so that they could return the evaluated entries to the appropriate residents. The entries were then submitted to the principal investigator at UAMS where the PGY level was removed before the entries were rated. The raters assigned an overall score to each entry and provided narrative comments to give feedback to the residents. Scored entries were then returned to the various institutions to be given back to the residents.

For the data analysis, each entry was given a single score, which was the average of the two scores. We conducted an analysis of variance to determine if there was a difference across levels of training. Lastly, we did a paired samples t test for those residents for whom we could match data in both years. Significance was set at 0.05. To evaluate the curricular impact, we also calculated an effect size for each institution across two years. The effect size is the difference in means divided by the standard deviation. In this case, we used the root mean squared for the standard deviation to account for the differences in standard deviation between the two years. Effect size greater than 0.8 was considered large (+19).

In 2000—2001 there were 33 entries and in 2001—2002 there were 46 entries. +Table 1 summarizes results overall and by institution. In both years no postgraduate year level had an average rating of 3.0 (competent). In fact, in both years we saw performance gradually decline across the years of training. However, the difference in performance across the four years in training was not statistically significant. The one institution implementing an intervention had substantial improvement in average scores for each year in training. Residents in year one of training rose from 2.67 (N=3) the first year to 3.60 (N=5) the second year. In the same program, PGY-2 residents’ scores increased from 2.50 (N=5) to 3.40 (N=5), PGY-3 scores rose from 2.20 (N=5) to 3.17 (N=6), and fourth year scores increased from 2.50 (N=2) to 3.33 (N=3). For each institution with two years of data, the effect size at each PGY level is in +Table 1. Only in the institution with the intervention was the effect size what would be considered large. Additionally, for the nine residents with matched entries across the two years, there was a significant improvement (p=0.006) from an average of 2.5 to an average of 3.6, indicating competency (see + Table 2).

This study indicates that although biopsychosocial formulation is generally accepted as an essential skill of practicing psychiatrists, there are deficits in psychiatric residents’ ability to generate biopsychosocial formulations as indicated by less than competent formulation averages. The results may be related to program emphasis and curriculum. As psychiatry has become more biologically oriented with the emergence of sophisticated imaging techniques, the exponential growth in our knowledge of neurotransmitter systems, and the explosion of psychopharmacologic agents, some residency programs may have begun to drift away from the psychological and sociocultural aspects of patient care. Indeed, only one of the four participating sites was self-described as "dynamically oriented."

The effect of this shift in educational focus was evident in the narrative comments provided by the raters. For entries scoring below competent, the most commonly assigned score was at level 2 (see +Figure 2). Comments most often cited "lack of integration" as the reason for assigning a low score. In other words, our residents may be eliciting the necessary information for the development of competent formulations, but they lack the ability to synthesize a coherent and succinct formulation. Failure to recognize the interplay and impact each component of the biopsychosocial model has on the other two was a commonly cited weakness as well.

Another possible explanation for the low scores, although indirect, could be the fact that formulation is no longer considered essential documentation by insurance companies or the Joint Commission on Accreditation of Healthcare Organizations (+20,+21). With the ever-increasing burden of documentation to ensure reimbursement and accreditation, educational focus at many training sites has drifted away from "unnecessary" documentation, including biopsychosocial formulations.

The fact that resident portfolio entry scores trended downward with more years of training is a concern since more advanced residents would be expected to have greater understanding and more experience in formulating cases. This trend may be reflective of greater emphasis on interviewing skills and diagnostic issues early in training with subsequent years more devoted to specialized treatments and more complex psychopharmacology. It may also be evidence of less interest, motivation, or investment by upper-level residents who are approaching the end of training and are facing other issues such as securing employment, written board preparation, etc.

Residents surveyed indicated that the most difficult part of completing the portfolio entry was actually formulating the case. Thirty-three residents completed the survey over 2 years, and of those, nearly half reported either difficulty organizing the data they had collected into a formulation or doing so in a concise manner. This was consistent with the raters’ narrative comments as previously discussed. One resident even indicated that he/she was uncertain as to what a biopsychosocial formulation should contain. Those responding to the survey also indicated that by focusing on the portfolio entry, they realized areas for needed improvement in their overall assessment of patients and the need for more experience in formulating cases. Other reported benefits were increased understanding of their patients and better supporting information for their diagnoses and treatment plans. A total of 66.7% responded that they agree or strongly agree that portfolio entries represent a good tool for demonstrating competency in biopsychosocial formulation, five were uncertain, two disagreed, and four did not answer the question.

Besides the small number of participants, there are several other limitations to this pilot study. One limitation was the inability to match data across years in training by individual residents. Only one program used the same unique code number for individual residents in both years of the study. This was also the program that implemented the intervention. There was an overall increase in the scores of that program, but the improvement in individual PGY-1 residents across the two years of the study illustrate the effect of the intervention as did the substantial improvement across years of training within the program. This method assumes that in general the first year classes in each year of the study were comparable in their baseline level of knowledge of the biopsychosocial model and baseline skill in case formulation. A second limitation is that there was no standardization of the curriculum in the participating residency programs. Despite the fact that standardized guidelines for the portfolio entry were provided, the methods of teaching the biopsychosocial case formulation was left up to each individual site. Residents may, therefore, have had very different experiences with regard to the biopsychosocial model and case formulation. Additionally, though faculty and board-certified raters were assumed capable of formulating cases using the biopsychosocial method, they were not formally assessed using this or any other method. Future studies may include matching individual resident performance across years of training, comparing scores from the portfolio method of assessing biopsychosocial formulation to ABPN oral board examination outcomes, and evaluating faculty members’ ability to competently formulate cases using the biopsychosocial model and portfolio method.

Biopsychosocial formulations appear difficult for residents. Residents indicate that they recognize their own weakness in biopsychosocial formulations and have indicated the desire for more training in this area, which they view as beneficial in the care of their patients. Across institutions performance is low in this area. Yet, the intervention in one institution indicates that we can improve resident competency in this area.

This paper was supported by the Stemmler Medical Education Research Fund, National Board of Medical Examiners, Grant #60-9899, Patricia O’Sullivan, Principal Investigator and the Teaching Scholars Program at the University of Arkansas for Medical Sciences