The American Association of Medical Colleges (AAMC) defines mission-based management as "a process for organizational decision making that is mission-driven, ensures internal accountability, distributes resources in alignment with organization-wide goals, and is based on timely, open and accurate information" (+1). The development of mission-based management programs has been the focus of a number of academic medical centers. However, it remains a controversial area since faculty are concerned that quantitative measurements do not accurately reflect their activities and may adversely affect their salaries, promotions, workload, and allocation of support. Moreover, faculty consider relative value units (RVUs), although generally acceptable as a metric for clinical productivity, as inappropriate for application to educational and research productivity.

Nevertheless, since the mid-1990s, American medical schools have been developing mission-based systems to prioritize their resource allocations and align faculty efforts with the mission of their school. Schools have been particularly interested in mission-based systems as a means of highlighting and enhancing their educational mission, since it is generally believed that this component of the mission has been greatly overshadowed by the revenue-generating capabilities of the clinical and research missions (+2—+15). One issue of Academic Medicine (Volume 77, 2002) devoted an entire section to this topic. As Mallon and Jones emphasized, when describing the experience of 41 medical schools using mission-based systems, schools that have been successful in implementing mission-based systems view the metrics as a process not only for allocation of resources, but more importantly for mission enhancement (+13). To date, there are only a few published methods describing productivity measures for nonclinical missions, such as education (+2—+6). Likewise, only a few of the published mission-based management systems have attempted to integrate the information from all missions for an individual faculty member (+7, +8, +16—+19). Mission-Based Reporting (MBR) at the University of California Davis School of Medicine (UC Davis) was designed with two guiding principles: simplicity and practicality. In contrast to more comprehensive systems, which account for all faculty activities and each unit of time on a daily, weekly, or monthly basis or the use of journal impact factor or article citation index (+4, +8, +11, +12), UC Davis MBR was designed to annually highlight major academic activities only. The system is easily transportable to other institutions. Its technical development and implementation are described in greater detail elsewhere (+14).

In 1998, Mission-Based Reporting was developed at UC Davis under the direction of two of the authors (TFA, LPH), who collaborated with faculty work groups charged with devising quantitative (time-based) and qualitative metrics that identify efforts in clinical work, research, education, and administrative/public service activities. Because academic requirements differ by faculty rank and series, as described in a subsequent section of this report, scoring algorithms that weighted activities to meet various criteria were developed.

The UC Davis Department of Psychiatry and Behavioral Sciences was an active participant in all aspects of MBR development, from faculty serving on initial work groups to piloting and full implementation. Aggregate MBR results create a unique department-wide picture of faculty productivity that would not be possible by any other means. The chairperson (R.E.H) and department administrator (N.C.S) use the annual results to assess faculty progress toward mutually defined objectives, consistent with the faculty member's academic series, to incentivize faculty effort through financial rewards and to plan future distributions of department funds.

Mission-Based Reporting is a password-protected, web-based, self-report system in which faculty members provide responses to specific questions about their activities for each of the University's four missions: research, teaching, clinical work, and administrative/public service. The basic data entry module is designed in four sections: 1) an identification section that classifies an individual faculty member by department, rank, series, and primary assignment (e.g., basic scientist, clinician, administrator); 2) an activity section in which faculty members enter annual activities quantitatively; 3) an evaluation section for qualitative assessment of performance on each mission; and 4) an automated summary "report card." As part of each department's annual budget submission, the dean's office requires projections (targets) of individual faculty effort by mission; thus, "budgeted" or "targeted" percent effort for each mission is entered before the faculty member begins to input data.

In actual practice, most faculty members complete online entry of their individualized MBR forms prior to the time of their annual evaluation meeting with their chairperson, which occurs in February for the University's Department of Psychiatry and Behavioral Sciences. This required meeting is intended principally to provide an individual assessment of the faculty member's academic progress and to set goals for the subsequent year, which begins July 1. Additionally, salaries for the subsequent year are set, and changes in activity or responsibility are noted. The chairperson and faculty member also review the completed year-to-date MBR form. Specific items such as publications, grants, and teaching assignments are verified. Any irregularities or discrepancies are reconciled and corrections are made to the online MBR database. The chair and faculty member agree to the subsequent year's allocations of effort for each of the missions, totaling 100%, becoming the targets upon which the following year's actual productivity will be gauged.

Based on the faculty member's responses to items in the activities section, the MBR program calculates a time estimation for each mission by means of weighted relative value units (RVUs) embedded in the program using rank- and faculty-series-specific algorithms. Mission-specific summary scores are created based on the sum of the entries. The scoring algorithm assumes a 50-hour workweek as representative of 100% total effort, and thus an "actual" total greater than 100% would indicate that a faculty member's total effort has exceeded the 50-hour workweek. Each mission's summary score is then compared to the previously entered "targeted" or "projected" percent effort estimated for that faculty member. A grand total representing the sum of the four mission totals is also computed and compared to the targeted total of 100% effort set previously.

For the qualitative metrics in the evaluation section, a list of standards reflects the quality of work in each mission. The standards are ranked from 0 to 5, and thus the evaluation section summarizes the quality of the faculty effort that was scored previously by the quantitative algorithms. The teaching mission is evaluated from the perspective of student and peers and is averaged to achieve a single evaluation score for teaching. Faculty members enter one standard for each mission. The MBR guidelines and a sample of a completed MBR form may be viewed at http://informatics.ucdmc.ucdavis.edu/mbrsample/mbrdoc.html and http://informatics.ucdmc.ucdavis.edu/mbrsample/mbrsample.html.

Mission-Based Reporting is a Java Servlet 2.1 compatible system, and thus can be implemented on a large variety of server environments. User summary reports are provided as Portable Document Format (PDF) files constructed "on-the-fly" from information in the database and submitted to the web browser whenever a user requests the report. A printed record is available for each individual faculty member, and printable summary reports compile data for the entire school, by department and by faculty rank and/or series across departments. This has yielded thought-provoking insights into the division of clinical and academic workload among these different populations and has enabled analysis according to the School of Medicine's stated academic objectives for rank and series (+16). Questions, however, have been put forth, such as: Do research faculty actually perform more research than clinical faculty, or do professors conduct more or less teaching than assistant professors?

The University of California employs medical school faculty in five academic series. The "regular" (tenure) and "in-residence" (IR, nontenure) series are considered research series. The "clinical-X" series (professor of clinical psychiatry) is equivalent to the teacher scholar/clinical investigator series in other universities. The "salaried clinical" series is for master clinicians and clinical supervisors who conduct little formal teaching or investigation. The "adjunct" series is for grant-funded research scientists. Each series has a unique set of criteria and expectations for promotion in rank from assistant professor to associate professor and then professor. All series faculty are expected to have their teaching evaluated at regular intervals by both students and peers. They are also expected to engage in public service. However, the amount and type of research productivity, teaching, and clinical activity differ among the three research-intensive series and the teacher-scholar and master clinician series.

Mission-Based Reporting was piloted using a small group of faculty in 1998—1999, tested on eight departments in 1999—2000, and has been in school-wide use for the past 2 academic years (2000—2001, 2001—2002). Faculty from the psychiatry department participated in the pilot phase, in the test year and in the two subsequent implementation years. The school-wide results for the two implementation years have been reported previously (+16). That report validated individual faculty MBR profiles with successes and failures of promotions, determined independently, without MBR information, by the customary faculty peer review committee. Successful productivity, as determined by MBR accounts, was highly correlated with success rates of independently assessed academic advancement. Similarly, low productivity of MBR was associated with higher denials by the promotions committee (+16).

This report describes the outcomes of 2 years of experience with MBR in the UC Davis School of Medicine and the psychiatry and behavioral sciences department. To further assess face validity of the MBR scoring algorithms in this department, two of the authors (L.P.H., T.F.A.) re-reviewed each psychiatry faculty member's MBR scores with the department chairperson (R.E.H.). The chairperson confirmed that his knowledge of faculty members' productivity in each mission corroborated with their MBR summary profile. That is, faculty who were the most productive research scientists had the highest scores in research, and the most prolific teachers scored highest in teaching, while the most active clinicians received the highest clinical scores. Conversely, faculty whose MBR summaries were below average had low productivity in those missions in the chairperson's view.

All 26 departments of the UC Davis School of Medicine participated in MBR in 2000—2001 and 2001—2002. A summary of completed MBR reports by rank and series is shown in +Table 1. Seventeen departments, including the psychiatry and behavioral sciences department, had 100% compliance in 2001—2002. Overall, 95% of all faculty completed records in 2001—2002, and 85% completed records in 2000—2001.

From the cumulative data for all departments in each year, overall summaries for the school were generated, which compared the average percent of targeted effort by mission to the average percent of actual effort. Comparing the school's targeted and actual performance with the psychiatry department's targeted and actual performance over the 2-year span is portrayed in +Table 2.

On average, all departments in the medical school met or exceeded the school's initial, budgeted targets in all missions in both years. In fact, the total percent of actual effort for the school in 2001—2002 was 153%, up from 135% in 2000—2001. This average "actual" department effort reflects the workload of the entire faculty, which includes generalists and specialists (e.g., surgeons, emergency room physicians, internists, pediatricians, psychiatrists).

The psychiatry and behavioral sciences department employs full-time faculty in all five of the faculty series. The data are based on 100% of responses from 35 faculty members in 2001—2002 and 31 faculty members in 2000—2001. As shown in +Table 2, the shift in targeted research effort (29% to 41%) conformed to a dean's directive to increase research productivity. +Figure 1 illustrates the total and actual results for the medical school and the psychiatry and behavioral sciences department during the 2001—2002 academic year.

As noted in +Figure 1, the department exceeded its 2001—2002 targets in all four areas. The department’s total effort (126%) indicates that, on average, faculty members worked 76 hours per week, more than the 50-hour per week considered representative of 100% effort. This figure includes writing manuscripts and grants and preparing lectures at home, all embedded in the MBR scoring algorithms. For illustrative purposes, faculty effort profiles by academic series for the 2001—2002 academic year are illustrated in +Figure 2.

Tenure track faculty members spend most of their time in research and the least time in clinical activity. The nontenure research faculty and the teaching scholar/clinical investigator faculty spend almost equal amounts of time in research, while the latter spend more time in teaching and clinical work than do research faculty. Adjunct faculty spend no time in clinical activity. The master clinicians spend most of their time seeing patients in clinical activity and in teaching medical students and supervising residents while performing patient care activities. Tenure track and teaching scholar faculty provide most of the department’s administrative/public service activity. An analysis of these departmental results might suggest to the chairperson that the nontenure research faculty need to increase their research productivity at the expense of either clinical or teaching efforts. The tenure track research faculty and the grant-funded adjunct faculty appear to be on track for academic advancement, although the tenure track faculty members, on average, seem too heavily engaged in administration. Closer analysis of these results demonstrated that two tenure track faculty members raised the overall average because they had undertaken full-time administrative positions. In another example, one master clinician was heavily engaged in clinical research, suggesting that a change in series to the teaching scholar/clinician investigator series may be appropriate.

In their survey of 41 medical schools involved in development of mission-based management systems, Mallon and Jones, noted that the common challenge for medical schools was the lack of a culture of self-evaluation using quantitative data and the faculty belief that their activities defy quantitative assessment (+13). Our experiences at UC Davis, in general, support these observations. An exception has been psychiatry, which used a quantitative tool for evaluation of performance before the MBR system was implemented. In fact, many quantitative items in MBR were borrowed and adapted from this tool. Consequently, it has not been difficult to obtain compliance from faculty in a department that embraced a quantitative culture.

Greater than 100% effort for a faculty member or a department is not astonishing since most faculty members have expanded clinical workloads related to a highly competitive managed care market. In addition, recent expansion of the medical school’s research grants and contracts and a change in the curriculum to small-group teaching requiring the involvement of more faculty have increased research and teaching efforts.

Several of the surgical subspecialties and some of the basic science departments, in contrast, continue to demonstrate reluctance about MBR. Strategies to change the culture and improve compliance include working closely with department chairpersons and demonstrating to them the utility of the system; continually modifying weighting algorithms in response to faculty feedback; simplifying faculty entry of data; and establishing strong, continuing support from the dean.

To generate further appeal, we have recently developed a secure data repository called My InfoVault for each faculty member that, when maintained will yield a current CV in multiple formats, an NIH biosketch, and/or an official form used in the University’s appointments and promotions, while simultaneously populating the field of MBR. This one-stop data entry module should encourage more faculty participation, given the redundant time, effort, and data elements required of these various formats.

Mission-Based Reporting must be viewed as a dynamic, interactive system that is beneficial to both faculty and administration. Despite efforts to gain acceptance, some faculty continue to believe that Albert Einstein would never have advanced had Mission-Based Reporting been in place during his time. Although this may be true, outstanding individual merit will always be recognized. MBR is merely a tool, an imperfect one at best, that assists chairpersons and school administrators in supporting specific objectives and progress of individual faculty and in advocating distinct missions.

Additionally, there have been other limitations to implementing MBR. Although MBR provides qualitative evaluations as well as quantitative assessments, the former are much more difficult to measure objectively. Research prestige, educational excellence, clinical expertise, and prominence in professional organizations are often viewed as capricious benchmarks. Consequently, department chairpersons and the medical school have not utilized the qualitative metrics effectively to date. Further work in this area is necessary.

Does MBR differ significantly from current practices in many universities where faculty provide written or verbal self-reports of their annual activities? The answer is yes. Mission-Based Reporting is automated and standardized. All faculty members respond to the same set of structured questions, making it feasible to compare performance across faculty within a given department and across departments. Automation provides an opportunity for data analysis from multiple perspectives: junior faculty versus senior faculty, faculty in different faculty series, individuals from one year to the next, and expected performances versus actual performances (+16).

The original intention of MBR at UC Davis was to provide department chairpersons with a management tool to aid them in their annual review of faculty performance. In addition, comparing aggregate profiles for the department to those of the medical school can assist a chairperson in defining the department’s role and contributions to the institution’s missions and to assess productivity. The profiles generated for a department graphically illustrate whether faculty efforts are headed in the direction the leadership intends, where deficits or excesses may lie, and what types of faculty may be needed to fill gaps.

The authors wish to acknowledge the valuable assistance and support of Benny Poon, programmer, and Michael Hogarth, M.D. of the Department of Medical Informatics, UC Davis Health System.