The benefits of information technology in medical training are diverse and well established (1–5). In fact, medical informatics skills acquisition during training is now one of nine objectives in the curriculum project facilitated by the Council on Graduate Medical Education for Undergraduate Medical Education for the 21st Century (UME-21) (6). Moreover, since 1999 the Accreditation Council for Graduate Medical Education’s (ACGME’s) General Competencies for residency training contain the requirement that residents “use information technology to manage information, access online medical information; and support their own education” (7). Additionally, the Alliance for Clinical Education notes in its most recent Guidebook for Clerkship Directors that the “use of information technology is no longer just a mandate from educators or professional bodies, but a vital part of the everyday practice of medical education” (8).
Despite that it is imperative that information technology (IT) instruction be added to existing medical education infrastructures, to our knowledge, with the exception of a single paper (9), there are no publicly available written guidelines or model program descriptions in the literature. Educators, therefore, must individually design their own ad hoc IT system blueprints de novo.
Educators may avoid many IT development pitfalls through pre-planning and forethought, including querying their trainees (medical students and residents) for their thoughts regarding system design. Most of today’s trainees are technologically savvy (10) and, as the central focus and ultimate end users, can provide invaluable feedback regarding initiation, refinement, and maturation of IT systems in medical training.
This topic, trainees’ perceptions regarding technology in medicine, is highly understudied. We found a myriad of small-scale, noncomparative, primarily single department studies involving experiences with a single type of technology in medicine. Moreover, in our manual review scrutinizing over 1,500 references from bibliographic databases, we could only locate two references with a broad-based, general examination of trainees’ perceptions toward technology in medicine (11, 12). Parenthetically, others have noticed a disproportionate number of descriptive studies regarding computer-assisted instruction in medical education within the published literature (13).
In order to assess and explore the richness of trainees’ feedback on the information technology they utilize, we conducted a pilot survey of active medical trainees in two teaching institutions, seeking their firsthand feedback regarding experiences with and perceptions of some of the essential elements of IT in medicine. We defined these essential elements to include their attitudes on its importance, their comfort and proficiency levels, desired didactic topics, learning style preferences, utilization patterns, and suggestions for improvement.
An invitation to an anonymous Web-based survey was e-mailed to 777 medical students and 731 residents (N=1,508) in two different institutions, Eastern Virginia Medical School and the University of California, Davis. This survey utilized the cluster-sampling technique. The survey, by design, was protected from indexing by search engines, such as Google or Yahoo, thus limiting access to only those who received the e-mail invitation containing the survey’s unique Web page address. The study was accepted by the Institutional Review Boards of both named institutions.
The survey consisted of 44 questions (Likert style) in six core target areas regarding IT in medicine, specifically asking for the students’ and residents’ 1) attitudes on its importance, 2) comfort levels, 3) proficiencies and desired didactic topics, 4) learning style preferences, 5) utilization patterns, and 6) suggestions for improvement. These areas were selected based on the literature of noncomparative studies and change management. The estimated completion time was approximately 7 minutes. There were no inducements, financial or otherwise, to complete this survey, nor were there any penalties for nonparticipation. Descriptive statistics were used to analyze the core target areas. The survey can be viewed in Figure 1.
From the 1,508 invitations, there were 160 respondents (11%). Of those, 62% of respondents were medical students (40% pre-clinical), and 33% were residents (one fellow). Five percent of respondents were listed as “other,” and their training level is unknown. To protect respondent anonymity, as per HIPAA regulation, no demographic information was collected.
Attitudes Toward Technology in Medicine
Ninety-six percent of respondents either agreed or agreed strongly with the statement, “Technology skills are important in medical training” (Question 2). Only 4% were neutral, and no one disagreed. Regarding the statement, “Teaching about technology skills should be part of my medical curriculum” (Question 3), 81% of respondents either agreed or agreed strongly, with only 6% disagreeing and 13% remaining neutral. Discomfort with technology in their careers (Question 4) was rare (5%), while 16% were neutral, and 79% were either comfortable or very comfortable. Responses to Question 3 (“Teaching about technology skills should be part of my medical curriculum”) are broken down by level of training (Question 1) and are shown in Figure 2. Residents collectively scored higher on this question (i.e., Strongly Agree + Agreee) than students, and there was a nonlinear increase of 18% from M1 to PGY-4.
Another area of interest concerned discerning the learning preferences of trainees regarding digital media versus traditional printed material. When asked whether they preferred digital or printed media when initially learning a medical topic (Question 5), over half of trainees (51%) preferred printed material (20% preferred digital media, 29% were neutral). When referencing a medical topic one has already learned (Question 6), nearly half (49%) preferred digital media over printed (23% preferred printed media, 28% were neutral). There were no discernable differences between students and residents. The respondents preferred to be taught about technology (Question 7) in small instructor-led groups (38%), and by peers (18%). They did not favor large instructor-led groups (1%).
IT Proficiencies and Desired Didactic Content
Trainees were given the following stem: “I am adequately proficient in technology in medicine, in the following areas” (Question 29). They were then given a list of different types of technologies and asked to self-assess whether they were proficient (“yes”) or not (“no”) with each type. The most highly scored proficiencies were “getting started with a computer” and “basic software issues”; the lowest scoring were collaborative digital systems, career, or project tracking software, and speech recognition technology. Complete results are shown in Figure 3.
So, which technological issues did the trainees feel they needed to learn more about? Survey recipients were posed the following statement: “I think it is important to my medical career that I learn more about [technology type]” (Questions 11–27), and were asked to rank the importance of each technology that followed. Their responses were analyzed, then averaged for each question (Figure 4). Residents tended to express a somewhat stronger interest than students in receiving education upon all listed technologies, except third-party medical software, Web-based course management systems, digital privacy, and career tracking software. This trend was most pronounced for the items “getting started with a computer” and “basic software issues” (factor of 2:1, residents to students, respectively). Figure 5 displays those technologies which might best justify educational resource expenditure.
Frequency of Use Patterns
The trainees were asked to pick their preferred reference media when providing patient care: “In my patient care, of the following references resources, I use ___ the most often” (Question 34). They chose Internet-based (40%) and PDA-based (30%) resources most frequently, followed by peer-attending inquiry (18%), printed books (11%), and other (1%). When separated, both students and residents relied most heavily on the Internet and least heavily on “other.”
Personal Digital Assistants
Given the ubiquity and interest concerning PDAs and their applications in medicine, we felt it necessary to discuss this topic separately. Our first question asked how useful a PDA was in the classroom setting (Question 36). Since the majority of M1 and M2 students spend most of their time in the classroom, we felt they would be best qualified to give an accurate assessment of this and filtered our data accordingly. Approximately 25% of preclinical students agreed (Agree + Strongly Agree), 23% disagreed (Disagree + Strongly Disagree), and 52% were neutral.
M1 and M2 students did not see an appreciable number of patients (Question 33); the majority (i.e., 96% of M1 and 91% of M2) saw five patients a week or fewer. They were therefore excluded from questions regarding clinical issues (Questions 37–43). Clinical trainees (M3 and above) were given the statement “Personal digital assistant availability is critical in patient care” (Question 37), and 62% of respondents agreed (27% Agree + 35% Strongly Agree) with this statement, 20% were neutral, and 18% disagreed (Disagree + Strongly Disagree). Students and residents averaged 63.3% and 61.6%, respectively, on Question 37, and PGY-1s were the highest scoring group at 90.1%.
How are these devices actually being utilized in patient care? We provided the stem: “In patient care, I use my personal digital assistant for ___,” and posed five tasks for the trainees to rate their respective frequency of use (Figure 6). Students and residents alike were most likely to use their PDAs as a “medication-related reference” (students averaged 73.8%, residents 52.5%), and secondmost likely to use them as a “non-medication-related reference source (e.g., disease information)” (students averaged 25.2%, residents 22.1%).
How satisfied were trainees with PDAs in meeting their clinical needs? We asked: “I find that personal digital assistants meet my clinical needs__” (Question 43). More than half (51%) of trainees picked “frequently,” 28% picked “sometimes,” and 15% answered either “infrequently” or “never.” Only 3% felt that PDAs meet their clinical needs “every time.” Students and residents scored similarly (50.2% and 54.9%, respectively).
Trainees were then asked to provide a narrative answer to our last question, “I have found through experience that personal digital assistants need improvement in the area of (if applicable)” (Question 44). The submitted comments were categorized (Table 1).
Opportunities for Improvement
Trainees were given the opportunity to enter multiple comments regarding the frustrations they had experienced with technology in medicine (Question 31). The submitted comments were categorized. Because several comments specifically related to the electronic medical record, these were subcategorized to one of the following: inadequacies of a particular electronic medical record (EMR), the frustrations in having split paper record/EMR, compatibility between different EMRs, and lack of a standard, universal EMR. Table 2 lists their categorized submissions.
Overall, trainees’ attitudes were positive about technology. Almost all respondents felt that technology skills were important in medical training, with a nonlinear increase from first-year medical students to fourth-year residents. Attitudes regarding whether teaching about technology skills should be part of the medical curriculum were robustly positive and consistent across all levels of training. Discomfort with technology in medicine was rare. These are positive findings for educators, and these trainee attitudes may facilitate IT implementation and development. While trainees uncomfortable with IT appear to be in the minority, it would be a mistake to simply disregard their situation. Instead, educators have opportunities to address this through programmatic customization or other necessary support.
The majority of trainees, both students and residents, preferred printed media over digital for initial learning, and the opposite for referencing material already learned. The reasons behind their preferences are unknown, as the trainees were not asked to explain. We suspect that printed media somehow allow easier note-taking and more comfort to someone seated (as may be supported by lackluster sales in the global eBook market). Conversely, digital media is more compact and lightweight, and allows quicker information searches, all clear advantages when seeing patients at the point of care. It is not yet clear how trainees’ preferences will change as digital media become more user-friendly. Educators might best appease their trainees by offering both types of media when applicable or affordable. Fortunately, some medical publishers are beginning to include digital media equivalent with a conventional paper textbook.
Instructor-led small groups were viewed as the best environment in which to receive instruction. In this arrangement, a trainee would receive the guidance and expertise of the instructor but also benefit from the immediate availability of peer assistance. Surprisingly, the second highest preference was “learning from peers,” ranked above most instructor-led variations, which may reflect trainees’ comfort in receiving more individualized instruction, with less extraneous information and freedom from rate-limiting slower learners.
IT Proficiencies and Desired Educational Content
The first step in assimilating IT education for trainees is assessing their proficiencies in the targeted areas. This serves several purposes: a) discerning a starting point for all, b) avoiding redundancy for subjects which trainees have already mastered, c) exposing weaknesses and pairing these with IT topics felt to be most important. Thus, the trainees were posed the following statement: “I am adequately proficient in technology in medicine, in the following areas,” and asked to answer “yes” or “no” to each listed area of technology. It was not surprising that elementary skills, such as “getting started with the computer” and “basic software issues,” were almost exclusively answered “yes” (already proficient), with a tiny percent responding negatively. This suggests that educators might be wise to shelve mass education regarding such basic skills. Remedial instruction should remain available as needed.
In general, the trainees felt it most important to learn about electronic medical records and accessing scientific/medical books, journals or databases on the Internet (or LAN). Third and fourth choices were learning more about personal digital assistant third-party software applications and Web-based course management systems, respectively.
It is unclear why residents tended to express stronger interest than students in receiving education upon all stated technologies except third-party medical software, Web-based course management systems, digital privacy, and career tracking software. We can only speculate. It could be that the student cohort is generally more technologically adept (mirroring societal adaptations over time) than the resident cohort. Or it could be that students are most focused upon, and therefore most interested in, the technologies that are immediately useful in their stage of training. Or residents may have a higher level of responsibility and therefore need more technological assistance. The answer is yet to be determined.
Subjects that were deemed highly important but with which trainees have low proficiency included videoconferencing or telemedicine, electronic medical records, patient simulator software, digital privacy, and career or project tracking/coordinating software.
Frequency of Use Patterns
That trainees preferred Internet-based or PDA-based reference media when providing patient care is important not only to educators but to departmental librarians as they consider what type of media to purchase for reference materials.
Personal Digital Assistants
Preclinical students noted that PDAs were not particularly useful in the classroom. This information may be useful to accepted medical school applicants prior to beginning their first year. On the other hand, over half of clinical trainees felt these devices were “critical” in patient care and met their clinical needs. When separated, clinical students and residents scored equally high regarding their sentiments upon the “critical” nature of the PDA in patient care (PGY-1s were the highest scoring group), the success of the PDA in meeting their clinical needs, and utilization of the PDA as a “non-medication-related reference source (e.g., disease information).” However, while both groups scored high, students were somewhat more likely to use their PDA as a “medication-related reference” (students averaged 73.8%, residents 52.5%). This information may prove useful to trainees considering a PDA purchase.
Though most educators are probably familiar with these patterns, not all administrators may be so informed. It was puzzling that trainees do not use these devices often for patient care logging, as they are sometimes purchased by administrations for this use. We speculate that perhaps they are instead using PC-based software or conventional paper media, as it is somewhat awkward to enter data into a PDA, despite the availability of mobile keyboards for these devices. Paper media may also be the least expensive.
Opportunities for Improvement
It appears that only a minority of trainees are frustrated by technology in medicine, as the top-rated category (EMR—inadequate) was only rated as frustrating by 7% of survey respondents. Even when all EMR-related categories were combined, the complainants only comprised 18% of the survey respondents. The shortcomings of the EMR may be a reflection of their immaturity, and hopefully users will report the problems encountered to the manufacturers so that refinements can follow.
Limitations of This Study
Certain qualitative issues were somewhat difficult to explore exhaustively in a structured survey format, as opposed to interpersonal interviews which, time and space permitting, may have augmented an understanding of the topic and explanatory power. Undergraduate and continuing education research has shown that the accuracy of self-assessment by adult learners has been suboptimal (14–16), limiting the validity of trainee proficiencies (Question 29). The return rate was adequate, albeit relatively small, potentially affecting precision and generalizability. It would be informative to survey trainees from additional institutions. Given that our survey was conducted online, it is possible that some trainees who had trouble using or accessing e-mail may have been inadvertently omitted from our results.
Despite medical training accrediting bodies’ established expectations that medical institutions teach about IT to their trainees, there is little published information regarding practical implementation of system development in this regard, nor are there any substantive reports regarding trainee’s feedback upon technology in medicine. This information is critical for educators if they are to optimize the medical training experience. This pilot study represents a first effort in this regard. Trainees provided insights regarding their experiences with technology in medicine, including attitudes on its importance, comfort levels, proficiencies and desired didactic topics, learning style preferences, utilization patterns, and suggestions for improvement.
Suggestions for Understanding Trainees’ Needs
If IT solutions in medical training are under consideration, query trainees for their input, preferably in the product selection stage.
When developing IT education for trainees, assess their proficiencies in the target area beforehand in order to discern a suitable starting point for all and avoid redundant training.
Solicit input from trainees regarding their desired didactic topics within IT in medicine.
Routinely gather feedback from trainees regarding the IT solutions they use, including the technologies' strengths and weaknesses, usage patterns, needed improvements, and frustrations the trainees encounter.
Identify trainees who are uncomfortable with IT in medicine and address their distress through elucidation of noxious elements, education, programmatic customization, or other necessary support.
Special thanks to Dr. Donald M. Hilty, University of California, Davis, for his suggestions and assistance.