Acad Psychiatry 30:470-479, December 2006
doi: 10.1176/appi.ap.30.6.470
© 2006 Academic Psychiatry
Students’ and Residents’ Perceptions Regarding Technology in Medical Training
Gregory W. Briscoe, M.D.,
Lisa G. Fore Arcand, Ed.D.,
Terence Lin, M.D.,
Joel Johnson, M.D.,
Aanmol Rai, M.D. and
Kevin Kollins, B.A.
Received January 31, 2006; revised June 12, 2006; accepted June 14, 2006. Drs. Briscoe, Fore Arcand, Rai and Mr. Kollins are affiliated with the Department of Psychiatry and Behavioral Sciences, Eastern Virginia Medical School, Norfolk, Virginia. Drs. Lin and Johnson are affiliated with the Department of Psychiatry and Behavioral Sciences, University of California, Davis, Sacramento, California. Address correspondence to Dr. Briscoe, Eastern Virginia Medical School, 825 Fairfax Avenue, Norfolk, VA 23507; briscogw{at}evms.edu (e-mail).
|
ABSTRACT
|
OBJECTIVE: This pilot study provides firsthand feedback from medical students and residents in training regarding their perceptions of technology in medicine. METHOD: The authors distributed an e-mail invitation to an anonymous Web-based survey to medical students and residents in two different U.S. training institutions. RESULTS: Respondents unanimously expressed that technology skills were important in medical training and felt it most important to learn about electronic medical records and accessing scientific information on the Internet. At the point of patient care, trainees’ preferred reference sources were the Internet and PDA, in that order. Most clinical trainees felt PDAs were critical in patient care and met their clinical needs, and they were most likely to use them for medication reference. The majority of trainees preferred printed media over digital media for initial learning, but the converse for referencing. Instructor-led small groups were viewed as the best environment in which to receive instruction. CONCLUSIONS: Trainees in medical education are technologically savvy and provide invaluable feedback regarding initiation, development and refinement of technological systems in medical training.
|
INTRODUCTION
|
The benefits of information technology in medical training are diverse and well established (1–5). In fact, medical informatics skills acquisition during training is now one of nine objectives in the curriculum project facilitated by the Council on Graduate Medical Education for Undergraduate Medical Education for the 21st Century (UME-21) (6). Moreover, since 1999 the Accreditation Council for Graduate Medical Education’s (ACGME’s) General Competencies for residency training contain the requirement that residents "use information technology to manage information, access online medical information; and support their own education" (7). Additionally, the Alliance for Clinical Education notes in its most recent Guidebook for Clerkship Directors that the "use of information technology is no longer just a mandate from educators or professional bodies, but a vital part of the everyday practice of medical education" (8).
Despite that it is imperative that information technology (IT) instruction be added to existing medical education infrastructures, to our knowledge, with the exception of a single paper (9), there are no publicly available written guidelines or model program descriptions in the literature. Educators, therefore, must individually design their own ad hoc IT system blueprints de novo.
Educators may avoid many IT development pitfalls through pre-planning and forethought, including querying their trainees (medical students and residents) for their thoughts regarding system design. Most of today’s trainees are technologically savvy (10) and, as the central focus and ultimate end users, can provide invaluable feedback regarding initiation, refinement, and maturation of IT systems in medical training.
This topic, trainees’ perceptions regarding technology in medicine, is highly understudied. We found a myriad of small-scale, noncomparative, primarily single department studies involving experiences with a single type of technology in medicine. Moreover, in our manual review scrutinizing over 1,500 references from bibliographic databases, we could only locate two references with a broad-based, general examination of trainees’ perceptions toward technology in medicine (11, 12). Parenthetically, others have noticed a disproportionate number of descriptive studies regarding computer-assisted instruction in medical education within the published literature (13).
In order to assess and explore the richness of trainees’ feedback on the information technology they utilize, we conducted a pilot survey of active medical trainees in two teaching institutions, seeking their firsthand feedback regarding experiences with and perceptions of some of the essential elements of IT in medicine. We defined these essential elements to include their attitudes on its importance, their comfort and proficiency levels, desired didactic topics, learning style preferences, utilization patterns, and suggestions for improvement.
|
Method
|
An invitation to an anonymous Web-based survey was e-mailed to 777 medical students and 731 residents (N=1,508) in two different institutions, Eastern Virginia Medical School and the University of California, Davis. This survey utilized the cluster-sampling technique. The survey, by design, was protected from indexing by search engines, such as Google or Yahoo, thus limiting access to only those who received the e-mail invitation containing the survey’s unique Web page address. The study was accepted by the Institutional Review Boards of both named institutions.
The survey consisted of 44 questions (Likert style) in six core target areas regarding IT in medicine, specifically asking for the students’ and residents’ 1) attitudes on its importance, 2) comfort levels, 3) proficiencies and desired didactic topics, 4) learning style preferences, 5) utilization patterns, and 6) suggestions for improvement. These areas were selected based on the literature of noncomparative studies and change management. The estimated completion time was approximately 7 minutes. There were no inducements, financial or otherwise, to complete this survey, nor were there any penalties for nonparticipation. Descriptive statistics were used to analyze the core target areas. The survey can be viewed in Figure 1.
|
Results
|
Survey Participants
From the 1,508 invitations, there were 160 respondents (11%). Of those, 62% of respondents were medical students (40% pre-clinical), and 33% were residents (one fellow). Five percent of respondents were listed as "other," and their training level is unknown. To protect respondent anonymity, as per HIPAA regulation, no demographic information was collected.
Attitudes Toward Technology in Medicine
Ninety-six percent of respondents either agreed or agreed strongly with the statement, "Technology skills are important in medical training" (Question 2). Only 4% were neutral, and no one disagreed. Regarding the statement, "Teaching about technology skills should be part of my medical curriculum" (Question 3), 81% of respondents either agreed or agreed strongly, with only 6% disagreeing and 13% remaining neutral. Discomfort with technology in their careers (Question 4) was rare (5%), while 16% were neutral, and 79% were either comfortable or very comfortable. Responses to Question 3 ("Teaching about technology skills should be part of my medical curriculum") are broken down by level of training (Question 1) and are shown in Figure 2. Residents collectively scored higher on this question (i.e., Strongly Agree + Agreee) than students, and there was a nonlinear increase of 18% from M1 to PGY-4.
Preferences in Learning
Another area of interest concerned discerning the learning preferences of trainees regarding digital media versus traditional printed material. When asked whether they preferred digital or printed media when initially learning a medical topic (Question 5), over half of trainees (51%) preferred printed material (20% preferred digital media, 29% were neutral). When referencing a medical topic one has already learned (Question 6), nearly half (49%) preferred digital media over printed (23% preferred printed media, 28% were neutral). There were no discernable differences between students and residents. The respondents preferred to be taught about technology (Question 7) in small instructor-led groups (38%), and by peers (18%). They did not favor large instructor-led groups (1%).
IT Proficiencies and Desired Didactic Content
Trainees were given the following stem: "I am adequately proficient in technology in medicine, in the following areas" (Question 29). They were then given a list of different types of technologies and asked to self-assess whether they were proficient ("yes") or not ("no") with each type. The most highly scored proficiencies were "getting started with a computer" and "basic software issues"; the lowest scoring were collaborative digital systems, career, or project tracking software, and speech recognition technology. Complete results are shown in Figure 3.
So, which technological issues did the trainees feel they needed to learn more about? Survey recipients were posed the following statement: "I think it is important to my medical career that I learn more about [technology type]" (Questions 11–27), and were asked to rank the importance of each technology that followed. Their responses were analyzed, then averaged for each question (Figure 4). Residents tended to express a somewhat stronger interest than students in receiving education upon all listed technologies, except third-party medical software, Web-based course management systems, digital privacy, and career tracking software. This trend was most pronounced for the items "getting started with a computer" and "basic software issues" (factor of 2:1, residents to students, respectively). Figure 5 displays those technologies which might best justify educational resource expenditure.
View larger version (35K):
[in this window]
[in a new window]
|
FIGURE 5. Trainees "Not Proficient" Rate Multiplied by Its Perceived Importance for Each Technology in Questions 11–27
|
Frequency of Use Patterns
The trainees were asked to pick their preferred reference media when providing patient care: "In my patient care, of the following references resources, I use ___ the most often" (Question 34). They chose Internet-based (40%) and PDA-based (30%) resources most frequently, followed by peer-attending inquiry (18%), printed books (11%), and other (1%). When separated, both students and residents relied most heavily on the Internet and least heavily on "other."
Personal Digital Assistants
Given the ubiquity and interest concerning PDAs and their applications in medicine, we felt it necessary to discuss this topic separately. Our first question asked how useful a PDA was in the classroom setting (Question 36). Since the majority of M1 and M2 students spend most of their time in the classroom, we felt they would be best qualified to give an accurate assessment of this and filtered our data accordingly. Approximately 25% of preclinical students agreed (Agree + Strongly Agree), 23% disagreed (Disagree + Strongly Disagree), and 52% were neutral.
M1 and M2 students did not see an appreciable number of patients (Question 33); the majority (i.e., 96% of M1 and 91% of M2) saw five patients a week or fewer. They were therefore excluded from questions regarding clinical issues (Questions 37–43). Clinical trainees (M3 and above) were given the statement "Personal digital assistant availability is critical in patient care" (Question 37), and 62% of respondents agreed (27% Agree + 35% Strongly Agree) with this statement, 20% were neutral, and 18% disagreed (Disagree + Strongly Disagree). Students and residents averaged 63.3% and 61.6%, respectively, on Question 37, and PGY-1s were the highest scoring group at 90.1%.
How are these devices actually being utilized in patient care? We provided the stem: "In patient care, I use my personal digital assistant for ___," and posed five tasks for the trainees to rate their respective frequency of use (Figure 6). Students and residents alike were most likely to use their PDAs as a "medication-related reference" (students averaged 73.8%, residents 52.5%), and secondmost likely to use them as a "non-medication-related reference source (e.g., disease information)" (students averaged 25.2%, residents 22.1%).
How satisfied were trainees with PDAs in meeting their clinical needs? We asked: "I find that personal digital assistants meet my clinical needs__" (Question 43). More than half (51%) of trainees picked "frequently," 28% picked "sometimes," and 15% answered either "infrequently" or "never." Only 3% felt that PDAs meet their clinical needs "every time." Students and residents scored similarly (50.2% and 54.9%, respectively).
Trainees were then asked to provide a narrative answer to our last question, "I have found through experience that personal digital assistants need improvement in the area of (if applicable)" (Question 44). The submitted comments were categorized (Table 1).
Opportunities for Improvement
Trainees were given the opportunity to enter multiple comments regarding the frustrations they had experienced with technology in medicine (Question 31). The submitted comments were categorized. Because several comments specifically related to the electronic medical record, these were subcategorized to one of the following: inadequacies of a particular electronic medical record (EMR), the frustrations in having split paper record/EMR, compatibility between different EMRs, and lack of a standard, universal EMR. Table 2 lists their categorized submissions.
|
Discussion
|
Attitudes
Overall, trainees’ attitudes were positive about technology. Almost all respondents felt that technology skills were important in medical training, with a nonlinear increase from first-year medical students to fourth-year residents. Attitudes regarding whether teaching about technology skills should be part of the medical curriculum were robustly positive and consistent across all levels of training. Discomfort with technology in medicine was rare. These are positive findings for educators, and these trainee attitudes may facilitate IT implementation and development. While trainees uncomfortable with IT appear to be in the minority, it would be a mistake to simply disregard their situation. Instead, educators have opportunities to address this through programmatic customization or other necessary support.
Preferences in Learning
The majority of trainees, both students and residents, preferred printed media over digital for initial learning, and the opposite for referencing material already learned. The reasons behind their preferences are unknown, as the trainees were not asked to explain. We suspect that printed media somehow allow easier note-taking and more comfort to someone seated (as may be supported by lackluster sales in the global eBook market). Conversely, digital media is more compact and lightweight, and allows quicker information searches, all clear advantages when seeing patients at the point of care. It is not yet clear how trainees’ preferences will change as digital media become more user-friendly. Educators might best appease their trainees by offering both types of media when applicable or affordable. Fortunately, some medical publishers are beginning to include digital media equivalent with a conventional paper textbook.
Instructor-led small groups were viewed as the best environment in which to receive instruction. In this arrangement, a trainee would receive the guidance and expertise of the instructor but also benefit from the immediate availability of peer assistance. Surprisingly, the second highest preference was "learning from peers," ranked above most instructor-led variations, which may reflect trainees’ comfort in receiving more individualized instruction, with less extraneous information and freedom from rate-limiting slower learners.
IT Proficiencies and Desired Educational Content
The first step in assimilating IT education for trainees is assessing their proficiencies in the targeted areas. This serves several purposes: a) discerning a starting point for all, b) avoiding redundancy for subjects which trainees have already mastered, c) exposing weaknesses and pairing these with IT topics felt to be most important. Thus, the trainees were posed the following statement: "I am adequately proficient in technology in medicine, in the following areas," and asked to answer "yes" or "no" to each listed area of technology. It was not surprising that elementary skills, such as "getting started with the computer" and "basic software issues," were almost exclusively answered "yes" (already proficient), with a tiny percent responding negatively. This suggests that educators might be wise to shelve mass education regarding such basic skills. Remedial instruction should remain available as needed.
In general, the trainees felt it most important to learn about electronic medical records and accessing scientific/medical books, journals or databases on the Internet (or LAN). Third and fourth choices were learning more about personal digital assistant third-party software applications and Web-based course management systems, respectively.
It is unclear why residents tended to express stronger interest than students in receiving education upon all stated technologies except third-party medical software, Web-based course management systems, digital privacy, and career tracking software. We can only speculate. It could be that the student cohort is generally more technologically adept (mirroring societal adaptations over time) than the resident cohort. Or it could be that students are most focused upon, and therefore most interested in, the technologies that are immediately useful in their stage of training. Or residents may have a higher level of responsibility and therefore need more technological assistance. The answer is yet to be determined.
Subjects that were deemed highly important but with which trainees have low proficiency included videoconferencing or telemedicine, electronic medical records, patient simulator software, digital privacy, and career or project tracking/coordinating software.
Frequency of Use Patterns
That trainees preferred Internet-based or PDA-based reference media when providing patient care is important not only to educators but to departmental librarians as they consider what type of media to purchase for reference materials.
Personal Digital Assistants
Preclinical students noted that PDAs were not particularly useful in the classroom. This information may be useful to accepted medical school applicants prior to beginning their first year. On the other hand, over half of clinical trainees felt these devices were "critical" in patient care and met their clinical needs. When separated, clinical students and residents scored equally high regarding their sentiments upon the "critical" nature of the PDA in patient care (PGY-1s were the highest scoring group), the success of the PDA in meeting their clinical needs, and utilization of the PDA as a "non-medication-related reference source (e.g., disease information)." However, while both groups scored high, students were somewhat more likely to use their PDA as a "medication-related reference" (students averaged 73.8%, residents 52.5%). This information may prove useful to trainees considering a PDA purchase.
Though most educators are probably familiar with these patterns, not all administrators may be so informed. It was puzzling that trainees do not use these devices often for patient care logging, as they are sometimes purchased by administrations for this use. We speculate that perhaps they are instead using PC-based software or conventional paper media, as it is somewhat awkward to enter data into a PDA, despite the availability of mobile keyboards for these devices. Paper media may also be the least expensive.
Opportunities for Improvement
It appears that only a minority of trainees are frustrated by technology in medicine, as the top-rated category (EMR—inadequate) was only rated as frustrating by 7% of survey respondents. Even when all EMR-related categories were combined, the complainants only comprised 18% of the survey respondents. The shortcomings of the EMR may be a reflection of their immaturity, and hopefully users will report the problems encountered to the manufacturers so that refinements can follow.
Limitations of This Study
Certain qualitative issues were somewhat difficult to explore exhaustively in a structured survey format, as opposed to interpersonal interviews which, time and space permitting, may have augmented an understanding of the topic and explanatory power. Undergraduate and continuing education research has shown that the accuracy of self-assessment by adult learners has been suboptimal (14–16), limiting the validity of trainee proficiencies (Question 29). The return rate was adequate, albeit relatively small, potentially affecting precision and generalizability. It would be informative to survey trainees from additional institutions. Given that our survey was conducted online, it is possible that some trainees who had trouble using or accessing e-mail may have been inadvertently omitted from our results.
|
Conclusions
|
Despite medical training accrediting bodies’ established expectations that medical institutions teach about IT to their trainees, there is little published information regarding practical implementation of system development in this regard, nor are there any substantive reports regarding trainee’s feedback upon technology in medicine. This information is critical for educators if they are to optimize the medical training experience. This pilot study represents a first effort in this regard. Trainees provided insights regarding their experiences with technology in medicine, including attitudes on its importance, comfort levels, proficiencies and desired didactic topics, learning style preferences, utilization patterns, and suggestions for improvement.
Suggestions for Understanding Trainees’ Needs
- If IT solutions in medical training are under consideration, query trainees for their input, preferably in the product selection stage.
- When developing IT education for trainees, assess their proficiencies in the target area beforehand in order to discern a suitable starting point for all and avoid redundant training.
- Solicit input from trainees regarding their desired didactic topics within IT in medicine.
- Routinely gather feedback from trainees regarding the IT solutions they use, including the technologies' strengths and weaknesses, usage patterns, needed improvements, and frustrations the trainees encounter.
- Identify trainees who are uncomfortable with IT in medicine and address their distress through elucidation of noxious elements, education, programmatic customization, or other necessary support.
|
ACKNOWLEDGMENTS
|
Special thanks to Dr. Donald M. Hilty, University of California, Davis, for his suggestions and assistance.
|
REFERENCES
|
- Baumgart DC: PDAs and their increasing integration with information technology in hospitals could change the way health care is delivered in the future. Lancet 2005; 366:1210–1222[CrossRef][Medline]
- Hilty DM, Hales DJ, Briscoe G, et al: APA Summit on Medical Student Education Task Force on Informatics and Technology: learning about computers and applying computer technology to education and practice. Acad Psychiatry 2006; 30:29–35[Abstract/Free Full Text]
- Medical School Objectives Project: Contemporary Issues in Medicine: Medical Informatics and Population. Washington, DC, Association of American Medical Colleges, 1998
- Kaushal R, Barker KN, Bates DW: How can information technology improve patient safety and reduce medication errors in children’s health care? Arch Pediatr Adolesc Med 2001; 155:1002–1007[Abstract/Free Full Text]
- Baumgart DC: Personal digital assistants in health care: experienced clinicians in the palm of your hand? Lancet 2005; 366:1210–1222[CrossRef][Medline]
- Council on Graduate Medical Education Resource Paper: Preparing learners for practice in a managed care environment. Washington, DC, Department of Health and Human Services, HRSA, 1997
- ACGME’s Residency Review and Institutional Review Committees, Practice-Based Learning And Improvement component, General Competencies, Version 1.3, established at February 1999 meeting. http://www.acgme.org/outcome/comp/compFull.asp
- O’Brien K, Chumley H: Courseware and Distance Learning, Evidence for Electronic Learning, Simulations, Technology in Clerkship Education, Guidebook for Clerkship Directors, 3rd ed Alliance for Clinical Education, 2005
- Huang MP, Alessi NE: An informatics curriculum for psychiatry. Acad Psychiatry 1998; 22:77–91[Abstract/Free Full Text]
- Parekh SG, Nazarian DG, Lim CK: Adoption of information technology by resident physicians. Clin Orthop Relat Res 2004; April:107-111
- Seago BL, Schlesinger JB, Hampton CL: Using a decade of data on medical student computer literacy for strategic planning. J Med Libr Assoc 2002; 90:202–209[Medline]
- Barrett JR, Strayer SM, Schubart JR: Assessing medical residents’ usage and perceived needs for personal digital assistants. Int J Med Inform 2004; 73:25–34[CrossRef][Medline]
- Letterie GS: Medical education as a science: the quality of evidence for computer-assisted instruction. Am J Obstet Gynecol 2003; 188:849–853[CrossRef][Medline]
- Tousignant M, DesMarchais JE: Accuracy of student self-assessment ability compared to their own performance in a problem-based learning medical program: a correlation study. Adv Health Sci Educ 2002; 7:19–27[CrossRef]
- Ward M, Gruppen L, Regehr G: Measuring self-assessment: current state of the art. Adv Health Sci Educ 2002; 7:63–80[CrossRef]
- Langendyk V: Not knowing that they do not know: self-assessment accuracy of third-year medical students. Med Educ 2006; 40:173–179[CrossRef][Medline]
Get information about faster international access.
Privacy Policy
Copyright © 2006
Academic Psychiatry.
All rights reserved.
Home
| Search
| Current Issue
| Past Issues
| Subscribe
| All APPI Journals
| Help
| Contact Us
|
TOP
ABSTRACT
INTRODUCTION
