Usability testing of mobile ICT for clinical settings: Methodological and practical challenges

From Clinfowiki
Jump to: navigation, search

Introduction: The majority of modern Electronic Patient Record (EPR) systems reside on stationary computers. This paradigm is not conducive to a clinical setting wherein health workers are extremely active while caring for patients. The growing ubiquity and enhanced functionality of mobile devices is aligned well with a number of studies that have uncovered benefits of mobile Information and Communications Technology (ICT). However, Norwegian researchers have identified intrinsic challenges for system design and usability evaluation. The aim of this study is to focus on the methodological and practical challenges relative to this new health care model.

Background: For the purposes of the study, mobile technology is defined as “technology that provides digital information and communication services to users on the move either through devices that are portable per se, or through fixed devices that are easily ready at hand at the users’ current physical position.” Usability is another term requiring a specific description. To paraphrase the ISO terminology, usability is context-dependant in that it is relative to particular users, goals and the surrounding environment. One must take into account the myriad variations in mobile ICT devices as they range from rudimentary single-function gadgets to complex tools capable of numerous operations.

Method: Funding was provided to build a usability laboratory for testing mobile applications in a realistic environment in order to prevent encroaching upon actual health care settings through field trials. The researchers installed movable walls to facilitate simulations of different Norwegian hospital configurations. They designed and furnished the laboratory after consulting with health workers. The team also installed video and audio recording equipment for constant surveillance of the trials.

The Two Experiments (Controlled):

Experiment 1: Combining Handheld Devices and Patient Terminals Some newer hospitals install bedside terminals, allowing for patient entertainment and web browsing via a touch screen interface, so a similar scene was created in the lab. Seven prototype PDAs were tested as terminal control consoles for the display of X-ray images that patients could view. Five physician and patient (emulated) pairs were recruited to test and rank the different PDAs based on usability. Evaluation factors: graphical user interface (GUI); patient terminal screen size and ergonomics; shared view versus hiding information on the PDA; and focus shifts and time away from the patient (comingling PDA and terminal was distracting).

Experiment 2: Automatic Identification of Patients at Point of Care Experiment 2 targeted assessing and comparing the usability of different sensor-based techniques for automatic patient identification during medicine administration, thereby attempting to decrease error and improve patient safety. The common methods in practice are a name search or selection from a patient list. Each take time and have a large potential for error. Researchers employed 2x2 combinations of two newer device (mobile PDAs; bedside stationary touch-screen terminals) and two sensing technologies: barcode token-based and WLAN positioning location-based. Test subjects ranked and explained their rankings according to four areas: time on computer devices versus time on patient; predictability and control; and integration with work situation.

Factors that affect the usability of mobile EPR and Resulting Consequences:

Usability of the graphical user interface: GUI usability issues were negligible for both experiments. The results were attributed to GUI simplicity in the prototypes, which would not likely be the case when reviewing a more fully functional and realistic mobile EPR system. Consequences: The authors recommend a full-scale mobile usability test and a separate desktop usability test when the mobile EPR GUI is complex.

Physical and bodily aspects of usability: Both experiments revealed screen size issues. PDAs were not appropriate for X-ray images in the first case. Conversely, the second experiment showed patients preferred medication lists on large screens. In reference to body movement, physicians in the first experience stated PDAs were more convenient because they did not have to lean over a patient’s bed to control a terminal. Some users in the second experiment like the bedside terminal concept as it freed their hands for other tasks. Experiment 2 also allowed for persistent computer access from any location. Consequences: Replicating the physical environment helps researchers understand point-of-care interaction and the precision of sensors.

Social aspects of usability: Both experiments exhibited issues regarding conflicts with shared and private data views. The study also indicated both experiments documented issues when the human-computer interaction time interfered with human-human interaction quality. Consequences: Introduce tests with multiple users concurrently interacting. Ensure that communication between users is captured through quality video and audio media.

Discussion: The study analysis and recommendations were based on a small sample population of tests and subjects. Also, device prototypes and use scenarios were relatively simple and did not completely emulate real-world situations. More studies are definitely required before formulating a conclusive list of usability issues. The structure laid out in this study can serve as a reasonable foundation for future evaluations.

Conclusion: Clinical work in hospitals is very mobile and involves intense information and communication sharing. The expected evolution is from solely stationary computers to a mixture of these systems with mobile devices operating at the point of care. As evidenced in the laboratory tests, usability extends beyond the GUI. Ergonomic and social aspects are also key attributes requiring attention. Mobile usability tests should be conducted in physical environments offering realism, yet do not impinge on privacy and ethics concerns.

Comments: The study is compelling in that it seeks results as near to real-world experiences as can be expected in a lab environment. A convertible layout provides prompt remodeling to accommodate many test settings. Earnest attempts should be made to obtain and configure more complex prototype devices. Some additional suggestions are to consider biometric identification methods, conduct focus group sessions from patient and provider camps before resuming tests, and leverage personal telehealth case studies to fully understand location tracking.