Difference between revisions of "The Cognitive Complexity of a Provider Order Entry Interface"

Revision as of 12:38, 5 November 2015

This is a review of the article by Horsky et al (2003) titled The Cognitive Complexity of a Provider Order Entry Interface.

The Cognitive Complexity of a Provider Order Entry Interface

Jan Horsky, MA, David R. Kaufman, PhD, and Vimla L. Patel, PhD

AMIA Annu Symp Proc. 2003; 2003: 294–298.

http://www.ncbi.nlm.nih.gov/pmc/articles/PMC1480200/

Background

Cognitive engineering is an approach that looks at the complexity of intellectual interaction of humans and machines. Cognition can be viewed as a process of coordinating internal and external representations in the users distributed resources framework. It is an approach that can be used to investigate the interaction of errors.

The development of Order entry systems was anticipated to deliver benefits such as the elimination of errors associated with hand-written orders and increase in speed and quality of communication between clinicians. However, newly adopted technologies can introduce new sources of error, as they tend to alter work hits and practices. CPOE is an inherently complex process and poor interface design not only slows down the clinician but can also introduce a source of error. ^[1]

The authors of this paper aim to evaluate a complex order entry system using the relatively new model of distributed resources framework.

Methods

This research was based on Norman’s theory of action - specifically the cognitive walkthrough and recent developments in analysis of distributed cognitive methods of human-computer interaction (HCI). ^[1]

To this end the study involved 7 internal medicine physicians with a 2-5 year range of clinical experience using a complex order entry system and consisted of two components, cognitive walkthrough and order entry.

System Walkthrough

Cognitive walkthrough involved analysis of the distribution of cognitive resources involved in performing tasks and the potential creation of errors. The task involved developing a problem representation of the clinical scenario by assessing the patient, recording pertinent findings and then entering orders as requested.

Order Entry

Order entry by the clinicians where they were given a written clinical scenario and asked to enter the appropriate orders using the think aloud protocol where they verbalize their thoughts during 30minute sessions. The subjects were videoed as they performed the tasks and their verbalizations were transcribed and coded for cognitive task analysis. The screen video was used to record mouse movements, mouse actions and screen transitions to enable analysis.

Results

For the system walkthroughs the users attention needed to be divided between treatment planning and managing system operations. To complete the task correctly the user needed to navigate through 12 system states, which made heavy demands on the users internal and external resources. From this analysis they inferred that the system required a steep learning curve, which could potentially lead to increased errors.

For the order entry no subject was able to produce a flawless set of orders as per reference model guidelines. Five subjects made errors of omission and incorrect entries were made by five subjects. The number of both types of errors per subject ranged from one to five.

Conclusion

This study took a dual approach by firstly conducting a distributed resources task analysis and then secondly conducting usability testing by asking experience clinicians to enter orders into the CPOE system.

The system walkthrough identified that the configuration of resources might lead to problems for the users in terms of cognitive complexity. The results bore out this as evidenced by the users actions and high error rate.

The design of the system may have contributed to the errors observed due to lack of clarity in the presentation of pick list for orders sets and the lack of easy backtracking or error recovery in the system.

Comments

This was an article that presented a very interesting approach to analysis of cognitive load coupled with the previously used think aloud technique for recording task performance. However the sample size was very small and results could not be used for any form of statistical analysis.

In addition the presentation of the background, methods and results could have been more structured in order to better convey to the reader the process and analysis used for the study.

Overall however, the study did identify that these methods could be used to investigate and identify the introduction and potential reasons for errors when clinicians interact with health technology such as CPOE systems. It would be interesting to review any studies with larger sample sizes that utilize these approaches within a more structured experimental design to investigate usability in clinical systems.

Related Articles

References

1. ↑ ^{1.0 1.1} Bates et al (2001) Reducing the Frequency of Errors in Medicine Using Information Technology JAMIA 2001;8(4):299–308. http://www.ncbi.nlm.nih.gov/pmc/articles/PMC1480200/ /