Alert Dwell Time: Introduction of a Measure to Evaluate Interruptive Clinical Decision Support Alerts

From Clinfowiki
Revision as of 13:08, 3 November 2015 by Sgriffith1 (Talk | contribs)

Jump to: navigation, search

This is a review of the 2015 paper by McDaniel, et al. [1]

Abstract

Metrics for evaluating interruptive prescribing alerts have many limitations. Additional methods are needed to identify opportunities to improve alerting systems and prevent alert fatigue. In this study, the authors determined whether alert dwell time—the time elapsed from when an interruptive alert is generated to when it is dismissed—could be calculated by using historical alert data from log files. Drug–drug interaction (DDI) alerts from 3 years of electronic health record data were queried. Alert dwell time was calculated for 25,965 alerts, including 777 unique DDIs. The median alert dwell time was 8 s (range, 1–4913 s). Resident physicians had longer median alert dwell times than other prescribers (P < .001). The 10 most frequent DDI alerts (n = 8759 alerts) had shorter median dwell times than alerts that only occurred once (P < .001). This metric can be used in future research to evaluate the effectiveness and efficiency of interruptive prescribing alerts.

Background

Previous efforts at measuring the effectiveness of Clinical Decision Support (CDS) prescribing alerts have tracked user satisfaction surveys and alert overrride rates. The authors investigate the "dwell time", or the time that elapses between alert presentation and dismissal, as a possible new method to evaluate CDS.

Methods

Clinical DDI alerts were monitored over a 3-year timeframe. Dwell time was calculated for a number of different subsets, including evaluation by DDI, clinician role, or whether the alert was dismissed or accepted.

Results

Dwell time was not significantly different across common drug alerts. Infrequent DDI generated a longer dwell time than commonly presented DDI. Pharmacists had on average, the shortest dwell time, and resident physicians the longest.

Conclusion

Dwell time can be successfully monitored in a large hospital CIS environment, and utilized as a metric in future CDS evaluations.

Comments

This is a pilot study, and the findings fell within the limited expectations of the study. Dwell time may be a more useful tool when implementing significant changes to a user interface. Also, it would be interesting to monitor dwell time changes with the deployment of new CDS rules. I suspect it is a metric that would be most useful in a dynamic environment.


References

  1. McDaniel, R. B., Burlison, J. D., Baker, D. K., Hasan, M., Robertson, J., Hartford, C., … Hoffman, J. M. (2015). Alert dwell time: introduction of a measure to evaluate interruptive clinical decision support alerts. Journal of the American Medical Informatics Association: JAMIA. http://doi.org/10.1093/jamia/ocv144