Difference between revisions of "Vendor Selection Criteria: Usability"

From Clinfowiki
Jump to: navigation, search
(→‎TURF Usability Framework)
(Created new headings, created new section "Outcomes of Poor Usability", added references, rearrangement of material, fixed other minor errors such as spelling and grammar)
 
Line 1: Line 1:
−
Previous studies demonstrated how usability measurements can be applied to the evaluation of EHR systems; however, most of these studies were conducted post implementation. If significant problems have been discovered with usability, at this point in the system development
+
Previous studies demonstrated how usability measurements can be applied to the evaluation of EHR systems; however, most of these studies were conducted post implementation. If significant problems have been discovered with usability at this point in the system development life cycle, it is usually too late to make any major modifications to the EHR system. Therefore, it is important to consider the usability criteria in early stages of the EHR implementation, and particularly during the process of vendor and product selection.  <ref name="Saiku"> I. Saiku. Including usability in the procurement process of healthcare IT Systems. <http://www.soberit.hut.fi/T-121/shared/thesis/di-Inkeri-Saiku.pdf</ref>   
−
life cycle it is usually too late to make any major modifications to the EHR system. Therefore, it is important to consider the usability criteria in early stages of the EHR implementation, and particularly during the process of vendor and product selection.  <ref name="Saiku"> I. Saiku. Including usability in the procurement process of healthcare IT Systems. <http://www.soberit.hut.fi/T-121/shared/thesis/di-Inkeri-Saiku.pdf</ref>   
+
  
−
A 2015 report by Ratwani et al. reviews complaince with the US Department of Health and Human Services’ Office of the National Coordinator for Health Information Technology (ONC) requirements for EHR vendor usability practices.  Compliance with the usability standards was weak, and the authors suggest that this lack of focus on usability may contribute significantly to end user frustration.  <ref name="ratwani">Ratwani, R. M., Benda, N. C., Hettinger, A. Z., & Fairbanks, R. J. (2015). Electronic Health Record Vendor Adherence to Usability Certification Requirements and Testing Standards. JAMA, 314(10), 1070–1071. http://jama.jamanetwork.com/article.aspx?articleid=2434673</ref>
+
== Definition ==
  
 
Usability describes how easy it is for users to accurately and efficiently accomplish a task while using a system. Usability can be defined in terms of three main attributes: useful, usable, and satisfaction. Even in the presence of all necessary features, the system can still have poor reliability, mismatch between interface and tasks, and lack critical functionalities that can impact end-user ability to conduct work. <ref name="Usability">AHRQ. EHR Usability Toolkit: A Background Report
 
Usability describes how easy it is for users to accurately and efficiently accomplish a task while using a system. Usability can be defined in terms of three main attributes: useful, usable, and satisfaction. Even in the presence of all necessary features, the system can still have poor reliability, mismatch between interface and tasks, and lack critical functionalities that can impact end-user ability to conduct work. <ref name="Usability">AHRQ. EHR Usability Toolkit: A Background Report
 
on Usability and Electronic Health Records. https://healthit.ahrq.gov/sites/default/files/docs/citation/EHR_Usability_Toolkit_Background_Report.pdf</ref>
 
on Usability and Electronic Health Records. https://healthit.ahrq.gov/sites/default/files/docs/citation/EHR_Usability_Toolkit_Background_Report.pdf</ref>
 +
 +
 +
== Methods ==
  
 
The following methods can be used to measure usability:
 
The following methods can be used to measure usability:
Line 18: Line 20:
 
* Workload Evaluation <ref name="Usability">AHRQ. EHR Usability Toolkit: A Background Report
 
* Workload Evaluation <ref name="Usability">AHRQ. EHR Usability Toolkit: A Background Report
 
on Usability and Electronic Health Records. https://healthit.ahrq.gov/sites/default/files/docs/citation/EHR_Usability_Toolkit_Background_Report.pdf</ref>
 
on Usability and Electronic Health Records. https://healthit.ahrq.gov/sites/default/files/docs/citation/EHR_Usability_Toolkit_Background_Report.pdf</ref>
 +
 +
 +
== Characteristics ==
  
 
Some key characteristics to consider when evaluating EHR usability:
 
Some key characteristics to consider when evaluating EHR usability:
Line 29: Line 34:
  
  
−
====TURF Usability Framework====
+
== TURF Usability Framework ==
  
 
When selecting an EMR from a Vendor, it is important to have a framework in which to compare each module of the EMR with the workflow of the users.  Usability is how useful, usable and satisfying a system is for the user and how well the user is able to complete a given task.  In a recent study, it was found the TURF analysis framework can be used to assess and create a satisfying and less complex interface.  TURF stands for  
 
When selecting an EMR from a Vendor, it is important to have a framework in which to compare each module of the EMR with the workflow of the users.  Usability is how useful, usable and satisfying a system is for the user and how well the user is able to complete a given task.  In a recent study, it was found the TURF analysis framework can be used to assess and create a satisfying and less complex interface.  TURF stands for  
Line 39: Line 44:
  
 
This framework can be used to leverage and compare the usability of the EMR vendor being considered.<ref name = "usability framework">Harrington,C., Wood,R., Breuer,J., Pinzon, O., Howell, R., et al. (2011).  Using a unified usability framework to dramatically improve the usability of an EMR module. American Medical Informatics Association. http://www.ncbi.nlm.nih.gov/pmc/articles/PMC3243270/</ref>
 
This framework can be used to leverage and compare the usability of the EMR vendor being considered.<ref name = "usability framework">Harrington,C., Wood,R., Breuer,J., Pinzon, O., Howell, R., et al. (2011).  Using a unified usability framework to dramatically improve the usability of an EMR module. American Medical Informatics Association. http://www.ncbi.nlm.nih.gov/pmc/articles/PMC3243270/</ref>
 +
 +
=== Reports on TURF ===
  
 
A paper written by Jiajie Zhang and Muhammad Walji (2011) describes TURF as a framework that "(1) describing, explaining, and predicting usability differences in terms of the representation effect; (2) for defining, evaluating, and measuring usability objectively; (3) for designing built-in good usability; and (4) for developing EHR usability guidelines and standards." <ref name= "Zhang & Walji"> Zhang, J., & Walji, M. (2011). TURF: toward a unified framework of EHR usability. Journal of Biomedical Informatics, 44(6), 1056-1067. doi:10.1016/j.jbi.2011.08.005</ref> They also define usability as useful, usable, and satisfying.  
 
A paper written by Jiajie Zhang and Muhammad Walji (2011) describes TURF as a framework that "(1) describing, explaining, and predicting usability differences in terms of the representation effect; (2) for defining, evaluating, and measuring usability objectively; (3) for designing built-in good usability; and (4) for developing EHR usability guidelines and standards." <ref name= "Zhang & Walji"> Zhang, J., & Walji, M. (2011). TURF: toward a unified framework of EHR usability. Journal of Biomedical Informatics, 44(6), 1056-1067. doi:10.1016/j.jbi.2011.08.005</ref> They also define usability as useful, usable, and satisfying.  
Line 45: Line 52:
 
*Usable is defined as a system that is easy to use and learn and is tolerant of mistakes. <ref name= "Zhang & Walji"> Zhang, J., & Walji, M. (2011). TURF: toward a unified framework of EHR usability. Journal of Biomedical Informatics, 44(6), 1056-1067. doi:10.1016/j.jbi.2011.08.005</ref>
 
*Usable is defined as a system that is easy to use and learn and is tolerant of mistakes. <ref name= "Zhang & Walji"> Zhang, J., & Walji, M. (2011). TURF: toward a unified framework of EHR usability. Journal of Biomedical Informatics, 44(6), 1056-1067. doi:10.1016/j.jbi.2011.08.005</ref>
 
*Satisfying is defined as users having good impressions concerning usefulness and usability. <ref name= "Zhang & Walji"> Zhang, J., & Walji, M. (2011). TURF: toward a unified framework of EHR usability. Journal of Biomedical Informatics, 44(6), 1056-1067. doi:10.1016/j.jbi.2011.08.005</ref>
 
*Satisfying is defined as users having good impressions concerning usefulness and usability. <ref name= "Zhang & Walji"> Zhang, J., & Walji, M. (2011). TURF: toward a unified framework of EHR usability. Journal of Biomedical Informatics, 44(6), 1056-1067. doi:10.1016/j.jbi.2011.08.005</ref>
 +
 +
 +
== Outcomes of Poor Usability ==
 +
 +
Many different articles have been released relaying information from different experts in the field of Health IT. These articles are written with a clear attempt to help vendors attain credibility for usability. One article points towards five different criteria that could be useful for creating better EHR usability; the five points are data entry, errors, metrics, training, and what the article calls “bottom line.”<ref name="insider">5 health IT insiders offer their takes on EHR usability http://www.healthcareitnews.com/news/five-health-it-insiders-offer-five-takes-ehr-usability</ref> Data entry, according to JiaJie Zhang in the article, is not optimized and can be improved.<ref name="insider"></ref> With the lack of usability, errors can be unprecedented and lead to unnecessary adverse effects, as noted by Robert Tennant.<ref name="insider"></ref> The goodness of usability can offer the vendors valuable metrics, as mentioned by Mary Kate Foley.<ref name="insider"></ref> The ability to have usability comes with the need for adequate training according to Dr. William F. Bria.<ref name="insider"></ref> And after all is said and done, the physicians would like to see good changes from the usability of EHRs, as noted by Rosemarie Nelson.<ref name="insider"></ref>
 +
 +
=== Special 2015 Report ===
 +
 +
A 2015 report by Ratwani et al. reviews compliance with the US Department of Health and Human Services’ Office of the National Coordinator for Health Information Technology (ONC) requirements for EHR vendor usability practices.  Compliance with the usability standards was weak, and the authors suggest that this lack of focus on usability may contribute significantly to end user frustration.  <ref name="ratwani">Ratwani, R. M., Benda, N. C., Hettinger, A. Z., & Fairbanks, R. J. (2015). Electronic Health Record Vendor Adherence to Usability Certification Requirements and Testing Standards. JAMA, 314(10), 1070–1071. http://jama.jamanetwork.com/article.aspx?articleid=2434673</ref>
 +
  
 
== References ==
 
== References ==
 
<references/>
 
<references/>

Latest revision as of 21:23, 23 September 2015

Previous studies demonstrated how usability measurements can be applied to the evaluation of EHR systems; however, most of these studies were conducted post implementation. If significant problems have been discovered with usability at this point in the system development life cycle, it is usually too late to make any major modifications to the EHR system. Therefore, it is important to consider the usability criteria in early stages of the EHR implementation, and particularly during the process of vendor and product selection. [1]

Definition

Usability describes how easy it is for users to accurately and efficiently accomplish a task while using a system. Usability can be defined in terms of three main attributes: useful, usable, and satisfaction. Even in the presence of all necessary features, the system can still have poor reliability, mismatch between interface and tasks, and lack critical functionalities that can impact end-user ability to conduct work. [2]


Methods

The following methods can be used to measure usability:

  • Literature Search
  • Heuristic Evaluation
  • Cognitive walkthrough
  • Laboratory Testing
  • Remote Evaluation
  • Usability Questionnaires
  • Predictive Modeling
  • Risk Assessment
  • Workload Evaluation [2]


Characteristics

Some key characteristics to consider when evaluating EHR usability:

  • Does the system have an easy to read interface with an user centered design (UCD)?[3]
  • Speed: How much time does it take for the user to complete a task? How much time between switching from one screen to another?
  • Juxtaposition: Are similar options placed too close to one another? Is it easy for the user to click on the wrong option?
  • Aesthetics: Is the EHR design aesthetically pleasing? Does the design make it intuitive for first time users to work with the EHR?[4]
  • Are the icons consistent and provide intuitive meaning to the user?[3]
  • Will the interface fully support the user's workflow and information needs?[3]
  • Can it display data that makes it easy to draw insights for the user?[3]


TURF Usability Framework

When selecting an EMR from a Vendor, it is important to have a framework in which to compare each module of the EMR with the workflow of the users. Usability is how useful, usable and satisfying a system is for the user and how well the user is able to complete a given task. In a recent study, it was found the TURF analysis framework can be used to assess and create a satisfying and less complex interface. TURF stands for

  • Task Analysis
  • User Analysis
  • Representational Analysis
  • Functional Analysis

This framework can be used to leverage and compare the usability of the EMR vendor being considered.[5]

Reports on TURF

A paper written by Jiajie Zhang and Muhammad Walji (2011) describes TURF as a framework that "(1) describing, explaining, and predicting usability differences in terms of the representation effect; (2) for defining, evaluating, and measuring usability objectively; (3) for designing built-in good usability; and (4) for developing EHR usability guidelines and standards." [6] They also define usability as useful, usable, and satisfying.

  • Useful is defined as a system where users "can accomplish their goals independent of how the system is implemented." [6]
  • Usable is defined as a system that is easy to use and learn and is tolerant of mistakes. [6]
  • Satisfying is defined as users having good impressions concerning usefulness and usability. [6]


Outcomes of Poor Usability

Many different articles have been released relaying information from different experts in the field of Health IT. These articles are written with a clear attempt to help vendors attain credibility for usability. One article points towards five different criteria that could be useful for creating better EHR usability; the five points are data entry, errors, metrics, training, and what the article calls “bottom line.”[7] Data entry, according to JiaJie Zhang in the article, is not optimized and can be improved.[7] With the lack of usability, errors can be unprecedented and lead to unnecessary adverse effects, as noted by Robert Tennant.[7] The goodness of usability can offer the vendors valuable metrics, as mentioned by Mary Kate Foley.[7] The ability to have usability comes with the need for adequate training according to Dr. William F. Bria.[7] And after all is said and done, the physicians would like to see good changes from the usability of EHRs, as noted by Rosemarie Nelson.[7]

Special 2015 Report

A 2015 report by Ratwani et al. reviews compliance with the US Department of Health and Human Services’ Office of the National Coordinator for Health Information Technology (ONC) requirements for EHR vendor usability practices. Compliance with the usability standards was weak, and the authors suggest that this lack of focus on usability may contribute significantly to end user frustration. [8]


References

  1. ↑ I. Saiku. Including usability in the procurement process of healthcare IT Systems. <http://www.soberit.hut.fi/T-121/shared/thesis/di-Inkeri-Saiku.pdf
  2. ↑ 2.0 2.1 AHRQ. EHR Usability Toolkit: A Background Report on Usability and Electronic Health Records. https://healthit.ahrq.gov/sites/default/files/docs/citation/EHR_Usability_Toolkit_Background_Report.pdf
  3. ↑ 3.0 3.1 3.2 3.3 Ratwani, R.M., Fairbanks, R.J., Hettinger, A.Z., Benda, N.C. (2015). Electronic health record usability: analysis of the user-centered design processes of eleven electronic health record vendors. Journal of the American Medical Informatics Association, 0:1–5. doi: 10.1093/jamia/ocv050
  4. ↑ Ash, Berg, & Coiera (2004). Some Unintended Consequences of Information Technology in Health Care: The Nature of Patient Care Information System-related Errors. Journal of the American Medical Informatics Association (11)104-112. DOI 10.1197/jamia.M1471
  5. ↑ Harrington,C., Wood,R., Breuer,J., Pinzon, O., Howell, R., et al. (2011). Using a unified usability framework to dramatically improve the usability of an EMR module. American Medical Informatics Association. http://www.ncbi.nlm.nih.gov/pmc/articles/PMC3243270/
  6. ↑ 6.0 6.1 6.2 6.3 Zhang, J., & Walji, M. (2011). TURF: toward a unified framework of EHR usability. Journal of Biomedical Informatics, 44(6), 1056-1067. doi:10.1016/j.jbi.2011.08.005
  7. ↑ 7.0 7.1 7.2 7.3 7.4 7.5 5 health IT insiders offer their takes on EHR usability http://www.healthcareitnews.com/news/five-health-it-insiders-offer-five-takes-ehr-usability
  8. ↑ Ratwani, R. M., Benda, N. C., Hettinger, A. Z., & Fairbanks, R. J. (2015). Electronic Health Record Vendor Adherence to Usability Certification Requirements and Testing Standards. JAMA, 314(10), 1070–1071. http://jama.jamanetwork.com/article.aspx?articleid=2434673
Retrieved from "https://www.clinfowiki.org/wiki/index.php?title=Vendor_Selection_Criteria:_Usability&oldid=25118"