Cross-vendor evaluation of key user-defined clinical decision support capabilities: a scenario-based assessment of certified electronic health records with guidelines for future development

From Clinfowiki
Jump to: navigation, search

This is a qualitative study by McCoy AB, Wright A, & Sittig DF. (2015) in the Journal of American Medical Informatics Association, entitled “Cross-vendor evaluation of key user-defined clinical decision support capabilities: a scenario-based assessment of certified electronic health records with guidelines for future development.” [1]


The mere presence of clinical decision support system (CDS) within the electronic health record (EHR) is not enough. An EHR with full CDS implementation capabilities is required. “The Office of the National Coordinator for Health Information Technology (ONC) developed the Permanent Certification Program for EHRs to certify that EHRs are capable of meeting the meaningful use criteria. Six organizations serve as ONC-Authorized Certification Bodies (ONC-ACBs): Surescripts LLC; ICSA Labs; SLI Global eaningful Useolutions; InfoGard Laboratories, Inc.; Certification Commission for Health Information Technology (CCHIT); and Drummond Group, Inc.” (McCoy et at. 2015; pg 1082). In order to achieve superior quality of care, improved patient safety and reduced healthcare cost, a high-quality clinical decision support is warranted.


The purpose of this study was to evaluate the implementation capabilities of a convenience sample of certified EHRs by assigning three simple CDS rules.


This was a descriptive, observational study using a small sample size of ONC-ACB EHRs. Only 3 basic user-defined CDS rules were established to be implemented in the EHRs. Also, the detailed ability of EHRs to implement CDS rules was analyzed.

According to the authors, the rules were:

  1. If a patient has amiodarone on his or her active medication list and does not have a thyroid-stimulating hormone (TSH) result recorded in the last 12 months, suggest ordering a TSH.
  2. If a patient has a hemoglobin A1c result >7% and does not have diabetes on his or her problem list, suggest adding diabetes to the problem list.
  3. If a patient has coronary artery disease on his or her problem list and does not have aspirin on the active medication list, suggest ordering aspirin.


Results of user-defined CDS rules using 8 EHRs were as follows: “Five EHRs were able to implement all 3 rules, and the remaining 3 EHRs were unable to fully implement any of the rules. One of these EHRs did not allow users to create rules at all” (McCoy et al, 2015, p1083). Overall, three patterns were recognized in the EHRs.

  1. EHRs that lacked customization or modification abilities did not support user-defined CDS rule implementation.
  2. EHRs that were basic and inflexible and EHRs that were more advanced varied from full- to partial- to no implementation. This type of pattern was referred as ‘bolt-on’.
  3. EHRs that allowed full implementation of all user-defined CDS rules. This pattern was ‘a platform’ approach.

Discussion & Comments

For the full potential of national EHR use to be realized, event-driven, action-oriented, real-time, point-of-care clinical decision support that is relevant and is incorporated seamlessly by users into their workflows are required. Alerts need to be able to be driven by time-limited rules that are able to use laboratory results, advanced Boolean logic, and permit actions directly from the alert interface. Despite certification using Clinical Decision Support Guidelines (2011, not 2014, criteria), 3 or 8 EHRs were unable to satisfactorily implement 3 evidence-based rules and test all portions of the interventions. In this probe of CDS capabilities, different specific deficiencies were explored, but most EHRs evaluated had significant limitations. Such implementation incapability has important implications about successful, meaningful future use of EHRs. These fell into 2 groups: a) attaining improved standards for basic CDS functionality, and b) sufficient training by vendors to healthcare organizations utilizing their systems so that all quality and safety features can actually be used.


The authors detailed the specific deficiencies in the sample EHRs studied, but since limitations were frequent in this convenience sample, called for significant improvements in our certification and implementation rules and regulations. Without performing substantial upgrades, the essential functions of CDS in EHRs will not be possible. To this end, they provided a Table of Guidelines for Clinical Decision Support Guidelines for Electronic Health Record Certification as a template for action.

Related articles

  1. Romano MJ, Stafford RS. Electronic health records and clinical decision support systems: impact on national ambulatory care quality. Arch Intern Med. 2011 May 23; 171(10):897-903.
  2. Phansalkar S, Zachariah M, Seidling HM, Mendes C, Volk L, Bates DW. Evaluation of medication alerts in electronic health records for compliance with human factors principles. J Am Med Inform Assoc. 2014 Oct; 21(e2):e332-40.
  3. Rothman B, Leonard JC, Vigoda MM. Future of electronic health records: implications for decision support. Mt Sinai J Med. 2012 Nov-Dec; 79(6):757-68.
  4. Ballard DW, Rauchwerger AS, Reed ME, Vinson DR, Mark DG, Offerman SR, Chettipally UK, Graetz I, Dayan P, Kuppermann N, et al. Emergency physicians' knowledge and attitudes of clinical decision support in the electronic health record: a survey-based study. Acad Emerg Med. 2013 Apr; 20(4):352-60.
  5. Häyrinen K, Saranto K, Nykänen P. Definition, structure, content, use and impacts of electronic health records: a review of the research literature. Int J Med Inform. 2008 May; 77(5):291-304.
  6. McCullagh LJ, Sofianou A, Kannry J, Mann DM, McGinn TG. User centered clinical decision support tools: adoption across clinician training level. Appl Clin Inform. 2014 Dec 17;5(4):1015-25. doi: 10.4338/ACI-2014-05-RA-0048.
  7. Kannry J, McCullagh L, Kushniruk A, Mann D, Edonyabo D, McGinn T. A Framework for Usable and Effective Clinical Decision Support: Experience from the iCPR Randomized Clinical Trial. EGEMS (Wash DC). 2015 Jul 9;3(2):1150. doi: 10.13063/2327-9214.1150.


  1. McCoy (2015). Cross-vendor evaluation of key user-defined clinical decision support capabilities: a scenario-based assessment of certified electronic health records with guidelines for future development. Journal of American Medical Informatics Association. .