Formative evaluation of the accuracy of a clinical decision support system for cervical cancer screening
This is a review on Casey, P. M., Chaundry, R., Greenes, R. A., Henry, M., Kastner, T. M., Liu, H., MacLaughlin, K. L., & Wagholikar, K. B. (2013) article, Formative evaluation of the accuracy of a clinical decision support system for cervical cancer screening [1].
Background and Objective
The article discusses the importance of a prototypical clinical decision support system (CDSS), which screens for cervical cancer. Although screening is generally correlated with the prevention of cervical cancer, cervical cancer is continuously associated with a majority of cancer-related deaths in females. In fact, approximately 530,000 women were diagnosed with cervical cancer in 2008 and 275,000 deaths were associated with cervical cancer [1]. Guidelines for the screening and surveillance of cervical cancer are complex in nature and fail to be recalled by health care providers. In a dire effort to maximize the optimization of cervical cancer screening, a prototype CDSS was developed to assess patient information in the electronic health record and suggest recommendations to care providers in accordance to specific guidelines. A meta-analysis comprised of a total of 42 studies conducted on a multinational scale reported that more than half of women diagnosed with cervical cancer either had no screening or inadequate screening. As more and more health care providers adopt the use of electronic health records in the United States, the use of CDSS as a mechanism to remind care providers and patients to adhere to screening guidelines will improve cervical cancer screening and surveillance rates.
Methods
Clinical decision support systems are developed to aid in prevention and diagnosis; however, ineffective implementation of CDSS impacts clinical outcomes due to fatigue, prolonged response time, and lack of accuracy and integration of workflow. The study utilized a web-based application to record recommendations of 89 potential end-users from a randomized sample of patients compared to those generated by CDSS. Recommendations that did not match between the random sample and CDSS were rectified by two experts in a dire effort to assess whether the provider recommendation or CDSS was more optimal. If the clinical decision support system proved to be less optimal than the provider recommendation, an error analysis was performed and the CDSS was improved to resolve errors accordingly.
Results
Twenty-eight of the 89 providers agreed to participate in the study; however, only 25 annotated a total of 175 test cases in accordance with their recommendations. Six of the cases were flagged with an error as the cases had bugs within the interface of the electronic health record system, which prevented providers from obtaining pathology reports. Of the remaining 169 cases, 75 cases had mismatched recommendations, of which 22 of those cases showed less than optimal CDSS. The error analysis led to the identification of 12 errors (modeling and programming errors) in the clinical decision support system. Analysis of research findings revealed areas within the guidelines that require decision support, especially among cases that demonstrate abnormal findings.
Comments
The use of CDSS to improve cervical cancer screening and surveillance has a unique set of potential advantages. However, CDSS also entails a subset of implications thereby requiring further research. The most significant criticism is the fact that the study involved only 25 providers who actually participated and annotated test cases. The study must therefore be performed on a larger scale to determine whether the results generated from future studies coincide and support data collected from this web-based application. Hence, verification from the CDSS recommendations is required to enhance the provision of feedback necessary in improving the optimization of the system. Based on prospective pilot studies, CDSS may potentially be implemented within clinical practice to improve clinical outcomes.