Feasibility of real-time satisfaction surveys through automated analysis of patients' unstructured comments and sentiments

From Clinfowiki
Jump to: navigation, search

First Review

This is a review of the article by Alemi et al., Feasibility of real-time satisfaction surveys through automated analysis of patients' unstructured comments and sentiments.


This article shows how sentiment analysis (an artificial intelligence procedure that classifies opinions expressed within the text can be used to design real-time satisfaction surveys. To improve participation, real-time surveys must be radically short. The shortest possible survey is a comment card. Patients' comments can be found online at sites organized for rating clinical care, within e-mails, in hospital complaint registries, or through simplified satisfaction surveys such as "Minute Survey." Sentiment analysis uses patterns among words to classify a comment into a complaint, or praise. It further classifies complaints into specific reasons for dissatisfaction, similar to broad categories found in longer surveys such as Consumer Assessment of Healthcare Providers and Systems. In this manner, sentiment analysis allows one to re-create responses to longer satisfaction surveys from a list of comments. To demonstrate, this article provides an analysis of sentiments expressed in 995 online comments made at the [|RateMDs.com]. We focused on pediatrician and obstetrician/gynecologist physicians in District of Columbia, Maryland, and Virginia. We were able to classify patients' reasons for dissatisfaction and the analysis provided information on how practices can improve their care. This article reports the accuracy of classifications of comments. Accuracy will improve as the number of comments received increases. In addition, we ranked physicians using the concept of time-to-next complaint. A time-between control chart was used to assess whether time-to-next complaint exceeded historical patterns and therefore suggested a departure from norms. These findings suggest that (1) patients' comments are easily available, (2) sentiment analysis can classify these comments into complaints/praise, and (3) time-to-next complaint can turn these classifications into numerical benchmarks that can trace impact of improvements over time. The procedures described in the article show that real-time satisfaction surveys are possible[1].

Background and Purpose

The authors saw a need for a system that collected feedback from patients because traditional means often took too long to be administered and collected after patient care had entered. This prevented providers from identifying and making use of windows opportunity for improvement actions. The expensive nature of traditional means also limited physicians to only collecting feedback from a sample of patients, not the full population. This often proved to be unreliable.


The authors suggested a two question survey, with the two questions being open ended [1]. The first question asked patients to rate their overall experience, while the second question asked them what worked well and what needs improvement. The open ended nature of these questions allowed patients to explain the reasoning behind their rating. A sentiment analysis tool was then developed and trained to process these surveys[1].


The authors were able to classify comments into praise or complaints and found that time-to-next complaint can turn these classifications into quantitative benchmarks that could be used to keep track of improvements over time[1].

Second Review

Add next review here.


  1. 1.0 1.1 1.2 1.3 Alemi, F., Torii, M., Clementz, L., & Aron, D. C. (2012). Feasibility of real-time satisfaction surveys through automated analysis of patients’ unstructured comments and sentiments. Quality Management in Health Care, 21(1), 9–19. http://doi.org/10.1097/QMH.0b013e3182417fc4