Features Related to Patient Portal User Satisfaction: N-Gram-Based Analysis of Users' Feedback

"U.S. health care spending grew 5.8 percent in 2015, reaching $3.2 trillion or $9,990 per person. As a share of the nation's Gross Domestic Product, health spending accounted for 17.8 percent" [1]. Therefore, an intensive national effort to improve healthcare using information technology (IT) with a focus on reducing costs and increasing quality of service is well underway. In this regard, patient portals, known as personal health records, show promise as tools that patients value and that can reduce healthcare cost and improve health. These Health Information Technology (HIT) are positioned as a central component of patient engagement through the potential to change the physician-patient relationship and enable chronic disease self-management. Patient portals can lead to improvements in clinical outcomes, patient behavior, and experiences. However, portal adoption is still low, due to technological limitations and to the lack of adaptability to primary care practice workflow [2]. Large studies in outpatient settings have found that providing patients with adequate functionalities leads to increases in patient satisfaction and then adherence to patient portal [3]. In fact, patient portal user satisfaction is increasingly recognized as an important component of quality [4]. However, little is known about the different patient portal characteristics that are associated with higher patient satisfaction. It seems there is insufficient evidence to support how portals empower patients and improve quality of care. According to the literature, many studies have addressed the relationship between the use of health information technologies and patients' satisfaction [4-7]. Despite the fact that there is some evidence that such technologies improve and enhance patients' satisfaction, there exist some inconsistencies in the findings and reported results [4]. The literature also highlighted the need for further research that focuses on use of the patient portal and measures of quality indicators such as medical outcomes, medication adherence, and patient satisfaction [5]. In this study, we systematically analyze users' reviews of mobile patient portal to extract features that are associated with patient satisfaction. To this end, we use user rating as a proxy for user satisfaction and adopt word-level n-grams to represent user reviews. We use MyChart reviews as Epic has captured significant market share with at least partial health information for 51% of the US population. It has been described as the default EHR choice not for its superior performance, but because other systems are considered inferior [8]. Specifically, in this research, we aim to identify predictors of patient satisfaction based on a systematic analysis of user feedback from actual use of patient portal. The data were collected using a web crawler. We obtain our data set consisting of 500 reviews. For data preprocessing, we removed stop words and represented user reviews using vectors of word-level n-grams weights. For the word n-grams, we include unigram, bigrams, and trigrams. We perform feature selection using the commonly used Chi-square (X2) method. To evaluate the predictive power of the features selected, we chose four evaluation metrics, precision, recall, accuracy, and F1 Score. Results analysis show that the majority of selected features are related to the ease of use of the patient portal, other features are related to specific features and functions the users can use within the application such as scheduling appointments, communication with health providers, using the calendar, etc. Results also report "touch id", fingerprint recognition feature, which is a security related feature that allows users to log in to their portal. The performance results of the features selected in predicting user satisfaction using different classifiers such as decision tree, linear SVC, ridge classifier, logistic regression, Bernoulli Naïve Bayes (NB), and random forest show very good performance with accuracy ranging 73%-80%, F1 ranging 81%-86%, Precision ranging 88%-96%, and Recall ranging 75%-79%.