JOURNAL OF MEDICAL INTERNET RESEARCH Jurkeviciute et al Original Paper Identifying the Value of an eHealth Intervention Aimed at Cognitive Impairments: Observational Study in Different Contexts and Service Models Monika Jurkeviciute1, MSc; Lex van Velsen2, PhD; Henrik Eriksson1, PhD; Svante Lifvergren1,3, MD, PhD; Pietro Davide Trimarchi4, PhD; Ulla Andin3, MD, PhD; Johan Svensson3,5, MD, PhD 1 Centre for Healthcare Improvement, Chalmers University of Technology, Gothenburg, Sweden 2 eHealth Group, Roessingh Research and Development, Enschede, Netherlands 3 Skaraborg Hospital Group, Lidköping, Sweden 4 IRCCS Fondazione Don Carlo Gnocchi, Milan, Italy 5 Department of Internal Medicine and Clinical Nutrition, Institute of Medicine, Sahlgrenska Academy, University of Gothenburg, Gothenburg, Sweden Corresponding Author: Monika Jurkeviciute, MSc Centre for Healthcare Improvement Chalmers University of Technology Vera Sandbergs Allé 8 Gothenburg, 41296 Sweden Phone: 46 766061558 Email: monika.jurkeviciute@chalmers.se Abstract Background: Value is one of the central concepts in health care, but it is vague within the field of summative eHealth evaluations. Moreover, the role of context in explaining the value is underexplored, and there is no explicit framework guiding the evaluation of the value of eHealth interventions. Hence, different studies conceptualize and operationalize value in different ways, ranging from measuring outcomes such as clinical efficacy or behavior change of patients or professionals to measuring the perceptions of various stakeholders or in economic terms. Objective: The objective of our study is to identify contextual factors that determine similarities and differences in the value of an eHealth intervention between two contexts. We also aim to reflect on and contribute to the discussion about the specification, assessment, and relativity of the “value” concept in the evaluation of eHealth interventions. Methods: The study concerned a 6-month eHealth intervention targeted at elderly patients (n=107) diagnosed with cognitive impairment in Italy and Sweden. The intervention introduced a case manager role and an eHealth platform to provide remote monitoring and coaching services to the patients. A model for evaluating the value of eHealth interventions was designed as monetary and nonmonetary benefits and sacrifices, based on the value conceptualizations in eHealth and marketing literature. The data was collected using the Mini–Mental State Examination (MMSE), the clock drawing test, and the 5-level EQ-5D (EQ-5D-5L). Semistructured interviews were conducted with patients and health care professionals. Monetary data was collected from the health care and technology providers. Results: The value of an eHealth intervention applied to similar types of populations but differed in different contexts. In Sweden, patients improved cognitive performance (MMSE mean 0.85, SD 1.62, P<.001), reduced anxiety (EQ-5D-5L mean 0.16, SD 0.54, P=.046), perceived their health better (EQ-5D-5L VAS scale mean 2.6, SD 9.7, P=.035), and both patients and health care professionals were satisfied with the care. However, the Swedish service model demonstrated an increased cost, higher workload for health care professionals, and the intervention was not cost-efficient. In Italy, the patients were satisfied with the care received, and the health care professionals felt empowered and had an acceptable workload. Moreover, the intervention was cost-effective. However, clinical efficacy and quality of life improvements have not been observed. We identified 6 factors that influence the value of eHealth intervention in a particular context: (1) service delivery design of the intervention (process of delivery), (2) organizational setup of the intervention (ie, organizational structure and professionals involved), (3) cost of different http://www.jmir.org/2020/10/e17720/ XSL• FO RenderX J Med Internet Res 2020 | vol. 22 | iss. 10 | e17720 | p. 1 (page number not for citation purposes) JOURNAL OF MEDICAL INTERNET RESEARCH Jurkeviciute et al treatments, (4) hourly rates of staff for delivering the intervention, (5) lifestyle habits of the population (eg, how physically active they were in their daily life and if they were living alone or with family), and (6) local preferences on the quality of patient care. Conclusions: Value in the assessments of eHealth interventions need to be considered beyond economic terms, perceptions, or behavior changes. To obtain a holistic view of the value created, it needs to be operationalized into monetary and nonmonetary outcomes, categorizing these into benefits and sacrifices. (J Med Internet Res 2020;22(10):e17720) doi: 10.2196/17720 KEYWORDS eHealth value; evaluation of value; eHealth intervention; cognitive impairment; role of context; cost benefit Introduction The Concept of Value in eHealth The concept of value has taken a central role in health care, including eHealth development and evaluation. This trend has been ongoing in health care since 2006, when Porter and Teisberg introduced the value-based health care concept to improve and innovate healthcare [1], resulting in increased interest in the value concept. For example, some argue that there has been a shift in the health care discourse, from improvement of quality to improvement of value [2]. Moreover, many improvements using the value concept have been reported in recent years [3]. However, the concept of value is somewhat unclear within the context of summative eHealth evaluation. Subsequently, it has been approached in different ways in various studies. Sometimes, value has been investigated as positive outcomes such as clinical efficacy or behavior change of patients or professionals [4,5]. Others have aimed to identify value through the perceptions of various stakeholders [6,7]. In order to arrange reimbursement for an eHealth service and to support decisions about investments in technology development or implementation, the value of eHealth has also been interpreted in economic terms (ie, whether health outcomes justify the costs) [8]. While different evaluation frameworks have addressed some aspects of value [9], there is no framework that explicitly guides the evaluation of value in eHealth interventions. The immature conceptual and methodological base of value in eHealth can create confusion due to the large number of studies that are hard to compare, learn from, and transfer from one context to another. To explore the conceptualizations of value beyond the area of eHealth, inspiration could be taken from other disciplines, such as the marketing of products and services. One way to approach value can be as benefits and sacrifices, which measure service quality in relation to cost [10] and can reflect both monetary and nonmonetary outcomes [11]. In addition, value occurs from the interaction between a subject (eg, a patient) and an object (an eHealth intervention) and is relative. Relativity means that value is comparative (signifying that the value of one eHealth intervention can be compared to the other), personal (what is valuable for one end-user or stakeholder is not necessarily valuable for the other), and situational (ie, value depends on the context of use) [10]. Assuming that value is relative, one can question the usefulness of evaluating a certain eHealth intervention in a specific context. How do the results translate from one context to the next when the concept of value, costs, and benefits are potentially different? Systematic reviews that have analyzed eHealth intervention studies in dementia care [12] and eHealth evaluation frameworks [13] revealed that the role of context has been neglected. The current discourse treats the context as the circumstances under which an intervention is effective or not [14]. However, there are no studies that have explicitly investigated how the context influences the value of the eHealth intervention. Knowledge of these contextual factors can help to translate the interventions into practice or new settings [15-17]. To sum up, one way of conceptualizing the assessment of value in eHealth interventions could be to combine the benefit-and-sacrifice approach from marketing [8] with the value-based health care logic that assesses patient outcomes against cost [1]. In this view, benefits could refer to financial earnings or savings as monetary benefits, and eHealth service quality or utility as nonmonetary benefits [18]. Sacrifices relate to the financial investment and expenditure as monetary sacrifices, and social disadvantages (ie, what it takes to provide the service physically or emotionally) as nonmonetary sacrifices. In addition, it might also be fruitful to add an emphasis on the context, which has been lacking in eHealth studies [12,13,17]. The proposed structure of assessing the value of an eHealth intervention is depicted in Textbox 1. Textbox 1. Structure of the value assessment of an eHealth intervention. Benefits • Monetary • Nonmonetary Sacrifices • Monetary • Nonmonetary http://www.jmir.org/2020/10/e17720/ XSL• FO RenderX J Med Internet Res 2020 | vol. 22 | iss. 10 | e17720 | p. 2 (page number not for citation purposes) JOURNAL OF MEDICAL INTERNET RESEARCH In this study, we identified the value of a nonpharmacological eHealth intervention, which combined an integrated care model with eHealth and targeted to treat cognitive impairment in elderly populations in Italy and Sweden. The 2 objectives that guided this study were (1) to identify the contextual factors that determined the similarities and differences in the value of an eHealth intervention between the 2 contexts, and (2) to reflect on and contribute to the discussion about the specification, assessment, and relativity of the “value” concept in the evaluation of eHealth interventions. Context This study was a part of the European Union–funded project “Digital Environment for Cognitive Inclusion” (DECI). The entire DECI project was performed over 3 years and focused on the development of digital solutions to improve the care of elderly individuals with mild cognitive impairment (MCI) and mild dementia (MD). In our study, we examined the implementation of a 6-month eHealth intervention among patients recruited in Sweden and Italy. The inclusion criteria for participants were ≥ 60 years of age, a diagnosis of MCI or a diagnosis of dementia according to DSM-5 criteria, a clinical dementia rating (CDR) of ≤ 1, living at home, and the ability to provide informed consent or the availability of a proxy for informed consent. The exclusion Jurkeviciute et al criteria were living in a care institution, previous or present major psychiatric illness (eg, schizophrenia, bipolar disorder, or recurrent major depression), previous or present major neurological illness other than MCI or MD (eg, stroke, multiple sclerosis, brain tumor, traumatic brain injury), the presence of other serious comorbidities (eg, severe chronic obstructive pulmonary disease, severe heart disease, or severe chronic kidney failure), a history of drug or alcohol abuse, severe sensory impairments (mainly visual and auditory), a history of intellectual disability or other developmental diseases, and a life expectancy of less than 1 year (as judged by a clinician). The specific procedures for the inclusion of patients in Italy and Sweden differed. After inclusion, the 6-month eHealth intervention was initiated. Clinical efficacy variables and quality of life were assessed at baseline and at the end of the 6-month intervention. Further data in terms of monetary and nonmonetary outcomes were obtained by performing semistructured interviews 6 months after the eHealth intervention and by collecting information from the health care providers after the intervention had been completed. The DECI Intervention The DECI intervention consisted of a set of integrated eHealth applications and a service model, as depicted in Figure 1. Figure 1. Components of the Digital Environment for Cognitive Inclusion (DECI) intervention. The digital component of the intervention was centered around a web-based portal that disclosed a set of several services via single sign-on. A digital platform that served as the data repository for a patient allowed for the following main functionalities: messaging among the key actors for a patient’s care (the patient, professional caregivers, family members, and informal caregivers), sending data to and retrieving data from a patient’s electronic medical record (EMR) and other digital http://www.jmir.org/2020/10/e17720/ XSL• FO RenderX applications, and the provision and processing of patient-centered surveys. A web-based service provided the Otago fall prevention program [19] (used for improving physical health [20]) through video instruction, accompanied by written instructions and a voice-over that pronounced these written instructions. Patients could indicate the difficulty they had with exercises; based upon this feedback, the service decided whether (1) they could continue to the next level of the Otago program J Med Internet Res 2020 | vol. 22 | iss. 10 | e17720 | p. 3 (page number not for citation purposes) JOURNAL OF MEDICAL INTERNET RESEARCH after completing a week, (2) they would remain at the same level, or (3) they would go down a level. Figure 2 depicts a Jurkeviciute et al screenshot of an exercise in the Otago program. Figure 2. Screenshot of the Otago exercise program. Smartbrain [21], a web-based collection of exercises aimed at cognitive stimulation normally provided to patients with MCI, Alzheimer disease, Parkinson disease, and similar conditions, offered patients different difficulty levels. In addition, a physical activity dashboard supported by the ADAMO care watch (Caretek s.r.l) and that included pedometer functionality [22] allowed care professionals to inspect a patient’s physical activity (in terms of daily steps and active hours) and provided the patient with daily feedback about his or her physical activity levels. The patients received a tablet for accessing digital services. Next, the case manager role was introduced as the organizational element of the intervention. The case manager was responsible for coordinating care and introducing the technology to the patients. The Italian Context of the DECI Intervention The Italian National Health System provides free health care for patients throughout the country. It is funded by taxes and charges for some services. The DECI study was carried out in the Milan area (Lombardy region) at the Istituto Palazzolo (IP), Fondazione Don Gnocchi (FDG). All services are integrated to offer a care program that is shared between the elderly patients and the caregivers, and to provide follow-up along all the clinical pathways. The DECI project mainly targeted patients enrolled at the Memory Clinic of the Istituto Palazzolo. Patients eligible for inclusion were identified using the IP-FDG Memory Clinic database and were then contacted to receive information on the study and a proposal of participation. Then, an inclusion visit was arranged for written informed consent, clinical anamnesis, and a neuropsychological and functional http://www.jmir.org/2020/10/e17720/ XSL• FO RenderX examination of patients that accepted participation and met the inclusion criteria. Afterward, patients received training and delivery of the study materials. If needed, a follow-up visit was arranged. Lastly, clinical and satisfaction assessments were performed at the exit visit. The Italian DECI team consisted of 1 senior physician specialized in geriatrics and dementia, 1 social worker (case manager), 2 neuropsychologists, and 1 engineer. A physiotherapist from the rehabilitation services gave advice but did not participate actively in visits and follow-up. The patients received a 10.1-inch Samsung Android tablet with a 4G SIM card to guarantee connectivity. The case manager and one of the neuropsychologists provided instructions and training for the patient regarding the DECI platform, usually during an individualized 90-minute meeting at the memory clinic. Related information was also available through the case manager by phone or through the message system of the DECI platform. The patients used the tablet for physical and cognitive training during the 6-month intervention. An online help desk by phone was established during working hours (Monday to Friday). Inclusion visits and follow-up were carried out at the memory clinic. The Swedish Context of the DECI Intervention Health care in Sweden is tax-funded and provided by the communities (elderly care) and the counties (specialized and primary care). The DECI study was carried out in the Skaraborg region, which has a permanent interorganizational network for integrated care that delivers mobile, coordinated person-centered care for patients with various chronic diseases. DECI patients receive mobile, networked care managed by the Swedish DECI team. J Med Internet Res 2020 | vol. 22 | iss. 10 | e17720 | p. 4 (page number not for citation purposes) JOURNAL OF MEDICAL INTERNET RESEARCH The study purpose was described in advertisements in local media and leaflets that were distributed in local care units. Then, patients who contacted the Swedish DECI team were invited for screening. Patients who met the inclusion criteria and agreed to participate were visited at home for written informed consent, clinical anamnesis, and neuropsychological and functional examination; inclusion and exclusion criteria were checked further. During subsequent visits, patients received training and delivery of the material. Lastly, clinical and satisfaction assessments were performed at the exit visit at the patient’s home. The DECI team consisted of 1 senior physician specialized in geriatrics from Skaraborg Hospital Group and 1 experienced nurse from one of the communities (case manager). An occupational therapist and a physiotherapist from 2 other communities in Skaraborg gave advice but did not participate in the home visits. All patients received a 10.1-inch Samsung Android tablet with a 4G SIM card to guarantee connectivity. Jurkeviciute et al The patients were assessed using memory tests. The case manager and the geriatrician provided instructions and training for the patient regarding the DECI platform, usually during a 1-hour long meeting at the patient’s home. Information was also given during subsequent visits at the patients’ home by the case manager, in some cases with the geriatrician also present. The patient used the tablet for physical and cognitive training during the 6-month intervention. An online help desk by phone was established during working hours (Monday to Friday). Follow-up was carried out at the patients’ homes. Methods Value Specification The evaluation conducted in this study was based on Textbox 1. In order to meet the requirements and aims of the DECI project, the model presented in Textbox 1 was populated with the variables collected during the project. The specification of value for the DECI intervention is depicted in Textbox 2. Textbox 2. Value specification in the Digital Environment for Cognitive Inclusion (DECI) intervention. Benefits • • Monetary • Income • Prevented cost of treatment Nonmonetary • Clinical efficacy • Quality of life • Patient satisfaction • Job satisfaction Sacrifices • • Monetary • Investment • Operating expenses • Cost of spent time Nonmonetary • Patient safety • Workload The monetary benefits were operationalized as income and the prevented cost of treatment. The nonmonetary benefits were expressed as clinical efficacy, quality of life, patient satisfaction, and job satisfaction. The monetary sacrifices were operationalized as investment, operating expenses, and the cost of spent time. The nonmonetary sacrifices were expressed as patient safety and workload. Data Collection and Analysis Data to assess the monetary and nonmonetary outcomes of DECI (Textbox 2) were collected from the Italian and Swedish http://www.jmir.org/2020/10/e17720/ XSL• FO RenderX health care providers involved in the project. Details of the collection and analysis of data are described below. Monetary Benefits Income Income data was related to the annual state reimbursement for the treatment of MCI for a single patient. The total yearly income was calculated by multiplying the projected yearly population in the treatment by the yearly state reimbursement per patient. J Med Internet Res 2020 | vol. 22 | iss. 10 | e17720 | p. 5 (page number not for citation purposes) JOURNAL OF MEDICAL INTERNET RESEARCH Prevented Cost of Treatment Prevented cost included the postponement of care and the prevented cost of treatment of falls. Postponement of care was based on the delayed conversion from MCI to MD. Data was collected using the Clinical Dementia Rating Scale (CDR) [23] at baseline and at follow-up after 6 months. The annual cost of treatment of MD was collected from the Italian and Swedish health care providers involved in the project. In the analysis of data, the conversion rate (changes in CDR after 6 months) was multiplied by the annual targeted population and the cost of MD treatment. Data regarding falls were theorized based on the assumptions extracted from the relevant literature, since such data was not collected in the DECI study. Elderly people (from 65 years of age and older) fall 0.33 times a year [24], which can create care needs such as visits to a general practitioner (GP; 9%), emergency visits (5%) [25], or treatments for fracture (3%, based on the professional judgment of health care professionals in the DECI study). Previous studies involving the Otago physical activity coaching program have demonstrated that it helps to prevent falls by 68% [26]. The cost information for GP visits, emergency visits, and treatments for fracture was collected from the Italian and Swedish health care providers involved in the project. During the data analysis, a preventable number of falls was calculated by multiplying 0.33 by 68% and by a targeted population size. Then, a number of prevented GP visits, emergency visits, and fractures were calculated by multiplying the number of falls by 9%, 5%, and 3%, respectively. To calculate the cost, the results were multiplied by the costs for a single GP visit, emergency visit, and fracture treatment. The results were summed up to calculate the total preventable costs due to falls. Nonmonetary Benefits Clinical Efficacy The Mini–Mental State Examination (MMSE) [27] and the clock drawing test (CDT) [28] were used to assess the cognitive performance of the patients. The instruments were administered at baseline (T0) and at follow-up after 6 months of intervention (T1). Within-group differences were calculated by comparing values at T1 with those at T0 using the Wilcoxon test. Between-group differences were analyzed by comparing the changes from baseline (T1–T0) using the Mann-Whitney U test. The statistical analysis was performed using SPSS for Windows (version 24; IBM Corp). A P value of <.05 was considered statistically significant. Quality of Life The EQ-5D-5L [29] questionnaire was used to estimate quality of life at baseline and at follow-up after 6 months. Within-group differences were calculated using the Wilcoxon test, and between-group differences were analyzed by comparing the changes from baseline using the Mann-Whitney U test. Patient Satisfaction Data were collected using semistructured interviews with patients in Sweden and Italy after 6 months of having used the DECI services services (the interview protocol can be found in http://www.jmir.org/2020/10/e17720/ XSL• FO RenderX Jurkeviciute et al Multimedia Appendix 1). The sampling was purposeful [30] in order to be careful not to create too big of a cognitive burden during the assessment activities. Thematic analysis [31] of the data helped to identify the perceived benefits by the patients and the necessary sacrifices in order to deliver it. Job Satisfaction Data were collected using in-depth semistructured interviews with the health care professionals providing the DECI services in Italy and Sweden services (the interview protocol can be found in Multimedia Appendix 2). Thematic analysis [31] of the data identified the perceived benefits by the health care professionals and the necessary sacrifices in order to deliver it. Monetary Sacrifices Cost of Spent Time The spent time included the direct provision of the DECI services and participation in the multi-disciplinary meetings (those hours did not overlap). The hours spent by healthcare professionals were collected from the Swedish and Italian healthcare providers. The cost was then calculated by multiplying the number of hours by the hourly tariff. The total cost of spent time was a sum of costs of the different professional categories. Investment Investment related to the one-time cost of starting to use the DECI technologies. Investment data were collected via email from the healthcare providers. Operating Expenses Operating expenses included the annual cost of hardware (tablets), servers, the usage fees of DECI technologies, maintenance, and a help-desk function. Data were collected from the healthcare and technology providers using an Excel file containing various categories of the operating expenses. The usage fees of the DECI technology were stable, except for the fee for the ADAMO activity monitoring device (a wristwatch) that depended on the number of users. Nonmonetary Sacrifices Patient Safety Data were collected using semistructured interviews with patients and health care professionals. The data were thematically analyzed in order to identify safety-related issues. Workload Data were collected through semistructured interviews with health care professionals. The data were thematically analyzed in order to identify the workload-related issues. Monetary Benefit/Sacrifice Ratio The ratio was calculated by dividing the sum of monetary benefits by the sum of monetary sacrifices (Ratio=Benefits/Sacrifices). The ratio was calculated for the scenarios at year 1, year 2, and year 3 of using the DECI services. J Med Internet Res 2020 | vol. 22 | iss. 10 | e17720 | p. 6 (page number not for citation purposes) JOURNAL OF MEDICAL INTERNET RESEARCH Ethics The DECI study was approved by the Ethical Committee of the Fondazione Don Carlo Gnocchi and the Regional Ethical Committee of Gothenburg. Jurkeviciute et al Results Demographic Characteristics Table 1 depicts the demographic characteristics of the patients involved in the study. Table 1. Demographic characteristics of patients in Italy (n=53) and Sweden (n=54) at baseline. Characteristics Patients in Italy (n=53) Patients in Sweden (n=54) P valuea Age in years, mean (SD) 77.6 (5.3) 74.8 (5.9) <.001 Gender, n (%) .66 Female 27 (51) 30 (56) Male 26 (49) 24 (44) Diagnosis, n (%) MCIb 39 (74) 49 (91) MDc 14 (26) 5 (9) 9.2 (4.3) 11.6 (2.9) <.001 MMSE (range 0-30), mean (SD) 26.6 (2.9) 28.2 (1.4) <.001 CDTe (range 0-5), mean (SD) 3.30 (1.38) 4.81 (0.52) <.001 Education years, mean (SD) d a .007 Differences between groups were examined using the Mann-Whitney U test for continuous variables and using chi-square tests for categorical variables. b MCI: mild cognitive impairment. c MD: mild dementia. d MMSE: Mini–Mental State Examination. e CDT: clock drawing test. Monetary Benefits costs) was zero, the Swedish preventable costs consisted of the prevention of falls only. Prevented Cost of Treatment Income The results in Italy showed a 10% lower conversion rate from MCI to MD compared to the conversion rates of patients in regular care. In the Swedish patients, who were younger than the Italian patients and had a less marked decrease in cognitive function at baseline (higher MMSE and CDT scores; Table 1), the results showed a 2% higher conversion rate from MCI to MD compared to the patients in regular care, thus bringing no preventable costs from the postponed care. Since the postponement of care (which is the element of the preventable The Italian and Swedish health care systems reimburse the cost of time when providing the eHealth-supported care. Therefore, the cost of time spent on providing the DECI treatment (a category in monetary sacrifices) was considered as income for year 1, year 2, and year 3. http://www.jmir.org/2020/10/e17720/ XSL• FO RenderX Summary of Monetary Benefits Table 2 shows a summary of the monetary benefits per patient in Italy and Sweden for 3 years. J Med Internet Res 2020 | vol. 22 | iss. 10 | e17720 | p. 7 (page number not for citation purposes) JOURNAL OF MEDICAL INTERNET RESEARCH Jurkeviciute et al Table 2. Monetary benefits, in euros (a currency exchange rate of EUR €1=US $1.18 is applicable). Monetary benefits in Italy and Sweden Year 1 (€) Year 2 (€) Year 3 (€) Total 191.696 287.544 335.468 Per patient 1916 1916 1916 Total 6155 9232 10.770 Per patient 12 12 12 Total 67.938 101.906 118.891 Per patient 679 679 679 Total 826.882 1.240.324 1.447.044 Per patient 1653 1653 1653 Preventable costs - Italy Preventable costs - Sweden Income - Italy Income - Sweden The large differences in monetary values between Italy and Sweden occurred due to the different targeted population sizes. In Italy, it was 100 patients in year 1, 150 patients in year 2, and 175 patients in year 3. In Sweden, it was 500 patients in year 1, 750 patients in year 2, and 875 patients in year 3. Nonmonetary Benefits Clinical Efficacy The mean (SD) changes in MMSE and CDT scores in Italy were -0.14 (2.86) and 0.23 (1.52), respectively; in Sweden, these changes were 0.85 (1.62) and -0.11 (0.57), respectively. In Italy, both MMSE and CDT scores at the 6-month follow-up were similar to those at baseline (P=.35 and P=.34, respectively). In Sweden, MMSE scores were higher (better) at the 6-month follow-up compared to those at baseline (P<.001), whereas CDT scores were unchanged (P=.18). When comparing the results in Italy versus those in Sweden, the changes in MMSE scores were significantly greater in the Swedish cohort (P=.004), whereas there were no differences in CDT scores (P=.15). Thus, as determined by MMSE scores, cognitive performance was improved in the Swedish study population but not in the Italian study population. Quality of Life In Italy, there was no difference between the 6-month follow-up and baseline in any of the EQ-5D-5L subscales (mobility, P=.41; self-care, P=.41; activity, P=.58; pain, P=.16; anxiety, P=.59) and the EQ-5D-5L visual analog scale (VAS), P=.53. In Sweden, there were improvements in the mean (SD) changes in the EQ-5D-5L subscale anxiety and the EQ-5D-5L VAS scale, which were 0.16 (0.54) and 2.6 (9.7), respectively. The 6-month follow-up values in these variables were also significantly different from those at baseline (P=.046 and P=.035, respectively), confirming beneficial effects in these variables in the Swedish cohort. Other EQ-5D-5L subscales were unchanged in Sweden: mobility, P=.20; self-care, P=.16; activity, P=.20; and pain, P=.26. However, when comparing the results in Italy versus those in Sweden, there were no between-group differences. Therefore, although some http://www.jmir.org/2020/10/e17720/ XSL• FO RenderX improvement was observed in Sweden, there was no difference in the effect on quality of life between the countries. Patient Satisfaction In Italy, 10 patients were interviewed. The patients perceived the DECI service as simple and as containing engaging exercises, which helped the patients become more physically and cognitively active. The patients were willing to try more advanced exercises matching their physical condition, and it was deemed that a customized exercise program could increase the motivation of the patients. Some patients struggled to navigate the technologies and sought help from health care professionals or family members. Like in Sweden, the help-desk function could help to reduce the load on clinicians for solving technical questions or problems. The activity monitoring watch was appreciated by the patients, but its design could be improved in order to meet the aesthetic standards of the elderly. In Sweden, 10 patients were interviewed. The DECI service was appreciated by the patients due to the clinician-monitored exercise possibilities at home and multiple home visits by health care professionals. The patients expressed a willingness to continue using the physical and cognitive activity programs. Previous information technology (IT) experience was determined to be helpful in navigating the tablet. Less experienced patients relied on family members for help, while others called health care professionals. Therefore, an IT help desk that helps to solve tablet-related issues is a necessary element of the service. The activity monitoring watch did not meet patients’ expectations without a pedometer and a display. Job Satisfaction In Italy, 4 health care professionals were interviewed. Their occupations were a geriatrician, a social worker, a neuropsychologist, and a clinical neuropsychologist. The health care professionals felt enabled to build relationships with patients through a dedicated case manager. The physical and cognitive activity coaching programs gave the professionals tools that could not be found in usual care practices. The exercises motivated and engaged the patients, which positively contributed to the job satisfaction of the professionals. The J Med Internet Res 2020 | vol. 22 | iss. 10 | e17720 | p. 8 (page number not for citation purposes) JOURNAL OF MEDICAL INTERNET RESEARCH ability to remotely monitor patients’ performances and adherence to the tools was seen as an additional value. The professionals also utilized the messaging and data sharing functions in the DECI platform, which facilitated the interdisciplinary work. In Sweden, 2 health care professionals were interviewed. Their occupations were a geriatrician and a nurse. The professionals were satisfied that the DECI services helped to form positive relationships with the patients while visiting them in their home environments. The case manager’s role was perceived as rewarding. Professionals could deepen their knowledge of the patients when observing physical status in their usual environment. The professionals felt empowered since they could offer digital tools to the patients with beneficial, customized exercises for different muscle groups and cognitive conditions. However, the messaging function in the DECI platform was perceived as having low value, since the mobile team was mostly on the road and used mobile phones for communication. Jurkeviciute et al Monetary Sacrifices Investment In Italy, the one-time investment concerned the cost of staff for server preparation in the hospital. In Sweden, the investment concerned the cost of staff for server preparation and installation of the DECI technologies. Additionally, investments in Sweden also entailed one-time costs for purchasing tablets, including 4G-sim cards for training for patients (n=52) and members of the DECI-team (n=2). Operating Expenses In both Italy and Sweden, the highest usage fee concerned the ADAMO activity monitoring device. The annual operating costs in Italy include purchasing the tablets (a stable number of tablets purchased for patients every year, depreciation in 12 months), server hosting, the configuration of the tablets, the personnel cost of maintenance and the help desk, licenses, and 4G connectivity. In Sweden, it consists of the server hosting, first-line help desk, and management costs. Operating expenses are depicted in Table 3. Table 3. Monetary sacrifices, in euros (a currency exchange rate of EUR €1=US $1.18 is applicable). Monetary sacrifices in Italy and Sweden Year 1 (€) Year 2 (€) Year 3 (€) Total 71.161 106.743 124.533 Per patient 711 711 711 Total 693.773 969.160 1.130.687 Per patient 1387 1387 1387 Total 140 0 0 Per patient 1.4 0 0 Total 18.636 18.636 18.636 Per patient 37 24 21 Total 126.746 171.896 194.471 Per patient 1267 1145 1111 Total 348.015 499.965 575.940 Per patient 696 666 658 Cost of spent time - Italy Cost of spent time - Sweden Investment - Italy Investment - Sweden Operating expenses - Italy Operating expenses - Sweden Cost of Spent Time The cost was calculated based on the hours spent on the DECI service provision and the multi-disciplinary meetings (Table 3). The following professional categories were involved in Italy: physician, nurse practitioner, physiotherapist, technician, case manager (social worker), and psychologist. In Sweden, the cost was calculated for a geriatrician, an occupational therapist, a physiotherapist, and a nurse practitioner. http://www.jmir.org/2020/10/e17720/ XSL• FO RenderX Table 3 shows a summary of the monetary sacrifices per patient in Italy and Sweden for 3 years. The large differences in monetary values between Italy and Sweden occurred due to the different targeted population sizes and care models. Nonmonetary Sacrifices Patient Safety In Italy, the ADAMO activity monitoring device caused an allergic reaction in 1 patient, due to sensitivity to nickel and J Med Internet Res 2020 | vol. 22 | iss. 10 | e17720 | p. 9 (page number not for citation purposes) JOURNAL OF MEDICAL INTERNET RESEARCH plastic. The health care professionals in Italy (n=4, 1 geriatrician, 1 social worker, 1 neuropsychologist, and 1 clinical neuropsychologist) and Sweden (n=2, 1 geriatrician and 1 nurse) noted that the physical activity training program Otago could be less safe for older people if used without supervision. Hence, the level of exercise difficulty is of high importance. Workload In Italy, the 4 health care professionals dedicated, on average, 29.3 hours a week to engage with the existing patients at the memory clinic that took part in the DECI study (the result is based on their self-reported data). A substantial amount of this time was dedicated to digital data entry for the DECI study. Jurkeviciute et al This time was used for patient inclusion, training, phone calls, solving technical issues, and digital data entry for the study. In Sweden, the 2 health care professionals spent, on average, 52.5 hours per week, thus reporting overtime. This time was used for the full-scale dementia examination, training, phone calls, home visits to the patients, solving technical issues, and digital data entry for the study. The result is based on their self-reported data. Monetary Benefit/Sacrifice Ratio Table 4 depicts a summary of monetary benefits and sacrifices, and provides a calculation of the benefit/sacrifice ratio in Italy and Sweden for 3 years. Table 4. Summary of the DECI scenario in monetary value, in euros (a currency exchange rate of EUR €1=US $1.18 is applicable). Benefits and sacrifices in Italy and Sweden Year 1 Year 2 Year 3 Total 259.633 389.450 454.359 Per patient 2596 2596 2596 Total 833.037 1.249.555 1.457.815 Per patient 1666 1666 1666 Total 198.047 278.639 319.004 Per patient 1980 1857 1822 Total 1.060.424 1.487.761 1.725.263 Per patient 2120 1983 1971 Benefit/sacrifice ratio - Italy 1.31 1.39 1.42 Benefit/ sacrifice ratio - Sweden 0.78 0.84 0.84 Total monetary benefits – Italy (€) Total monetary benefits – Sweden (€) Total monetary sacrifices – Italy (€) Total monetary sacrifices – Sweden (€) The benefit/sacrifice ratio showed that the Italian intervention could bring positive monetary value from the first year onward. In Sweden, the intervention did not bring monetary value during the first 3 years. However, the gap between monetary benefits and sacrifices reduces with the growing number of patients. Discussion Principal Findings This study was guided by 2 objectives: (1) to identify the contextual factors that determine the similarities and differences in the value of an eHealth intervention between 2 contexts, and (2) to reflect on and contribute to the discussion about the specification, assessment, and the relativity of the “value” concept in evaluating eHealth interventions. This study was based on the implementation of an eHealth platform for remote home monitoring of physical and cognitive activity for people suffering from cognitive impairment in Italy and Sweden. The findings of this study show that there is a differing value derived from the implementation of the same eHealth technology to similar types of populations in different contexts. http://www.jmir.org/2020/10/e17720/ XSL• FO RenderX We translated value into benefits and sacrifices and assessed these for the intervention in 2 countries. In Sweden, the identified benefits of the eHealth intervention included improved cognitive performance assessed by the MMSE, reduced patient anxiety assessed by the EQ-5D-5L, better perceived health estimated using the EQ-5D-5L VAS scale, and satisfaction with the care received. However, these benefits can require sacrifices, such as an increased cost and higher workload for health care professionals. With the service model of Sweden (home visits), a lower number of patients could be visited per day. Additionally, the relatively high hourly rates of the staff increased the cost of the intervention. In Italy, the identified benefits included patient satisfaction with the care received, empowered health care professionals, and acceptable workloads. Moreover, for the Italian patients, who were older and had a more marked decrease in cognitive function (lower MMSE and CDT scores) at baseline than the Swedish patients, the intervention could bring positive monetary value from the first year onward. This was the result of a higher preventable cost of treatment and state reimbursement, in comparison to the cost that was based on relatively lower hourly staff rates and the service model that reduced the time spent per patient (the intervention was implemented on a sample of existing patients J Med Internet Res 2020 | vol. 22 | iss. 10 | e17720 | p. 10 (page number not for citation purposes) JOURNAL OF MEDICAL INTERNET RESEARCH of the clinic and the visits were performed at the clinic). However, the clinical efficacy and quality-of-life improvements have not been observed over the course of the 6-month intervention in Italy. In this study, the monetary side of value was influenced by factors such as (1) the service delivery design of the intervention (process of delivery), (2) the organizational setup of the intervention (ie, organizational structure and professionals involved), (3) the cost of different treatments, and (4) the hourly rates of staff for delivering the intervention. These factors affected the cost-effectiveness through the expenses incurred (including potentially preventable costs due to the intervention) and necessary investments. The nonmonetary side of the value of the intervention was also influenced by the service delivery design and organizational setup of the intervention, as well as by a fifth factor: (5) the lifestyle habits of the population (eg, Jurkeviciute et al how physically active they were in their daily lives and if they were living alone or with family). Finally, the value of eHealth should be seen against the sixth factor: (6) local preferences on the quality of patient care. This study showed that even the non–cost-efficient intervention can be viewed as valuable locally and deemed worthy of implementation. In such a case, local preferences on the quality of patient care can be a decisive factor. Particular positive nonmonetary outcomes might be valued highly enough to proceed with adopting the eHealth-supported service model. Moreover, it should not be neglected that the service delivery design and organizational set-up of the intervention can be adjusted to make it more cost-efficient. A summary of the contextual factors affecting the value of eHealth intervention is provided in Textbox 3. Textbox 3. Conceptual model for eHealth value specification and the influencing contextual factors. Contextual factors (benefits) • • Monetary • Service delivery design of an intervention (process) • Organizational setup of an intervention (structure) • Cost of different treatments • Hourly rates of staff for delivering an intervention Nonmonetary • Service delivery design of an intervention (process) • Organizational setup of an intervention (structure) • Lifestyle habits of the population • Local preferences on the quality of patient care Contextual factors (sacrifices) • • Monetary • Service delivery design of an intervention (process) • Organizational setup of an intervention (structure) • Cost of different treatments • Hourly rates of staff for delivering an intervention Nonmonetary • Service delivery design of an intervention (process) • Organizational setup of an intervention (structure) Limitations The identified contextual factors affecting the value of eHealth interventions could be limited because the study was based on a summative eHealth evaluation conducted in 2 countries. To study 2 countries in-depth demanded time and effort, and including more countries was not feasible. Other contextual setups could enrich the list of the factors and need to be investigated further. Also, the study was constrained by a 6-month follow-up time for the patients. A longer follow-up http://www.jmir.org/2020/10/e17720/ XSL• FO RenderX time could enrich the contextual factors and the proposed eHealth value conceptualization by revealing long-term effects. Comparison with Prior Work The conceptual model proposed for assessing the value of eHealth interventions (Textbox 3) was built on prominent eHealth evaluation frameworks [32,33], in addition to previous conceptualizations of value in the eHealth [4-8] and marketing literature [10]. We argue that value needs to be operationalized in both monetary and nonmonetary outcomes, and our model J Med Internet Res 2020 | vol. 22 | iss. 10 | e17720 | p. 11 (page number not for citation purposes) JOURNAL OF MEDICAL INTERNET RESEARCH suggests categorizing them into benefits and sacrifices. In practice, an evaluation study on value needs to adapt the model to its needs by populating the model with the themes of evaluation. It is important not to overlook the nonmonetary aspects that can reveal a broader and more accurate view of the value created (in contrast to the cost-versus-outcomes view [1,8,34,35]). We propose that all the parts of the model (monetary and nonmonetary benefits/sacrifices) need to be assessed in order to obtain a holistic view of the value created. Regarding the conceptualization of value for eHealth interventions, our study showed that an overly limited view on value is obtained if assessing it as only positive outcomes, such as behavior change or clinical efficacy [4,5], perceptions regarding the added value by various stakeholders [6,7], or economic outcomes [8]. The view of value as clinical efficacy [4] is sometimes not possible when studies have a shorter follow-up time. Furthermore, this study showed that economic outcomes might be an overly limited measure of value, when the non–cost-efficient intervention can be viewed as valuable locally for other positive outcomes and deemed worthy for implementation in practice despite the increased cost. We suggest the following conceptualization of value for eHealth interventions: Jurkeviciute et al Value is a holistic view of the created monetary and nonmonetary benefits of eHealth that require monetary and nonmonetary sacrifices in a particular context. Finally, we argue that more studies need to apply a holistic view to the summative assessments of value, rather than focusing on defining value for the stakeholders during the design process [36]. This holistic value assessment could help to clarify which adjustments could be made to reduce sacrifices and maximize benefits. Moreover, such assessments (especially across multiple contexts) could improve the transferability of eHealth interventions. Conclusions This study offers a step toward better conceptualization of the value concept within the eHealth context. We argue that it should be interpreted as both monetary and nonmonetary benefits and sacrifices achieved in a context. Next, we argue that without considering value beyond economic terms or assessing it as merely perceptions, a full picture of the value created might be missed. We offer a model that provides some conceptual considerations for the assessment of value of eHealth. Applied within a summative evaluation in this cross-context study, it can be a useful starting point for future research. Acknowledgments The collection of data used in this study was financed by the European Union’s Horizon 2020 research and innovation program under grant agreement no. 643588. The funding organization had no role in designing the study, collecting and analyzing the data, or in preparing this manuscript. Conflicts of Interest LvV works at one of the enterprises that supplied an eHealth solution to the DECI project and the current study. Multimedia Appendix 1 Patient interview protocol. [DOCX File , 17 KB-Multimedia Appendix 1] Multimedia Appendix 2 Healthcare professional interview protocol. [DOCX File , 17 KB-Multimedia Appendix 2] References 1. 2. 3. 4. 5. 6. 7. Porter M, Teisberg E. Redefining Competition in Health Care. Boston: Harvard Business School Press; 2006. Gray M. Value: Operations Research and the new health care paradigm. Oper Res Health Care 2012;1:20-21 [FREE Full text] [doi: 10.1016/j.orhc.2012.01.002] Colldén C, Gremyr I, Hellström A, Sporraeus D. A value-based taxonomy of improvement approaches in healthcare. J Health Organ Manag 2017 Jun 19;31(4):445-458. [doi: 10.1108/jhom-08-2016-0162] Ahern DK, Kreslake JM, Phalen JM. What is eHealth (6): perspectives on the evolution of eHealth research. J Med Internet Res 2006 Mar;8(1):e4 [FREE Full text] [doi: 10.2196/jmir.8.1.e4] [Medline: 16585029] Talboom-Kamp EP, Verdijk NA, Harmans LM, Numans ME, Chavannes NH. An eHealth Platform to Manage Chronic Disease in Primary Care: An Innovative Approach. Interact J Med Res 2016 Feb 09;5(1):e5. [doi: 10.2196/ijmr.4217] Runz-Jørgensen SM, Schiøtz ML, Christensen U. Perceived value of eHealth among people living with multimorbidity: a qualitative study. J Comorb 2017;7(1):96-111 [FREE Full text] [doi: 10.15256/joc.2017.7.98] [Medline: 29359124] Ariens LF, Schussler-Raymakers FM, Frima C, Flinterman A, Hamminga E, Arents BW, et al. Barriers and Facilitators to eHealth Use in Daily Practice: Perspectives of Patients and Professionals in Dermatology. J Med Internet Res 2017 Sep 05;19(9):e300 [FREE Full text] [doi: 10.2196/jmir.7512] [Medline: 28874336] http://www.jmir.org/2020/10/e17720/ XSL• FO RenderX J Med Internet Res 2020 | vol. 22 | iss. 10 | e17720 | p. 12 (page number not for citation purposes) JOURNAL OF MEDICAL INTERNET RESEARCH 8. 9. 10. 11. 12. 13. 14. 15. 16. 17. 18. 19. 20. 21. 22. 23. 24. 25. 26. 27. 28. 29. 30. 31. Bergmo TS. How to Measure Costs and Benefits of eHealth Interventions: An Overview of Methods and Frameworks. J Med Internet Res 2015 Nov;17(11):e254 [FREE Full text] [doi: 10.2196/jmir.4521] [Medline: 26552360] Greenhalgh T, Wherton J, Papoutsi C, Lynch J, Hughes G, A'Court C, et al. Beyond Adoption: A New Framework for Theorizing and Evaluating Nonadoption, Abandonment, and Challenges to the Scale-Up, Spread, and Sustainability of Health and Care Technologies. J Med Internet Res 2017 Nov 01;19(11):e367 [FREE Full text] [doi: 10.2196/jmir.8775] [Medline: 29092808] Gummerus J. Value creation processes and value outcomes in marketing theory. Mark Theory 2013 Jan 24;13(1):19-46. [doi: 10.1177/1470593112467267] Ulaga W, Chacour S. Measuring Customer-Perceived Value in Business Markets. Ind Mark Manag 2001 Aug;30(6):525-540. [doi: 10.1016/s0019-8501(99)00122-4] Christie HL, Bartels SL, Boots LM, Tange HJ, Verhey FR, de Vugt ME. A systematic review on the implementation of eHealth interventions for informal caregivers of people with dementia. Internet Interv 2018 Sep;13:51-59. [doi: 10.1016/j.invent.2018.07.002] Eslami Andargoli A, Scheepers H, Rajendran D, Sohal A. Health information systems evaluation frameworks: A systematic review. Int J Med Inform 2017 Jan;97:195-209. [doi: 10.1016/j.ijmedinf.2016.10.008] Glasgow RE, Phillips SM, Sanchez MA. Implementation science approaches for integrating eHealth research into practice and policy. Int J Med Inform 2014 Jul;83(7):e1-e11. [doi: 10.1016/j.ijmedinf.2013.07.002] Edwards N, Barker PM. The Importance of Context in Implementation Research. J Acquir Immune Defic Syndr 2014;67:S157-S162. [doi: 10.1097/qai.0000000000000322] Vernooij-Dassen M, Moniz-Cook E. Raising the standard of applied dementia care research: addressing the implementation error. Aging Ment Health 2014 May 12;18(7):809-814. [doi: 10.1080/13607863.2014.899977] Ammenwerth E, Brender J, Nykänen P, Prokosch H, Rigby M, Talmon J. Visions and strategies to improve evaluation of health information systems. In: International Journal of Medical Informatics. 2004 Jun Presented at: Visions and strategies to improve evaluation of health information systems: Reflections and lessons based on the HIS-EVAL workshop in Innsbruck. Int J Med Inform ;73(6); 2004; Innsbruk p. 479-491. [doi: 10.1016/j.ijmedinf.2004.04.004] Brady MK, Robertson CJ. An exploratory study of service value in the USA and Ecuador. Int J Serv Ind Manag 1999 Dec;10(5):469-486. [doi: 10.1108/09564239910289003] Campbell AJ, Robertson MC, Gardner MM, Norton RN, Tilyard MW, Buchner DM. Randomised controlled trial of a general practice programme of home based exercise to prevent falls in elderly women. BMJ 1997 Oct 25;315(7115):1065-1069. [doi: 10.1136/bmj.315.7115.1065] Dekker-van Weering M, Jansen-Kosterink S, Frazer S, Vollenbroek-Hutten M. User Experience, Actual Use, and Effectiveness of an Information Communication Technology-Supported Home Exercise Program for Pre-Frail Older Adults. Front. Med 2017 Nov 27;4:208. [doi: 10.3389/fmed.2017.00208] Tárraga L, Boada M, Modinos G, Espinosa A, Diego S, Morera A, et al. A randomised pilot study to assess the efficacy of an interactive, multimedia tool of cognitive stimulation in Alzheimer's disease. J Neurol Neurosurg Psychiatry 2006 Oct;77(10):1116-1121 [FREE Full text] [doi: 10.1136/jnnp.2005.086074] [Medline: 16820420] Magistro D, Brustio PR, Ivaldi M, Esliger DW, Zecca M, Rainoldi A, et al. Validation of the ADAMO Care Watch for step counting in older adults. PLoS ONE 2018 Feb 9;13(2):e0190753. [doi: 10.1371/journal.pone.0190753] Morris JC. Clinical Dementia Rating: A Reliable and Valid Diagnostic and Staging Measure for Dementia of the Alzheimer Type. Int. Psychogeriatr 2005 Jan 10;9(S1):173-176. [doi: 10.1017/s1041610297004870] Gillespie L, Robertson M, Gillespie W, Sherrington C, Gates S, Clemson L, et al. Interventions for preventing falls in older people living in the community. Cochrane database syst rev 2012;9:CD007146. [doi: 10.1002/14651858.cd007146.pub3] Berg WP, Alessio HM, Mills EM, Tong C. Circumstances and consequences of falls in independent community-dwelling older adults. Age Ageing 1997;26(4):261-268. [doi: 10.1093/ageing/26.4.261] Thomas S, Mackintosh S, Halbert J. Does the 'Otago exercise programme' reduce mortality and falls in older adults?: a systematic review and meta-analysis. Age Ageing 2010 Sep 04;39(6):681-687. [doi: 10.1093/ageing/afq102] Folstein MF, Folstein SE, McHugh PR. Mini-mental state: a practical method for grading the cognitive state of patients for the clinician. Int. J. Geriat. Psychiatry 1975;12(3):189-198. [doi: 10.1002/(sici)1099-1166(199805)13:5<285::aid-gps753>3.0.co;2-v] Shulman KI. Clock-drawing: is it the ideal cognitive screening test? Int J Geriat Psychiatry 2000 Jun;15(6):548-561. [doi: 10.1002/1099-1166(200006)15:6<548::aid-gps242>3.0.co;2-u] EuroQol Group. EuroQol - a new facility for the measurement of health-related quality of life. Health Policy 1990 Dec;16(3):199-208. [doi: 10.1016/0168-8510(90)90421-9] Palinkas LA, Horwitz SM, Green CA, Wisdom JP, Duan N, Hoagwood K. Purposeful Sampling for Qualitative Data Collection and Analysis in Mixed Method Implementation Research. Adm Policy Ment Health 2015 Sep;42(5):533-544 [FREE Full text] [doi: 10.1007/s10488-013-0528-y] [Medline: 24193818] Aronson J. A pragmatic view of thematic analysis. Qual Rep(1). 1995. URL: https://nsuworks.nova.edu/tqr/vol2/iss1/3/ [accessed 2019-12-28] http://www.jmir.org/2020/10/e17720/ XSL• FO RenderX Jurkeviciute et al J Med Internet Res 2020 | vol. 22 | iss. 10 | e17720 | p. 13 (page number not for citation purposes) JOURNAL OF MEDICAL INTERNET RESEARCH 32. 33. 34. 35. 36. Jurkeviciute et al Kidholm K, Ekeland AG, Jensen LK, Rasmussen J, Pedersen CD, Bowes A, et al. A model for assessment of telemedicine applications: mast. Int J Technol Assess Health Care 2012 Jan;28(1):44-51. [doi: 10.1017/S0266462311000638] [Medline: 22617736] Lampe K, Mäkelä M, Garrido MV, Anttila H, Autti-Rämö I, Hicks NJ, et al. The HTA Core Model: A novel method for producing and reporting health technology assessments. Int J Technol Assess Health Care 2009 Dec 23;25(S2):9-20. [doi: 10.1017/s0266462309990638] Roski J, Bo-Linn GW, Andrews TA. Creating value in health care through big data: opportunities and policy implications. Health Aff 2014 Jul;33(7):1115-1122. [doi: 10.1377/hlthaff.2014.0147] [Medline: 25006136] Porter ME. What Is Value in Health Care? N Engl J Med 2010 Dec 23;363(26):2477-2481. [doi: 10.1056/NEJMp1011024] van Gemert-Pijnen JE, Nijland N, van Limburg M, Ossebaard HC, Kelders SM, Eysenbach G, et al. A holistic framework to improve the uptake and impact of eHealth technologies. J Med Internet Res 2011 Dec;13(4):e111 [FREE Full text] [doi: 10.2196/jmir.1672] [Medline: 22155738] Abbreviations CDR: clinical dementia rating CDT: clock drawing test DECI: Digital Environment for Cognitive Inclusion FDG: Fondazione Don Gnocchi GP: general practitioner IP: Istituto Palazzolo IT: information technology MCI: mild cognitive impairment MD: mild dementia MMSE: Mini–Mental State Examination Edited by G Eysenbach; submitted 07.01.20; peer-reviewed by E Aramaki, F Adebesin; comments to author 10.03.20; revised version received 17.05.20; accepted 14.06.20; published 08.10.20 Please cite as: Jurkeviciute M, van Velsen L, Eriksson H, Lifvergren S, Trimarchi PD, Andin U, Svensson J Identifying the Value of an eHealth Intervention Aimed at Cognitive Impairments: Observational Study in Different Contexts and Service Models J Med Internet Res 2020;22(10):e17720 URL: http://www.jmir.org/2020/10/e17720/ doi: 10.2196/17720 PMID: 33064089 ©Monika Jurkeviciute, Lex van Velsen, Henrik Eriksson, Svante Lifvergren, Pietro Davide Trimarchi, Ulla Andin, Johan Svensson. Originally published in the Journal of Medical Internet Research (http://www.jmir.org), 11.10.2020. This is an open-access article distributed under the terms of the Creative Commons Attribution License (https://creativecommons.org/licenses/by/4.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work, first published in the Journal of Medical Internet Research, is properly cited. The complete bibliographic information, a link to the original publication on http://www.jmir.org/, as well as this copyright and license information must be included. http://www.jmir.org/2020/10/e17720/ XSL• FO RenderX J Med Internet Res 2020 | vol. 22 | iss. 10 | e17720 | p. 14 (page number not for citation purposes)