DOI: https://doi.org/10.1145/3290605.3300475
CHI '19: Proceedings of CHI Conference on Human Factors in Computing Systems, Glasgow, Scotland UK, May 2019
In the last decade, the number of articles on HCI and health has increased dramatically. We extracted 139 papers on depression, anxiety and bipolar health issues from 10 years of SIGCHI conference proceedings. 72 of these were published in the last two years. A systematic analysis of this growing body of literature revealed that most innovation happens in automated diagnosis, and self-tracking, although there are innovative ideas in tangible interfaces. We noted an overemphasis on data production without consideration of how it leads to fruitful interventions. Moreover, we see a need to promote ethical practices for involvement of people living with affective disorders. Finally, although only 16 studies evaluate technologies in a clinical context, several forms of support and intervention illustrate how rich insights are gained from evaluations with real patients. Our findings highlight potential for growth in the design space of affective health technologies.
ACM Reference Format:
Pedro Sanches, Axel Janson, Pavel Karpashevich, Camille Nadal, Chengcheng Qu, Claudia Dauden Roquet, Muhammad Umair, Charles Windlin, Gavin Doherty, Kristina Höök, and Corina Sas. 2019. HCI and Affective Health: Taking stock of a decade of studies and charting future research directions. In CHI Conference on Human Factors in Computing Systems Proceedings (CHI 2019), May 4–9, 2019, Glasgow, Scotland UK. ACM, New York, NY, USA 19 Pages. https://doi.org/10.1145/3290605.3300475
Mental health and wellbeing is a rapidly growing area within Human-Computer Interaction (HCI). In recent years, many have developed systems, theoretical stances, and methodologies that aim at positioning Information and Communication Technologies (ICT) as an important component of therapies, prevention strategies or self-management for people dealing with affective disorders, as well as their peers, caregivers or clinical staff. Affective disorders, such as bipolar or depression, are a category of mental health illness characterized by a distorted emotional state which interferes with the ability to function [3].
Research on mental health in HCI sometimes spans the full spectrum of the development cycle: from gathering requirements for designing technologies for specific groups of users, to designing novel prototypes and finally to evaluating the uptake of novel technologies by users in healthcare contexts. These evaluations range from design feedback studies to clinical studies aiming to systematically establish their effect.
Designing healthcare systems supporting users with mental health conditions is by its very nature a delicate endeavor, addressing a vulnerable user group, requiring that ethical considerations are taken into account.
As we entered into a novel project1 aiming to design systems supporting processes of emotion regulation, we saw the need to gain a better understanding of the research front in HCI. Our aim was to both inform our own work and contribute to the field by synthesizing recent research in the area of mental health (as has recently been carried out for health research with low-SES and minority individuals [149]), and chart future directions. The systematic review of literature herein addresses three main research questions. RQ1: What is the design space of technologies for affective health explored so far in the field of HCI?; RQ2: Which ethical approaches are followed or developed within the design of these technologies, regarding sensitive user groups?; RQ3: Has HCI work on affective health been validated through clinical trials, and how?
With the first research question, we aim to broadly characterize the design space of technologies in HCI in regard to the main function of the system or methodology being developed. What functions do the developed systems perform (e.g. diagnosing, logging data, or social support). We were curious to know what kinds of designs were most common, which therapies they were built on, and what opportunities might exist to further explore innovative designs. The second research question aims at delineating the space of how ethical guidelines and considerations are approached in the design and evaluation of these systems, with special attention to how vulnerable populations are treated. The third research question aims at characterizing how clinical evaluations have been employed in HCI. While HCI research does not necessarily need to involve clinical studies (as we will discuss in detail below), we would like to see if there were any such studies at all, and if so, what feedback they would provide into our own design processes.
After a systematic analysis of 139 papers in the ACM digital library, we found that only 16 papers describe a clinical study, and only 48 papers explicitly mention and deal with ethical issues. Much of the research either deals with diagnosis (23.7%) or self-tracking (20.1%) and supports a narrow range of therapeutic methods for affective disorders. Based on our findings, we propose three future research directions. First, HCI ought to be pushing the state of the art on technological affordances, for example on novel tangible interfaces or data-driven technologies, while engaging with recent advances in therapies for affective disorders [66]. In the past, new technologies, e.g. virtual reality, enabled new kinds of therapies, e.g. virtual reality (VR) exposure therapy, while at the same time therapies, e.g. cognitive-behavioral therapy, enabled novel design spaces for HCI, e.g. computerized cognitive behavioral therapy (CBT). We would like to encourage collaboration in the research fronts of the two fields, as only then can new therapies and technologies feed on each other in novel and interesting ways. Second, we would like to see more new designs being taken all the way to clinical trials, as only then can their benefits for affective health be determined. At the same time, there is lot to learn from patients in clinical settings, which can also spur novel designs that can actually make a difference in the lives of people living with mental ill-health. Finally, we would like to see more ethically sensitive design practices being applied to this area. For example, more participatory design methods including the voices of people living with affective disorders, as the literature often exposes limited understanding of their realities, and more consideration of risks and benefits of developing technology in this domain.
The most common affective spectrum disorders are depression and bipolar disorders, with a very high co-morbidity of anxiety disorders [2, 3, 78]. We have chosen to limit the search terms to the most common affective spectrum disorders, which are estimated by 2020 to be the highest ranked cause of disease in the developed world [39]. This represents an important, tractable, and coherent subset of the large volume of mental health research in HCI. Therefore we searched for papers that mentioned ”depression”, ”anxiety” or ”bipolar” in the ACM digital library in the period 2007 — 2017 (when we started the literature survey), published in all SIGCHI sponsored conferences and affiliated publication venues. A pilot study was first performed through a keyword search in titles and abstract in the ACM digital library. The initial results (94 papers) were coded inductively by a team of three researchers to articulate the research questions and delineate the scope. Since the pilot study revealed that the keyword search was missing relevant papers, a full-text search was performed instead for the same terms, capturing publications that mention these terms anywhere in the paper. This search yielded 2775 papers. We then developed strict relevance criteria to select papers for inclusion, which yielded a final corpus of 139 publications.
A paper was classified as relevant if at least one of these conditions apply: 1) addresses depression, anxiety, or bipolar as a main disorder, 2) addresses general mental health issues, of which depression, anxiety and bipolar are included or 3) addresses other mental or physical conditions; however, depression, anxiety, bipolar are explicitly identified as caused by the condition (e.g. anxiety and panic attacks for autistic children). A paper was classified as irrelevant if: 1) depression, anxiety and bipolar are not mentioned at all, 2) Other definitions of depression, bipolar, anxiety are used in the paper — “depression” as financial recession, or ”bipolar” as a type of a transistor — or 3) the terms are used in ”general sense”, as the name of the metrics or a short-term mood.
Code | Frequency (%) |
---|---|
Development cycle | |
Requirements and Context | 50 (36.0%) |
Design and Development | 31 (22.3%) |
Non-clinical Evaluation | 42 (30.2%) |
Clinical Evaluation | 16 (11.5%) |
Main stakeholder | |
Adult sufferers | 84 (60.4%) |
Children sufferers | 12 (8.6%) |
Peers and Significant others | 3 (2.2%) |
Caregivers | 4 (2.9%) |
Clinical staff | 5 (3.6%) |
Researchers | 31 (22.3%) |
User journey | |
Prevention | 4 (2.9%) |
Diagnosis and Triage | 29 (20.9%) |
Treatment/intervention | 37 (26.6%) |
Self management and maintenance | 32 (23.0%) |
No specific stage | 37 (26.6%) |
Outcome | |
Clinical | 18 (12.9%) |
Proximal | 27 (19.4%) |
Adherence | 8 (5.8%) |
Other | 18 (12.9%) |
No outcome measured | 68 (48.9%) |
Primary system function | |
Diagnosis | 33 (23.7%) |
Self-tracking | 28 (20.1%) |
Biofeedback | 12 (8.6%) |
Structured psychological interventions | 13 (9.4%) |
Social support | 14 (10.1%) |
Mindfulness | 9 (6.5%) |
Other | 30 (21.6%) |
Ethical principles | |
Autonomy | 25 (18.0%) |
Nonmaleficence | 18 (12.9%) |
Beneficence | 3 (2.2%) |
Justice | 2 (1.4%) |
No mention of ethics | 91 (65.5%) |
One coder downloaded the full text of each paper in the initial corpus (2775), and interpreted the text against the relevance criteria, yielding 139 relevant papers and 2636 irrelevant papers. We then followed a two-step process. First, the 139 papers considered relevant were divided into 5 groups of 2 coders (10 independent coders in total) for deductive coding, using the manual described below (see Table 1 for code distribution). Secondly, following the initial coding, each set of papers within a code was subsequently inductively analyzed by the same coding teams in order to extract commonalities and differences, e.g. papers marked as clinical evaluations were analyzed for common themes and compared to papers who report non-clinical evaluations.
Development cycle. Requirements and context refers to studies aimed at informing design work, design and development presents novel designs but lack validation. When there is validation, some work performed non-clinical evaluation, i.e. typical user studies, whereas clinical evaluation studies involve people with mental health difficulties and typically in a clinical setting.
Main stakeholder. We distinguished between two types of primary target user: adult or children and adolescents. Some work was aimed at significant others or peers of sufferers, whereas some research was aimed at caregivers or clinical staff dealing with sufferers. Finally, papers aimed at informing future research were marked for researchers.
User Journey. We constructed a simple taxonomy for steps, based on health care models [121]. Systems can be used for prevention, diagnosis and triage, treatment/intervention or self management and maintenance.
Outcome. Outcomes of systems can be measured through clinically validated tools, although the use of a clinically validated tool does not mean that a clinical trial procedure was followed. Outcomes could also be measured through proximal indexes (e.g. sleep) impacting mental health, or through adherence, which measures engagement with a particular system or following through a particular therapy.
Primary system function. As for characterizing the main function of the systems, some were aimed at automatic diagnosis, whereas others aimed at intervening in sufferers’ lives, either by supporting self-tracking, biofeedback training, implementing structured psychological interventions, e.g. computerized cognitive behavior therapy tools, encouraging mindfulness or promoting social support.
Ethical principles. We applied a coding scheme of four principles informed by healthcare ethics [18]: autonomy, capturing the respect for the decision making ability of autonomous persons through supporting information, its understanding and consent; non-maleficence or the explicit intention of not causing harm; beneficence focusing not only on preventing harm but also on providing benefits and on balancing benefits against risks and costs; and justice which captures fair distribution of benefits, risks and costs to all people irrespectively of social class, race, gender or other forms of discrimination.
Broadly characterizing the design space, we can say that the largest proportion of HCI work focuses on diagnosing affective disorders, followed by self-tracking technologies. It is also worth noting that there was some overlap across the different categories, as the initial coding was done to obtain the primary function performed by the system.
Across the different system functions, the most prevalent therapy cited was cognitive behavioral therapy (CBT) – 13 papers based their designs on CBT [12, 31, 41, 53, 89, 107, 130, 131, 144, 147, 157, 176, 182]. Other therapies cited include for example: Respiratory Biofeedback-assisted Therapy [43], Virtual Reality Exposure Therapy [29, 87], Interpersonal Social Rhythm Therapy [110, 117] and Emotion Regulation Therapy [72, 113] There is space for more engagement with existing treatment methods or emotion regulation theories. This said, not all design explorations need to be based on an existing therapy method. There is space for innovation and for treatments that might circumvent the issue through other means - engaging with caregivers, focusing on empathy or diagnosis. We now consider the different categories of systems to better describe the overall design space.
Diagnosis. Based on various forms of data, these systems focus on diagnosing an affective disorder. Five papers in this category report on design of new diagnostic systems [34, 69, 79, 125, 160]. Nine suggest novel approaches / frameworks / methods for more accurate diagnosis [9, 20, 21, 35, 37, 48, 64, 82, 84], while the majority (19) present the results of the studies performed in the existing systems, discover patterns and correlations in data [10, 26, 36, 45, 46, 47, 49, 51, 52, 74, 76, 104, 105, 122, 136, 141, 142, 150, 161].
Three papers look at measuring user behavior when using technology (computers, or performing tasks) and mapping it to stress and anxiety symptoms [82, 105, 150]. Another strand (14) tries to extract features from popular social media platforms (Facebook, Twitter, Reddit, Lifejournal) trying to correlate users’ content and online behaviour with affective disorder symptoms [9, 10, 35, 36, 45, 46, 47, 48, 49, 74, 79, 104, 122, 161].
Six papers introduced multimodal approaches, frameworks and systems combining visual, audio[20, 21, 52], as well verbal cues [64], biosensor data[84] and self-reporting [37]. Such a synergy is reported as more accurate for detecting sub-types of affective disorders, for example melancholia[20, 21].
There is also an emerging trend of studies correlating participants’ mobility patterns (based on GPS data from the smartphones) with affective health state[76, 136, 160] and using the obtained data for predicting changes in depressive mood[34] or social anxiety[26] .
Self-tracking. Self-tracking technologies are systems that make use of personal data collection about the sufferers. A majority (19) of papers present novel designs [11, 13, 17, 59, 60, 60, 61, 75, 77, 81, 88, 89, 110, 115, 120, 130, 157, 163, 179], whereas nine [8, 15, 56, 58, 90, 111, 117, 148, 159] are concerned with methodologies for developing self-tracking technologies attending to the specificities of affective disorders.
To our surprise, save some exceptions [69, 81, 89, 130, 157], most systems or system proposals (24 out of 29) in this category are aimed at multiple stakeholders. Self-reflection is often a stated aim of these systems, but in this domain self-tracking goes beyond the individual, and data may be shared with caregivers, family members or clinical staff.
Regarding reasons for collecting data and sharing with others, seven systems [13, 17, 60, 61, 110, 115, 163] are aimed at encouraging the patient to adhere to a particular course of treatment, a medication prescription or a set of healthy routines, such as exercising. These are often crucial components of any therapy for affective disorders. Some [59, 117, 120] developed machine learning and data analytics to trigger automatic reminders and generate advice based on self-tracked data.
There are also other aims for self-tracking. Some systems are aimed at supporting communication between clinical staff and patients [15] or family members[75], promoting empathy with mental health sufferers [179] or sharing data with others for increasing visibility of the condition of sufferers [88]. In these cases, self-tracking can have the goal of strengthening the social network of sufferers and create spaces for communicating that may complement face-to-face interactions.
Biofeedback. A biofeedback loop mirrors bodily signals that are hard to consciously experience (such as breathing patterns, heart rate variability or sweating patterns) back to the user. In the medical literature, biofeedback has been shown to have positive effects, as it provides individuals with access to their psychophysiological processes and can help regulate bodily responses to stress or other mental fluctuations [143].
All 12 of the articles presented systems that engaged end-users in biofeedback loops. Input included: breathing [63, 166, 169]; heart rate [91, 103, 153]; electroencephalography (EEG) [72]; bodily movements [119]; indoor-positioning [88]; electro-dermal activity (EDA) [110]; one of the biofeedback systems was more of a toolkit, allowing many different biosignals as input [113]. Apart from the diversity in biosignal input, there was also a richness in outputs that these biofeedback systems produced. Most common were forms of visual and/or auditory feedback [63, 72, 91, 110, 119, 169] and haptics [113, 147], but there was also a touchable animated crystal ball [153], underwater swimming in a VR-environment [166], a game changing its difficulty level [103], and an alarm preventing the sleepwalkers from entering dangerous areas [88].
All in all, we note that this is a viable and thriving research area with much potential. But it is also an area where we need to question the basic idea that biofeedback, without careful scaffolding for how to enact change, can result in healthy emotion regulation. Biofeedback should be accompanied with specific therapeutic guidance, and attention should be given to actuation as much as sensing.
Structured Psychological Interventions. The category describes technological projects and studies built upon different interventions. The majority of the publications (8 out 13) refer to CBT as the therapy of choice in their studies. Among those, several authors turn their attention to more specific CBT subsets — CBT for insomnia[182], computerized CBT[131] or common CBT components such as mood charting[107], as well as using context-aware CBT[12] and CBT with life coaches[144]. Other structured psychological interventions in the categories are Assessment and Commitment Therapy (ACT) [94] and Virtual Reality Exposure Therapy[29, 87], as well generic tools, not assigning them to particular therapies[162, 176].
Nine of the articles describe the design and evaluation process of novel technological solutions[12, 31, 41, 53, 94, 107, 144, 162, 176]. These include the design of a wearable device for public speaking anxiety [31], an intervention system with biosensing, self-reporting and decision support [176], a clinical study of a CBT-based therapeutic programme for depression [53] and a coach-supported CBT platform [144]. Two papers study contextual aspects of using computerized CBT [131, 182], and three discuss design options for future systems: a smartphone-based CBT system for unipolar and bipolar disorders [12], or a system for treatment of social phobias[29] and anxiety disorders[87]. We note that CBT is constantly being developed and specialized for different settings, making it an active and important research area. However, other well established and emerging therapies (ACT, DBT, Interpersonal therapy) have received relatively less attention from the community and could hold promise for future designs.
Mindfulness. Mindfulness can be defined [181] as acceptance of the moment through non-judgment. In this category, five papers are novel designs [14, 133, 146, 154, 168], two are studies of previously developed applications [137, 153] and two present studies for understanding how to better design mindfulness in general, outside of therapeutic settings [80, 181].
Research engaging more seriously with therapeutic settings proposes mindfulness as a complement to different therapies. Thieme et al. [153], ,154] talk about how to design for a clinical setting for mental health patients (women in a secure unit). The focus is on a method to innovate outside traditional healthcare practices, with a focus on complex mental health conditions. Here, mindfulness appears within a context of issues such as acceptance and distress tolerance, through teaching the person to accept the things they cannot change. Seo et al. [146] developed a system aimed at promoting calmness and a feeling of being at ease through touching and interacting with plants. It was tested in children with autism and older adults living in a senior home. Although mindfulness was not a very prominent category in the papers we reviewed, we were pleased to see innovative work integrating mindfulness into the specificities of therapeutic contexts.
Social Support. Work in this category is motivated by research showing that belonging to a community of mutual obligations has mental health benefits. Specifically, interaction with family and friends can significantly contribute to positive health outcomes for people with depression [118].
A majority of the papers (10) present results of studies rather than designing or developing a novel system. They present characteristics of social support in patient-caregiver dyads [4, 19, 156, 178] and how patients look for social support via social media [50, 71, 73, 132]. Some recommend using preventative diagnostics to direct the conversation between patients or moderate their content consumption . Four papers specifically investigate cultural aspects of mental health [30, 73, 100, 178].
Four present novel systems, of which three describe the design and development of assistive social robots [98] or conveying empathy towards the patient by either ”being in the same misery” [164] or emulating symptoms [97]. Wallbaum et al. [170] propose a tangible interface that allows for implicit communication between patients and their relatives to emulate social support.
Overall, we found that research on social support systems has revealed patterns in different cultural contexts while investigating social support with the help of, for example, participatory design methodology [19, 30, 98]. However, we would like to see more novel designs of systems that foster and support beneficial human interactions.
Other functions. In the ‘other’-category, comprising 30 papers, nine describe novel design ideas that do not fall into the other categories above [28, 29, 57, 93, 114, 167, 171, 173, 174] – the rest (21 papers) mainly focus on providing design requirements. Out of the novel design ideas, some stick out as particularly interesting. For example, there are two systems engaging with our somatic selves. The first one, the art piece the rest is construction, lets participants emulate the experience of living with anxiety, exploring one particular somatic symptom: impaired motor performance (shaking, trembling, tremors) through haptics and visualizations, to enhance empathy [174]. The second paper engaging with somatics presents three haptic systems: one emulates touch therapy, the second is a vest that hugs the wearer, distributing pressure in ways that have been shown to lessen panic attacks in autistic children, and the third generates controlled pain as a form of sensory grounding for persons with tendencies towards self-harm [167].
Three papers present systems for users with autism [28, 114, 171] – none of these are addressing affective disorder as such, but rather ways of coping. One was developed in a participatory design process, bringing out a down-to-earth solution to panic attacks through a mobile app with a “panic button”, allowing the user to get in contact with someone trusted to talk them through the situation at hand [114]. Living with affective disorders will, for many, not be a state to be cured, but rather a life-long adjustment, which is why the design space for coping-strategies is an interesting and underdeveloped area in HCI.
A striking finding is that from the total of 139 papers, two thirds (91 papers) do not mention any ethical concerns or values. Such limited emphasis is disconcerting given the heightened vulnerability of people with mental ill health. One explanation is that such vulnerability is considered accounted for through the institutional ethics approval processes, but this assumption may hide specific ethical concerns relating to affective disorders or the system at hand.
Within the remaining 48 papers addressing ethical issues, more than half (25 papers) referred to the topic of autonomy, one third (18 papers) touched upon the issue of non-maleficence, i.e., not causing harm to the others, and only a few papers mentioned the ethical value of beneficence, i.e., providing benefits which are balanced benefits against risks and cost (3 papers), and justice, i.e., fair distribution of benefits, risks and costs to all people (2 papers). With respect to the value of autonomy, 14 papers focus on supporting the decision making ability of autonomous persons, 6 papers touched upon the privacy of personal data, while 5 papers acknowledged that the opinions of multiple stakeholders have been considered in the design. We now discuss the key findings for each of these ethical principles.
Autonomy. Papers addressing the ethical principle of autonomy focused on two main areas: respect for the voice of people living with affective disorders, and for their data privacy.
The majority of the 48 papers touching on ethics focused on the voice of adults living with affective disorders (26 papers), and to a lesser extent on children (5 papers). Studies engaging adults included people living with serious mental illness, their caregivers and clinicians [163], people living with depression [26] or anxiety, and their clinicians [115, 168], or people with the experience of self-harm [22]. Fewer papers have engaged with the age [112], or gender aspects [154] of mental health, while the focus on children or teenagers included autism [58, 114], or cyberbullying [7]. Participatory design methods are the dominant approach, often involving several stakeholders. Novel approaches have started to emerge, such as methodologies for transforming hackathons into safe spaces for engaging with vulnerable users throughout the entire design process [22].
Most papers addressed the value of autonomy with respect to the privacy of sensitive data. Such data could be extracted from one's digital footprint, or captured explicitly through self tracking technologies.
Although public, user generated data shared on social media or online forums is highly personal and sensitive, addressing mental health concerns and being shared for the main purpose of accessing social support. As several papers indicated [35, 45, 49, 124], when such data is repurposed for the different goal of academic research, the issue of volunteering consent and protecting the anonymity of people who generated the data becomes important.
Good practices addressing these issues have started to emerge, with emphasis on protecting anonymity rather than on informed consent. Two noticeable exceptions include Pater et al. [124] who contacted and asked permission from the owners of public pictures to be analyzed, and Homan et al. [74] who also used informed consent combined with a respondent-driven sampling method to protect data owners’ privacy at the recruitment stage. Their study explored signs of depression in a social network designed to support LGBT youth at risk of self harming [74]. An important work is that of Manikonda and De Choudhury [104] who have developed guidelines to support the design of interventions for mental health on social media; guidelines which include seeking voluntary consent, and developing privacy and security protocols to protect people being studied throughout the entire research process, from data gathering and analysis to the development of interventions.
Good practices for protecting the anonymity of users of social media whose public content is analyzed include de-identification [35, 36, 104, 172] and paraphrasing [6] of personal data, or recreation of such data [6]. For example, De Choudhury et al. [46] applied de-identification of mothers’ public expression on Twitter posts used to predict postpartum changes, De Choudhury et al. [49] paraphrased Twitter posts in their exploration of gender and cultural differences in social disclosure, while Andalibi et al. [6] paraphrased quotes and recreated Instagram images in their analysis of instagramers’ experiences of depression. A similar approach was taken for the exploration of social media communities focused on mental health and suicide support such as Reddit [48], where public posts analyzed to identify individuals likely to engage in suicidal ideation, were previously de-identified and paraphrased.
Apart from data shared on social media, autonomy concerns have been also raised with regard to tracking technologies. The additional challenge of some vulnerable groups is that their autonomy could be claimed by their social support network, collectivized by healthcare services, or both [15]. For example, mobile apps for self-reporting symptoms of depression must be designed carefully due to the risks to the autonomy of individuals or groups of users such as pregnant women [15], while people living with depression raised privacy concerns regarding the uncertainty of who may be able to access their data [53]. In addition, privacy controls for tracking devices to support aging in place has been also critiqued [112] highlighting the risk of older adults’ data getting shared with their social support network without fully informed consent.
Privacy concerns regarding mobile-based interventions for mental health were also raised among user groups where the sharing of mobile phones is an accepted norm, i.e., teenagers [107], or where the ownership of such devices is limited due to socio-economic status [22].
Non-maleficence. The principle of non-maleficence was targeted through: participant recruitment, diagnostic claim, and providing feedback of negative states.
Firstly, participant recruitment focused on identifying people whose participation in a study may seriously impact their mental health. In their recruitment of people living with depression, two studies addressed this issue by employing a screening process. This process involved exclusion criteria such as suicidality or psychosis [53], and interview with a clinician who provided clearance for study participation [53, 73].
Secondly, many studies reporting analysis of personal data shared on social media have also raised concerns on the diagnostic claim: how and to whom is this being communicated. For example, in a study predicting postpartum depression from Facebook data, De Choudhury et al. [47] questioned the impact of these predictions on people's professional and social identities and the risk of perpetuating stigma and discrimination. Chancellor and colleagues described a classifier to identify pro-eating disorder posts on social media [36], also raising concerns of how automated diagnosing may negatively impact users. We additionally identified three emerging good practices addressing this concern. First is the avoidance of any diagnostic claim being made in De Choudhury et al. [49], as this would require medical expertise. Second is sensitively sharing the predictions of mental ill health privately with the user [45]. The third is to support users in making sense of these predictions, for example through sharing them with their support group [45] or medical staff [46].
Finally, the feedback provided by tracking technologies to people living with affective disorders is most likely to reflect negative emotional or behavioral patterns, which in turn can increase their mental ill-health symptoms [54, 117]. A similar argument is made regarding the impact of tracking technologies on older adults’ emotional and physical wellbeing whose feedback can threaten one's positive self-concept and damage wellbeing [112]. This concern is also relevant for younger people such as students using self tracking for stress management; with findings showing that participants with mental health issues reported increased negative experiences with the feedback of negative data [90]. Such experiences included guilt, disappointment, and embarrassment about their data. Authors proposed that special care must be paid to scaffold students during self-tracking for mental wellbeing. They also recommended mechanisms for framing negative data in a way that does not further demotivate students affected by ill-health. This paper offers a candid account of the potentially harmful implications of self-tracking technologies for vulnerable users suffering from mental ill-health, raising awareness of the need to account and limit them.
Two emerging research directions addressing these challenges are worth mentioning. The first has a focus on supporting reflection on tracked data. For example, Echo is an emotion tracking app supporting technology mediated reflection through which people could discover positive aspects in originally negative experiences [81]. The option to engage in reflection may not be taken on by all users, and those who chose to merely record fail to experience the benefits of reflection. Others argued for the value of increasing user's feeling of control by augmenting tracking technologies with support for reflection [16], or increasing user engagement with technology through DIY approaches to design [148]. Another emerging HCI area aims to acknowledge and proactively support the often invisible emotional labor [116, 175] of researchers engaged in health technologies. For example, Moncur [116] argued that working with sensitive groups such as people diagnosed with depression is emotionally taxing, increasing researchers’ vulnerability to wellbeing risks. To address this risk, the paper identifies supporting mechanisms such as reflection, counselling, peer support from friends, family as well as preparedness [116].
Beneficence. Beneficence is arguably relevant to all HCI work on mental health, as ultimately this is about doing good. However, such a broad framing renders this concept less useful and to break it down, we were inspired by the approach taken by Bates [16] who defined beneficence as “an affirmative ethic in the sense that it demands that actions must be taken when the opportunity arises to actively contribute to the health and welfare of clients” underpinned by “doing the right thing”, and “doing it well”.
We would argue that as HCI researchers we hold the ethical responsibility to recognize when the technologies we contribute to and research are appropriated in harmful ways. For example, within the broader discussion of online activities’ impact on wellbeing, Pater and Mynatt [123] defined digital self-harm as SNS-based communication, through user generated or consumed content, which fosters intentional self-harm. They call for researchers to be aware of how our tools support such negative behaviors to arise and spread, and to take ethical responsibility for addressing these issues. A similar call for action regards the negative affective and social implications of excessive smartphone use; with Faiola and Srinivas [55] arguing for researchers’ responsibility to confront this issue through visionary designs supporting social engagement and self-responsibility. Beneficence also means doing things well, and in this respect Topham et al. [157] argued for researchers’ ethical responsibility to ensure that their mental health technologies are grounded in solid and valid principles to maximize the benefits and limit harm. An important prerequisite for doing things well is the empathy of HCI researchers working on sensitive topics, and in particular when designing for people with affective disorders. Several studies highlighted the importance of empathy supported by design methods such as role-play [108], person-focused or experience-centered design [154], or phronesis as an ethical approach to design celebrating wisdom of taking actions for doing good in a situated context [15, 16, 151]
Justice. Justice is an ethical principle focusing on fair distribution of benefits, risks and costs. The few papers addressing justice argue for researchers’ responsibility to design for the wider possible user groups [131], including those underprivileged by the digital divide [22], by leveraging their strengths rather than weaknesses [96].
While HCI techniques and design research can help to explore user needs, clinical constraints, and the design space of affective health technologies, it is only by evaluating technologies with people with real mental health difficulties that many questions relating to design can be answered. Demonstrating feasibility and providing initial evidence of benefit also serves to show that ideas proposed are suitable for more formal summative clinical evaluation. Only 16 studies report clinical evaluations, i.e. with clients of a mental health service, or who meet the formal criteria for a specific mental health problem. As a proportion of the overall corpus, this number appears low. This may reflect the need to have completed a development process through several iterations (as described in [60, 107, 110]), resulting in a system which is both refined and stable enough to ethically be used in an evaluation, and the need for potentially extensive collaboration with mental health researchers, not just practitioners (see section 4). Nevertheless, these studies illustrate the richness of the data which can be gathered from clinical evaluation.
Patient group. A large majority of these studies (12 out of 16) explore solutions for adult sufferers, and 2 studies primarily address clinician tools. Two studies address children with affective health problems [41, 107]. While there are compelling reasons to address mental health problems in childhood and adolescence [41], and good arguments for the use of technology with children and adolescents [107], these lower numbers may reflect the additional practical and ethical difficulties of working with an especially vulnerable population on a topic of such sensitivity.
Form of solution. Two studies address analysis of corpora which might inform the development of tools for diagnosis or symptom tracking [21, 52], and one the gathering of a corpus of eye tracking data for attentional retraining [69]. Thus the majority (12 out of 16) of clinical evaluations were concerned with technologies to directly support people with affective health issues rather than analysis of data and clinician tools.
Many of the papers explored novel technologies, with 9 papers investigating the use of sensors and data analysis, and other papers exploring the use of games [41] and Virtual Reality [147]. Technology for the treatment of mental health difficulties is the subject of 8 out of 16 of the clinical evaluations, while 5 focus on tools for self-management, and 3 address diagnosis. It is encouraging that half of the studies are thus concerned with treatment interventions. Assessment and self management are also an important part of day-to-day mental health practice, and for bipolar disorder in particular, long term self-management is a central feature [11, 110]. Within the subset of papers reporting clinical evaluations, three studies [41, 53, 144] were based on CBT as a treatment model, which has been identified as a treatment approach which is both widely used and particularly amenable to support using technology [41]. One further paper [94] was based on Acceptance Commitment Therapy, which differs in several ways from traditional CBT, for example in promoting acceptance of negative thoughts rather than challenging them.
Evaluation methods. The studies aimed to capture a range of outcomes, including engagement, satisfaction [41], usability [115, 144], and perceived benefit [11], to formally measured psychological outcomes (10 studies). One study reported clinical measures only, and one paper survey results only. However, the majority reported a rich mixture of quantitative and qualitative data. Adherence (6 studies) and usage were the most frequently reported quantitative measures.
Four studies reported pre-post measures using validated measures. No studies used a randomized control trial design for effectiveness, although one study with stress as a proximal outcome randomised people to three different versions of a program [93]. Only one study measured maintenance of gains at follow-up [41]. The length of studies varied widely, ranging from lab studies, through to studies that lasted over 20 weeks. Of particular interest was the MONARCA system, as a second study on the system presented refinements over the original design [11, 60]. A sustained program of research and development is important for maximising the chances of bringing technology to clinical practice.
One study used clinical measures to give a breakdown of results (including adherence and overall usage time) by depression severity, facilitating discussion of the needs of different client groups [53]. Other studies looked at patterns of use, identifying issues such as re-engagement [107], and the relationship between coach calls and use of an online intervention [144].
Qualitative data was gathered by questionnaire and interview for 10 studies, and was often related to specific aspects of the technology. Issues discussed with respect to qualitative data included reasons for engagement (or non-engagement), comparisons to existing paper-based materials, comments on how the technology impacted on their treatment or illness, and requests for additional functionality. The qualitative data was also related to quantitative data, for example, in explaining the reasons behind different patterns of engagement.
Patient insights. Two papers presented detailed clinical case studies providing rich insight into the treatment of children with mental health difficulties, including severe difficulties [41, 107]. Case studies included the background of the client, the nature of their difficulties, and how the technology-supported treatment progressed, including clinical measures. Case studies have long been used in mental health, and may provide a means for conveying a richer sense of the people involved and the progress of treatment.
Several studies touch upon how their illness may impact on technology use, including comments from clients that they did not wish to use their phone [107] or read a lot of material [53] when depressed, and how the different phases of bipolar disorder would affect their tendency to log data [110]. Another feature of both MoodRhythm [110] and the MONARCA project [11] was a combination of participatory design and clinical use of the system, such that participants in the design process were also using the system to manage their illness over an extended period of time.
The studies contained several examples of appropriation and unexpected uses of technology, including therapists signing up face to face clients for an online intervention as an adjunct to treatment [53], and clients using mood diaries to broach embarrassing topics [107]. Several studies also incorporated an element of clinicians working out how best to use technology in their practice [41, 53, 60, 107].
Here we reflect on our analysis to chart future research directions, focusing on both the need to think about what technologies afford in terms of therapeutic potential, and the need to expand the range of therapeutic practices supported by systems.
We have seen a predominance of data-driven systems, that both produce and depend on digital mental health data streams for decision-support and self-monitoring goals. This is visible by the predominance of the categories diagnosis and self-tracking, a total of 43.8% of the papers. These types of systems have been proposed as having the potential to change how health care is to be provided, not only by providing immediate support to a sufferer, through improving adherence to a treatment or predicting episodes, as we have seen in the reviewed papers, but also by aggregating different health data streams across patients (big data) and helping see population-wide trends, providing the possibility of advancing theoretical frameworks for mental health and providing evidence for effectiveness of different therapies by making use of the multivariate nature of available data from different sufferers [24, 32, 158]. However, we would also like to point out interesting and under-represented research directions in the analyzed corpus.
There is a type of non-data driven work which focuses on actuation aimed at altering somatic perception of users through tangible interfaces. This work may involve touch therapy [146], promoting empathy [174] or simply artifacts for helping sufferers cope in the real world [154, 167]. In contrast with data-driven systems, this work aims at providing new types of technology-assisted coping mechanisms outside of virtual or online environments. We have seen this work appear in some of the mindfulness and biofeedback research and some of the work we marked as other. We would like to see more of this type of work, integrated into therapeutic settings.
Supporting new therapeutic methods. Within the mental health discipline, there are huge variety of therapies associated with a number of major schools or theoretical models. While it makes sense for HCI research to start with those such as CBT which are widely used and which have a good evidence base, there is scope for HCI research to explore means of computerized support for a much wider range of therapies [40]. For example, compassion-focused therapy shows potential for sufferers who have failed to reap benefits from CBT [92]. Some opportunities for support may be straightforward, whereas others, particularly those with a more embodied element, will require more creative ideation and exploration of the design space [44, 165]. As well as existing but unexplored interventions, such as those based on memory [129], there are concepts like the extended model of emotion regulation [67, 68] that could spur new approaches. Only with greater cooperation between psychology, HCI researchers and patients can we realize the potential of technologies within affective health.
Supporting anonymity, consent, and privacy. Most HCI research accounting for the voice of adults living with mental ill-health included multiple stakeholders through participatory design methods. Our findings suggest extending the exploration of mental health technologies more fully along the lifespan. In this way, the increased vulnerability due to young or old age could be better addressed by adopting specific alternative design methods developed for these user groups. We also want to bring to attention the emerging good practices for protecting anonymity of social media users and seeking their voluntary consent. For example, we can think of more engaging and easier to understand multimodal consent forms and social media's terms of services. We also draw attention to the compelling concerns of data and device privacy posed by self-tracking technologies for mental health as a call to action for the HCI community, which we could start to explore by integrating tracking technologies with edge computing [62].
Supporting understanding, reflection, and positive change. Many papers analyzing user generated data on social media pointed to the challenge of proposing a diagnosis. Good practices in this respect are the avoidance of such claims, and the sensitive communication of mental ill-health prediction with additional support for sense-making, for which we may draw inspiration from medical humanities research [42]. With regard to the potential for harm of self-tracking for mental health, we argue for an ethical responsibility for researchers to design more innovative mental health technologies that leverage less the tracked data and more its understanding [138], reflection [139], and actionability [140] for positive behavior change [46, 47, 117]. We also draw attention to the importance of providing better peer and institutional support so that researchers can continue to work in this challenging and important area, without burning out. For example, we may rethink how such support can be explicitly factored into institutional ethics or research funds.
Supporting empathy, inclusiveness, and avoiding harmful appropriation. Most HCI research on technologies for mental health has engaged with people with mental ill-health. Researchers conducting clinical evaluations need to recognize the increased vulnerability of some participants, and carefully consider how to apply strategies such as screening in the recruitment process to reduce the risk of harm, while also preserving the validity of the work to real patients, and supporting the autonomy of potential participants.
While only a handful of papers touched upon the ethical principles of beneficence and justice, we agree with powerful arguments put forward regarding researchers’ responsibility to anticipate and recognize when technologies are appropriated in harmful ways and to engage in addressing these issues [55, 123]. We would also argue for the need for novel design methods and approaches [15, 108, 154], to strengthen researchers’ empathy for the wider vulnerable groups on both sides of the digital divide [22, 46, 131] .
Working with clinical researchers. Mental health is a unique and sensitive research setting, and the low number of clinical evaluations likely reflects a range of difficulties. Conducting clinical evaluations of affective health technologies will require sustained collaboration with mental health researchers in most cases. This is perhaps one of the main reasons behind the low number of clinical studies in the corpus. While interdisciplinary collaboration is not uncommon within HCI, Blandford et al. [23] discuss a number of important ways in which health and HCI research differs, ranging from research methods (literature review, development and evaluation practices all differ substantially) and ethical perspectives (preventing harm vs. individual rights), through to publication practices (long, exploratory papers in HCI vs. short but varied papers in health research).
Working with clinical interventions. As clinical research comes to consider how technology might be used in a wider range of therapies, there is an opportunity for HCI research to contribute in terms of understanding requirements and context, how people interact with technology, methods for ideation of novel solutions, and exploring the strengths and challenges associated with multiple different possible designs.
Mobile health technologies, the proliferation of sensors, and the emergence of machine learning techniques have opened up many possibilities for the use of technology in affective health. These technologies are powerful but also potentially problematic, and so there is a need for exploration of all aspects of these technologies with real people before progressing to trials whose main purpose is establishing effectiveness.
Working with clinical trials. The development of complex healthcare interventions has been described as a process with multiple stages of progression from modeling and exploratory trials through to randomised controlled trials and implementation studies [33]. While this is clearly applicable to the development of technology based affective health interventions, the long time frame of this validation process is difficult to reconcile with the demands of rapidly and continually changing changing technology. Due to ethical issues around novel technologies, and access barriers to clinical settings [108], we are not arguing that every novel design should be tested in clinical trials. But we contend that research in this area should be more informed by the needs of real patients and have, as an ultimate goal, evidence-based design development - whether the resulting artifact is aimed to be used in or outside of clinical settings.
In a world where technology development can never really stop, we should consider how we might incorporate HCI elements into clinical evaluation protocols as well as during implementation. The advantage of doing so from a HCI perspective is the prospect of receiving detailed clinical outcome data and demographics, together with more familiar forms of data such as engagement and reports of user experience. Conversely, as a discipline we need to be able to clearly articulate and demonstrate what HCI has to offer, for example in terms of helping to achieve better engagement and thus enabling better science [53, 93], or showing how HCI research can contribute to refining designs to improve effectiveness [60], as development of new interventions and accumulating an evidence base progresses. Ultimately, research in this area should leverage mutually beneficial collaborations between HCI, clinical psychology and other relevant disciplines.
A systematic analysis of the fast growing body of literature on depression, anxiety and bipolar health issues from 10 years of SIGCHI conference proceedings revealed that most innovation happens in automated diagnosis and self-tracking. Moreover, we see a need to promote ethical practices for involvement of people living with affective disorders. Finally, although relatively few studies evaluate technologies in a clinical context, the varied forms of support and intervention they investigate illustrate how rich insights are gained from evaluations with real patients. Our findings highlight potential for growth in the design space of affective health technologies.
This work has been supported by AffecTech: Personal Technologies for Affective Health, Innovative Training Network funded by the H2020 People Programme under Marie Sklodowska-Curie grant agreement No 722022, the Swedish Foundation for Strategic Research project RIT15-0046, and Science Foundation Ireland Grant 12/CE/I2267 to the Adapt Centre.
1AffecTech - Personal Technologies for Affective Health http://www.affectech.org/
This work is licensed under a Creative Commons Attribution International 4.0 License .
CHI '19, May 04–09, 2019, Glasgow, Scotland UK
© 2019 Copyright held by the owner/author(s).
ACM ISBN 978-1-4503-5970-2/19/05.
DOI: https://doi.org/10.1145/3290605.3300475