1 Introduction
With the aging of the population, the prevalence of dementia is rapidly increasing, affecting nearly 55 million people worldwide and expected to reach 70 million by 2030 [
74].
People with Dementia (PwD) face a decline in cognitive ability that impairs their daily activities, impacting their quality of life and often requiring assistance from caregivers. This burden often falls on family members, who may experience significant stress and economic hardship [
11].
PwD often exhibit
Behavioral and Psychological Symptoms of Dementia (BPSD) that affect their relationships with caregivers and close family members, leading to social disengagement and isolation [
42]. These manifestations, which include apathy, agitation, aggression, mood disorders, psychosis, and sleep disturbances, affect up to 90% of PwD over the course of their illness. The etiology of BPSD is multifactorial, involving complex interactions between neurobiological, psychological, and social factors. Social and environmental elements, including caregiver interactions and physical surroundings, also play crucial roles in the manifestation and severity of BPSD. The assessment and management of BPSD require a multidisciplinary approach, integrating standardized assessment tools like the
Neuropsychiatric Inventory (NPI). Additionally, these challenging behaviors affect caregivers differently, and strategies for dealing with BPSD need to address the needs of both PwD and their caregivers [
12]. While there is currently no known cure for dementia, treatments mainly address the symptoms rather than the causes of the disease. These treatments could involve pharmacological and/or non-pharmacological interventions [
13].
There is evidence supporting the efficacy of a variety of non-pharmacological interventions for dementia in addressing BPSDs, including music therapy, physical exercises, and cognitive rehabilitation [
48]. Besides, most of these practices have no harmful effects and require moderate investment [
59]. Non-pharmacological interventions for BPSDs are not usually directed at one specific symptom. Most of them address multiple symptoms simultaneously or have broader effects on well-being. Some interventions may have primary targets but often yield secondary benefits across other symptoms, as they often co-occur and address underlying factors like unmet needs or environmental stressors.
Non-pharmacological interventions for dementia are increasingly incorporating the use of assistive technologies. These include the use of virtual and augmented reality; tablet-based cognitive training programs and ambient assisted living technologies, that adapt the environment to respond to the user needs. Among these technologies,
Socially Assistive Robots (SARs) have shown considerable potential in aiding individuals with dementia who experience challenging behaviors [
10]. SARs have been designed to function as companions [
61], exercise coaches [
22], and daily living assistants [
46].
Developers of SARs for dementia care face a myriad of tensions and tradeoffs for
Human–Robot Interaction (HRI) design. PwD exhibit visual, cognitive, communication, or hearing deficits and other diverse abilities in communication that tend to deteriorate over time [
24]. These challenges influence how the user engages with a social robot, the central focus of interaction design in HRI [
56].
The field of
Human-Centered Artificial Intelligence (HCAI) has emerged from the growing recognition that the design of intelligent systems must prioritize human needs and values [
4]. This approach is particularly crucial when developing AI solutions for vulnerable populations, such as PwD and their informal caregivers. HCAI integrates multiple perspectives: First a user-centric perspective ensuring that AI systems are tailored to the needs and context of end users [
9]. Second, HCAI emphasizes addressing ethical concerns like privacy, fairness, and inclusivity. Finally, HCAI encourages synergistic collaboration between humans and AI agents, while ensuring human oversight and control. In applications involving social robots in domains like dementia care, this symbiotic partnership takes center stage with an emphasis on facilitating natural interaction between the person with dementia and the robot and empowering caregivers by augmenting their capabilities rather than replacing them.
Several methods and techniques using notions from human-centered design have been used to involve PwD at different stages of the design and evaluation of SARs technologies, and there is general agreement on the challenges that this involves. For example, several PwD are non-verbal so it is very hard to co-design and elicit their needs to establish requirements; there is a need for research that describe how such user-centered design methods could be selected and applied in this context [
64]. Thus, there is an untapped potential to understand what techniques from a typical user-centered design process work for this population and in this context, and how to combine or adapt current user-centered design techniques with those related to health, like person-centered treatments [
64] by placing special focus in designing for appropriate interaction and engagement. Currently, there is little support for researchers to facilitate the process of characterizing and designing SARs behaviors as appropriate exchanges and interactions with PwD. Without a proper design of these SARs behaviors and interactions, robots may fail to support their interests and needs.
In this article, we present a framework for SARs designed specifically to assist individuals with dementia, taking into account their unique experiences, needs, and challenges. The proposed framework aims to help create better robots that offer companionship and stimulation to PwD, alleviate caregiver burden, and enhance the quality of life for both PwD and their caregivers.
Leveraging years of experience in designing HRI for PwD, we propose a human-centered framework composed by three cycles, illustrated with the robotic platform called EVA (
Section 3). In the first cycle, called
Engagement, we emphasize the importance of designing alternatives of prototypes of SARs and their behaviors until one prototype reaches a level of maturity and interaction that is found engaging. In the second cycle,
Automation, we seek to translate those engaging behaviors found in the first cycle into it’s autonomous version that can work without human intervention. In the third cycle,
Efficacy, we leverage long-term engagement to test for efficacy in terms of measurable health, behavioral, or quality of life outcomes. We illustrate the use of the framework with a case study, that involved a 4-year development process of an SAR used to guide a
Cognitive Stimulation Therapy (CST) with PwD. Health, behavioral, and quality of life outcomes resulting from the use of this proposed SAR showed a reduction in problematic behaviors and an improvement in quality of life (
Section 4). Most of the lessons learned from this case study derived in the proposal of the framework and the EVA robotic platform. We close highlighting the significance of our contribution and discussing directions for future work (
Section 6).
2 Related Work in HRI for Dementia Care
HRI research in the domain of dementia care often concentrates on one of two areas: examining human factors to assess the adoption of social robots or devising solutions to address technical hurdles, such as improving the interplay between sensors, actuators, and software [
67].
2.1 Prototyping Platforms for HRI
Several platforms have been proposed to support the development of social robots. For example, the Bonsai modeling framework, proposed in [
38], interlinks usability research with system development. The framework incorporates an architecture of reusable components that provide abstractions to sensors and actuators. The Bonsai framework has been implemented in the BIRON robotic platform.
A common solution to deal with the complexity of designing and evaluating HRI is the use of
Visual Programming Languages (VPLs) associated to robotics platforms. These VPLs allow for the rapid prototyping and modification of interactions. Some VPLs also facilitate conducting interaction sessions using a
Wizard-of-Oz (WoZ) approach [
39,
63].
Implementing robot behaviors that adhere to human social norms is challenging. The RoVer authoring environment uses formal-verification techniques to ensure that designed interactions satisfy social norms [
55]. The environment includes a VPL to facilitate the design of the interactions.
Several social robotic platforms have become available to help HRI researchers design, customize, and conduct interventions. Furhat Robotics
1 is a commercial platform that offers a tabletop social robot that includes microphones, a camera, and motors to mimic human head movements and a projector to display a personalized face [
1]. Several research teams have used the Furhat robot to conduct HRI research with different populations, such as older adults with depression [
66] and children [
53]. QTrobot
2 is a commercial robotic platform that includes services to operate and program a humanoid social robot, including speech and image recognition, body tracking, and body gesture. QTRobot has been used in diverse contexts such as regular and special needs education, and health with children with autism [
15].
Overall, this body of work has moved one step further, in using the lessons learned when developing specific prototypes to propose technical platforms aimed at facilitating the development of robots. Yet, effective solutions need to consider both, the technical and the human-factors perspectives, which intersect on interaction design and that goes beyond usability. However, little has been said as to how design challenges and tensions could be resolved with such tools opening opportunities to further understand the design space of HRI for dementia.
2.2 Design Frameworks for Social Robots
A design framework offers a structured approach or methodology that provides a set of guidelines, principles, and best practices for designing a particular product, system, or process.
Acknowledging the multidisciplinary nature and complexity of designing social robots, multiple efforts have been made to provide comprehensive design frameworks to assist in this task. We exclude from this discussion software frameworks aimed at facilitating the development of applications for social robots such as in [
27,
60,
70].
Using a participatory design approach, Axelsson et al. [
6] propose the use of canvases as a visual tool to support collaboration between different domain experts involved in the design process. The framework emphasizes a user-centered approach with a focus on interaction design, incorporates input from different stakeholders and it involves all phases of the process from conception to testing. It focuses on conceptual design with no direct mapping to implementation platforms or tools.
Participatory design approaches have been used in the design of SARs for older adults, actively involving therapists and older adults [
36,
71]. There has also been participatory design work with caregivers of PwD [
28].
Another recent initiative with a focus on user-centered design is presented in [
35]. It focuses on human–robot communication extending the 5Ws model of mass communication by defining aspects such as who is the source, what is the message, what channel is used, and to whom it is directed. For instance, the channel could be verbal communication or body language.
The integration of human-centered and universal design methodologies led to the development of a human-oriented framework for HRI presented in [
40]. The framework facilitates the identification of user needs, its mapping to design goals and to the development of a functioning prototype of service robots. The approach uses mixed methods and covers stages from requirement formulation to evaluation. The framework focuses on technical design aspects and makes use of universal design principles so that the product addresses the needs of as many uses as possible.
All these design frameworks incorporate user-centered and iterative design strategies that help identify user requirements and map them to design parameters. They can be generally applied to several application domains. They are oriented toward social robots, but not specifically SARs and thus the assessment of the efficacy of a robot-guided intervention is not considered.
2.3 SARs for Dementia Care
The context of dementia care imposes design challenges, contradictions, and tensions that call for the need of design insights specific for the development of robots in this context. However, little has been said as to how such design insights can be incorporated in the HCAI design process of SARs for dementia care. Moreover, as several interventions have focused on using commercially available robots, there is an untapped potential to understand the design space of SARs placing an emphasis on understanding what behaviors a robot must incorporate, and what techniques from successful therapeutic interventions could affect the design of behaviors and interactions of SARs for dementia care and derived in measuring for efficacy with promising healthcare outcomes.
Research in SARs for dementia has often focused of acting as companions. PARO is perhaps the most well-known SAR for dementia care. It emulates a baby seal. Its therapeutic role has been extensively investigated as in [
37] showing a positive effect on PwD on affection and communication. It has also been shown to reduce psychological symptoms of dementia such anxiety and apathy [
47].
In the realm of assisting in activities of daily living, the Robot Ed was designed to guide PwD when preparing tea and proving to be feasible and usable [
7].
Other initiatives have focused on understanding how techniques from successful therapeutic interventions could be used to inform the design of SARs behaviors and their interactions for therapy delivery. For example, Mohan and Kuchenbecker [
44] developed a tool to facilitate the design of exercise and therapy interactions with a humanoid robot to be used in WoZ experiments. Similarly, the PARO robot has been successfully used in a multi-sensory behavioral therapy with PwD [
75]. In particular, personalized social interaction is another characteristic that has been shown to positively impact PwD–robot interaction and can contribute to creating a strong bond between the user and the robot [
22].
In general, research in SARs has been more focused in understanding the technical aspects of developing SARs or to evaluate usability aspects or adoption of commercially available robots in a particular context. Yet, no specific tools or methodologies from user-centered design leveraging person-centered health strategies have been proposed to encompass other aspects that go beyond feasibility and usability. We argue than only after PwD truly engage in the use of SARs, researchers will able to understand the effect in terms of efficacy of health outcomes derived from an intervention delivered and guided by an SAR. In light of this body of work, this article makes the following contributions:
—
A framework specifying the user-centered and person-centered techniques researchers can use and combine when designing, developing, and evaluating SARs in the context of dementia dare. To our knowledge, this is the first and only framework providing a comprehensive support throughout the complete end-to-end development process of SARs with PwD. It shows what methods are useful for eliciting initial conceptual prototypes, those for designing autonomous behavior and those relevant for the assessment of health outcomes resulting from the use of such prototypes.
—
Empirical evidence via a case study demonstrating how the framework could be used in practice to design and evaluate SARs to support dementia care.
—
A set of design recommendations to inform the HCAI design and development of SARs to support dementia care placing especial emphasis in personalization, tolerance, and proactive behavior.
3 A Framework to Design Person-Centered Interactions and Behaviors of SARs to Support Dementia Care
SARs can be designed to support PwD and/or their caregivers (both formal and informal). Most PwD experience symptoms that are strongly influenced by background and environmental aspects that also change over time, with the trajectory of the disease, progressing very differently, both between and within individuals [
41]. Thus, the importance of incorporating the perspectives, interests, experiences, and context of PwD and their caregivers in the design, development, and evaluation of SARs, and for the technology to be sufficiently flexible to be tailored to each individual and its context.
With this aim we have developed a framework that has evolved with our research in HRI for dementia. The framework consists of an iterative, human-centered methodology for the design and evaluation of a social robot to support dementia care and has been applied in the development of HRI solutions on an open and extensible robotic platform called Embodied Voice Agent (EVA).
The methodology and the robotic EVA platform are aimed at being flexible enough to be applied to support PwD and their caregivers in a variety of tasks and circumstances, each of which might demand additional development and evaluation.
3.1 The Three-Cycle Methodology for the Development of SARs for Dementia
The methodology proposed is grounded on an iterative human-centered design and evaluation approach, anchored in design insights specific to dementia care.
Figure 1 illustrates the approach proposed to design and evaluate SARs for dementia care. Each cycle of the framework includes participatory design and evaluation activities with different stakeholders. With each iteration, the technology achieves increasing levels of maturity, its evaluation becomes more demanding, and the evidence of its effectiveness more conclusive.
3.1.1 Engagement Cycle.
The initial cycle focuses on technology acceptance by way of engagement. At this stage, an initial prototype is developed and evaluated with stakeholder participation. Its aim is to gather initial evidence that the SAR can be useful and will be accepted by its target users. The engagement with the external stimulus, in our case SARs, results in a change in affect that influences the effectiveness of the intervention [
14]. Engaging PwD in activities with appropriate external stimulus has shown to produce beneficial effects [
50]. Therefore, this cycle aims to promote PwDs’ engagement with the SARs.
The process initiates by gathering information about the social and cultural context of the potential users that can influence the acceptance and adoption of the technology [
19]. We then use this knowledge in the design of the interaction paradigm and features of the SAR. We need to understand the specific challenges, needs, experiences, and desires of PwDs and their caregivers that we aim to address, to propose an initial set of opportunities for the use of an SAR and its services.
Several methods can be used for this purpose, but they should be aimed at understanding the domain of application and the main affordances required from the SAR, as well as potential risks associated with its use by PwD and caregivers. Qualitative data collection methods, such as interviews, shadowing, and direct observation, can be used to identify opportunities for the use of SARs as well as the strategies used by caregivers to establish effective communication and interaction with PwD.
The characteristics of the target population should also be defined. It is important to consider that PwD experience dementia differently [
12] and that the disease progresses over decades at different speeds and non-linearly [
33]. Common HCI constructs, such as scenarios and personas, are useful during this interaction co-design phase.
The use of focus groups is recommended during co-design. Since caregivers and PwD may have low technology literacy, we propose the use of creativity prompters—artifacts that show the potential and capabilities of the technology to facilitate participants to explore and envision the design space [
54], to encourage the participation of PwD and caregivers.
In the implementation phase, an early SAR prototype is developed or customized. A tangible prototype helps to get a better understanding of SAR–PwD interactions and improving the stakeholders involvement. This helps to leverage PwD lived technology experiences, which they can obtain through interaction with a physical robot [
51]. Thus, designers and HRI researchers can iterate over this design based on real interactions and feedback from PwD and caregivers. The robot’s autonomy is not the aim of this phase; thus, the use of rapid prototyping and WoZ approaches is recommended.
Finally, to assess user acceptance we propose a focus on engagement as a measure of success [
14]. Five dimensions proposed by Cohen-Mansfield et al. [
14] may be used to measure the PwD’s engagement: (1) rate of refusal of the SAR; (2) duration of time that PwD was occupied or involved with the SAR; (3) level of attention to the SAR; (4) attitude toward the SAR, and (5) actions toward the SAR such as holding it, talking to it. If the PwD does not engage with the external stimulus (SAR), we cannot expect the robot to have the desired behavior or impact on the health outcomes that will be evaluated in the next cycles [
14].
Intent to adopt can be assessed with an early prototype using a WoZ approach or videos depicting the interactions, two techniques frequently used in HRI studies [
57,
73]. These are useful iterative design tools of robot behaviors [
20,
39]. The evaluation can be conducted directly with PwD and caregivers. Benchmarks have been proposed for evaluating SAR systems taking into consideration technical, social, and assistive perspectives [
23].
Table 1 indicates in which cycle of the methodology each of these benchmarks are addresses, either during the design and/or evaluation phases. The assessment at the end of cycle 1 focuses on evaluating the technical issues of safety and scalability, and the social issue related to the understanding of the domain.
3.1.2 Automation Cycle.
The second cycle focuses on developing and evaluating the SAR at the desired level of autonomy that allows it to perform adequately in the application domain. The focus at this stage is on interaction design, which relates to the benchmarks of autonomy and imitation for SAR development (see
Table 1).
Design for autonomy should consider the role of the caregiver. Will the caregiver participate directly in the interaction and/or configure the robot? Mechanisms need to be defined for identifying user actions and states that will trigger responses from the SAR. With regard to imitation, which refers to robot’s capability of displaying a consistent and credible personality to be considered as an appropriate interlocutor [
31], affordances should be designed in the robot to allow the user to assess the capabilities of the robot.
In its initial stage, findings and data from the engagement cycle are used to guide the process of automating the social robot to replicate successful interactions achieved using the WoZ approach. Thus, data from real interactions are analyzed to discover patterns, preferences, and other additional insights to automate the interaction’s content and robot’s behavior.
Several strategies can be used to design autonomous HRIs. On the one hand, robot utterances should be clear and short. Wording could be personalized to promote engagement. For instance some users might respond well to praise while others might find it patronizing. Defining a limited interaction script can facilitate constraining possible answers. Also, robot utterances could be classified as those requiring understanding the reply to advance the flow of the conversation, expecting a reply, but not requiring to continue the conversation and those when a response is not expected. Finally, strategies should be used to deal with communication breakdowns, which for dementia could include rephrasing the utterance or even changing the conversation topic.
The implementation of the autonomous interactions designed in the previous phase can be supported by AI-supported services such as Text-To-Speech, Speech-To-Text (STT), and Natural Language Understanding (NLU). Also, sensors in the robot or the environment can be used to detect events that trigger robot behaviors.
The assessment of autonomy can be conducted with an increasingly functional and Autonomous Robot (AR), including the capacity of the robot to appropriately recognize the situation and act accordingly. Breakdowns in communication between PwD and the robot, or accuracy in the recognition of user behaviors by the robot are two of the metrics that can be used to assess the prototype at this stage. Special care should be given to unintended consequences that could arise from the SAR acting autonomously, such as producing confusion or agitation. In any case, the use of the autonomous SAR for dementia should be continuously monitored. Strategies should be considered to minimize the effects of potential recognition errors.
3.1.3 Efficacy Cycle.
The third cycle focuses on assessing the SAR as an assistive technology and its efficacy on PwD and their caregivers. While most social robotics technologies don’t need to reach this phase of evaluation, all SAR system should, as they are designed to act and support people in a healthcare context.
The initial phase of this cycle focuses on the design of the study. The outcomes that are usually measured in dementia interventions can be categorized as focusing on cognitive function, functional performance, and quality of life/behavior [
29]. Since there is currently no known cure for dementia, cognitive outcomes are usually used only for initial assessment. Furthermore, metrics based on functional ability and quality of life focus on what individuals feel or are able to do, so they are more relevant to patients and caregivers [
29]. Thus, based on the purpose of the SAR, appropriate instruments should be selected to measure improvements on functional performance, addressing BPSD, and/or improving the quality of life, of both, PwD and caregivers. The focus in this cycle changes from measuring the effectiveness of PwD–SAR interactions to assessing the impact of the intervention on the care of PwD and their quality of life.
After the study design, an initial phase of data gathering with participants will be conducted to personalize the behaviors of the robot to each individual and establish a baseline for the desirable outcomes.
Next, the intervention is conducted with the SAR. Evidence-based non-pharmacological interventions should be used to highlight the role of the SAR and compare it with similar studies. These interventions include Reminiscence Therapy, Doll Therapy, Reality Orientation and CST [
8].
Finally, the results from the intervention are analyzed to assess the efficacy of the intervention conducted by the SAR, make recommendations for improvement and propose additional studies. With respect to the benchmarks for evaluating SAR interactions, this cycle focuses on the impact the technology will have on the care of its users and the quality of life of PwD and caregivers (
Table 1).
3.2 The EVA Robotic Platform
EVA
3 is an open source social robotics platform aimed at supporting research in HRI. It was originally developed to support our research on the use of conversational robots for dementia care [
16]. EVA provides most of the resources needed to build a fully functional social robot, at an affordable cost and to design and create interactions for specific contexts and populations. EVA’s repository provides all the elements to make your own social robot, such as 3D models, schematics, software, and guidelines to assemble it using open-hardware solutions such as the Raspberry Pi and Arduino. The basic version of the robot EVA includes a voice interface (microphone array and speaker) and a 5-inch touchscreen to manage the robot’s basic features. Furthermore, EVA integrates a ring of LEDs in its chest to display light animations, such as to display emotions and indicate that the robot is listening or talking. The screen in the head is used to display facial expressions, denoting states such as attention and joy. In addition to the above elements, the intermediate version (see
Figure 2(b)) includes two servo-motors to provide 2 df to the robot’s head and a depth camera (Intel RealSense) for tasks involving computer vision. The complete (mobile) version includes a mobile platform based on TurtleBot to explore more complex body gestures and mobility features.
Front-end and back-end modules compose the platform’s software architecture. Front-end includes (1) the operator module—operating the robot’s behavior enabling its features and abilities in real-time; (2) the configuration module—configuring the interaction and robot features such as selecting pre-programmed routines, language, editing behaviors; (3) the programming module—create and edit interactions using VPL, and (4) the facial expressions renderer that display the appropriate face animation. The back-end manages the logic and hardware components of the robot. It includes modules to create a local server, manage the robot’s behavior, execute interactions, and store and load pre-programmed routines. The front-end technology stack includes AngularJS, Unity, and Bootstrap. The back-end employs NodeJS, MongoDB, and Python scripts.
EVA has features that include basic natural language processing, non-verbal synthesis of emotions, and speech. The robot can operate in two modes, autonomous and operated. In autonomous mode, EVA processes user utterances to produce verbal responses, actions (such as playing music or expressing emotions), or a combination of both. On the other hand, operated mode employs a remote web application to control the robot’s behavior, activate pre-defined skills and can support interactions using WoZ.
In the next section, we describe how the proposed framework, using the methodology and the EVA platform, was used in the design and evaluation of an SAR to guide a CST with older adults with dementia. The framework, however, could be applied to other robotic platforms with similar affordances.
4 An SAR as a Facilitator of a CST for PwD
We used the framework to design, develop, and assess an SAR to guide a CST for PwD. A CST consists of group sessions led by a human facilitator aimed at actively mentally stimulate PwD through cognitive activities and reminiscence, multi-sensory stimulation, and group social contact [
72]. The CST is an evidence-based non-pharmacological intervention program that has been successfully used to improve or stabilize cognitive function, the performance of daily activities, behavior, mood, and quality of life [
49]. The effects from a CST program may support dealing with BPSD [
62].
We aimed to design an SAR as a therapy facilitator in order to guide therapeutic sessions autonomously. However, in the beginning, it was not clear which factors should be considered to promote a successful interaction between PwD and an SAR. Thus, we followed and refined the framework described in
Section 3 to design robot features to achieve a successful interaction with PwD.
We next describe the three cycles for designing and assessing an SAR to guide cognitive stimulation sessions with PwD.
4.1 Engagement Cycle
We followed the first cycle of the framework to gather and analyze data aimed at informing the design of the SAR and evaluating its acceptance. Furthermore, during this cycle, we designed and developed the first version of the platform EVA.
4.1.1 Initial Understanding of the Domain.
We conducted two initial studies to determine the social and affective needs of PwD who live in a nursing home to envision what features, strategies, and activities should be conducted by a social robot.
First, we conducted a contextual study using semi-structured interviews with four formal caregivers, one therapist, one neuropsychologist, and one geriatrician. This qualitative study aimed to determine the effective strategies to interact, communicate, frequent dementia-related behaviors symptoms and how to deal with them. The interviewees’ experience of the day-by-day interaction with PwD helped us better understand the challenges of dementia care and how to deal with them.
Next, we conducted another qualitative study based on passive observation in the nursing home facilities. During one day, research team members observed the activities and the interaction between residents and caregivers. Using this immersive technique we observed how the caregivers used personalized strategies to deal with problematic behaviors, such as depression, anxiety, wandering, apathy, and aggression.
We next describe the main findings from these studies.
—
Verbal Communication. The primary and most effective interaction between caregivers and PwD is verbal communication over body language or another type of communication.
—
Personalized Social Interaction. The most common strategy to deal with problematic behaviors is social interaction. Caregivers know the favorite topic of conversations and the personality of each resident. Thus, they use this knowledge to enact social interactions to calm, distract, and relax them.
—
Music. Our analysis showed that music is an essential factor as an ice-breaker, and it can create an initial bond with the PwD and help the acceptance of the new caregiver in their environment.
4.1.2 Interaction Co-Design.
Based on the findings from the first phase, we created scenarios where the SAR can support dementia care issues. Thus, the second phase aimed to get feedback from a new group of caregivers to evaluate if these scenarios were realistic and feasible or how to improve them. We conducted a focus group co-design workshop with nine formal caregivers who cared for different PwD at another nursing home.
As proposed in the methodology, we used creativity prompters to encourage the participation of the caregivers. The first prompter was an initial prototype of a social robot. The second creativity prompter was a set of scenarios based on the results of the study conducted in the previous phase. Both creativity prompters promoted the reflection and discussion to get feedback from the caregivers in order to create, improve, or discard scenarios.
During this co-design process, the research team and caregivers exchanged and shared their expertise.
(1)
Technology Introduction. The caregivers interacted with a social robot created using an early version of the EVA platform. The participants used and explored the features and capabilities of the social robot (e.g., ask date and time, small talk, tell jokes, complete sayings, and ask for music tracks) during an open interaction with it. Participants responded to a couple of open questions: (1) What do you think about EVA? (2) Do you think that EVA can enact an interaction with a PwD?
(2)
Scenario Assessment. We created four scenarios based on information gathered in the first stage: (1) personalized conversations for anxiety; (2) music therapy to distract and relax; (3) conversation to deal with depression; (4) use spatial and temporal orientation and storyteller capabilities for wandering. As part of the creativity triggers, we asked four questions to encourage the active participation of caregivers: (1) Is this a realistic scenario? (2) Have you ever used this kind of intervention strategy? (3) What elements or situations of the scenario are not realistic? (4) What would you change in the scenario to make it more realistic? Finally, we asked the participants to envision new scenarios with the current or possible new features for EVA.
We used an inductive approach [
65] to analyze data gathered from the transcripts of the workshop. The results showed that scenarios 1 and 2, personalized conversations and music therapy, were considered the most promising. However, the participants coincided that a combination of both scenarios might be more effective at engaging PwD. In addition, the results coincide with those of the first phase about the relevance and importance of personalized interactions and the use of music as an essential feature. Finally, participants suggested that group sessions with the social robot would be more effective for PwD because they would feel more comfortable accompanied by their fellow residents, which might increase their participation and enjoyment.
4.1.3 Implement Support for Interaction.
We designed an interactive group session based on the results of the previous phase. The session included elements of reminiscence (personalized conversations) and music therapy. The concept of the designed session is based on a music therapy session where the social robot EVA asked participants for their favorite music to play it. During the session, EVA encouraged and motivated participants to sing and clap their hands. During pauses, EVA approached each participant to enact a personalized conversation as an element of a reminisce therapy.
At an early stage of the robot prototype, we used the WoZ approach to implement the session. We implemented the interactive session pre-programming a set of utterances (e.g., greetings, motivational phrases, name of participants, conversational connectors, and farewell). During the session, the human operator can manipulate the robot’s behavior in real-time, such as reproducing a song requested by a participant or improvising personalized responses and behaviors when a participant makes an unexpected question or comment during the session.
4.1.4 Assessment of Engagement.
We conducted a study to assess the adoption of the robot by PwD who live in a private nursing home with around 50 residents in the city of Ensenada, Mexico [
16]. The study aimed at evaluating the effectiveness of these personalized conversational strategies in the domain of PwD–robot interaction and how their use impacts PwD engagement with the robot.
A total of 12 PwD who lived in a nursing home participated in the study (
M \(=\) 80.25 years, SD
\(=\) 6.70), but our analysis focused on five who attended at least 10 of 12 group sessions. Participants’
Mini-Mental State Examination (MMSE) scores (
M \(=\) 14.10, SD
\(=\) 4.58) denoted a mild to moderate stage of dementia. The robot EVA guided the session described in the previous section (conversations and music therapy elements), with a research team member operating the robot using the WoZ module. The use of personalized conversational strategies proved effective at increasing the interaction between PwD and the social robot EVA. Results showed a significant increase in the number of utterances from the PwD to the robot and the number of sustained conversations (at least four utterances exchanged between the participant and the robot) [
16].
Figure 3 shows heatmap graphs comparing the activity (number of utterances and expressions of enjoyment, measured by counting, in the video from each session, instances of events such as laughs, singing and clapping.) of a group of participants between their first and last session with the SAR. While they only emit a few utterances to the SAR in the first session, there was a significant increase in both activities, the number of utterances and expressions of enjoyment in the last session.
Figure 3 also shows how the sessions changed from a mainly music-based (eight songs) to a conversation-based session with only three songs. This result show how participants adopted the social robot, increasing their interaction with it over time. It also illustrates how a preliminary evaluation with an early prototype allowed to identify strategies to engage participant interaction that were not originally mentioned by the caregivers.
4.2 Automation Cycle
Although the result of the previous cycle showed a successful engagement with the first prototype of the SAR, the robot’s behavior was controlled by a human operator under the WoZ approach. The automation cycle focuses on automating the social robot to replicate the successful interaction under the WoZ approach.
4.2.1 Analysis of HRIs.
During the engagement study, we gathered data from the video recordings of the sessions, session logs, and filed notes and analyzed them to guide the implementation of the autonomous features of the social robot. The analysis focused on conversations and preferences of the participants to personalize the session. We next describe the main findings that influenced the automated redesign of the robot.
—
Restricted Therapy Script. All participants successfully interacted with the robot showing frequent periods of intense engagement. Most participants seemed to remember the robot and the general aspects of the session. However, they did not recall specific elements of the session, although these were very similar (e.g., conversation topics, played songs, name of the robot). Thus, we propose using a restricted therapy script with few alternatives, focused on those activities that each participant enjoyed.
—
Personalization. In addition to personalized content (e.g., conversation topics, favorite music), it was necessary to include personalization aspects such as how to structure a question/comment and what kind of response to expect from participants. For example, some participants could not respond to open questions but had no problem choosing from alternatives provided to them; others needed more motivation to participate, while some were talkative.
—
Consistent Replies. Participants often replied with very similar or even the same response to a specific comment or query by the SAR. For example, replies for greetings, motivational phrases, and farewell were pretty similar and predictable. Moreover, each participant changed the length of the answers, with some tending to provide short and concrete answers, while others usually provided lengthy replies even to questions that required just a simple yes or no answer.
4.2.2 Design of Autonomous Behaviors.
In this stage, we designed a script template based on the session used in the engagement study that proved effective in engaging PwD. This template includes elements such as greeting, music therapy, conversation topic, and farewell, in addition to two new aspects: complete wisdom sayings and relaxation.
Furthermore, we designed participant profiles that include information on their musical preferences, preferred topic of conversation, type of utterances, motivational level, and average response length. The SAR uses this information for two primary purposes. First, the robot can tailor the content of a session based on the template script, using the preferences and traits of the participants. Second, the robot can personalize the communication with the PwD, knowing the type of utterances to use and what kind of responses are expected.
4.2.3 Implementation of Autonomous Behaviors.
We extended the EVA platform with two additional modules: the Script Generator and Conversation Manager.
The Script Generator module creates a tailored script using the profiles of the group of participants that will be involved in a session. The script defines the activities sequence and turns of the participants.
The Conversation Manager module is responsible for managing the session sequence following the script. It uses the participants’ profiles to formulate the utterance and the type of response it expects to answer. Once the STT and NLU extract the intent of the voice participant input, the module processes it in order to enact the appropriate action, which could be: an utterance, emotion reaction, play a song, change the activity, or finish the session. In the conversation segments, the Conversation Manager module manages the participant answer to modify the conversation path in order to try to follow the conversation. The Conversation Manager uses the responses length of the participants’ profile to decide when and how to provide the following action, giving a little more time to participants to express themselves but without taking control of the session.
4.2.4 Assessment of Autonomy.
Finally, in this phase, we conducted a study to assess the effectiveness of the autonomous features of the social robot. This study was conducted in the same nursing home as the engagement study [
16]. Ten people participated in this study, including selected six participants from the previous study (
M \(=\) 77.22 years, SD
\(=\) 6.53). We compared six sessions of the AR condition with the last six sessions from the adoption study described in
Section 4.1.4 that used the WoZ condition.
In the WoZ condition, participants responded to queries directed to them by the robot 86% of the time, while in the AR condition, participants responded 96% of the time (W
\(=\) 46.5, p-value
\(=\) 0.0049*) (see
Figure 4(a)). We also registered a significant increase (W
\(=\) 48, p-value
\(=\) 0.0078*) in the rate of responses to open queries from the robot (those not directed to one participant in particular), from 65% in the WoZ condition to 81% in the AR condition (see
Figure 4(b)).
On average, the participants directed 2.37 utterances per minute at the robot during the WoZ condition and 2.70 utterances per minute (t \(=\) 1.5054, df \(=\) 23.388, p-value \(=\) 0.1456) in the AR condition.
In terms of perceived enjoyment, there was a significant increase (t \(=\) 3.6937, df \(=\) 24.409, p-value \(=\) 0.0011*) in the number of expressions of enjoyment per minute in the AR condition.
These somewhat surprising results can be partially explained by the fact that the autonomous version of the robot takes advantage of the lessons learned in the WoZ study. This version incorporates a more restricted script based on the utterances and interactions that proved more successful and tailoring utterances to each individual which increased the response rate.
The results of this study showed that when acting in fully autonomous mode, EVA is as effective in engaging participants in the interaction as with the WoZ condition. With the cumulative evidence of the acceptance of the robot and its robustness operating autonomously, the next iteration consist on evaluating the efficacy of the SAR as an assistive technology.
4.3 Efficacy Assessment Cycle
To evaluate the SAR as an assistive technology we conducted an intervention with people in dementia who live in a in the same nursing home as the engagement study [
17]. Our aim was to assess the efficacy of the CST on the frequency and intensity of problematic behaviors and quality of life of the PwD. The efficacy of interventions for BPSD is typically assessed through a combination of standardized assessment tools, such as the NPI instrument, clinical observations, and caregiver reports.
4.3.1 Study Design.
The study design used mixed-methods, including the use of geriatric instruments and the analysis of interviews with caregivers.
The CST included 14 therapeutic sessions conducted by the robot EVA. A session was designed for three participants for approximately 30 minutes. The therapeutic session includes elements of musicotherapy, reminiscence, cognitive games (complete to wisdom sayings), and relaxation.
A total of 10 participants were recruited and were assigned randomly to a specific group session. Inclusion criteria was as follows: (1) diagnosed with early and middle stage of dementia, (2) ability to speak, (3) appropriate level of diction, and (4) an adequate level of hearing. Family members and caregivers who approved the participation of their relative should sign a consent letter. Ethics approval for this study was granted by the Bioethics Commitment of CICESE (CBE/PRES-O/001).
The study was conducted in a lounge of the geriatric residence where all participants live. In each session, a human facilitator welcomes the participants and introduces the robot EVA to them. During the meeting, the robot EVA acts as facilitator, guiding the sequence of recreational activities, as well as turns for each participant. Participant profiles are used by EVA’s Script Generator module to create an interaction script for the session.
To assess the frequency and intensity of dementia-related behavioral symptoms from PwDs we used the
Neuropsychiatric Inventory-Nursing Home (NPI-NH) version instrument [
18]. To evaluate changes in the quality of life participants we used the Qualidem instrument [
21].
Caregivers from the geriatric residence staff were recruited to act as proxies to complete the NPI-NH and Qualidem instruments. We gathered three caregivers assessment for each participant, before and after the intervention.
In addition, we conducted semi-structured interviews with caregivers, for each participant, to get an in-depth understanding of changes in behavior and their possible causes. Interviews were performed before the intervention, in the middle of it and 1 week after it was completed.
4.3.2 Initial Data Gathering.
We interviewed caregivers to learn about the preferences and profile of the participants to personalize the robot interactions. This included information such as cities were they have lived, meals they liked, and some of the music they enjoy listening to.
As indicated above we also completed the MMSE, NPI-NH, and Qualidem instruments with the help of the caregivers (three per participant) to be used as baseline.
4.3.3 Intervention.
Fourteen group sessions of CST with the robot EVA were conducted for 9 weeks. A total of nine residents (six females and three males), aged between 74 and 95 (M \(=\) 83.77, SD \(=\) 8.13), participated in the study. Their MMSE scores denote early to moderate-stage dementia (M \(=\) 14.57, SD \(=\) 3.57).
In addition, six caregivers participated in the study (four females and two males) with an average of 3.2 years of experience as a caregiver in the geriatric residence.
4.3.4 Analysis and Results.
Data analysis was performed in eight participants. Originally we recruited nine participants, but one of them (P3) left the residence and the study after six sessions due to health issues.
Table 2 shows results obtained from MMSE. NPI-NH and Qualidem scores before and after the CST facilitated by the robot EVA. Results from the Wilcoxon test show a significant decrease in NPI-NH total scores and an increase in Qualidem total scores. In particular, the NPI-NH sub-domains of delusions, agitation/aggression, euphoria/exaltation show a statistically significant decrease. With respect to Qualidem, the constructs of positive self-image and Feeling at home show significant increase. No statistically significant decrease in MSSE was recorded.
A total of 48 interviews were analyzed using the inductive coding method, half of them conducted in the middle of the study and the half after its conclusion. Steps two and three of the inductive coding process were performed independently by two coders. A total of 20 themes emerged from the analysis, that were grouped into four main categories: behavior and mood, daily activities, socialization, and prevalence of the impact [
17].
Overall the study provides evidence of the efficacy of the CST, guided by an autonomous SAR, at reducing the manifestation of problematic behaviors and increasing the quality of life of participants. The qualitative analysis supports this result and provides additional evidence that these effects persist beyond the individual sessions.
5 Design Insights for SAR for Dementia Care
Based on the lessons learned when using the proposed framework we derived the following design insights for SAR for dementia care. These insights are purposely broad and can be applied to different dementia contexts.
Personalized and Restricted Interactions. Personalization is essential in the interaction with PwD, such as including the use of topics of interest to the individual. Furthermore, the different ways in which PwD experience the disease and how these changes as the disease progresses emphasize the need for a person-centered approach to dementia care which should be incorporated in an SAR aimed at assisting PwD and their caregivers.
The importance of personalized interventions has long been recognized in dementia care. Person-centered models in dementia care highlight the importance of tailoring the care to the interests, abilities, history, and personality of PwD and their caregivers [
32]. Non-pharmacological interventions individually tailored to PwD are considered to offer a more effective response to behavioral disorders and should be prioritized in the treatment of these disorders [
13]. Person-centered dementia care emphasizes [
3]: treating the person with dignity and respect; understanding their history, lifestyle, and preferences; looking at situations from the perspective of the PwD; providing opportunities for the person to have conversations and relationships with other people; and ensuring the person has the opportunity to try new things or take part in activities they enjoy.
For example, in the case study, using the name of the person with dementia at the beginning of an utterance increased the number of times the user responded. Therefore, we recommend personalizing the interaction using elements (e.g., names, jokes, conversation topics, music, photos) that participants enjoy. Moreover, the interactions with PwD should not be complex, using short-simple sentences and be clear on the messages being conveyed. We designed SARs social behaviors following specific guidelines like having pre-defined utterances with no more than six words; reducing the speed of the voice synthesizer to 70%; including a 10 seconds delay between utterances; and enabling users to override the system by using a button to repeat the last utterance. The robot hence uses this restricted interaction to redirect the interaction and reengage an individual who has lost focus during the interaction or task.
Tolerance to Communication Breakdowns. PwD may act in different and unpredictable ways that the robot cannot process and respond correctly, generating communication breakdowns. Moreover, the technological limitations of the robot might be the cause of the breakdown, such as failing to understand a user’s utterance. However, PwD exhibited tolerance to these breakdowns and resilience to the recovery strategies of the robot. For example, in the case study, when the robot asked a person for a song name, at times multiple participants responded simultaneously. If the robot cannot interpret the response, even if it repeats the question, it is better for the robot to announce that it will play one of the songs, rather than asking for a third time. We found out that on these instances the participant never minded the mistake and instead exhibited a positive behavior, like start singing along the song being played. Furthermore, we specifically designed for consistent replies, by leveraging the similarities in the way individuals replied to queries from the SAR, and vice versa. So, even though these responses from the robot were incorrect, users found most of them to be familiar and as a consequence they overlooked the error. Thus, it is important to be able to detect communication breakdowns and design strategies to recover from such breakdowns by sustaining the interaction’s fluency.
Translational Science. Interacting with PwD is challenging. Evidence-based practice of dementia care includes therapies, activities, and guidelines for establishing and improving collaboration, interaction, and communication with PwD [
2]. Caregivers, families, and clinicians use and tailor these recommendations for their care recipient. Gathering information from these sources (organizations and experts) supports understanding the context of the use of SAR in dementia and cognitive impairment care [
34].
For example, we found during our field work that prompting and rewards are efficient strategies caregivers use to redirect both the attention and the actions of PwD. Hence, the robot often uses praise as a form of engagement to re-gain the attention of the user and as a sort of strategy to let them know they are in the right path. Also, our SARs use a lot of verbal stimulation, mimicking both the verbal and modeling prompting, caregivers provide to PwD in real life.
Thus, we recommend exploring how to translate successful strategies, therapeutic techniques and elements for dementia care from PwD–caregiver/families/clinician interaction to a PwD–robot interaction.
Proactive over Reactive. PwD may appear to be withdrawn and exhibit a loss of initiative. However, they often respond to gentle prompts and encouragement if someone else takes the lead [
68]. Thus, it is imperative to consider a proactive personality for the robot. We recommended designing interactions guided by the robot using motivational and personalized elements to encourage and engage participants.
Actually, in the first cycle of case study 1 conducted with caregivers, they recommended for the robot to take a more passive role. So, in the initial design the robot would tell the users that it could play any song they liked and waited for an answer. When we changed the interaction to be more proactive, with the robot suggesting the song or providing limited options engagement improved considerably.
Clearly a robot aimed at assisting challenging behaviors, such as disruptive eating, needs to be proactive. But this behavior should be supported by accurate inferences and appropriate interactions to achieve the desired result.
6 Discussion and Limitations
There has been increasing interest in the development of SARs for dementia care [
10,
26]. The design of interactions between PwD and SARs is particularly challenging. We have shown how several design recommendations for interaction design from the HRI literature apply or can be adapted for the specific challenges of interacting with a PwD. We have described a framework that evolved during the 4-year development of an SAR aimed at guiding a CST. The framework was also successfully followed in the design of a second SAR as an eating companion [
5].
Table 3 summarizes the methods and tools that were used during the development of the SARs described in the previous sections, at each phase of the development framework. For instance, to obtain an initial understanding of the domain, we conducted interviews with stakeholders and non-participatory observations (shadowing), while to assess adoption a WoZ study was used in the case study. This list is not exhaustive, but can be seen as a preliminary list of alternatives that can be expanded with new methods or tools. For instance, we are currently working on a simulator of the EVA robot aimed at facilitating preliminary iterative design and evaluation of HRI.
The framework we have proposed is similar to the Bonsai framework described in [
38], in that it emphasizes iterative development and provides primitive programming abstractions and a robotic platform. The Bonsai framework is not as comprehensive, as it can only be applied to phases 3 and 4 of our cycles 1 and 2. In contrast, the framework proposed is oriented toward assistive robots for dementia and it includes a third cycle for evaluating the efficacy of the robot as an assistive technology.
Tianyi Gu and LaRoche [
67] proposed a software framework for dementia care robots. The platform facilitates caregivers the work of changing care protocols, such as medication reminders or preventing wandering, using a planning domain definition language. The framework was evaluated for acceptance using UTAUT. In contrast, with our work, this framework focuses on the technical software components of the robot (implemented on top of a Pioneer 3DX robotics platform) and does not propose user-centered design methods for SAR design nor does it consider an assessment of the efficacy of using the SAR with PwD.
While not properly presented as a design framework, Gasteiger et al. [
25] present the methods used in the participatory design and evaluation of an SAR for mood stabilization and cognitive improvement of older adults. The approach included six phases with some similarities to the ones we have proposed, such as scenario design technical development, and effectiveness and usability evaluation. Their approach however is descriptive, focusing on how it was done, rather than prescriptive with an emphasis on how it should be done, and in this regard the three-cycle methodology goes one step further.
A scoping review by Cibulka et al. [
52] on the use of SAR for dementia, after analyzing 90 relevant articles, states the need for a conceptual framework for effective and safe implementation of social robots for PwD. This work identifies the opportunity of incorporating IoT to gather physiological and behavioral data to support PwD–robot interaction. However, the framework makes no specific proposal regarding methods of tools for SAR development other than involving seniors in the development phase. Our work responds to the need for a dementia-centered concept and implementation framework for SAR proposed by the authors.
The importance and challenges to co-design of technology with PwD have been identified in the HCI literature for several years [
30,
43]. Recommendations and guidelines include incorporating seniors, PwD, and caregivers in participatory design sessions, usually in a group setting, and tailoring methods such as workshops or interviews to elicit feedback [
69]. This line of work has more recently been extended to the conceptual design of SAR for dementia. In [
45] for instance, design workshops with caregivers were used to create scenarios and prototypes of robots. These methods are also incorporated in the initial phases of the first cycle of the proposed framework and their use illustrated with a case study.
Similarly as other HRI design tools, the platform EVA includes a VPL for the rapid prototyping of interactions, and a WoZ component that allows an operator to remotely control the robot. The VPL facilitates the development of relatively simple interactions, but it can also be extended with new components. Some of the components of the framework are designed to facilitate PwD–robot communication, such as the ability to reduce the speed of the speech synthesized to facilitate comprehension by the PwD.
While PwD and caregivers participated in several of the design/evaluation phases as emphasized by the framework, PwD were not active informants or co-designers of the SARs described. There have been some significant efforts in this regard, but mainly with people at early stages of dementia [
64]. In contrast we worked with people with moderate dementia. They were however active users who influenced the design of the SAR thru their interactions with the robot. In addition, caregivers were actively involved during the design and evaluation of both SAR.
The robotic platform EVA has a limited variability in its embodiment which constitutes an additional limitation in the design space that can be covered with it. Other SARs, with quite different embodiment have been successful in dementia care, such as the furred baby seal PARO that invites to be cuddled [
58] or the humanoid torso robot Bandit, whose 19 controllable df allows it to represent a variety of upper-body movements for rehabilitation [
22]. In contrast, EVA uses conversation as its main mode of interaction, which has been recognized as one of the most effective strategies in dealing with behavioral disorders of dementia [
13].