research-article

Open access

Studying Exploration & Long-Term Use of Voice Assistants by Older Adults

Authors:

Pooja Upadhyay,

Sharon Heung,

Shiri Azenkot,

Robin N. BrewerAuthors Info & Claims

CHI '23: Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems

Article No.: 848, Pages 1 - 11

https://doi.org/10.1145/3544548.3580925

Published: 19 April 2023 Publication History

All formats PDF

Abstract

While past research has examined older adults’ voice assistant (VA) use, it is unclear whether VAs provide enough value to sustain use when compared to technologies such as smartphones. Research also suggests that barriers around structured command input may limit use. In order to investigate these gaps in adoption, we conducted interviews with ten older adults in a long-term care community who have adopted Alexa devices for at least one year. Participants learned to use Alexa through a training program that encouraged exploration. They used Alexa to complement their daily routines, improve their mood, engage in cognitively stimulating activities, and support socialization with others. We discuss our findings in the context of prior work, describe strategies to promote VA learning and adoption, and present design recommendations to support aging.

1 Introduction

Voice assistants (VAs), such as Amazon’s Alexa and Google Home, are low-cost voice-enabled computing devices that use voice and conversation as the primary interaction modality. In recent years, they have become more popular with common uses such as retrieving information, playing music, calling others, and controlling one’s smart home. VAs offer an accessible entry-point to computing due to their hands-free modality, compared to computers and smartphones. VAs are also positioned to provide better information access for users such as older adults, who may not use computers and mobile devices due to late-life vision or motor disability.

Prior research has explored how older adults interact with voice assistants. Most studies have focused on short-term use by older adults with no prior experience [11, 23, 24], and have shown that older adults find VAs useful for alarms, reminders, music [11], general information searches [24], and casual conversation including small talk [11, 24]. Kim et al. deployed Google home devices with older adults and found music and search as the most commonly used features [1, 11]. Pradhan et al. deployed VAs with older adults who lived in a low-income facility [23]. The researchers found that some features such as playing music and supporting memory (e.g., with timers and reminders) decreased over time. On the other hand, the researchers found information seeking to be useful, providing high value in the absence of other technology use. However, these interactions could be susceptible to a novelty effect, which may wear off over time, as suggested by [23]. As such, experienced users may have different perceptions and VA use over a longer period.

While most work studied short-term use, one study explored long-term use to find that older adults ended up limiting their VA use or abandoned Alexa devices [32]. The researchers found that VA skills such as weather, information searches, and music provide limited value due to the ease of using other screen-based technologies (e.g., smartphones) [32]. Other researchers suggest that there are barriers to older adult’s VA adoption, both in terms of understanding how to operate VAs (e.g., difficulty phrasing, discovering, and remembering commands), and evaluating the value VAs provide in comparison to other technologies [11, 23, 32]. Thus, there is a gap in understanding how older adults overcome barriers and what value they find in using VAs after longer-term engagement.

To address this gap, we partnered with Soundmind ¹, a startup that incorporated VAs into long-term care communities to support older adult residents and staff. They developed custom VA or skills such as applications to access community events and meal information. Soundmind deployed VA devices in several long-term care communities in a large metropolitan area. As part of their deployment, they also conducted initial training to introduce residents to the potential benefits of an Alexa device through in-person demonstrations and distributing printed flyers regularly in the initial phase. We partnered with one of these communities to investigate the following research questions:

•

RQ1: How did the residents learn to use voice assistants?

•

RQ2: How did the residents use voice assistants to support their routines?

•

RQ3: What were residents’ perceptions of voice assistants’ conversational capacities?

To address these questions, we interviewed ten older adults who participated in Soundmind’s program and used an Amazon Alexa voice assistant for at least one year in one care community. During each interview, we asked participants how they learned to use Alexa and how they typically use Alexa. We concluded the interviews with a contextual demonstration, asking participants to engage with Alexa. We found that our participants learned about VAs through flyers that introduced them to popular Alexa skills and associated commands, and through trial-and-error approaches. Although past research suggests that popular VA skills (such as music and general information searches) alone are not enough to sustain use over time, our findings provide evidence that participants used structured training programs to engage in these tasks, understand Alexa’s conversational capacity, and sustain voice assistant use. Participants used Alexa to support and enhance their routines.

From these findings, we discuss factors that may have contributed to our participants adopting voice assistants, such as how training facilitated command discovery associated to skills and exposed participants to Alexa’s conversational boundaries. We present design recommendations for training strategies to expose older adults to system capacities, encourage exploration, and support them in appropriating skills found to be personally relevant for sustained VA use. We add to a growing body of work exploring older adults’ VA use. Although we interviewed older adults in a long-term care community, our findings could also apply to those aging-in-place. We extend existing research on older adults’ voice assistant use by describing (1) their learning and skill adoption process and (2) sustained, routine use by experienced voice assistant users.

2 Related Works

Voice assistants provide accessible ways for older adults to interact with computing [20, 28], mitigating challenges using computers and smart devices, such as using small fonts or buttons [11]. Furthermore, hands-free interactions can be accessible for those with visual and motor impairments [3]. While VAs are promising in the context of aging, prior work has shown that older adults have complex reasoning behind their use and non-use. While some research reports that older adults may have incomplete models of how VAs work [11, 13], other studies show that older adults have a clear grasp of what the system is capable of and its conversational limitations [11, 24].

This section reviews past work that reports how VAs’ skills may or may not provide utility to older adults (section 2.1) and how their conversational capacities are perceived as useful or limited (section 2.2). While one study describes long-term VA use [32], most of these studies [6, 11, 24] recruited older adults with limited to no prior VA experience or studied those who used VAs over shorter periods (from one hour to four months of VA use). As novelty effects may impact voice assistant use, we extend this prior research by studying older adults who have used voice assistants for at least one year to understand what factors may contribute to sustained use.

2.1 Voice Assistant Use by Older Adults

Research ston VA use suggests that older adults’ VA use resembles general use by all populations [1] such as listening to music, searching for general and health information, engaging in casual conversations, checking the date, time, or weather, and setting reminders [10, 11, 23, 25, 32]. However, qualitative studies have uncovered more complex reasoning behind older adults’ decisions to use or not use specific VA skills.

Trajkova et al. found that older adults limited or abandoned VAs because they struggled to find valuable use cases. The researchers conducted focus groups with older adults in a long-term care community who used VAs for a year. The long-term care community deployed VAs intending to connect residents to information, but participants preferred to use screen-based devices for information searches, music, and weather. For instance, the weather app on the phone could provide information about the entire week. Participants preferred searching on a laptop or smartphone because they were accustomed to doing so [32]. In addition, their participants reported frustrations with VAs repeatedly misunderstanding music retrieval commands, compared to the simplicity of selecting music on the phone. As such, VAs provided limited utility, primarily used for setting alarms for sleeping, cooking, and appointments. In summary, while older adults found VAs useful for voice-based alarms and timers, the perceived utility of VAs seemed to diminish over time due to preferences for using screen-based technologies for music and information searches [32].

In a three-week deployment study, Pradhan et al. distributed VAs in the homes of older adults who were novice VA users and either had limited computer and smartphone access or preferred not to use them. Their participants saw VAs as fulfilling needs (e.g., general information searches) that a computer might otherwise provide and expressed a preference to use VAs over computers for information as VAs were faster, involved fewer steps, and avoided steps like typing in passwords. Participants used VAs to seek information on topics including health, public figures, local information, and food. However, the researchers reported a novelty effect where use declined over time. During the study, participants found it challenging to remember structured command names, which may have led to non-use [23].

Specific to health information through VAs, Shalini et al. found that older adults preferred accessing health information across modalities [29]. Brewer et al. (2022) showed how older adults reformulated their health queries when VAs provided unhelpful responses [6]. Pradhan et al. found that while older adults used VAs to ask health questions, they were concerned about information credibility [23].

2.2 Older Adults’ Conversational Interactions with Voice Assistants

Echoing a broader challenge with the limitations of natural language processing (NLP) systems discussed in prior work [8], a structured nature of dialog with VAs poses limitations in conversational interaction. Older adults may face difficulty discovering and remembering commands associated with VA skills [23, 31]. To mitigate these challenges, older adults often employ trial-and-error approaches [6, 11], often giving up after rephrasing their commands several times and receiving no response from the system [23]. Further, such structured command names limited older adults’ ability to discover new skills [23]. For instance, Pradhan et al. found that although participants expected VAs to be useful for alarms and reminders, commands associated with setting up an alarm or reminder, changing or canceling one (e.g., "cancel reminder on May 31st") caused frustrations and cessation of the use of these skills. Further, their participants expected the system to be able to understand complex commands [23], highlighting the limitations of accessing skills through structured commands.

Researchers have also highlighted limitations in complex two-way conversations. Porcheron et al. argue against calling transactions with VAs a "conversation" [22]. Conversational datasets typically used to train such VAs draw from typed text instances extracted from online forums and social media, which may be more representative of younger adults’ speech styles [27]. Older adults’ conversational styles may involve different syntactic structures, pauses, speeds, or volumes, which NLP systems may not be trained to recognize [11, 28]. Pradhan et al. found "pauses" in conversation leading to frustration for older adults when the system forgets important information context after a pause [23]. Researchers have attributed these limitations to initially designing VAs to support adjacent pairs of questions and answers rather than a turn-by-turn conversation that builds on previous context [22].

Past research on older adults’ VA use looks into their conversational perceptions. Research suggests that after gaining experience with VAs, older adults identified the systems’ limitations (e.g., lack of ability to follow up and inability to mimic a two-way conversation). Despite knowing these limitations, users expressed conversational and companionship as a benefit [11], based on having casual conversations [23] in a question-and-answer format [11]. As such, the nature of conversations, and resulting companionship, with VAs may not be similar to conversations between people. Our study explores how older adults find conversational value in interactions with VAs despite knowing their natural language limitations.

3 Methods

To understand voice assistant use, we conducted semi-structured interviews with ten older adults who used Amazon Alexa devices for at least a year as part of a pilot voice assistant program in their assisted living community.

3.1 Alexa Program for Long-Term Care

We collaborated with Soundmind to investigate voice assistant use among older adults who participated in their pilot program. Soundmind designed Amazon Alexa skills (or applications) for long-term care community residents. These skills provided local updates about community meals, and events (Care plan skill). Soundmind provided live VA demonstrations and subsequent paper training materials These training materials typically included a summary of commonly used Alexa’s skills (e.g., games, music) and the associated commands to activate these skills. For residents who expressed interest and consented to participate in the program, Soundmind installed Amazon Echo and Echo Dot devices in their residents’ rooms and in select common community spaces, such as the lounge area where group activities are held.

3.2 Participant Recruitment & Demographics

After IRB approval from our institutions, we collaborated with one assisted living community in the company’s program located in a large city in the U.S., where Soundmind deployed its pilot program in select common areas and consented residents’ rooms. We recruited residents by sending flyers to the community, which were printed and shared with residents by a community staff member. The staff member supported us with finding and scheduling participants who used Alexa for at least a year. If a resident indicated an interest in the study, the staff member shared the consent form, which described that their participation was voluntary, audio would be recorded, and transcripts would be anonymized. We did not collect information about participants’ health or disability since this information is sensitive.

We recruited ten residents from the assisted living community for interviews (5 male, 5 female, ages 70-80, average age = 78 years old). Each participant owned an Alexa-powered device distributed by Soundmind and used the device for at least one year. Table 1 provides more details about resident demographics and their self-reported technology use.

Table 1:

Participant ID	Age	Gender	Technology Use
1	77	M	TV for news and mental stimulation, Amazon Tablet for Alexa, Zoom, Android phone to ask Google, Zoom
2	74	M	iPad for emails to friends, lists from organizations, Facebook
3	73	F	TV for uplifting shows, news, entertainment. Flip phone for call, text, sending photographs
4	86	M	iPhone for Siri and calls
5	73	M	Computer for specific programs
6	79	F	Information not available
7	76	F	Phone for calls, Radio for news, info on nutrition, Computer for writing
8	83	F	Tablet for Facebook and email
9	84	F	Information not available
10	70	M	Computer for games and email, Phone for calls, TV for news

Table 1: Participant’s self-reported demographics and general technology use

3.3 Interviews

We conducted semi-structured interviews to understand how residents learned and used Alexa in their community since they began participating in the program. Two research team members conducted and recorded 45-90 minute phone interviews with residents in November and December of 2020. During the interview, the interviewer asked residents questions in three categories:

•

Voice Assistant Mental Models. Questions included: “How would you describe the Alexa to someone who doesn’t know about it?” “Where do you think Alexa’s responses come from?”

•

Alexa Adoption and Use. Questions included: “Describe how you learned how to use Alexa?” “Can you tell me about how you use Alexa?” “How often do you use Alexa?” We asked follow-up questions about how often they used certain skills and, if at all, how they incorporated Alexa into their routines.

•

Contextual Demonstration. We asked residents to engage Alexa through scenarios and demonstrate how they would use Alexa for conversation, connect with others, support health and well-being, and search for information. These scenarios were chosen based on common VA uses from prior work [6, 10, 11, 24, 25, 32]. For each scenario, we asked participants to reflect on their interaction and whether anything could have been improved. If they expressed engaging in this before with their VA, we asked them to describe how they learned to do so and to describe the experience.

Each interview ended with demographic questions. Residents were compensated $30 for their participation.

3.4 Qualitative Analysis

We used a third-party service to transcribe each interview. The two coders conducted open coding [26], applied descriptive and in-vivo codes, compared their codes to data, and standardized the codebook language. The coders developed the final codebook iteratively by individually coding three transcripts and comparing the codes over three stages. Iterative coding and comparing quotations to resolve disagreements helped maintain codebook reliability [19]. Then the two coders combined descriptive codes into categories and created a stabilized codebook, coding all ten transcripts. At the end of the coding process, there were 129 descriptive codes in 21 broad categories. Some examples of code categories and descriptive codes from our codebook include:

•

Value in using or not using Alexa over other technology: “Affordance of hands-free”, “Affordance of visual modality”, “Affordance of voice modality”, “Easier, quicker to use Alexa for information”, “Valuable to use Alexa at that point of time for information”, “Verifying what they know with Alexa”, “Uses other technology dependent on ease of use, location, type of information.”

•

Ways to use Alexa: Use depending on the time of day, Use passively/in the background, Using only desired features, Motivations behind use, Using Alexa with others, Using Alexa throughout the day

•

Alexa Expectations: Aspirational uses, Frustrations, Satisfied despite flaws, Wants access to more features or more variety of features, Wants reminders, Wants more detail in information provided by Alexa

Finally, the two coders discussed the codebook application with the team of researchers. Two authors coded and analyzed the coded data following a thematic analysis approach [4]. Using a Miro board, the coders organized groups of codes using a digital affinity mapping process to identify emergent themes (e.g., “ Alexa is a machine with human qualities"). Following this process, we did another round of analysis while writing the findings section to understand how themes (i.e., training, exploration, participants’ models, and Alexa skill use) may be related.

4 Findings

In this section, we first present how participants learned to use Alexa (RQ1). Then, we detail how participants used Alexa to support routines and community living (RQ2). Lastly, we share how participants described Alexa’s conversational value and limitations (RQ3).

4.1 Learning How to Use Alexa

Participants first learned to use Alexa through Soundmind’s training process, including printed flyers describing potential use cases and in-person demonstrations with the Alexa device. Participants also described learning through trial and error, similar to how they interact with other technologies.

Most participants (P2, P3, P4, P8, P9, P10) described learning from the flyers provided by Soundmind. Community staff shared one-page flyers with residents weekly. P2 described it as a “form above Alexa on our wall,” showing how the information on the flier could easily be accessed while interacting with the device. Flyers included examples of searching for information (P2 - “ask about flowers”, P3 - “Happy birthday in Korean”), asking for the weather and important dates (P3 - “How many days to thanksgiving”) and commands related to existing VA skills like playing games (P1 - “new games you can play”). P3 listed examples showing a variety of commands described in the flyers: "What are you doing for your birthday?, How do you say happy birthday in Korean? [and] How cold [will] it be tomorrow?”.

In addition to the flyers, several participants (P2, P3, P5, P6, P7, P8) mentioned that they learned how to use Alexa by observing Alexa use through in-person training. By attending a staff demonstration with Alexa, P8 became interested in Alexa. Similarly, P6 said, “one of the social workers came up and stay[ed] with me in the room and show[ed] me what to do…I thought [it was] very interesting. and it’s something I never had before.”

Participants explained that this training process guided their initial discovery of Alexa’s capabilities. For instance, P4 referred to the printed flyer to justify why he used skills for music, alarms, and meal information. After familiarizing himself with these skills, P4 expressed confidence in learning to use Alexa, saying:

“The only thing in the training I had is that paper they gave me, the directions on what to do. So that’s all I know. After you use it you get used to it [Alexa], you know…” (P4)

Participants also described learning how to use Alexa as an ongoing process, “learn[ing] more all the time” and “discover[ing] Alexa". P5 explained that they “learned more all the time”, and P8 explained it as “discover Alexa…day by day” (P5, P8). P6 described how the flier helped her understand how to craft questions to Alexa, “trying to ask her some questions which I never did before getting used to her”. Some participants learned to use Alexa in the same way they would learn to use other technologies. Two participants described trial-and-error learning strategies (P1, P2). For example, P2 said, “I hit and [it] misfires, kind of like [what] I do on my iPad. I try something [with Alexa] and see if it works, like a question… and then I try something else.” P1 compared learning to use Alexa with learning to use his smartphone and tablet:

“Like with the phone, I find something new…accidentally and I [say] ‘Oh my god, I got that on it too?’... it’s trial and error. Like the tablet that I’ve got… I’ll start to learn it a little every day… the same with Alexa. You have so much on her I know that. I had no idea that it’s there but I was never given a list of everything that she can do. Let me put it that way.” (P1)

Our findings show that engaging with the training process prompted participants to discover new VA skills relevant to their everyday life, learn conversational strategies to issue commands, and continue to use Alexa.

4.2 Incorporating Alexa into Daily Life

After understanding the participant’s initial learning processes, we sought to understand how they routinely used Alexa. Participants incorporated Alexa’s skills to support everyday tasks, entertain themselves, manage their mood, and socialize with other residents. This included using Alexa’s skills to support everyday routines through activities at specific times with Alexa, playing music throughout the day, seeking information and stimulating games, and having casual conversations (see Table 2).

Table 2:

P ID	Date/Time	Weather	Alarm	Games	Jokes	Music	Facts	Care Plan
1	Y	Y	Y	Y	Y	Y	Y	N
2	Y	Y	N	N	N	N	Y	Y
3	N	Y	Y	Y	Y	Y	Y	Y
4	N	N	Y	N	N	Y	Y	Y
5	Y	Y	N	N	Y	N	Y	N
6	N	Y	N	N	Y	Y	Y	N
7	Y	N	Y	N	Y	Y	Y	Y
8	N	Y	Y	Y	Y	Y	Y	Y
9	N	N	N	Y	N	Y	N	N
10	Y	Y	N	Y	N	Y	Y	N

Table 2: Participants’ Routine Alexa Use

4.2.1 Supporting Everyday Routines.

All participants used Alexa daily, sharing that they make commands twice or more each day. P10 used Alexa even more frequently, “15-20 times[ a day and] sometimes more depending.” Most often, participants used it for productive and informative tasks such as setting alarms and inquiring about the date, time, weather, and schedule.

Aligning with prior work, participants used Alexa as an alarm or timer (P1, P3, P4, P7, P8). P3 recalled Alexa’s reliability in waking her up: “every once in a while, I might take a little nap for about a half an hour and give her the time, and she always wakes me up at that time.” P1 used Alexa with his phone alarms to ensure he woke up in the morning. Other participants used the timer skill while cooking (P4). Echoing past findings about difficulties around command discoverability, participants did not know how to set up their own reminders, but expressed the desire to do so (P4, P5, P7). P7 suggested that Alexa could be useful to remind her to place an important call to her doctor; “I had something out on my back, and they don’t know if it’s cancerous or not. I’d like her [Alexa] to remind me the day that I’m supposed to find out, so I can call the doctor.” P4 echoed that reminders could be useful, but that he had no experience creating reminders:

“You could apply this [Alexa] and I’m pretty sure in a lot of other ways. It could be a reminder of appointments you might have, things like that. But I haven’t put that [to] use yet.” (P4)

Participants mentioned using Alexa’s skills in many ways to supplement their daily routines, unrelated to scheduling-related skills (e.g. calendar, alarm etc). For instance, P4 mentioned doing their “Crossword” in the morning newspaper with Alexa (using Alexa to ask for spellings) for the first hour, followed by “soft music while shaving and walking around”, and then “Alexa lets me know what time I’m going to have breakfast so the next 15 minutes I put the paper down and wait for the food” (P4). Similar to P4, other participants too used the Care Plan skill developed by Soundmind to inquire about the daily menu and scheduled activities. P5 explained that he liked to ask about the “schedule for the day” in the morning. P7 described how they start their routine with Alexa in the morning when “she wakes me up, then I ask for news, weather”. P1 describes Alexa as providing “Music to get dressed to…when I am in the bathroom taking a shower”. Others mentioned using music to relax to in the afternoon [P8] or night [P3].

While the Care Plan skill was useful for meal and daily schedule information, participants described the schedule announcements as frustrating. For P3 and P10, Alexa would announce the schedule too frequently, so they decided to use a printed schedule instead.

"In the beginning, I used to ask [Alexa] about the schedule, but that drove me nuts. Every 15 minutes, it would say - you have this activity, that activity. So once that happened to me, I just canceled [it] because I have a [paper] schedule that they give out and it tells me what’s happening, what day and what time." (P3)

These findings show how older adults found and selected relevant voice assistant skills (e.g. local information, alarms, news, spellings, music) to supplement or enhance their existing daily routines, using Alexa in the background while doing other activities. Commands around remembering appointments and creating reminders were not easy to discover. While local information relevant to their daily routines was found useful, participants faced frustrations with unprompted frequent schedule reminders.

4.2.2 Improving Mood.

Several participants described that using Alexa to listen to music, relaxing sounds, or meditation helped improve their mood. Participants used Alexa for music, leveraging voice to issue a single command to retrieve a music genre rather than a specific track (e.g. play pop, Sinatra), and leveraging its hands-free affordances by using music commands while doing other activities (relaxing, going to sleep).

As it was a skill commonly discussed in training, participants played music at different times of the day. P1 scheduled Alexa to play music at a certain time in the evening, explaining that “right now [he has] music at four o’clock.” For P3, playing music at night was beneficial: “I ask [Alexa] at nighttime to play the sounds of the ocean [because] it’s kind of soothing for me.” Participants (P2, P7, P10) also mentioned using Alexa’s music skills while doing other activities, e.g., shaving or relaxing. P4 described that Alexa’s music has become a part of his morning routine: “when I get up in the morning, I put on something soft [music] when I’m shaving, walking around and everything. And then I shut it off.”

Multiple participants enjoyed being able to retrieve diverse artists (P3, P6, P10), music from Broadway shows (P1, P8), radio stations (P3, P6) and different genres (P1, P9, P10); one participant described Alexa as a “music reservoir” (P10). P10 preferred to use a hands-free device instead of a computer to play music, saying:

“There are times when I’m tired of watching television and I don’t want to go to my computer. And I’ll have music on… And I can get the type of music I want at that particular time. So you know if I want Bizet, then I’ll ask to play Bizet…then I’ll ask it to play George and so on. I like that it’s a music reservoir and [it] plays different songs.” (P10)

Some participants decided what music to play based on how they were feeling (P4, P6, P8). For instance, P4 played “soft Jazz when relaxing” and “pop music when excited.” P8 communicated to Alexa how they were feeling before asking to play music, using the command, “Alexa, I’m sad, play show music.” P9 found comfort in using Alexa for music during a time when she lost mobility due to an injury:

"I happen to like [Alexa] very much. Because in the beginning of the pandemic… I fell from a crack in the sidewalk. So I needed to lie down, face down on my bed or on my back. So I would sit in the big chair and go to sleep. And I would use Alexa [to] play music. The music is what hooked me in the beginning, I was always a Frank Sinatra fan.” (P9)

One participant demonstrated how Alexa may be limited in its capacities in saving and retrieving a saved music track. P7 used Alexa to find a guided meditation track she really enjoyed, but could not find anymore to incorporate this into her routine. P7 further explained that if Alexa is able to retain her music preferences, Alexa could possibly remember her preferences with the meditation track as well. P7 stated that “she [Alexa] does [remember what I listened to] with music... She says: ‘you listened to so and so last night, Would you like to hear it again?”

Although some participants owned other devices that could play music, the interviews showed that they preferred doing so with the voice assistant based on the ability to retrieve, select, and play music through simple commands (e.g. a radio station, jazz music, Sinatra). Additionally, participants could start, stop, or change the music without interrupting their ongoing activities (relaxing, recovery).

4.2.3 Stimulating Cognition Through Learning.

Participants used Alexa for cognitive stimulation by learning about new commands, asking for different types of information, playing games, or using it to support mentally stimulating offline activities.

Participants expressed that they enjoyed learning new skills and using Alexa was one way “to be mentally challenged”, “learn every day” (P10), and keep them from boredom (P4). P1 noticed how using Alexa on his tablet gave him a different weather-related response than the one on his Amazon Echo and said, “see? She just stimulated my brain with this” (P1).

Our participants found that the ability to ask questions to learn at any desired time provided daily mental stimulation. Almost all (See Table 2) participants described asking Alexa about facts typically to “keep updated” (P2) and “answer their own questions” (P1). Participants asked questions to verify facts (e.g., “the population of Dakota”, P1), learn about current affairs (e.g., “presidential elections”, P6, P10), receive information about their local environments (e.g., “dine-in restaurants near me”, P3), and learn about resources (e.g., “unemployment information”, P4). P10 enjoyed using Alexa to learn new facts and recounted that “the other day [he] was watching [something] and [he] couldn’t remember something so [he] asked her [Alexa]... it’s like talking to an encyclopedia.” In this example, P10 used Alexa as a memory aid to support recalling facts.

Participants clarified why they used Alexa for information instead of other sources. P1 acknowledged that he can “put on a channel on the TV and I have 24-hour news but if you need something really quick” he would use Alexa. P10 echoed this utility and said:

“I find Alexa a lot faster, quicker, and easier rather than going online verifying the data [...] I ask questions that I can get but it would take time to get…I find Alexa a lot faster, quicker, and easier rather than going online.”

P1 and P10 found it useful to verify what they might already know but might be time-consuming to find on their own.

Some participants played games such as “question of the day” (P1), trivia (P3, P9), guessing games (P8), and Jeopardy (P10). P1 explained:

“Coming into this place, I never dealt with Alexa before. I knew of Alexa… because I saw the commercials on TV for Alexa and Siri… I enjoy having it, it’s informative, it keeps me stimulated mentally. I play question of the day with it, I play Jeopardy with it, How to Be a Millionaire, you know, I have different things with it. So for me, I find it to be helpful and handy." (P1)

Other participants incorporated Alexa into informative offline games they currently play. P4 used Alexa to assist with his daily crossword puzzles in the morning paper, checking spellings and word definitions – “I would read something and say, ‘what does this word mean?’ And it’d spell it out. And then it would tell me what it means.”

A few participants expressed their desire to engage in additional creative activities with Alexa. For example, P4 said “I would like her to show me every day the good art that I love (Michelangelo and Dali).” and explained that it should be possible to connect Alexa to the TV, but does not know how to do so, “I know it can be done. I’m positive. If they did this [create Alexa), they can do that.if we connect it to the TV, it should be able to use the bottom of the screen.” P5 wanted to use Alexa to read audiobooks "Alexa, open book on War of the Worlds,” a command Alexa did not understand. These quotes show when participants wanted to use VAs and how they used them for cognitive stimulation.

4.2.4 Facilitating Social Activities.

Participants described using Alexa to engage with other people in the room or in group settings with staff members, where Alexa could support human-human interaction rather than interaction between the user and the device.

To some participants, Alexa was a conversation starter. For example, when P2 invited guests over, Alexa sometimes spoke and “they [would] get a kick out of that.” To make others laugh, P4 intentionally prompted Alexa to tell a joke when others were in his room, saying, “sometimes someone comes into my room to fix something. And I’ll ask them: do you want to laugh? If they say yes, I play [Alexa].” P5 even introduced visiting friends to Alexa: "I have my friend here, she’s so and so, say hello to her."

In community common areas, the staff used Alexa to facilitate group activities such as trivia, listening to music together, or as background music during group activities like exercise. P6 explained that staff members typically played music that aligns with a group’s consensus since “everybody likes a certain station.” Some participants recounted conflict during group use. For instance, P3 described a situation with conflicting musical preferences:

“They [the residents] can ask Alexa down there [lobby], and they love Frank Sinatra. I would rather hear Cher, but you know, or Tony Bennett, because there are a lot of older residents there. And sometimes there is one person that dominates it.” (P3)

As a result of this conflict, P3 did not use Alexa in the common area. P3 also explained another instance where Alexa continued to automatically play music which did not fit their group exercise class, which led them to ask Alexa to stop playing the music, and request that Alexa play another song.

“We had the exercise director there, she would ask it to play certain music, and it always, most of the time, would start out playing that particular song, but then it would go into other songs… sometimes it was frustrating. And then the exercise person would just tell it to cancel.” (P3).

We argue that voice assistants have the potential to further support group activities and older adult communities.

4.3 Perceptions of Alexa’s Conversational Capabilities

Participants had varied interpretations of how exactly Alexa works, which may have affected use and adoption over time. Some participants did not know where Alexa’s responses come from, while others explained its responses were from the internet, a computer, or programmed by “IT.” Regardless, participants understood Alexa’s natural language capacities and valued conversational utility, given its limitations.

4.3.1 Adapting to Alexa’s Limitations.

Participants noted limitations to Alexa’s conversational capabilities, especially Alexa’s inability to answer all questions.

Participants described their interaction with Alexa as a “one-way conversation” (P1), or “[not a] real conversation” (P8). Participants often compared Alexa’s conversational ability to that of a human. For instance, P1 elaborated on Alexa’s inability to ask follow-up questions, saying:

“She can give me certain information. But if I have another question just to simply say follow up, like I’m saying to you, I’m not going to get a response from her like I do from you. So no, she is what she is [not a human]...She’s just Alexa.” (P1)

Participants also described a need for more “more extensive, more detailed” answers (P5), and “longer” answers (P7).

Based on interactions with Alexa, participants formed their own mental models of Alexa’s response boundaries. P10 shared that Alexa is capable of answering “specific” questions, saying “Alexa, what happened during the second World War? It gives me a specific answer because I was specific and it was correct." Similarly P5 explained that she has “learn[ed] how to ask the question”. When Alexa could not answer P5’s questions, she often tried “another way of asking the question”. P2 described a strategy where he waits before repeating questions to Alexa.

However, Alexa’s inability to answer questions was frustrating. Like several other participants, P2 managed his frustration to statements like “I’m not quite sure how to help you” by reminding himself that Alexa was not human, saying:

“Sometimes you’ll ask a question and she’ll act confused so that’s sort of frustrating... she gets mixed up when you give her [a] command [that is a] very rational question and she’ll tell you that she doesn’t understand that…or she can’t help you with it. But then you have to realize that it’s only a technology; it’s not a human being.” (P2)

Although all participants personified Alexa, many participants reminded the interviewers that Alexa is not a “live person” and “just a machine” (P7). Being a machine, Alexa could not “replace the human touch” (P2). Although some participants found Alexa able to provide company, P10 stated “human interaction helps with loneliness,” not Alexa (P10). P5 wanted Alexa to be more human-like, explaining that if he could change Alexa, he would have it “be like a person or something” (P5).

These quotes show how participants built an understanding of Alexa’s conversational capacity, acknowledging and accepting system limitations, and adjusting their interaction styles for use over time.

4.3.2 Alexa as a Conversation Partner.

Participants found conversations with Alexa to be valuable, describing a companionship that had attributes that were different from human companionship. For example, P2 described Alexa as non-intrusive: “a companion...that is always there but non-intrusive and it’s a wonderful addition to our daily life.” Another resident described their interaction with Alexa as socializing with a friend because Alexa was always there to listen to their questions. P8 explained, “I talk to Alexa. She’s my friend. Because I ask her and she answers. She listens better than our kids.” Participants valued Alexa because of its politeness and humor (P1, P2, P3, P6). For instance, P6 mentioned that she would say to Alexa “have a nice night [and] she always tells me you have a pleasant evening too” (P6). P1 felt a sense of independence in simply being able to ask Alexa questions and sometimes he answers them himself:

“It’s nice. It’s a voice. It’s something, you know. I mean, I talk to myself, like, I think a lot of people do you know. What’s good is I answer my own questions, I think it’s healthy. But it’s nice in the fact that I had something like that. So how I would describe it is, it’s very helpful.” (P1)

Participants described Alexa as “very fun” (P6) and was “happy she makes me laugh” (P7). P7 discovered silly noises (i.e., fart sounds), and P1 used Alexa’s “joke of the day” feature, saying that “because they’re so corny, I laughed” (P1). As part of these interactions, participants also explored how Alexa responded to human-like questions. For instance, P1 asked “Alexa, are you married?”. After one interaction, P8 asked Alexa, “what happened, you have a sore throat?". P3 engaged with the voice agent’s programmed set of responses to its birthday by asking, "Alexa, what are you doing for your birthday?". P8 demonstrated a similar conversation with Alexa:

Alexa: "It’s my birthday today".
P8: "Today is your birthday?"
Alexa: "My birthday cake! [pause] I forgot to blow the candles out, oww".
P8: "Ah, what happened, did that burn?"

P4 shared a command he had never tried before. He received an unexpected, yet funny reply:

P4: “Hello, Alexa. I’m feeling fine.
Alexa: “No, fine is not an emotion.”
P4: “That’s a great answer [laughs]... That was a five [rating Alexa’s response on a scale of 1-5] because she got me.”

Some participants felt their conversations with Alexa were especially valuable when they felt lonely. P2 explained that Alexa is “like having another person in the room.” Another resident explained that Alexa was the “best thing invented” for a “lonely person” (P8). P8 explained that residents were isolated in their rooms due to COVID-related restrictions, so she appreciated talking to Alexa and “the pleasure of [Alexa] answering back.” P1 detailed that Alexa mitigated feelings of loneliness:

“Most of us are alone, you know, we come in alone. Being that I live alone, you know, and also more so with this virus, ‘she’ [...] becomes a part of you in a sense that you can talk [about] things.” (P3)

These quotes suggest that despite noting its limitations in conversational capacity, Alexa was helpful beyond productivity for participants and could be used to support companionship, especially when isolated from others (e.g. post-surgery, pandemic).

5 Discussion

In our study, we provide empirical data showing how older adult long-term care residents learned and used Amazon Alexa voice assistants. We detail how training encouraged participants to discover new features support routines and attempts to work around limited conversational capacity with VAs.

In this section, we contextualize how Soundmind’s training, VA exploration, feature awareness, and conversational capacities contributed to participants adopting VAs (RQ1 & RQ2) and argue for better conversational support (RQ3). We extend prior work by discussing factors that contributed to participants’ Alexa over time.

5.1 Designing Voice Assistants for Sustained Use

Past research reports that older adults use voice assistants for checking the date, time, or weather; playing music; and searching for general information [11, 23, 32], similar to how younger adults use voice assistants [1]. However, prior work also suggests that these activities may not be enough for sustained use, resulting in older adults abandoning their VA devices [32]. In contrast, we present empirical evidence of residents who have adopted Alexa into their everyday routines for at least over a year. In this section, we discuss factors that may have contributed to their adoption and sustained use, and present design recommendations for training and VA skills to support older adults across living environments.

5.1.1 Supporting Mixed-Modality Training (RQ1).

Researchers have identified potential barriers to older adults’ voice assistant use. For example, older adults may face challenges in discovering commands related to unknown or new skills, [23, 28] and remembering the language used to initiate complex commands (e.g. reminders) [23]. Also, older adults may prefer to use existing screen-based devices for tasks that can also be accomplished with voice assistants such as information seeking or playing music [32], attributed to VA command unawareness. Our findings indicate that VAs may not clearly communicate how they function, or what utilities they provide. As such, our participants benefitted from a training component. To address such discoverability challenges, Soundmind designed a training process with in-person demonstrations and frequently updated flyers listing potential commands and skills. Training materials provided examples of search queries and popular VA skills (e.g., games, meditation, music), which were helpful for envisioning how voice assistants could be useful and made to fit within their routines. Through the initial training process, participants learned how to structure commands so that the VA could understand and identified skills that would support their existing routines. For example, they requested music while showering, requested a “type of music” rather than a specific track, asked for word spelling and definitions while completing their morning crossword puzzle, and quickly cross-checked facts. Thus, we argue that designing for sustained VA use for older adults should start with improving training processes to support learning and command discovery.

Participants stored training flyers close to the device, which allowed them to learn incrementally and explore new skills. The skills presented on the flyers were curated for older adults living in long-term care communities and participants found the skills to be relevant. Regularly updated flyers in a one-page format that could be kept close to the device were more effective than a one-time flyer researchers have used in the past [23]. Similar types of curated training materials and continuous training processes could introduce older adults in other residential settings or those aging-in-place /updateto explore how VAs can address their individual needs.

In summary, we recommend that VA learning and exploration should occur off-device (i.e., flyers and in-person demonstrations) and with the device (i.e., ‘how to’ commands). Given that traditional training materials such as manuals may exclude newly developed skills, we also recommend providing on-device support for older adults to ask about capabilities or uses they would like to explore. For instance, VAs could have “how to” conversations that inform users about new commands and skills.

5.1.2 Designing for Meaningful Voice Assistant Use (RQ2).

Through our interviews, we surface how participants used Alexa in meaningful ways that contributed to their sustained use over time. In this section, we discuss valued VA uses and present design recommendations for designing multi-step skill scaffolding, incorporating learning and cognitive stimulation skills into training resources, and supporting group interactions.

Design multi-step skill scaffolding: Our findings align with prior research on the potential utility of alarms, timers, and reminders to help maintain older adults’ routines [23]. While past research has identified medication reminders as being potentially useful for older adults[21], our participants did not know how to set up reminders. The language associated with these commands tends to require more complexity than other commands (e.g., playing music). For example, setting, modifying, or canceling a reminder may require including the date, time, and frequency. For instance, another useful skill in supporting participants’ routines was the Care Plan application designed and implemented by Soundmind. The Care Plan skill provided information about community activities and the menu of the day, which was found to be useful. Building on weaknesses of multi-step reminders and strengths of skills like Care Plan, we recommend that designers incorporate custom skill scaffolding to support multi-step commands and applications relevant to older adults, including community information, schedule details, and health-related reminders.

Incorporate learning and cognitive stimulation skills: Participants described how using VAs for cognitive stimulation supported their desire for learning. For some participants, simply interacting with Alexa was stimulating, while others intentionally used Alexa to acquire information about artists, jokes, or specific topics of interest like “flowers” and “World War II.” Participants referred to Alexa as an “encyclopedia” and used it as a memory aid, refreshing and fact-checking their existing knowledge and learning new information. Learning and memory support are often motivating factors for why older adults engage online [5, 9, 33]. Therefore, we argue VAs should continue to support older adults in maintaining cognitive health. And, VA training materials can better incorporate skills that allow people to learn new knowledge.

Provide improved sound interaction: Music was the most commonly discussed VA skill, helping participants with their mood and complementing their daily routines. Similar to prior work discovered, our participants found great value in using Alexa for music during routine activities such as while shaving in the morning [11]. Music was also a way for participants to pursue their fandom (e.g. playing their favorite artist), structure their day (meditation track and soothing sounds at night), and improve their mood (e.g. playing music depending on how they were feeling). It is important to note that our participants requested music in various ways, such as specifying the track, artist, or genre; or requesting a radio station, music from a movie, or music from a Broadway show. As such, they often created their own short commands to retrieve a type of music content, letting the VA select and continue playing that type of music, rather than using a specific retrieval command for a song that may be prone to errors in retrieval. Training flyers may have increased music-related command awareness. We recommend researchers and designers investigate more nuanced music-related interactions such as creating playlists, bookmarking desired content, and retrieving saved content.

Support group interactions: Our participants used VAs during group activities and to support social interactions with other older adults. Using VAs to support groups of older adults and care staff was a use case desired by older adults residing in long-term care communities in prior work [32]. Since most researchers have primarily studied group voice assistant use in home settings with children and parents [2, 14, 17, 18], we encourage future work to explore VA group use in community living. For example, new skills could adjust music based on activity, support playlist development, or engage small groups in entertainment/games. Open questions include resolving conflict based on VA recommendations in group settings.

5.2 Designing VAs as Conversational Partners (RQ3)

Our findings revealed how older adults explored and critiqued Alexa’s conversational boundaries. We extend prior work discussing voice assistant anthropomorphization [28] to reflect on how older adults navigated VAs’ conversational limitations and engage with broader social concepts such as companionship. Prior work suggests that experienced users find conversation and companionship beneficial [11]. Our findings show how this happened when older adults explored the system through enough training and self-exploration to differentiate between pre-programmed aspects of the voice agents (e.g. conversational sequences like Alexa’s birthday), and the capabilities of the natural language processing to engage in natural conversation.

While our participants enjoyed casual conversations with voice assistants, they also highlighted VAs inability to understand and respond to detailed, follow-up questions. These concerns have been echoed in recent work aiming to improve conversational ability that accounts for context [12, 22]. While preserving conversational context, there may be times when conversational context could be ignored. For example, voice assistants that do not preserve context could be useful for older adults who may have a tendency to repeat questions the questions they recently asked, due to cognitive concerns. However, it may not be appropriate to retain certain conversations for privacy reasons (e.g., health). Designers could explore ways for users to choose when they want VAs to remember conversations, and future research could investigate when “always listening” may not be beneficial.

Participants described how VAs could be useful as companions as they appreciated having control over initiating conversations with voice assistants. The “non-intrusive” nature of Alexa was appreciated, reinforcing how older adults are seeking agency in their technology use. Accounting for agency in VA use is particularly important in long-term care settings where technology can often be designed to support the goals of other stakeholders (e.g., family members, care community staff/management, and medical professionals) [15]. For example, health and AI researchers have studied how voice assistants can identify, detect, and report cognitive decline [16], but older adults may not want this to take place [7]. Rather than focusing on detection and identifying decline, we argue that voice assistants should be used as tools to empower older adults, in supporting their own efforts to stay cognitively active.

5.3 Limitations & Future Work

One limitation of this work is that assisted living residents self-selected to participate in the study, which meant that we did not interview residents who abandoned Alexa or chose to unenroll from the VA program. As prior work presents perspectives of VA non-use [32], perspectives from older adult VA users provide a unique contribution to understanding older adults’ VA interactions. Additionally, we did not collect demographic information about disability for privacy purposes. Instead, we provide examples in the findings of participants disclosing how disability affected their Alexa use (e.g., tremors). Lastly, we report on a small sample of older adults, which is not intended to be generalizable to all older adults [30], rather to provide a nuanced empirical description of how this group of older adult participants use Alexa.

Future work could unpack differences in VA use by environment type (i.e., residential and community settings, assisted living, independent living). In addition, it may be insightful to conduct an in-depth temporal analysis of Alexa usage over a longer period of time and engage older adults in participatory methods to build VA skills that support their unique routine and care needs.

6 Conclusion

To understand how older adults in long-term care communities routinely engage with Alexa, we conducted an interview study with ten residents within one community. In contrast to past research which suggests that common VA skills are not enough for older adults to sustain use, we provide evidence of residents applying strategies learned from a training process to adopt such skills (e.g., alarm, music, games) into their routines. While Alexa provided responses that supported daily tasks, participants benefited from its conversational nature and recognized its limitations. This study provides empirical evidence of how training impacts adoption and how voice assistants can be used to support routine use for older adults aging in place.

Acknowledgments

We thank Soundmind for their collaboration in making this study possible. We also thank Tawfiq Ammari and Bruna Oewel for their contributions in early drafts of this work. Lastly, we thank Lucy Jiang and our anonymous reviewers for helping improve and proofread the paper.

Footnote

Acquired by Speak2 Family - https://www.speak2family.com/

Supplementary Material

MP4 File (3544548.3580925-talk-video.mp4)

Pre-recorded Video Presentation

Download
18.20 MB

References

[1]

Tawfiq Ammari, Jofish Kaye, Janice Y Tsai, and Frank Bentley. 2019. Music, Search, and IoT: How People (Really) Use Voice Assistants.ACM Trans. Comput. Hum. Interact. 26, 3 (2019), 17–1.

P ID	Date/Time	Weather	Alarm	Games	Jokes	Music	Facts	Care Plan
1	Y	Y	Y	Y	Y	Y	Y	N
2	Y	Y	N	N	N	N	Y	Y
3	N	Y	Y	Y	Y	Y	Y	Y
4	N	N	Y	N	N	Y	Y	Y
5	Y	Y	N	N	Y	N	Y	N
6	N	Y	N	N	Y	Y	Y	N
7	Y	N	Y	N	Y	Y	Y	Y
8	N	Y	Y	Y	Y	Y	Y	Y
9	N	N	N	Y	N	Y	N	N
10	Y	Y	N	Y	N	Y	Y	N

P ID	Date/Time	Weather	Alarm	Games	Jokes	Music	Facts	Care Plan
1	Y	Y	Y	Y	Y	Y	Y	N
2	Y	Y	N	N	N	N	Y	Y
3	N	Y	Y	Y	Y	Y	Y	Y
4	N	N	Y	N	N	Y	Y	Y
5	Y	Y	N	N	Y	N	Y	N
6	N	Y	N	N	Y	Y	Y	N
7	Y	N	Y	N	Y	Y	Y	Y
8	N	Y	Y	Y	Y	Y	Y	Y
9	N	N	N	Y	N	Y	N	N
10	Y	Y	N	Y	N	Y	Y	N

Abstract

1 Introduction

2 Related Works

2.1 Voice Assistant Use by Older Adults

2.2 Older Adults’ Conversational Interactions with Voice Assistants

3 Methods

3.1 Alexa Program for Long-Term Care

3.2 Participant Recruitment & Demographics

3.3 Interviews

3.4 Qualitative Analysis

4 Findings

4.1 Learning How to Use Alexa

4.2 Incorporating Alexa into Daily Life

4.2.1 Supporting Everyday Routines.

4.2.2 Improving Mood.

4.2.3 Stimulating Cognition Through Learning.

4.2.4 Facilitating Social Activities.

4.3 Perceptions of Alexa’s Conversational Capabilities

4.3.1 Adapting to Alexa’s Limitations.

4.3.2 Alexa as a Conversation Partner.

5 Discussion

5.1 Designing Voice Assistants for Sustained Use

5.1.1 Supporting Mixed-Modality Training (RQ1).

5.1.2 Designing for Meaningful Voice Assistant Use (RQ2).

5.2 Designing VAs as Conversational Partners (RQ3)

5.3 Limitations & Future Work

6 Conclusion

Acknowledgments

Footnote

Supplementary Material

References

Cited By

Index Terms

Recommendations

An Empirical Study of Older Adult’s Voice Assistant Use for Health Information Seeking

Technological acceptance of voice assistants in older adults: an online co-creation experience

Understanding Older Adults' Perceptions of In-Home Sensors Using an Obtrusiveness Framework

Comments

Information

Published In

Sponsors

Publisher

Publication History

Check for updates

Author Tags

Qualifiers

Conference

Acceptance Rates

Upcoming Conference

Contributors

Other Metrics

Bibliometrics

Article Metrics

Other Metrics

Citations

Cited By

View options

PDF

eReader

HTML Format

Login options

Full Access

Figures

Other

Share

Share this Publication link

Share on social media

Affiliations

P ID	Date/Time	Weather	Alarm	Games	Jokes	Music	Facts	Care Plan
1	Y	Y	Y	Y	Y	Y	Y	N
2	Y	Y	N	N	N	N	Y	Y
3	N	Y	Y	Y	Y	Y	Y	Y
4	N	N	Y	N	N	Y	Y	Y
5	Y	Y	N	N	Y	N	Y	N
6	N	Y	N	N	Y	Y	Y	N
7	Y	N	Y	N	Y	Y	Y	Y
8	N	Y	Y	Y	Y	Y	Y	Y
9	N	N	N	Y	N	Y	N	N
10	Y	Y	N	Y	N	Y	Y	N