Influence of Personality and Communication Behavior of a Conversational Agent on User Experience and Social Presence in Augmented Reality

Abstract

A virtual embodiment can benefit conversational agents, but it is unclear how their personalities and non-verbal behavior influence the User Experience and Social Presence in Augmented Reality (AR). We asked 30 users to converse with a virtual assistant who gives recommendations about city activities. The participants interacted with two different personalities: Sammy, a cheerful blue mouse, and Olive, a serious green human-like agent. Each was presented with two body languages - happy/friendly and annoyed/unfriendly. We conclude how agent representation and humor affect User Experience aspects, and that body language is significant in the evaluation and perception of the AR agent.

Human-centered computingHuman computer interaction (HCI)Interaction paradigmsMixed/augmented reality; Human-centered computingInteraction design;

Introduction

In the last decade, smart speech assistants have become more popular, while at the same time Augmented Reality (AR) applications have emerged as a powerful tool to help users in training, work, tourism, healthcare, or marketing. Mobile AR is beneficial for several reasons. First, it can be used on any mobile device, such as a smartphone or tablet, and second, it allows adding virtual objects to the real world, making it possible to use mobile AR applications everywhere, indoors or outdoors. Virtual Assistants (VAs) are digitally generated characters that provide information by voice, recognize the basic language, and can interact with the user by speech. Soon, speech assistants could be developed further, and we could have virtual assistants as part of each pair of AR glasses. To use all the benefits that visual embodiment offers, we should consider the different types of appearance and the body language of the VA. How we look and express ourselves verbally and non-verbally influences how we are perceived in our interactions with other humans daily. We expect that to apply also in the interaction between a user and a virtual assistant. Virtual characters who show emotions may increase the user’s appreciation of a system. Showing emotion is closely related to body language, and the positive effect of polite gestures is already demonstrated in human-human communication [7]. Such positive effects may also hold when communicating with a virtual assistant. This study can help better understand if the character and the non-verbal behavior of the speech assistant influence the User Experience, Social Presence, and overall User Perception in AR.

1 Experiment Design

It was decided to design two conversational agents who give recommendations about activities in the city with two different virtual bodies and speaking voices. Both of them were presented with two Communication Behaviors, expressed with different body gestures. Based on related work [5],[4], which stated that users did not prefer the more anthropomorphic appearance of the agent, it was decided to develop Sammy as a non-human-like agent. An animated blue mouse character was chosen that has a big head and a cheerful facial expression (Fig. 1a, 1b). Sammy has a male voice and gives the impression of a virtual pet, a little helper, or a character from an animated kids’ movie. Olive was created with the motivation to have a colder character with a less friendly personality. Some studies showed that a more anthropomorphic appearance would deliver a better experience to the users [8]. Olive was designed to have a more anthropomorphic appearance, but still not entirely human-like (Fig. 1c, 1d). Non-behavioral cues for warmth and coldness of the agents indicated in the research of Cuddy et al. [1] were considered when designing the two opposite communication behaviors. Also, non-verbal behavior in Embodied Conversational Agents has been shown to be really important for User Evaluation [3]. The Communication Behavior, presented in the A version, was generally designed as more polite and friendlier to the user. Communication Behavior B was designed to be less friendly, and the agents were presented as disengaged and annoyed with the interaction.

(a)

(b)

(c)

(d)

Figure 1: The four versions of the virtual assistant.

The user tests were designed as within-subject tests, which means each user interacted with the four different versions of the virtual assistant. The order in which the four assistants were used was randomized to prevent sequence effects After each condition, the participant was asked to fill out the User Experience Questionnaire plus (UEQ+) [6], the Networked Minds Social Presence Measure for social presence [2], and some questions related to User Perception. At the end was asked ‘Did you think the body language of the virtual assistant matched their appearance and answers?’ with ‘yes’ or ‘no’ answers, which was followed by the question ‘Why/Why not?’, requiring a short answer. The final questionnaire included some demographics, and some final questions.

2 Results

There were 30 participants aged between 19 and 35, from which 14 were female, and 16 were male, with an average age of 23.8 (SD = 2.94). They were students of different local universities, and most were familiar with some possible activities. This study used a two-way repeated measures ANOVA to determine if the different Personality and Communication Behavior influenced User Experience and Social Presence aspects in AR. An overview of all significant effects is given in Table 1.

Table 1: Two-way repeated-measures ANOVA significant results

Parameter	Effect	$df_{n}$	$df_{d}$	$\hphantom{F}F$	$p\hphantom{p.}$	$\eta_{G}^{2}$
Personality	UX_Attractiveness	1	29	18.905	<.001	.395
Personality	UX_Stimulation	1	29	8.719	<.01	.231
Personality	UX_Novelty	1	29	5.078	<.05	.149
Personality	UX_Usefulness	1	29	7.354	<.05	.202
Personality	UX_Response.Behavior	1	29	13.918	<.01	.324
Personality	SP_Co-presence	1	29	6.374	<.05	.180
Personality	SP_Perceived.Emotional.Interdependence	1	29	5.232	<.05	.153
Comm_Behavior	UX_Attractiveness	1	29	13.000	<.001	.310
Comm_Behavior	UX_Dependability	1	29	10.387	<.01	.264
Comm_Behavior	UX_Stimulation	1	29	6.707	<.05	.188
Comm_Behavior	UX_Novelty	1	29	5.208	<.05	.152
Comm_Behavior	UX_Visual.Aesthetics	1	29	7.785	<.01	.212
Comm_Behavior	UX_Response.Behavior	1	29	10.240	<.01	.261
Comm_Behavior	UX_Response.Quality	1	29	7.191	<.05	.199
Comm_Behavior	SP_Perceived.Message.Understanding	1	29	8.835	<.01	.234

Participants rated the assistant named Sammy as more attractive (M = 5.733, SE = 0.155) than Olive (M = 5.033, SE = 0.187). Furthermore, Communication Behavior A (M = 5.829, SE = 0.152) was rated more attractive than Communication Behavior B (M = 4.938, SE = 0.231). In terms of dependability, communication behavior significantly influenced user perceptions. Communication Behavior A (M = 5.504, SE = 0.150) was rated more dependable than Communication Behavior B (M = 4.917, SE = 0.186). Participants found Sammy (M = 5.379, SE = 0.187) more stimulating than Olive (M = 4.963, SE = 0.170). Also, Communication Behavior A (M = 5.392, SE = 0.186) was more stimulating than Communication Behavior B (M = 4.950, SE = 0.184). Participants perceived Sammy (M = 5.442, SE = 0.211) as more novel than Olive (M = 5.125, SE = 0.237). Additionally, Communication Behavior A (M = 5.446, SE = 0.211) was considered more novel than Communication Behavior B (M = 5.121, SE = 0.238). Communication Behavior A (M = 5.592, SE = 0.191) was rated higher in visual aesthetics than Communication Behavior B (M = 5.092, SE = 0.231). Participants considered Sammy (M = 5.483, SE = 0.204) more useful than Olive (M = 5.092, SE = 0.223). Both personality and communication behavior significantly influenced the perceived response behavior of virtual assistants. Sammy (M = 5.113, SE = 0.196) was perceived as having better response behavior compared to Olive (M = 4.454, SE = 0.179). Furthermore, Communication Behavior A (M = 5.117, SE = 0.197) was rated more polite and trustworthy than Communication Behavior B (M = 4.450, SE = 0.195). Communication Behavior A (M = 5.633, SE = 0.204) was rated higher in response quality than Communication Behavior B (M = 5.342, SE = 0.225). When analysing Social Presence, users gave Sammy a higher co-presence score (M = 4.206, SE = 0.116) compared to Olive (M = 4.039, SE = 0.140). Communication Behavior A (M = 3.461, SE = 0.081) received a higher score in perceived message understanding than Communication Behavior B (M = 3.261, SE = 0.064). Sammy (M = 2.931, SE = 0.133) was rated higher than Olive (M = 2.744, SE = 0.129) in perceived emotional interdependence. In terms of user perception, most users (24 out of 30) preferred Sammy over Olive. The users liked Sammy’s funnier and more likable appearance, his ability to make jokes, and his friendliness. Those who preferred Olive mentioned her beauty, sympathy, and confidence. When asked about their favorite version, 56.7% chose Sammy A, 16.7% chose Sammy B, and 13.3% each chose Olive A and Olive B. Olive B was voted the least enjoyable by 70% of users, while Olive A and Sammy B received 13.3% of the votes each. Negative feedback for Olive B focused on her rudeness and unfriendly body language. Sammy A’s body language was seen as consistent with his appearance and answers by 93.3% of users, while Olive A received 70% agreement. Sammy B and Olive B had lower percentages of agreement.

3 Discussion & Conclusion

From the UX results we can argue that both the friendly appearance and the positive body language significantly affect how attractive, stimulating, and novel the virtual assistant is and how appropriate their response is. Non-verbal communication results are important because positive body language alone influences how predictable and appealing the users think the virtual assistant is, and showing that the personality does not have an effect on visual aesthetics, but the Communication Behavior does, and so the friendly, excited body language makes the assistant significantly more appealing to the user. It could be that for the context of CAs in AR for giving recommendations, nonverbal behavior includes a set of more sought-after traits. The results showed that a major effect played the Personality for the extent to which the user’s emotional state is affected by that of the virtual assistant. Sammy, or the more friendly one, scored higher in perceived emotional interdependence. Furthermore, from the two different personalities again, Sammy evoked a significantly higher sense of co-presence. This could also be because a less anthropomorphic character represented him. Those results suggest that people feel more engaged and emotionally affected when talking to a character that seems from the imaginary world. For the overall User Perception, one of the most important conclusions is that the four versions of the virtual assistant were perceived differently, even though the base dialog system and given information were the same. The interaction with the assistants was described with words that people would use for human-to-human conversations. Moreover, although Sammy was chosen as more likable by 80% of the people, some people still related to the more neutral anthropomorphic female assistant, which may be explained by the fact that people are different, and the users are looking for personalities in real life, as well as in the virtual world, that are suitable for them. It could be argued that the Non-verbal Communication Behavior of the assistants played an even bigger role in user evaluation than the Personality.

3.1 Acknowledgements

This work was partly funded by the german Federal Ministry of Education and Research (BMBF) under the Software Campus program (grant number 01IS17052).