Cooking Game: A Serious Game Using a Social Robot

In recent years a demographic transition has started as a result of the reducing birth rate in developed countries. It is estimated that by 2050, people over 60 will make over 20% of the population [9]. This doubles the proportion in 2015, making it an important topic. In care-giving, this implies at least double the current workload, making it not sustainable in many countries. Part of this work regards activities for cognitive and physical training. Nonetheless, not all elders can get easy access to this activities. Moreover, as the number of older adults increases, technological complexity also increases. Efforts to make new technologies more accessible are being done, but the technological age gap still exists. This is a helping factor in the isolation of older adults from new generations.

Each problem has its own approaches. Regarding the lack of caregivers, some families typically give informal care to the elderly, which can impact the well-being of both parts. On the aspect of loneliness and lack of stimuli, accessibility has been a focus of technological development thanks to the 2020 pandemic. Nonetheless, most solutions are not accessible enough for all elder. Approaches from social robotics may help solve both problems at once. As social robots are designed for natural interaction, they require low technological knowledge and may be easily included in care-giving environments. Our proposal follows this, by exploring a serious game with the social robot Pepper, inspired by Eleonora Zedda et al. [10]. This game is designed with natural interaction for user stimuli and entertainment and by inciting memorization of cooking recipes. As information about interaction in this context is sparse [8], our work takes a step further evaluating the usability of the game with real users. Moreover, to differ our work, we modified the original game structure in order to give a more complete experience.

In section 2 an overview of recent works on social robots and the elderly is shown. Following in 3, the structure and design of the game is explained. Section 4 develops the experimental setting to evaluate use of the application in a realistic context. In 5 the obtained results from the experimentation are shown and commented. And finally in section 6 the conclusions and future work are stated.

The game developed for this work, titled Pepper's Cooking Game, has been designed for the humanoid robot Pepper by SoftBank Robotics [3]. The game structure is divided into three sections: (1) introduction, where the user and Pepper introduce each other and, if the user desires, can short tutorial of the game is given; (2) level selection, where the user can navigate a menu where they can choose the difficulty of the levels to play and later which specific level (recipe) they want to play; and (3) play phase, in which Pepper explains a recipe and then does a questionnaire about it.

During the entire game, since many basic dialogues take place (i.e. "What's your name?"), automatic body movements accompany these and similar lines in (1) and (2). However, to increase user engagement during explanations, some modifications are done to how Pepper communicates, including speech speed and animations. These changes help give Pepper a teacher role at the start of (3).

On the questionnaire part of (3), the user is tested with timed questions. The duration of the questions is lower for harder recipes. On each question, Pepper firstly formulates the question and then the answers one by one, and on screen the answers are shown either with text or with images (see Fig. 1). Afterwards, the timer starts and the user can answer using either their voice or the tablet's touchscreen. Since the user has three attempts for each question, four types of feedback have been defined, as shown in Fig. 2. At the same time a feedback is shown, Pepper performs some animations to convey some emotions to the user (see Fig. 3). On the condition where the user answers correctly, Pepper does a celebration, such as raising one or both arms up and shaking them. In this condition Pepper also formulates a cheering sentence such as "You are doing great, keep it up" or "Perfect, [user name], that was the right answer". If any of the other cases is met, Pepper does a negation animation, as shaking its head while looking down. In this scenarios Pepper uses sentences according to the game state, for example: "Try again" if it was not the last attempt, "Oh no!, you have run out of attempts" if the user failed its last attempt, or "Time up!" if the timer runs out.

Two images showing the user interface of the questions. Different questions are shown on each example. On the left four options appear, each one on a different box. All of them are formulated in text, saying "Peas", "Egg", "Leg of lamb" and "Dough". The right image shows three options, all of them with images. The three options displayed are "Leg of lamb", "Peas", and "Dough".

Four images showing the feedback types. On the top left a green background is set and the answer given by the user is seen. Also, the sentence "Perfect, Joan, that was the right answer" shows on screen. For the top right, an orange background is set, with the sentence "That was not it" showing. Also, the answer given by the user is shown. For both images at the bottom, the background is set to a red colour. On the left, the sentence "That was not the right answer" can be read, and both the correct answer and the users’ answer are shown. On the right image, the sentence "Oh no!, you have run out of time to answer" can be read, and only the correct answer can be read.

After the questionnaire on (3), the results are given by showing on screen the number of right answers and the total number of questions. Depending on the rate of right answers, a different final feedback is given. For a rate lower than 40%, a bronze medal is shown on screen and Pepper says "Good job! Try again to keep improving". If the rate is between 40% and 70%, a silver medal is shown and the sentence used is "Congrats! You have guessed most questions! See you soon!" (see right Figure 3). At last, for over 70% guess rate, a gold medal appears and the sentence "Excellent! You have done an amazing job!" is used. For all these cases Pepper also claps its hands. An extra case is triggered if the user gets a perfect score, where Pepper turns around its vertical axis with a hand up while saying "That was incredible! You got a perfect score!".

Three images showing some animations for Pepper. The left image shows a cheering animation where Pepper raises both arms up. At the same time on its tablet the feedback screen for the right answer can be seen. The middle image shows an animation when the user provides a wrong answer. This animation shows Pepper swinging its arm across its heap while shaking its head. On the tablet the corresponding feedback screen can be seen. The right image shows the celebration animation at the end of the questionnaire when the user has guessed between 40% and 70% of the answers. It shows Pepper clapping its hands while a silver medal is displayed on screen.

An experiment to test the interactive capabilities of the game has been carried out. Its design focuses on evaluating the usability of the game when the user interacts for the first time with the system.

This experiment has been evaluated both qualitatively and quantitatively. The qualitative analysis uses observations and comments from the users along the experiment. The quantitative analysis uses a System Usability Scale (SUS) questionnaire with extra questions on the interactive features of Pepper answered by the users. Due to the reduced sample size, results should be considered with caution.

As the older people's’ demography increases, so does technological complexity. Social robots are proposed for introducing technology in elderly care. In this work, a serious game implementation is selected. To evaluate user acceptance of the implementation, a first-contact experiment has been carried out.

The main result from this work regards the natural interaction features of Pepper. The users showed preference on them over the tablet, as they do not require learning. The use of speech and body language appears to be key for helping the users in feeling comfortable with the system. The help of the tablet for extra information should be noted, but reducing its complexity. This work also shows that users with partial physical impairments can use Pepper.

In future work, we are working on integrating XAI into the system for studying the cognitive benefits of the game. Within the framework of HRI our goal is to analyze interaction and relation with robots in different situations. One of the working experiences at the University of Zaragoza will focus on analyzing aspects of interaction, trust, and empathy in interaction with the robot "Sanbot Elf" in recreational environments. For this, it is intended to work with natural language processing technologies, facial recognition techniques, and empathic dialogues for conducting tourist visits in an indoor venue within the city of Teruel. The visit will incorporate dialogues and interactive recreational activities, both conversational and tactile. Also, due to the different features of the Sanbot and Pepper robots, we aim to analyze if the unique qualities of each robot impact users’ views on how they interact with them.