Social Cognition in Human-Robot Interaction: Putting the ‘H’ back in ‘HRI’

by Elef Schellen, Jairo Pérez-Osorio and Agnieszka Wykowska (Istituto Italiano di Tecnologia)

The Social Cognition in Human-Robot Interaction (S4HRI) research line at the Istituto Italiano di Tecnologia (IIT) applies methods from experimental psychology and cognitive neuroscience to human-robot interaction studies. With this approach, we maintain excellent experimental control, without losing ecological validity and generalisability, and thus we can provide reliable results informing about robot design that best evokes mechanisms of social cognition in the human interaction partner.
A major goal in the field of Human-Robot Interaction (HRI) is determining the factors required for social attunement between a human and a robot agent. When socially attuned with others, humans employ specialised cognitive mechanisms leading to effective communication and cooperation. Eliciting attunement in interaction with artificial agents (robots, in this case) will therefore allow these mechanisms to be brought to bear on HRI, improving cooperation between humans and robots. 

At present, HRI research gives more weight to external validity; introducing a robotic system into its intended environment (or a close approximation thereof), and relying on questionnaires and task efficiency as outcome variables. This approach is valuable in that it provides practical insights that aid the further development of a system; though, it seldom sheds light on the underlying neuro-
cognitive mechanisms of the human during interaction with a robot [1].

The S4HRI research line emphasises the fundamental neuro-cognitive mechanisms (often implicit or automatic) that humans use to ensure efficient social interaction. These mechanisms include, for example, joint attention, where two agents attend to the same object or event in environment, or spatial perspective taking, where one represents space from the point of view of their interaction partner. The presence of mechanisms such as these contribute to a generalised attitude or stance that people take on when interacting with (artificial) agents. People can take on a stance toward another agent in which they assume that this agent has certain intentions, and use this to predict the agent’s actions. This is known as the intentional stance [2], and is generally how humans interact with each other. We are interested in finding out whether taking on this attitude towards a robot will make interacting with it more natural and efficient.  

Figure 1. An example experimental setup where a participant is involved in an interaction with the humanoid robot iCub [3], while neuro-cognitive mechanisms are measured through performance data, as well as eye tracking and EEG.

One current line of research that is illustrative of our goals and methods examines predictive processing of robot’s eye movements in the context of action expectations. For this series of studies, we designed an interactive protocol with the iCub [3] robot (Figure 1) by modifying an established experimental paradigm in cognitive psychology [4]. Similarly to previous findings [4], we observed that people engaged in joint attention with iCub by following its gaze. However, the level of engagement depended on whether iCub’s behaviour confirmed participants’ expectations or violated them (i.e., iCub either looked at an object that it was expected to manipulate, or looked at an unexpected object). This example study serves as a proof of concept that well-established paradigms in experimental psychology can be implemented in more naturalistic HRI scenarios, while still maintaining excellent experimental control. In follow-up studies we will examine how violation of expectations, as well as other subtle behavioural characteristics of the robot, influence the adoption of intentional stance towards iCub.

S4HRI’s research goes beyond just focusing on the behavioural parameters involved in HRI. With the aid of tools like electroencephalography (EEG) and eye-tracking, we are further able to investigate neuro-cognitive processes during HRI. We are currently using EEG to investigate the neural mechanisms underlying adoption of intentional stance as well as the engagement in joint attention with iCub. The methodology employed in the S4HRI research line emphasises high internal validity, creating highly controlled experiments designed to prevent from potential confounding factors that could otherwise influence the results. By using this approach, we can target and isolate very specific mechanisms that are relatively foundational in their role in social cognition, and that have a ripple effect for more complex social interaction. Consequently, fundamental findings resulting from our research can be used to inform practical design of future robotic systems.

To conclude, we highlight that despite the rapidly changing nature of the field, and the constant pushing the envelope in terms of technology, studying fundamental mechanisms of human information processing remains essential. We aim to provide insights into the fundamentals of social cognition as it pertains to robots, and with these results provide designers of future robotic systems with a solid understanding of the mechanisms of human information processing. This is applicable not only in social robotics but also in current industrial applications, with the rise of industrial robots that use social cues to communicate efficiently and intuitively with users.

The S4HRI research line [L1] is partially funded by the ERC starting grant “Intentional Stance for Social Attunement” (InStance), Grant Agreement No. ERC-2016-STG-715058, awarded to Agnieszka Wykowska [L2], and hosted at the Istituto Italiano di Tecnologia (IIT, Genova, Italy) [L3]. In order to meet the challenges of the interdisciplinary aims of the research line, we are collaborating closely with the iCub facility at IIT [L4].

Links:
[L1] https://kwz.me/htp
[L2] https://instanceproject.eu
[L3] https://www.iit.it
[L4] https://www.iit.it/research/lines/icub

References:
[1] Kachouie et al.: “Socially Assistive Robots in Elderly Care: A Mixed-Method Systematic Literature Review”, International Journal of Human–Computer Interaction, 2014
[2] D.C. Dennett: “True believers: the intentional strategy and why it works”, in: O’Connor, T., Robb, D. (Eds.) Philosophy of Mind: Contemporary Readings (Routledge, London) pp. 370-390, 2003.
[3] Metta et al.: “The iCub humanoid robot: An open-systems platform for research in cognitive development”, Neural Networks, 2010.
[4] Perez‐Osorio et al.: “Expectations regarding action sequences modulate electrophysiological correlates of the gaze‐cueing effect”, Psychophysiology, 2017.

Please contact:
Agnieszka Wykowska
Istituto Italiano di Tecnologia
+39 010 8172 242
This email address is being protected from spambots. You need JavaScript enabled to view it.
https://www.iit.it/people/agnieszka-wykowska