Effects of Reward Schedule and Avatar Visibility on Joint Agency During VR Collaboration
Pages 4372 - 4382
Abstract
Joint agency, a group-level sense of agency, has been studied as an essential social cognitive element while engaging in collaborative tasks. The joint agency has been actively investigated in diverse contexts (e.g., performance, reward schedules, and predictability), yet the studies were mostly conducted in traditional 2D computer environments. Since virtual reality (VR) is an emerging technology for remote collaboration, we aimed to probe the effects of traditional reward schedule factors along with novel VR features (i.e., avatar visibility) on joint agency during remote collaboration. In this study, we implemented an experiment based on a card-matching game to test the effects of the reward schedule (fair or equal) and the counterpart's avatar hand visibility (absent or present) on the sense of joint agency. The results showed that participants felt a higher sense of joint agency when the reward was distributed equally regardless of the individual performance and when the counterpart's avatar hand was present. Moreover, the effects of reward schedule and avatar hand visibility interacted, with a bigger amount of deficit for the absent avatar hand when the reward was distributed differentially according to performance. Interestingly, the sense of joint agency was strongly correlated to the level of collaborative performance, as well as to perceptions of other social cognitive factors, including cooperativeness, reward fairness, and social presence. These results contribute to the understanding of joint agency perceptions during VR collaboration and provide design guidelines for remote collaborative tasks and environments for users' optimal social experience and performance.
References
[1]
J. S. Adams. “Inequity in social exchange”. in Advances in experimental social psychology, vol. 2, pp. 267–299. Elsevier, 1965. 2.
[2]
B. A. Bettencourt, M. B. Brewer, M. R. Croak, and N. Miller. Cooperation and the reduction of intergroup bias: The role of reward structure and social orientation. Journal of experimental social psychology, 28 (4): pp. 301–319, 1992. 2.
[3]
H.-W. Bierhoff and E. Rohmann. “Justice in performance situations: Compromise between equity and equality”. in Justice and Conflicts, pp. 135–152. Springer, 2011. 2.
[4]
N. K. Bolt and J. D. Loehr. The predictability of a partner's actions modulates the sense of joint agency. Cognition, 161: pp. 60–65, 2017. 2, 9.
[5]
N. K. Bolt, E. M. Poncelet, B. G. Schultz, and J. D. Loehr. Mutual coordination strengthens the sense of joint agency in cooperative joint action. Consciousness and cognition, 46: pp. 173–187, 2016. 2, 4.
[6]
A. W. Cappelen, T. Eichele, K. Hugdahl, K. Specht, E. Ø. Sørensen, and B. Tungodden. Equity theory and fair inequality: A neuroeconomic study. in Proceedings of the National Academy of Sciences, 111 (43): pp. 15368–15372, 2014. 2.
[7]
P. S. Cho, N. Escoffier, Y. Mao, C. Green, and R. C. Davis. Beyond physical entrainment: competitive and cooperative mental stances during identical joint-action tasks differently affect inter-subjective neural synchrony and judgments of agency. Social Neuroscience, 15 (3): pp. 368–379, 2020. 2.
[8]
P. I. Cornelio Martinez, E. Maggioni, K. Hornbæk, M. Obrist, and S. Subramanian. Beyond the libet clock: modality variants for agency measurements. in Proceedings of the 2018 CHI Conference on Human Factors in Computing Systems, pp. 1–14, 2018. 2.
[9]
C. De Vicariis, V. T. Chackochan, and V. Sanguineti. Game theory and partner representation in joint action: toward a computational theory of joint agency. Phenomenology and the Cognitive Sciences, pp. 1–30, 2022. 2, 9.
[10]
M. Deutsch. Equity, equality, and need: What determines which value will be used as the basis of distributive justice? Journal of Social issues, 31 (3): pp. 137–149, 1975. 2, 8.
[11]
J. Dokic. “Affordances and the sense of joint agency”. in Neuropsychology of the sense of agency, pp. 23–43. Springer, 2010. 1.
[12]
J. P. Freiwald, J. Schenke, N. Lehmann-Willenbrock, and F. Steinicke. Effects of avatar appearance and locomotion on co-presence in virtual reality collaborations. in Mensch und Computer 2021, pp. 393–401. 2021. 3.
[13]
R. Frömer, H. Lin, C. Dean Wolf, M. Inzlicht, and A. Shenhav. Expectations of reward and efficacy guide cognitive control allocation. Nature Communications, 12 (1): pp. 1–11, 2021. 9.
[14]
M. Gallotti and C. D. Frith. Social cognition in the we-mode. Trends in cognitive sciences, 17 (4): pp. 160–165, 2013. 2.
[15]
G. Gamelin, A. Chellali, S. Cheikh, A. Ricca, C. Dumas, and S. Otmane. Point-cloud avatars to improve spatial communication in immersive collaborative virtual environments. Personal and Ubiquitous Computing, 25 (3): pp. 467–484, 2021. 2, 3.
[16]
D. A. Grant. The latin square principle in the design and analysis of psychological experiments. Psychological bulletin, 45 (5): p. 427, 1948. 3.
[17]
I. Gumilar, E. Sareen, R. Bell, A. Stone, A. Hayati, J. Mao, A. Barde, A. Gupta, A. Dey, G. Lee et al., A comparative study on inter-brain synchrony in real and virtual environments using hyperscanning. Computers & Graphics, 94: pp. 62–75, 2021. 2.
[18]
P. Haggard. Sense of agency in the human brain. Nature Reviews Neuro-science, 18 (4): pp. 196–207, 2017. 2.
[19]
C. Harms and F. Biocca. Internal consistency and reliability of the networked minds measure of social presence. in Seventh annual international workshop: Presence, vol. 2004. Universidad Politecnica de Valencia Valencia, Spain, 2004. 4.
[20]
P. Heidicker, E. Langbehn, and F. Steinicke. Influence of avatar appearance on presence in social vr. in 2017 IEEE symposium on 3D user interfaces (3DUI), pp. 233–234. IEEE, 2017. 2, 3, 9.
[21]
G. C. Homans. Social behavior: Its elementary forms. 1974. 2.
[22]
I. Hudson and J. Hurter. Avatar types matter: review of avatar literature for performance purposes. in International conference on virtual, augmented and mixed reality, pp. 14–21. Springer, 2016. 2, 3.
[23]
M. A. F. Ismail and S. Shimada. 'robot'hand illusion under delayed visual feedback: Relationship between the senses of ownership and agency. PloS one, 11 (7): p. e0159619, 2016. 2.
[24]
C. Jeunet, L. Albert, F. Argelaguet, and A. Lécuyer. “do you feel in control?”: towards novel approaches to characterise, manipulate and measure the sense of agency in virtual environments. IEEE transactions on visualization and computer graphics, 24 (4): pp. 1486–1495, 2018. 2.
[25]
A. Jing, K. Gupta, J. McDade, G. Lee, and M. Billinghurst. Near-gaze visualisations of empathic communication cues in mixed reality collaboration. in ACM SIGGRAPH 2022 Posters, pp. 1–2. 2022. 2.
[26]
A. Jing, K. May, B. Matthews, G. Lee, and M. Billinghurst. The impact of sharing gaze behaviours in collaborative mixed reality. in Proceedings of the ACM on Human-Computer Interaction, 6(CSCW2): pp. 1–27, 2022. 2.
[27]
D. Kim, S. Kim, J.-E. Shin, B. Yoon, J. Kim, J. Lee, and W. Woo. The effects of spatial configuration on relative translation gain thresholds in redirected walking. Virtual Reality, pp. 1–18, 2022. 2.
[28]
S. Le Bars, A. Devaux, T. Nevidal, V. Chambon, and E. Pacherie. Agents' pivotality and reward fairness modulate sense of agency in cooperative joint action. Cognition, 195: p. 104117, 2020. 2, 3, 4, 9.
[29]
K.-Y. Liu, S.-K. Wong, M. Volonte, E. Ebrahimi, and S. V. Babu. Investigating the effects of leading and following behaviors of virtual humans in collaborative fine motor tasks in virtual reality. In 2022 IEEE Conference on Virtual Reality and 3D User Interfaces (VR), pp. 330–339. IEEE, 2022. 3.
[30]
J. D. Loehr. Shared credit for shared success: Successful joint performance strengthens the sense of joint agency. Consciousness and Cognition, 66: pp. 79–90, 2018. 1, 2, 9.
[31]
J. D. Loehr. The sense of agency in joint action: An integrative review. Psychonomic Bulletin & Review, 29 (4): pp. 1089–1117, 2022. 2.
[32]
J.-L. Lugrin, M. Ertl, P. Krop, R. Klüpfel, S. Stierstorfer, B. Weisz, M. Rück, J. Schmitt, N. Schmidt, and M. E. Latoschik. Any “body” there? avatar visibility effects in a virtual reality game. In 2018 IEEE conference on virtual reality and 3D user interfaces (VR), pp. 17–24. IEEE, 2018. 3.
[33]
C. G. McClintock and S. P. McNeel. Reward and score feedback as determinants of cooperative and competitive game behavior. Journal of Personality and Social Psychology, 4 (6): p. 606, 1966. 2.
[34]
F. V. Muth, R. Wirth, and W. Kunde. Temporal binding past the libet clock: Testing design factors for an auditory timer. Behavior Research Methods, 53 (3): pp. 1322–1341, 2021. 2.
[35]
S. S. Obhi and P. Hall. Sense of agency and intentional binding in joint action. Experimental brain research, 211 (3): pp. 655–662, 2011. 2.
[36]
M. Osumi, S. Nobusako, T. Zama, N. Yokotani, S. Shimada, T. Maeda, and S. Morioka. The relationship and difference between delay detection ability and judgment of sense of agency. PloS one, 14 (7): p. e0219222, 2019. 2.
[37]
E. Pacherie. 14 the phenomenology of joint action: Self-agency versus joint agency. Joint attention: New developments in psychology, philosophy of mind, and social neuroscience, p. 343, 2012. 1, 2.
[38]
T. Piumsomboon, G. A. Lee, J. D. Hart, B. Ens, R. W. Lindeman, B. H. Thomas, and M. Billinghurst. Mini-me: An adaptive avatar for mixed reality remote collaboration. in Proceedings of the 2018 CHI conference on human factors in computing systems, pp. 1–13, 2018. 2.
[39]
N. Sebanz, H. Bekkering, and G. Knoblich. Joint action: bodies and minds moving together. Trends in cognitive sciences, 10 (2): pp. 70–76, 2006. 1.
[40]
A. Seemann. Joint agency: intersubjectivity, sense of control, and the feeling of trust. Inquiry, 52 (5): pp. 500–515, 2009. 1.
[41]
M. Shiraishi and S. Shimada. Inter-brain synchronization during a cooperative task reflects the sense of joint agency. Neuropsychologia, 154: p. 107770, 2021. 2.
[42]
C. A. Silver, B. W. Tatler, R. Chakravarthi, and B. Timmermans. Social agency as a continuum. Psychonomic Bulletin & Review, 28 (2): pp. 434–453, 2021. 2, 8.
[43]
H. J. Smith and M. Neff. Communication behavior in embodied virtual reality. in Proceedings of the 2018 CHI conference on human factors in computing systems, pp. 1–12, 2018. 3.
[44]
C. Vesper, E. Abramova, J. Bütepage, F. Ciardo, B. Crossey, A. Effenberg, D. Hristova, A. Karlinsky, L. McEllin, S. R. Nijssen et al., Joint action: mental representations, shared information and general mechanisms for coordinating with others. Frontiers in psychology, p. 2039, 2017. 9.
[45]
P. Wang, X. Bai, M. Billinghurst, S. Zhang, D. Han, M. Sun, Z. Wang, H. Lv, and S. Han. Haptic feedback helps me? a vr-sar remote collaborative system with tangible interaction. International Journal of Human–Computer Interaction, 36 (13): pp. 1242–1257, 2020. 2.
[46]
P. Wang, X. Bai, M. Billinghurst, S. Zhang, X. Zhang, S. Wang, W. He, Y. Yan, and H. Ji. Ar/mr remote collaboration on physical tasks: A review. Robotics and Computer-Integrated Manufacturing, 72: p. 102071, 2021. 2.
[47]
L. Zaadnoordijk, M. Meyer, M. Zaharieva, F. Kemalasari, S. van Pelt, and S. Hunnius. From movement to action: An eeg study into the emerging sense of agency in early infancy. Developmental Cognitive Neuroscience, 42: p. 100760, 2020. 2.
[48]
T. Zama, Y. Takahashi, and S. Shimada. Simultaneous eeg-nirs measurement of the inferior parietal lobule during a reaching task with delayed visual feedback. Frontiers in human neuroscience, 13: p. 301, 2019. 2.
[49]
C. Zhou, X. Cheng, C. Liu, and P. Li. Interpersonal coordination enhances brain-to-brain synchronization and influences responsibility attribution and reward allocation in social cooperation. NeuroImage, 252: p. 119028, 2022. 2.
[50]
Z. Zhou, J. Christensen, J. A. Cummings, and J. D. Loehr. Not just in sync: Relations between partners' actions influence the sense of joint agency during joint action. Consciousness and Cognition, 111: p. 103521, 2023. 2.
Recommendations
BeHere: a VR/SAR remote collaboration system based on virtual replicas sharing gesture and avatar in a procedural task
AbstractIn this paper, we focus on the significance of remote collaboration using virtual replicas, avatar, and gesture on a procedural task in industry; thus, we present a Virtual Reality (VR)/Spatial Augmented Reality (SAR) remote collaboration system, ...
Effects of Avatar Transparency on Social Presence in Task-Centric Mixed Reality Remote Collaboration
Despite the importance of avatar representation on user experience for Mixed Reality (MR) remote collaboration involving various device environments and large amounts of task-related information, studies on how controlling visual parameters for avatars ...
ShareYourReality: Investigating Haptic Feedback and Agency in Virtual Avatar Co-embodiment
CHI '24: Proceedings of the 2024 CHI Conference on Human Factors in Computing SystemsVirtual co-embodiment enables two users to share a single avatar in Virtual Reality (VR). During such experiences, the illusion of shared motion control can break during joint-action activities, highlighting the need for position-aware feedback ...
Comments
Information & Contributors
Information
Published In
1077-2626 © 2023 IEEE. Personal use is permitted, but republication/redistribution requires IEEE permission. See https://www.ieee.org/publications/rights/index.html for more information.
Publisher
IEEE Educational Activities Department
United States
Publication History
Published: 03 October 2023
Qualifiers
- Research-article
Contributors
Other Metrics
Bibliometrics & Citations
Bibliometrics
Article Metrics
- 0Total Citations
- 0Total Downloads
- Downloads (Last 12 months)0
- Downloads (Last 6 weeks)0
Reflects downloads up to 09 Nov 2024
Other Metrics
Citations
View Options
View options
Get Access
Login options
Check if you have access through your login credentials or your institution to get full access on this article.
Sign in