Open-Sourcing VR2Gather: A Collaborative Social VR System for Adaptive Multi-Party Real Time Communication
Pages 11210 - 11213
Abstract
Social Virtual Reality is envisioned to transform how individuals communicate remotely, offering a sense of immersion and co-presence within a virtual space. Current platforms enabling remote social interactions rely on synthetic user representations. We address this limitation by enabling realistic human representation through volumetric content capture, encoding and transmission. Specifically, we present an extended version of VR2Gather, now a fully open source Unity package, available at https://github.com/cwi-dis/VR2Gather-acmmm-oss. Our platform is a customisable system to transmit volumetric content in a multi-party real-time environment, easy to integrate into existing applications.
Supplemental Material
MP4 File - oss0006-video.mp4
This video introduces the main features of the VR2Gather platform. It introduces the concept of Social VR and highlights the fact that VR2Gather employs volumetric real-time point cloud representations of users. It presents the components of the system that are responsible for point cloud acquisition, registration and compression as well as the components responsible for facilitating communication and data transmission over the Internet. The project was realised by the DIS group at CWI in Amsterdam with the help of open source contributors on Github and financial support from the European Union. The project is available at https://github.com/cwi-dis/vr2gather.
- Download
- 166.27 MB
References
[1]
A. Abilkaiyrkyzy, A. Elhagry, F. Laamarti, and A. Elsaddik. 2023. Metaverse key requirements and platforms survey. IEEE Access (2023).
[2]
S. J. Friston, B. J. Congdon, D. Swapp, L. Izzouzi, K. Brandstätter, D. Archer, O. Olkkonen, F. J. Thiel, and A. Steed. 2021. Ubiq: A system to build flexible social virtual reality experiences. In Proceedings of the 27th ACM symposium on virtual reality software and technology. 1--11.
[3]
S. Gunkel, R. Hindriks, K. M. El Assal, H. M. Stokking, S. Dijkstra-Soudarissanane, F. Haar, and O Niamut. 2021. VRComm: an end-to-end web system for real-time photorealistic social VR communication. In Proceedings of the 12th ACM multimedia systems conference. 65--79.
[4]
Hupont Torres Isabelle, Charisi Vasiliki, De Prato Giuditta, Pogorzelska Katarzyna, Schade Sven, Kotsev Alexander, Sobolewski Maciej, Duch Brown Nestor, Calza Elisa, Dunker Cesare, and Others. 2023. Next Generation Virtual Worlds: Societal, Technological, Economic and Policy Challenges for the EU. Technical Report. Joint Research Centre.
[5]
J. Jansen, S. Subramanyam, R. Bouqueau, G. Cernigliaro, M. : Cabré, F. Pérez, and P. Cesar. 2020. A pipeline for multiparty volumetric video conferencing: transmission of point clouds over low latency DASH. In Proceedings of the 11th ACM Multimedia Systems Conference. 341--344.
[6]
S. Lee, I. Viola, S. Rossi, Z. Guo, I. Reimat, K. Ławicka, A. Striner, and P. Cesar. 2024. Designing and Evaluating a VR Lobby for a Socially Enriching Remote Opera Watching Experience. IEEE Transactions on Visualization and Computer Graphics (2024).
[7]
J. Li, S. Subramanyam, J. Jansen, Y. Mei, I. Reimat, K. Ławicka, and P. Cesar. 2021. Evaluating the user experience of a photorealistic social VR movie. In 2021 IEEE International Symposium on Mixed and Augmented Reality (ISMAR). IEEE, 284--293.
[8]
Rufael Mekuria, Kees Blom, and Pablo Cesar. 2016. Design, implementation, and evaluation of a point cloud codec for tele-immersive video. IEEE Transactions on Circuits and Systems for Video Technology, Vol. 27, 4 (2016), 828--842.
[9]
R. Mekuria, P. Cesar, I. Doumanis, and A. Frisiello. 2015. Objective and subjective quality assessment of geometry compression of reconstructed 3D humans in a 3D virtual room. In Applications of Digital Image Processing XXXVIII, Vol. 9599. SPIE, 537--549.
[10]
Mario Montagud, Jie Li, Gianluca Cernigliaro, Abdallah El Ali, Sergi Fernández, and Pablo Cesar. 2022. Towards socialVR: evaluating a novel technology for watching videos together. Virtual Reality, Vol. 26, 4 (2022), 1593--1613.
[11]
I. Reimat, E. Alexiou, J. Jansen, I. Viola, S. Subramanyam, and P. Cesar. 2021. CWIPC-SXR: Point cloud dynamic human dataset for social XR. In Proceedings of the 12th ACM Multimedia Systems Conference. 300--306.
[12]
I. Reimat, Y. Mei, E. Alexiou, J. Jansen, J. Li, S. Subramanyam, I. Viola, J. Oomen, and P. Cesar. 2022. Mediascape XR: A Cultural Heritage Experience in Social VR. In Proceedings of the 30th ACM International Conference on Multimedia. 6955--6957.
[13]
Thomas Röggla, David A Shamma, Julie R Williamson, Irene Viola, Silvia Rossi, and Pablo Cesar. 2024. A Platform for Collecting User Behaviour Data during Social VR Experiments Using Mozilla Hubs. In Proceedings of the 16th International Workshop on Immersive Mixed and Virtual Environment Systems. 41--44.
[14]
S. Subramanyam, I. Viola, J. Jansen, E. Alexiou, A. Hanjalic, and P. Cesar. 2022. Evaluating the impact of tiled user-adaptive real-time point cloud streaming on VR remote communication. In Proceedings of the 30th ACM International Conference on Multimedia. 3094--3103.
[15]
I. Viola, J. Jansen, S. Subramanyam, I. Reimat, and P. Cesar. 2023. VR2Gather: A collaborative social VR system for adaptive multi-party real-time communication. IEEE MultiMedia (2023).
Index Terms
- Open-Sourcing VR2Gather: A Collaborative Social VR System for Adaptive Multi-Party Real Time Communication
Recommendations
Measuring User Quality of Experience in Social VR systems
AltMM'18: Proceedings of the 3rd International Workshop on Multimedia Alternate RealitiesVirtual Reality (VR) is a computer-generated experience that can simulate physical presence in real or imagined environments [7]. A social VR system is an application that allows multiple users to join a collaborative Virtual Environment (VE), such as a ...
Social VR: A New Medium for Remote Communication and Collaboration
CHI EA '21: Extended Abstracts of the 2021 CHI Conference on Human Factors in Computing SystemsWe are facing increasingly pressure on reducing travel and working remotely. Tools that support effective remote communication and collaboration are much needed. Social Virtual Reality (VR) is an emerging medium, which invites multiple users to join a ...
Impact of Long-Term Use of an Avatar to IVBO in the Social VR
Human Interface and the Management of Information. Information Presentation and VisualizationAbstractThe illusion of virtual body ownership (IVBO) gives us the feeling of perceiving a virtual body as our own body. It affects our behavior and the sense of presence. The recent emergence of lower-cost VR devices and social VR environments have ...
Comments
Information & Contributors
Information
Published In
October 2024
11719 pages
ISBN:9798400706868
DOI:10.1145/3664647
- General Chairs:
- Jianfei Cai,
- Mohan Kankanhalli,
- Balakrishnan Prabhakaran,
- Susanne Boll,
- Program Chairs:
- Ramanathan Subramanian,
- Liang Zheng,
- Vivek K. Singh,
- Pablo Cesar,
- Lexing Xie,
- Dong Xu
Copyright © 2024 Owner/Author.
This work is licensed under a Creative Commons Attribution International 4.0 License.
Sponsors
Publisher
Association for Computing Machinery
New York, NY, United States
Publication History
Published: 28 October 2024
Check for updates
Author Tags
Qualifiers
- Research-article
Funding Sources
- HORIZON EUROPE
Conference
MM '24
Sponsor:
MM '24: The 32nd ACM International Conference on Multimedia
October 28 - November 1, 2024
Melbourne VIC, Australia
Acceptance Rates
MM '24 Paper Acceptance Rate 1,150 of 4,385 submissions, 26%;
Overall Acceptance Rate 2,145 of 8,556 submissions, 25%
Contributors
Other Metrics
Bibliometrics & Citations
Bibliometrics
Article Metrics
- 0Total Citations
- 112Total Downloads
- Downloads (Last 12 months)112
- Downloads (Last 6 weeks)50
Reflects downloads up to 12 Jan 2025
Other Metrics
Citations
View Options
Login options
Check if you have access through your login credentials or your institution to get full access on this article.
Sign in