Towards Distributed and Interactive Multi-cam and Multi-device VR360 Video Experiences
June 2023
Pages 407 - 410
Abstract
The production and consumption of multimedia content is continuously increasing, and this particularly affects to immersive formats, like VR360 video. Even though significant advances have been witnessed with regard to the processing, delivery and consumption of interactive VR360 video, key challenges and research questions still need to be addressed to efficiently provide interactive multi-camera and multi-user VR360 video services over distributed and heterogeneous environments. This research work aims at providing novel and efficient contributions to overcome existing limitations in this topic. First, it will develop an end-to-end modular web-based VR360 video platform, including the measurement of Quality of Service (QoS) and activity metrics, to be used as a research testbed. Second, it will provide lightweight yet efficient viewport-aware video processing and delivery strategies to dynamically concentrate the video resolution on the user’s viewport, with a single stream and decoding process through the web browser. Third, it will propose innovative encoding, signaling and synchronization solutions to enable an effective support for multi-camera and multi-device VR360 services, in a synchronized manner and with the lowest latency possible. Fourth, it will explore how to effectively provide social viewing scenarios between remote users while watching the same or related VR360 videos, assisted with interactive and guiding techniques.
References
[1]
Fernando Boronat, Mario Montagud, Pau Salvador, and Javier Pastor. 2021. Wersync: A web platform for synchronized social viewing enabling interaction and collaboration. Journal of Network and Computer Applications 175 (2021), 102939. https://doi.org/10.1016/j.jnca.2020.102939
[2]
Federico Chiariotti. 2021. A survey on 360-degree video: Coding, quality of experience and streaming. Computer Communications 177 (2021), 133–155. https://doi.org/10.1016/j.comcom.2021.06.029
[3]
Ching-Ling Fan, Wen-Chih Lo, Yu-Tung Pai, and Cheng-Hsin Hsu. 2019. A Survey on 360° Video Streaming: Acquisition, Transmission, and Display. ACM Comput. Surv. 52, 4, Article 71 (aug 2019), 36 pages. https://doi.org/10.1145/3329119
[4]
Sergi Fernandez, Isaac Fraile, David Gómez, Mario Montagud, and Jose Antonio Nuñez. 2018. ImmersiaTV: an end-to-end toolset to enable customizable and immersive multi-screen TV experiences. In Workshop day on Virtual Reality, co-located with ACM TVX, Seoul, 2018. Association for Computing Machinery (ACM), Recercat.
[5]
Miguel Fernández-Dasi, Mario Montagud, Miguel Ángel Torres-Font, and Miguel Garcia-Pineda. 2021. Plataforma modular para la codificación y distribución interactiva de contenidos VR360 basada en campo de visión. (2021).
[6]
David Gómez, Juan A. Núñez, Isaac Fraile, Mario Montagud, and Sergi Fernández. 2018. TiCMP: A Lightweight and Efficient Tiled Cubemap Projection Strategy for Immersive Videos in Web-Based Players. In Proceedings of the 28th ACM SIGMM Workshop on Network and Operating Systems Support for Digital Audio and Video (Amsterdam, Netherlands) (NOSSDAV ’18). Association for Computing Machinery, New York, NY, USA, 1–6. https://doi.org/10.1145/3210445.3210456
[7]
Xueshi Hou, Sujit Dey, Jianzhong Zhang, and Madhukar Budagavi. 2021. Predictive Adaptive Streaming to Enable Mobile 360-Degree and VR Experiences. IEEE Transactions on Multimedia 23 (2021), 716–731. https://doi.org/10.1109/TMM.2020.2987693
[8]
Dave Kosiur. 2001. Understanding policy-based networking. Vol. 20. John Wiley & Sons.
[9]
Wen-Chih Lo, Chih-Yuan Huang, and Cheng-Hsin Hsu. 2018. Edge-Assisted Rendering of 360° Videos Streamed to Head-Mounted Virtual Reality. In 2018 IEEE International Symposium on Multimedia (ISM). 44–51. https://doi.org/10.1109/ISM.2018.00016
[10]
Mario Montagud, Isaac Fraile, Einar Meyerson, and Sergi Fernández. 2020. Open-source accessibility-enabled V360 player. ACM IMX (2020).
[11]
Mario Montagud, Einar Meyerson, Isaac Fraile, and Sergi Fernández. 2019. Modular Testbed for KPI Monitoring in Omnidirectional Video Streaming Scenarios. In The Eleventh International Conference on Advances in Multimedia (MMEDIA 2019), València (Spain), March 2019.
[12]
Zdravko Paladin, Zarko Luksic, Nexhat Kapidani, Mario Montagud, Miguel Fernández-Dasi, Soundarya Srinidhi, Thomas Wöllert, and George Boustras. 2023. The 5G-supported Unmanned Aerial Vehicles for Emergency Cases Response. 6.
[13]
Sylvia Rothe, Alexander Schmidt, Mario Montagud, Daniel Buschek, and Heinrich Hußmann. 2021. Social viewing in cinematic virtual reality: a design space for social movie applications. Virtual Reality 25, 3 (2021), 613–630. https://doi.org/10.1007/s10055-020-00472-4
[14]
Petr Yurievich Timokhin, Mikhail Vasilievich Mikhaylyuk, and Evgeniy Mikhaylovich Vozhegov. 2020. Efficient methods and algorithms to synthesize 360-degree video based on cubemap projection of virtual environment. Proceedings of the Institute for System Programming of the RAS 32, 4 (2020), 73–88. https://doi.org/10.15514/ISPRAS-2020-32(4)-5
[15]
Mai Xu, Chen Li, Shanyi Zhang, and Patrick Le Callet. 2020. State-of-the-Art in 360° Video/Image Processing: Perception, Assessment and Compression. IEEE Journal of Selected Topics in Signal Processing 14, 1 (2020), 5–26. https://doi.org/10.1109/JSTSP.2020.2966864
[16]
Abid Yaqoob, Ting Bi, and Gabriel-Miro Muntean. 2020. A Survey on Adaptive 360° Video Streaming: Solutions, Challenges and Opportunities. IEEE Communications Surveys & Tutorials 22, 4 (2020), 2801–2838. https://doi.org/10.1109/COMST.2020.3006999
Index Terms
- Towards Distributed and Interactive Multi-cam and Multi-device VR360 Video Experiences
Recommendations
Enabling and Understanding Interactive Social VR360 Video Viewing
IMX '23: Proceedings of the 2023 ACM International Conference on Interactive Media ExperiencesThis paper reports on the research being done towards enabling and understanding interactive social VR360 video viewing scenarios, by exclusively relying on web-based technologies, and using different types of consumption devices. After motivating the ...
Design, development and evaluation of adaptative and interactive solutions for high-quality viewport-aware VR360 video processing and delivery: research proposal
MMSys '22: Proceedings of the 13th ACM Multimedia Systems ConferenceThis paper contains the research proposal of Miguel Fernández Dasí that was presented at the MMSys 2022 doctoral symposium.1 The production and consumption of multimedia content is continuously increasing, and this particularly affects to immersive ...
Viewport-driven DASH media playback for interactive storytelling: a seamless non-linear storyline experience
MMSys '19: Proceedings of the 10th ACM Multimedia Systems ConferenceOver the past few years, the HTTP Adaptive Streaming (HAS) technologies, e.g. the MPEG-DASH standard (DASH), became the predominant form of online video streaming. 360 Virtual Reality (VR) content have recently emerged on video streaming platform as a ...
Comments
Information & Contributors
Information
Published In
June 2023
465 pages
ISBN:9798400700286
DOI:10.1145/3573381
Copyright © 2023 Owner/Author.
Permission to make digital or hard copies of part or all of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for third-party components of this work must be honored. For all other uses, contact the Owner/Author.
Sponsors
Publisher
Association for Computing Machinery
New York, NY, United States
Publication History
Published: 29 August 2023
Check for updates
Author Tags
Qualifiers
- Extended-abstract
- Research
- Refereed limited
Funding Sources
Conference
IMX '23: ACM International Conference on Interactive Media Experiences
June 12 - 15, 2023
Nantes, France
Acceptance Rates
Overall Acceptance Rate 69 of 245 submissions, 28%
Contributors
Other Metrics
Bibliometrics & Citations
Bibliometrics
Article Metrics
- 0Total Citations
- 27Total Downloads
- Downloads (Last 12 months)27
- Downloads (Last 6 weeks)0
Other Metrics
Citations
View Options
Get Access
Login options
Check if you have access through your login credentials or your institution to get full access on this article.
Sign inFull Access
View options
View or Download as a PDF file.
PDFeReader
View online with eReader.
eReaderHTML Format
View this article in HTML Format.
HTML Format