Design, development and evaluation of adaptative and interactive solutions for high-quality viewport-aware VR360 video processing and delivery: research proposal

This paper contains the research proposal of Miguel Fernández Dasí that was presented at the MMSys 2022 doctoral symposium.¹ The production and consumption of multimedia content is continuously increasing, and this particularly affects to immersive formats, like VR360 video. Even though significant advances have been witnessed with regard to the processing, delivery and consumption of interactive VR360 video, key challenges and research questions still need to be addressed to efficiently provide multi-camera and multi-user VR360 video services over distributed and heterogeneous environments. This research work aims at providing novel and efficient contributions to overcome existing limitations in this topic. First, it will develop an end-to-end modular VR360 video platform, including the measurement of Quality of Service (QoS) and activity metrics, to be used as a research testbed. Second, it will provide web-compliant viewport-aware video processing and delivery strategies to dynamically concentrate the video resolution on the users' viewport, with a single stream and decoding process through the web browser. Third, it will propose innovative encoding, signaling and synchronization solutions to enable an effective support for multi-camera VR360 services, with personalized, fast and in-sync switching features. Fourth, it will explore how to effectively provide social viewing scenarios between remote users while watching the same or related VR360 videos. The work also plans to contribute with low-latency delivery pipelines as well as with tools and algorithms to assess and model the Quality of Experience (QoE).