Will Dynamic Foveation Boost Cloud VR Gaming Experience?
Authors: 
Jia-Wei Fang, Kuan-Yu Lee, Teemu Kämäräinen, Matti Siekkinen, Cheng-Hsin HsuAuthors Info & Claims
Jia-Wei Fang, Kuan-Yu Lee, Teemu Kämäräinen, Matti Siekkinen, Cheng-Hsin HsuAuthors Info & ClaimsPages 29 - 35
Abstract
Cloud Virtual Reality (VR) gaming offloads the computationally-intensive rendering tasks from resource-limited Head-Mounted Displays (HMDs) to cloud servers, which consume a staggering amount of bandwidth for high-quality gaming experiences. One way to cope with such high bandwidth demands is to capitalize on human vision systems by allocating a higher bitrate to the foveal region of HMD viewport, which is known as foveation in the literature. Although foveation was employed by remote VR gaming, existing open-source projects all adopt static foveation, in which the HMD gamer gaze position is assumed to be fixed at the viewport center. In this paper, we construct the very first cloud VR gaming system that supports dynamic foveation. That is, the real-time gaze positions of gamers are streamed from eye-trackers on HMDs to cloud servers, which in turn adjust the foveation parameters, such as foveal region size/location and peripheral region quality degradation, accordingly. Using our developed cloud VR gaming system, we design and carry out a user study using a game called Fruit Ninja VR 2 to find the foveation parameters in static and dynamic foveation for maximizing the gaming Quality of Experience (QoE) in Mean Opinion Score (MOS). With the chosen foveation parameters, we found that, compared to cloud VR gaming without foveation, static foveation leads to a MOS increase of 0.60 and a bitrate reduction of 8.71%. Furthermore, adopting dynamic foveation results in an additional 0.60 increase on MOS while saving 9.81% bitrate, compared to static foveation. Our findings demonstrate the potential of dynamic foveation in cloud VR gaming, which dictates both high visual quality and short response time. The optimization techniques developed in this and follow-up work could benefit other cloud-rendered applications that typically have less strict requirements than cloud VR gaming.
References
[1]
2018. Cloud VR network solution white paper. https://reurl.cc/ml3OqV.
[2]
2020. Cloud gaming market size. https://reurl.cc/b7RLRM.
[3]
2020. The official website of NVIDIA CloudXR. https://reurl.cc/10djxm.
[4]
2020. Virtual reality market size, share & trends analysis report by technology. https://reurl.cc/MRdadk.
[5]
2021. The github of ALXR. https://github.com/korejan/ALVR/releases.
[6]
2021. PICO Neo 3 Pro Eye. https://reurl.cc/Nqgpep.
[7]
2022. Meta Quest Pro. https://reurl.cc/6NMEoV.
[8]
2023. OpenXR SDK document. https://reurl.cc/qkk393.
[9]
R. Albert, A. Patney, D. Luebke, and J. Kim. 2017. Latency requirements for foveated rendering in virtual reality. ACM Transactions on Applied Perception 14, 4 (2017), 1--13.
[10]
F. Frieß, M. Braun, V. Bruder, S. Frey, G. Reina, and T. Ertl. 2020. Foveated encoding for large high-resolution displays. IEEE Transactions on Visualization and Computer Graphics 27, 2 (2020), 1850--1859.
[11]
B. Guenter, M. Finch, S. Drucker, D. Tan, and J. Snyder. 2012. Foveated 3D graphics. ACM Transactions on Graphics 31, 6 (2012), 1--10.
[12]
M. Hegazy, K. Diab, M. Saeedi, B. Ivanovic, I. Amer, Y. Liu, G. Sines, and M. Hefeeda. 2019. Content-aware video encoding for cloud gaming. In Proc. of ACM Multimedia Systems Conference (MMSys'19). 60--73.
[13]
L. Hsiao, B. Krajancich, P. Levis, G. Wetzstein, and K. Winstein. 2022. Towards retina-quality VR video streaming: 15ms could save you 80% of your bandwidth. ACM SIGCOMM Computer Communication Review 52, 1 (2022), 10--19.
[14]
C. Hsu, A. Chen, C. Hsu, C. Huang, C. Lei, and K. Chen. 2017. Is foveated rendering perceivable in virtual reality? Exploring the efficiency and consistency of quality assessment methods. In Proc. of ACM International Conference on Multimedia (MM'17). 55--63.
[15]
C. Huang, C. Hsu, Y. Chang, and K. Chen. 2013. GamingAnywhere: An open cloud gaming system. In Proc. of ACM Multimedia Systems Conference (MMSys'13). 36--47.
[16]
G. Illahi, T. Gemert, M. Siekkinen, E. Masala, A. Oulasvirta, and A. Ylä-Jääski. 2020. Cloud gaming with foveated video encoding. ACM Transactions on Multimedia Computing, Communications, and Applications 16, 1 (2020), 1--24.
[17]
G. Illahi, M. Siekkinen,T. Kämäräinen, and A. Ylä-Jääski. 2021. Foveated streaming of real-time graphics. In Proc. of ACM Multimedia Systems Conference (MMSys'21). 214--226.
[18]
T. Installations and L. Line. 1999. Subjective video quality assessment methods for multimedia applications. Networks 910, 37 (1999), 5.
[19]
Y. Jin, M. Chen, T. Goodall, A. Patney, and A. Bovik. 2021. Subjective and objective quality assessment of 2D and 3D foveated video compression in virtual reality. IEEE Transactions on Image Processing 30 (2021), 5905--5919.
[20]
P. Kortum and W. Geisler. 1996. Implementation of a foveated image coding system for image bandwidth reduction. In Proc. of Human Vision and Electronic Imaging, Vol. 2657. 350--360.
[21]
I. Mohammadi, M. Hashemi, and M. Ghanbari. 2015. An object-based framework for cloud gaming using player's visual attention. In Proc. of IEEE International Conference on Multimedia & Expo Workshops (ICMEW'15). 1--6.
[22]
A. Patney, M. Salvi, J. Kim, A. Kaplanyan, C. Wyman, N. Benty, D. Luebke, and A. Lefohn. 2016. Towards foveated rendering for gaze-tracked virtual reality. ACM Transactions on Graphics 35, 6 (2016), 1--12.
[23]
M. Romero-Rondón, L. Sassatelli, F. Precioso, and R. Aparicio-Pardo. 2018. Foveated streaming of virtual reality videos. In Proc. of ACM Multimedia Systems Conference (MMSys'18). 494--497.
[24]
J. Ryoo, K. Yun, D. Samaras, S. Das, and G. Zelinsky. 2016. Design and evaluation of a foveated video streaming service for commodity client devices. In Proc. of ACM Multimedia Systems Conference (MMSys'16). 1--11.
[25]
Sctanf. 2021. The linear and non-linear function of foveated warping method in ALVR. https://reurl.cc/3O5N10.
[26]
S. Shi and C. Hsu. 2015. A survey of interactive remote rendering systems. Comput. Surveys 47, 4 (2015), 1--29.
[27]
F. Tong. 2018. Foundations of vision. Stevens' Handbook of Experimental Psychology and Cognitive Neuroscience 2 (2018), 1--61.
[28]
Valve. 2023. SteamVR. https://www.steamvr.com/en/.
[29]
W. Zou, S. Feng, X. Mao, F. Yang, and Z. Ma. 2021. Enhancing quality of experience for cloud virtual reality gaming: an object-aware video encoding. In Proc. of IEEE International Conference on Multimedia & Expo Workshops (ICMEW'21). 1--6.
Index Terms
- Will Dynamic Foveation Boost Cloud VR Gaming Experience?
Recommendations
Modeling Gamer Quality-of-Experience Using a Real Cloud VR Gaming Testbed
MMVE '23: Proceedings of the 15th International Workshop on Immersive Mixed and Virtual Environment SystemsCloud Virtual Reality (VR) gaming offloads computationally-intensive VR games to resourceful data centers. Ensuring good Quality of Experience (QoE) in cloud VR gaming, however, is inherently challenging as VR gamers demand high visual quality, short ...
Performance Measurements on a Cloud VR Gaming Platform
QoEVMA'20: Proceedings of the 1st Workshop on Quality of Experience (QoE) in Visual Multimedia ApplicationsAs cloud gaming and Virtual Reality (VR) games become popular in the game industry, game developers engage in these fields to boost their sales. Because cloud gaming possesses the merit of lifting computation loads from client devices to servers, it ...
Player Experience in a VR and Non-VR Multiplayer Game
VRIC '18: Proceedings of the Virtual Reality International Conference - Laval VirtualSeveral studies compared the player experience of Virtual Reality (VR) games and non-VR games. However, there is a lack of research on the specific subject of VR and non-VR multiplayer games. Therefore, this work explores how implementing a multiplayer ...
Comments
Information & Contributors
Information
Published In
June 2023
77 pages
ISBN:9798400701849
DOI:10.1145/3592473
- Co-chairs:
- Shervin Shirmohammadi,
- Mohamed Hefeeda,
- Program Co-chairs:
- Roger Zimmermann,
- Carsten Griwodz,
- Mea Wang
Copyright © 2023 ACM.
Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than the author(s) must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected].
Sponsors
In-Cooperation
Publisher
Association for Computing Machinery
New York, NY, United States
Publication History
Published: 07 June 2023
Check for updates
Author Tags
Qualifiers
- Research-article
Conference
NOSSDAV '23
Sponsor:
NOSSDAV '23: 33rd Workshop on Network and Operating System Support for Digital Audio and Video
June 7 - 10, 2023
BC, Vancouver, Canada
Acceptance Rates
Overall Acceptance Rate 118 of 363 submissions, 33%
Contributors
Other Metrics
Bibliometrics & Citations
Bibliometrics
Article Metrics
- 0Total Citations
- 178Total Downloads
- Downloads (Last 12 months)68
- Downloads (Last 6 weeks)2
Reflects downloads up to 13 Jan 2025
Other Metrics
Citations
View Options
Login options
Check if you have access through your login credentials or your institution to get full access on this article.
Sign in