research-article

Open access

CloudVR: Cloud Accelerated Interactive Mobile Virtual Reality

Authors:

Teemu Kämäräinen,

Matti Siekkinen,

Jukka Eerikäinen,

Antti Ylä-JääskiAuthors Info & Claims

MM '18: Proceedings of the 26th ACM international conference on Multimedia

Pages 1181 - 1189

https://doi.org/10.1145/3240508.3240620

Published: 15 October 2018 Publication History

Abstract

High quality immersive Virtual Reality experience currently requires a PC setup with cable connected head mounted display, which is expensive and restricts user mobility. This paper presents CloudVR which is a system for cloud accelerated interactive mobile VR. It is designed to provide short rotation and interaction latencies through panoramic rendering and dynamic object placement. CloudVR also includes rendering optimizations to reduce server-side computational load and bandwidth requirements between the server and client. Performance measurements with a CloudVR prototype suggest that the optimizations make it possible to double the server's framerate and halve the amount of bandwidth required and that small objects can be quickly moved at run time to client device for rendering to provide shorter interaction latency. A small-scale user study indicates that CloudVR users do not notice small network latencies (20ms) and even much longer ones (100-200ms) become non-trivial to detect when they do not affect the interaction with objects. Finally, we present a design of CloudVR extension to multi-user scenarios.

References

[1]

ISO/IEC 23009. 2012. Dynamic adaptive streaming over HTTP (DASH). (2012).

[2]

Omid Abari, Dinesh Bharadia, Austin Duffield, and Dina Katabi. 2017. Enabling High-quality Untethered Virtual Reality. Proceedings of the 14th USENIX Conference on Networked Systems Design and Implementation (NSDI'17). USENIX Association, Berkeley, CA, USA, 531--544. http://dl.acm.org/citation.cfm?id=3154630.3154674

Digital Library

[3]

Rachel Albert, Anjul Patney, David Luebke, and Joohwan Kim. 2017. Latency Requirements for Foveated Rendering in Virtual Reality. ACM Trans. Appl. Percept., Vol. 14, 4, Article 25 (Sept. 2017), bibinfonumpages13 pages.

Digital Library

[4]

Giovanni Avveduto, Franco Tecchia, Marcello Carrozzino, and Massimo Bergamasco. 2016. A Scalable Cluster-Rendering Architecture for Immersive Virtual Environments. In International Conference on Augmented Reality, Virtual Reality and Computer Graphics . Springer, 102--119.

[5]

Yanan Bao, Huasen Wu, Tianxiao Zhang, Albara Ah Ramli, and Xin Liu. 2016. Shooting a moving target: Motion-prediction-based transmission for 360-degree videos. In Big Data (Big Data), 2016 IEEE International Conference on. IEEE, 1161--1170.

[6]

Kevin Boos, David Chu, and Eduardo Cuervo. 2016. FlashBack: Immersive Virtual Reality on Mobile Devices via Rendering Memoization. In Proceedings of the 14th Annual International Conference on Mobile Systems, Applications, and Services (MobiSys '16). ACM, New York, NY, USA, 291--304.

Digital Library

[7]

Wei Cai, Ryan Shea, Chun-Ying Huang, Kuan-Ta Chen, Jiangchuan Liu, Victor CM Leung, and Cheng-Hsin Hsu. 2016. The future of cloud gaming {point of view}. Proc. IEEE, Vol. 104, 4 (2016), 687--691.

[8]

Sharon Choy, Bernard Wong, Gwendal Simon, and Catherine Rosenberg. 2012. The Brewing Storm in Cloud Gaming: A Measurement Study on Cloud to End-user Latency. In Proceedings of the 11th Annual Workshop on Network and Systems Support for Games (NetGames '12). IEEE Press, Piscataway, NJ, USA, Article 2, bibinfonumpages6 pages. http://dl.acm.org/citation.cfm?id=2501560.2501563

Digital Library

[9]

Xavier Corbillon, Francesca De Simone, and Gwendal Simon. 2017a. 360Degreee Video Head Movement Dataset. In Proceedings of the 8th ACM on Multimedia Systems Conference (MMSys'17). ACM, New York, NY, USA, 199--204.

Digital Library

[10]

Xavier Corbillon, Alisa Devlic, Gwendal Simon, and Jacob Chakareski. 2017b. Optimal Set of 360-Degree Videos for Viewport-Adaptive Streaming. In Proceedings of the 2017 ACM on Multimedia Conference (MM '17). ACM, New York, NY, USA, 943--951.

Digital Library

[11]

Facebook. 2018. 360-Capture-SDK. (2018). https://github.com/facebook/360-Capture-SDK

[12]

PA Howarth and PJ Costello. 1997. The occurrence of virtual simulation sickness symptoms when an HMD was used as a personal viewing system. Displays, Vol. 18, 2 (1997), 107--116.

[13]

Chun-Ying Huang, Kuan-Ta Chen, De-Yu Chen, Hwai-Jung Hsu, and Cheng-Hsin Hsu. 2014. GamingAnywhere: The First Open Source Cloud Gaming System. ACM Trans. Multimedia Comput. Commun. Appl., Vol. 10, 1s, Article 10 (Jan. 2014), bibinfonumpages25 pages.

Digital Library

[14]

Teemu K"am"ar"ainen, Matti Siekkinen, Antti Yl"a-J"a"aski, Wenxiao Zhang, and Pan Hui. 2017. Dissecting the End-to-end Latency of Interactive Mobile Video Applications. In Proceedings of the 18th International Workshop on Mobile Computing Systems and Applications (HotMobile '17). ACM, New York, NY, USA, 61--66.

Digital Library

[15]

FAA Kingdom and N Prins. 2010. Psychophysics: a practical introduction. (2010).

[16]

Zeqi Lai, Y Charlie Hu, Yong Cui, Linhui Sun, and Ningwei Dai. 2017. Furion: Engineering high-quality immersive virtual reality on today's mobile devices. In Proceedings of the 23rd International Conference on Mobile Computing and Networking (MobiCom'17). ACM, Snowbird, Utah, USA .

Digital Library

[17]

Kyungmin Lee, David Chu, Eduardo Cuervo, Johannes Kopf, Yury Degtyarev, Sergey Grizan, Alec Wolman, and Jason Flinn. 2015. Outatime: Using Speculation to Enable Low-Latency Continuous Interaction for Mobile Cloud Gaming. In Proceedings of the 13th Annual International Conference on Mobile Systems, Applications, and Services (MobiSys '15). ACM, New York, NY, USA, 151--165.

Digital Library

[18]

Yeng-Ting Lee, Kuan-Ta Chen, Han-I Su, and Chin-Laung Lei. 2012. Are All Games Equally Cloud-gaming-friendly?: An Electromyographic Approach. In Proceedings of the 11th Annual Workshop on Network and Systems Support for Games (NetGames '12). IEEE Press, Piscataway, NJ, USA, Article 3, bibinfonumpages6 pages. http://dl.acm.org/citation.cfm?id=2501560.2501564

Digital Library

[19]

OculusRift-Blog.com. {n. d.}. John Carmack's Delivers Some Home Truths On Latency . ({n. d.}). http://oculusrift-blog.com/john-carmacks-message-of-latency/.

[20]

Feng Qian, Lusheng Ji, Bo Han, and Vijay Gopalakrishnan. 2016. Optimizing 360 Video Delivery over Cellular Networks. In Proceedings of the 5th Workshop on All Things Cellular: Operations, Applications and Challenges (ATC '16). ACM, New York, NY, USA, 1--6.

Digital Library

[21]

Bernhard Reinert, Johannes Kopf, Tobias Ritschel, Eduardo Cuervo, David Chu, and Hans-Peter Seidel. 2016. Proxy-guided Image-based Rendering for Mobile Devices. In Computer Graphics Forum, Vol. 35. Wiley Online Library, 353--362.

Digital Library

[22]

Chen Tao. 2018. clumsy . (2018). https://jagt.github.io/clumsy/

[23]

Unity. 2017. Unity Documentation. Cluster Rendering. (Nov. 2017). Retrieved Nov. 19, 2017 from https://docs.unity3d.com/Manual/ClusterRendering.html

[24]

Chenglei Wu, Zhihao Tan, Zhi Wang, and Shiqiang Yang. 2017. A Dataset for Exploring User Behaviors in VR Spherical Video Streaming. In Proceedings of the 8th ACM on Multimedia Systems Conference (MMSys'17). ACM, New York, NY, USA, 193--198.

Digital Library

Cited By

Batiuk AKulyk Y(2024)Integration of cloud technologies into virtual realityUkrainian Journal of Information Technology10.23939/ujit2024.01.1096:1(109-119)Online publication date: 2024
https://doi.org/10.23939/ujit2024.01.109
Rossi HMitra KÅhlund CCotanis I(2024)QoE Models for Virtual Reality Cloud-based First Person Shooter Game over Mobile Networks2024 20th International Conference on Network and Service Management (CNSM)10.23919/CNSM62983.2024.10814458(1-5)Online publication date: 28-Oct-2024
https://doi.org/10.23919/CNSM62983.2024.10814458
Wu XZhao KChen XLiang TCai JKankanhalli MPrabhakaran BBoll SSubramanian RZheng LSingh VCesar PXie LXu D(2024)Edge-assisted Real-time Dynamic 3D Point Cloud Rendering for Multi-party Mobile Virtual RealityProceedings of the 32nd ACM International Conference on Multimedia10.1145/3664647.3681650(2824-2832)Online publication date: 28-Oct-2024
https://dl.acm.org/doi/10.1145/3664647.3681650
Show More Cited By

Index Terms

CloudVR: Cloud Accelerated Interactive Mobile Virtual Reality
1. Computer systems organization
  1. Architectures
    1. Distributed architectures
      1. Cloud computing
2. Computing methodologies
  1. Computer graphics
    1. Graphics systems and interfaces
      1. Virtual reality

Recommendations

FlashBack: Immersive Virtual Reality on Mobile Devices via Rendering Memoization
MobiSys '16: Proceedings of the 14th Annual International Conference on Mobile Systems, Applications, and Services

Virtual reality head-mounted displays (VR HMDs) are attracting users with the promise of full sensory immersion in virtual environments. Creating the illusion of immersion for a near-eye display results in very heavy rendering workloads: low latency, ...
Mixed Reality MIDI Keyboard Demonstration
AM '17: Proceedings of the 12th International Audio Mostly Conference on Augmented and Participatory Sound and Music Experiences

The Mixed Reality MIDI Keyboard is a prototype designed to augment virtual reality experiences through the inclusion of a physical interface which aligns the user's senses with the virtual environment. It also serves as a platform on which the uses of ...
INSIDE: Using a Cubic Multisensory Controller for Interaction With a Mixed Reality Environment

The field of electronic games is a great experimental area that contributes to the implementation of new technologies in a variety of applications. This article explores the field of mixed reality through the use of physical computing for the ...

Comments

Information & Contributors

Information

Published In

cover image ACM Conferences

MM '18: Proceedings of the 26th ACM international conference on Multimedia

October 2018

2167 pages

ISBN:9781450356657

DOI:10.1145/3240508

General Chairs:
Susanne Boll
University of Oldenburg, Germany
,
Kyoung Mu Lee
Seoul National University, Korea
,
Jiebo Luo
University of Rochester, USA
,
Wenwu Zhu
Tsinghua University, China
,
Program Chairs:
Hyeran Byun
Yonsei University, Korea
,
Chang Wen Chen
State Univ. Of New York at Buffalo, USA
,
Rainer Lienhart
University of Augsburg, Germany
,
Tao Mei
JD AI, China

Copyright © 2018 Owner/Author.

Permission to make digital or hard copies of part or all of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for third-party components of this work must be honored. For all other uses, contact the Owner/Author.

Sponsors

SIGMM: ACM Special Interest Group on Multimedia

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 15 October 2018

Check for updates

Author Tags

Qualifiers

Research-article

Funding Sources

Nokia Center for Advanced Research
Academy of Finland

Conference

MM '18

Sponsor:

SIGMM

MM '18: ACM Multimedia Conference

October 22 - 26, 2018

Seoul, Republic of Korea

Acceptance Rates

MM '18 Paper Acceptance Rate 209 of 757 submissions, 28%;

Overall Acceptance Rate 2,145 of 8,556 submissions, 25%

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

39
Total Citations
View Citations
1,866
Total Downloads

Downloads (Last 12 months)338
Downloads (Last 6 weeks)35

Reflects downloads up to 19 Feb 2025

Other Metrics

View Author Metrics

Citations

Cited By

Batiuk AKulyk Y(2024)Integration of cloud technologies into virtual realityUkrainian Journal of Information Technology10.23939/ujit2024.01.1096:1(109-119)Online publication date: 2024
https://doi.org/10.23939/ujit2024.01.109
Rossi HMitra KÅhlund CCotanis I(2024)QoE Models for Virtual Reality Cloud-based First Person Shooter Game over Mobile Networks2024 20th International Conference on Network and Service Management (CNSM)10.23919/CNSM62983.2024.10814458(1-5)Online publication date: 28-Oct-2024
https://doi.org/10.23919/CNSM62983.2024.10814458
Wu XZhao KChen XLiang TCai JKankanhalli MPrabhakaran BBoll SSubramanian RZheng LSingh VCesar PXie LXu D(2024)Edge-assisted Real-time Dynamic 3D Point Cloud Rendering for Multi-party Mobile Virtual RealityProceedings of the 32nd ACM International Conference on Multimedia10.1145/3664647.3681650(2824-2832)Online publication date: 28-Oct-2024
https://dl.acm.org/doi/10.1145/3664647.3681650
Liu TLucas JHe SLiu TWang XWang W(2024)Improving Resource and Energy Efficiency for Cloud 3D through Excessive Rendering ReductionProceedings of the Nineteenth European Conference on Computer Systems10.1145/3627703.3650064(317-332)Online publication date: 22-Apr-2024
https://dl.acm.org/doi/10.1145/3627703.3650064
Zhou YPopescu V(2024)CloVR: Fast-Startup Low-Latency Cloud VRIEEE Transactions on Visualization and Computer Graphics10.1109/TVCG.2024.337205930:5(2337-2346)Online publication date: May-2024
https://doi.org/10.1109/TVCG.2024.3372059
Mo SXin M(2024)BSTG-Trans: A Bayesian Spatial-Temporal Graph Transformer for Long-Term Pose ForecastingIEEE Transactions on Multimedia10.1109/TMM.2023.326921926(673-686)Online publication date: 2024
https://doi.org/10.1109/TMM.2023.3269219
Chen YOmoma SKwon HInaltekin HGorlatova M(2024)Quantifying and Exploiting VR Frame Correlations: An Application of a Statistical Model for Viewport PoseIEEE Transactions on Mobile Computing10.1109/TMC.2024.339977023:12(11466-11482)Online publication date: Dec-2024
https://doi.org/10.1109/TMC.2024.3399770
Li LHuang YQiao XMeng YYu DRen PDustdar S(2024)Toward Distributed Collaborative Rendering Service for Immersive Mobile WebIEEE Network10.1109/MNET.133.220052438:3(137-145)Online publication date: May-2024
https://doi.org/10.1109/MNET.133.2200524
Santos J(2024)Towards Cloud-Native Virtual Reality Applications: State-Of-The-Art and Open Challenges2024 IEEE Symposium on Computers and Communications (ISCC)10.1109/ISCC61673.2024.10733608(1-6)Online publication date: 26-Jun-2024
https://doi.org/10.1109/ISCC61673.2024.10733608
Rossi HMitra KLarsson SÅhlund CCotanis I(2024)Subjective QoE Assessment for Virtual Reality Cloud-based First-Person Shooter GameICC 2024 - IEEE International Conference on Communications10.1109/ICC51166.2024.10622467(4698-4703)Online publication date: 9-Jun-2024
https://doi.org/10.1109/ICC51166.2024.10622467
Show More Cited By

View Options

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

Figures

Tables

Media

View Table of Conten