research-article

Open access

A Framework for Adaptive Residual Streaming for Single-Player Cloud Gaming

Authors:

Magda El-ZarkiAuthors Info & Claims

ACM Transactions on Multimedia Computing, Communications, and Applications (TOMM), Volume 15, Issue 2s

Article No.: 66, Pages 1 - 23

https://doi.org/10.1145/3336498

Published: 19 July 2019 Publication History

All formats PDF

Abstract

Applying cloud technology to 3D interactive multimedia applications is a promising way to provide flexible and cost-efficient online high-bandwidth immersive services to a large population of end users. One main reason cloud systems are popular among users is the fact that it relaxes the hardware requirements for high-end interactive visual applications. As most of the computational tasks are done on cloud servers, users no longer need to upgrade their hardware as frequently to keep up with the ever-increasing high-end computing requirements of the latest applications. Moreover, cloud systems make it easier for a user to enjoy applications on different platforms, including mobile devices that are usually not powerful enough to run high-end, memory-intensive services. In short, applying cloud technology to high-end immersive applications has advantages in cost-efficiency and flexibility both for the end users and the service providers. However, there are two main drawbacks to applying cloud technology to 3D interactive multimedia services: (1) high-bandwidth utilization and (2) latency. In this article, we propose a framework that addresses the two problems for single-player cloud gaming by using a combination of collaborative rendering, progressive meshes, and 3D image warping techniques. The experimental results show that the proposed system can reduce the bandwidth usage and improve the visual quality by utilizing local computing power on the client. The results also show that the interaction latency can be reduced somewhat by sacrificing some degree of visual quality in the end system.

References

[1]

Agam Shah. 2016. The PC upgrade cycle slows to every five to six years, Intel’s CEO says. Retrieved on June 30, 2019 from https://www.pcworld.com/article/3078010/hardware/the-pc-upgrade-cycle-slows-to-every-five-to-six-years-intels-ceo-says.html.

[2]

Amazon Web Services. 2018. Amazon EC2 Pricing. Retrieved on June 30, 2019 from https://aws.amazon.com/ec2/pricing/.

[3]

Hamed Ahmadi, Saman Zad Tootaghaj, Mahmoud Reza Hashemi, and Shervin Shirmohammadi. 2014. A game attention model for efficient bit rate allocation in cloud gaming. Multimedia Systems 20, 5 (01 Oct. 2014), 485--501.

Digital Library

[4]

Tom Beigbeder, Rory Coughlan, Corey Lusher, John Plunkett, Emmanuel Agu, and Mark Claypool. 2004. The effects of loss and latency on user performance in unreal tournament 2003®. In Proceedings of the 3rd ACM SIGCOMM Workshop on Network and System Support for Games (NetGames’04). ACM, New York, 144--151.

Digital Library

[5]

W. Cai, H. C. B. Chan, X. Wang, and V. C. M. Leung. 2015. Cognitive resource optimization for the decomposed cloud gaming platform. IEEE Transactions on Circuits and Systems for Video Technology 25, 12 (Dec. 2015), 2038--2051.

Digital Library

[6]

D. Chen and M. El-Zarki. 2017. Impact of information buffering on a flexible cloud gaming system. In Proceedings of the 15th Annual Workshop on Network and Systems Support for Games (NetGames). 1--6.

Digital Library

[7]

De-Yu Chen and Magda El-Zarki. 2018. Improving the quality of 3D immersive interactive cloud based services over unreliable network. In Proceedings of the 10th International Workshop on Immersive Mixed and Virtual Environment Systems (MMVE’18). ACM, New York, 28--33.

Digital Library

[8]

Seong-Ping Chuah and Ngai-Man Cheung. 2014. Layered coding for mobile cloud gaming. In Proceedings of the International Workshop on Massively Multiuser Virtual Environments (MMVE’14). ACM, New York, Article 4, 6 pages.

Digital Library

[9]

M. Claypool, D. Finkel, A. Grant, and M. Solano. 2012. Thin to win? Network performance analysis of the OnLive thin client game system. In Proceedings of the 11th Annual Workshop on Network and Systems Support for Games (NetGames). 1--6.

Digital Library

[10]

Cyril Crassin, David Luebke, Michael Mara, Morgan McGuire, Brent Oster, Peter Shirley, Peter-Pike Sloan, and Chris Wyman. 2015. CloudLight: A system for amortizing indirect lighting in real-time rendering. Journal of Computer Graphics Techniques (JCGT) 4, 4 (15 Oct. 2015), 1--27. http://jcgt.org/published/0004/04/01/.

[11]

Eduardo Cuervo, Alec Wolman, Landon P. Cox, Kiron Lebeck, Ali Razeen, Stefan Saroiu, and Madanlal Musuvathi. 2015. Kahawai: High-quality mobile gaming using GPU offload. In Proceedings of the 13th Annual International Conference on Mobile Systems, Applications, and Services (MobiSys’15). ACM, New York, 121--135.

Digital Library

[12]

Matthias Dick, Oliver Wellnitz, and Lars Wolf. 2005. Analysis of factors affecting players’ performance and perception in multiplayer games. In Proceedings of 4th ACM SIGCOMM Workshop on Network and System Support for Games (NetGames’05). ACM, New York, 1--7.

Digital Library

[13]

Yong Xue Eu, Jermyn Tanu, Justin Jieting Law, Muhammad Hanif B Ghazali, Shuan Siang Tay, Wei Tsang Ooi, and Anand Bhojan. 2016. SuperStreamer: Enabling progressive content streaming in a game engine. In Proceedings of the 24th ACM International Conference on Multimedia (MM’16). ACM, New York, 737--738.

Digital Library

[14]

Michael Garland and Paul S. Heckbert. 1997. Surface simplification using quadric error metrics. In Proceedings of the 24th Annual Conference on Computer Graphics and Interactive Techniques (SIGGRAPH’97). ACM Press/Addison-Wesley Publishing Co., New York, 209--216.

Digital Library

[15]

Mahdi Hemmati, Abbas Javadtalab, Ali Asghar Nazari Shirehjini, Shervin Shirmohammadi, and Tarik Arici. 2013. Game as video: Bit rate reduction through adaptive object encoding. In Proceedings of the 23rd ACM Workshop on Network and Operating Systems Support for Digital Audio and Video (NOSSDAV’13). ACM, New York, 7--12.

Digital Library

[16]

H. Hong, C. Hsu, T. Tsai, C. Huang, K. Chen, and C. Hsu. 2015. Enabling adaptive cloud gaming in an open-source cloud gaming platform. IEEE Transactions on Circuits and Systems for Video Technology 25, 12 (Dec. 2015), 2078--2091.

Digital Library

[17]

Hugues Hoppe. 1996. Progressive meshes. In Proceedings of the 23rd Annual Conference on Computer Graphics and Interactive Techniques (SIGGRAPH’96). ACM, New York, 99--108.

Digital Library

[18]

Hugues Hoppe. 1997. View-dependent refinement of progressive meshes. In Proceedings of the 24th Annual Conference on Computer Graphics and Interactive Techniques (SIGGRAPH’97). ACM Press/Addison-Wesley Publishing Co., New York, 189--198.

Digital Library

[19]

H. Hoppe. 1998. Smooth view-dependent level-of-detail control and its application to terrain rendering. In Proceedings of Visualization ’98 (Cat. No.98CB36276). 35--42.

Digital Library

[20]

Chun-Ying Huang, Kuan-Ta Chen, De-Yu Chen, Hwai-Jung Hsu, and Cheng-Hsin Hsu. 2014. GamingAnywhere: The first open source cloud gaming system. ACM Trans. Multimedia Comput. Commun. Appl. 10, 1s, Article 10 (Jan. 2014), 25 pages.

Digital Library

[21]

Kyungmin Lee, David Chu, Eduardo Cuervo, Johannes Kopf, Yury Degtyarev, Sergey Grizan, Alec Wolman, and Jason Flinn. 2015. Outatime: Using speculation to enable low-latency continuous interaction for mobile cloud gaming. In Proceedings of the 13th Annual International Conference on Mobile Systems, Applications, and Services (MobiSys’15). ACM, New York, 151--165.

Digital Library

[22]

J. Levenberg. 2002. Fast view-dependent level-of-detail rendering using cached geometry. In Proceedings of IEEE Visualization, 2002 (VIS 2002). 259--265.

Digital Library

[23]

Marc Levoy. 1995. Polygon-assisted JPEG and MPEG compression of synthetic images. In Proceedings of the 22nd Annual Conference on Computer Graphics and Interactive Techniques (SIGGRAPH’95). ACM, New York, 21--28.

Digital Library

[24]

Xiaofei Liao, Li Lin, Guang Tan, Hai Jin, Xiaobin Yang, Wei Zhang, Bo Li, Xiaofei Liao, Li Lin, Guang Tan, Hai Jin, Xiaobin Yang, Wei Zhang, and Bo Li. 2016. LiveRender: A cloud gaming system based on compressed graphics streaming. IEEE/ACM Trans. Netw. 24, 4 (Aug. 2016), 2128--2139.

Digital Library

[25]

Y. Liu, S. Dey, and Y. Lu. 2015. Enhancing video encoding for cloud gaming using rendering information. IEEE Transactions on Circuits and Systems for Video Technology 25, 12 (Dec. 2015), 1960--1974.

Digital Library

[26]

Leonard McMillan, Jr. 1997. An Image-based Approach to Three-dimensional Computer Graphics. Ph.D. Dissertation. University of North Carolina at Chapel Hill, Department of Computer Science.

Digital Library

[27]

F. Messaoudi, G. Simon, and A. Ksentini. 2015. Dissecting games engines: The case of Unity3D. In Proceedings of the International Workshop on Network and Systems Support for Games (NetGames). 1--6.

Digital Library

[28]

S. Möller, S. Schmidt, and S. Zadtootaghaj. 2018. New ITU-T standards for gaming QoE evaluation and management. In Proceedings of the 10th International Conference on Quality of Multimedia Experience (QoMEX). 1--6.

[29]

A. K. Moorthy and A. C. Bovik. 2011. Blind image quality assessment: From natural scene statistics to perceptual quality. IEEE Transactions on Image Processing 20, 12 (Dec. 2011), 3350--3364.

Digital Library

[30]

S. Mowlaei, S. Schmidt, S. Zadtootaghaj, and S. Möller. 2018. Know your game: A bottom-up approach for gaming research. In Proceedings of the 10th International Conference on Quality of Multimedia Experience (QoMEX). 1--3.

[31]

X. Nan, X. Guo, Y. Lu, Y. He, L. Guan, S. Li, and B. Guo. 2014. A novel cloud gaming framework using joint video and graphics streaming. In Proceedings of the IEEE International Conference on Multimedia and Expo (ICME). 1--6.

[32]

I. Nave, H. David, A. Shani, Y. Tzruya, A. Laikari, P. Eisert, and P. Fechteler. 2008. Games@large graphics streaming architecture. In Proceedings of the IEEE International Symposium on Consumer Electronics. 1--4.

[33]

Andreas Petlund, Kristian Evensen, Pål Halvorsen, and Carsten Griwodz. 2008. Improving application layer latency for reliable thin-stream game traffic. In Proceedings of the 7th ACM SIGCOMM Workshop on Network and System Support for Games (NetGames’08). ACM, New York, 91--96.

Digital Library

[34]

Peter Quax, Patrick Monsieurs, Wim Lamotte, Danny De Vleeschauwer, and Natalie Degrande. 2004. Objective and subjective evaluation of the influence of small amounts of delay and jitter on a recent first person shooter game. In Proceedings of 3rd ACM SIGCOMM Workshop on Network and System Support for Games (NetGames’04). ACM, New York, 152--156.

Digital Library

[35]

S. Schmidt, S. Möller, and S. Zadtootaghaj. 2018. A comparison of interactive and passive quality assessment for gaming research. In Proceedings of the 10th International Conference on Quality of Multimedia Experience (QoMEX). 1--6.

[36]

Shu Shi, Cheng-Hsin Hsu, Klara Nahrstedt, and Roy Campbell. 2011. Using graphics rendering contexts to enhance the real-time video coding for mobile cloud gaming. In Proceedings of the 19th ACM International Conference on Multimedia (MM’11). ACM, New York, 103--112.

Digital Library

[37]

Shervin Shirmohammadi. 2013. Adaptive streaming in mobile cloud gaming. IEEE COMSOC Multimedia Communications Technical Committee E-Letter (2013), 20--23.

[38]

S. Wang and S. Dey. 2010. Rendering adaptation to address communication and computation constraints in cloud mobile gaming. In Proceedings of the IEEE Global Telecommunications Conference (GLOBECOM 2010). 1--6.

[39]

Z. Wang and Q. Li. 2011. Information content weighting for perceptual image quality assessment. IEEE Transactions on Image Processing 20, 5 (May 2011), 1185--1198.

Digital Library

[40]

Zhou Wang, H. R. Sheikh, and A. C. Bovik. 2002. No-reference perceptual quality assessment of JPEG compressed images. In Proceedings of the International Conference on Image Processing, Vol. 1. I--I.

[41]

Andrew D. Wilson. 2017. Fast lossless depth image compression. In Proceedings of the 2017 ACM International Conference on Interactive Surfaces and Spaces (ISS’17). ACM, New York, 100--105.

Digital Library

[42]

S. Zadtootaghaj, N. Barman, S. Schmidt, M. G. Martini, and S. Möller. 2018. NR-GVQM: A no reference gaming video quality metric. In Proceedings of the IEEE International Symposium on Multimedia (ISM). 131--134.

[43]

S. Zadtootaghaj, S. Schmidt, and S. Möller. 2018. Modeling gaming QoE: Towards the impact of frame rate and bit rate on cloud gaming. In Proceedings of the 10th International Conference on Quality of Multimedia Experience (QoMEX). 1--6.

[44]

Liang Zhang and W. J. Tam. 2005. Stereoscopic image generation based on depth images for 3D TV. IEEE Transactions on Broadcasting 51, 2 (June 2005), 191--199.

Cited By

Yang YJoukovsky BOramas Mogrovejo JTuytelaars TDeligiannis N(2024)SNIPPET: A Framework for Subjective Evaluation of Visual Explanations Applied to DeepFake DetectionACM Transactions on Multimedia Computing, Communications, and Applications10.1145/3665248Online publication date: 22-May-2024
https://doi.org/10.1145/3665248
Chen QHuang TLiu Q(2024)SWRM: Similarity Window Reweighting and Margin for Long-Tailed RecognitionACM Transactions on Multimedia Computing, Communications, and Applications10.1145/364381620:6(1-18)Online publication date: 8-Mar-2024
https://dl.acm.org/doi/10.1145/3643816
Liang RZhang SZhang WZhang GTang J(2024)Nonlocal Hybrid Network for Long-tailed Image ClassificationACM Transactions on Multimedia Computing, Communications, and Applications10.1145/363025620:4(1-22)Online publication date: 11-Jan-2024
https://dl.acm.org/doi/10.1145/3630256
Show More Cited By

Index Terms

A Framework for Adaptive Residual Streaming for Single-Player Cloud Gaming

Recommendations

Towards Self-Adaptive Cloud Collaborations
IC2E '13: Proceedings of the 2013 IEEE International Conference on Cloud Engineering

In multi-tenant collaborative cloud systems, the choice of cloud providers is based on pre-negotiated terms and conditions to help meet Service Level Agreements (SLAs) and streamline resource provisioning. However, these collaborations are not self-...
Designing a Framework for Cloud Service Agreement for Cloud Environments

Cloud Computing has emerged as the prime IT computing model for an on-demand access using a pool of shared resources with least IT support. Cloud computing is starting to replace the legacy office IT infrastructure and helpdesk support system. Corporate ...
Cloud computing service framework for bioinformatics tools
BIBM '15: Proceedings of the 2015 IEEE International Conference on Bioinformatics and Biomedicine (BIBM)

With the rapid growth of biological technology, large amount of biological data can be produced in few days or months. Many of common-used tools become computation-consuming in analyzing such big biological data. Cloud computing has emerged to provide ...

Comments

Information & Contributors

Information

Published In

cover image ACM Transactions on Multimedia Computing, Communications, and Applications

ACM Transactions on Multimedia Computing, Communications, and Applications Volume 15, Issue 2s

Special Section on Cross-Media Analysis for Visual Question Answering, Special Section on Big Data, Machine Learning and AI Technologies for Art and Design and Special Section on MMSys/NOSSDAV 2018

April 2019

381 pages

ISSN:1551-6857

EISSN:1551-6865

DOI:10.1145/3343360

Editor:
Alberto Del Bimbo
University of Firenze, Italy

Issue’s Table of Contents

Copyright © 2019 ACM.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 19 July 2019

Accepted: 01 May 2019

Revised: 01 May 2019

Received: 01 December 2018

Published in TOMM Volume 15, Issue 2s

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Research-article
Research
Refereed

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

18
Total Citations
View Citations
1,454
Total Downloads

Downloads (Last 12 months)266
Downloads (Last 6 weeks)22

Reflects downloads up to 30 Aug 2024

Other Metrics

View Author Metrics

Citations

Cited By

Yang YJoukovsky BOramas Mogrovejo JTuytelaars TDeligiannis N(2024)SNIPPET: A Framework for Subjective Evaluation of Visual Explanations Applied to DeepFake DetectionACM Transactions on Multimedia Computing, Communications, and Applications10.1145/3665248Online publication date: 22-May-2024
https://doi.org/10.1145/3665248
Chen QHuang TLiu Q(2024)SWRM: Similarity Window Reweighting and Margin for Long-Tailed RecognitionACM Transactions on Multimedia Computing, Communications, and Applications10.1145/364381620:6(1-18)Online publication date: 8-Mar-2024
https://dl.acm.org/doi/10.1145/3643816
Liang RZhang SZhang WZhang GTang J(2024)Nonlocal Hybrid Network for Long-tailed Image ClassificationACM Transactions on Multimedia Computing, Communications, and Applications10.1145/363025620:4(1-22)Online publication date: 11-Jan-2024
https://dl.acm.org/doi/10.1145/3630256
Tian GPan LLiu S(2024)An online cost optimization approach for edge resource provisioning in cloud gamingJournal of Network and Computer Applications10.1016/j.jnca.2024.104008232(104008)Online publication date: Dec-2024
https://doi.org/10.1016/j.jnca.2024.104008
Chen ZYang MZhang S(2023)Complementary Coarse-to-Fine Matching for Video Object SegmentationACM Transactions on Multimedia Computing, Communications, and Applications10.1145/359649619:6(1-21)Online publication date: 12-Jul-2023
https://dl.acm.org/doi/10.1145/3596496
Baldovino B(2022)An Overview of the Networking Issues of Cloud GamingJournal of Innovation Information Technology and Application (JINITA)10.35970/jinita.v4i2.15814:2(120-132)Online publication date: 23-Dec-2022
https://doi.org/10.35970/jinita.v4i2.1581
Tan YTan ANge NBhojan AAmirpour HViola IVega M(2022)DHR: Distributed Hybrid Rendering for Metaverse ExperiencesProceedings of the 1st Workshop on Interactive eXtended Reality10.1145/3552483.3556455(51-59)Online publication date: 14-Oct-2022
https://dl.acm.org/doi/10.1145/3552483.3556455
Tan MYuan FYu JWang GGu X(2022)Fine-grained Image Classification via Multi-scale Selective Hierarchical Biquadratic PoolingACM Transactions on Multimedia Computing, Communications, and Applications10.1145/349222118:1s(1-23)Online publication date: 25-Jan-2022
https://dl.acm.org/doi/10.1145/3492221
Alhilal ABraud THan BHui P(2022)Nebula: Reliable Low-latency Video Transmission for Mobile Cloud GamingProceedings of the ACM Web Conference 202210.1145/3485447.3512276(3407-3417)Online publication date: 25-Apr-2022
https://dl.acm.org/doi/10.1145/3485447.3512276
Yan CMeng LLi LZhang JWang ZYin JZhang JSun YZheng B(2022)Age-Invariant Face Recognition by Multi-Feature Fusionand Decomposition with Self-attentionACM Transactions on Multimedia Computing, Communications, and Applications10.1145/347281018:1s(1-18)Online publication date: 25-Jan-2022
https://dl.acm.org/doi/10.1145/3472810
Show More Cited By

View Options

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

HTML Format

View this article in HTML Format.

Get Access

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Article

Media

Figures

Other

Tables

View Issue’s Table of Contents