research-article

Dissecting the End-to-end Latency of Interactive Mobile Video Applications

Authors:

Teemu Kämäräinen,

Matti Siekkinen,

Antti Ylä-Jääski,

Pan HuiAuthors Info & Claims

HotMobile '17: Proceedings of the 18th International Workshop on Mobile Computing Systems and Applications

Pages 61 - 66

https://doi.org/10.1145/3032970.3032985

Published: 21 February 2017 Publication History

Abstract

In this paper we measure the step-wise latency in the pipeline of three kinds of interactive mobile video applications that are rapidly gaining popularity, namely Remote Graphics Rendering (RGR), Mobile Augmented Reality (MAR), and Mobile Virtual Reality (MVR). The applications differ from each other by the way in which the user interacts with the application, i.e., video I/O and user controls, however in all the user experience is highly sensitive to end-to-end latency. Long latency between a user control event and display update renders the application unusable. Hence, understanding the nature and origins of latency of these applications is of paramount importance. We show through extensive measurements that control input and display buffering have a substantial effect on the overall delay. Our results shed light on the latency bottlenecks and the maturity of technology for seamless user experience with these applications.

References

[1]

M. Abrash. What VR Could, Should, and almost certainly will be within two years. http://media.steampowered.com/apps/abrashblog/Abrash\%20Dev\%20Days\%202014.pdf.

[2]

R. S. Allison, L. R. Harris, M. Jenkin, U. Jasiobedzka, and J. E. Zacher. Tolerance of temporal delay in virtual environments. In Proceedings of the Virtual Reality 2001 Conference (VR'01), VR '01, pages 247-, Washington, DC, USA, 2001. IEEE Computer Society.

Digital Library

[3]

J. Beyer, R. Varbelow, J.-N. Antons, and S. Zander. A method for feedback delay measurement using a low-cost arduino microcontroller. In Proceedings of the 7th International Workshop on Quality of Multimedia Experience (QoMEX), IEEE, 2015.

[4]

K. Boos, D. Chu, and E. Cuervo. Flashback: Immersive virtual reality on mobile devices via rendering memoization. In Proceedings of the 14th Annual International Conference on Mobile Systems, Applications, and Services, MobiSys '16, pages 291--304, New York, NY, USA, 2016. ACM.

Digital Library

[5]

E. Cattan, A. Rochet-Capellan, and F. Bérard. A predictive approach for an end-to-end touch-latency measurement. In Proceedings of the 2015 International Conference on Interactive Tabletops & Surfaces, ITS '15, pages 215--218, New York, NY, USA, 2015. ACM.

Digital Library

[6]

C.-M. Chang, C.-H. Hsu, C.-F. Hsu, and K.-T. Chen. Performance measurements of virtual reality systems: Quantifying the timing and positioning accuracy. In Proceedings of the 2016 ACM on Multimedia Conference, MM '16, pages 655--659. ACM, 2016.

Digital Library

[7]

K.-T. Chen, Y.-C. Chang, H.-J. Hsu, D.-Y. Chen, C.-Y. Huang, and C.-H. Hsu. On the quality of service of cloud gaming systems. Trans. Multi., 16(2):480--495, Feb. 2014.

Digital Library

[8]

S. Choy, B. Wong, G. Simon, and C. Rosenberg. A hybrid edge-cloud architecture for reducing on-demand gaming latency. Multimedia Systems, 20(5):503--519, 2014.

Digital Library

[9]

M. Claypool and D. Finkel. The effects of latency on player performance in cloud-based games. In 2014 13th Annual Workshop on Network and Systems Support for Games, pages 1--6. IEEE, 2014.

Digital Library

[10]

S. Clinch, J. Harkes, A. Friday, N. Davies, and M. Satyanarayanan. How close is close enough? understanding the role of cloudlets in supporting display appropriation by mobile users. In PerCom, 2012 IEEE International Conference on, pages 122--127, March 2012.

[11]

J. Deber, B. Araujo, R. Jota, C. Forlines, D. Leigh, S. Sanders, and D. Wigdor. Hammer time!: A low-cost, high precision, high accuracy tool to measure the latency of touchscreen devices. In Proceedings of the 2016 CHI Conference on Human Factors in Computing Systems, pages 2857--2868. ACM, 2016.

Digital Library

[12]

J. Deber, R. Jota, C. Forlines, and D. Wigdor. How much faster is fast enough?: User perception of latency & latency improvements in direct and indirect touch. In Proceedings of the 33rd Annual ACM Conference on Human Factors in Computing Systems, pages 1827--1836. ACM, 2015.

Digital Library

[13]

C.-Y. Huang, C.-H. Hsu, Y.-C. Chang, and K.-T. Chen. GamingAnywhere: an open cloud gaming system. In Proceedings of the 4th ACM multimedia systems conference, pages 36--47. ACM, 2013.

Digital Library

[14]

Z. Ivkovic, I. Stavness, C. Gutwin, and S. Sutcliffe. Quantifying and mitigating the negative effects of local latencies on aiming in 3d shooter games. In Proceedings of the 33rd Annual ACM Conference on Human Factors in Computing Systems, CHI '15, pages 135--144, New York, NY, USA, 2015. ACM.

Digital Library

[15]

P. Jain, J. Manweiler, and R. Roy Choudhury. Overlay: Practical mobile augmented reality. In Proc. of the 13th Annual International Conference on Mobile Systems, Applications, and Services, MobiSys '15, pages 331--344, New York, NY, USA, 2015. ACM.

Digital Library

[16]

M. Jarschel, D. Schlosser, S. Scheuring, and T. Hoßfeld. An evaluation of qoe in cloud gaming based on subjective tests. In Innovative Mobile and Internet Services in Ubiquitous Computing (IMIS), 2011 Fifth International Conference on, pages 330--335. IEEE, 2011.

Digital Library

[17]

J. J. Jerald. Scene-motion-and latency-perception thresholds for head-mounted displays. PhD thesis, University of North Carolina at Chapel Hill, 2010.

[18]

M. Koudritsky. Github. Walt Latency Timer. https://github.com/google/walt.

[19]

B. Lee and A. Oulasvirta. Modelling error rates in temporal pointing. In Proc. of the 2016 CHI Conference on Human Factors in Computing Systems, CHI '16, pages 1857--1868. ACM, 2016.

Digital Library

[20]

K. Lee, D. Chu, E. Cuervo, J. Kopf, Y. Degtyarev, S. Grizan, A. Wolman, and J. Flinn. Outatime: Using speculation to enable low-latency continuous interaction for mobile cloud gaming. In Proceedings of the 13th Annual International Conference on Mobile Systems, Applications, and Services, MobiSys '15, pages 151--165, New York, NY, USA, 2015. ACM.

Digital Library

[21]

Y.-T. Lee, K.-T. Chen, H.-I. Su, and C.-L. Lei. Are all games equally cloud-gaming-friendly? an electromyographic approach. In Network and Systems Support for Games (NetGames), 2012 11th Annual Workshop on, pages 1--6. IEEE, 2012.

Digital Library

[22]

OculusRift-Blog.com. John Carmack's Delivers Some Home Truths On Latency. http://oculusrift-blog.com/john-carmacks-message-of-latency/682/.

[23]

K. Raaen, R. Eg, and C. Griwodz. Can gamers detect cloud delay? In Proceedings of the 13th Annual Workshop on Network and Systems Support for Games, page 6. IEEE Press, 2014.

Digital Library

[24]

M. Satyanarayanan, P. Bahl, R. Caceres, and N. Davies. The case for vm-based cloudlets in mobile computing. IEEE pervasive Computing, 8(4):14--23, 2009.

Digital Library

Cited By

Schmid AHalbhuber DFischer TWimmer RHenze N(2023)Small Latency Variations Do Not Affect Player Performance in First-Person ShootersProceedings of the ACM on Human-Computer Interaction10.1145/36110277:CHI PLAY(197-216)Online publication date: 29-Sep-2023
https://dl.acm.org/doi/10.1145/3611027
Schmid AWimmer R(2023)Measuring the Latency of Graphics Frameworks on X11-Based SystemsExtended Abstracts of the 2023 CHI Conference on Human Factors in Computing Systems10.1145/3544549.3585779(1-7)Online publication date: 19-Apr-2023
https://dl.acm.org/doi/10.1145/3544549.3585779
Liu XZhang YYan ZGe Y(2023)Defining ‘seamlessly connected’International Journal of Human-Computer Studies10.1016/j.ijhcs.2023.103068177:COnline publication date: 26-Jul-2023
https://dl.acm.org/doi/10.1016/j.ijhcs.2023.103068
Show More Cited By

Recommendations

Affordable system for measuring motion-to-photon latency of virtual reality in mobile devices
SA '17: SIGGRAPH Asia 2017 Posters

Recently, virtual reality on the mobile phone is getting more and more popular. In virtual reality, motion-to-photon latency is a very important element which causes simulator sickness. In this paper, we construct a low-cost system with high accuracy ...
Building and Employing Cross-Reality

This issue's Works in Progress department lists five projects with a focus on cross reality. The first two involve using Project Wonderland for collaborative projects. The next three use virtual and augmented reality to enhance user capabilities and ...
Welcome to the Jungle: Using ARTV to Balance Media Immersion and Reality Awareness for Passengers on Public Transport
IMX '24: Proceedings of the 2024 ACM International Conference on Interactive Media Experiences

Head-mounted displays (HMDs) offer new possibilities for reclaiming and improving passenger travel time. Common Virtual Reality (VR) HMDs provide an immersive mobile media viewing experience, overcoming the screen size constraints of mobile and seat-...

Comments

Information & Contributors

Information

Published In

cover image ACM Conferences

HotMobile '17: Proceedings of the 18th International Workshop on Mobile Computing Systems and Applications

February 2017

116 pages

ISBN:9781450349079

DOI:10.1145/3032970

Copyright © 2017 ACM.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

Sponsors

SIGMOBILE: ACM Special Interest Group on Mobility of Systems, Users, Data and Computing

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 21 February 2017

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Research-article
Research
Refereed limited

Funding Sources

Suomen Akatemia
Nokia Center for Advanced Research

Conference

HotMobile '17

Sponsor:

SIGMOBILE

HotMobile '17: The 18th International Workshop on Mobile Computing Systems and Applications

February 21 - 22, 2017

CA, Sonoma, USA

Acceptance Rates

Overall Acceptance Rate 96 of 345 submissions, 28%

Upcoming Conference

HOTMOBILE '25

Sponsor:
sigmobile

The 26th International Workshop on Mobile Computing Systems and Applications

February 26 - 27, 2025

La Quinta , CA , USA

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

14
Total Citations
View Citations
628
Total Downloads

Downloads (Last 12 months)32
Downloads (Last 6 weeks)3

Reflects downloads up to 16 Oct 2024

Other Metrics

View Author Metrics

Citations

Cited By

Schmid AHalbhuber DFischer TWimmer RHenze N(2023)Small Latency Variations Do Not Affect Player Performance in First-Person ShootersProceedings of the ACM on Human-Computer Interaction10.1145/36110277:CHI PLAY(197-216)Online publication date: 29-Sep-2023
https://dl.acm.org/doi/10.1145/3611027
Schmid AWimmer R(2023)Measuring the Latency of Graphics Frameworks on X11-Based SystemsExtended Abstracts of the 2023 CHI Conference on Human Factors in Computing Systems10.1145/3544549.3585779(1-7)Online publication date: 19-Apr-2023
https://dl.acm.org/doi/10.1145/3544549.3585779
Liu XZhang YYan ZGe Y(2023)Defining ‘seamlessly connected’International Journal of Human-Computer Studies10.1016/j.ijhcs.2023.103068177:COnline publication date: 26-Jul-2023
https://dl.acm.org/doi/10.1016/j.ijhcs.2023.103068
Choi JJeong SKo J(2022)Emulating Your eXtended World: An Emulation Environment for XR App Development2022 IEEE 19th International Conference on Mobile Ad Hoc and Smart Systems (MASS)10.1109/MASS56207.2022.00025(131-139)Online publication date: Oct-2022
https://doi.org/10.1109/MASS56207.2022.00025
Stauffert JNiebling FLatoschik M(2020)Latency and Cybersickness: Impact, Causes, and Measures. A ReviewFrontiers in Virtual Reality10.3389/frvir.2020.5822041Online publication date: 26-Nov-2020
https://doi.org/10.3389/frvir.2020.582204
Choi JKo JSong JKim MLane NBalan R(2019)RemoteGL - Towards Low-latency Interactive Cloud Graphics Experience for Mobile Devices (demo)Proceedings of the 17th Annual International Conference on Mobile Systems, Applications, and Services10.1145/3307334.3328587(693-694)Online publication date: 12-Jun-2019
https://dl.acm.org/doi/10.1145/3307334.3328587
Li YGao WEskicioglu RMottola LPriyantha B(2019)DeltaVRProceedings of the 18th International Conference on Information Processing in Sensor Networks10.1145/3302506.3310385(13-24)Online publication date: 16-Apr-2019
https://dl.acm.org/doi/10.1145/3302506.3310385
Baig GHe JQureshi MQiu LChen GChen PHu YAgarwal SGreenstein BBalasubramanian AGollakota SZhang X(2019)JigsawThe 25th Annual International Conference on Mobile Computing and Networking10.1145/3300061.3300127(1-16)Online publication date: 5-Aug-2019
https://dl.acm.org/doi/10.1145/3300061.3300127
Kämäräinen TSiekkinen MEerikäinen JYlä-Jääski ABoll SMu Lee KLuo JZhu WByun HWen Chen CLienhart RMei T(2018)CloudVRProceedings of the 26th ACM international conference on Multimedia10.1145/3240508.3240620(1181-1189)Online publication date: 15-Oct-2018
https://dl.acm.org/doi/10.1145/3240508.3240620
Tan ZLi YLi QZhang ZLi ZLu S(2018)Supporting Mobile VR in LTE NetworksProceedings of the ACM on Measurement and Analysis of Computing Systems10.1145/31794112:1(1-31)Online publication date: 3-Apr-2018
https://dl.acm.org/doi/10.1145/3179411
Show More Cited By

View Options

Get Access

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Media

Figures

Other

Tables

View Table of Contents