research-article

C2: consumption context cognizant ABR streaming for improved QoE and resource usage tradeoffs

Authors:

Chinmaey Shende,

Subhabrata Sen,

Bing WangAuthors Info & Claims

MMSys '22: Proceedings of the 13th ACM Multimedia Systems Conference

Pages 109 - 122

https://doi.org/10.1145/3524273.3528185

Published: 05 August 2022 Publication History

Abstract

Smartphones have emerged as ubiquitous platforms for people to consume content in a wide range of consumption contexts (C2), e.g., over cellular or WiFi, playing back audio and video directly on phone or through peripheral devices such as external screens or speakers, etc. In this paper, we argue that a user's specific C2 is an important factor to consider in Adaptive Bitrate (ABR) streaming. We examine the current practice of using C2 in four popular ABR players, and identify various limitations in existing treatments that have a detrimental impact on network resource usage and user experience. We then develop practical best-practice guidelines for C2-cognizant ABR streaming. Instantiating these guidelines, we develop a proof-of-concept implementation in the widely used state-of-the-art ExoPlayer platform and demonstrate that it leads to significantly better tradeoffs in terms of user experience and resource usage.

References

[1]

[n.d.]. Advanced Audio Coding. https://en.wikipedia.org/wiki/Advanced_Audio_Coding.

[2]

[n.d.]. DASH Industry Forum. https://dashif.org/.

[3]

[n.d.]. Dolby Digital Plus. https://en.wikipedia.org/wiki/Dolby_Digital_Plus.

[4]

[n.d.]. Everything You Need to Know About Screen Mirroring iPhone and iPad. https://www.airsquirrels.com/reflector/resources/everything-you-need-to-know-about-screen-mirroring-iphone-ios-ipad-ipados.

[5]

[n.d.]. ExoPlayer. https://github.com/google/ExoPlayer/tree/r2.16.1.

[6]

[n.d.]. HTTP Live Streaming in Dolby. https://developer.dolby.com/platforms/apple/hls-in-dolby/.

[7]

[n.d.]. Surround sound. https://en.wikipedia.org/wiki/Surround_sound.

[8]

[n.d.]. Tailgate party. https://en.wikipedia.org/wiki/Tailgate_party.

[9]

[n.d.]. Video scaler. https://en.wikipedia.org/wiki/Video_scaler.

[10]

2015. GstPEAQ - A GStreamer plugin for Perceptual Evaluation of Audio Quality (PEAQ) v0.6.1. https://github.com/HSU-ANT/gstpeaq.

[11]

2018. Information technology-Multimedia application format (MPEG-A)-Part19: Common media application format (CMAF) for segmented media. Standard ISO/IEC 23000-19:2018, International Organization for Standardization and International Electrotechnical Commission. https://www.iso.org/standard/71975.html.

[12]

2020. ViSQOL v3.1.0. https://github.com/google/visqol.

[13]

2021. Pixel 4a HDMI Output? https://www.reddit.com/r/GooglePixel/comments/i3n3t4/pixel_4a_hdmi_output/.

[14]

2021. You can now play videos up to 4K60 on mobile regardless of screen resolution. https://www.reddit.com/r/youtube/comments/ln3dy8/you_can_now_play_videos_up_to_4k60_on_mobile/.

[15]

2021. YouTube for Android now giving 4K playback option even if don't have a 4K screen. https://www.gsmarena.com/youtube_for_android_showing_4k_playback_option_for_some_users-news-47842.php.

[16]

2021. YouTube on Android now lets you watch 4K videos on FHD screens. https://www.slashgear.com/youtube-on-android-now-lets-you-watch-4k-videos-on-fhd-screens-21660339.

[17]

Zahaib Akhtar, Yun Seong Nam, Ramesh Govindan, Sanjay Rao, Jessica Chen, Ethan Katz-Bassett, Bruno Ribeiro, Jibin Zhan, and Hui Zhang. 2018. Oboe: Auto-tuning Video ABR Algorithms to Network Conditions. In SIGCOMM.

[18]

Apple. [n.d.]. AVFoundation. https://developer.apple.com/av-foundation/.

[19]

Apple. [n.d.]. AVPlayer. https://developer.apple.com/documentation/avfoundation/avplayer.

[20]

Apple. 2017. Apple's HTTP Live Streaming. https://goo.gl/eyDmBc.

[21]

ATSC (Advanced Television Systems Committee). 2012. ATSC Standard: Digital Audio Compression (AC-3, E-AC-3).

[22]

Abdelhak Bentaleb, Bayan Taani, Ali C. Begen, Christian Timmerer, and Roger Zimmermann. 2019. A Survey on Bitrate Adaptation Schemes for Streaming Media Over HTTP. IEEE Communications Surveys & Tutorials 21, 1 (2019).

[23]

Michael Chinen, Felicia S. C. Lim, Jan Skoglund, Nikita Gureev, Feargus O'Gorman, and Andrew Hines. 2020. ViSQOL v3: An Open Source Production Ready Objective Speech and Audio Metric. In Proc. of International Workshop on Quality of Multimedia Experience (QoMEX).

[24]

L. De Cicco, S. Mascolo, and V. Palmisano. 2011. Feedback Control for Adaptive Live Video Streaming. In ACM MMSys.

[25]

William Cooper, Sue Farrell, and Kumar Subramanian. 2017. QBR Metadata to Improve Streaming Efficiency and Quality. In SMPTE.

[26]

DASH Industry Forum. [n.d.]. dash.js. https://goo.gl/XJcciV.

[27]

Corbin Davenport. 2019. Pixel 4 has USB video output disabled in software. https://www.androidpolice.com/2019/11/03/pixel-4-has-usb-video-output-disabled-in-software/.

[28]

Luca De Cicco, Vito Caldaralo, Vittorio Palmisano, and Saverio Mascolo. 2013. ELASTIC: a client-side controller for dynamic adaptive streaming over HTTP (DASH). In Proc. of Packet Video Workshop (PV). IEEE.

[29]

Google. 2014. ExoPlayer: Adaptive video streaming on Android - YouTube. https://www.youtube.com/watch?v=6VjF638VObA.

[30]

Google. 2016. ExoPlayer. https://github.com/google/ExoPlayer.

[31]

Google. 2019. Shaka Player. https://github.com/google/shaka-player.

[32]

Joe Hindy. 2021. 5 best screen mirroring apps for Android and other ways too. https://www.androidauthority.com/best-screen-mirroring-apps-android-ways-807191/.

[33]

A. Hines, J. Skoglund, A. C. Kokaram, and N. Harte. 2015. ViSQOL: an objective speech quality model. EURASIP Journal on Audio, Speech, and Music Processing (2015).

[34]

Tony Hoffman. 2021. The Best Portable Projectors for 2022. https://www.pcmag.com/picks/the-best-portable-projectors/.

[35]

Te-Yuan Huang, Ramesh Johari, Nick McKeown, Matthew Trunnell, and Mark Watson. 2014. A buffer-based approach to rate adaptation: Evidence from a large video streaming service. In Proc. of ACM SIGCOMM.

Digital Library

[36]

R. Huber and B. Kollmeier. 2006. PEMO-Q: A new method for objective audio quality assessment using a model of auditory perception. IEEE Audio, Speech, Language Process 14, 6 (November 2006), 1902--1911.

Digital Library

[37]

International Organization for Standardization. 2012. ISO/IEC DIS 23009-1.2 Dynamic adaptive streaming over HTTP (DASH).

[38]

ITU. 2001. ITU-R Recommendation BS.1387-1: Method for objective measurements of perceived audio quality (PEAQ).

[39]

ITU. 2012. Recommendation ITU-TP.1201.2: Parametric non-intrusive assessment of audiovisual media streaming quality - Higher resolution application area.

[40]

ITU. 2014. ITU-T Rec. P.863: Perceptual objective listening quality assessment.

[41]

ITU. 2017. Recommendation ITU-T P.1203.2: Parametric bitstream-based quality assessment of progressive download and adaptive audiovisual streaming services over reliable transport - Audio quality estimation module.

[42]

K. Kellogg and T. Garcia. 2021. 13 Best Mini Projects for Indoor and Outdoor Use. https://www.teenvogue.com/story/best-mini-projectors/.

[43]

Jonathan Kua, Grenville Armitage, and Philip Branch. 2017. A survey of rate adaptation techniques for dynamic adaptive streaming over HTTP. IEEE Communications Surveys & Tutorials 19, 3 (2017).

Digital Library

[44]

Lynn La. 2022. Top foldable phones for 2022: Motorola Razr 2020, Galaxy Flip, Galaxy Fold 2 and more. https://www.cnet.com/tech/mobile/best-foldable-phones/. access in January 2022.

[45]

Zhi Li, Anne Aaron, Ioannis Katsavounidis, Anush Moorthy, and Megha Manohara. 2016. Toward A Practical Perceptual Video Quality Metric. https://goo.gl/ptjrWv.

[46]

Zhi Li, Ali Begen, Joshua Gahm, Yufeng Shan, Bruce Osler, and David Oran. 2014. Streaming video over HTTP with consistent quality. In ACM MMSys.

[47]

Linux. 2014. tc. https://linux.die.net/man/8/tc.

[48]

Hongzi Mao, Ravi Netravali, and Mohammad Alizadeh. 2017. Neural Adaptive Video Streaming with Pensieve. In Proc. of ACM SIGCOMM.

Digital Library

[49]

Ricky KP Mok, Xiapu Luo, Edmond WW Chan, and Rocky KC Chang. 2012. QDASH: a QoE-aware DASH system. In ACM MMSys.

[50]

Mridula Nimawat. 2021. 15 Best Free Screen Mirroring Apps For Android & iPhone [2022]. https://wethegeek.com/screen-mirroring-apps/.

[51]

Yanyuan Qin, Shuai Hao, Krishna R Pattipati, Feng Qian, Subhabrata Sen, Bing Wang, and Chaoqun Yue. 2018. ABR streaming of VBR-encoded videos: characterization, challenges, and solutions. In CoNext. ACM.

[52]

Yanyuan Qin, Shuai Hao, Krishna R Pattipati, Feng Qian, Subhabrata Sen, Bing Wang, and Chaoqun Yue. 2019. Quality-aware strategies for optimizing ABR video streaming QoE and reducing data usage. In MMSys. ACM.

[53]

Yanyuan Qin, Ruofan Jin, Shuai Hao, Krishna R Pattipati, Feng Qian, Subhabrata Sen, Chaoqun Yue, and Bing Wang. 2020. A Control Theoretic Approach to ABR Video Streaming: A Fresh Look at PID-based Rate Adaptation. IEEE Transactions on Mobile Computing 19, 11 (2020).

[54]

Yanyuan Qin, Subhabrata Sen, and Bing Wang. 2019. ABR Streaming with Separate Audio and Video Tracks: Measurements and Best Practices. In CoNext. ACM.

[55]

Yanyuan Qin, Chinmaey Shende, Cheonjin Park, Subhabrata Sen, and Bing Wang. 2021. DataPlanner: Data-budget Driven Approach to Resource-efficient ABR Streaming. In ACM MMSys.

[56]

Chandan KA Reddy, Vishak Gopal, and Ross Cutler. 2020. DNSMOS: A Non-Intrusive Perceptual Objective Speech Quality metric to evaluate Noise Suppressors. arXiv e-prints (2020), arXiv-2010.

[57]

M. Seufert, S. Egger, M. Slanina, T. Zinner, T. Hoßfeld, and P. Tran-Gia. 2015. A Survey on Quality of Experience of HTTP Adaptive Streaming. IEEE Communications Surveys & Tutorials 17, 1 (2015).

[58]

SimilarTech Ltd. 2019. Facebook Video vs Shaka Player. https://www.similartech.com/compare/facebook-video-vs-shaka-player

[59]

Colm Sloan, Naomi Harte, Damien Kelly, Anil C. Kokaram, and Andrew Hines. 2017. Objective Assessment of Perceptual Audio Quality Using ViSQOLAudio. IEEE Transactions on Broadcasting 63, 4 (2017).

[60]

Kevin Spiteri, Ramesh Sitaraman, and Daniel Sparacio. 2018. From Theory to Practice: Improving Bitrate Adaptation in the DASH Reference Player. In MMSys.

[61]

Kevin Spiteri, Rahul Urgaonkar, and Ramesh K Sitaraman. 2016. BOLA: Near-Optimal Bitrate Adaptation for Online Videos. In INFOCOM. IEEE.

[62]

Olivia Tambini. 2021. SpatialAudio: our guide to immersive speakers, headphones, and streaming services. https://www.techradar.com/news/spatial-audio-your-complete-guide-to-immersive-speakers-headphones-and-streaming-services.

[63]

Ewdison Then. 2019. Pixel 4 USB-C video output exists but is disabled in source code. https://www.slashgear.com/pixel-4-usb-c-video-output-exists-but-is-disabled-in-source-code-03598259/.

[64]

Guibin Tian and Yong Liu. 2012. Towards agile and smooth video adaptation in dynamic HTTP streaming. In ACM CoNEXT.

[65]

Xiph. 2016. Xiph Video Test Media. https://media.xiph.org/video/derf/.

[66]

Xiaoqi Yin, Abhishek Jindal, Vyas Sekar, and Bruno Sinopoli. 2015. A Control-Theoretic Approach for Dynamic Adaptive Video Streaming over HTTP. In SIGCOMM. ACM.

[67]

C. Yue, S. Sen, B. Wang, Y. Qin, and F. Qian. 2020. Energy Considerations for ABR Video Streaming to Smartphones: Measurements, Models and Insights. In ACM MMSys.

[68]

Diekola Yusuf. 2019. 5 Best Projector Phones For Presenters. https://www.novabach.com/5-best-projector-phones-for-presenters/.

Cited By

Park CShende CSen SWang B(2024)C2: ABR Streaming in Cognizant of Consumption Context for Improved QoE and Resource Usage TradeoffsACM Transactions on Multimedia Computing, Communications, and Applications10.1145/365251720:9(1-27)Online publication date: 16-Aug-2024
https://dl.acm.org/doi/10.1145/3652517
Li WHuang JSu QJiang WWang J(2024)VASE: Enhancing Adaptive Bitrate Selection for VBR-Encoded Audio and Video Content With Deep Reinforcement LearningIEEE Transactions on Mobile Computing10.1109/TMC.2024.344837023:12(14889-14902)Online publication date: Dec-2024
https://doi.org/10.1109/TMC.2024.3448370

Index Terms

C2: consumption context cognizant ABR streaming for improved QoE and resource usage tradeoffs
1. Information systems
  1. Information systems applications
    1. Multimedia information systems
      1. Multimedia streaming

Recommendations

C2: ABR Streaming in Cognizant of Consumption Context for Improved QoE and Resource Usage Tradeoffs
Smartphones have emerged as ubiquitous platforms for people to consume content in a wide range of consumption contexts (C2), e.g., over cellular or WiFi, playing back audio and video directly on phone or through peripheral devices such as external screens ...
Improving QoE of ABR Streaming Sessions through QUIC Retransmissions
MM '18: Proceedings of the 26th ACM international conference on Multimedia

While adaptive bitrate (ABR) streaming has contributed significantly to the reduction of video playout stalling, ABR clients continue to suffer from the variation of bit rate qualities over the duration of a streaming session. Similar to stalling, these ...
Towards a Benchmark for Software Resource Efficiency
ICPE '21: Companion of the ACM/SPEC International Conference on Performance Engineering

Data centers already account for over 250TWh of energy usage every year and their energy demand will grow above 1PWh until 2030 even in the best-case scenarios of some studies. As this demand cannot be met with renewable sources as of today, this growth ...

Comments

Information & Contributors

Information

Published In

cover image ACM Conferences

MMSys '22: Proceedings of the 13th ACM Multimedia Systems Conference

June 2022

432 pages

ISBN:9781450392839

DOI:10.1145/3524273

General Chairs:
Niall Murray
Technological University of the Shannon: Midlands Midwest
,
Gwendal Simon
Synamedia
,
Mylene Farias
University of Brasilia
,
Program Chairs:
Irene Viola
Centrum Wiskunde & Informatica
,
Mario Montagud
i2CAT Foundation & University of Valencia

Copyright © 2022 ACM.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

Sponsors

SIGMM: ACM Special Interest Group on Multimedia

In-Cooperation

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 05 August 2022

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Research-article

Conference

MMSys '22

Sponsor:

SIGMM

MMSys '22: 13th ACM Multimedia Systems Conference

June 14 - 17, 2022

Athlone, Ireland

Acceptance Rates

Overall Acceptance Rate 176 of 530 submissions, 33%

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

2
Total Citations
View Citations
114
Total Downloads

Downloads (Last 12 months)13
Downloads (Last 6 weeks)0

Reflects downloads up to 01 Jan 2025

Other Metrics

View Author Metrics

Citations

Cited By

Park CShende CSen SWang B(2024)C2: ABR Streaming in Cognizant of Consumption Context for Improved QoE and Resource Usage TradeoffsACM Transactions on Multimedia Computing, Communications, and Applications10.1145/365251720:9(1-27)Online publication date: 16-Aug-2024
https://dl.acm.org/doi/10.1145/3652517
Li WHuang JSu QJiang WWang J(2024)VASE: Enhancing Adaptive Bitrate Selection for VBR-Encoded Audio and Video Content With Deep Reinforcement LearningIEEE Transactions on Mobile Computing10.1109/TMC.2024.344837023:12(14889-14902)Online publication date: Dec-2024
https://doi.org/10.1109/TMC.2024.3448370

View Options

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Media

Figures

Other

Tables

View Table of Contents