research-article

QoE-Fair DASH Video Streaming Using Server-side Reinforcement Learning

Authors:

Sa’di Altamimi,

Shervin ShirmohammadiAuthors Info & Claims

ACM Transactions on Multimedia Computing, Communications, and Applications (TOMM), Volume 16, Issue 2s

Article No.: 68, Pages 1 - 21

https://doi.org/10.1145/3397227

Published: 21 June 2020 Publication History

Abstract

To design an optimal adaptive video streaming method, video service providers need to consider both the efficiency and the fairness of the Quality of Experience (QoE) of their users. In Reference [8], we proposed a server-side QoE-fair rate adaptation method that considers both efficiency and fairness of the QoE. The server uses Reinforcement Learning (RL) to select a bitrate for each client sharing the same bottleneck link to the server in a way that achieves fairness among concurrent DASH clients and imposes that bitrate by dynamically modifying the client’s Media Presentation Description (MPD) file. In this article, we extend that work to minimize the number of actions the server needs to take to keep the system in its equilibrium state. By incorporating a Recurrent Neural Network, specifically an LSTM model, we modify the server’s training algorithm to achieve improvements in both the quality and the quantity of actions the server takes to guide the client. Performance evaluation of the modified algorithm for clients running both homogeneous and heterogeneous adaptation algorithms showed that the number of server actions dropped by 14% and 22%, respectively, while QoE-fairness improved by at least 6% and 10%, respectively.

References

[1]

2012. ITU-T Recommendation G.1070 Opinion Model for Video Applications.

[2]

2014. ISO/IEC 23009-1, Information Technology - Dynamic Adaptive Streaming Over HTTP (DASH) - Part 1: Media Presentation Description and Segment Formats.

[3]

2020. Global Internet Phenomena Report. Sandvine, September 2019. Retrieved March 29, 2020 from https://www.sandvine.com/press-releases/sandvine-releases-2019-global-internet-phenomena-report.

[4]

2020. The Mobile Internet Phenomena Report. Sandvine. Retrieved March 29, 2020 from https://www.sandvine.com/press-releases/sandvine-releases-2020-mobile-internet-phenomena-report-youtube-is-over-25-of-all-mobile-traffic.

[5]

Saamer Akhshabi, Lakshmi Anantakrishnan, Ali C. Begen, and Constantine Dovrolis. 2012. What happens when HTTP adaptive streaming players compete for bandwidth? In Proceedings of the 22nd International Workshop on Network and Operating System Support for Digital Audio and Video (NOSSDAV’12). ACM, New York, NY, 9--14.

Digital Library

[6]

Saamer Akhshabi, Lakshmi Anantakrishnan, Constantine Dovrolis, and Ali Begen. 2013. Server-based traffic shaping for stabilizing oscillating adaptive streaming players. In Proceeding of the 23rd ACM Workshop on Network and Operating Systems Support for Digital Audio and Video (NOSSDAV’13). ACM, 19--24.

Digital Library

[7]

Zahaib Akhtar, Yun Seong Nam, Ramesh Govindan, Sanjay Rao, Jessica Chen, Ethan Katz-Bassett, Bruno Ribeiro, Jibin Zhan, and Hui Zhang. 2018. Oboe: Auto-tuning video ABR algorithms to network conditions. In Proceedings of the 2018 Conference of the ACM Special Interest Group on Data Communication (SIGCOMM’18). CM, New York, NY, 44--58.

Digital Library

[8]

Sa’di Altamimi and Shervin Shirmohammadi. 2019. Client-server cooperative and fair DASH video streaming. In Proceedings of the 29th ACM Workshop on Network and Operating Systems Support for Digital Audio and Video (NOSSDAV’19). ACM, New York, NY, USA, 1--6.

Digital Library

[9]

Abdelhak Bentaleb, Ali C. Begen, and Roger Zimmermann. 2018. ORL-SDN: Online reinforcement learning for SDN-enabled HTTP adaptive streaming. ACM Trans. Multimedia Comput. Commun. Appl. 14, 3, Article 71 (Aug. 2018), 28 pages.

Digital Library

[10]

Tyler Chesebro and Aleks Kamko. 2016. Learning Atari: An Exploration of the A3C Reinforcement Learning Method. Available: https://bcourses.berkeley.edu/files/70573736/download?download_frd=1.

[11]

Federal Communications Commission. [n.d.]. Raw Data - Measuring Broadband America. Retrieved from https://www.fcc.gov/reports-research/reports.

[12]

M. Gadaleta, F. Chiariotti, M. Rossi, and A. Zanella. 2017. D-DASH: A deep Q-learning framework for DASH video streaming. IEEE Trans. Cogn. Commun. Netw. 3, 4 (2017), 703--718.

[13]

Mahdi Hemmati. 2017. New Bandwidth Allocation Methods to Provide Quality-of-Experience Fairness for Video Streaming Services. Ph.D. Dissertation. University of Ottawa, School of Electrical Engineering 8 Computer Science.

[14]

Mahdi Hemmati, Abdulsalam Yassine, and Shervin Shirmohammadi. 2015. A Dec-POMDP model for congestion avoidance and fair allocation of network bandwidth in rate-adaptive video streaming. In Proceedings of the 2015 IEEE Symposium Series on Computational Intelligence. 1182--1189.

[15]

T. Hoßfeld, L. Skorin-Kapov, P. E. Heegaard, and M. Varela. 2017. Definition of QoE fairness in shared systems. IEEE Commun. Lett. 21, 1 (2017), 184--187. https://ieeexplore.ieee.org/document/7588099.

[16]

T. Huang, X. Yao, C. Wu, R. Zhang, Z. Pang, and L. Sun. 2019. Tiyuntsong: A self-play reinforcement learning approach for ABR video streaming. In Proceedings of the 2019 IEEE International Conference on Multimedia and Expo (ICME’19). 1678--1683.

[17]

Tianchi Huang, Chao Zhou, Rui-Xiao Zhang, Chenglei Wu, Xin Yao, and Lifeng Sun. 2019. Comyco: Quality-aware adaptive video streaming via imitation learning. In Proceedings of the 27th ACM International Conference on Multimedia (MM’19). ACM, New York, NY, 429--437.

Digital Library

[18]

Te-Yuan Huang, Nikhil Handigol, Brandon Heller, Nick McKeown, and Ramesh Johari. 2012. Confused, timid, and unstable: Picking a video streaming rate is hard. In Proceedings of the 2012 Internet Measurement Conference (IMC’12). ACM, New York, NY, 225--238.

Digital Library

[19]

R. Jain, D.-M. Chiu, W. R. Hawe, T. Zinner, T. Hoßfeld, and P. Tran-Gia. 1984. A quantitative measure of fairness and discrimination for resource allocation in shared computer system. Eastern Res. Lab. (1984), Hudson, MA.

[20]

Junchen Jiang, Vyas Sekar, and Hui Zhang. 2014. Improving fairness, efficiency, and stability in HTTP-based adaptive video streaming with festive. IEEE/ACM Trans. Netw. 22, 1 (Feb. 2014), 326--340.

Digital Library

[21]

Vijay Konda and John Tsitsiklis. 2003. On actor-critic algorithms. SIAM J. Contr. Optim. 42, 4 (2003), 1143--1166.

Digital Library

[22]

Stefan Lederer, Christopher Muller, and Christian Timmerer. 2012. Dynamic adaptive streaming over HTTP dataset. In Proceedings of the 3rd Multimedia Systems Conference (MMSys’12). ACM, New York, NY, 89--94.

Digital Library

[23]

De Cicco Luca, Saverio Mascolo, and Vittorio Palmisano. 2011. Feedback control for adaptive live video streaming. In Proceedings of the 2nd Annual ACM Conference on Multimedia Systems (MMSys’11). ACM, 145--156.

[24]

Hongzi Mao, Ravi Netravali, and Mohammad Alizadeh. 2017. Neural adaptive video streaming with pensieve. Proceedings of the ACM Special Interest Group on Data Communication (SIGCOMM’17).

Digital Library

[25]

Volodymyr Mnih, Koray Kavukcuoglu, David Silver, et al. 2015. Human-level control through deep reinforcement learning. Nature 25, 2, 529--533. http://dx.doi.org/10.1038/518486a

[26]

A. Nadembega, A. Hafid, and T. Taleb. 2014. An integrated predictive mobile-oriented bandwidth-reservation framework to support mobile multimedia streaming. IEEE Trans. Wireless Commun. 13, 12 (2014), 6863--6875.

[27]

Vikram Nathan, Vibhaalakshmi Sivaraman, Ravichandra Addanki, Mehrdad Khani, Prateesh Goyal, and Mohammad Alizadeh. 2019. End-to-end transport for video QoE fairness. In Proceedings of the ACM Special Interest Group on Data Communication (SIGCOMM’19). ACM, New York, NY, 408--423.

Digital Library

[28]

Haakon Riiser, Paul Vigmostad, Carsten Griwodz, and Halvorsen Pal. 2013. Commute path bandwidth traces from 3G networks: Analysis and applications. In Proceedings of the 4th ACM Multimedia Systems Conference (MMSys’13). ACM, New York, NY, 114--118.

Digital Library

[29]

J. Padhye S. Floyd, M. Handley and J. Widmer. 2008. TCP Friendly Rate Control (TFRC): Protocol Specification (Proposed Standard). Retrieved from https://tools.ietf.org/html/rfc5348.

[30]

M. Seufert, S. Egger, M. Slanina, T. Zinner, T. Hoßfeld, and P. Tran-Gia. 2015. A survey on quality of experience of HTTP adaptive streaming. IEEE Commun. Surv. Tutor. 17, 1 (2015), 469--492.

Digital Library

[31]

M. Seufert, N. Wehner, and P. Casas. 2019. A fair share for all: TCP-inspired adaptation logic for QoE fairness among heterogeneous HTTP adaptive video streaming clients. IEEE Trans. Netw. Serv. Manag. 16, 2 (2019), 475--488.

[32]

Ashkan Sobhani, Abdulsalam Yassine, and Shervin Shirmohammadi. 2017. A video bitrate adaptation and prediction mechanism for HTTP adaptive streaming. ACM Trans. Multimedia Comput. Commun. Appl. 13, 2, (Mar. 2017), 25 pages.

Digital Library

[33]

Kevin Spiteri, Rahul Urgaonkar, and Ramesh K. Sitaraman. 2016. BOLA: Near-optimal bitrate adaptation for online videos. In Proceedings of the 35th Annual IEEE International Conference on Computer Communications (INFOCOM’16). 1--9.

[34]

Richard Sutton and Andrew Barto. 1998. Reinforcement Learning: An Introduction. MIT Press.

Digital Library

[35]

The PyTorch team. 2020. PyTorch Neural Network Framework. Retrieved from https://pytorch.org/docs/stable/nn.html#lstm.

[36]

Xiaoqi Yin, Abhishek Jindal, Vyas Sekar, and Bruno Sinopoli. 2015. A control-theoretic approach for dynamic adaptive video streaming over HTTP. In Proceedings of the 2015 ACM Conference on Special Interest Group on Data Communication (SIGCOMM’15). ACM, New York, NY, 325--338.

Digital Library

Cited By

Yuan YWang WWang YAdhatarao SRen BZheng KFu X(2024)Joint Optimization of QoE and Fairness for Adaptive Video Streaming in Heterogeneous Mobile EnvironmentsIEEE/ACM Transactions on Networking10.1109/TNET.2023.327772932:1(50-64)Online publication date: Mar-2024
https://doi.org/10.1109/TNET.2023.3277729
Li ZZhong PHuang JGao FWang J(2024)Achieving QoE Fairness in Bitrate Allocation of 360° Video StreamingIEEE Transactions on Multimedia10.1109/TMM.2023.327728626(1169-1178)Online publication date: 1-Jan-2024
https://dl.acm.org/doi/10.1109/TMM.2023.3277286
Wang HZhong ZYin JXu YYang L(2024)Enhancing Real-Time Video Streaming with Joint Frame Size and Rate Adaptation2024 IEEE International Symposium on Circuits and Systems (ISCAS)10.1109/ISCAS58744.2024.10557847(1-5)Online publication date: 19-May-2024
https://doi.org/10.1109/ISCAS58744.2024.10557847
Show More Cited By

Index Terms

QoE-Fair DASH Video Streaming Using Server-side Reinforcement Learning
1. Information systems
  1. Information systems applications
    1. Multimedia information systems
      1. Multimedia streaming

Recommendations

Client-server cooperative and fair DASH video streaming
NOSSDAV '19: Proceedings of the 29th ACM Workshop on Network and Operating Systems Support for Digital Audio and Video

Adaptive video streaming over HTTP, such as the MPEG-DASH standard, is now widely used by video service provides to stream their videos to users. But DASH and similar methods are known to suffer from two practical challenges: on the one hand, clients ...
DQ-DASH: A Queuing Theory Approach to Distributed Adaptive Video Streaming

The significant popularity of HTTP adaptive video streaming (HAS), such as Dynamic Adaptive Streaming over HTTP (DASH), over the Internet has led to a stark increase in user expectations in terms of video quality and delivery robustness. This situation ...
Reinforcement learning-based rate adaptation in dynamic video streaming
Abstract
Video streaming stands out as the most significant traffic type consumed by mobile devices. This increased demand has been a major driver for research on bitrate adaptation algorithms. Bitrate adaptation ensures high user-perceived quality, which, ...

Comments

Information & Contributors

Information

Published In

cover image ACM Transactions on Multimedia Computing, Communications, and Applications

ACM Transactions on Multimedia Computing, Communications, and Applications Volume 16, Issue 2s

Special Issue on Smart Communications and Networking for Future Video Surveillance and Special Section on Extended MMSYS-NOSSDAV 2019 Best Papers

April 2020

291 pages

ISSN:1551-6857

EISSN:1551-6865

DOI:10.1145/3407689

Editor:
Alberto Del Bimbo
University of Firenze, Italy

Issue’s Table of Contents

Copyright © 2020 ACM.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 21 June 2020

Online AM: 07 May 2020

Accepted: 01 April 2020

Revised: 01 March 2020

Received: 01 December 2019

Published in TOMM Volume 16, Issue 2s

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Research-article
Research
Refereed

Funding Sources

Cisco Systems

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

18
Total Citations
View Citations
592
Total Downloads

Downloads (Last 12 months)88
Downloads (Last 6 weeks)17

Reflects downloads up to 26 Jul 2024

Other Metrics

View Author Metrics

Citations

Cited By

Yuan YWang WWang YAdhatarao SRen BZheng KFu X(2024)Joint Optimization of QoE and Fairness for Adaptive Video Streaming in Heterogeneous Mobile EnvironmentsIEEE/ACM Transactions on Networking10.1109/TNET.2023.327772932:1(50-64)Online publication date: Mar-2024
https://doi.org/10.1109/TNET.2023.3277729
Li ZZhong PHuang JGao FWang J(2024)Achieving QoE Fairness in Bitrate Allocation of 360° Video StreamingIEEE Transactions on Multimedia10.1109/TMM.2023.327728626(1169-1178)Online publication date: 1-Jan-2024
https://dl.acm.org/doi/10.1109/TMM.2023.3277286
Wang HZhong ZYin JXu YYang L(2024)Enhancing Real-Time Video Streaming with Joint Frame Size and Rate Adaptation2024 IEEE International Symposium on Circuits and Systems (ISCAS)10.1109/ISCAS58744.2024.10557847(1-5)Online publication date: 19-May-2024
https://doi.org/10.1109/ISCAS58744.2024.10557847
Liu HChen QLiu P(2023)An Optimization Method of Large-Scale Video Stream Concurrent Transmission for Edge ComputingMathematics10.3390/math1112262211:12(2622)Online publication date: 8-Jun-2023
https://doi.org/10.3390/math11122622
Nami SPakdaman FHashemi MShirmohammadi S(2023)BL-JUNIPER: A CNN-Assisted Framework for Perceptual Video Coding Leveraging Block-Level JNDIEEE Transactions on Multimedia10.1109/TMM.2022.318725925(5077-5092)Online publication date: 1-Jan-2023
https://dl.acm.org/doi/10.1109/TMM.2022.3187259
Kim MChung K(2023)HTTP adaptive streaming scheme based on reinforcement learning with edge computing assistanceJournal of Network and Computer Applications10.1016/j.jnca.2023.103604213(103604)Online publication date: May-2023
https://doi.org/10.1016/j.jnca.2023.103604
Kim MChung K(2022)Reinforcement Learning-Based Adaptive Streaming Scheme with Edge Computing AssistanceSensors10.3390/s2206217122:6(2171)Online publication date: 10-Mar-2022
https://doi.org/10.3390/s22062171
Liu YWei DZhang CLi W(2022)Distributed Bandwidth Allocation Strategy for QoE Fairness of Multiple Video Streams in Bottleneck LinksFuture Internet10.3390/fi1405015214:5(152)Online publication date: 18-May-2022
https://doi.org/10.3390/fi14050152
Hsu CHung THsu C(2022)Optimizing Immersive Video Coding Configurations Using Deep Learning: A Case Study on TMIVACM Transactions on Multimedia Computing, Communications, and Applications10.1145/347119118:1(1-25)Online publication date: 27-Jan-2022
https://dl.acm.org/doi/10.1145/3471191
Karagkioules TPaschos GLiakopoulos NFiandrotti ATsilimantos DCagnazzo M(2022)Online Learning for Adaptive Video Streaming in Mobile NetworksACM Transactions on Multimedia Computing, Communications, and Applications10.1145/346081918:1(1-22)Online publication date: 27-Jan-2022
https://dl.acm.org/doi/10.1145/3460819
Show More Cited By

View Options

Get Access

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Article

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

HTML Format

View this article in HTML Format.

Media

Figures

Other

Tables

View Issue’s Table of Contents