research-article

Higher quality live streaming under lower uplink bandwidth: an approach of super-resolution based video coding

Authors:

Zhenhui YuanAuthors Info & Claims

NOSSDAV '21: Proceedings of the 31st ACM Workshop on Network and Operating Systems Support for Digital Audio and Video

Pages 74 - 81

https://doi.org/10.1145/3458306.3458874

Published: 02 July 2021 Publication History

Abstract

With the growing popularity of live streaming, high video quality and low latency with limited uplink bandwidth have become a significant challenge. In this study, we propose Live Super-Resolution Based Video Coding (LiveSRVC), a novel video uploading framework that improves the quality of live streaming with low latency under limited uplink bandwidth. We design a new super-resolution-based key frame coding module to improve the coding compression efficiency. LiveSRVC dynamically selects the bitrate and the compression ratio of key frames, mitigating the influence of uplink bandwidth capacity on live streaming quality. Trace-driven emulations verify that LiveSRVC can provide the same quality while reducing up to 50% of the required bandwidth compared to the original encoding method (H.264). LiveSRVC consumes at least 10X less GPU occupation time compared to the method of reconstructing all frames with super-resolution.

References

[1]

2020. H.264 Video Encoding Guide. https://trac.ffmpeg.org/wiki/Encode/H.264/. Last accessed: 03 April 2021.

[2]

Eirikur Agustsson and Radu Timofte. 2017. NITRE 2017 Challenge on Single Image Super-Resolution: Dataset and Study. In The IEEE Conference on Computer Vision and Pattern Recognition (CVPR) Workshops.

[3]

Namhyuk Ahn, Byungkon Kang, and Kyung-Ah Sohn. 2018. Fast, accurate, and lightweight super-resolution with cascading residual network. In Proceedings of the European Conference on Computer Vision (ECCV). 252--268.

Digital Library

[4]

Apple. 2020. HLS Authoring Specification for Apple Devices. https://developer.apple.com/documentation/http_live_streaming/hls_authoring_specification_for_apple_devices. Last accessed: 03 April 2021.

[5]

Athula Balachandran, Vyas Sekar, Aditya Akella, Srinivasan Seshan, Ion Stoica, and Hui Zhang. 2013. Developing a predictive model of quality of experience for internet video. ACM SIGCOMM Computer Communication Review 43, 4 (2013), 339--350.

Digital Library

[6]

Fernanda Brandi, Ricardo de Queiroz, and Debargha Mukherjee. 2008. Super resolution of video using key frames. In 2008 IEEE International Symposium on Circuits and Systems. IEEE, 1608--1611.

[7]

Jose Caballero, Christian Ledig, Andrew Aitken, Alejandro Acosta, Johannes Totz, Zehan Wang, and Wenzhe Shi. 2017. Real-time video super-resolution with spatio-temporal networks and motion compensation. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. 4778--4787.

[8]

Pradeep Dogga, Sandip Chakraborty, Subrata Mitra, and Ravi Netravali. 2019. Edge-based Transcoding for Adaptive Live Video Streaming. In 2nd {USENIX} Workshop on Hot Topics in Edge Computing (HotEdge 19).

[9]

Chao Dong, Chen Change Loy, Kaiming He, and Xiaoou Tang. 2015. Image super-resolution using deep convolutional networks. IEEE transactions on pattern analysis and machine intelligence 38, 2 (2015), 295--307.

[10]

Oliver L. Haimson and John C. Tang. 2017. What Makes Live Events Engaging on Facebook Live, Periscope, and Snapchat. In The ACM 2018 CHI Conference on Human Factors in Computing Systems. 48--60.

[11]

Tianchi Huang, Chao Zhou, Rui-Xiao Zhang, Chenglei Wu, Xin Yao, and Lifeng Sun. 2019. Comyco: Quality-aware adaptive video streaming via imitation learning. In Proceedings of the 27th ACM International Conference on Multimedia. 429--437.

Digital Library

[12]

Bert Hubert et al. 2002. Linux Advanced Routing & Traffic Control HOWTO. https://tldp.org/HOWTO/Adv-Routing-HOWTO/. Last accessed: 03 April 2021.

[13]

Robert Keys. 1981. Cubic convolution interpolation for digital image processing. IEEE transactions on acoustics, speech, and signal processing 29, 6 (1981), 1153--1160.

[14]

Jaehong Kim, Youngmok Jung, Hyunho Yeo, Juncheol Ye, and Dongsu Han. 2020. Neural-Enhanced Live Streaming: Improving Live Video Ingest via Online Learning. In Proceedings of the Annual conference of the ACM Special Interest Group on Data Communication on the applications, technologies, architectures, and protocols for computer communication. 107--125.

Digital Library

[15]

Jiwon Kim, Jung Kwon Lee, and Kyoung Mu Lee. 2016. Accurate image super-resolution using very deep convolutional networks. In Proceedings of the IEEE conference on computer vision and pattern recognition. 1646--1654.

[16]

S Shunmuga Krishnan and Ramesh K Sitaraman. 2013. Video stream quality impacts viewer behavior: inferring causality using quasi-experimental designs. IEEE/ACM Transactions on Networking 21, 6 (2013), 2001--2014.

Digital Library

[17]

Yue Li, Dong Liu, Houqiang Li, Li Li, Feng Wu, Hong Zhang, and Haitao Yang. 2017. Convolutional neural network-based block up-sampling for intra frame coding. IEEE Transactions on Circuits and Systems for Video Technology 28, 9 (2017), 2316--2330.

[18]

Zhi Li, Anne Aaron, Ioannis Katsavounidis, Anush Moorthy, and Megha Manohara. 2016. Toward a practical perceptual video quality metric. The Netflix Tech Blog 6 (2016), 2.

[19]

Bee Lim, Sanghyun Son, Heewon Kim, Seungjun Nah, and Kyoung Mu Lee. 2017. Enhanced deep residual networks for single image super-resolution. In Proceedings of the IEEE conference on computer vision and pattern recognition workshops. 136--144.

[20]

Hongwei Lin, Xiaohai He, Linbo Qing, Qizhi Teng, and Songfan Yang. 2019. Improved low-bitrate HEVC video coding using deep learning based super-resolution and adaptive block patching. IEEE Transactions on Multimedia 21, 12 (2019), 3010--3023.

Digital Library

[21]

Zhicong Lu, Haijun Xia, Seongkook Heo, and Daniel Wigdor. 2018. You Watch, You Give, and You Engage: A Study of Live Streaming Practices in China. In The ACM 2018 CHI Conference on Human Factors in Computing Systems.

Digital Library

[22]

Darijo Raca, Jason J Quinlan, Ahmed H Zahran, and Cormac J Sreenan. 2018. Beyond throughput: a 4G LTE dataset with channel and context metrics. In Proceedings of the 9th ACM Multimedia Systems Conference. 460--465.

Digital Library

[23]

Heiko Schwarz, Detlev Marpe, and Thomas Wiegand. 2007. Overview of the scalable video coding extension of the H. 264/AVC standard. IEEE Transactions on circuits and systems for video technology 17, 9 (2007), 1103--1120.

Digital Library

[24]

SPEEDTEST. 2019. SPEEDTEST Market Report. https://www.speedtest.net/reports/zh/. Last accessed: 03 April 2021.

[25]

Ziyi Wang, Yong Cui, Xiaoyu Hu, Xin Wang, Wei Tsang Ooi, and Yi Li. 2020. MueltiLive: Adaptive Bitrate Control for Low-delay Multi-party Interactive Live Streaming. In IEEE INFOCOM 2020-IEEE Conference on Computer Communications. IEEE, 1093--1102.

[26]

Zhimin Xu, Xinggong Zhang, and Zongming Guo. 2018. QoE-driven adaptive K-push for HTTP/2 live streaming. IEEE Transactions on Circuits and Systems for Video Technology 29, 6 (2018), 1781--1794.

[27]

Hyunho Yeo, Youngmok Jung, Jaehong Kim, Jinwoo Shin, and Dongsu Han. 2018. Neural adaptive content-aware internet video delivery. In 13th {USENIX} Symposium on Operating Systems Design and Implementation ({OSDI} 18). 645--661.

[28]

Xiaoqi Yin, Abhishek Jindal, Vyas Sekar, and Bruno Sinopoli. 2015. A control-theoretic approach for dynamic adaptive video streaming over HTTP. In Proceedings of the 2015 ACM Conference on Special Interest Group on Data Communication. 325--338.

Digital Library

[29]

Rui-Xiao Zhang, Tianchi Huang, Ming Ma, Haitian Pang, Xin Yao, Chenglei Wu, and Lifeng Sun. 2019. Enhancing the crowdsourced live streaming: a deep reinforcement learning approach. In Proceedings of the 29th ACM Workshop on Network and Operating Systems Support for Digital Audio and Video. 55--60.

Digital Library

[30]

Yinjie Zhang, Yuanxing Zhang, Yi Wu, Yu Tao, Kaigui Bian, Pan Zhou, Lingyang Song, and Hu Tuo. 2020. Improving Quality of Experience by Adaptive Video Streaming with Super-Resolution. In IEEE INFOCOM 2020-IEEE Conference on Computer Communications. IEEE, 1957--1966.

Digital Library

Cited By

Fang HZhao HShi JZhang MWu GChou YWang FLiu JCai JKankanhalli MPrabhakaran BBoll SSubramanian RZheng LSingh VCesar PXie LXu D(2024)Robust Live Streaming over LEO Satellite Constellations: Measurement, Analysis, and Handover-Aware AdaptationProceedings of the 32nd ACM International Conference on Multimedia10.1145/3664647.3680712(5958-5966)Online publication date: 28-Oct-2024
https://dl.acm.org/doi/10.1145/3664647.3680712
Xu ZLi QShi WJiang YYuan ZZhang PMuntean GChua TNgo CKa-Wei Lee RKumar RLauw H(2024)NCTM: A Novel Coded Transmission Mechanism for Short Video DeliveriesProceedings of the ACM Web Conference 202410.1145/3589334.3645387(2847-2858)Online publication date: 13-May-2024
https://dl.acm.org/doi/10.1145/3589334.3645387
Hu KZou LJiang JMuntean G(2024)CoSPAM: Multi-Robot Collaboration Simultaneous Path Planning and Semantic Mapping2024 IEEE 99th Vehicular Technology Conference (VTC2024-Spring)10.1109/VTC2024-Spring62846.2024.10683290(1-7)Online publication date: 24-Jun-2024
https://doi.org/10.1109/VTC2024-Spring62846.2024.10683290
Show More Cited By

Index Terms

Higher quality live streaming under lower uplink bandwidth: an approach of super-resolution based video coding
1. Information systems
  1. Information systems applications
    1. Multimedia information systems
      1. Multimedia streaming
2. Networks
  1. Network algorithms
    1. Control path algorithms
      1. Network resources allocation

Recommendations

Neural-Enhanced Live Streaming: Improving Live Video Ingest via Online Learning
SIGCOMM '20: Proceedings of the Annual conference of the ACM Special Interest Group on Data Communication on the applications, technologies, architectures, and protocols for computer communication

Live video accounts for a significant volume of today's Internet video. Despite a large number of efforts to enhance user quality of experience (QoE) both at the ingest and distribution side of live video, the fundamental limitations are that streamer's ...
An SDN-aided low-latency live video streaming over HTTP
Abstract
Dynamic adaptive streaming over HTTP (DASH) is the crucial factor in the rapid penetration of over-the-top (OTT) service providers for on-demand video streaming. It can also be used for live video streaming by the OTT providers. The recent ...
Optimizing CDN Infrastructure for Live Streaming with Constrained Server Chaining
ISPA '11: Proceedings of the 2011 IEEE Ninth International Symposium on Parallel and Distributed Processing with Applications

Content Delivery Networks (CDNs) are increasingly being used to deliver live streaming on today's Internet. The new application type of live streaming exposes unique characteristics and challenges that require more advanced design of CDN infrastructure. ...

Comments

Information & Contributors

Information

Published In

cover image ACM Conferences

NOSSDAV '21: Proceedings of the 31st ACM Workshop on Network and Operating Systems Support for Digital Audio and Video

July 2021

128 pages

ISBN:9781450384353

DOI:10.1145/3458306

Program Chairs:
Andra Lutu
Telefonica, Spain
,
Gwendal Simon
Huawei, France
,
Mylene C. Q. Farias
University of Brasilia, Brazil

Copyright © 2021 ACM.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

Sponsors

SIGMM: ACM Special Interest Group on Multimedia

In-Cooperation

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 02 July 2021

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Research-article

Funding Sources

National Natural Science Foundation of China

Conference

MMSys '21

Sponsor:

SIGMM

MMSys '21: 12th ACM Multimedia Systems Conference

September 28 - October 1, 2021

Istanbul, Turkey

Acceptance Rates

NOSSDAV '21 Paper Acceptance Rate 15 of 52 submissions, 29%;

Overall Acceptance Rate 118 of 363 submissions, 33%

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

13
Total Citations
View Citations
544
Total Downloads

Downloads (Last 12 months)77
Downloads (Last 6 weeks)5

Reflects downloads up to 25 Jan 2025

Other Metrics

View Author Metrics

Citations

Cited By

Fang HZhao HShi JZhang MWu GChou YWang FLiu JCai JKankanhalli MPrabhakaran BBoll SSubramanian RZheng LSingh VCesar PXie LXu D(2024)Robust Live Streaming over LEO Satellite Constellations: Measurement, Analysis, and Handover-Aware AdaptationProceedings of the 32nd ACM International Conference on Multimedia10.1145/3664647.3680712(5958-5966)Online publication date: 28-Oct-2024
https://dl.acm.org/doi/10.1145/3664647.3680712
Xu ZLi QShi WJiang YYuan ZZhang PMuntean GChua TNgo CKa-Wei Lee RKumar RLauw H(2024)NCTM: A Novel Coded Transmission Mechanism for Short Video DeliveriesProceedings of the ACM Web Conference 202410.1145/3589334.3645387(2847-2858)Online publication date: 13-May-2024
https://dl.acm.org/doi/10.1145/3589334.3645387
Hu KZou LJiang JMuntean G(2024)CoSPAM: Multi-Robot Collaboration Simultaneous Path Planning and Semantic Mapping2024 IEEE 99th Vehicular Technology Conference (VTC2024-Spring)10.1109/VTC2024-Spring62846.2024.10683290(1-7)Online publication date: 24-Jun-2024
https://doi.org/10.1109/VTC2024-Spring62846.2024.10683290
Zhang DZhu HShen KWang DWang F(2024)DSJA: Distributed Server-Driven Joint Route Scheduling and Streaming Adaptation for Multi-Party Realtime Video StreamingIEEE Transactions on Mobile Computing10.1109/TMC.2023.333967123:7(7680-7694)Online publication date: Jul-2024
https://doi.org/10.1109/TMC.2023.3339671
Zhang LZhou HWang HCui L(2024)TBSR: Tile-Based 360° Video Streaming with Super-Resolution on Commodity Mobile DevicesIEEE INFOCOM 2024 - IEEE Conference on Computer Communications10.1109/INFOCOM52122.2024.10621078(501-510)Online publication date: 20-May-2024
https://doi.org/10.1109/INFOCOM52122.2024.10621078
Chen QWen WQin J(2024)GlobalSRNeural Networks10.1016/j.neunet.2024.106686180:COnline publication date: 1-Dec-2024
https://dl.acm.org/doi/10.1016/j.neunet.2024.106686
Dong PLi SGong XZhang L(2024)HVASR: Enhancing 360-Degree Video Delivery with Viewport-Aware Super ResolutionInformation Sciences10.1016/j.ins.2024.121609(121609)Online publication date: Nov-2024
https://doi.org/10.1016/j.ins.2024.121609
Yu QLi QHe RShi WJiang YDasari MJiang JGorlatova M(2023)RTCSR: Zero-latency Aware Super-resolution for WebRTC Mobile Video StreamingProceedings of the 2023 Workshop on Emerging Multimedia Systems10.1145/3609395.3610601(54-59)Online publication date: 10-Sep-2023
https://dl.acm.org/doi/10.1145/3609395.3610601
Wang FLi QShi WTyson GJiang YMa LZhang PLan YLi ZEl Saddik AMei TCucchiara RBertini MTobon Vallejo DAtrey PHossain M(2023)Reparo: QoE-Aware Live Video Streaming in Low-Rate Networks by Intelligent Frame RecoveryProceedings of the 31st ACM International Conference on Multimedia10.1145/3581783.3613441(9194-9204)Online publication date: 26-Oct-2023
https://dl.acm.org/doi/10.1145/3581783.3613441
Yu QLi QHe RTyson GShi WLv JYuan ZZhang PLan YLi Z(2023)BiSR: Bidirectionally Optimized Super-Resolution for Mobile Video StreamingProceedings of the ACM Web Conference 202310.1145/3543507.3583519(3121-3131)Online publication date: 30-Apr-2023
https://dl.acm.org/doi/10.1145/3543507.3583519
Show More Cited By

View Options

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Figures

Tables

Media

View Table of Conten