research-article

OASIS: Collaborative Neural-Enhanced Mobile Video Streaming

Authors:

Zhi-Li ZhangAuthors Info & Claims

MMSys '24: Proceedings of the 15th ACM Multimedia Systems Conference

Pages 45 - 55

https://doi.org/10.1145/3625468.3647610

Published: 17 April 2024 Publication History

Abstract

Neural-enhanced video streaming (e.g., super-resolution) is an ongoing revolution which can provide extremely high-quality video streaming services breaking the restriction of bandwidth. However, such enhancements require intense computation power that is not affordable for a single mobile device, which hinders their real-world deployment. To address the limitation, we propose OASIS, the first system that facilitates multiple users in close proximity to execute intense neural-enhanced video streaming in realtime. To this end, OASIS intelligently distributes computation tasks among multiple mobile devices, selects appropriate video bitrates and super-resolution models, and optimizes video chunk delivery. As a result, the expensive neural-enhanced streaming is done through distributed collaboration, achieving optimal quality of experience (QoE). We implement and evaluate OASIS on commodity smartphones from different vendors, under various network and computation conditions. Extensive experiments demonstrate the high efficiency of OASIS: it improves the video streaming QoE by 40%-200% and reduces each participant's energy consumption by 60% when the system scales up from a single device to six devices.

References

[1]

2012. MPEG-DASH. https://dashif.org/.

[2]

2019. Red Dead Redemption 2: Official Gameplay Video. https://www.youtube.com/watch?v=Dw_oH5oiUSE/.

[3]

2020. 57 Fascinating and Incredible YouTube Statistics. https://www.brandwat.ch.com/blog/youtube-stats/.

[4]

2022. NSD. https://developer.android.com/develop/connectivity/wifi.

[5]

2023. AI Benchmark Performance Ranking. https://ai-benchmark.com/ranking.html

[6]

2023. Expanding the Galaxy Ecosystem: Ultimate Connected Experiences Between Galaxy Devices. https://news.samsung.com/us/samsung-expanding-galaxy-ecosystem-ultimate-connected-experience-between-devices

[7]

2023. Huawei introduces a new era of cross-device collaboration with Super Device. https://consumer.huawei.com/sg/press/news/2022/news-220223/

[8]

2023. Use Continuity to work across Apple devices. https://support.apple.com/guide/mac-help/work-across-devices-using-continuity-mchl1d734309/mac

[9]

2023. Video Super-Resolution on MSU Video Super Resolution Benchmark: Detail Restoration. https://paperswithcode.com/sota/video-super-resolution-on-msu-vsr-benchmark

[10]

Kelvin CK Chan, Xintao Wang, Ke Yu, Chao Dong, and Chen Change Loy. 2021. BasicVSR: The search for essential components in video super-resolution and beyond. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. 4947--4956.

[11]

Byung-Gon Chun, Sunghwan Ihm, Petros Maniatis, Mayur Naik, and Ashwin Patti. 2011. Clonecloud: elastic execution between mobile device and cloud. In Proceedings of the sixth conference on Computer systems. 301--314.

Digital Library

[12]

Mallesham Dasari, Kumara Kahatapitiya, Samir R Das, Aruna Balasubramanian, and Dimitris Samaras. 2022. Swift: Adaptive video streaming with layered neural codecs. In 19th USENIX Symposium on Networked Systems Design and Implementation (NSDI 22). 103--118.

[13]

Lester Randolph Ford and Delbert R Fulkerson. 1956. Maximal flow through a network. Canadian journal of Mathematics 8 (1956), 399--404.

[14]

Karim Habak, Mostafa Ammar, Khaled A Harras, and Ellen Zegura. 2015. Femto clouds: Leveraging mobile devices to provide cloud service at the edge. In 2015 IEEE 8th international conference on cloud computing. IEEE, 9--16.

Digital Library

[15]

Ahmad Hassan, Arvind Narayanan, Anlan Zhang, Wei Ye, Ruiyang Zhu, Shuowei Jin, Jason Carpenter, Z Morley Mao, Feng Qian, and Zhi-Li Zhang. 2022. Vivisecting mobility management in 5G cellular networks. In Proceedings of the ACM SIGCOMM 2022 Conference. 86--100.

Digital Library

[16]

Jian He, Mubashir Adnan Qureshi, Lili Qiu, Jin Li, Feng Li, and Lei Han. 2018. Favor: Fine-grained video rate adaptation. In Proceedings of the 9th ACM Multimedia Systems Conference. 64--75.

Digital Library

[17]

Te-Yuan Huang, Ramesh Johari, Nick McKeown, Matthew Trunnell, and Mark Watson. 2014. A buffer-based approach to rate adaptation: Evidence from a large video streaming service. In Proceedings of the 2014 ACM conference on SIGCOMM. 187--198.

Digital Library

[18]

Andrey Ignatov, Andres Romero, Heewon Kim, and Radu Timofte. 2021. Realtime video super-resolution on smartphones with deep learning, mobile ai 2021 challenge: Report. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition. 2535--2544.

[19]

Junchen Jiang, Vyas Sekar, and Hui Zhang. 2012. Improving fairness, efficiency, and stability in http-based adaptive video streaming with festive. In Proceedings of the 8th international conference on Emerging networking experiments and technologies. 97--108.

Digital Library

[20]

Lorenzo Keller, Anh Le, Blerim Cici, Hulya Seferoglu, Christina Fragouli, and Athina Markopoulou. 2012. Microcast: Cooperative video streaming on smartphones. In Proceedings of the 10th international conference on Mobile systems, applications, and services. 57--70.

Digital Library

[21]

Jaehong Kim, Youngmok Jung, Hyunho Yeo, Juncheol Ye, and Dongsu Han. 2020. Neural-enhanced live streaming: Improving live video ingest via online learning. In Proceedings of the Annual conference of the ACM Special Interest Group on Data Communication on the applications, technologies, architectures, and protocols for computer communication. 107--125.

Digital Library

[22]

Kyu-Han Kim and Kang G Shin. 2005. Improving TCP performance over wireless networks with collaborative multi-homed mobile hosts. In Proceedings of the 3rd international conference on Mobile systems, applications, and services. 107--120.

Digital Library

[23]

Zhi Li, Xiaoqing Zhu, Joshua Gahm, Rong Pan, Hao Hu, Ali C Begen, and David Oran. 2014. Probe and adapt: Rate adaptation for HTTP video streaming at scale. IEEE Journal on Selected Areas in Communications 32, 4 (2014), 719--733.

[24]

Jingyun Liang, Jiezhang Cao, Yuchen Fan, Kai Zhang, Rakesh Ranjan, Yawei Li, Radu Timofte, and Luc Van Gool. 2022. Vrt: A video restoration transformer. arXiv preprint arXiv:2201.12288 (2022).

[25]

Hongzi Mao, Ravi Netravali, and Mohammad Alizadeh. 2017. Neural adaptive video streaming with pensieve. In Proceedings of the Conference of the ACM Special Interest Group on Data Communication. 197--210.

Digital Library

[26]

Arvind Narayanan, Xumiao Zhang, Ruiyang Zhu, Ahmad Hassan, Shuowei Jin, Xiao Zhu, Xiaoxuan Zhang, Denis Rybkin, Zhengxuan Yang, Zhuoqing Morley Mao, et al. 2021. A variegated look at 5G in the wild: performance, power, and QoE implications. In Proceedings of the 2021 ACM SIGCOMM 2021 Conference. 610--625.

Digital Library

[27]

Yunzhe Ni, Feng Qian, Taide Liu, Yihua Cheng, Zhiyao Ma, Jing Wang, Zhongfeng Wang, Gang Huang, Xuanzhe Liu, and Chenren Xu. 2023. {POLYCORN}: Data-driven Cross-layer Multipath Networking for High-speed Railway through Composable Schedulerlets. In 20th USENIX Symposium on Networked Systems Design and Implementation (NSDI 23). 1325--1340.

[28]

Cătălin Nicutar, Dragoş Niculescu, and Costin Raiciu. 2014. Using cooperation for low power low latency cellular connectivity. In Proceedings of the 10th ACM International on Conference on emerging Networking Experiments and Technologies. 337--348.

Digital Library

[29]

Sangeun Oh, Hyuck Yoo, Dae R Jeong, Duc Hoang Bui, and Insik Shin. 2017. Mobile plus: Multi-device mobile platform for cross-device functionality sharing. In Proceedings of the 15th Annual International Conference on Mobile Systems, Applications, and Services. 332--344.

Digital Library

[30]

Devdeep Ray, Jack Kosaian, KV Rashmi, and Srinivasan Seshan. 2019. Vantage: optimizing video upload for time-shifted viewing of social live streams. In Proceedings of the ACM Special Interest Group on Data Communication. 380--393.

Digital Library

[31]

Susanna Schwarzmann, Clarissa Cassales Marquezan, Marcin Bosk, Huiran Liu, Riccardo Trivisonno, and Thomas Zinner. 2019. Estimating video streaming QoE in the 5G architecture using machine learning. In Proceedings of the 4th Internet-QoE Workshop on QoE-based Analysis and Management of Data Communication Networks. 7--12.

Digital Library

[32]

Susanna Schwarzmann, Clarissa Cassales Marquezan, Riccardo Trivisonno, Shinichi Nakajima, Vincent Barriac, and Thomas Zinner. 2022. Ml-based qoe estimation in 5g networks using different regression techniques. IEEE Transactions on Network and Service Management 19, 3 (2022), 3516--3532.

[33]

Ashish Sharma, Vishnu Navda, Ramachandran Ramjee, Venkata N Padmanabhan, and Elizabeth M Belding. 2009. Cool-tether: energy efficient on-the-fly wifi hotspots using mobile phones. In Proceedings of the 5th international conference on Emerging networking experiments and technologies. 109--120.

Digital Library

[34]

Kevin Spiteri, Rahul Urgaonkar, and Ramesh K Sitaraman. 2020. BOLA: Near-optimal bitrate adaptation for online videos. IEEE/ACM Transactions on Networking 28, 4 (2020), 1698--1711.

Digital Library

[35]

Yi Sun, Xiaoqi Yin, Junchen Jiang, Vyas Sekar, Fuyuan Lin, Nanshu Wang, Tao Liu, and Bruno Sinopoli. 2016. CS2P: Improving video bitrate selection and adaptation with data-driven throughput prediction. In Proceedings of the 2016 ACM SIGCOMM Conference. 272--285.

Digital Library

[36]

Xuedou Xiao, Wei Wang, Taobin Chen, Yang Cao, Tao Jiang, and Qian Zhang. 2019. Sensor-augmented neural adaptive bitrate video streaming on UAVs. IEEE Transactions on Multimedia 22, 6 (2019), 1567--1576.

[37]

Xiufeng Xie, Xinyu Zhang, Swarun Kumar, and Li Erran Li. 2015. piStream: Physical layer informed adaptive video streaming over LTE. In Proceedings of the 21st Annual International Conference on Mobile Computing and Networking. 413--425.

Digital Library

[38]

Shichang Xu, Subhabrata Sen, Z Morley Mao, and Yunhan Jia. 2017. Dissecting VOD services for cellular: performance, root causes and best practices. In Proceedings of the 2017 Internet Measurement Conference. 220--234.

Digital Library

[39]

Francis Y Yan, Hudson Ayers, Chenzhi Zhu, Sadjad Fouladi, James Hong, Keyi Zhang, Philip Alexander Levis, and Keith Winstein. 2020. Learning in situ: a randomized experiment in video streaming. In NSDI, Vol. 20. 495--511.

[40]

Hyunho Yeo, Chan Ju Chong, Youngmok Jung, Juncheol Ye, and Dongsu Han. 2020. NEMO: enabling neural-enhanced video streaming on commodity mobile devices. In Proceedings of the 26th Annual International Conference on Mobile Computing and Networking. 1--14.

Digital Library

[41]

Hyunho Yeo, Youngmok Jung, Jaehong Kim, Jinwoo Shin, and Dongsu Han. 2018. Neural adaptive content-aware internet video delivery. In 13th USENIX Symposium on Operating Systems Design and Implementation (OSDI 18). 645--661.

[42]

Hyunho Yeo, Hwijoon Lim, Jaehong Kim, Youngmok Jung, Juncheol Ye, and Dongsu Han. 2022. NeuroScaler: neural video enhancement at scale. In Proceedings of the ACM SIGCOMM 2022 Conference. 795--811.

Digital Library

[43]

Xiaoqi Yin, Abhishek Jindal, Vyas Sekar, and Bruno Sinopoli. 2015. A control-theoretic approach for dynamic adaptive video streaming over HTTP. In Proceedings of the 2015 ACM Conference on Special Interest Group on Data Communication. 325--338.

Digital Library

[44]

Zhengdong Zhang and Vivienne Sze. 2017. FAST: A framework to accelerate super-resolution processing on compressed videos. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition Workshops. 19--28.

[45]

Xiao Zhu, Jiachen Sun, Xumiao Zhang, Y Ethan Guo, Feng Qian, and Z Morley Mao. 2020. MPBond: efficient network-level collaboration among personal mobile devices. In Proceedings of the 18th International Conference on Mobile Systems, Applications, and Services. 364--376.

Digital Library

[46]

Xuan Kelvin Zou, Jeffrey Erman, Vijay Gopalakrishnan, Emir Halepovic, Rittwik Jana, Xin Jin, Jennifer Rexford, and Rakesh K Sinha. 2015. Can accurate predictions improve video streaming in cellular networks?. In Proceedings of the 16th International Workshop on Mobile Computing Systems and Applications. 57--62.

Digital Library

Cited By

Zhu RZhu XZhang AZhang XSun JQian FQiu HMao ZLee MShu YLiu JTan RHe YChen J(2024)Boosting Collaborative Vehicular Perception on the Edge with Vehicle-to-Vehicle CommunicationProceedings of the 22nd ACM Conference on Embedded Networked Sensor Systems10.1145/3666025.3699328(141-154)Online publication date: 4-Nov-2024
https://dl.acm.org/doi/10.1145/3666025.3699328

Index Terms

OASIS: Collaborative Neural-Enhanced Mobile Video Streaming

Recommendations

NeuroScaler: neural video enhancement at scale
SIGCOMM '22: Proceedings of the ACM SIGCOMM 2022 Conference

High-definition live streaming has experienced tremendous growth. However, the video quality of live video is often limited by the streamer's uplink bandwidth. Recently, neural-enhanced live streaming has shown great promise in enhancing the video ...
Neural-Enhanced Live Streaming: Improving Live Video Ingest via Online Learning
SIGCOMM '20: Proceedings of the Annual conference of the ACM Special Interest Group on Data Communication on the applications, technologies, architectures, and protocols for computer communication

Live video accounts for a significant volume of today's Internet video. Despite a large number of efforts to enhance user quality of experience (QoE) both at the ingest and distribution side of live video, the fundamental limitations are that streamer's ...
NEMO: enabling neural-enhanced video streaming on commodity mobile devices
MobiCom '20: Proceedings of the 26th Annual International Conference on Mobile Computing and Networking

The demand for mobile video streaming has experienced tremendous growth over the last decade. However, existing methods of video delivery fall short of delivering high-quality video. Recent advances in neural super-resolution have opened up the ...

Comments

Information & Contributors

Information

Published In

cover image ACM Conferences

MMSys '24: Proceedings of the 15th ACM Multimedia Systems Conference

April 2024

557 pages

ISBN:9798400704123

DOI:10.1145/3625468

Copyright © 2024 ACM.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than the author(s) must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected].

Sponsors

SIGMM: ACM Special Interest Group on Multimedia

In-Cooperation

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 17 April 2024

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Badges

Best Paper

Author Tags

Qualifiers

Research-article
Research
Refereed limited

Funding Sources

NSF (National Science Foundation)

Conference

MMSys '24

Sponsor:

SIGMM

MMSys '24: ACM Multimedia Systems Conference 2024

April 15 - 18, 2024

Bari, Italy

Acceptance Rates

Overall Acceptance Rate 176 of 530 submissions, 33%

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

1
Total Citations
View Citations
330
Total Downloads

Downloads (Last 12 months)330
Downloads (Last 6 weeks)13

Reflects downloads up to 13 Jan 2025

Other Metrics

View Author Metrics

Citations

Cited By

Zhu RZhu XZhang AZhang XSun JQian FQiu HMao ZLee MShu YLiu JTan RHe YChen J(2024)Boosting Collaborative Vehicular Perception on the Edge with Vehicle-to-Vehicle CommunicationProceedings of the 22nd ACM Conference on Embedded Networked Sensor Systems10.1145/3666025.3699328(141-154)Online publication date: 4-Nov-2024
https://dl.acm.org/doi/10.1145/3666025.3699328

View Options

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Media

Figures

Other

Tables

View Table of Contents