A Salience & Motion State Based Quality Adjustment for 360-Degree Video Transmission
Authors: 
Nan Zong, Jialiang Lu, Yutong Li, Zhenyu XieAuthors Info & Claims
Nan Zong, Jialiang Lu, Yutong Li, Zhenyu XieAuthors Info & ClaimsPages 43 - 48
Abstract
The rapid development of Virtual Reality technology and the video sharing market of VOD (video-on-demand) stream nowadays makes online 360-degree video a new form of internet media. In the past few years, many 360-degree video delivery schemes are proposed. Yet, high fps requires a very high network band, otherwise will suffer from significant video quality drop. In this paper, we are focusing on using salient object detection and frame motion state detection for quality adjustment to reset 360-degree image quality of salient foreground and unimportant background. The system predicts the salient portion of video and then separate one frame in to foreground frame and back ground frame. Then by combining MS SSIM (Modified Structural Similarity of Images) algorithm, it also detects important frames and estimate the motion state of each view. The following procedure includes methods which lower the quality of dynamic background view and meanwhile maintain the high quality of foreground. This stage separates a 360-degree video into two videos with a much smaller file size of the background video. The core contribution of the scheme is that it is able to detect salient foreground and less noticed dynamic background of VR graphics. As a result in the transmission of the VR graphics, the network can execute a parallel transmission of a foreground and a economized background, costing less network resources.The results prove that this proposed scheme can effectively reduce global transmission bandwidth of the network with a wide adaptation of all genres of 360-degree videos.
References
[1]
Ming Tang, Vincent W.S. Wong. Online Bitrate Selection for Viewport Adaptive 360-Degree Video Streaming. arXiv:2005.02479(cs.NI), 2020.
[2]
Te-Yuan Huang, Ramesh Johari, Nick McKeown, Matthew Trunnell, and Mark Watson. A buffer-based approach to rate adaptation: Evidence from a large video streaming service. ACM SIGCOMM, 2014.
[3]
Yuhang Song, Mai Xu, Jianyi Wang, Minglang Qiao, Liangyu Huo. Predicting Head Movement in Panoramic Video: A Deep Reinforcement Learning Approach. arXiv: 1710.10755 [cs.CV], IEEE Transactions on Pattern Analysis and Machine Intelligence, Jul 24, 2018.
[4]
Mu Mu, Murtada Dohan, Alison Goodyear, Gary Hill, Cleyon Johns, Andreas Mauthe. User Attention and Behaviour in Virtual Reality Art Encounter. arXiv:2005.10161 [cs.HC], 2020.
[5]
Shuang Li, Peter Mathews. Can Image Retrieval help Visual Saliency Detection?. arXiv: 1709.08172 [cs.CV], 2017.
[6]
Ninh Dang Tran, Hans-Jürgen Zepernick. Spherical Lightweight Data Hiding in 360-Degree Videos With Equirectangular. In IEEE International Conference on Advanced Technologies for Communications, Hanoi, Vietnam, 2019.
[7]
Wei Zeng, Mingqiang Yang, and Zhenxing Cui, "Ultra-Low Bit Rate Facial Coding Hybrid Model Based on Saliency Detection," Journal of Image and Graphics, Vol. 3, No. 1, pp. 25--29, June 2015.
[8]
Nian Liu, Junwei Han, Ming-Hsuan Yang. PiCANet: Learning Pixel-wise Contextual Attention for Saliency Detection. arXiv: 1708.06433 [cs.CV], 2018.
[9]
Xuebin Qin, Zichen Zhang, Chenyang Huang, Chao Gao, Masood Dehghan, Martin Jagersand. BASNet: Boundary-Aware Salient Object Detection. In IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), Long Beach, CA, USA, 2019.
[10]
Zhe Wu, Li Su, Qingming Huang. Stacked Cross Refinement Network for Edge-Aware Salient Object Detection. In IEEE/CVF International Conference on Computer Vision (ICCV), Seoul, Korea, 2019.
[11]
Yusra Al-Najjar. Comparison of Image Quality Assessment: PSNR, HVS, SSIM, UIQI. International Journal of Scientific & Engineering Research, 2012.
[12]
Zhou Wang, Alan Bovik, Hamid Rahim Sheikh, Eero P. Simoncelli. Image Quality Assessment: From Error Visibility to Structural Similarity. In IEEE Transactions on Image Processing 13(4):600--612, May 2004.
[13]
Z. Wang, E.P. Simoncelli, A.C. Bovik. Multiscale structural similarity for image quality assessment. In the Thrity-Seventh Asilomar Conference on Signals, Systems & Computers, IEEE, Pacific Grove, CA, USA, 2003.
[14]
Yi Zhang, Lu Zhang, Wassim Hamidouche, Olivier Deforges. A FIXATION-BASED 360° BENCHMARK DATASET FOR SALIENT OBJECT DETECTION. In ICIP 2020. arXiv:2001.07960 [cs.CV], 2020.
Index Terms
- A Salience & Motion State Based Quality Adjustment for 360-Degree Video Transmission
Recommendations
Saliency-driven rate-distortion optimization for 360-degree image coding
Abstract360-degree images allow an immersive experience. They offer multiple views of a scene and the viewpoint can be selected by the user. However, the huge amount of data that is necessary for real-time transmission of 360-degree image and video ...
Immersive $$360^{\circ }$$360? video user experience: impact of different variables in the sense of presence and cybersickness
Virtual Reality (VR) has been recently gaining interest from researchers and companies, contributing to the development of the associated technologies that aim to transport its users to a virtual environment by the stimulation of their senses. ...
A survey on 360-degree video: Coding, quality of experience and streaming
AbstractThe commercialization of Virtual Reality (VR) headsets has made immersive and 360-degree video streaming the subject of intense interest in the industry and research communities. While the basic principles of video streaming are the ...
Comments
Information & Contributors
Information
Published In
August 2020
108 pages
ISBN:9781450375504
DOI:10.1145/3421537
Copyright © 2020 ACM.
Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]
Publisher
Association for Computing Machinery
New York, NY, United States
Publication History
Published: 05 October 2020
Check for updates
Author Tags
Qualifiers
- Research-article
- Research
- Refereed limited
Conference
BDIOT 2020
BDIOT 2020: 2020 the 4th International Conference on Big Data and Internet of Things
August 22 - 24, 2020
Singapore, Singapore
Acceptance Rates
Overall Acceptance Rate 75 of 136 submissions, 55%
Contributors
Other Metrics
Bibliometrics & Citations
Bibliometrics
Article Metrics
- 0Total Citations
- 49Total Downloads
- Downloads (Last 12 months)5
- Downloads (Last 6 weeks)0
Reflects downloads up to 12 Sep 2024
Other Metrics
Citations
View Options
Get Access
Login options
Check if you have access through your login credentials or your institution to get full access on this article.
Sign in