research-article

Depth Personalization and Streaming of Stereoscopic Sports Videos

Authors:

Kiana Calagari,

Krzysztof Templin,

Wojciech Matusik,

Mohamed HefeedaAuthors Info & Claims

ACM Transactions on Multimedia Computing, Communications, and Applications (TOMM), Volume 12, Issue 3

Article No.: 41, Pages 1 - 23

https://doi.org/10.1145/2890103

Published: 08 March 2016 Publication History

Abstract

Current three-dimensional displays cannot fully reproduce all depth cues used by a human observer in the real world. Instead, they create only an illusion of looking at a three-dimensional scene. This leads to a number of challenges during the content creation process. To assure correct depth reproduction and visual comfort, either the acquisition setup has to be carefully controlled or additional postprocessing techniques have to be applied. Furthermore, these manipulations need to account for a particular setup that is used to present the content, for example, viewing distance or screen size. This creates additional challenges in the context of personal use when stereoscopic content is shown on TV sets, desktop monitors, or mobile devices. We address this problem by presenting a new system for streaming stereoscopic content. Its key feature is a computationally efficient depth adjustment technique which can automatically optimize viewing experience for videos of field sports such as soccer, football, and tennis. Additionally, the method enables depth personalization to allow users to adjust the amount of depth according to their preferences. Our stereoscopic video streaming system was implemented, deployed, and tested with real users.

References

[1]

3 DeeCentral. 2015. 3DeeCentral Web site. Retrieved from http://www.3deecentral.com/.

[2]

3 DVisionLive. 2015. 3DVisionLive Web site. Retrieved from https://www.3dvisionlive.com/.

[3]

Thomas Brox, Andrés Bruhn, Nils Papenberg, and Joachim Weickert. 2004. High accuracy optical flow estimation based on a theory for warping. In Proc. of European Conference on Computer Vision (ECCV’04). 25--36.

[4]

Peter Burt and Bela Julesz. 1980. Modifications of the classical notion of Panum’s fusional area. Perception 9, 6 (1980), 671--682.

[5]

Kiana Calagari, Krzysztof Templin, Tarek Elgamal, Khaled Diab, Piotr Didyk, Wojciech Matusik, and Mohamed Hefeeda. 2014. Anahita: A system for 3d video streaming with depth customization. In Proc. of the ACM International Conference on Multimedia (ACM MM’14). 337--346.

Digital Library

[6]

Pablo Carballeira, Julián Cabrera, Antonio Ortega, Fernando Jaureguizar, and Narciso García. 2012. A framework for the analysis and optimization of encoding latency for multiview video. IEEE J. Select. Top. Signal Process. 6, 5 (2012), 583--596.

[7]

Ben E. Coutant and Gerald Westheimer. 1993. Population distribution of stereoscopic ability. Ophthal. Physiol. Opt. 13, 1 (1993), 3--7.

[8]

Khaled Diab, Tarek Elgamal, Kiana Calagari, and Mohamed Hefeeda. 2014. Storage optimization for 3d streaming systems. In Proc. of ACM Conference on Multimedia Systems (MMSys’14). 59--69.

Digital Library

[9]

Piotr Didyk, Tobias Ritschel, Elmar Eisemann, Karol Myszkowski, and Hans-Peter Seidel. 2011. A perceptual model for disparity. ACM Trans. Graphics 30, 4 (2011), 96:1--96:9.

Digital Library

[10]

Displaybank Co. 2010. 3D TV industry trend and market forecast. Special Report. Retrieved from http://www.displaybank.com/research_file/710527.pdf.

[11]

Ahmet Ekin, A. Murat Tekalp, and Rajiv Mehrotra. 2003. Automatic soccer video analysis and summarization. IEEE Transactions on Image Processing 12, 7 (2003), 796--807.

Digital Library

[12]

Christoph Fehn. 2003. A 3d-tv system based on video plus depth information. In Asilomar Conference on Signals, Systems and Computers 2, 1529--1533.

[13]

Jonathan Freeman and Steve E. Avons. 2000. Focus group exploration of presence through advanced broadcast services. In Proc. of SPIE Human Vision and Electronic Imaging Conference. 530--539.

[14]

C. Göktuğ Gurler, Burak Görkemli, Görkem Saygili, and others. 2011. Flexible transport of 3-d video over networks. Proc. IEEE 99, 4 (2011), 694--707.

[15]

Ahmed Hamza and Mohamed Hefeeda. 2014. A DASH-based free-viewpoint video streaming system. In Proc. of ACM NOSSDAV’14 Workshop, in Conjunction with ACM Multimedia Systems (MMSys’14) Conference. 55--60.

Digital Library

[16]

Nicolas S. Holliman. 2004. Mapping perceived depth to regions of interest in stereoscopic images. In Proc. of SPIE Stereoscopic Displays and Virtual Reality Systems Conference. 117--128.

[17]

ISO/IEC 23009-1:2012. 2012. Information technology -- Dynamic adaptive streaming over HTTP (DASH) -- Part 1: Media presentation description and segment formats.

[18]

ITU-R BT.2021. 2012. Subjective methods for the assessment of stereoscopic 3D TV systems. International Telecommunication Union, Geneva, Switzerland.

[19]

Mathias Johanson. 2001. Stereoscopic video transmission over the internet. In Proc. of IEEE Workshop on Internet Applications (WIAPP’01). 12--19.

Digital Library

[20]

Hideaki Kimata, Katsuhiko Fukazawa, Akio Kameda, Yoshie Yamaguchi, and Norihiko Matsuura. 2011. Interactive 3D multi-angle live streaming system. In Proc. of IEEE International Symposium on Consumer Electronics (ISCE’01). 576--579.

[21]

Manuel Lang, Alexander Hornung, Oliver Wang, Steven Poulakos, Aljoscha Smolic, and Markus Gross. 2010. Nonlinear disparity mapping for stereoscopic 3D. ACM Trans. Graphics 29, 3 (2010), 75:1--75:10.

Digital Library

[22]

Jian-Guang Lou, Hua Cai, and Jiang Li. 2005. A real-time interactive multi-view video system. In Proc. of ACM International Conference on Multimedia (ACMMM’05). 161--170.

Digital Library

[23]

Bernard Mendiburu. 2012. 3D Movie Making: Stereoscopic Digital Cinema from Script to Screen. CRC Press.

[24]

Thomas Oskam, Alexander Hornung, Huw Bowles, Kenny Mitchell, and Markus Gross. 2011. OSCAM - optimized stereoscopic camera control for interactive 3D. ACM Trans. Graphics 30, 6 (2011), 189:1--189:8.

Digital Library

[25]

Dawid Pajak, Robert Herzog, Radosław Mantiuk, Piotr Didyk, Elmar Eisemann, Karol Myszkowski, and Kari Pulli. 2014. Perceptual depth compression for stereo applications. Comput. Graphics Forum 33, 2 (2014), 195--204.

Digital Library

[26]

Sylvain Paris and Frédo Durand. 2009. A fast approximation of the bilateral filter using a signal processing approach. Int. J. Comput. Vision 81, 1 (2009), 24--52.

Digital Library

[27]

Selen Pehlivan, Anil Aksay, Cagdas Bilen, Gozde Bozdagi Akar, and M. Reha Civanlar. 2006. End-to-end stereoscopic video streaming system. In Proc. of IEEE International Conference on Multimedia and Expo (ICME’06). 2169--2172.

[28]

André Redert, Marc Op de Beeck, Christoph Fehn, Wijnand Jsselsteijn, Marc Pollefeys, Luc Van Gool, Eyal Ofek, Ian Sexton, and Philip Surman. 2002. ATTEST: Advanced three-dimensional television system technologies. In Proc. of International Symposium on 3D Data Processing Visualization and Transmission. 313--319.

[29]

Takashi Shibata, Joohwan Kim, David M. Hoffman, and Martin S. Banks. 2011. The zone of comfort: Predicting visual discomfort with stereo displays. J. Vision 11, 8 (2011), 11:1--11:29.

[30]

Aljoscha Smolic, Peter Kauff, Sebastian Knorr, Alexander Hornung, Matthias Kunter, Marcus Mller, and Manuel Lang. 2011. Three-dimensional video postproduction and processing. Proc. IEEE 99, 4 (2011), 607--625.

[31]

Iraj Sodagar. 2011. The MPEG-DASH standard for multimedia streaming over the internet. IEEE Multimed. Mag. 18, 4 (2011), 62--67.

Digital Library

[32]

Thomas Stockhammer. 2011. Dynamic adaptive streaming over HTTP: Standards and design principles. In Proc. of ACM Conference on Multimedia Systems (MMSys’11). 133--144.

Digital Library

[33]

Wa James Tam, Filippo Speranza, Sumio Yano, Koichi Shimono, and Hiroshi Ono. 2011. Stereoscopic 3D-tv: Visual comfort. IEEE Trans. Broadcast. 57, 2 (2011), 335--346.

[34]

Jonathan R. Thorpe and Mark J. Russell. 2011. Perceptual effects when scaling screen size of stereo 3D presentations. In Proc. of Society of Motion Picture and Television Engineers Conferences (SMPTE’11). 1--10.

[35]

Trivido. 2015. Trivido Web site. Retrieved from http://www.trivido.com/.

[36]

Ugo Capeto. 2013. Depth Map Automatic Generator (DMAG). Retrieved from http://3dstereophoto.blogspot.com/2013/04/depth-map-automatic-generator-dmag.html.

[37]

Anthony Vetro, Thomas Wiegand, and Gary J. Sullivan. 2011. Overview of the stereo and multiview video coding extensions of the H. 264/MPEG-4 AVC standard. Proc. IEEE 99, 4 (2011), 626--642.

[38]

Wanmin Wu, Ahsan Arefin, Gregorij Kurillo, Pooja Agarwal, Klara Nahrstedt, and Ruzena Bajcsy. 2011. Color-plus-depth level-of-detail in 3D tele-immersive video: A psychophysical approach. In Proc. of ACM International Conference on Multimedia (ACM MM’11). 13--22.

Digital Library

[39]

Baicheng Xin, Ronggang Wang, Zhenyu Wang, Wenmin Wang, Chenchen Gu, Quanzhan Zheng, and Wen Gao. 2012. AVS 3D video streaming system over internet. In Proc. of IEEE International Conference on Signal Processing, Communication and Computing (ICSPCC’12). 286--289.

[40]

Qingxiong Yang, Liang Wang, and Narendra Ahuja. 2010. A constant-space belief propagation algorithm for stereo matching. In Proc. of IEEE Conference on Computer Vision and Pattern Recognition (CVPR’10). 1458--1465.

[41]

YouTube. Retrieved from http://www.youtube.com/.

Cited By

Liu SChen Q(2022)College Sports Multimedia Network Based on Wireless Communication and Deep Learning Network EnvironmentWireless Communications & Mobile Computing10.1155/2022/32676392022Online publication date: 1-Jan-2022
https://dl.acm.org/doi/10.1155/2022/3267639
Cánovas ATaha MLloret JTomas J(2019)A cognitive network management system to improve QoE in stereoscopic IPTV serviceInternational Journal of Communication Systems10.1002/dac.399232:12Online publication date: 17-May-2019
https://doi.org/10.1002/dac.3992

Index Terms

Depth Personalization and Streaming of Stereoscopic Sports Videos
1. Computing methodologies
  1. Computer graphics
    1. Image manipulation
      1. Image processing
2. Information systems
  1. Information systems applications
    1. Multimedia information systems
      1. Multimedia streaming

Recommendations

Gradient-based 2D-to-3D Conversion for Soccer Videos
MM '15: Proceedings of the 23rd ACM international conference on Multimedia

A wide spread adoption of 3D videos and technologies is hindered by the lack of high-quality 3D content. One promising solution to address this problem is to use automated 2D-to-3D conversion. However, current conversion methods, while general, produce ...
Anahita: A System for 3D Video Streaming with Depth Customization
MM '14: Proceedings of the 22nd ACM international conference on Multimedia

Producing high-quality stereoscopic 3D content requires significantly more effort than preparing regular video footage. In order to assure good depth perception and visual comfort, 3D videos need to be carefully adjusted to specific viewing conditions ...
Nonlinear disparity mapping for stereoscopic 3D

This paper addresses the problem of remapping the disparity range of stereoscopic images and video. Such operations are highly important for a variety of issues arising from the production, live broadcast, and consumption of 3D content. Our work is ...

Comments

Information & Contributors

Information

Published In

cover image ACM Transactions on Multimedia Computing, Communications, and Applications

ACM Transactions on Multimedia Computing, Communications, and Applications Volume 12, Issue 3

June 2016

227 pages

ISSN:1551-6857

EISSN:1551-6865

DOI:10.1145/2901366

Editor:
Alberto Del Bimbo
University of Firenze, Italy

Issue’s Table of Contents

Copyright © 2016 ACM.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 08 March 2016

Accepted: 01 January 2016

Revised: 01 January 2016

Received: 01 June 2015

Published in TOMM Volume 12, Issue 3

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Research-article
Research
Refereed

Funding Sources

Qatar Computing Research Institute (QCRI)

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

2
Total Citations
View Citations
254
Total Downloads

Downloads (Last 12 months)7
Downloads (Last 6 weeks)0

Reflects downloads up to 16 Oct 2024

Other Metrics

View Author Metrics

Citations

Cited By

Liu SChen Q(2022)College Sports Multimedia Network Based on Wireless Communication and Deep Learning Network EnvironmentWireless Communications & Mobile Computing10.1155/2022/32676392022Online publication date: 1-Jan-2022
https://dl.acm.org/doi/10.1155/2022/3267639
Cánovas ATaha MLloret JTomas J(2019)A cognitive network management system to improve QoE in stereoscopic IPTV serviceInternational Journal of Communication Systems10.1002/dac.399232:12Online publication date: 17-May-2019
https://doi.org/10.1002/dac.3992

View Options

Get Access

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Article

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Media

Figures

Other

Tables

View Issue’s Table of Contents