research-article

Dynamic 3D point cloud streaming: distortion and concealment

Authors:

Cheng-Hsin HsuAuthors Info & Claims

NOSSDAV '21: Proceedings of the 31st ACM Workshop on Network and Operating Systems Support for Digital Audio and Video

Pages 98 - 105

https://doi.org/10.1145/3458306.3458876

Published: 02 July 2021 Publication History

Abstract

We present a study on the impact of packet loss on dynamic 3D point cloud streaming, encoded with MPEG Video-based Point Cloud Compression (V-PCC) standard. We show the distortion when different channels of V-PCC bitstream are lost, with the loss of occupancy and geometry data impacting the quality most significantly. Our results point to the need for better error concealment techniques. We end the paper by presenting preliminary thoughts and experimental results of two naive error concealment techniques in the point cloud domain, for attributes and geometry data, respectively, and highlight the limitations of each.

References

[1]

Ghassan AlRegib and Yucel Altunbasak. 2005. 3TP: An application-Layer protocol for streaming 3-D models. IEEE Transactions on Multimedia 7, 6 (2005), 1149--1156.

Digital Library

[2]

Jill Boyce and Robert Gaglianello. 1998. Packet loss effects on MPEG video sent over the public Internet. In Proc. of ACM International Conference on Multimedia (MM'98). Bristol, UK, 181--190.

Digital Library

[3]

Bing-Yu Chen and Tomoyuki Nishita. 2002. Multiresolution streaming mesh with shape preserving and QoS-like controlling. In Proc. of ACM International Conference on 3D Web Technology (Web3D'02). Tempe, Arizona, 35--42.

Digital Library

[4]

Wei Cheng and Wei Tsang Ooi. 2008. Receiver-driven view-dependent streaming of progressive mesh. In Proc. of ACM International Workshop on Network and Operating Systems Support for Digital Audio and Video (NOSSDAV'08). Braunschweig, Germany, 9--14.

Digital Library

[5]

Alvaro Collet, Ming Chuang, Pat Sweeney, Don Gillett, Dennis Evseev, David Calabrese, Hugues Hoppe, Adam Kirk, and Steve Sullivan. 2015. High-quality streamable free-viewpoint video. ACM Transactions on Graphics 34, 4 (2015), 1--13.

Digital Library

[6]

Thomas Forgione, Axel Carlier, Géraldine Morin, Wei Tsang Ooi, Vincent Charvillat, and Praveen Kumar Yadav. 2018. DASH for 3D networked virtual environment. In Proc. of ACM International Conference on Multimedia (MM'18). Seoul, Republic of Korea, 1910--1918.

Digital Library

[7]

Zeqing Fu, Wei Hu, and Zongming Guo. 2020. 3D dynamic point cloud inpainting via temporal consistency On graphs. In Proc. of IEEE International Conference on Multimedia and Expo (ICME'20). London, UK, 1--6.

[8]

D Graziosi, O Nakagami, S Kuma, A Zaghetto, T Suzuki, and A Tabatabai. 2020. An overview of ongoing point cloud compression standardization activities: Video-based (V-PCC) and geometry-based (G-PCC). APSIPA Transactions on Signal and Information Processing 9 (2020), e13.

[9]

André Guéziec, Gabriel Taubin, Bill Horn, and Francis Lazarus. 1999. A framework for streaming geometry in VRML. IEEE Computer Graphics and Applications 19, 2 (1999), 68--78.

Digital Library

[10]

Ju He, Zeqing Fu, Wei Hu, and Zongming Guo. 2019. Point cloud attribute inpainting in graph spectral domain. In Proc. of IEEE International Conference on Image Processing (ICIP'19). Taipei, Taiwan, 4385--4389.

[11]

Sahar Hojati, Mohammad Kazemi, and Payman Moallem. 2020. Error concealment with parallelogram partitioning of the lost area. Springer Multimedia Tools and Applications 2020, 79 (2020), 7449--7469.

[12]

Shun Yun Hu, Ting Hao Huang, Shao Chen Chang, Wei Lun Sung, Jehn Ruey Jiang, and Bing Yu Chen. 2008. FLoD: A framework for peer-to-peer 3D streaming. In Proc. of IEEE Conference on Computer Communications (INFOCOM'08). Phoenix, AZ, 1373--1381.

[13]

Wei Hu, Zeqing Fu, and Zongming Guo. 2019. Local frequency interpretation and non-local self-similarity on graph for point cloud inpainting. IEEE Transactions on Image Processing 28, 8 (2019), 4087--4100.

[14]

Byongsu Hwang, Jonghyon Jo, and Cholsu Ri. 2019. An improved multi-directional interpolation for spatial error concealment. Springer Multimedia Tools and Applications 78, 2 (2019), 2587--2598.

Digital Library

[15]

Euee S. Jang, Marius Preda, Khaled Mammou, Alexis M. Tourapis, Jungsun Kim, Danillo B. Graziosi, Sungryeul Rhyu, and Madhukar Budagavi. 2019. Video-based point-cloud-compression standard in MPEG: From evidence collection to committee draft [Standards in a nutshell]. IEEE Signal Processing Magazine 36, 3 (2019), 118--123.

[16]

Junsik Kim, Jiheon Im, Sungryeul Rhyu, and Kyuheon Kim. 2020. 3D motion estimation and compensation method for video-based point cloud compression. IEEE Access 8 (2020), 83538--83547.

[17]

Li Li, Zhu Li, Vladyslav Zakharchenko, Jianle Chen, and Houqiang Li. 2020. Advanced 3D motion prediction for video-based dynamic point cloud compression. IEEE Transactions on Image Processing 29 (2020), 289--302.

Digital Library

[18]

Ketan Mayer-Patel, Long Le, and Georg Carle. 2002. An MPEG performance model and its application to adaptive forward error correction. In Proc. of ACM International Conference on Multimedia (MM'02). Juan les Pins, France, 1--10.

Digital Library

[19]

MPEG. 2021. MPEG Datasets. http://mpegfs.int-evry.fr/MPEG/PCC/DataSets/pointCloud/CfP/datasets/.

[20]

MPEG/N0038 2020. Common Test Conditions for V3C and V-PCC. Document. ISO/IEC JTC1/SC29/WG7 MPEG 3D Graphics Coding. Meeting held online.

[21]

MPEG/N19579 2020. Text of ISO/IEC FDIS 23090-5 Visual Volumetric Video-based Coding and Video-based Point Cloud Compression. Document. ISO/IEC JTC1/SC29/WG11 MPEG 3DG. Meeting held online.

[22]

Mu Mu, Roswitha Gostner, Andreas Mauthe, Gareth Tyson, and Francisco Garcia. 2009. Visibility of individual packet loss on H.264 encoded video stream: A user study on the impact of packet loss on perceived video quality. In Multimedia Computing and Networking 2009, Vol. 7253. International Society for Optics and Photonics, San Jose, CA, 725302.

[23]

Szymon Rusinkiewicz and Marc Levoy. 2001. Streaming QSplat: A viewer for networked visualization of large, dense models. In Proc. of ACM Symposium on Interactive 3D Graphics (I3D'01). Chapel Hill, NC, 63--68.

Digital Library

[24]

Arun Sankisa, Arjun Punjabi, and Aggelos K Katsaggelos. 2018. Video error concealment using deep neural networks. In Proc. of IEEE International Conference on Image Processing (ICIP'18). Athens, Greece, 380--384.

[25]

Arun Sankisa, Arjun Punjabi, and Aggelos K Katsaggelos. 2020. Temporal capsule networks for video motion estimation and error concealment. Springer Signal, Image and Video Processing 14 (2020), 1--9.

[26]

Socrates Varakliotis, Jörn Ostermann, and Vicky Hardman. 2001. Coding of animated 3-D wireframe models for Internet streaming applications. In Proc. of IEEE International Conference on Multimedia and Expo (ICME'01). Tokyo, Japan, 353--356.

[27]

Yao Wang, Amy R Reibman, and Shunan Lin. 2004. Multiple description coding for video delivery. Proc. IEEE 93, 1 (2004), 57--70.

[28]

Cheng-Hao Wu, Chih-Fan Hsu, Ting-Chun Kuo, Carsten Griwodz, Michael Riegler, Géraldine Morin, and Cheng-Hsin Hsu. 2020. PCC arena: A benchmark platform for point cloud compression algorithms. In Proc. of ACM International Workshop on Immersive Mixed and Virtual Environment Systems (MMVE'20). Istanbul, Turkey, 1--6.

Digital Library

[29]

Zhenyu Yang, Klara Nahrstedt, Yi Cui, Bin Yu, Jin Liang, Sang-hack Jung, and Ruzena Bajscy. 2005. TEEVE: The next generation architecture for tele-immersive environments. In Proc. of IEEE International Symposium on Multimedia (ISM'05). Irvine, CA, 112--119.

Digital Library

[30]

Rui Zhang, Shankar L Regunathan, and Kenneth Rose. 2000. Video coding with optimal inter/intra-mode switching for packet loss resilience. IEEE Journal on Selected Areas in Communications 18, 6 (2000), 966--976.

Digital Library

Cited By

Li XGui SLuo YJi SSalakhutdinov RKolter ZHeller KWeller AOliver NScarlett JBerkenkamp F(2024)Graph structure extrapolation for out-of-distribution generalizationProceedings of the 41st International Conference on Machine Learning10.5555/3692070.3693185(27846-27874)Online publication date: 21-Jul-2024
https://dl.acm.org/doi/10.5555/3692070.3693185
Van Damme Svan der Hooft JDe Turck FTorres Vega M(2024)Analyzing the applicability of psychometric QoE modeling for projection-based point cloud video quality assessmentJournal on Image and Video Processing10.1186/s13640-024-00655-y2024:1Online publication date: 19-Nov-2024
https://dl.acm.org/doi/10.1186/s13640-024-00655-y
Jahan MBentaleb A(2024)GESA: Exploring Loss-based Adversarial Attacks in Volumetric Media Streaming2024 IEEE 7th International Conference on Multimedia Information Processing and Retrieval (MIPR)10.1109/MIPR62202.2024.00061(342-348)Online publication date: 7-Aug-2024
https://doi.org/10.1109/MIPR62202.2024.00061
Show More Cited By

Index Terms

Dynamic 3D point cloud streaming: distortion and concealment

Recommendations

Error Concealment of Dynamic 3D Point Cloud Streaming
MM '22: Proceedings of the 30th ACM International Conference on Multimedia

Recently standardized MPEG Video-based Point Cloud Compression (V-PCC) codec has shown promise in achieving a good rate-distortion ratio of dynamic 3D point cloud compression. Current error concealment methods of V-PCC, however, lead to significantly ...
Modeling of transmission-loss-induced distortion in decoded video

This paper analyzes the distortion in decoded video caused by random packet losses in the underlying transmission network. A recursion model is derived that relates the average channel-induced distortion in successive P-frames. The model is applicable ...
Codeblock-Based Error Concealment for JPEG2000 Coded Image Transmission over RTP

JPEG2000 compression standard considers a block of wavelet coefficients, called codeblock, as the smallest coding unit that being independently entropy-coded. In this paper, we propose a codeblock-based concealment technique for JPEG2000 images to ...

Comments

Information & Contributors

Information

Published In

cover image ACM Conferences

NOSSDAV '21: Proceedings of the 31st ACM Workshop on Network and Operating Systems Support for Digital Audio and Video

July 2021

128 pages

ISBN:9781450384353

DOI:10.1145/3458306

Program Chairs:
Andra Lutu
Telefonica, Spain
,
Gwendal Simon
Huawei, France
,
Mylene C. Q. Farias
University of Brasilia, Brazil

Copyright © 2021 ACM.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

Sponsors

SIGMM: ACM Special Interest Group on Multimedia

In-Cooperation

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 02 July 2021

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Research-article

Funding Sources

Singapore Ministry of Education
Ministry of Science and Technology of Taiwan

Conference

MMSys '21

Sponsor:

SIGMM

MMSys '21: 12th ACM Multimedia Systems Conference

September 28 - October 1, 2021

Istanbul, Turkey

Acceptance Rates

NOSSDAV '21 Paper Acceptance Rate 15 of 52 submissions, 29%;

Overall Acceptance Rate 118 of 363 submissions, 33%

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

6
Total Citations
View Citations
499
Total Downloads

Downloads (Last 12 months)84
Downloads (Last 6 weeks)10

Reflects downloads up to 28 Jan 2025

Other Metrics

View Author Metrics

Citations

Cited By

Li XGui SLuo YJi SSalakhutdinov RKolter ZHeller KWeller AOliver NScarlett JBerkenkamp F(2024)Graph structure extrapolation for out-of-distribution generalizationProceedings of the 41st International Conference on Machine Learning10.5555/3692070.3693185(27846-27874)Online publication date: 21-Jul-2024
https://dl.acm.org/doi/10.5555/3692070.3693185
Van Damme Svan der Hooft JDe Turck FTorres Vega M(2024)Analyzing the applicability of psychometric QoE modeling for projection-based point cloud video quality assessmentJournal on Image and Video Processing10.1186/s13640-024-00655-y2024:1Online publication date: 19-Nov-2024
https://dl.acm.org/doi/10.1186/s13640-024-00655-y
Jahan MBentaleb A(2024)GESA: Exploring Loss-based Adversarial Attacks in Volumetric Media Streaming2024 IEEE 7th International Conference on Multimedia Information Processing and Retrieval (MIPR)10.1109/MIPR62202.2024.00061(342-348)Online publication date: 7-Aug-2024
https://doi.org/10.1109/MIPR62202.2024.00061
Yang MLuo ZHu MChen MWu D(2023)A Comparative Measurement Study of Point Cloud-Based Volumetric Video CodecsIEEE Transactions on Broadcasting10.1109/TBC.2023.324340769:3(715-726)Online publication date: Oct-2023
https://doi.org/10.1109/TBC.2023.3243407
Thomos NMaugey TToni L(2022)Machine Learning for Multimedia CommunicationsSensors10.3390/s2203081922:3(819)Online publication date: 21-Jan-2022
https://doi.org/10.3390/s22030819
Chujo YKanai KKatto J(2022)Implementation and Demonstration of Real-time Point Cloud Streaming System using HoloLens2022 IEEE International Conference on Consumer Electronics - Taiwan10.1109/ICCE-Taiwan55306.2022.9869044(221-222)Online publication date: 6-Jul-2022
https://doi.org/10.1109/ICCE-Taiwan55306.2022.9869044

View Options

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Figures

Tables

Media

View Table of Conten