research-article

FSVVD: A Dataset of Full Scene Volumetric Video

Authors:

Fangxin WangAuthors Info & Claims

MMSys '23: Proceedings of the 14th Conference on ACM Multimedia Systems

Pages 410 - 415

https://doi.org/10.1145/3587819.3592551

Published: 08 June 2023 Publication History

Get Access

Abstract

Recent years have witnessed a rapid development of immersive multimedia which bridges the gap between the real world and virtual space. Volumetric videos, as an emerging representative 3D video paradigm that empowers extended reality, stand out to provide unprecedented immersive and interactive video watching experience. Despite the tremendous potential, the research towards 3D volumetric video is still in its infancy, relying on sufficient and complete datasets for further exploration. However, existing related volumetric video datasets mostly only include a single object, lacking details about the scene and the interaction between them. In this paper, we focus on the current most widely used data format, point cloud, and for the first time release a full-scene volumetric video dataset that includes multiple people and their daily activities interacting with the external environments. Comprehensive dataset description and analysis are conducted, with potential usage of this dataset. The dataset and additional tools can be accessed via the following website: https://cuhksz-inml.github.io/full_scene_volumetric_video_dataset/.

References

[1]

Anargyros Chatzitofis, Leonidas Saroglou, Prodromos Boutis, Petros Drakoulis, Nikolaos Zioulis, Shishir Subramanyam, Bart Kevelham, Caecilia Charbonnier, Pablo César, Dimitrios Zarpalas, Stefanos D. Kollias, and Petros Daras. 2020. HUMAN4D: A Human-Centric Multimodal Dataset for Motions and Immersive Media. IEEE Access 8 (2020), 176241--176262.

Crossref

Google Scholar

[2]

Serhan Gül, Dimitri Podborski, Thomas Buchholz, Thomas Schierl, and Cornelius Hellge. 2020. Low-latency cloud-based volumetric video streaming using head motion prediction. In Proceedings of the 30th ACM Workshop on Network and Operating Systems Support for Digital Audio and Video (NOSSDAV), 2020. ACM.

Digital Library

Google Scholar

[3]

Bo Han, Yu Liu, and Feng Qian. 2020. ViVo: visibility-aware mobile volumetric video streaming. In Proceedings of the 26th Annual International Conference on Mobile Computing and Networking (MobiCom), 2020. ACM.

Digital Library

Google Scholar

[4]

Yili Jin, Junhua Liu, Fangxin Wang, and Shuguang Cui. 2022. Where Are You Looking?: A Large-Scale Dataset of Head and Gaze Behavior for 360-Degree Videos and a Pilot Study. In Proceedings of the 30th ACM International Conference on Multimedia (MM), 2022. ACM.

Digital Library

Google Scholar

[5]

Yili Jin, Junhua Liu, Fangxin Wang, and Shuguang Cui. 2023. Ebublio: Edge Assisted Multi-User 360-Degree Video Streaming. IEEE Internet Things J. (2023).

Google Scholar

[6]

Junhua Liu, Boxiang Zhu, Fangxin Wang, Yili Jin, Wenyi Zhang, Zihan Xu, and Shuguang Cui. 2023. CaV3: Cache-Assisted Viewport Adaptive Volumetric Video Streaming. In Proceedings of the IEEE Conference on Virtual Reality and 3D User Interfaces (VR), 2023. IEEE.

Crossref

Google Scholar

[7]

Yu Liu, Bo Han, Feng Qian, Arvind Narayanan, and Zhi-Li Zhang. 2022. Vues: Practical Mobile Volumetric Video Streaming through Multiview Transcoding. In Proceedings of the 28th Annual International Conference on Mobile Computing And Networking (MobiCom), 2022. ACM.

Digital Library

Google Scholar

[8]

Kaichun Mo, Paul Guerrero, Li Yi, Hao Su, Peter Wonka, Niloy J. Mitra, and Leonidas J. Guibas. 2019. StructureNet: hierarchical graph networks for 3D shape generation. ACM Trans. Graph. 38, 6 (2019), 242:1--242:19.

Digital Library

Google Scholar

[9]

Rafael Pagés, Emin Zerman, Konstantinos Amplianitis, Jan Ondřej, and Aljosa Smolic. 2021. Volograms & V-SENSE Volumetric Video Dataset. ISO/IEC JTC1/SC29/WG07 MPEG2021/m56767 (2021).

Google Scholar

[10]

Ignacio Reimat, Evangelos Alexiou, Jack Jansen, Irene Viola, Shishir Subramanyam, and Pablo Cesar. 2021. CWIPC-SXR: Point Cloud Dynamic Human Dataset for Social XR. In Proceedings of the 12th ACM Multimedia Systems Conference (MMSys), 2021. ACM.

Digital Library

Google Scholar

[11]

Vladimiros Sterzentsenko, Alexandros Doumanoglou, Spyridon Thermos, Nikolaos Zioulis, Dimitrios Zarpalas, and Petros Daras. 2020. Deep Soft Procrustes for Markerless Volumetric Sensor Alignment. In Proceedings of the IEEE Conference on Virtual Reality and 3D User Interfaces (VR), 2020.

Crossref

Google Scholar

[12]

Fan Wei, Guanghua Xu, Qingqiang Wu, Jiachen Kuang, Peiyuan Tian, Penglin Qin, and Zejiang Li. 2022. Azure Kinect Calibration and Parameter Recommendation in Different Scenarios. IEEE Sensors Journal 22, 10 (2022), 9733--9742.

Crossref

Google Scholar

[13]

Jae Shin Yoon, Zhixuan Yu, Jaesik Park, and Hyun Soo Park. 2023. HUMBI: A Large Multiview Dataset of Human Body Expressions and Benchmark Challenge. IEEE Trans. Pattern Anal. Mach. Intell. 45, 1 (2023), 623--640.

Crossref

Google Scholar

[14]

Anlan Zhang, Chendong Wang, Xing Liu, Bo Han, and Feng Qian. 2020. Mobile Volumetric Video Streaming Enhanced by Super Resolution. In Proceedings of the 18th International Conference on Mobile Systems, Applications, and Services (MobiSys), 2020. ACM.

Digital Library

Google Scholar

Cited By

View all

Jin YDuan XWang FLiu XCai JKankanhalli MPrabhakaran BBoll SSubramanian RZheng LSingh VCesar PXie LXu D(2024)HeadsetOff: Enabling Photorealistic Video Conferencing on Economical VR HeadsetsProceedings of the 32nd ACM International Conference on Multimedia10.1145/3664647.3681432(7928-7936)Online publication date: 28-Oct-2024
https://dl.acm.org/doi/10.1145/3664647.3681432
Yin DShi JZhang MHuang ZLiu JDong FCai JKankanhalli MPrabhakaran BBoll SSubramanian RZheng LSingh VCesar PXie LXu D(2024)FSVFG: Towards Immersive Full-Scene Volumetric Video Streaming with Adaptive Feature GridProceedings of the 32nd ACM International Conference on Multimedia10.1145/3664647.3680908(11089-11098)Online publication date: 28-Oct-2024
https://dl.acm.org/doi/10.1145/3664647.3680908
Jin YZhang WXu ZWang FLiu X(2024)Privacy-Preserving Gaze-Assisted Immersive Video StreamingIEEE Transactions on Mobile Computing10.1109/TMC.2024.345251023:12(15098-15113)Online publication date: Dec-2024
https://doi.org/10.1109/TMC.2024.3452510
Show More Cited By

Index Terms

FSVVD: A Dataset of Full Scene Volumetric Video
1. Computing methodologies
  1. Computer graphics
    1. Shape modeling
      1. Volumetric models
2. Information systems
  1. Information systems applications
    1. Multimedia information systems
      1. Multimedia databases

Recommendations

Split Rendering for Mixed Reality: Interactive Volumetric Video in Action
SA '20: SIGGRAPH Asia 2020 XR

This demo presents a mixed reality (MR) application that enables free-viewpoint rendering of interactive high-quality volumetric video (VV) content on Nreal Light MR glasses, web browsers via WebXR and Android devices via ARCore. The application uses a ...
Towards Volumetric Video Realism in Extended Reality: Challenges and Opportunities
IXR '23: Proceedings of the 2nd International Workshop on Interactive eXtended Reality

Advances in volumetric capture, compression, and rendering techniques have enabled the possibilities of telepresence in an extended reality (XR) environment. Live or pre-recorded volumetric video of an avatar can be streamed over the network and ...
Volumetric Video Use Cases for XR Immersive Streaming
ICEMT '24: Proceedings of the 2024 8th International Conference on Education and Multimedia Technology

This paper proposes a summarized analyses for volumetric video streaming use cases and its applicability for immersive extended reality applications. It also closely evaluates and illustrates the usability of standardized technologies for one of the use ...

Comments

Information & Contributors

Information

Published In

MMSys '23: Proceedings of the 14th ACM Multimedia Systems Conference

June 2023

495 pages

ISBN:9798400701481

DOI:10.1145/3587819

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than the author(s) must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected].

In-Cooperation

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 08 June 2023

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Research-article

Conference

MMSys '23

Sponsor:

SIGMM

MMSys '23: 14th Conference on ACM Multimedia Systems

June 7 - 10, 2023

BC, Vancouver, Canada

Acceptance Rates

Overall Acceptance Rate 176 of 530 submissions, 33%

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

6
Total Citations
View Citations
224
Total Downloads

Downloads (Last 12 months)115
Downloads (Last 6 weeks)7

Reflects downloads up to 27 Jan 2025

Other Metrics

View Author Metrics

Citations

Cited By

View all

Jin YDuan XWang FLiu XCai JKankanhalli MPrabhakaran BBoll SSubramanian RZheng LSingh VCesar PXie LXu D(2024)HeadsetOff: Enabling Photorealistic Video Conferencing on Economical VR HeadsetsProceedings of the 32nd ACM International Conference on Multimedia10.1145/3664647.3681432(7928-7936)Online publication date: 28-Oct-2024
https://dl.acm.org/doi/10.1145/3664647.3681432
Yin DShi JZhang MHuang ZLiu JDong FCai JKankanhalli MPrabhakaran BBoll SSubramanian RZheng LSingh VCesar PXie LXu D(2024)FSVFG: Towards Immersive Full-Scene Volumetric Video Streaming with Adaptive Feature GridProceedings of the 32nd ACM International Conference on Multimedia10.1145/3664647.3680908(11089-11098)Online publication date: 28-Oct-2024
https://dl.acm.org/doi/10.1145/3664647.3680908
Jin YZhang WXu ZWang FLiu X(2024)Privacy-Preserving Gaze-Assisted Immersive Video StreamingIEEE Transactions on Mobile Computing10.1109/TMC.2024.345251023:12(15098-15113)Online publication date: Dec-2024
https://doi.org/10.1109/TMC.2024.3452510
Wei LLiu YWang FWang D(2024)FewVV: Few-Shot Adaptive Bitrate Volumetric Video Streaming With Prompted Online AdaptationIEEE Internet of Things Journal10.1109/JIOT.2024.342497711:19(32055-32066)Online publication date: 1-Oct-2024
https://doi.org/10.1109/JIOT.2024.3424977
Wu YHu KShao QChen JChen DWu J(2024)TeleOR: Real-Time Telemedicine System for Full-Scene Operating RoomMedical Image Computing and Computer Assisted Intervention – MICCAI 202410.1007/978-3-031-72089-5_59(628-638)Online publication date: 3-Oct-2024
https://doi.org/10.1007/978-3-031-72089-5_59
Hu KYang HJin YLiu JChen YZhang MWang FEl Saddik AMei TCucchiara RBertini MTobon Vallejo DAtrey PHossain M(2023)Understanding User Behavior in Volumetric Video Watching: Dataset, Analysis and PredictionProceedings of the 31st ACM International Conference on Multimedia10.1145/3581783.3613810(1108-1116)Online publication date: 26-Oct-2023
https://dl.acm.org/doi/10.1145/3581783.3613810

View Options

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

View options

PDF

View or Download as a PDF file.

PDF

eReader

View online with eReader.

eReader

Abstract

References

Cited By

Index Terms

Recommendations

Split Rendering for Mixed Reality: Interactive Volumetric Video in Action

Towards Volumetric Video Realism in Extended Reality: Challenges and Opportunities

Volumetric Video Use Cases for XR Immersive Streaming

Comments

Information

Published In

Sponsors

In-Cooperation

Publisher

Publication History

Permissions

Check for updates

Author Tags

Qualifiers

Conference

Acceptance Rates

Contributors

Other Metrics

Bibliometrics

Article Metrics

Other Metrics

Citations

Cited By

Login options

Full Access

View options

PDF

eReader

Share

Share this Publication link

Share on social media

Affiliations