research-article

Open access

Point Cloud Interaction and Manipulation in Virtual Reality

Authors:

Daniel Garrido,

A. Augusto Sousa,

Daniel Castro SilvaAuthors Info & Claims

AIVR 2021: 2021 5th International Conference on Artificial Intelligence and Virtual Reality (AIVR)

Pages 15 - 20

https://doi.org/10.1145/3480433.3480437

Published: 08 November 2021 Publication History

All formats PDF

Abstract

The use of virtual reality technologies for data visualization and analysis has been an emerging topic of research in the past years. However, one type of data has been left neglected, the point cloud. While some strides have been made in the visualization and analysis of point clouds in immersive environments, these have yet to be used for direct manipulation interactions. It is hypothesized that as with other types of data, bringing direct interactions and 3D visualization to point clouds may increase the ease of performing basic handling tasks. An immersive application for virtual reality HMDs was developed in Unity to help research this hypothesis. It is capable of parsing classified point cloud files with extracted objects and representing them in a virtual environment. Several editing tools were also developed, designed with the HMD controllers in mind. The end result allows the user to perform basic transformative tasks to the point cloud with an ease of use and intuitive feeling unmatched by the traditional desktop-based tools.

References

[1]

Ayman Ammoura. 2001. DIVE-ON: From Databases to Virtual Reality. XRDS 7, 3 (March 2001), 4-ff. https://doi.org/10.1145/367884.367891

Digital Library

[2]

F. Bernadini, J. Mittleman, H. Rushmeier, C. Silva, and G. Taubin. 1999. The ball-pivoting algorithm for surface reconstruction. IEEE Transactions on Visualization and Computer Graphics 5, 4 (October 1999), 349-359. https://doi.org/10.1109/2945.817351

Digital Library

[3]

Ciro Donalek, S. G. Djorgovski, Alex Cioc, Anwell Wang, Jerry Zhang, Elizabeth Lawler, Stacy Yeh, Ashish Mahabal, Matthew Graham, Andrew Drake, Scott Davidoff, Jeffrey S. Norris, and Giuseppe Longo. 2014. Immersive and collaborative data visualization using virtual reality platforms. In 2014 IEEE International Conference on Big Data. 609-614. https://doi.org/10.1109/BigData.2014.7004282

[4]

Seth M. Feeman, Landon B. Wright, John L. Salmon. 2018. Exploration and evaluation of CAD modeling in virtual reality. Computer-Aided Design and Applications 15, 6 (2018), 892-904. https://doi.org/10.1080/16864360.2018.1466807

[5]

A. Kharroubi, R. Hajji, R. Billen, F. Poux. 2019. Classification and Integration of Massive 3d Points Clouds in a Virtual Reality (vr) Environment. ISPRS - International Archives of the Photogrammetry, Remote Sensing and Spatial Information Sciences XLII-2/W17 (December 2019), 165-171. https://doi.org/10.5194/isprs-archives-XLII-2-W17-165-2019

[6]

D. Lee, W. Muir, S. Beeston, S. Bates, S. D. Schofield, M. J. Edwards, and R. D. Green. 2018. Analysing Forests Using Dense Point Clouds. In 2018 International Conference on Image and Vision Computing New Zealand. https://doi.org/10.1109/IVCNZ.2018.8634651

[7]

P. Lubos, R. Beimler, M. Lammers, and F. Steinicke. 2014. Touching the Cloud: Bimanual annotation of immersive point clouds. In 2014 IEEE Symposium on 3D User Interfaces. 191-192. https://doi.org/10.1109/3DUI.2014.6798885

[8]

Stefan Marks, Javier E. Estevez, and Andy M. Connor. 2014. Towards the Holodeck: Fully Immersive Virtual Reality Visualisation of Scientific and Engineering Data. In Proceedings of the 29th International Conference on Image and Vision Computing New Zealand. Hamilton, New Zealand, 42–47. https://doi.org/10.1145/2683405.2683424

Digital Library

[9]

Sven Oesau. 2015. Geometric modeling of indoor scenes from acquired point data. PhD Thesis, Université Nice Sophia Antipolis.

[10]

Noritaka Osawa, Kikuo Asai, Yuji Y. Sugimoto. 2000. Immersive Graph Navigation Using Direct Manipulation and Gestures. In Proceedings of the ACM Symposium on Virtual Reality Software and Technology. Seoul, Korea, 147–152.

Digital Library

[11]

Xavier Roynard, Jean-Emmanuel Deschaud, and François Goulette. 2017. Paris-Lille-3D: A large and high-quality ground-truth urban point cloud dataset for automatic segmentation and classification. The International Journal of Robotics Research 37, 6 (May 2018), 545-557. https://doi.org/10.1177/0278364918767506

Digital Library

[12]

Jonathan Dyssel Stets, Yongbin Sun, Wiley Corning, and Scott W. Greenwald. 2017. Visualization and Labeling of Point Clouds in Virtual Reality. In SIGGRAPH Asia 2017 Posters. Bangkok, Thailand, Article 31, 1–2. https://doi.org/10.1145/3145690.3145729

Digital Library

[13]

F. Thiel, S. Discher, R. Richter, and J. Döllner. 2018. Interaction and Locomotion Techniques for the Exploration of Massive 3D Point Clouds in VR Environments. ISPRS - International Archives of the Photogrammetry, Remote Sensing and Spatial Information Sciences XLII-4 (October 2018), 623-630. https://doi.org/10.5194/isprs-archives-XLII-4-623-2018

[14]

Martin Weinmann. 2016. Preliminaries of 3D Point Cloud Processing. In: Reconstruction and Analysis of 3D Scenes. Springer, Cham. https://doi.org/10.1007/978-3-319-29246-5_2

[15]

Josef Wolfartsberger. 2019. Analyzing the potential of Virtual Reality for engineering design review. Automation in Construction 104 (August 2019), 27 – 37. https://doi.org/10.1016/j.autcon.2019.03.018

Cited By

Bai HLi XMeng QZhuo SYan L(2025)MFFTNet: A Novel 3D Point Cloud Segmentation Network Based on Multi-Scale Feature Fusion and Transformer ArchitectureIEEE Access10.1109/ACCESS.2025.352824513(9462-9472)Online publication date: 2025
https://doi.org/10.1109/ACCESS.2025.3528245
Chidambaram SWilliams ABai MVirk SHaffner PLease MLi E(2024)Annorama: Enabling Immersive At-Desk Annotation Experiences in Virtual Reality with 3D Point Cloud DioramasProceedings of the 2024 ACM Symposium on Spatial User Interaction10.1145/3677386.3682081(1-13)Online publication date: 7-Oct-2024
https://dl.acm.org/doi/10.1145/3677386.3682081
Yu ZLin TGumhold SMcGinity M(2024)An Immersive Method for Extracting Structural Information from Unorganized Point Clouds2024 Nicograph International (NicoInt)10.1109/NICOInt62634.2024.00014(24-31)Online publication date: 14-Jun-2024
https://doi.org/10.1109/NICOInt62634.2024.00014
Show More Cited By

Index Terms

Point Cloud Interaction and Manipulation in Virtual Reality
1. Computing methodologies
  1. Computer graphics
    1. Graphics systems and interfaces
      1. Virtual reality
2. Human-centered computing
  1. Human computer interaction (HCI)
    1. Interaction paradigms
      1. Mixed / augmented reality
      2. Virtual reality

Index terms have been assigned to the content through auto-classification.

Recommendations

Mixed Reality MIDI Keyboard Demonstration
AM '17: Proceedings of the 12th International Audio Mostly Conference on Augmented and Participatory Sound and Music Experiences

The Mixed Reality MIDI Keyboard is a prototype designed to augment virtual reality experiences through the inclusion of a physical interface which aligns the user's senses with the virtual environment. It also serves as a platform on which the uses of ...
Multi-sensorial virtual reality and augmented human food interaction
MHFI '16: Proceedings of the 1st Workshop on Multi-sensorial Approaches to Human-Food Interaction

In the field of virtual reality (VR) research, media technologies to create a realistic feeling of being present in a real/virtual world by duplicating multi-sensory information have been studied over a long period. Recently, technologies for multi-...
A Framework for Virtual Reality with Tangible Augmented Reality-Based User Interface

In this paper, we propose a framework for virtual reality, I²-NEXT, which enables users to interact with virtual objects by tangible objects in immersive networked virtual environment. The primary goal of this framework is to support rapid development ...

Comments

Information & Contributors

Information

Published In

cover image ACM Other conferences

AIVR 2021: 2021 5th International Conference on Artificial Intelligence and Virtual Reality (AIVR)

July 2021

134 pages

ISBN:9781450384148

DOI:10.1145/3480433

Copyright © 2021 Owner/Author.

This work is licensed under a Creative Commons Attribution International 4.0 License.

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 08 November 2021

Check for updates

Author Tags

Qualifiers

Research-article
Research
Refereed limited

Conference

AIVR 2021

AIVR 2021: 2021 the 5th International Conference on Artificial Intelligence and Virtual Reality

July 23 - 25, 2021

Kumamoto, Japan

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

12
Total Citations
View Citations
2,587
Total Downloads

Downloads (Last 12 months)863
Downloads (Last 6 weeks)137

Reflects downloads up to 08 Feb 2025

Other Metrics

View Author Metrics

Citations

Cited By

Bai HLi XMeng QZhuo SYan L(2025)MFFTNet: A Novel 3D Point Cloud Segmentation Network Based on Multi-Scale Feature Fusion and Transformer ArchitectureIEEE Access10.1109/ACCESS.2025.352824513(9462-9472)Online publication date: 2025
https://doi.org/10.1109/ACCESS.2025.3528245
Chidambaram SWilliams ABai MVirk SHaffner PLease MLi E(2024)Annorama: Enabling Immersive At-Desk Annotation Experiences in Virtual Reality with 3D Point Cloud DioramasProceedings of the 2024 ACM Symposium on Spatial User Interaction10.1145/3677386.3682081(1-13)Online publication date: 7-Oct-2024
https://dl.acm.org/doi/10.1145/3677386.3682081
Yu ZLin TGumhold SMcGinity M(2024)An Immersive Method for Extracting Structural Information from Unorganized Point Clouds2024 Nicograph International (NicoInt)10.1109/NICOInt62634.2024.00014(24-31)Online publication date: 14-Jun-2024
https://doi.org/10.1109/NICOInt62634.2024.00014
Ton BTempert NPlass D(2024)Immersive visualisation of point cloud data of railway environments2024 10th International Conference on Virtual Reality (ICVR)10.1109/ICVR62393.2024.10868576(233-239)Online publication date: 24-Jul-2024
https://doi.org/10.1109/ICVR62393.2024.10868576
Murtiyoso ACabo CSingh AObaya DCherlet WStoddart JFol CBeloiu Schwenke MRehush NStereńczak KCalders KGriess VMokroš M(2024)A Review of Software Solutions to Process Ground-based Point Clouds in Forest ApplicationsCurrent Forestry Reports10.1007/s40725-024-00228-210:6(401-419)Online publication date: 17-Aug-2024
https://doi.org/10.1007/s40725-024-00228-2
Zhang TQian GXie JYang J(2024)FastPCI: Motion-Structure Guided Fast Point Cloud Frame InterpolationComputer Vision – ECCV 202410.1007/978-3-031-72904-1_15(251-267)Online publication date: 21-Nov-2024
https://doi.org/10.1007/978-3-031-72904-1_15
Wang YHuang SGao YWang ZWang RSheng KZhang BLiu SEl Saddik AMei TCucchiara RBertini MTobon Vallejo DAtrey PHossain M(2023)Transferring CLIP's Knowledge into Zero-Shot Point Cloud Semantic SegmentationProceedings of the 31st ACM International Conference on Multimedia10.1145/3581783.3612107(3745-3754)Online publication date: 27-Oct-2023
https://doi.org/10.1145/3581783.3612107
Zhang XHuang ZHuang X(2023)MAVR: Multi-Functional Point Cloud Annotations Using Virtual Reality2023 IEEE High Performance Extreme Computing Conference (HPEC)10.1109/HPEC58863.2023.10363493(1-6)Online publication date: 25-Sep-2023
https://doi.org/10.1109/HPEC58863.2023.10363493
Zheng ZWu DLu RLu FChen GJiang C(2023)NeuralPCI: Spatio-Temporal Neural Field for 3D Point Cloud Multi-Frame Non-Linear Interpolation2023 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)10.1109/CVPR52729.2023.00094(909-918)Online publication date: Jun-2023
https://doi.org/10.1109/CVPR52729.2023.00094
Dekker BTon BMeijer JBouali NLinssen JAhmed F(2023)Point Cloud Analysis of Railway Infrastructure: A Systematic Literature ReviewIEEE Access10.1109/ACCESS.2023.333704911(134355-134373)Online publication date: 2023
https://doi.org/10.1109/ACCESS.2023.3337049
Show More Cited By

View Options

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

HTML Format

View this article in HTML Format.

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

Figures

Tables

Media

View Table of Conten