Point Cloud Upsampling with Geometric Algebra Driven Inverse Heat Dissipation
Authors: 
Wenqiang Xu, Wenrui Dai, Ziyang Zheng, Chenglin Li, Junni Zou, Hongkai XiongAuthors Info & Claims
Wenqiang Xu, Wenrui Dai, Ziyang Zheng, Chenglin Li, Junni Zou, Hongkai XiongAuthors Info & ClaimsPages 7385 - 7394
Abstract
Point cloud upsampling is crucial for 3D reconstruction, with recent research significantly benefitting from the advances in deep learning technologies. The majority of existing methods, which focus on a sequence of processes including feature extraction, augmentation, and the reconstruction of coordinates, encounter significant challenges in interpreting the geometric attributes they uncover, particularly with respect to the intricacies of transitioning feature dimensionality. In this paper, we delve deeper into modeling Partial Differential Equations (PDEs) specifically tailored for the inverse heat dissipation process in dense point clouds. Our goal is to detect gradients within the dense point cloud data distribution and refine the accuracy of interpolated points' positions along with their complex geometric nuances through a systematic iterative approximation method. Simultaneously, we adopt multivectors from geometric algebra as the primary tool for representing the geometric characteristics of point clouds, moving beyond the conventional vector space representations. The use of geometric products of multivectors enables us to capture the complex relationships between scalars, vectors, and their components more effectively. This methodology not only offers a robust framework for depicting the geometric features of point clouds but also enhances our modeling capabilities for inverse heat dissipation PDEs. Through both qualitative and quantitative assessments, we demonstrate that our results significantly outperform existing state-of-the-art techniques in terms of widely recognized point cloud evaluation metrics and 3D visual reconstruction fidelity.
References
[1]
Marc Alexa, Johannes Behr, Daniel Cohen-Or, Shachar Fleishman, David Levin, and Claudio T. Silva. 2003. Computing and Rendering Point Set Surfaces. IEEE Transactions on Visualization and Computer Graphics, Vol. 9, 1 (2003), 3--15.
[2]
Jean Babaud, Andrew P Witkin, Michel Baudin, and Richard O Duda. 1986. Uniqueness of the Gaussian Kernel for Scale-Space Filtering. IEEE Transactions on Pattern Analysis and Machine Intelligence, Vol. 8, 1 (1986), 26--33.
[3]
Johann Brehmer, Pim de Haan, Sönke Behrends, and Taco S Cohen. 2023. Geometric Algebra Transformer. In Advances in Neural Information Processing Systems 36, A. Oh, T. Naumann, A. Globerson, K. Saenko, M. Hardt, and S. Levine (Eds.). Curran Associates, Inc., 35472--35496.
[4]
Wanquan Feng, Jin Li, Hongrui Cai, Xiaonan Luo, and Juyong Zhang. 2022. Neural points: Point cloud representation with neural fields for arbitrary upsampling. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. 18633--18642.
[5]
Ian Goodfellow, Jean Pouget-Abadie, Mehdi Mirza, Bing Xu, David Warde-Farley, Sherjil Ozair, Aaron Courville, and Yoshua Bengio. 2014. Generative Adversarial Nets. In Advances in Neural Information Processing Systems 27. Curran Associates, Inc., 2672--2680.
[6]
Yun He, Danhang Tang, Yinda Zhang, Xiangyang Xue, and Yanwei Fu. 2023. Grad-PU: Arbitrary-scale point cloud upsampling via gradient descent with learned distance functions. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. 5354--5363.
[7]
Robert Held, Ankit Gupta, Brian Curless, and Maneesh Agrawala. 2012. 3D puppetry: a kinect-based interface for 3D animation. In UIST, Vol. 12. 423--434.
[8]
Jonathan Ho, Ajay Jain, and Pieter Abbeel. 2020. Denoising diffusion probabilistic models. In Advances in Neural Information Processing Systems 33. 6840--6851.
[9]
Emiel Hoogeboom and Tim Salimans. 2023. Blurring diffusion models. In The Eleventh International Conference on Learning Representations.
[10]
Hugues Hoppe, Tony DeRose, Tom Duchamp, John McDonald, and Werner Stuetzle. 1992. Surface reconstruction from unorganized points. In Proceedings of the 19th Annual Conference on Computer Graphics and Interactive Techniques. 71--78.
[11]
Hui Huang, Dan Li, Hao Zhang, Uri Ascher, and Daniel Cohen-Or. 2009. Consolidation of Unorganized Point Clouds for Surface Reconstruction. ACM Transactions on Graphics (TOG), Vol. 28, 5 (2009), 1--7.
[12]
Hui Huang, Shihao Wu, Minglun Gong, Daniel Cohen-Or, Uri Ascher, and Hao Zhang. 2013. Edge-aware point set resampling. ACM Transactions on Graphics (TOG), Vol. 32, 1 (2013), 1--12.
[13]
Michael Kazhdan and Hugues Hoppe. 2013. Screened Poisson Surface Reconstruction. ACM Transactions on Graphics (ToG), Vol. 32, 3 (2013), 1--13.
[14]
Alex H Lang, Sourabh Vora, Holger Caesar, Lubing Zhou, Jiong Yang, and Oscar Beijbom. 2019. PointPillars: Fast encoders for object detection from point clouds. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. 12697--12705.
[15]
Ruihui Li, Xianzhi Li, Chi-Wing Fu, Daniel Cohen-Or, and Pheng-Ann Heng. 2019. PU-GAN: a point cloud upsampling adversarial network. In Proceedings of the IEEE/CVF International Conference on Computer Vision. 7203--7212.
[16]
Ruihui Li, Xianzhi Li, Pheng-Ann Heng, and Chi-Wing Fu. 2021. Point cloud upsampling via disentangled refinement. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern recognition. 344--353.
[17]
Yaron Lipman, Daniel Cohen-Or, David Levin, and Hillel Tal-Ezer. 2007. Parameterization-Free Projection for Geometry Reconstruction. ACM Transactions on Graphics (ToG), Vol. 26, 3 (2007), 22--es.
[18]
Zhijian Liu, Haotian Tang, Yujun Lin, and Song Han. 2019. Point-Voxel CNN for Efficient 3D Deep Learning. In Advances in Neural Information Processing Systems 32, H. Wallach, H. Larochelle, A. Beygelzimer, F. dtextquotesingle Alché-Buc, E. Fox, and R. Garnett (Eds.). Curran Associates, Inc., 965--975.
[19]
Chen Long, WenXiao Zhang, Ruihui Li, Hao Wang, Zhen Dong, and Bisheng Yang. 2022. PC2-PU: Patch correlation and point correlation for effective point cloud upsampling. In Proceedings of the 30th ACM International Conference on Multimedia. 2191--2201.
[20]
Shitong Luo and Wei Hu. 2021. Diffusion probabilistic models for 3D point cloud generation. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. 2837--2845.
[21]
Zhaoyang Lyu, Zhifeng Kong, Xudong Xu, Liang Pan, and Dahua Lin. 2021. A conditional point diffusion-refinement paradigm for 3D point cloud completion. arXiv preprint arXiv:2112.03530 (2021).
[22]
Richard A Newcombe, Shahram Izadi, Otmar Hilliges, David Molyneaux, David Kim, Andrew J Davison, Pushmeet Kohi, Jamie Shotton, Steve Hodges, and Andrew Fitzgibbon. 2011. KinectFusion: Real-time dense surface mapping and tracking. In 2011 10th IEEE International Symposium on Mixed and Augmented Reality. Ieee, 127--136.
[23]
Alex Nichol, Heewoo Jun, Prafulla Dhariwal, Pamela Mishkin, and Mark Chen. 2022. Point-E: A system for generating 3D point clouds from complex prompts. arXiv preprint arXiv:2212.08751 (2022).
[24]
Anh Viet Phan, Minh Le Nguyen, Yen Lam Hoang Nguyen, and Lam Thu Bui. 2018. DGCNN: A convolutional neural network over large-scale labeled graphs. Neural Networks, Vol. 108 (2018), 533--543.
[25]
Charles R Qi, Hao Su, Kaichun Mo, and Leonidas J Guibas. 2017. PointNet: Deep learning on point sets for 3D classification and segmentation. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. 652--660.
[26]
Charles Ruizhongtai Qi, Li Yi, Hao Su, and Leonidas J Guibas. 2017. PointNet: Deep hierarchical feature learning on point sets in a metric space. In Advances in Neural Information Processing Systems 30. 5099--5108.
[27]
Guocheng Qian, Abdulellah Abualshour, Guohao Li, Ali Thabet, and Bernard Ghanem. 2021. PU-GCN: Point cloud upsampling using graph convolutional networks. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. 11683--11692.
[28]
Yue Qian, Junhui Hou, Sam Kwong, and Ying He. 2020. PUGeo-Net: A geometry-centric network for 3D point cloud upsampling. In European Conference on Computer Vision. Springer, 752--769.
[29]
Shi Qiu, Saeed Anwar, and Nick Barnes. 2022. PU-Transformer: Point cloud upsampling transformer. In Proceedings of the Asian Conference on Computer Vision. 2475--2493.
[30]
Wentao Qu, Yuantian Shao, Lingwu Meng, Xiaoshui Huang, and Liang Xiao. 2024. A Conditional Denoising Diffusion Probabilistic Model for Point Cloud Upsampling. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR). 20786--20795.
[31]
Severi Rissanen, Markus Heinonen, and Arno Solin. 2023. Generative modelling with inverse heat dissipation. In The Eleventh International Conference on Learning Representations.
[32]
Robin Rombach, Andreas Blattmann, Dominik Lorenz, Patrick Esser, and Björn Ommer. 2022. High-resolution image synthesis with latent diffusion models. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. 10684--10695.
[33]
Chitwan Saharia, Jonathan Ho, William Chan, Tim Salimans, David J Fleet, and Mohammad Norouzi. 2022. Image Super-Resolution via Iterative Refinement. IEEE Transactions on Pattern Analysis and Machine Intelligence, Vol. 45, 4 (2022), 4713--4726.
[34]
José Miguel Santana, Jochen Wendel, Agustín Trujillo, José Pablo Suárez, Alexander Simons, and Andreas Koch. 2017. Multimodal location based services?semantic 3D city data as virtual and augmented reality. In Progress in Location-based Services 2016. Springer, 329--353.
[35]
Andrés Serna, Beatriz Marcotegui, Franccois Goulette, and Jean-Emmanuel Deschaud. 2014. Paris-rue-Madame database: A 3D mobile laser scanner dataset for benchmarking urban detection, segmentation and classification methods. In 4th International Conference on Pattern Recognition, Applications and Methods (ICPRAM 2014).
[36]
Jascha Sohl-Dickstein, Eric Weiss, Niru Maheswaranathan, and Surya Ganguli. 2015. Deep unsupervised learning using nonequilibrium thermodynamics. In International Conference on Machine Learning. PMLR, 2256--2265.
[37]
Yang Song, Jascha Sohl-Dickstein, Diederik P Kingma, Abhishek Kumar, Stefano Ermon, and Ben Poole. 2020. Score-based generative modeling through stochastic differential equations. arXiv preprint arXiv:2011.13456 (2020).
[38]
RW Sumner and J Po-povic. [n.,d.]. Mesh data from deformation transfer for triangle meshes (2004).
[39]
Yan Wang, Wei-Lun Chao, Divyansh Garg, Bharath Hariharan, Mark Campbell, and Kilian Q Weinberger. 2019. Pseudo-LiDAR from visual depth estimation: Bridging the gap in 3D object detection for autonomous driving. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. 8445--8453.
[40]
Andrew P Witkin. 1987. Scale-space filtering. In Readings in Computer Vision. Elsevier, 329--332.
[41]
Shihao Wu, Hui Huang, Minglun Gong, Matthias Zwicker, and Daniel Cohen-Or. 2015. Deep points consolidation. ACM Transactions on Graphics (ToG), Vol. 34, 6 (2015), 1--13.
[42]
Shuquan Ye, Dongdong Chen, Songfang Han, Ziyu Wan, and Jing Liao. 2021. Meta-PU: An Arbitrary-Scale Upsampling Network for Point Cloud. IEEE Transactions on Visualization And Computer Graphics, Vol. 28, 9 (2021), 3206--3218.
[43]
Wang Yifan, Shihao Wu, Hui Huang, Daniel Cohen-Or, and Olga Sorkine-Hornung. 2019. Patch-based progressive 3d point set upsampling. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. 5958--5967.
[44]
Lequan Yu, Xianzhi Li, Chi-Wing Fu, Daniel Cohen-Or, and Pheng-Ann Heng. 2018. PU-Net: Point cloud upsampling network. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. 2790--2799.
[45]
Cem Yuksel. 2015. Sample elimination for generating poisson disk sample sets. Computer Graphics Forum, Vol. 34, 2 (2015), 25--32.
[46]
Hengshuang Zhao, Li Jiang, Jiaya Jia, Philip HS Torr, and Vladlen Koltun. 2021. Point Transformer. In Proceedings of the IEEE/CVF International Conference on Computer Vision. 16259--16268.
[47]
Linqi Zhou, Yilun Du, and Jiajun Wu. 2021. 3D shape generation and completion through point-voxel diffusion. In Proceedings of the IEEE/CVF International Conference on Computer Vision. 5826--5835.
Index Terms
- Point Cloud Upsampling with Geometric Algebra Driven Inverse Heat Dissipation
Recommendations
Double Conformal Geometric Algebra for Quadrics and Darboux Cyclides
CGI '16: Proceedings of the 33rd Computer Graphics InternationalWe introduce Clifford geometric algebra based multivector modeling of quartic and general quadric surfaces, Darboux cyclides, Dupin cyclies, tori and pairs of standard CGA objects. A computer visualization is briefly described.
SSPU-Net: Self-Supervised Point Cloud Upsampling via Differentiable Rendering
MM '21: Proceedings of the 29th ACM International Conference on MultimediaPoint clouds obtained from 3D sensors are usually sparse. Existing methods mainly focus on upsampling sparse point clouds in a supervised manner by using dense ground truth point clouds. In this paper, we propose a self-supervised point cloud upsampling ...
Point Cloud Upsampling via a Coarse-to-Fine Network
MultiMedia ModelingAbstractPoint clouds captured by 3D scanning are usually sparse and noisy. Reconstructing a high-resolution 3D model of an object is a challenging task in computer vision. Recent point cloud upsampling approaches aim to generate a dense point set, while ...
Comments
Information & Contributors
Information
Published In
October 2024
11719 pages
ISBN:9798400706868
DOI:10.1145/3664647
- General Chairs:
- Jianfei Cai,
- Mohan Kankanhalli,
- Balakrishnan Prabhakaran,
- Susanne Boll,
- Program Chairs:
- Ramanathan Subramanian,
- Liang Zheng,
- Vivek K. Singh,
- Pablo Cesar,
- Lexing Xie,
- Dong Xu
Copyright © 2024 ACM.
Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than the author(s) must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected].
Sponsors
Publisher
Association for Computing Machinery
New York, NY, United States
Publication History
Published: 28 October 2024
Check for updates
Author Tags
Qualifiers
- Research-article
Funding Sources
Conference
MM '24
Sponsor:
MM '24: The 32nd ACM International Conference on Multimedia
October 28 - November 1, 2024
Melbourne VIC, Australia
Acceptance Rates
MM '24 Paper Acceptance Rate 1,150 of 4,385 submissions, 26%;
Overall Acceptance Rate 2,145 of 8,556 submissions, 25%
Contributors
Other Metrics
Bibliometrics & Citations
Bibliometrics
Article Metrics
- 0Total Citations
- 73Total Downloads
- Downloads (Last 12 months)73
- Downloads (Last 6 weeks)56
Reflects downloads up to 23 Dec 2024
Other Metrics
Citations
View Options
Login options
Check if you have access through your login credentials or your institution to get full access on this article.
Sign in