research-article

Neural-Singular-Hessian: Implicit Neural Representation of Unoriented Point Clouds by Enforcing Singular Hessian

Authors:

Peng-Shuai Wang,

Shuangmin Chen,

Changhe TuAuthors Info & Claims

ACM Transactions on Graphics (TOG), Volume 42, Issue 6

Article No.: 274, Pages 1 - 14

https://doi.org/10.1145/3618311

Published: 05 December 2023 Publication History

Abstract

Neural implicit representation is a promising approach for reconstructing surfaces from point clouds. Existing methods combine various regularization terms, such as the Eikonal and Laplacian energy terms, to enforce the learned neural function to possess the properties of a Signed Distance Function (SDF). However, inferring the actual topology and geometry of the underlying surface from poor-quality unoriented point clouds remains challenging. In accordance with Differential Geometry, the Hessian of the SDF is singular for points within the differential thin-shell space surrounding the surface. Our approach enforces the Hessian of the neural implicit function to have a zero determinant for points near the surface. This technique aligns the gradients for a near-surface point and its on-surface projection point, producing a rough but faithful shape within just a few iterations. By annealing the weight of the singular-Hessian term, our approach ultimately produces a high-fidelity reconstruction result. Extensive experimental results demonstrate that our approach effectively suppresses ghost geometry and recovers details from unoriented point clouds with better expressiveness than existing fitting-based methods.

Supplemental Material

MP4 File

supplemental

Download
239.65 MB

ZIP File

supplemental

Download
88.34 MB

References

[1]

Matan Atzmon and Yaron Lipman. 2020. SAL: Sign Agnostic Learning of Shapes From Raw Data. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR).

[2]

Matan Atzmon and Yaron Lipman. 2021. SALD: Sign Agnostic Learning with Derivatives. In International Conference on Learning Representations (ICLR).

[3]

Michele Audin, Mihai Damian, and Reinie Erné. 2014. Morse theory and Floer homology. Springer.

[4]

Ma Baorui, Liu Yu-Shen, Zwicker Matthias, and Han Zhizhong. 2022b. Surface Reconstruction from Point Clouds by Learning Predictive Context Priors. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR).

[5]

Ma Baorui, Liu Yu-Shen, and Han Zhizhong. 2022a. Reconstructing Surfaces for Sparse Point Clouds with On-Surface Priors. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR).

[6]

Ma Baorui, Han Zhizhong, Liu Yu-Shen, and Zwicker Matthias. 2021. Neural-Pull: Learning Signed Distance Functions from Point Clouds by Learning to Pull Space onto Surfaces. In Proceedings of the International Conference on Machine Learning (ICML).

[7]

Yizhak Ben-Shabat, Chamin Hewa Koneputugodage, and Stephen Gould. 2022. DiGS: Divergence guided shape implicit neural representation for unoriented point clouds. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR).

[8]

Federica Bogo, Javier Romero, Gerard Pons-Moll, and Michael J. Black. 2017. Dynamic FAUST: Registering Human Bodies in Motion. In Proceedings IEEE Conf. on Computer Vision and Pattern Recognition (CVPR).

[9]

Alexandre Boulch and Renaud Marlet. 2022. POCO: Point Convolution for Surface Reconstruction. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR).

[10]

Fatih Calakli and Gabriel Taubin. 2011. SSD: Smooth Signed Distance Surface Reconstruction. Computer Graphics Forum 30, 7 (2011), 1993--2002.

[11]

Jonathan C Carr, Richard K Beatson, Jon B Cherrie, Tim J Mitchell, W Richard Fright, Bruce C McCallum, and Tim R Evans. 2001. Reconstruction and representation of 3D objects with radial basis functions. In Proceedings of the 28th annual conference on Computer graphics and interactive techniques.

Digital Library

[12]

Rohan Chabra, Jan E Lenssen, Eddy Ilg, Tanner Schmidt, Julian Straub, Steven Lovegrove, and Richard Newcombe. 2020. Deep Local Shapes: Learning local sdf priors for detailed 3d reconstruction. In Computer Vision-ECCV 2020: 16th European Conference, Glasgow, UK, August 23--28, 2020, Proceedings, Part XXIX 16. Springer, 608--625.

[13]

Eric Chan, Marco Monteiro, Petr Kellnhofer, Jiajun Wu, and Gordon Wetzstein. 2021. pi-GAN: Periodic Implicit Generative Adversarial Networks for 3D-Aware Image Synthesis. In Proceedings IEEE Conf. on Computer Vision and Pattern Recognition (CVPR).

[14]

Angel X. Chang, Thomas Funkhouser, Leonidas Guibas, Pat Hanrahan, Qixing Huang, Zimo Li, Silvio Savarese, Manolis Savva, Shuran Song, Hao Su, Jianxiong Xiao, Li Yi, and Fisher Yu. 2015. ShapeNet: An Information-Rich 3D Model Repository. Technical Report arXiv:1512.03012 [cs.GR].

[15]

Julian Chibane, Aymen Mir, and Gerard Pons-Moll. 2020. Neural Unsigned Distance Fields for Implicit Function Learning. In Advances in Neural Information Processing Systems (NeurIPS).

[16]

Philipp Erler, Paul Guerrero, Stefan Ohrhallinger, Niloy J Mitra, and Michael Wimmer. 2020. Points2surf: learning implicit surfaces from point clouds. In Proceedings of the European Conference on Computer Vision (ECCV).

Digital Library

[17]

Amos Gropp, Lior Yariv, Niv Haim, Matan Atzmon, and Yaron Lipman. 2020. Implicit Geometric Regularization for Learning Shapes. In Proceedings of the International Conference on Machine Learning (ICML).

[18]

Hugues Hoppe, Tony DeRose, Tom Duchamp, John Alan McDonald, and Werner Stuetzle. 1992. Surface reconstruction from unorganized points. Proceedings of the 19th annual conference on Computer graphics and interactive techniques.

Digital Library

[19]

Fei Hou, Chiyu Wang, Wencheng Wang, Hong Qin, Chen Qian, and Ying He. 2022. Iterative Poisson Surface Reconstruction (IPSR) for Unoriented Points. ACM Trans. Graph. 41, 4 (2022), 13 pages.

Digital Library

[20]

Jiahui Huang, Hao-Xiang Chen, and Shi-Min Hu. 2022a. A Neural Galerkin Solver for Accurate Surface Reconstruction. ACM Trans. Graph. 41, 6 (2022), 16 pages.

Digital Library

[21]

Zhiyang Huang, Nathan Carr, and Tao Ju. 2019. Variational Implicit Point Set Surfaces. ACM Trans. Graph. 38, 4 (2019), 13 pages.

Digital Library

[22]

Zhangjin Huang, Yuxin Wen, Zihao Wang, Jinjuan Ren, and Kui Jia. 2022b. Surface Reconstruction from Point Clouds: A Survey and a Benchmark. arXiv:arXiv:2205.02413

[23]

Chiyu Jiang, Avneesh Sud, Ameesh Makadia, Jingwei Huang, Matthias Nießner, Thomas Funkhouser, et al. 2020. Local implicit grid representations for 3d scenes. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR).

[24]

Yiwei Jin, Diqiong Jiang, and Ming Cai. 2020. 3D reconstruction using deep learning: a survey. Commun. Inf. Syst. (2020).

[25]

Kazhdan, Fei Hou, Chiyu Wang, Wencheng Wang, Hong Qin, Chen Qian, and Ying He. 2013. Screened Poisson surface reconstruction. ACM Trans. Graph. 32, 3 (2013), 13 pages.

[26]

Misha Kazhdan, Ming Chuang, Szymon M. Rusinkiewicz, and Hugues Hoppe. 2020. Poisson Surface Reconstruction with Envelope Constraints. Computer Graphics Forum 30, 7 (2020), 1993--2002.

[27]

Michael M. Kazhdan, Matthew Bolitho, and Hugues Hoppe. 2006. Poisson surface reconstruction. In Eurographics Symposium on Geometry Processing.

[28]

Diederik P. Kingma and Jimmy Ba. 2014. Adam: A Method for Stochastic Optimization.

[29]

Sebastian Koch, Albert Matveev, Zhongshi Jiang, Francis Williams, Alexey Artemov, Evgeny Burnaev, Marc Alexa, Denis Zorin, and Daniele Panozzo. 2019. ABC: A Big CAD Model Dataset For Geometric Deep Learning. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR).

[30]

Ravikrishna Kolluri. 2008. Provably good moving least squares. ACM Transactions on Algorithms (TALG) (2008).

[31]

Oliver Laric. 2012. Three D Scans. http://threedscans.com/info/.

[32]

Thomas Lewiner, Hélio Lopes, Antônio Wilson Vieira, and Geovan Tavares. 2003. Efficient implementation of marching cubes' cases with topological guarantees. Journal of Graphics Tools 8, 2 (2003), 1--15.

[33]

Manyi Li, Falai Chen, Wenping Wang, and Changhe Tu. 2016. Sparse RBF surface representations. Computer Aided Geometric Design 48 (2016), 49--59.

Digital Library

[34]

Siyou Lin, Dong Xiao, Zuoqiang Shi, and Bin Wang. 2022. Surface Reconstruction from Point Clouds without Normals by Parametrizing the Gauss Formula. ACM Trans. Graph. 42, 2 (2022), 19 pages.

[35]

Yaron Lipman. 2021. Phase Transitions, Distance Functions, and Implicit Neural Representations. In Proceedings of the International Conference on Machine Learning (ICML).

[36]

Ilya Loshchilov and Frank Hutter. 2017. SGDR: Stochastic Gradient Descent with Warm Restarts. In International Conference on Learning Representations (ICLR).

[37]

Daniel Mayost. 2014. Applications of the signed distance function to surface geometry. University of Toronto (Canada).

[38]

Ishit Mehta, Michael Gharbi, Connelly Barnes, Eli Shechtman, Ravi Ramamoorthi, and Manmohan Chandraker. 2021. Modulated Periodic Activations for Generalizable Local Functional Representations. In Proceedings of the IEEE/CVF International Conference on Computer Vision (ICCV).

[39]

Lars Mescheder, Michael Oechsle, Michael Niemeyer, Sebastian Nowozin, and Andreas Geiger. 2019. Occupancy Networks: Learning 3D Reconstruction in Function Space. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR).

[40]

Yutaka Ohtake, Alexander G. Belyaev, Marc Alexa, Greg Turk, and Hans-Peter Seidel. 2003. Multi-level partition of unity implicits. ACM SIGGRAPH 2005 Courses (2003).

Digital Library

[41]

Barrett O'Neill. 2006. Chapter 5 - Shape Operators. In Elementary Differential Geometry (Second Edition) (second edition ed.), Barrett O'Neill (Ed.). Academic Press, Boston, 202--262.

[42]

A Cengiz Öztireli, Gael Guennebaud, and Markus Gross. 2009. Feature preserving point set surfaces based on non-linear kernel regression. In Computer Graphics Forum.

[43]

Jeong Joon Park, Peter Florence, Julian Straub, Richard Newcombe, and Steven Love-grove. 2019. DeepSDF: Learning Continuous Signed Distance Functions for Shape Representation. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR).

[44]

Songyou Peng, Chiyu "Max" Jiang, and Yiyi Liao. 2021. Shape As Points: A Differentiable Poisson Solver. In Advances in Neural Information Processing Systems (NeurIPS).

[45]

Charles R. Qi, Hao Su, Kaichun Mo, and Leonidas J. Guibas. 2017. PointNet: Deep Learning on Point Sets for 3D Classification and Segmentation. In Proceedings IEEE Conf. on Computer Vision and Pattern Recognition (CVPR).

[46]

Sashank J. Reddi, Satyen Kale, and Sanjiv Kumar. 2018. On the Convergence of Adam and Beyond. In International Conference on Learning Representations (ICLR).

[47]

Levent Sagun, Utku Evci, V Ugur Guney, Yann Dauphin, and Leon Bottou. 2017. Empirical analysis of the hessian of over-parametrized neural networks. arXiv preprint arXiv:1706.04454 (2017).

[48]

C. Schroers, S. Setzer, and J. Weickert. 2014. A Variational Taxonomy for Surface Reconstruction from Oriented Points. Computer Graphics Forum 33, 5 (2014), 195--204.

Digital Library

[49]

Silvia Sellán and Alec Jacobson. 2022. Stochastic Poisson Surface Reconstruction. ACM Transactions on Graphics (TOG) (2022).

[50]

Chen Shen, James F. O'Brien, and Jonathan R. Shewchuk. 2004. Interpolating and Approximating Implicit Surfaces from Polygon Soup. In ACM SIGGRAPH 2004 Papers.

[51]

Vincent Sitzmann, Julien N.P. Martel, Alexander W. Bergman, David B. Lindell, and Gordon Wetzstein. 2020. Implicit Neural Representations with Periodic Activation Functions. In Advances in Neural Information Processing Systems (NeurIPS).

[52]

Peng Songyou, Niemeyer Michael, Mescheder Lars, Pollefeys Marc, and Andreas Geiger. 2020. Convolutional Occupancy Networks. In Proceedings of the European Conference on Computer Vision (ECCV).

[53]

Raphael Sulzer, Loic Landrieu, Renaud Marlet, and Bruno Vallet. 2023. A Survey and Benchmark of Automatic Surface Reconstruction from Point Clouds.

[54]

Jiapeng Tang, Jiabao Lei, Dan Xu, Feiying Ma, Kui Jia, and Lei Zhang. 2021b. SA-ConvONet: Sign-Agnostic Optimization of Convolutional Occupancy Networks. In Proceedings of the IEEE/CVF International Conference on Computer Vision.

[55]

Jia-Heng Tang, Weikai Chen, jie Yang, Bo Wang, Songrun Liu, Bo Yang, and Lin Gao. 2021a. OctField: Hierarchical Implicit Functions for 3D Modeling. In Advances in Neural Information Processing Systems (NeurIPS).

[56]

Stéfan van der Walt, Johannes L. Schönberger, Juan Nunez-Iglesias, François Boulogne, Joshua D. Warner, Neil Yager, Emmanuelle Gouillart, Tony Yu, and the scikit-image contributors. 2014. scikit-image: image processing in Python. PeerJ 2 (2014), e453.

[57]

Ignacio Vizzo, Xieyuanli Chen, Nived Chebrolu, Jens Behley, and Cyrill Stachniss. 2021. Poisson Surface Reconstruction for LiDAR Odometry and Mapping. In IEEE International Conference on Robotics and Automation (ICRA).

[58]

Peng-Shuai Wang, Yang Liu, and Xin Tong. 2022. Dual Octree Graph Networks for Learning Adaptive Volumetric Shape Representations. ACM Trans. Graph. 41, 4 (2022), 15 pages.

Digital Library

[59]

Francis Williams, Teseo Schneider, Claudio Silva, Denis Zorin, Joan Bruna, and Daniele Panozzo. 2019. Deep geometric prior for surface reconstruction. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR). IEEE.

[60]

Francis Williams, Matthew Trager, Joan Bruna, and Denis Zorin. 2021. Neural splines: Fitting 3d surfaces with infinitely-wide neural networks. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR).

[61]

Rui Xu, Zixiong Wang, Zhiyang Dou, Chen Zong, Shiqing Xin, Mingyan Jiang, Tao Ju, and Changhe Tu. 2022. RFEPS: Reconstructing Feature-line Equipped Polygonal Surface. ACM Transactions on Graphics (TOG) (2022), 15 pages.

Digital Library

[62]

Jianglong Ye, Yuntao Chen, Naiyan Wang, and Xiaolong Wang. 2022. GIFS: Neural Implicit Function for General Shape Representation. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR).

[63]

Wang Yifan, Lukas Rahmann, and Olga Sorkine-hornung. 2021. Geometry-Consistent Neural Shape Representation with Implicit Displacement Fields. In International Conference on Learning Representations (ICLR).

[64]

Wang Yifan, Shihao Wu, Cengiz Oztireli, and Olga Sorkine-Hornung. 2020. Iso-Points: Optimizing Neural Implicit Surfaces with Hybrid Representations. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR).

[65]

Jingyang Zhang, Yao Yao, Shiwei Li, Tian Fang, David McKinnon, Yanghai Tsin, and Long Quan. 2022. Critical regularizations for neural surface reconstruction in the wild. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. 6270--6279.

[66]

Junsheng Zhou, Baorui Ma, Liu Yu-Shen, Fang Yi, and Han Zhizhong. 2022. Learning Consistency-Aware Unsigned Distance Functions Progressively from Raw Point Clouds. In Advances in Neural Information Processing Systems (NeurIPS).

[67]

Qingnan Zhou and Alec Jacobson. 2016. Thingi10K: A Dataset of 10,000 3D-Printing Models. arXiv preprint arXiv:1605.04797 (2016).

Cited By

Anand VMaurya A(2024)A Survey on Recommender Systems Using Graph Neural NetworkACM Transactions on Information Systems10.1145/369478443:1(1-49)Online publication date: 26-Nov-2024
https://dl.acm.org/doi/10.1145/3694784
Guo DLi MYang YLi SWang G(2024)Barrier-Augmented Lagrangian for GPU-based Elastodynamic ContactACM Transactions on Graphics10.1145/368798843:6(1-17)Online publication date: 19-Nov-2024
https://dl.acm.org/doi/10.1145/3687988
Lyu AZhao SXian CCen ZCai HFang G(2024)Accelerate Neural Subspace-Based Reduced-Order Solver of Deformable Simulation by Lipschitz OptimizationACM Transactions on Graphics10.1145/368796143:6(1-10)Online publication date: 19-Nov-2024
https://dl.acm.org/doi/10.1145/3687961
Show More Cited By

Index Terms

Neural-Singular-Hessian: Implicit Neural Representation of Unoriented Point Clouds by Enforcing Singular Hessian
1. Computing methodologies
  1. Computer graphics
    1. Shape modeling
      1. Point-based models

Recommendations

Frequency-based representation of 3D point-based surfaces using spherical harmonics

In this paper, we propose a precise frequency-based representation for oriented point-based surfaces using spherical harmonics. The representation can be useful in many applications, such as filtering, progressive transmission and coding of 3D surfaces. ...
Visualizing Nonmanifold and Singular Implicit Surfaces with Point Clouds

We use octree spatial subdivision to generate point clouds on complex nonmanifold implicit surfaces in order to visualize them. The new spatial subdivision scheme only uses point sampling and an interval exclusion test. The algorithm includes a test for ...
Hessian-based robust ray-tracing of implicit surfaces on GPU
SA '18: SIGGRAPH Asia 2018 Technical Briefs

In recent years, the Ray Tracing of Implicit Surfaces on a GPU has been studied by many researchers. However, the existing methods have challenges that mainly includes solving for self-intersecting surfaces. General solutions for Ray Tracing suffer from ...

Comments

Information & Contributors

Information

Published In

cover image ACM Transactions on Graphics

ACM Transactions on Graphics Volume 42, Issue 6

December 2023

1565 pages

ISSN:0730-0301

EISSN:1557-7368

DOI:10.1145/3632123

Issue’s Table of Contents

Copyright © 2023 ACM.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than the author(s) must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected].

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 05 December 2023

Published in TOG Volume 42, Issue 6

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Research-article

Funding Sources

Natural Science Foundation of Shandong Province
SpecialFunds for Taishan Scholars Project
National Key R&D Program of China
National Natural Science Foundation of China

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

18
Total Citations
View Citations
301
Total Downloads

Downloads (Last 12 months)208
Downloads (Last 6 weeks)19

Reflects downloads up to 08 Feb 2025

Other Metrics

View Author Metrics

Citations

Cited By

Anand VMaurya A(2024)A Survey on Recommender Systems Using Graph Neural NetworkACM Transactions on Information Systems10.1145/369478443:1(1-49)Online publication date: 26-Nov-2024
https://dl.acm.org/doi/10.1145/3694784
Guo DLi MYang YLi SWang G(2024)Barrier-Augmented Lagrangian for GPU-based Elastodynamic ContactACM Transactions on Graphics10.1145/368798843:6(1-17)Online publication date: 19-Nov-2024
https://dl.acm.org/doi/10.1145/3687988
Lyu AZhao SXian CCen ZCai HFang G(2024)Accelerate Neural Subspace-Based Reduced-Order Solver of Deformable Simulation by Lipschitz OptimizationACM Transactions on Graphics10.1145/368796143:6(1-10)Online publication date: 19-Nov-2024
https://dl.acm.org/doi/10.1145/3687961
Yuan CShi HLan LQiu YYuksel CWang HJiang CWu KYang Y(2024)Volumetric Homogenization for Knitwear SimulationACM Transactions on Graphics10.1145/368791143:6(1-19)Online publication date: 19-Nov-2024
https://dl.acm.org/doi/10.1145/3687911
Yu XYuan ZGuo YLiu YLiu JLi YCao YLiang DQi X(2024)TEXGen: a Generative Diffusion Model for Mesh TexturesACM Transactions on Graphics10.1145/368790943:6(1-14)Online publication date: 19-Nov-2024
https://dl.acm.org/doi/10.1145/3687909
Lin SShi ZLiu Y(2024)Fast and Globally Consistent Normal Orientation based on the Winding Number Normal ConsistencyACM Transactions on Graphics10.1145/368789543:6(1-19)Online publication date: 19-Nov-2024
https://dl.acm.org/doi/10.1145/3687895
Xu LLiu YXu TChen ETang Y(2024)Graph Augmentation Empowered Contrastive Learning for RecommendationACM Transactions on Information Systems10.1145/3677377Online publication date: 12-Jul-2024
https://dl.acm.org/doi/10.1145/3677377
Zeng KShi HLin JLi SCheng JWang KLi ZYang KCai JKankanhalli MPrabhakaran BBoll SSubramanian RZheng LSingh VCesar PXie LXu D(2024)MambaMOS: LiDAR-based 3D Moving Object Segmentation with Motion-aware State Space ModelProceedings of the 32nd ACM International Conference on Multimedia10.1145/3664647.3680578(1505-1513)Online publication date: 28-Oct-2024
https://dl.acm.org/doi/10.1145/3664647.3680578
Dong QXu RWang PChen SXin SJia XWang WTu C(2024)NeurCADRecon: Neural Representation for Reconstructing CAD Surfaces by Enforcing Zero Gaussian CurvatureACM Transactions on Graphics10.1145/365817143:4(1-17)Online publication date: 19-Jul-2024
https://dl.acm.org/doi/10.1145/3658171
Wang YWang CGong BXue T(2024)Bilateral Guided Radiance Field ProcessingACM Transactions on Graphics10.1145/365814843:4(1-13)Online publication date: 19-Jul-2024
https://dl.acm.org/doi/10.1145/3658148
Show More Cited By

View Options

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Article

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Figures

Tables

Media

View Issue’s Table of Contents