research-article

Open access

DEF: deep estimation of sharp geometric features in 3D shapes

Authors:

Albert Matveev,

Ruslan Rakhimov,

Alexey Artemov,

Gleb Bobrovskikh,

Vage Egiazarian,

Emil Bogomolov,

Daniele Panozzo,

Evgeny BurnaevAuthors Info & Claims

ACM Transactions on Graphics (TOG), Volume 41, Issue 4

Article No.: 108, Pages 1 - 22

https://doi.org/10.1145/3528223.3530140

Published: 22 July 2022 Publication History

Abstract

We propose Deep Estimators of Features (DEFs), a learning-based framework for predicting sharp geometric features in sampled 3D shapes. Differently from existing data-driven methods, which reduce this problem to feature classification, we propose to regress a scalar field representing the distance from point samples to the closest feature line on local patches. Our approach is the first that scales to massive point clouds by fusing distance-to-feature estimates obtained on individual patches.

We extensively evaluate our approach against related state-of-the-art methods on newly proposed synthetic and real-world 3D CAD model benchmarks. Our approach not only outperforms these (with improvements in Recall and False Positives Rates), but generalizes to real-world scans after training our model on synthetic data and fine-tuning it on a small dataset of scanned data.

We demonstrate a downstream application, where we reconstruct an explicit representation of straight and curved sharp feature lines from range scan data.

We make code, pre-trained models, and our training and evaluation datasets available at https://github.com/artonson/def.

Supplemental Material

MP4 File

presentation

Download
886.10 MB

SRT File

presentation

Download
15.93 KB

ZIP File

supplemental material

Download
3.53 MB

References

[1]

D. Bazazian, J. R. Casas, and J. Ruiz-Hidalgo. 2015. Fast and Robust Edge Extraction in Unorganized Point Clouds. In 2015 International Conference on Digital Image Computing: Techniques and Applications (DICTA). 1--8.

[2]

D. Bazazian and ME. Parés. 2021. EDC-Net: Edge Detection Capsule Network for 3D Point Clouds. Applied Sciences 11, 4: 1833 (2021), 1--16.

[3]

Yuanhao Cao, Liangliang Nan, and Peter Wonka. 2016. Curve networks for surface reconstruction. arXiv preprint arXiv:1603.08753 (2016).

[4]

Paolo Cignoni, Marco Callieri, Massimiliano Corsini, Matteo Dellepiane, Fabio Ganovelli, and Guido Ranzuglia. 2008. Meshlab: an open-source mesh processing tool. In Eurographics Italian chapter conference, Vol. 2008. Salerno, Italy, 129--136.

[5]

Joel II Daniels, Linh K Ha, Tilo Ochotta, and Claudio T Silva. 2007. Robust smooth feature extraction from point clouds. In IEEE International Conference on Shape Modeling and Applications 2007 (SMI'07). IEEE, 123--136.

Digital Library

[6]

Joel Daniels Ii, Tilo Ochotta, Linh K Ha, and Cláudio T Silva. 2008. Spline-based feature curves from point-sampled geometry. The Visual Computer 24, 6 (2008), 449--462.

Digital Library

[7]

Kris Demarsin, Denis Vanderstraeten, Tim Volodine, and Dirk Roose. 2007. Detection of closed sharp edges in point clouds using normal estimation and graph theory. Computer-Aided Design 39, 4 (2007), 276--283.

Digital Library

[8]

WA Falcon. 2019. PyTorch Lightning. GitHub. Note: https://github.com/PyTorchLightning/pytorch-lightning 3 (2019).

[9]

Shachar Fleishman, Daniel Cohen-Or, and Cláudio T Silva. 2005. Robust moving least-squares fitting with sharp features. ACM transactions on graphics (TOG) 24, 3 (2005), 544--552.

[10]

Adrien Gaidon, Qiao Wang, Yohann Cabon, and Eleonora Vig. 2016. Virtual worlds as proxy for multi-object tracking analysis. In Proceedings of the IEEE conference on computer vision and pattern recognition. 4340--4349.

[11]

T. Hackel, J. D. Wegner, and K. Schindler. 2016. Contour Detection in Unstructured 3D Point Clouds. In 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR). 1610--1618.

[12]

Timo Hackel, Jan D. Wegner, and Konrad Schindler. 2017. Joint classification and contour extraction of large 3D point clouds. ISPRS Journal of Photogrammetry and Remote Sensing 130 (2017), 231--245.

[13]

Ankur Handa, Viorica Patraucean, Vijay Badrinarayanan, Simon Stent, and Roberto Cipolla. 2016. Understanding real world indoor scenes with synthetic data. In Proceedings of the IEEE conference on computer vision and pattern recognition. 4077--4085.

[14]

JH Hannay and JF Nye. 2004. Fibonacci numerical integration on a sphere. Journal of Physics A: Mathematical and General 37, 48 (2004), 11591.

[15]

Richard Hartley and Andrew Zisserman. 2004. Multiple View Geometry in Computer Vision (2 ed.). Cambridge University Press.

[16]

Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun. 2016. Deep residual learning for image recognition. In Proceedings of the IEEE conference on computer vision and pattern recognition. 770--778.

[17]

Peter Henderson, Jieru Hu, Joshua Romoff, Emma Brunskill, Dan Jurafsky, and Joelle Pineau. 2020. Towards the systematic reporting of the energy and carbon footprints of machine learning. Journal of Machine Learning Research 21, 248 (2020), 1--43.

[18]

Chems-Eddine Himeur, Thibault Lejemble, Thomas Pellegrini, Mathias Paulin, Loic Barthe, and Nicolas Mellado. 2021. PCEDNet: A Lightweight Neural Network for Fast and Interactive Edge Detection in 3D Point Clouds. ACM Transactions on Graphics (TOG) 41, 1 (2021), 1--21.

Digital Library

[19]

Hui Huang, Shihao Wu, Minglun Gong, Daniel Cohen-Or, Uri Ascher, and Hao Richard Zhang. 2013. Edge-aware point set resampling. ACM transactions on graphics (TOG) 32, 1 (2013), 9.

[20]

Peter J Huber et al. 1973. Robust regression: asymptotics, conjectures and Monte Carlo. The annals of statistics 1, 5 (1973), 799--821.

[21]

Ehsan Imani and Martha White. 2018. Improving Regression Performance with Distributional Losses (Proceedings of Machine Learning Research, Vol. 80), Jennifer Dy and Andreas Krause (Eds.). PMLR, Stockholmsmässan, Stockholm Sweden, 2157--2166. http://proceedings.mlr.press/v80/imani18a.html

[22]

Tejas Khot, Shubham Agrawal, Shubham Tulsiani, Christoph Mertz, Simon Lucey, and Martial Hebert. 2019. Learning Unsupervised Multi-View Stereopsis via Robust Photometric Consistency. arXiv:1905.02706 [cs.CV]

[23]

Sangpil Kim, Hyung-gun Chi, Xiao Hu, Qixing Huang, and Karthik Ramani. 2020. A Large-scale Annotated Mechanical Components Benchmark for Classification and Retrieval Tasks with Deep Neural Networks. In Proceedings of 16th European Conference on Computer Vision (ECCV).

Digital Library

[24]

Diederik P Kingma and Jimmy Ba. 2014. Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014).

[25]

Sebastian Koch, Albert Matveev, Zhongshi Jiang, Francis Williams, Alexey Artemov, Evgeny Burnaev, Marc Alexa, Denis Zorin, and Daniele Panozzo. 2019. ABC: A big CAD model dataset for geometric deep learning. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. 9601--9611.

[26]

Eric-Tuan Lê, Minhyuk Sung, Duygu Ceylan, Radomir Mech, Tamy Boubekeur, and Niloy J Mitra. 2021. CPFN: Cascaded Primitive Fitting Networks for High-Resolution Point Clouds. In Proceedings of the IEEE/CVF International Conference on Computer Vision. 7457--7466.

[27]

Kai Wah Lee and Pengbo Bo. 2016. Feature curve extraction from point clouds via developable strip intersection. Journal of Computational Design and Engineering 3, 2 (2016), 102--111.

[28]

Lingxiao Li, Minhyuk Sung, Anastasia Dubrovina, Li Yi, and Leonidas J Guibas. 2019. Supervised fitting of geometric primitives to 3d point clouds. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. 2652--2660.

[29]

Y. Lin, C. Wang, B. Chen, D. Zai, and J. Li. 2017. Facet Segmentation-Based Line Segment Extraction for Large-Scale Point Clouds. IEEE Transactions on Geoscience and Remote Sensing 55, 9 (2017), 4839--4854.

[30]

Yangbin Lin, Cheng Wang, Jun Cheng, Bili Chen, Fukai Jia, Zhonggui Chen, and Jonathan Li. 2015. Line segment extraction for large scale unorganized point clouds. ISPRS Journal of Photogrammetry and Remote Sensing 102 (2015), 172--183.

[31]

Yujia Liu, Stefano D'Aronco, Konrad Schindler, and Jan Dirk Wegner. 2021. PC2WF: 3D Wireframe Reconstruction from Raw Point Clouds. CoRR abs/2103.02766 (2021). arXiv:2103.02766 https://arxiv.org/abs/2103.02766

[32]

Albert Matveev, Alexey Artemov, Denis Zorin, and Evgeny Burnaev. 2021. 3D Parametric Wireframe Extraction Based on Distance Fields. In 2021 4th International Conference on Artificial Intelligence and Pattern Recognition (Xiamen, China) (AIPR 2021). Association for Computing Machinery, New York, NY, USA, 316--322.

Digital Library

[33]

Quentin Mérigot, Maks Ovsjanikov, and Leonidas J Guibas. 2010. Voronoi-based curvature and feature estimation from point clouds. IEEE Transactions on Visualization and Computer Graphics 17, 6 (2010), 743--756.

Digital Library

[34]

Open CASCADE Technology OCCT 2021. Open CASCADE Technology OCCT. https://www.opencascade.com/. Accessed: 2021-06-01.

[35]

Parasolid: 3D Geometric Modeling Engine 2021. Parasolid: 3D Geometric Modeling Engine. https://www.plm.automation.siemens.com/global/en/products/plm-components/parasolid.html. Accessed: 2021-06-01.

[36]

Adam Paszke, Sam Gross, Francisco Massa, Adam Lerer, James Bradbury, Gregory Chanan, Trevor Killeen, Zeming Lin, Natalia Gimelshein, Luca Antiga, Alban Desmaison, Andreas Kopf, Edward Yang, Zachary DeVito, Martin Raison, Alykhan Tejani, Sasank Chilamkurthy, Benoit Steiner, Lu Fang, Junjie Bai, and Soumith Chintala. 2019. PyTorch: An Imperative Style, High-Performance Deep Learning Library. In Advances in Neural Information Processing Systems 32, H. Wallach, H. Larochelle, A. Beygelzimer, F. d Alche-Buc, E. Fox, and R. Garnett (Eds.). Curran Associates, Inc., 8024--8035. http://papers.neurips.cc/paper/9015-pytorch-an-imperative-style-high-performance-deep-learning-library.pdf

Digital Library

[37]

Charles Ruizhongtai Qi, Li Yi, Hao Su, and Leonidas J Guibas. 2017. Pointnet++ : Deep hierarchical feature learning on point sets in a metric space. In Advances in neural information processing systems. 5099--5108.

[38]

Prashant Raina, Sudhir Mudur, and Tiberiu Popa. 2019. Sharpness fields in point clouds using deep learning. Computers & Graphics 78 (2019), 37--53.

[39]

Range Vision Spectrum 2021. RangeVision Spectrum - a new 3D high-resolution scanner. https://rangevision.com/en/products/spectrum/. Accessed: 2021-06-01.

[40]

Olaf Ronneberger, Philipp Fischer, and Thomas Brox. 2015. U-net: Convolutional networks for biomedical image segmentation. In International Conference on Medical image computing and computer-assisted intervention. Springer, 234--241.

[41]

Gopal Sharma, Difan Liu, Subhransu Maji, Evangelos Kalogerakis, Siddhartha Chaudhuri, and Radomír Měch. 2020. Parsenet: A parametric surface fitting network for 3d point clouds. In European Conference on Computer Vision. Springer, 261--276.

Digital Library

[42]

Maria-Laura Torrente, Silvia Biasotti, and Bianca Falcidieno. 2018. Recognition of feature curves on 3D shapes using an algebraic approach to Hough transforms. Pattern Recognition 73 (2018), 111--130.

[43]

Xiaogang Wang, Yuelang Xu, Kai Xu, Andrea Tagliasacchi, Bin Zhou, Ali Mahdavi-Amiri, and Hao Zhang. 2020. PIE-NET: Parametric Inference of Point Cloud Edges. Advances in Neural Information Processing Systems 33 (2020).

[44]

Yue Wang, Yongbin Sun, Ziwei Liu, Sanjay E Sarma, Michael M Bronstein, and Justin M Solomon. 2019. Dynamic graph cnn for learning on point clouds. ACM Transactions on Graphics (TOG) 38, 5 (2019), 1--12.

Digital Library

[45]

Christopher Weber, Stefanie Hahmann, and Hans Hagen. 2010. Sharp feature detection in point clouds. In 2010 Shape Modeling International Conference. IEEE, 175--186.

Digital Library

[46]

Karl D. D. Willis, Yewen Pu, Jieliang Luo, Hang Chu, Tao Du, Joseph G. Lambourne, Armando Solar-Lezama, and Wojciech Matusik. 2020. Fusion 360 Gallery: A Dataset and Environment for Programmatic CAD Reconstruction. arXiv preprint arXiv:2010.02392 (2020).

[47]

Nan Xue, Song Bai, Fudong Wang, Gui-Song Xia, Tianfu Wu, and Liangpei Zhang. 2019. Learning attraction field representation for robust line segment detection. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. 1595--1603.

[48]

Omry Yadan. 2019. Hydra - A framework for elegantly configuring complex applications. Github. https://github.com/facebookresearch/hydra

[49]

Lequan Yu, Xianzhi Li, Chi-Wing Fu, Daniel Cohen-Or, and Pheng-Ann Heng. 2018. EC-Net: an Edge-aware Point set Consolidation Network. In Proceedings of the European Conference on Computer Vision (ECCV). 386--402.

Digital Library

[50]

Igor Zacharov, Rinat Arslanov, Maksim Gunin, Daniil Stefonishin, Andrey Bykov, Sergey Pavlov, Oleg Panarin, Anton Maliutin, Sergey Rykovanov, and Maxim Fedorov. 2019. "Zhores"-Petaflops supercomputer for data-driven modeling, machine learning and artificial intelligence installed in Skolkovo Institute of Science and Technology. Open Engineering 9, 1 (2019), 512--520.

Cited By

Xie YTu ZYang TZhang YZhou X(2025)EdgeFormer: local patch-based edge detection transformer on point cloudsPattern Analysis & Applications10.1007/s10044-024-01386-628:1Online publication date: 1-Mar-2025
https://dl.acm.org/doi/10.1007/s10044-024-01386-6
Liu YChen JPan SCohen-Or DZhang HHuang H(2024)Split-and-Fit: Learning B-Reps via Structure-Aware Voronoi PartitioningACM Transactions on Graphics10.1145/365815543:4(1-13)Online publication date: 19-Jul-2024
https://dl.acm.org/doi/10.1145/3658155
Di SDeng HChen DMao XZou YWu LGuo YLin YZhang L(2024)PECo: A Point-Edge Collaborative Framework for Global-Aware Urban Building Contouring From Unstructured Point CloudsIEEE Transactions on Geoscience and Remote Sensing10.1109/TGRS.2023.334280762(1-19)Online publication date: 2024
https://doi.org/10.1109/TGRS.2023.3342807
Show More Cited By

Index Terms

DEF: deep estimation of sharp geometric features in 3D shapes
1. Computing methodologies

Recommendations

Classification of radiolarian images with hand-crafted and deep features

Radiolarians are planktonic protozoa and are important biostratigraphic and paleoenvironmental indicators for paleogeographic reconstructions. Radiolarian paleontology still remains as a low cost and the one of the most convenient way to obtain dating ...
Generative adversarial network based synthetic data training model for lightweight convolutional neural networks
Abstract
Inadequate training data is a significant challenge for deep learning techniques, particularly in applications where data is difficult to get, and publicly available datasets are uncommon owing to ethical and privacy concerns. Various approaches, ...
Deep feature extraction with tri-channel textual feature map for text classification
Abstract
The complexity and diversity of texts make it difficult for shallow text classification models to capture deeper text features. Therefore, this paper takes advantage of the BiLSTM-CNN hybrid network based on the self-attention mechanism to ...
Highlights
- We propose a novel text feature representation in the form of a tri-channel textual feature map.
- We designed a deep feature extraction network to capture deeper features in the text.
- We construct a deep feature extraction text ...

Comments

Information & Contributors

Information

Published In

cover image ACM Transactions on Graphics

ACM Transactions on Graphics Volume 41, Issue 4

July 2022

1978 pages

ISSN:0730-0301

EISSN:1557-7368

DOI:10.1145/3528223

Issue’s Table of Contents

Copyright © 2022 ACM.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 22 July 2022

Published in TOG Volume 41, Issue 4

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Research-article

Funding Sources

Analytical center under the RF Government

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

24
Total Citations
View Citations
1,056
Total Downloads

Downloads (Last 12 months)381
Downloads (Last 6 weeks)43

Reflects downloads up to 27 Jan 2025

Other Metrics

View Author Metrics

Citations

Cited By

Xie YTu ZYang TZhang YZhou X(2025)EdgeFormer: local patch-based edge detection transformer on point cloudsPattern Analysis & Applications10.1007/s10044-024-01386-628:1Online publication date: 1-Mar-2025
https://dl.acm.org/doi/10.1007/s10044-024-01386-6
Liu YChen JPan SCohen-Or DZhang HHuang H(2024)Split-and-Fit: Learning B-Reps via Structure-Aware Voronoi PartitioningACM Transactions on Graphics10.1145/365815543:4(1-13)Online publication date: 19-Jul-2024
https://dl.acm.org/doi/10.1145/3658155
Di SDeng HChen DMao XZou YWu LGuo YLin YZhang L(2024)PECo: A Point-Edge Collaborative Framework for Global-Aware Urban Building Contouring From Unstructured Point CloudsIEEE Transactions on Geoscience and Remote Sensing10.1109/TGRS.2023.334280762(1-19)Online publication date: 2024
https://doi.org/10.1109/TGRS.2023.3342807
Xin XHuang WZhong SZhang MLiu ZXie Z(2024)Accurate and complete line segment extraction for large-scale point cloudsInternational Journal of Applied Earth Observation and Geoinformation10.1016/j.jag.2024.103728128(103728)Online publication date: Apr-2024
https://doi.org/10.1016/j.jag.2024.103728
Cao LXu YGuo JLiu X(2024)WireframeNetComputers and Graphics10.1016/j.cag.2023.07.015115:C(226-235)Online publication date: 1-Feb-2024
https://dl.acm.org/doi/10.1016/j.cag.2023.07.015
Fugacci URomanengo CFalcidieno BBiasotti S(2024)Reconstruction and Preservation of Feature Curves in 3D Point Cloud ProcessingComputer-Aided Design10.1016/j.cad.2023.103649167:COnline publication date: 4-Mar-2024
https://dl.acm.org/doi/10.1016/j.cad.2023.103649
Ning XShi QLiu YJin HWang YZhang XGuo J(2024)DRC-NET: Density Reweighted Convolution Network for Edge Curve ExtractionPattern Recognition and Computer Vision10.1007/978-981-97-8490-5_29(406-420)Online publication date: 18-Oct-2024
https://dl.acm.org/doi/10.1007/978-981-97-8490-5_29
Burnaev EMironov EShpilman AMironenko MKatalevsky D(2023)Practical AI Cases for Solving ESG ChallengesSustainability10.3390/su15171273115:17(12731)Online publication date: 23-Aug-2023
https://doi.org/10.3390/su151712731
Zhao TBusé LCohen-Steiner DBoubekeur TThiery JAlliez P(2023)Variational Shape Reconstruction via Quadric Error MetricsACM SIGGRAPH 2023 Conference Proceedings10.1145/3588432.3591529(1-10)Online publication date: 23-Jul-2023
https://dl.acm.org/doi/10.1145/3588432.3591529
Li YLiu SYang XGuo JGuo JGuo Y(2023)Surface and Edge Detection for Primitive Fitting of Point CloudsACM SIGGRAPH 2023 Conference Proceedings10.1145/3588432.3591522(1-10)Online publication date: 23-Jul-2023
https://dl.acm.org/doi/10.1145/3588432.3591522
Show More Cited By

View Options

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Article

Figures

Tables

Media

View Issue’s Table of Contents