research-article

Projective analysis for 3D shape segmentation

Authors:

Daniel Cohen-Or,

Baoquan ChenAuthors Info & Claims

ACM Transactions on Graphics (TOG), Volume 32, Issue 6

Article No.: 192, Pages 1 - 12

https://doi.org/10.1145/2508363.2508393

Published: 01 November 2013 Publication History

Abstract

We introduce projective analysis for semantic segmentation and labeling of 3D shapes. The analysis treats an input 3D shape as a collection of 2D projections, labels each projection by transferring knowledge from existing labeled images, and back-projects and fuses the labelings on the 3D shape. The image-space analysis involves matching projected binary images of 3D objects based on a novel bi-class Hausdorff distance. The distance is topology-aware by accounting for internal holes in the 2D figures and it is applied to piecewise-linearly warped object projections to compensate for part scaling and view discrepancies. Projective analysis simplifies the processing task by working in a lower-dimensional space, circumvents the requirement of having complete and well-modeled 3D shapes, and addresses the data challenge for 3D shape analysis by leveraging the massive available image data. A large and dense labeled set ensures that the labeling of a given projected image can be inferred from closely matched labeled images. We demonstrate semantic labeling of imperfect (e.g., incomplete or self-intersecting) 3D models which would be otherwise difficult to analyze without taking the projective analysis approach.

Supplementary Material

ZIP File (a192-wang.zip)

Supplemental material.

Download
8.11 MB

References

[1]

Baddeley, A. J. 1992. An error metric for binary images. In IEEE Workshop on Robust Computer Vision, 59--78.

[2]

Bai, X., Yang, X., Latecki, L. J., Liu, W., and Tu, Z. 2010. Learning context-sensitive shape similarity by graph transduction. PAMI 32, 5, 861--874.

Digital Library

[3]

Belongie, S., Malik, J., and Puzicha, J. 2002. Shape matching and object recognition using shape context. PAMI 24, 4, 509--522.

Digital Library

[4]

Berg, A. C., Berg, T. L., and Malik, J. 2005. Shape matching and object recognition using low distortion correspondence. In CVPR, 26--33.

Digital Library

[5]

Berndt, D., and Clifford, J. 1994. Using dynamic time warping to find patterns in time series. In KDD workshop, 359--370.

[6]

Blake, A., Rother, C., Brown, M., Perez, P., and Torr, P. 2004. Interactive image segmentation using an adaptive gmmrf model. In ECCV, 428--441.

[7]

Boykov, Y., Veksler, O., and Zabih, R. 2001. Fast approximate energy minimization via graph cuts. PAMI 23, 11, 1222--1239.

Digital Library

[8]

Chang, J. Y., Raskar, R., and Agrawal, A. 2009. 3D pose estimation and segmentation using specular cues. In CVPR, 1706--1713.

[9]

Chen, D., Tian, X., Shen, Y., and Ouhyoung, M. 2003. On visual similarity based 3D model retrieval. CGF (EUROGRAPHICS) 22, 3, 223--232.

[10]

Chen, X., Golovinskiy, A., and Funkhouser, T. 2009. A benchmark for 3D mesh segmentation. ACM Trans. Graph. (SIGGRAPH) 28, 3, 1--12.

Digital Library

[11]

Chen, X., Li, Q., Song, Y., Jin, X., and Zhao, Q. 2012. Supervised geodesic propagation for semantic label transfer. In ECCV, 553--565.

Digital Library

[12]

Csurka, G., Dance, C., Fan, L., Willamowski, J., and Bray, C. 2004. Visual categorization with bags of keypoints. In Workshop on statistical learning in computer vision, ECCV, vol. 1, 22.

[13]

Cyr, C. M., and Kimia, B. B. 2001. 3D object recognition using shape similiarity-based aspect graph. In ICCV, 254--261.

[14]

Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., and FeiFei, L. 2009. Imagenet: A large-scale hierarchical image database. In CVPR, 248--255.

[15]

Duchenne, O., Joulin, A., and Ponce, J. 2011. A graph-matching kernel for object categorization. In ICCV, 1792--1799.

Digital Library

[16]

Eitz, M., Richter, R., Boubekeur, T., Hildebrand, K., and Alexa, M. 2012. Sketch-based shape retrieval. ACM Trans. Graph. (SIGGRAPH) 31, 4, 31--40.

Digital Library

[17]

Ferrari, V., Tuytelaars, T., and Gool, L. V. 2004. Integrating multiple model views for object recognition. In CVPR, 105--112.

Digital Library

[18]

Fleming, R. W., and Singh, M. 2009. Visual perception of 3D shape. In ACM SIGGRAPH 2009 Courses, 24:1--24:94.

Digital Library

[19]

Golovinskiy, A., and Funkhouser, T. 2009. Consistent segmentation of 3D models. Computers & Graphics (SMI) 33, 3, 262--269.

Digital Library

[20]

Kalogerakis, E., Hertzmann, A., and Singh, K. 2010. Learning 3D mesh segmentation and labeling. ACM Trans. Graph. (SIGGRAPH) 29, 3, 1--11.

Digital Library

[21]

Klein, P. N., Sebastian, T. B., and Kimia, B. B. 2001. Shape matching using edit-distance: an implementation. In SODA, 781--790.

Digital Library

[22]

Koenderink, J. J. 1984. What does the occluding contour tell us about solid shape. Perception 13, 321--330.

[23]

Lai, K., Bo, L., Ren, X., and Fox, D. 2012. Detection-based object labeling in 3D scenes. In ICRA, 1330--1337.

[24]

Laurentini, A. 1994. The visual hull concept for silhouette-based image understanding. PAMI 16, 2, 150--162.

Digital Library

[25]

Leordeanu, M., and Hebert, M. 2005. A spectral technique for correspondence problems using pairwise constraints. In ICCV, 1482--1489.

Digital Library

[26]

Li, Y., Zheng, Q., Sharf, A., Cohen-Or, D., Chen, B., and Mitra, N. 2011. 2D-3D fusion for layer decomposition of urban facades. In ICCV, 882--889.

Digital Library

[27]

Lindstrom, P., and Turk, G. 2000. Image-driven simplification. ACM Trans. Graph. 19, 3, 204--241.

Digital Library

[28]

Ling, H., and Jacobs, D. 2007. Shape classification using the inner-distance. PAMI 29, 2, 286--299.

Digital Library

[29]

Liu, R., and Zhang, H. 2007. Mesh segmentation via spectral embedding and contour analysis. CGF (EUROGRAPHICS) 26, 3, 385--394.

[30]

Liu, C., Yuen, J., and Torralba, A. 2011. Nonparametric scene parsing via label transfer. PAMI 33, 12, 2368--2382.

Digital Library

[31]

Liu, C., Yuen, J., and Torralba, A. 2011. SIFT flow: Dense correspondence across scenes and its applications. PAMI 33, 5, 978--994.

Digital Library

[32]

Makadia, A., and Daniilidis, K. 2010. Spherical correlation of visual representations for 3D model retrieval. IJCV 89, 2--3, 193--210.

Digital Library

[33]

Murase, H., and Nayar, S. K. 1995. Visual learning and recognition of 3-d objects from appearance. IJCV 14, 1, 5--24.

Digital Library

[34]

Oliva, A., and Torralba, A. 2001. Modeling the shape of the scene: A holistic representation of the spatial envelope. IJCV 42, 3, 145--175.

Digital Library

[35]

Pepik, B., Gehler, P., Stark, M., and Schiele, B. 2012. 3D²pm-3D deformable part models. In ECCV, 356--370.

Digital Library

[36]

Rother, C., Kolmogorov, V., and Blake, A. 2004. Grabcut: Interactive foreground extraction using iterated graph cuts. ACM Trans. Graph. (SIGGRAPH) 23, 3, 309--314.

Digital Library

[37]

Shamir, A. 2008. A survey on mesh segmentation techniques. CGF 27, 6, 1539--1556.

[38]

Shotton, J., Winn, J., Rother, C., and Criminisi, A. 2006. Textonboost: Joint appearance, shape and context modeling for multi-class object recognition and segmentation. In ECCV, 1--15.

Digital Library

[39]

Sidi, O., van Kaick, O., Kleiman, Y., Zhang, H., and Cohen-Or, D. 2011. Unsupervised co-segmentation of a set of shapes via descriptor-space spectral clustering. ACM Trans. Graph. (SIGGRAPH Asia) 30, 6, 126:1--126:10.

Digital Library

[40]

Telea, A., and Van Wijk, J. J. 1999. Simplified representation of vector fields. In IEEE Visualization, 35--507.

Digital Library

[41]

van Kaick, O., Tagliasacchi, A., Sidi, O., Zhang, H., Cohen-Or, D., Wolf, L., and Hamarneh, G. 2011. Prior knowledge for part correspondence. CGF (EUROGRAPHICS) 30, 2, 553--562.

[42]

Wang, Y., Asafi, S., van Kaick, O., Zhang, H., Cohen-Or, D., and Chen, B. 2012. Active co-analysis of a set of shapes. ACM Trans. Graph. (SIGGRAPH Asia) 31, 6, 165--174.

Digital Library

[43]

Xiao, J., Hays, J., Ehinger, K. A., Oliva, A., and Torralba, A. 2010. Sun database: Large-scale scene recognition from abbey to zoo. In CVPR, 3485--3492.

[44]

Xu, K., Li, H., Zhang, H., Cohen-Or, D., Xiong, Y., and Cheng, Z. 2010. Style-content separation by anisotropic part scales. ACM Trans. Graph. (SIGGRAPH Asia) 29, 5.

Digital Library

[45]

Xu, K., Zheng, H., Zhang, H., Cohen-Or, D., Liu, L., and Xiong, Y. 2011. Photo-inspired model-driven 3D object modeling. ACM Trans. Graph. (SIGGRAPH) 30, 4, 80:1--80:10.

Digital Library

[46]

Zhang, H., Xiao, J., and Quan, L. 2010. Supervised label transfer for semantic segmentation of street scenes. In ECCV, 561--574.

Digital Library

Cited By

Shu ZLi SXin SLiu L(2025)3D Shape Segmentation With Potential Consistency Mining and EnhancementIEEE Transactions on Multimedia10.1109/TMM.2024.352167427(133-144)Online publication date: 2025
https://doi.org/10.1109/TMM.2024.3521674
Shu ZWu TShen JXin SLiu L(2024)Semi-Supervised 3D Shape Segmentation via Self RefiningIEEE Transactions on Image Processing10.1109/TIP.2024.337420033(2044-2057)Online publication date: 2024
https://doi.org/10.1109/TIP.2024.3374200
Shu ZSun XPang CXin S(2024)3D Shape Segmentation via Attentive Nonuniform DownsamplingIEEE Transactions on Circuits and Systems for Video Technology10.1109/TCSVT.2024.343209534:12(12184-12196)Online publication date: Dec-2024
https://doi.org/10.1109/TCSVT.2024.3432095
Show More Cited By

Index Terms

Projective analysis for 3D shape segmentation
1. Computing methodologies
  1. Computer graphics
    1. Shape modeling
2. Theory of computation
  1. Randomness, geometry and discrete structures

Recommendations

Semi-Supervised Co-Analysis of 3D Shape Styles from Projected Lines

We present a semi-supervised co-analysis method for learning 3D shape styles from projected feature lines, achieving style patch localization with only weak supervision. Given a collection of 3D shapes spanning multiple object categories and styles, we ...
A 3D shape descriptor based on spectral analysis of medial axis

The medial axis of a 3D shape is widely known for its ability as a compact and complete shape representation. However, there is still lack of a generative description defined over the medial axis directly which limits its actual application to 3D shape ...
Fine-grained semi-supervised labeling of large shape collections

In this paper we consider the problem of classifying shapes within a given category (e.g., chairs) into finer-grained classes (e.g., chairs with arms, rocking chairs, swivel chairs). We introduce a multi-label (i.e., shapes can belong to multiple ...

Comments

Information & Contributors

Information

Published In

cover image ACM Transactions on Graphics

ACM Transactions on Graphics Volume 32, Issue 6

November 2013

671 pages

ISSN:0730-0301

EISSN:1557-7368

DOI:10.1145/2508363

Issue’s Table of Contents

Copyright © 2013 ACM.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 01 November 2013

Published in TOG Volume 32, Issue 6

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Research-article

Funding Sources

National Natural Science Foundation of China
Ministry of Science and Technology of the People's Republic of China
Israel Science Foundation
Guangdong Science and Technology Program
Natural Sciences and Engineering Research Council of Canada
Shenzhen Science and Innovation Program

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

90
Total Citations
View Citations
881
Total Downloads

Downloads (Last 12 months)28
Downloads (Last 6 weeks)7

Reflects downloads up to 11 Feb 2025

Other Metrics

View Author Metrics

Citations

Cited By

Shu ZLi SXin SLiu L(2025)3D Shape Segmentation With Potential Consistency Mining and EnhancementIEEE Transactions on Multimedia10.1109/TMM.2024.352167427(133-144)Online publication date: 2025
https://doi.org/10.1109/TMM.2024.3521674
Shu ZWu TShen JXin SLiu L(2024)Semi-Supervised 3D Shape Segmentation via Self RefiningIEEE Transactions on Image Processing10.1109/TIP.2024.337420033(2044-2057)Online publication date: 2024
https://doi.org/10.1109/TIP.2024.3374200
Shu ZSun XPang CXin S(2024)3D Shape Segmentation via Attentive Nonuniform DownsamplingIEEE Transactions on Circuits and Systems for Video Technology10.1109/TCSVT.2024.343209534:12(12184-12196)Online publication date: Dec-2024
https://doi.org/10.1109/TCSVT.2024.3432095
Dutt NMuralikrishnan SMitra N(2024)Diffusion 3D Features (Diff3F) Decorating Untextured Shapes with Distilled Semantic Features2024 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)10.1109/CVPR52733.2024.00430(4494-4504)Online publication date: 16-Jun-2024
https://doi.org/10.1109/CVPR52733.2024.00430
Kim JMo KSung MWoo W(2023)Seg&Struct: The Interplay Between Part Segmentation and Structure Inference for 3D Shape Parsing2023 IEEE/CVF Winter Conference on Applications of Computer Vision (WACV)10.1109/WACV56688.2023.00128(1226-1235)Online publication date: Jan-2023
https://doi.org/10.1109/WACV56688.2023.00128
Yu FQian YGil-Ureta FJackson BBennett EZhang H(2023)HAL3D: Hierarchical Active Learning for Fine-Grained 3D Part Labeling2023 IEEE/CVF International Conference on Computer Vision (ICCV)10.1109/ICCV51070.2023.00086(865-875)Online publication date: 1-Oct-2023
https://doi.org/10.1109/ICCV51070.2023.00086
Liu RAigerman NKim VHanocka R(2023)DA Wand: Distortion-Aware Selection Using Neural Mesh Parameterization2023 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)10.1109/CVPR52729.2023.01606(16739-16749)Online publication date: Jun-2023
https://doi.org/10.1109/CVPR52729.2023.01606
Wu LHou YXu JZhao Y(2022)Robust Mesh Segmentation Using Feature-Aware Region FusionSensors10.3390/s2301041623:1(416)Online publication date: 30-Dec-2022
https://doi.org/10.3390/s23010416
Shi WLi Z(2022)A light and efficient attention model for 3D shape part-segmentationProceedings of the 5th International Conference on Computer Science and Software Engineering10.1145/3569966.3570055(287-292)Online publication date: 21-Oct-2022
https://dl.acm.org/doi/10.1145/3569966.3570055
Wen TZhuang JDu YYang LXu J(2021)Dual-Sampling Attention Pooling for Graph Neural Networks on 3D MeshComputer Methods and Programs in Biomedicine10.1016/j.cmpb.2021.106250208:COnline publication date: 1-Sep-2021
https://dl.acm.org/doi/10.1016/j.cmpb.2021.106250
Show More Cited By

View Options

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Article

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Figures

Tables

Media

View Issue’s Table of Contents