Few-shot 3D Point Cloud Semantic Segmentation with Prototype Alignment
Author: 
Maolin WeiAuthors Info & Claims
Maolin WeiAuthors Info & ClaimsPages 195 - 200
Abstract
Semantic Segmentation for 3D point clouds has made great progress in recent years. Most existing approaches for 3D point cloud segmentation are fully supervised, and they require a large number of well-annotated data for training. The training data is cost and quite difficult to obtain. Moreover, these fully supervised approaches cannot segment new classes well that are unseen in the training process. Thus, Few-shot segmentation has been developed to mitigate these limitations by learning to perform segment from a few labeled examples. In this paper, we propose a method to more adequately utilize information of query set and support set to promote performance of semantic segmentation for 3D point clouds. Specifically, we first extract support and query features and generate multiple prototypes to map the distribution of point clouds. Then we apply a transductive label propagation method to exploit the relations between labeled multi-prototypes and unlabeled points, and between pairs of unlabeled points. Finally, we utilize query points and predicted query masks to perform segmentation for support points. Our proposed method shows improvements for specific classes on S3DIS dataset compared to baselines in 2/3-way 1-shot point cloud semantic segmentation.
References
[1]
C. Choy, J. Gwak, and S. Savarese. 4d spatio-temporal convnets: Minkowski convolutional neural networks. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 3075–3084, 2019.
[2]
B. Graham, M. Engelcke, and L. Van Der Maaten. 3d semantic segmentation with submanifold sparse convolutional networks. In Proceedings of the IEEE conference on computer vision and pattern recognition, pages 9224–9232, 2018.
[3]
J. Huang and S. You. Point cloud labeling using 3d convolutional neural network. In 2016 23rd International Conference on Pattern Recognition (ICPR), pages 2670–2675. IEEE, 2016.
[4]
Q. Huang, W. Wang, and U. Neumann. Recurrent slice networks for 3d segmentation of point clouds. In Proceedings of the IEEE conference on computer vision and pattern recognition, pages 2626–2635, 2018.
[5]
L. Landrieu and M. Simonovsky. Large-scale point cloud semantic segmentation with superpoint graphs. In Proceedings of the IEEE conference on computer vision and pattern recognition, pages 4558–4567, 2018.
[6]
F. J. Lawin, M. Danelljan, P. Tosteberg, G. Bhat, F. S. Khan, and M. Felsberg. Deep projective 3d semantic segmentation. In International Conference on Computer Analysis of Images and Patterns, pages 95–107. Springer, 2017.
[7]
C. R. Qi, H. Su, K. Mo, and L. J. Guibas. Pointnet: Deep learning on point sets for 3d classification and segmentation. In Proceedings of the IEEE conference on computer vision and pattern recognition, pages 652–660, 2017.
[8]
C. R. Qi, L. Yi, H. Su, and L. J. Guibas. Pointnet++: Deep hierarchical feature learning on point sets in a metric space. Advances in neural information processing systems, 30, 2017.
[9]
B. Wu, A. Wan, X. Yue, and K. Keutzer. Squeezeseg: Convolutional neural nets with recurrent crf for real-time roadobject segmentation from 3d lidar point cloud. In 2018 IEEE International Conference on Robotics and Automation (ICRA), pages 1887–1893. IEEE, 2018.
[10]
S. Xie, J. Gu, D. Guo, C. R. Qi, L. Guibas, and O. Litany. Pointcontrast: Unsupervised pre-training for 3d point cloud understanding. In European conference on computer vision, pages 574–591. Springer, 2020.
[11]
S. Guinard and L. Landrieu. Weakly supervised segmentation-aided classification of urban scenes from 3d lidar point clouds. In ISPRS Workshop 2017, 2017.
[12]
X. Xu and G. H. Lee. Weakly supervised semantic point cloud segmentation: Towards 10x fewer labels. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 13706–13715, 2020.
[13]
J. Mei, B. Gao, D. Xu, W. Yao, X. Zhao, and H. Zhao. Semantic segmentation of 3d lidar data in dynamic scene using semi-supervised learning. IEEE Transactions on Intelligent Transportation Systems, 21(6):2496–2509, 2019.
[14]
X. Li, L. Feng, L. Li, and C. Wang. Few-shot meta-learning on point cloud for semantic segmentation. arXiv preprint arXiv:2104.02979, 2021.
[15]
G. Sharma, B. Dash, A. RoyChowdhury, M. Gadelha, M. Loizou, L. Cao, R.Wang, E. Learned-Miller, S. Maji, and E. Kalogerakis. Prifit: Learning to fit primitives improves few shot point cloud segmentation. In Computer Graphics Forum, volume 41, pages 39–50. Wiley Online Library, 2022.
[16]
N. Zhao, T.-S. Chua, and G. H. Lee. Few-shot 3d point cloud semantic segmentation. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 8873–8882, 2021.
[17]
K. Wang, J. H. Liew, Y. Zou, D. Zhou, and J. Feng. Panet: Few-shot image semantic segmentation with prototype alignment. In Proceedings of the IEEE/CVF International Conference on Computer Vision, pages 9197–9206, 2019.
[18]
O. Vinyals, C. Blundell, T. Lillicrap, D. Wierstra, Matching networks for one shot learning. Advances in neural information processing systems, 29, 2016.
[19]
J. Snell, K. Swersky, and R. Zemel. Prototypical networks for few-shot learning. Advances in neural information processing systems, 30, 2017.
[20]
N. Dong and E. P. Xing. Few-shot semantic segmentation with prototype learning. In BMVC, volume 3, 2018.
[21]
C. Zhang, G. Lin, F. Liu, J. Guo, Q. Wu, and R. Yao. Pyramid graph networks with connection attentions for regionbased one-shot semantic segmentation. In Proceedings of the IEEE/CVF International Conference on Computer Vision, pages 9587–9595, 2019.
[22]
C. Zhang, G. Lin, F. Liu, R. Yao, and C. Shen. Canet: Class-agnostic segmentation networks with iterative refinement and attentive few-shot learning. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 5217–5226, 2019.
[23]
F. Sung, Y. Yang, L. Zhang, T. Xiang, P. H. Torr, and T. M. Hospedales. Learning to compare: Relation network for fewshot learning. In Proceedings of the IEEE conference on computer vision and pattern recognition, pages 1199–1208, 2018.
[24]
Y. Wang, Y. Sun, Z. Liu, S. E. Sarma, M. M. Bronstein, and J. M. Solomon. Dynamic graph cnn for learning on point clouds. Acm Transactions On Graphics (tog), 38(5):1–12, 2019.
[25]
I. Armeni, O. Sener, A. R. Zamir, H. Jiang, I. Brilakis, M. Fischer, and S. Savarese. 3d semantic parsing of largescale indoor spaces. In Proceedings of the IEEE conference on computer vision and pattern recognition, pages 1534–1543, 2016.
Index Terms
- Few-shot 3D Point Cloud Semantic Segmentation with Prototype Alignment
Recommendations
Enhancing Few-Shot 3D Point Cloud Semantic Segmentation through Bidirectional Prototype Learning
ICRAI '23: Proceedings of the 2023 9th International Conference on Robotics and Artificial IntelligenceIn recent years, significant strides have been made in point cloud semantic segmentation, which, however, are unspectacular when the training is deprived of sufficient densely-annotated samples, especially with the face of new classes unseen during the ...
Crossmodal Few-shot 3D Point Cloud Semantic Segmentation
MM '22: Proceedings of the 30th ACM International Conference on MultimediaRecently, few-shot 3D point cloud semantic segmentation methods have been introduced to mitigate the limitations of existing fully supervised approaches, i.e., heavy dependence on labeled 3D data and poor capacity to generalize to new categories. ...
Cross-Domain Few-Shot Semantic Segmentation
Computer Vision – ECCV 2022AbstractFew-shot semantic segmentation aims at learning to segment a novel object class with only a few annotated examples. Most existing methods consider a setting where base classes are sampled from the same domain as the novel classes. However, in many ...
Comments
Information & Contributors
Information
Published In
March 2023
293 pages
ISBN:9781450398329
DOI:10.1145/3589883
Copyright © 2023 ACM.
Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than the author(s) must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected].
Publisher
Association for Computing Machinery
New York, NY, United States
Publication History
Published: 27 June 2023
Check for updates
Author Tags
Qualifiers
- Research-article
- Research
- Refereed limited
Conference
ICMLT 2023
ICMLT 2023: 2023 8th International Conference on Machine Learning Technologies
March 10 - 12, 2023
Stockholm, Sweden
Contributors
Other Metrics
Bibliometrics & Citations
Bibliometrics
Article Metrics
- 0Total Citations
- 73Total Downloads
- Downloads (Last 12 months)28
- Downloads (Last 6 weeks)2
Reflects downloads up to 13 Jan 2025
Other Metrics
Citations
View Options
Login options
Check if you have access through your login credentials or your institution to get full access on this article.
Sign inFull Access
View options
View or Download as a PDF file.
PDFeReader
View online with eReader.
eReaderHTML Format
View this article in HTML Format.
HTML Format