Cited By
View all- Gan JZhang GKou KXiong Y(2024)Flexible asymmetric convolutional attention network for LiDAR semanticApplied Intelligence10.1007/s10489-024-05525-854:8(6718-6737)Online publication date: 1-Apr-2024
PointFusionNet: Point feature fusion network for 3D point clouds analysis
Authors: 
Pan Liang, Zhijun Fang, Bo Huang, Heng Zhou, Xianhua Tang, Cengsi ZhongAuthors Info & Claims
Pan Liang, Zhijun Fang, Bo Huang, Heng Zhou, Xianhua Tang, Cengsi ZhongAuthors Info & ClaimsAbstract
The 3D point clouds is an important type of geometric data structure, and the analysis of 3D point clouds based on deep learning is a very challenging task due to the disorder and irregularity. In existing research, RS-CNN provides an effective and promising method to obtain shape features on disordered point clouds directly, which encodes local features effectively. However, RS-CNN fails to consider point-wise features and global features, which are conducive to point clouds better. In this paper, we proposed PointFusionNet, which solves these problems effectively by fusing point-wise features, local features, and global features. We have designed Feature Fusion Convolution (FF-Conv) and Global Relationship Reasoning Module (GRRM) to build PointFusionNet. The point-wise features were fused with their corresponding local features in the FF-Conv and then mapped into a high-dimensional space to extract richer local features. The GRRM inferred the relationship between various parts, in order to capture global features for enriching the content of the feature descriptor. Therefore the PointFusionNet is suitable for point clouds classification and semantic segmentation by using the two distinctive modules. The PointFusionNet has been tested on ModelNet40 and ShapeNet part datasets, and the experiments show that PointFusionNet has a competitive advantage in shape classification and part segmentation tasks.
References
[1]
Feng M, Wang Y, Liu J, Zhang L, Zaki HF, and Mian A Benchmark data set and method for depth estimation from light field images IEEE Trans Image Process 2018 27 7 3586-3598
[2]
Zhou Y, Tuzel O (2018) Voxelnet: End-to-end learning for point cloud based 3d object detection. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp 4490–4499
[3]
Qi CR, Liu W, Wu C, Su H, Guibas L (2018) Frustum pointnets for 3d object detection from rgb-d data. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp 918–927
[4]
Ku J, Mozifian M, Lee J, Harakeh A, Waslander SL (2018) Joint 3d proposal generation and object detection from view aggregation. In: 2018 IEEE/RSJ International conference on intelligent robots and systems (IROS). IEEE, pp 1–8
[5]
Liu Z, Chen H, Di H, Tao Y, Gong J, Xiong G, Qi J (2018) Real-time 6d lidar slam in large scale natural terrains for ugv. In: 2018 IEEE Intelligent vehicles symposium (IV). IEEE, pp 662–667
[6]
Zhu Y, Mottaghi R, Kolve E, Lim JJ, Gupta A, Fei-Fei L, Farhadi A (2017) Target-driven visual navigation in indoor scenes using deep reinforcement learning. In: 2017 IEEE International conference on robotics and automation (ICRA). IEEE, pp 3357–3364
[7]
Biswas J, Veloso M (2012) Depth camera based indoor mobile robot localization and navigation. In: 2012 IEEE International conference on robotics and automation. IEEE, pp 1697–1702
[8]
Golovinskiy A, Kim VG, Funkhouser T (2009) Shape-based recognition of 3d point clouds in urban environments. In: 2009 IEEE 12Th international conference on computer vision. IEEE, pp 2154–2161
[9]
Wang Z, Lu F (2019) Voxsegnet: Volumetric cnns for semantic part segmentation of 3d shapes. IEEE transactions on visualization and computer graphics
[10]
Wang DZ, Posner I (2015) Voting for voting in online point cloud object detection. In: Robotics: Science and systems, vol 1, pp 10–15607
[11]
Riegler G, Osman Ulusoy A, Geiger A (2017) Octnet: Learning deep 3d representations at high resolutions. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp 3577–3586
[12]
Maturana D, Scherer S (2015) Voxnet: a 3d convolutional neural network for real-time object recognition. In: 2015 IEEE/RSJ International conference on intelligent robots and systems (IROS). IEEE, pp 922–928
[13]
Qi CR, Su H, Mo K, Guibas L (2017) Pointnet: Deep learning on point sets for 3d classification and segmentation. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 652–660
[14]
Qi CR, Yi L, Su H, Guibas L (2017) Pointnet++: Deep hierarchical feature learning on point sets in a metric space. In: Advances in neural information processing systems, pp 5099–5108
[15]
Wang Y, Sun Y, Liu Z, Sarma SE, Bronstein MM, and Solomon JM Dynamic graph cnn for learning on point clouds ACM Trans Graph (TOG) 2019 38 5 1-12
[16]
Liu Y, Fan B, Xiang S, Pan C (2019) Relation-shape convolutional neural network for point cloud analysis. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp 8895–8904
[17]
Zhou H, Fang Z, Gao Y, Huang B, Zhong C, Shang R (2020) Feature fusion network based on attention mechanism for 3d semantic segmentation of point clouds. Pattern Recognition Letters
[18]
Li X, Wang W, Hu X, Yang J (2019) Selective kernel networks. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 510–519
[19]
Cao Y, Xu J, Lin S, Wei F, Hu H (2019) Gcnet: Non-local networks meet squeeze-excitation networks and beyond. In: Proceedings of the IEEE International Conference on Computer Vision Workshops, pp 0–0
[20]
Chen Y, Rohrbach M, Yan Z, Shuicheng Y, Feng J, Kalantidis Y (2019) Graph-based global reasoning networks. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp 433–442
[21]
Qi CR, Su H (2016) Nießner, M., Dai, A., Yan, M., Guibas, L.J.: Volumetric and multi-view cnns for object classification on 3d data. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 5648–5656
[22]
Gadelha M, Wang R, Maji S (2018) Multiresolution tree networks for 3d point cloud processing. In: Proceedings of the European Conference on Computer Vision (ECCV), pp 103–118
[23]
Wu Z, Song S, Khosla A, Yu F, Zhang L, Tang X, Xiao J (2015) 3d shapenets: a deep representation for volumetric shapes. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 1912–1920
[24]
Klokov R, Lempitsky V (2017) Escape from cells: Deep kd-networks for the recognition of 3d point cloud models. In: Proceedings of the IEEE International Conference on Computer Vision, pp 863–872
[25]
Wang PS, Liu Y, Guo YX, Sun CY, and Tong X O-cnn: Octree-based convolutional neural networks for 3d shape analysis ACM Trans Graph (TOG) 2017 36 4 1-11
[26]
Tatarchenko M, Dosovitskiy A, Brox T (2017) Octree generating networks: Efficient convolutional architectures for high-resolution 3d outputs. In: Proceedings of the IEEE International Conference on Computer Vision, pp 2088–2096
[27]
Wang PS, Sun CY, Liu Y, and Tong X Adaptive o-cnn: a patch-based deep representation of 3d shapes ACM Trans Graph (TOG) 2018 37 6 1-11
[28]
Le T, Duan Y (2018) Pointgrid: a deep network for 3d shape understanding. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 9204–9214
[29]
Feng Y, Zhang Z, Zhao X, Ji R, Gao Y (2018) Gvcnn: Group-view convolutional neural networks for 3d shape recognition. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp 264–272
[30]
Guo H, Wang J, Gao Y, Li J, and Lu H Multi-view 3d object retrieval with deep embedding network IEEE Trans Image Process 2016 25 12 5526-5537
[31]
Han Z, Shang M, Liu Z, Vong CM, Liu YS, Zwicker M, Han J, and Chen CP Seqviews2seqlabels: Learning 3d global features via aggregating sequential views by rnn with attention IEEE Trans Image Process 2018 28 2 658-672
[32]
Su H, Maji S, Kalogerakis E, Learned-Miller E (2015) Multi-view convolutional neural networks for 3d shape recognition. In: Proceedings of the IEEE international conference on computer vision, pp 945–953
[33]
Xie J, Dai G, Zhu F, Wong EK, and Fang Y Deepshape: Deep-learned shape descriptor for 3d shape retrieval IEEE Trans Pattern Anal Mach Intell 2016 39 7 1335-1345
[34]
Shen Y, Feng C, Yang Y, Tian D (2018) Mining point cloud local structures by kernel correlation and graph pooling. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 4548–4557
[35]
Li J, Chen B, Hee Lee G (2018) So-net: Self-organizing network for point cloud analysis. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 9397–9406
[36]
Komarichev A, Zhong Z, Hua J (2019) A-cnn: Annularly convolutional neural networks on point clouds. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp 7421–7430
[37]
Wang L, Huang Y, Hou Y, Zhang S, Shan J (2019) Graph attention convolution for point cloud semantic segmentation. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp 10296–10305
[38]
Zhang Y, Rabbat M (2018) A graph-cnn for 3d point cloud classification. In: 2018 IEEE International conference on acoustics, speech and signal processing (ICASSP). IEEE, pp 6279–6283
[39]
Simonovsky M, Komodakis N (2017) Dynamic edge-conditioned filters in convolutional neural networks on graphs. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 3693–3702
[40]
Song L, Li Y, Li Z, Yu G, Sun H, Sun J, Zheng N (2019) Learnable tree filter for structure-preserving feature transform. In: Advances in neural information processing systems, pp 1709–1719
[41]
Yi L, Kim VG, Ceylan D, Shen IC, Yan M, Su H, Lu C, Huang Q, Sheffer A, and Guibas L A scalable active framework for region annotation in 3d shape collections ACM Trans Graph (TOG) 2016 35 6 1-12
[42]
Kingma DP, Ba J (2014) Adam: A method for stochastic optimization. arXiv:1412.6980
[43]
He K, Zhang X, Ren S, Sun J (2015) Delving deep into rectifiers: Surpassing human-level performance on imagenet classification. In: Proceedings of the IEEE international conference on computer vision, pp 1026–1034
[44]
Hua BS, Tran MK, Yeung SK (2018) Pointwise convolutional neural networks. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp 984–993
[45]
Xie S, Liu S, Chen Z, Tu Z (2018) Attentional shapecontextnet for point cloud recognition. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp 4606–4615
[46]
Wang C, Samari B, Siddiqi K (2018) Local spectral graph convolution for point set feature learning. In: Proceedings of the European conference on computer vision (ECCV), pp 52–66
[47]
Li Y, Bu R, Sun M, Wu W, Di X, Chen B (2018) Pointcnn: Convolution on x-transformed points. In: Advances in neural information processing systems, pp 820–830
[48]
Atzmon M, Maron H, Lipman Y (2018) Point convolutional neural networks by extension operators. arXiv:1803.10091
[49]
Xu Y, Fan T, Xu M, Zeng L, Qiao Y (2018) Spidercnn: Deep learning on point sets with parameterized convolutional filters. In: Proceedings of the European Conference on Computer Vision (ECCV), pp 87–102
[50]
Chang AX, Funkhouser T, Guibas L, Hanrahan P, Huang Q, Li Z, Savarese S, Savva M, Song S, Su H et al (2015) Shapenet: An information-rich 3d model repository. arXiv:1512.03012
[51]
Huang Q, Wang W, Neumann U (2018) Recurrent slice networks for 3d segmentation of point clouds. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp 2626–2635
[52]
Su H, Jampani V, Sun D, Maji S, Kalogerakis E, Yang MH, Kautz J (2018) Splatnet: Sparse lattice networks for point cloud processing. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp 2530–2539
Index Terms
- PointFusionNet: Point feature fusion network for 3D point clouds analysis
Index terms have been assigned to the content through auto-classification.
Recommendations
Feature line extraction from unorganized noisy point clouds using truncated Fourier series
The detection of feature lines is important for representing and understanding geometric features of 3D models. In this paper, we introduce a new and robust method for extracting feature lines from unorganized point clouds. We use a one-dimensional ...
A statistical approach for extraction of feature lines from point clouds
This paper firstly introduces the use of a statistical model based on the Poisson distribution as a tool for extracting feature points from point clouds. When the features on a model are non-uniform or the surfaces are not completely smooth, the ...
Robust and Accurate Feature Detection on Point Clouds
AbstractGeometric feature detection on surfaces is a crucial task for the characterization and understanding of geometry shapes. In this paper, we present a robust and reliable approach for accurately capturing local surface variations at ...
Comments
Information & Contributors
Information
Published In
© Springer Science+Business Media, LLC, part of Springer Nature 2020.
Publisher
Kluwer Academic Publishers
United States
Publication History
Published: 01 April 2021
Accepted: 05 October 2020
Author Tags
Qualifiers
- Research-article
Funding Sources
- the National Natural Science Foundation of China
Contributors
Other Metrics
Bibliometrics & Citations
Bibliometrics
Article Metrics
- View Citations1Total Citations
- 0Total Downloads
- Downloads (Last 12 months)0
- Downloads (Last 6 weeks)0
Reflects downloads up to 01 Sep 2024
Other Metrics
Citations
Cited By
View all- Gan JZhang GKou KXiong Y(2024)Flexible asymmetric convolutional attention network for LiDAR semanticApplied Intelligence10.1007/s10489-024-05525-854:8(6718-6737)Online publication date: 1-Apr-2024
View Options
View options
Get Access
Login options
Check if you have access through your login credentials or your institution to get full access on this article.
Sign in