Feature Fusion based Re-voting for 3D Object Detection
Authors: 
Hang Yu, Jun Wei, Jinhe Su, and Niansheng LiuAuthors Info & Claims
Hang Yu, Jun Wei, Jinhe Su, and Niansheng LiuAuthors Info & ClaimsOctober 2021
Pages 652 - 658
Abstract
3D object detection based on point cloud is a challenging visual task, which is helpful to the realization of various 3D visual applications. A few recent works based votenet recognize objects by using hough voting. However, the voting strategy in votenet can only obtain some sampling points from incomplete surfaces and chaotic backgrounds, without considering the features and position relation of the original cloud points. In this work, we introduce a new 3D object detection method called feature fusion based revoting network (FFRNet). Our method adds a supervision mechanism to the simple voting mechanism and fuses the feature of seed points and voting points to increase the richness of information in the re-voting module. The feature fusion operation enhances the acquisition of effective information of the original surface points, So as to achieve more reliable and flexible object positioning and category prediction results. We validate our model on the challenging ScanNet V2 dataset, advancing votenet results by 3.6 [email protected].
References
[1]
John McCormac, Ronald Clark, Michael Bloesch, Andrew Davison, and Stefan Leutenegger. Fusion++: Volumetric object-level SLAM. In 2018 International Conference on 3D Vision (3DV), pages 32--41. IEEE, 2018. 1.
[2]
Shuran Song and Jianxiong Xiao. Deep sliding shapes for amodal 3D object detection in RGB-D images. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pages 808--816, 2016. 1, 6.
[3]
Charles R Qi, Hao Su, Kaichun Mo, and Leonidas J Guibas. PointNet: Deep learning on point sets for 3D classification and segmentation. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pages 652--660, 2017. 1, 3.
[4]
Charles Ruizhongtai Qi, Li Yi, Hao Su, and Leonidas J Guibas. PointNet++: Deep hierarchical feature learning on point sets in a metric space. In Advances in neural information processing systems, pages 5099--5108, 2017. 1.
[5]
Angela Dai, Angel X Chang, Manolis Savva, Maciej Halber, Thomas Funkhouser, and Matthias Nießner. Scannet: Richly-annotated 3d reconstructions of indoor scenes. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pages 5828--5839, 2017. 2, 5.
[6]
Ji Hou, Angela Dai, and Matthias Nießner. 3d-sis: 3D semantic instance segmentation of RGB-D scans. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pages 4421--4430, 2019. 1, 3, 6, 7.
[7]
Charles R Qi, Or Litany, Kaiming He, and Leonidas J Guibas. Deep Hough voting for 3D object detection in point clouds. arXiv preprint arXiv:1904.09664, 2019. 1, 2, 3, 4, 6.
[8]
Bo Yang, Jianan Wang, Ronald Clark, Qingyong Hu, Sen Wang, Andrew Markham, and Niki Trigoni. Learning object bounding boxes for 3D instance segmentation on point clouds. arXiv preprint arXiv:1906.01140, 2019. 3
[9]
Bastian Leibe, Ales Leonardis, and Bernt Schiele. Combined object categorization and segmentation with an implicit shape model. In Workshop on statistical learning in computer vision, ECCV, volume 2, page 7, 2004
[10]
Yangyan Li, Rui Bu, Mingchao Sun, Wei Wu, Xinhan Di, and Baoquan Chen. PointCNN: Convolution on xtransformed points. In Advances in Neural Information Processing Systems, pages 820--830, 2018. 3.
[11]
Matan Atzmon, Haggai Maron, and Yaron Lipman. Point convolutional neural networks by extension operators. arXiv preprint arXiv:1803.10091, 2018. 3.
[12]
Dahua Lin, Sanja Fidler, and Raquel Urtasun. Holistic scene understanding for 3D object detection with RGBD cameras. In ICCV, pages 1417--1424, 2013. 2.
[13]
Shuran Song and Jianxiong Xiao. Sliding shapes for 3D object detection in depth images. In ECCV, pages 634--651. Springer, 2014. 1, 2.
[14]
Joseph Redmon, Santosh Divvala, Ross Girshick, and Ali Farhadi. You only look once: Unified, real-time object detection. In CVPR, pages 779--788, 2016. 3
[15]
Zhile Ren and Erik B Sudderth. Three-dimensional object detection and layout prediction using clouds of oriented gradients. In CVPR, pages 1525--1533, 2016. 2, 6, 12.
[16]
Shaoshuai Shi, Chaoxu Guo, Li Jiang, Zhe Wang, Jianping Shi, Xiaogang Wang, and Hongsheng Li. PV-RCNN: Pointvoxel feature set abstraction for 3D object detection. In CVPR, pages 10529--10538, 2020. 2, 5.
[17]
Charles R Qi, Li Yi, Hao Su, and Leonidas J Guibas. Pointnet++: Deep hierarchical feature learning on point sets in a metric space. arXiv preprint arXiv:1706.02413, 2017. 1, 2, 3, 4, 5, 11.
[18]
Charles R Qi, Hao Su, Kaichun Mo, and Leonidas J Guibas. PointNet: Deep learning on point sets for 3D classification and segmentation. In CVPR, pages 652--660, 2017. 2, 5.
[19]
Subhransu Maji and Jitendra Malik. Object detection using a max-margin hough transform. 2009. 2.
[20]
Jean Lahoud and Bernard Ghanem. 2D-driven 3D object detection in RGB-D images. In Proceedings of the IEEE International Conference on Computer Vision, pages 4622--4630, 2017. 3, 6Geometric Design, 43:82--94, 2016. 3
[21]
Qian Xie, Yu-Kun Lai, Jing Wu, Zhoutao Wang, Yiming Zhang, Kai Xu, and Jun Wang. MLCVNet: Multi-level context votenet for 3D object detection. In CVPR, pages 10447--10456, 2020. 1, 3, 6, 7, 8, 11, 12, 13.
[22]
Ross Girshick. Fast R-CNN. In Proceedings of the IEEE international conference on computer vision, pages 1440--1448, 2015. 2.
[23]
Shaoqing Ren, Kaiming He, Ross Girshick, and Jian Sun. Faster R-CNN: Towards real-time object detection with region proposal networks. In Advances in neural information processing systems, pages 91--99, 2015. 2.
[24]
Zaiwei Zhang, Bo Sun, Haitao Yang, and Qixing Huang. H3DNet: 3D object detection using hybrid geometric primitives. In ECCV, 2020. 1, 3, 6, 7, 8, 11, 12, 1.
[25]
Alex Krizhevsky, Ilya Sutskever, and Geoffrey E Hinton. Imagenet classification with deep convolutional neural networks. In Advances in neural information processing systems, pages 1097--1105, 2012. 2.
[26]
Yifei Shi, Angel X Chang, Zhelun Wu, Manolis Savva, and Kai Xu. Hierarchy denoising recursive autoencoders for 3D scene layout prediction. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pages 1771--1780, 2019. 3.
[27]
Shuran Song and Jianxiong Xiao. Deep sliding shapes for amodal 3D object detection in RGB-D images. In CVPR, pages 808--816, 2016. 1, 2, 5, 6, 12.
[28]
Charles R Qi, Wei Liu, Chenxia Wu, Hao Su, and Leonidas J Guibas. Frustum pointnets for 3d object detection from RGB-D data. In CVPR, pages 918--927, 2018. 2, 4, 6, 12.
[29]
Li Yi, Wang Zhao, He Wang, Minhyuk Sung, and Leonidas J Guibas. GSPN: Generative shape proposal network for 3d instance segmentation in point cloud. In CVPR, pages 3947- 3956, 2019. 1, 2, 6.
[30]
Ji Hou, Angela Dai, and Matthias Nießner. 3D-SIS: 3D semantic instance segmentation of RGB-D scans. In CVPR, pages 4421--4430, 2019. 1, 2, 6.
[31]
Liangliang Nan, Ke Xie, and Andrei Sharf. A search-classify approach for cluttered indoor scene understanding. ACM Transactions on Graphics (TOG), 31(6):137, 2012. 3.
[32]
Yangyan Li, Angela Dai, Leonidas Guibas, and Matthias Nießner. Database-assisted object retrieval for real-time 3D reconstruction. In Computer Graphics Forum, volume 34, pages 435--446. Wiley Online Library, 2015. 3.
Index Terms
- Feature Fusion based Re-voting for 3D Object Detection
Recommendations
Multi-feature Fusion VoteNet for 3D Object Detection
In this article, we propose a Multi-feature Fusion VoteNet (MFFVoteNet) framework for improving the 3D object detection performance in cluttered and heavily occluded scenes. Our method takes the point cloud and the synchronized RGB image as inputs to ...
3D object detection based on the fusion of projected point cloud and image features
EITCE '22: Proceedings of the 2022 6th International Conference on Electronic Information Technology and Computer EngineeringThe complementary advantages of point cloud and image can provide more accurate 3D and semantic information to the model. Aiming at the problems that most existing methods adopt a single fusion strategy and thus fail to achieve deep fusion of image and ...
Point cloud 3D object detection method based on density information-local feature fusion
AbstractNowadays, three-dimensional (3D) point cloud is widely used in unmanned driving, high-precision mapping, robot grasping, mapping and virtual reality (VR) / augmented reality (AR), etc. Especially, many studies have focused on object detection ...
Comments
Information & Contributors
Information
Published In
October 2021
1723 pages
ISBN:9781450384322
DOI:10.1145/3501409
Copyright © 2021 ACM.
Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]
Publisher
Association for Computing Machinery
New York, NY, United States
Publication History
Published: 31 December 2021
Check for updates
Author Tags
Qualifiers
- Research-article
- Research
- Refereed limited
Conference
EITCE 2021
EITCE 2021: 2021 5th International Conference on Electronic Information Technology and Computer Engineering
October 22 - 24, 2021
Xiamen, China
Acceptance Rates
EITCE '21 Paper Acceptance Rate 294 of 531 submissions, 55%;
Overall Acceptance Rate 508 of 972 submissions, 52%
Contributors
Other Metrics
Bibliometrics & Citations
Bibliometrics
Article Metrics
- 0Total Citations
- 35Total Downloads
- Downloads (Last 12 months)11
- Downloads (Last 6 weeks)1
Other Metrics
Citations
View Options
Get Access
Login options
Check if you have access through your login credentials or your institution to get full access on this article.
Sign in