Abstract
The advent of low-cost LiDAR in recent years makes it feasible for LiDAR to be used in visual surveillance applications such as detection and tracking of players in a football game. However, the extreme sparsity of point cloud acquired by such LiDAR is a challenge for object detection and tracking in large-scale scenes. To alleviate this problem, we propose a method of multi-object detection and tracking from sparse point clouds comprising a short-term tracklet regression stage and a 3D D-IoU data association stage. In the former stage, temporal information is aggregated by the proposed temporal fusion module to predict short-term tracklets formed by three bounding boxes. In the latter stage, the Distance-IoU scores of current tracklets and historical trajectories are computed to associate the data using Hungarian matching algorithm. To reduce the cost of manual annotations, we build a simulated point cloud dataset using Google Research Football for training. A real test dataset of football game is acquired by Livox Mid-100 LiDAR. Our experimental results on both datasets show that fusing multi-frames conduces to improving detection and tracking performance from sparse point clouds. Our 3D D-IoU tracking method also gets a promising performance on the nuScenes autonomous driving dataset.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Similar content being viewed by others
Notes
- 1.
We plan to make both datasets publicly available.
References
Buric, M., Ivasic-Kos, M., Pobar, M.: Player tracking in sports videos. In: CloudCom (2019)
Caesar, H., Bankiti, V., Lang, A.H.: nuScenes: a multimodal dataset for autonomous driving. CoRR (2019)
Chiu, H., Prioletti, A., Li, J., Bohg, J.: Probabilistic 3D multi-object tracking for autonomous driving. CoRR (2020)
Geiger, A., Lenz, P., Urtasun, R.: Are we ready for autonomous driving? the KITTI vision benchmark suite. In: CVPR (2012)
Kurach, K., et al.: Google research football: a novel reinforcement learning environment. In: AAAI (2020)
Lang, A.H., Vora, S., Caesar, H., Zhou, L., Yang, J., Beijbom, O.: PointPillars: fast encoders for object detection from point clouds. In: CVPR (2019)
Li, B.: 3D fully convolutional network for vehicle detection in point cloud. In: IROS (2017)
Lin, T., Goyal, P., Girshick, R.B., He, K., Dollár, P.: Focal loss for dense object detection. IEEE Trans. Pattern Anal. Mach. Intell. 42, 318–327 (2020)
Liu, W., et al.: SSD: single shot multibox detector. In: Leibe, B., Matas, J., Sebe, N., Welling, M. (eds.) ECCV 2016. LNCS, vol. 9905, pp. 21–37. Springer, Cham (2016). https://doi.org/10.1007/978-3-319-46448-0_2
Luo, W., Yang, B., Urtasun, R.: Fast and furious: real time end-to-end 3D detection, tracking and motion forecasting with a single convolutional net. In: CVPR (2018)
Manafifard, M., Ebadi, H., Moghaddam, H.A.: A survey on player tracking in soccer videos. Comput. Vis. Image Underst. 159, 19–46 (2017)
Qi, C.R., Litany, O., He, K., Guibas, L.J.: Deep hough voting for 3D object detection in point clouds. In: ICCV (2019)
Qi, C.R., Su, H., Mo, K., Guibas, L.J.: PointNet: deep learning on point sets for 3D classification and segmentation. In: CVPR (2017)
Qi, H., Feng, C., Cao, Z., Zhao, F., Xiao, Y.: P2B: point-to-box network for 3D object tracking in point clouds. In: CVPR (2020)
Rahman, M.M., Tan, Y., Xue, J., Lu, K.: Recent advances in 3D object detection in the era of deep neural networks: a survey. IEEE Trans. Image Process. 29, 2947–2962 (2020)
Shi, S., Wang, X., Li, H.: Pointrcnn: 3D object proposal generation and detection from point cloud. In: CVPR (2019)
Su, H., Maji, S., Kalogerakis, E., Learned-Miller, E.G.: Multi-view convolutional neural networks for 3D shape recognition. In: ICCV. IEEE Computer Society (2015)
Sun, P., et al.: Scalability in perception for autonomous driving: waymo open dataset. CoRR (2019)
Wang, S., Sun, Y., Liu, C., Liu, M.: Pointtracknet: an end-to-end network for 3-D object detection and tracking from point clouds. IEEE Rob. Autom. Lett. 5, 3206–3212 (2020)
Weng, X., Wang, J., Held, D., Kitani, K.: 3D multi-object tracking: a baseline and new evaluation metrics. In: IROS (2020)
Weng, X., Wang, Y., Man, Y., Kitani, K.M.: GNN3DMOT: graph neural network for 3D multi-object tracking with 2D–3D multi-feature learning. In: CVPR (2020)
Yang, Y., Xu, M., Wu, W., Zhang, R., Peng, Y.: 3D multiview basketball players detection and localization based on probabilistic occupancy. In: DICTA (2018)
Yang, Z., Sun, Y., Liu, S., Jia, J.: 3DSSD: point-based 3D single stage object detector. In: CVPR (2020)
Yin, J., Shen, J., Guan, C., Zhou, D., Yang, R.: LiDAR-based online 3D video object detection with graph-based message passing and spatiotemporal transformer attention. In: CVPR (2020)
Zheng, Z., Wang, P., Liu, W., Li, J., Ye, R., Ren, D.: Distance-IoU loss: faster and better learning for bounding box regression. In: AAAI (2020)
Acknowledgments
The work was supported by the National Key Research and Development Program of China under Grant 2018AAA0102803.
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2021 Springer Nature Switzerland AG
About this paper
Cite this paper
Zhang, M., Pan, Z., Feng, J., Zhou, J. (2021). 3D Multi-object Detection and Tracking with Sparse Stationary LiDAR. In: Ma, H., et al. Pattern Recognition and Computer Vision. PRCV 2021. Lecture Notes in Computer Science(), vol 13019. Springer, Cham. https://doi.org/10.1007/978-3-030-88004-0_2
Download citation
DOI: https://doi.org/10.1007/978-3-030-88004-0_2
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-030-88003-3
Online ISBN: 978-3-030-88004-0
eBook Packages: Computer ScienceComputer Science (R0)