Location via proxy:   [ UP ]  
[Report a bug]   [Manage cookies]                
Skip to main content
Springer Nature Link
Log in

Real-Time And Robust 3D Object Detection with Roadside LiDARs

  • Conference paper
  • First Online:
Proceedings of the 12th International Scientific Conference on Mobility and Transport

Part of the book series: Lecture Notes in Mobility ((LNMOB))

  • 558 Accesses

Abstract

This work aims to address the challenges in autonomous driving by focusing on the 3D perception of the environment using roadside LiDARs. We design a 3D object detection model that can detect traffic participants in roadside LiDARs in real-time. Our model uses an existing 3D detector as a baseline and improves its accuracy. To prove the effectiveness of our proposed modules, we train and evaluate the model on three different vehicle and infrastructure datasets. To show the domain adaptation ability of our detector, we train it on an infrastructure dataset from China and perform transfer learning on a different dataset recorded in Germany. We do several sets of experiments and ablation studies for each module in the detector that show that our model outperforms the baseline by a significant margin, while the inference speed is at 45 Hz (22 ms). We make a significant contribution with our LiDAR-based 3D detector that can be used for smart city applications to provide connected and automated vehicles with a far-reaching view. Vehicles that are connected to the roadside sensors can get information about other vehicles around the corner to improve their path and maneuver planning and to increase road traffic safety.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution

Subscribe and save

Springer+ Basic
$34.99 /Month
  • Get 10 units per month
  • Download Article/Chapter or eBook
  • 1 Unit = 1 Article or 1 Chapter
  • Cancel anytime
Subscribe now

Buy Now

Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 189.00
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 249.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info
Hardcover Book
USD 249.99
Price excludes VAT (USA)
  • Durable hardcover edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

Similar content being viewed by others

References

  1. Caesar H, Bankiti V, Lang AH, Vora S, Liong VE, Xu Q, Krishnan A, Pan Y, Baldan G, Beijbom O (2020) Nuscenes: a multimodal dataset for autonomous driving. In: Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pp 11,621–11,631

    Google Scholar 

  2. Cress C, Zimmer W, Strand L, Fortkord M, Dai S, Lakshminarasimhan V, Knoll A (2022) A9-dataset: multi-sensor infrastructure-based dataset for mobility research. arXiv preprint

    Google Scholar 

  3. Croce N (2021) Openlabel version 1.0.0 standardization project by asam association for standardization of automation and measuring systems. https://www.asam.net/standards/detail/openlabel/. Accessed 12 Nov 2021

  4. Deng J, Zhou W, Zhang Y, Li H (2021) From multi-view to hollow-3d: Hallucinated hollow-3d r-cnn for 3d object detection. IEEE Trans Circuits Syst Video Technol

    Google Scholar 

  5. Dosovitskiy A, Ros G, Codevilla F, Lopez A, Koltun V (2017) Carla: an open urban driving simulator. In: Conference on robot learning. PMLR, pp 1–16

    Google Scholar 

  6. Eldar Y, Lindenbaum M, Porat M, Zeevi YY (1997) The farthest point strategy for progressive image sampling. IEEE Trans Image Process 6(9):1305–1315

    Article  Google Scholar 

  7. Fan L, Xiong X, Wang F, Wang N, Zhang Z (2021) Rangedet: in defense of range view for lidar-based 3d object detection. In: Proceedings of the IEEE/CVF international conference on computer vision, pp 2918–2927

    Google Scholar 

  8. Geiger A, Lenz P, Stiller C, Urtasun R (2013) Vision meets robotics: the kitti dataset. Int J Robot Res 32(11):1231–1237

    Article  Google Scholar 

  9. Geiger A, Lenz P, Urtasun R (2012) Are we ready for autonomous driving? the kitti vision benchmark suite. In: 2012 IEEE conference on computer vision and pattern recognition. IEEE, pp 3354–3361

    Google Scholar 

  10. Graham B, Engelcke M, Van Der Maaten L (2018) 3d semantic segmentation with submanifold sparse convolutional networks. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 9224–9232

    Google Scholar 

  11. He C, Zeng H, Huang J, Hua X.S, Zhang L (2020) Structure aware single-stage 3d object detection from point cloud. In: Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pp 11,873–11,882

    Google Scholar 

  12. He K, Zhang X, Ren S, Sun J (2016) Deep residual learning for image recognition. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 770–778

    Google Scholar 

  13. Hu J, Shen L, Sun G (2018) Squeeze-and-excitation networks. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 7132–7141

    Google Scholar 

  14. Ioffe S, Szegedy C (2015) Batch normalization: accelerating deep network training by reducing internal covariate shift. In: International conference on machine learning. PMLR, pp 448–456

    Google Scholar 

  15. Kloeker L, Kotulla C, Eckstein L (2020) Real-time point cloud fusion of multi-lidar infrastructure sensor setups with unknown spatial location and orientation. In: 2020 IEEE 23rd international conference on intelligent transportation systems (ITSC). IEEE, pp 1–8

    Google Scholar 

  16. Krämmer A, Schöller C, Gulati D, Lakshminarasimhan V, Kurz F, Rosenbaum D, Lenz C, Knoll A (2019) Providentia-a large-scale sensor system for the assistance of autonomous vehicles and its evaluation. arXiv preprint arXiv:1906.06789

  17. Lang AH, Vora S, Caesar H, Zhou L, Yang J, Beijbom O (2019) Pointpillars: fast encoders for object detection from point clouds. In: Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pp 12,697–12,705

    Google Scholar 

  18. Liang Z, Zhang M, Zhang Z, Zhao X, Pu S (2020) Rangercnn: towards fast and accurate 3d object detection with range image representation. arXiv preprint arXiv:2009.00206

  19. Lin TY, Goyal P, Girshick R, He K, Dollár P (2017) Focal loss for dense object detection. In: Proceedings of the IEEE international conference on computer vision, pp 2980–2988

    Google Scholar 

  20. Liu B, Wang M, Foroosh H, Tappen M, Pensky M (2015) Sparse convolutional neural networks. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 806–814

    Google Scholar 

  21. Liu Z, Zhao X, Huang T, Hu R, Zhou Y, Bai X (2020) Tanet: robust 3d object detection from point clouds with triple attention. Proceedings of the AAAI conference on artificial intelligence 34:11677–11684

    Article  Google Scholar 

  22. Nair V, Hinton GE (2010) Rectified linear units improve restricted boltzmann machines. In: Icml

    Google Scholar 

  23. Noh J, Lee S, Ham B (2021) Hvpr: hybrid voxel-point representation for single-stage 3d object detection. In: Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pp 14,605–14,614

    Google Scholar 

  24. Paigwar A, Sierra-Gonzalez D, Erkent Ö, Laugier C (2021) Frustum-pointpillars: a multi-stage approach for 3d object detection using rgb camera and lidar. In: Proceedings of the IEEE/CVF international conference on computer vision, pp 2926–2933

    Google Scholar 

  25. Qi CR, Su H, Mo K, Guibas LJ (2017) Pointnet: deep learning on point sets for 3d classification and segmentation. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 652–660

    Google Scholar 

  26. Qi CR, Yi L, Su H, Guibas LJ (2017) Pointnet++ deep hierarchical feature learning on point sets in a metric space. In: Proceedings of the 31st international conference on neural information processing systems, pp 5105–5114

    Google Scholar 

  27. Shi S, Guo C, Jiang L, Wang Z, Shi J, Wang X, Li H (2020) Pv-rcnn: point-voxel feature set abstraction for 3d object detection. In: Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pp 10,529–10,538

    Google Scholar 

  28. Shi S, Wang X, Li H (2019) Pointrcnn: 3d object proposal generation and detection from point cloud. In: Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pp 770–779

    Google Scholar 

  29. Triess LT, Dreissig M, Rist CB, Zöllner JM (2021) A survey on deep domain adaptation for lidar perception. In: 2021 IEEE intelligent vehicles symposium workshops (IV workshops), pp 350–357. IEEE

    Google Scholar 

  30. Wang H, Zhang X, Li J, Li Z, Yang L, Pan S, Deng Y (2021) Ips300+: a challenging multimodal dataset for intersection perception system. arXiv preprint arXiv:2106.02781

  31. Yan Y, Mao Y, Li B (2018) Second: sparsely embedded convolutional detection. Sensors 18(10):3337

    Article  Google Scholar 

  32. Yang Z, Sun Y, Liu S, Jia J (2020) 3dssd: point-based 3d single stage object detector. In: Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pp 11,040–11,048

    Google Scholar 

  33. Yang Z, Sun Y, Liu S, Shen X, Jia J (2019) Std: sparse-to-dense 3d object detector for point cloud. In: Proceedings of the IEEE/CVF international conference on computer vision, pp 1951–1960

    Google Scholar 

  34. Yu J, Jiang Y, Wang Z, Cao Z, Huang T (2016) Unitbox: an advanced object detection network. In: Proceedings of the 24th ACM international conference on multimedia, pp 516–520

    Google Scholar 

  35. Zhang W, Li W, Xu D (2021) Srdan: scale-aware and range-aware domain adaptation network for cross-dataset 3d object detection. In: Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pp 6769–6779

    Google Scholar 

  36. Zheng W, Tang W, Chen S, Jiang L, Fu CW (2020) Cia-ssd: confident iou-aware single-stage object detector from point cloud. arXiv preprint arXiv:2012.03015

  37. Zheng W, Tang W, Jiang L, Fu C.W (2021) Se-ssd: self-ensembling single-stage object detector from point cloud. In: Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pp 14,494–14,503

    Google Scholar 

  38. Zhou X, Zimmer W, Erçelik E, Knoll A (2021) Real-time lidar-based 3d object detection on the highway. Master’s thesis, Technische Universität München. Unpublished thesis

    Google Scholar 

  39. Zhou Y, Tuzel O (2018) Voxelnet: end-to-end learning for point cloud based 3d object detection. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 4490–4499

    Google Scholar 

  40. Zimmer W, Rangesh A, Trivedi M (2019) 3d bat: a semi-automatic, web-based 3d annotation toolbox for full-surround, multi-modal data streams. In: 2019 IEEE intelligent vehicles symposium (IV). IEEE, pp 1816–1821

    Google Scholar 

Download references

Acknowledgements

This work was funded by the Federal Ministry of Transport and Digital Infrastructure, Germany as part of the research project Providentia++ (Grant Number: 01MM19008A). The authors would like to express their gratitude to the funding agency and to the numerous students at TUM who have contributed to the creation of the first batch of the A9-Dataset.

Author information

Authors and Affiliations

  1. Department of Computer Engineering, School of Computation, Information and Technology, Technical University of Munich (TUM), 85748, Garching, Germany

    Walter Zimmer, Jialong Wu, Xingcheng Zhou & Alois C. Knoll

Authors
  1. Walter Zimmer
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Jialong Wu
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Xingcheng Zhou
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Alois C. Knoll
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Walter Zimmer .

Editor information

Editors and Affiliations

  1. TUM School of Engineering and Design, Technical University of Munich, Munich, Germany

    Constantinos Antoniou

  2. TUM School of Engineering and Design, Technical University of Munich, Munich, Germany

    Fritz Busch

  3. TUM Asia Pte Ltd., Singapore, Singapore

    Andreas Rau

  4. Railway Engineering, TUM Asia Pte Ltd., Singapore, Singapore

    Mahesh Hariharan

Rights and permissions

Reprints and permissions

Copyright information

© 2023 The Author(s), under exclusive license to Springer Nature Singapore Pte Ltd.

About this paper

Check for updates. Verify currency and authenticity via CrossMark

Cite this paper

Zimmer, W., Wu, J., Zhou, X., Knoll, A.C. (2023). Real-Time And Robust 3D Object Detection with Roadside LiDARs. In: Antoniou, C., Busch, F., Rau, A., Hariharan, M. (eds) Proceedings of the 12th International Scientific Conference on Mobility and Transport. Lecture Notes in Mobility. Springer, Singapore. https://doi.org/10.1007/978-981-19-8361-0_13

Download citation

  • DOI: https://doi.org/10.1007/978-981-19-8361-0_13

  • Published:

  • Publisher Name: Springer, Singapore

  • Print ISBN: 978-981-19-8360-3

  • Online ISBN: 978-981-19-8361-0

  • eBook Packages: EngineeringEngineering (R0)

Publish with us

Policies and ethics

Search

Navigation