research-article

Design Guidelines on Deep Learning–based Pedestrian Detection Methods for Supporting Autonomous Vehicles

Authors:

Azzedine Boukerche,

Mingzhi ShaAuthors Info & Claims

ACM Computing Surveys (CSUR), Volume 54, Issue 6

Article No.: 133, Pages 1 - 36

https://doi.org/10.1145/3460770

Published: 03 August 2021 Publication History

Abstract

Intelligent transportation systems (ITS) enable transportation participants to communicate with each other by sending and receiving messages, so that they can be aware of their surroundings and facilitate efficient transportation through better decision making. As an important part of ITS, autonomous vehicles can bring massive benefits by reducing traffic accidents. Correspondingly, much effort has been paid to the task of pedestrian detection, which is a fundamental task for supporting autonomous vehicles. With the progress of computational power in recent years, adopting deep learning–based methods has become a trend for improving the performance of pedestrian detection. In this article, we present design guidelines on deep learning–based pedestrian detection methods for supporting autonomous vehicles. First, we will introduce classic backbone models and frameworks, and we will analyze the inherent attributes of pedestrian detection. Then, we will illustrate and analyze representative pedestrian detectors from occlusion handling, multi-scale feature extraction, multi-perspective data utilization, and hard negatives handling these four aspects. Last, we will discuss the developments and trends in this area, followed by some open challenges.

References

[1]

2018. Taxonomy and Definitions for Terms Related to Driving Automation Systems for On-Road Motor Vehicles. In SAE MOBILUS. Retrieved from https://www.sae.org/standards/content/j3016_201806/.

[2]

2020. 6 Key Connectivity Requirements of Autonomous Driving. In IEEE Spectrum. Retrieved from https://spectrum.ieee.org/transportation/advanced-cars/6-key-connectivity-requirements-of-autonomous-driving.

[3]

2020. New Level 3 Autonomous Vehicles Hitting the Road in 2020. In IEEE Innovation at Work. Retrieved from https://innovationatwork.ieee.org/new-level-3-autonomous-vehicles-hitting-the-road-in-2020/.

[4]

T. Ahonen, A. Hadid, and M. Pietikainen. 2006. Face description with local binary patterns: Application to face recognition. IEEE Trans. Pattern Anal. Mach. Intell. 28, 12 (2006), 2037–2041.

Digital Library

[5]

E. Alemneh, S. Senouci, and P. Brunet. 2017. PV-Alert: A fog-based architecture for safeguarding vulnerable road users. In Proc. IEEE GIIS. 9–15.

[6]

Anelia Angelova, Alex Krizhevsky, Vincent Vanhoucke, Abhijit Ogale, and Dave Ferguson. 2015. Real-time pedestrian detection with deep network cascades. In Proc. BMVC.

[7]

Felix Assion, Peter Schlicht, Florens Greßner, Wiebke Gunther, Fabian Huger, Nico Schmidt, and Umair Rasheed. 2019. The attack generator: A systematic approach towards constructing adversarial attacks. In Proc. IEEE CVPR Workshops. 1370–1379.

[8]

S. Baidya, Y. J. Ku, H. Zhao, J. Zhao, and S. Dey. 2020. Vehicular and edge computing for emerging connected and autonomous vehicle applications. In Proc. IEEE DAC. 1–6.

[9]

Serge Belongie, Jitendra Malik, and Jan Puzicha. 2002. Shape matching and object recognition using shape contexts. IEEE Trans. Pattern Anal. Mach. Intell. 24, 4 (2002), 509–522.

Digital Library

[10]

Rodrigo Benenson, Markus Mathias, Radu Timofte, and Luc Van Gool. 2012. Pedestrian detection at 100 frames per second. In Proc. IEEE CVPR. 2903–2910.

[11]

Azzedine Boukerche, Burak Kantarci, and Cem Kaptan. 2020. Towards ensuring the reliability and dependability of vehicular crowd-sensing data in GPS-less location tracking. Pervas. Mob. Comput. 68 (2020), 101248.

Digital Library

[12]

M. Braun, S. Krebs, F. Flohr, and D. M. Gavrila. 2019. EuroCity persons: A novel benchmark for person detection in traffic scenes. IEEE Trans. Pattern Anal. Mach. Intell. 41, 8 (2019), 1844–1861.

[13]

Garrick Brazil, Xi Yin, and Xiaoming Liu. 2017. Illuminating pedestrians via simultaneous detection & segmentation. In Proc. IEEE ICCV. 4950–4959.

[14]

Zhaowei Cai, Quanfu Fan, Rogerio S. Feris, and Nuno Vasconcelos. 2016. A unified multi-scale deep convolutional neural network for fast object detection. In Proc. ECCV. 354–370.

[15]

Zhaowei Cai, Mohammad Saberian, and Nuno Vasconcelos. 2015. Learning complexity-aware cascades for deep pedestrian detection. In Proc. IEEE ICCV. 3361–3369.

Digital Library

[16]

Mingju Chen, Xiaofeng Han, Hua Zhang, Guojun Lin, and M. M. Kamruzzaman. 2019. Quality-guided key frames selection from video stream based on object detection. J. Vis. Commun. Image R. 65 (2019), 102678.

[17]

Shang-Tse Chen, Cory Cornelius, Jason Martin, and Duen Horng Polo Chau. 2018. Shapeshifter: Robust physical adversarial attack on faster r-cnn object detector. In Proc. ECML PKDD. 52–68.

[18]

Yu Cheng, Duo Wang, Pan Zhou, and Tao Zhang. 2018. Model compression and acceleration for deep neural networks: The principles, progress, and challenges. IEEE Signal Process. Mag. 35, 1 (2018), 126–136.

[19]

Siew-Kei Lam, Chengju Zhou, and Meiqing Wu. 2019. SSA-CNN: Semantic Self-Attention CNN for Pedestrian Detection. [Online]. Retrieved from http://arxiv.org/abs/1902.09080.

[20]

Cheng Chi, Shifeng Zhang, Junliang Xing, Zhen Lei, Stan Z. Li, and Xudong Zou. 2020. PedHunter: Occlusion robust pedestrian detector in crowded scenes. In Proc. AAAI. 10639–10646.

[21]

Corinna Cortes and Vladimir Vapnik. 1995. Support-vector networks. Mach. Learn. 20, 3 (1995), 273–297.

[22]

N. Dalal and B. Triggs. 2005. Histograms of oriented gradients for human detection. In Proc. IEEE CVPR. 886–893.

[23]

Arthur Daniel Costea and Sergiu Nedevschi. 2016. Semantic channels for fast pedestrian detection. In Proc. IEEE CVPR. 2360–2368.

[24]

J. Deng, W. Dong, R. Socher, L. Li, Kai Li, and Li Fei-Fei. 2009. ImageNet: A large-scale hierarchical image database. In Proc. IEEE CVPR. 248–255.

[25]

Piotr Dollár, Ron Appel, Serge Belongie, and Pietro Perona. 2014. Fast feature pyramids for object detection. IEEE Trans. Pattern Anal. Mach. Intell. 36, 8 (2014), 1532–1545.

Digital Library

[26]

Piotr Dollár, Ron Appel, and Wolf Kienzle. 2012. Crosstalk cascades for frame-rate pedestrian detection. In Proc. ECCV. 645–659.

[27]

Piotr Dollar, Serge Belongie, and Pietro Perona. 2010. The fastest pedestrian detector in the west. In Proc. BMVC. 68.1–68.11.

[28]

Piotr Dollár, Zhuowen Tu, Pietro Perona, and Serge Belongie. 2009. Integral channel features. In Proc. BMVC. 244.1–244.11.

[29]

P. Dollar, C. Wojek, B. Schiele, and P. Perona. 2009. Pedestrian detection: A benchmark. In Proc. IEEE CVPR. 304–311.

[30]

P. Dollar, C. Wojek, B. Schiele, and P. Perona. 2012. Pedestrian Detection: An evaluation of the state of the art. IEEE Trans. Pattern Anal. Mach. Intell. 34, 4 (2012), 743–761.

Digital Library

[31]

Fabio Henrique Kiyoiti dos Santos Tanaka and Claus Aranha. 2019. Data Augmentation Using GANs. [Online]. Retrieved from http://arxiv.org/abs/1904.09135.

[32]

Kaiwen Duan, Song Bai, Lingxi Xie, Honggang Qi, Qingming Huang, and Qi Tian. 2019. Centernet: Keypoint triplets for object detection. In Proc. IEEE ICCV. 6569–6578.

[33]

A. Ess, B. Leibe, and L. Van Gool. 2007. Depth and appearance for mobile scene analysis. In Proc. IEEE ICCV. 1–8.

[34]

Pedro Felzenszwalb, David McAllester, and Deva Ramanan. 2008. A discriminatively trained, multiscale, deformable part model. In Proc. IEEE CVPR. 1–8.

[35]

Pedro F. Felzenszwalb, Ross B. Girshick, David McAllester, and Deva Ramanan. 2009. Object detection with discriminatively trained part-based models. IEEE Trans. Pattern Anal. Mach. Intell. 32, 9 (2009), 1627–1645.

Digital Library

[36]

Yoav Freund and Robert E. Schapire. 1997. A decision-theoretic generalization of on-line learning and an application to boosting. JCSS 55, 1 (1997), 119–139.

Digital Library

[37]

M. Frid-Adar, E. Klang, M. Amitai, J. Goldberger, and H. Greenspan. 2018. Synthetic data augmentation using GAN for improved liver lesion classification. In Proc. IEEE ISBI. 289–293.

[38]

J. Funke, M. Brown, S. M. Erlien, and J. C. Gerdes. 2017. Collision Avoidance and stabilization for autonomous vehicles in emergency scenarios. IEEE Trans. Control Syst. Technol. 25, 4 (2017), 1204–1216.

[39]

A. Geiger, P. Lenz, and R. Urtasun. 2012. Are we ready for autonomous driving? The KITTI vision benchmark suite. In Proc. IEEE CVPR. 3354–3361.

[40]

Ross Girshick. 2015. Fast R-CNN. In Proc. IEEE ICCV. 1440–1448.

[41]

Ross Girshick, Jeff Donahue, Trevor Darrell, and Jitendra Malik. 2014. Rich feature hierarchies for accurate object detection and semantic segmentation. In Proc. IEEE CVPR. 580–587.

Digital Library

[42]

Marc Green. 2000. “How long does it take to stop?” Methodological Analysis of Driver Perception-Brake Times. Transp. Hum. Factors 2, 3 (2000), 195–216.

[43]

X. Guo, Y. Li, and H. Ling. 2017. LIME: Low-light image enhancement via illumination map estimation. IEEE Trans. Image Process. 26, 2 (2017), 982–993.

Digital Library

[44]

Rajesh Gupta, Sudeep Tanwar, Neeraj Kumar, and Sudhanshu Tyagi. 2020. Blockchain-based security attack resilience schemes for autonomous vehicles in industry 4.0: A systematic review. Comput. Electr. Eng. 86 (2020), 106717.

[45]

Hangil Choi, S. Kim, Kihong Park, and K. Sohn. 2016. Multi-spectral pedestrian detection based on accumulated object proposal with fully convolutional networks. In Proc. IEEE ICPR. 621–626.

[46]

Bharath Hariharan, Pablo Arbeláez, Ross Girshick, and Jitendra Malik. 2014. Simultaneous detection and segmentation. In Proc. ECCV. 297–312.

[47]

Irtiza Hasan, Shengcai Liao, Jinpeng Li, Saad Ullah Akram, and Ling Shao. 2020. Pedestrian Detection: The Elephant In The Room. [Online]. Retrieved from https://arxiv.org/abs/2003.08799.

[48]

A. Hbaieb, J. Rezgui, and L. Chaari. 2019. Pedestrian Detection for autonomous driving within cooperative communication system. In Proc. IEEE WCNC. 1–6.

[49]

Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun. 2015. Spatial pyramid pooling in deep convolutional networks for visual recognition. IEEE Trans. Pattern Anal. Mach. Intell. 37, 9 (2015), 1904–1916.

Digital Library

[50]

Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun. 2016. Deep residual learning for image recognition. In Proc. IEEE CVPR. 770–778.

[51]

Geoffrey E. Hinton, Simon Osindero, and Yee-Whye Teh. 2006. A fast learning algorithm for deep belief nets. Neural Comput. 18, 7 (2006), 1527–1554.

Digital Library

[52]

Mazin Hnewa and Hayder Radha. 2020. Rain-adaptive intensity-driven object detection for autonomous vehicles. In Proc. SAE WCX.

[53]

Jan Hosang, Mohamed Omran, Rodrigo Benenson, and Bernt Schiele. 2015. Taking a deeper look at pedestrians. In Proc. IEEE CVPR. 4073–4082.

[54]

Andrew G. Howard, Menglong Zhu, Bo Chen, Dmitry Kalenichenko, Weijun Wang, Tobias Weyand, Marco Andreetto, and Hartwig Adam. 2017. MobileNets: Efficient Convolutional Neural Networks for Mobile Vision Applications. [Online]. Retrieved from http://arxiv.org/abs/1704.04861.

[55]

Jie Hu, Li Shen, and Gang Sun. 2018. Squeeze-and-excitation networks. In Proc. IEEE CVPR. 7132–7141.

[56]

Shengyuan Hu, Tao Yu, Chuan Guo, Wei-Lun Chao, and Kilian Q. Weinberger. 2019. A new defense against adversarial images: Turning a weakness into a strength. In Proc. NIPS. 1635–1646.

[57]

Xiaowei Hu, Chi-Wing Fu, Lei Zhu, and Pheng-Ann Heng. 2019. Depth-attentional features for single-image rain removal. In Proc. IEEE CVPR.

[58]

Qiangui Huang, Kevin Zhou, Suya You, and Ulrich Neumann. 2018. Learning to prune filters in convolutional neural networks. In Proc. IEEE WACV. 709–718.

[59]

Soonmin Hwang, Jaesik Park, Namil Kim, Yukyung Choi, and In So Kweon. 2015. Multispectral pedestrian detection: Benchmark dataset and baselines. In Proc. IEEE CVPR.

[60]

Forrest N. Iandola, Matthew W. Moskewicz, Khalid Ashraf, Song Han, William J. Dally, and Kurt Keutzer. 2016. SqueezeNet: AlexNet-level accuracy with 50x fewer parameters and <1MB model size. [Online]. Retrieved from http://arxiv.org/abs/1602.07360.

[61]

Sergey Ioffe and Christian Szegedy. 2015. Batch normalization: Accelerating deep network training by reducing internal covariate shift. In Proc. ICML. 448–456.

[62]

Max Jaderberg, Karen Simonyan, Andrew Zisserman, et al. 2015. Spatial transformer networks. In Proc. NIPS. 2017–2025.

[63]

Steve T. K. Jan, Joseph Messou, Yen-Chen Lin, Jia-Bin Huang, and Gang Wang. 2019. Connecting the digital and physical world: Improving the robustness of adversarial attacks. In Proc. AAAI, Vol. 33. 962–969.

[64]

I. Jegham and A. Ben Khalifa. 2017. Pedestrian detection in poor weather conditions using moving camera. In Proc. IEEE AICCSA. 358–362.

[65]

Xiaoheng Jiang, Yanwei Pang, Xuelong Li, and Jing Pan. 2016. Speed up deep neural network based pedestrian detection by sharing features across multi-scale models. Neurocomputing 185 (2016), 163–170.

Digital Library

[66]

Shu Wang Jingjing Liu, Shaoting Zhang and Dimitris Metaxas. 2016. Multispectral deep neural networks for pedestrian detection. In Proc. BMVC. 73.1–73.13.

[67]

S. Kato, S. Tokunaga, Y. Maruyama, S. Maeda, M. Hirabayashi, Y. Kitsukawa, A. Monrroy, T. Ando, Y. Fujii, and T. Azumi. 2018. Autoware on Board: Enabling autonomous vehicles with embedded systems. In Proc. IEEE ICCPS.

[68]

Kaveh Bakhsh Kelarestaghi, Mahsa Foruhandeh, Kevin Heaslip, and Ryan Gerdes. 2019. Survey on vehicular ad hoc networks and its access technologies security vulnerabilities and countermeasures. [Online]. Retrieved from http://arxiv.org/abs/1903.01541.

[69]

Jong Hyun Kim, Ganbayar Batchuluun, and Kang Ryoung Park. 2018. Pedestrian detection based on faster R-CNN in nighttime by fusing deep convolutional features of successive images. Expert Syst. Appl. 114 (2018), 15–33.

[70]

Alex Krizhevsky, Ilya Sutskever, and Geoffrey E. Hinton. 2012. Imagenet classification with deep convolutional neural networks. In Proc. NIPS. 1097–1105.

[71]

S. S. S. Kruthiventi, P. Sahay, and R. Biswal. 2017. Low-light pedestrian detection from RGB images using multi-modal knowledge distillation. In Proc. IEEE ICIP. 4207–4211.

[72]

M. Kutila, P. Pyykonen, H. Holzhuter, M. Colomb, and P. Duthon. 2018. Automotive LiDAR performance verification in fog and rain. In Proc. IEEE ITSC. 1695–1701.

[73]

Hei Law and Jia Deng. 2018. CornerNet: Detecting objects as paired keypoints. In Proc. ECCV. 734–750.

[74]

Y. Lecun, L. Bottou, Y. Bengio, and P. Haffner. 1998. Gradient-based learning applied to document recognition. Proc. IEEE 86, 11 (1998), 2278–2324.

[75]

J. Levinson, J. Askeland, J. Becker, J. Dolson, D. Held, S. Kammel, J. Z. Kolter, D. Langer, O. Pink, V. Pratt, M. Sokolsky, G. Stanek, D. Stavens, A. Teichman, M. Werling, and S. Thrun. 2011. Towards fully autonomous driving: Systems and algorithms. In IEEE IV. 163–168.

[76]

Bai Li, Changyou Chen, Wenlin Wang, and Lawrence Carin. 2019. Certified adversarial robustness with additive noise. In Proc. NIPS. 9464–9474.

[77]

G. Li, Y. Yang, and X. Qu. 2019. Deep learning approaches on pedestrian detection in hazy weather. IEEE Trans. Ind. Electron. (2019), 1–1. Early Access.

[78]

Jianan Li, Xiaodan Liang, ShengMei Shen, Tingfa Xu, Jiashi Feng, and Shuicheng Yan. 2017. Scale-aware fast R-CNN for pedestrian detection. IEEE Trans. Multimedia 20, 4 (2017), 985–996.

[79]

Junwei Liang, Lu Jiang, and Alexander Hauptmann. 2020. SimAug: Learning Robust Representations from Simulation for Trajectory Prediction. [Online]. Retrieved from https://arxiv.org/abs/2004.02022.

[80]

Chunze Lin, Jiwen Lu, Gang Wang, and Jie Zhou. 2018. Graininess-aware deep feature learning for pedestrian detection. In Proc. ECCV. 732–747.

[81]

Shih-Chieh Lin, Yunqi Zhang, Chang-Hong Hsu, Matt Skach, Md E. Haque, Lingjia Tang, and Jason Mars. 2018. The Architectural implications of autonomous driving: Constraints and acceleration. SIGPLAN Not. 53, 2 (2018), 751–766.

Digital Library

[82]

Tsung-Yi Lin, Piotr Dollár, Ross Girshick, Kaiming He, Bharath Hariharan, and Serge Belongie. 2017. Feature pyramid networks for object detection. In Proc. IEEE CVPR. 2117–2125.

[83]

Tsung-Yi Lin, Priya Goyal, Ross Girshick, Kaiming He, and Piotr Dollar. 2017. Focal loss for dense object detection. In Proc. IEEE ICCV. 2980–2988.

[84]

Tsung-Yi Lin, Michael Maire, Serge Belongie, James Hays, Pietro Perona, Deva Ramanan, Piotr Dollár, and C. Lawrence Zitnick. 2014. Microsoft COCO: Common objects in context. In Proc. ECCV. 740–755.

[85]

Songtao Liu, Di Huang, and Yunhong Wang. 2019. Adaptive NMS: Refining pedestrian detection in a crowd. In Proc. IEEE CVPR. 6459–6468.

[86]

Wei Liu, Dragomir Anguelov, Dumitru Erhan, Christian Szegedy, Scott Reed, Cheng-Yang Fu, and Alexander C. Berg. 2016. SSD: Single shot multibox detector. In Proc. ECCV. 21–37.

[87]

Wei Liu, Shengcai Liao, Weidong Hu, Xuezhi Liang, and Xiao Chen. 2018. Learning efficient single-stage pedestrian detectors by asymptotic localization fitting. In Proc. ECCV. 618–634.

[88]

Wei Liu, Shengcai Liao, Weiqiang Ren, Weidong Hu, and Yinan Yu. 2019. High-level semantic feature detection: A new perspective for pedestrian detection. In Proc. IEEE CVPR. 5187–5196.

[89]

Ping Luo, Yonglong Tian, Xiaogang Wang, and Xiaoou Tang. 2014. Switchable deep network for pedestrian detection. In Proc. IEEE CVPR. 899–906.

Digital Library

[90]

Abdelhamid Mammeri and Azzedine Boukerche. 2017. Inter-vehicle communication of warning information: an experimental study. Wireless Netw. 23, 6 (2017), 1837–1848.

Digital Library

[91]

Ignacio Martinez-Alpiste, Gelayol Golcarenarenji, Qi Wang, and Jose Maria Alcaraz-Calero. 2020. Real-time low-pixel infrared human detection from unmanned aerial vehicles. In Proc. DIVANet. 9–15.

Digital Library

[92]

S. K. Maurya and A. Choudhary. 2018. Deep learning–based vulnerable road user detection and collision avoidance. In Proc. IEEE ICVES. 1–6.

[93]

Tsubasa Minematsu, Hideaki Uchiyama, Atsushi Shimada, Hajime Nagahara, and Rin-ichiro Taniguchi. 2017. Adaptive background model registration for moving cameras. Pattern Recogn. Lett. 96, C (2017), 86–95.

[94]

Vinod Nair and Geoffrey E. Hinton. 2010. Rectified linear units improve restricted boltzmann machines. In Proc. ICML. 807–814.

[95]

Hyeonwoo Noh, Seunghoon Hong, and Bohyung Han. 2015. Learning deconvolution network for semantic segmentation. In Proc. IEEE ICCV. 1520–1528.

Digital Library

[96]

Junhyug Noh, Soochan Lee, Beomsu Kim, and Gunhee Kim. 2018. Improving occlusion and hard negative handling for single-stage pedestrian detectors. In Proc. IEEE CVPR.

[97]

Junhyug Noh, Soochan Lee, Beomsu Kim, and Gunhee Kim. 2018. Improving occlusion and hard negative handling for single-stage pedestrian detectors. In Proc. IEEE CVPR. 966–974.

[98]

Nvidia. [n.d.]. NVIDIA DRIVE AGX PEGASUS. [Online]. Retrieved from https://www.nvidia.com/en-us/self-driving-cars/drive-platform/hardware/.

[99]

Nvidia. [n.d.]. NVIDIA DRIVE SIM AND DRIVE CONSTELLATION. [Online]. Retrieved from https://www.nvidia.com/en-us/self-driving-cars/drive-constellation/.

[100]

W. Ouyang and X. Wang. 2012. A discriminative deep model for pedestrian detection with occlusion handling. In Proc. IEEE CVPR. 3258–3265.

[101]

Wanli Ouyang and Xiaogang Wang. 2013. Joint deep learning for pedestrian detection. In Proc. IEEE ICCV. 2056–2063.

Digital Library

[102]

Wanli Ouyang and Xiaogang Wang. 2013. Single-pedestrian detection aided by multi-pedestrian detection. In Proc. IEEE CVPR. 3198–3205.

Digital Library

[103]

Wanli Ouyang, Xingyu Zeng, and Xiaogang Wang. 2013. Modeling mutual visibility relationship in pedestrian detection. In Proc. IEEE CVPR. 3222–3229.

Digital Library

[104]

W. Ouyang, H. Zhou, H. Li, Q. Li, J. Yan, and X. Wang. 2018. Jointly learning deep features, deformable parts, occlusion and classification for pedestrian detection. IEEE Trans. Pattern Anal. Mach. Intell. 40, 8 (2018), 1874–1887.

[105]

Sakrapee Paisitkriangkrai, Chunhua Shen, and Anton Van Den Hengel. 2014. Strengthening the effectiveness of pedestrian detection with spatially pooled features. In Proc. ECCV. 546–561.

[106]

A. Palffy, J. F. P. Kooij, and D. M. Gavrila. 2019. Occlusion aware sensor fusion for early crossing pedestrian detection. In Proc. IEEE IV. 1768–1774.

[107]

Yanwei Pang, Jin Xie, Muhammad Haris Khan, Rao Muhammad Anwer, Fahad Shahbaz Khan, and Ling Shao. 2019. Mask-guided attention network for occluded pedestrian detection. In Proc. IEEE ICCV. 4967–4975.

[108]

C. P. Papageorgiou, M. Oren, and T. Poggio. 1998. A general framework for object detection. In Proc. IEEE ICCV. 555–562.

[109]

Constantine P. Papageorgiou, Michael Oren, and Tomaso Poggio. 1998. A general framework for object detection. In Proc. IEEE ICCV. 555–562.

[110]

Dennis Park, C. Lawrence Zitnick, Deva Ramanan, and Piotr Dollár. 2013. Exploring weak stabilization for motion feature extraction. In Proc. IEEE CVPR. 2882–2889.

Digital Library

[111]

Joel Pereira, Cristiano Premebida, Alireza Asvadi, F. Cannata, Luis Garrote, and U. J. Nunes. 2019. Test and evaluation of connected and autonomous vehicles in real-world scenarios. In Proc. IEEE IV. 14–19.

[112]

J. Redmon and A. Farhadi. 2017. YOLO9000: Better, faster, stronger. In Proc. IEEE CVPR. 6517–6525.

[113]

Kui Ren, Tianhang Zheng, Zhan Qin, and Xue Liu. 2020. Adversarial attacks and defenses in deep learning. Engineering 6, 3 (2020), 346–360.

[114]

Shaoqing Ren, Kaiming He, Ross Girshick, and Jian Sun. 2015. Faster R-CNN: Towards real-time object detection with region proposal networks. In Proc. NIPS. 91–99.

[115]

Dominik Scherer, Andreas Müller, and Sven Behnke. 2010. Evaluation of pooling operations in convolutional architectures for object recognition. In Proc. ICANN. 92–101.

[116]

M. Sha and A. Boukerche. 2020. Semantic fusion-based pedestrian detection for supporting autonomous vehicles. In Proc. IEEE ISCC. 618–623.

[117]

Weijing Shi, Mohamed Baker Alawieh, Xin Li, and Huafeng Yu. 2017. Algorithm and hardware implementation for visual perception system in autonomous vehicle: A survey. Integration 59 (2017), 148–156.

Digital Library

[118]

Abhinav Shrivastava, Abhinav Gupta, and Ross Girshick. 2016. Training region-based object detectors with online hard example mining. In Proc. IEEE CVPR. 761–769.

[119]

Abdul Jabbar Siddiqui and Azzedine Boukerche. 2020. Adversarial patches-based attacks on automated vehicle make and model recognition systems. In Proc. Q2SWinet. 115–121.

Digital Library

[120]

Karen Simonyan and Andrew Zisserman. 2015. Very deep convolutional networks for large-scale image recognition. [Online]. Retrieved from http://arxiv.org/abs/1409.1556. In Proc. ICLR.

[121]

Tao Song, Leiyu Sun, Di Xie, Haiming Sun, and Shiliang Pu. 2018. Small-scale pedestrian detection based on topological line localization and temporal feature aggregation. In Proc. ECCV. 536–551.

[122]

Jack Stewart. [n.d.]. Self-Driving Cars Use Crazy Amounts of Power, and It’s Becoming a Problem. [Online]. Retrieved from https://www.wired.com/story/self-driving-cars-power-consumption-nvidia-chip/.

[123]

Peng Sun, Noura AlJeri, and Azzedine Boukerche. 2020. An energy-efficient proactive handover scheme for vehicular networks based on passive RSU detection. IEEE Trans. Sustain. Comput. 5, 1 (2020), 37–47.

[124]

Siyu Tang, Mykhaylo Andriluka, and Bernt Schiele. 2014. Detection and tracking of occluded people. Int. J. Comput. Vis. 110, 1 (2014), 58–69.

[125]

Yonglong Tian, Ping Luo, Xiaogang Wang, and Xiaoou Tang. 2015. Deep learning strong parts for pedestrian detection. In Proc. IEEE ICCV. 1904–1912.

Digital Library

[126]

Yonglong Tian, Ping Luo, Xiaogang Wang, and Xiaoou Tang. 2015. Pedestrian detection aided by deep learning semantic tasks. In Proc. IEEE CVPR. 5079–5087.

[127]

Cosmin Toca, Mihai Ciuc, and Carmen Patrascu. 2015. Normalized autobinomial Markov channels for pedestrian detection. In Proc. BMVC. 175.1–175.13.

[128]

Alexander Toshev and Christian Szegedy. 2014. DeepPose: Human pose estimation via deep neural networks. In Proc. IEEE CVPR.

Digital Library

[129]

Jasper R. R. Uijlings, Koen E. A. Van De Sande, Theo Gevers, and Arnold W. M. Smeulders. 2013. Selective search for object recognition. Int. J. Comput. Vision 104, 2 (2013), 154–171.

Digital Library

[130]

Jessica Van Brummelen, Marie O’Brien, Dominique Gruyer, and Homayoun Najjaran. 2018. Autonomous vehicle perception: The technology of today and tomorrow. Trans. Res. Part C Emerg. Technol. 89 (2018), 384–406.

[131]

Jaycil Z. Varghese, Randy G. Boone, et al. 2015. Overview of autonomous vehicle sensors and systems. In Proc. IEOM. 178–191.

[132]

Paul Viola and Michael J. Jones. 2004. Robust real-time face detection. Int. J. Comput. Vis 57, 2 (2004), 137–154.

Digital Library

[133]

Tuan-Hung Vu, Anton Osokin, and Ivan Laptev. 2015. Context-aware CNNs for person head detection. In Proc. IEEE ICCV. 2893–2901.

Digital Library

[134]

Heng Wang, Bin Wang, Bingbing Liu, Xiaoli Meng, and Guanghong Yang. 2017. Pedestrian recognition and tracking using 3D LiDAR for autonomous vehicle. Rob. Auton. Syst. 88 (2017), 71–78.

Digital Library

[135]

Shiguang Wang, Jian Cheng, Haijun Liu, Feng Wang, and Hui Zhou. 2018. Pedestrian detection via body part semantic and contextual information with DNN. IEEE Trans. Multimedia 20, 11 (2018), 3148–3159.

Digital Library

[136]

Tianyu Wang, Xin Yang, Ke Xu, Shaozhe Chen, Qiang Zhang, and Rynson W. H. Lau. 2019. Spatial attentive single-image deraining with a high quality real rain dataset. In Proc. IEEE CVPR.

[137]

Xiaoyu Wang, Tony X. Han, and Shuicheng Yan. 2009. An HOG-LBP human detector with partial occlusion handling. In Proc. IEEE ICCV. 32–39.

[138]

Xiaolong Wang, Abhinav Shrivastava, and Abhinav Gupta. 2017. A-fast-rcnn: Hard positive generation via adversary for object detection. In Proc. IEEE CVPR. 2606–2615.

[139]

Xinlong Wang, Tete Xiao, Yuning Jiang, Shuai Shao, Jian Sun, and Chunhua Shen. 2018. Repulsion loss: Detecting pedestrians in a crowd. In Proc. IEEE CVPR. 7774–7783.

[140]

Christian Wojek and Bernt Schiele. 2008. A performance evaluation of single and multi-feature people detection. In Proc. JPRS. 82–91.

Digital Library

[141]

Sanghyun Woo, Jongchan Park, Joon-Young Lee, and In So Kweon. 2018. Cbam: Convolutional block attention module. In Proc. ECCV. 3–19.

[142]

Cihang Xie, Jianyu Wang, Zhishuai Zhang, Yuyin Zhou, Lingxi Xie, and Alan Yuille. 2017. Adversarial examples for semantic segmentation and object detection. In Proc. IEEE ICCV. 1369–1378.

[143]

H. Xiong, F. B. Flohr, S. Wang, B. Wang, J. Wang, and K. Li. 2019. Recurrent neural network architectures for vulnerable road user trajectory prediction. In Proc. IEEE IV. 171–178.

[144]

Bin Yang, Junjie Yan, Zhen Lei, and Stan Z. Li. 2015. Convolutional channel features. In Proc. IEEE ICCV. 82–90.

[145]

S. Zang, M. Ding, D. Smith, P. Tyler, T. Rakotoarivelo, and M. A. Kaafar. 2019. The impact of adverse weather conditions on autonomous vehicles: How rain, snow, fog, and hail affect the performance of a self-driving car. IEEE Veh. Technol. Mag. 14, 2 (2019), 103–111.

[146]

He Zhang and Vishal M. Patel. 2018. Densely connected pyramid dehazing network. In Proc. IEEE CVPR.

[147]

Jialiang Zhang, Lixiang Lin, Yun-chen Chen, Yao Hu, Steven C. H. Hoi, and Jianke Zhu. 2019. CSID: Center, Scale, Identity and Density-aware Pedestrian Detectionin a Crowd. [Online]. Retrieved from https://arxiv.org/abs/1910.09188.

[148]

Liliang Zhang, Liang Lin, Xiaodan Liang, and Kaiming He. 2016. Is faster r-cnn doing well for pedestrian detection? In Proc. ECCV. 443–457.

[149]

Shanshan Zhang, Christian Bauckhage, and Armin B. Cremers. 2014. Informed haar-like features improve pedestrian detection. In Proc. IEEE CVPR.

[150]

Shanshan Zhang, Rodrigo Benenson, Mohamed Omran, Jan Hosang, and Bernt Schiele. 2016. How far are we from solving pedestrian detection? In Proc. IEEE CVPR. 1259–1267.

[151]

S. Zhang, R. Benenson, and B. Schiele. 2017. CityPersons: A diverse dataset for pedestrian detection. In Proc. IEEE CVPR. 4457–4465.

[152]

Shanshan Zhang, Rodrigo Benenson, Bernt Schiele, et al. 2015. Filtered channel features for pedestrian detection. In Proc. IEEE CVPR. 1751–1760.

[153]

Shifeng Zhang, Longyin Wen, Xiao Bian, Zhen Lei, and Stan Z. Li. 2018. Occlusion-aware R-CNN: Detecting pedestrians in a crowd. In Proc. ECCV. 637–653.

[154]

Shanshan Zhang, Jian Yang, and Bernt Schiele. 2018. Occluded pedestrian detection through guided attention in CNNs. In Proc. IEEE CVPR. 6995–7003.

[155]

Xiaowei Zhang, Li Cheng, Bo Li, and Hai-Miao Hu. 2018. Too far to see? Not really!–pedestrian detection with scale-aware localization policy. IEEE Trans. Image Process. 27, 8 (2018), 3703–3715.

[156]

Zhen Jia, A. Balasuriya, and S. Challa. 2006. Recent developments in vision based target tracking for autonomous vehicles navigation. In Proc. IEEE Trans. Intell. Transp. Syst.765–770.

[157]

Chunluan Zhou and Junsong Yuan. 2017. Multi-label learning of part detectors for heavily occluded pedestrian detection. In Proc. IEEE ICCV. 3486–3495.

[158]

Chunluan Zhou and Junsong Yuan. 2018. Bi-box regression for pedestrian detection and occlusion estimation. In Proc. ECCV. 135–151.

[159]

Yousong Zhu, Jinqiao Wang, Chaoyang Zhao, Haiyun Guo, and Hanqing Lu. 2016. Scale-adaptive deconvolutional regression network for pedestrian detection. In Proc. ACCV. 416–430.

Cited By

Kumar Jain DZhao XGarcia SNeelakandan S(2024)Robust multi-modal pedestrian detection using deep convolutional neural network with ensemble learning modelExpert Systems with Applications: An International Journal10.1016/j.eswa.2024.123527249:PAOnline publication date: 1-Sep-2024
https://dl.acm.org/doi/10.1016/j.eswa.2024.123527
Sumi ASantha T(2024)Analysis of Machine Learning Approaches to Detect Pedestrian Under Different Scale Using Frame Level Difference FeatureComputational Sciences and Sustainable Technologies10.1007/978-3-031-50993-3_37(464-473)Online publication date: 3-Feb-2024
https://doi.org/10.1007/978-3-031-50993-3_37
Merino Gómez EMoral Andrés FReviriego Vasallo P(2023)Arquitecturas inconclusas: una perspectiva desde la Inteligencia ArtificialEGA Revista de Expresión Gráfica Arquitectónica10.4995/ega.2023.1905328:48(254-267)Online publication date: 24-Jul-2023
https://doi.org/10.4995/ega.2023.19053
Show More Cited By

Index Terms

Design Guidelines on Deep Learning–based Pedestrian Detection Methods for Supporting Autonomous Vehicles
1. Computing methodologies
  1. Artificial intelligence
    1. Computer vision
      1. Computer vision problems
        Object detection
  2. Machine learning
    1. Machine learning approaches
      1. Neural networks
2. Information systems
  1. Information retrieval

Recommendations

Dynamic intersections and self-driving vehicles
ICCPS '18: Proceedings of the 9th ACM/IEEE International Conference on Cyber-Physical Systems

Connected and automated vehicles are expected to be at the core of future intelligent transportation systems. One of the main practical challenges for self-driving vehicles on public roads is safe cooperation and collaboration among multiple vehicles ...
A novel visibility semantic feature-aided pedestrian detection scheme for autonomous vehicles
Abstract
Intelligent transportation systems (ITS) have become a popular method for enhancing transportation safety and efficiency. As essential participants of ITS, autonomous vehicles need to detect pedestrians accurately. In this paper, we ...
Performance evaluation of CNN-based pedestrian detectors for autonomous vehicles
Abstract
With the widespread application of deep learning methodologies, many fields including Intelligent Transportation Systems (ITS) have integrated neural network-based models. In return, the promising performance of neural network-based ...

Comments

Information & Contributors

Information

Published In

cover image ACM Computing Surveys

ACM Computing Surveys Volume 54, Issue 6

Invited Tutorial

July 2022

799 pages

ISSN:0360-0300

EISSN:1557-7341

DOI:10.1145/3475936

Editor:
Albert Zomaya
University of Sydney, Australia

Issue’s Table of Contents

Copyright © 2021 ACM.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 03 August 2021

Accepted: 01 April 2021

Revised: 01 December 2020

Received: 01 March 2020

Published in CSUR Volume 54, Issue 6

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Research-article
Research
Refereed

Funding Sources

NSERC-SPG
NSERC-CREATE TRANSIT
NSERC-DISCOVERY
Canada Research Chairs Program

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

14
Total Citations
View Citations
672
Total Downloads

Downloads (Last 12 months)159
Downloads (Last 6 weeks)9

Reflects downloads up to 30 Aug 2024

Other Metrics

View Author Metrics

Citations

Cited By

Kumar Jain DZhao XGarcia SNeelakandan S(2024)Robust multi-modal pedestrian detection using deep convolutional neural network with ensemble learning modelExpert Systems with Applications: An International Journal10.1016/j.eswa.2024.123527249:PAOnline publication date: 1-Sep-2024
https://dl.acm.org/doi/10.1016/j.eswa.2024.123527
Sumi ASantha T(2024)Analysis of Machine Learning Approaches to Detect Pedestrian Under Different Scale Using Frame Level Difference FeatureComputational Sciences and Sustainable Technologies10.1007/978-3-031-50993-3_37(464-473)Online publication date: 3-Feb-2024
https://doi.org/10.1007/978-3-031-50993-3_37
Merino Gómez EMoral Andrés FReviriego Vasallo P(2023)Arquitecturas inconclusas: una perspectiva desde la Inteligencia ArtificialEGA Revista de Expresión Gráfica Arquitectónica10.4995/ega.2023.1905328:48(254-267)Online publication date: 24-Jul-2023
https://doi.org/10.4995/ega.2023.19053
Jin ZWang WWang YWang GZhang Q(2023)Design of automatic tobacco trash detection system based on quadratic fuzzy theoryApplied Mathematics and Nonlinear Sciences10.2478/amns.2023.2.005389:1Online publication date: 9-Oct-2023
https://doi.org/10.2478/amns.2023.2.00538
Zhang CZheng SWu HGu ZSun WYang L(2023)AttentionTrack: Multiple Object Tracking in Traffic Scenarios Using Features AttentionIEEE Transactions on Intelligent Transportation Systems10.1109/TITS.2023.331522225:2(1661-1674)Online publication date: 22-Sep-2023
https://dl.acm.org/doi/10.1109/TITS.2023.3315222
Bai ZNayak SZhao XWu GBarth MQi XLiu YSisbot EOguchi K(2023)Cyber Mobility Mirror: A Deep Learning-Based Real-World Object Perception Platform Using Roadside LiDARIEEE Transactions on Intelligent Transportation Systems10.1109/TITS.2023.326828124:9(9476-9489)Online publication date: 1-Sep-2023
https://dl.acm.org/doi/10.1109/TITS.2023.3268281
Pan Z(2023)Multi-Scale Occluded Pedestrian Detection Based on Deep Learning2023 International Conference on Evolutionary Algorithms and Soft Computing Techniques (EASCT)10.1109/EASCT59475.2023.10393467(1-6)Online publication date: 20-Oct-2023
https://doi.org/10.1109/EASCT59475.2023.10393467
Cardaioli MConti MOrazi GTricomi PTsudik G(2023)BLUFADER: Blurred face detection & recognition for privacy-friendly continuous authenticationPervasive and Mobile Computing10.1016/j.pmcj.2023.10180192(101801)Online publication date: May-2023
https://doi.org/10.1016/j.pmcj.2023.101801
Jain DZhao XGonzález-Almagro GGan CKotecha K(2023)Multimodal pedestrian detection using metaheuristics with deep convolutional neural network in crowded scenesInformation Fusion10.1016/j.inffus.2023.02.01495:C(401-414)Online publication date: 1-Jul-2023
https://dl.acm.org/doi/10.1016/j.inffus.2023.02.014
Devipriya APrabakar DSingh LOliver AQamar SAzeem A(2023)Machine learning-driven pedestrian detection and classification for electric vehicles: integrating Bayesian component network analysis and reinforcement region-based convolutional neural networksSignal, Image and Video Processing10.1007/s11760-023-02681-117:8(4475-4483)Online publication date: 3-Aug-2023
https://doi.org/10.1007/s11760-023-02681-1
Show More Cited By

View Options

Get Access

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Article

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

HTML Format

View this article in HTML Format.

Media

Figures

Other

Tables

View Issue’s Table of Contents