A Constructive Review on Pedestrian Action Detection, Recognition and Prediction
Authors: 
Md Akib Shahriar Khan, 
Md Jannatul Baki Showmik, Tanvir Ahmed, 
A F M Saifuddin SaifAuthors Info & Claims
Md Akib Shahriar Khan, Md Jannatul Baki Showmik, Tanvir Ahmed, A F M Saifuddin SaifAuthors Info & ClaimsPages 367 - 376
Abstract
Analysis of pedestrian activities in the video sequences is an intriguing domain that incorporates vast applications, such as autonomous driving systems, traffic control systems and interactions between people and computers. The primary focus of this research was on evaluating several strategies to analyse pedestrian activities effectively. The constructive comparison included three main steps, i.e. detection of the pedestrian, recognition of their actions and prediction about the activity of the pedestrian. Changes in activities of pedestrians, dynamic background, moving camera, view angle and processing time made it more challenging. Recent approaches were justified and compared based on precision accuracy, processing time and minimum resource allocation. The results were also compared by a series of state-of-the-art research datasets with provided significant observations in terms of greater accuracy which can lead to the construction of an extremely improvised system that would save pedestrian people from road accidents and assist autonomous driving systems. The purpose of this study is to discuss the current progress using different approaches.
References
[1]
A.S. Saif, M.A.S. Khan, A.M. Hadi, R.P. Karmoker, and J.J. Gomes, Aggressive Action Estimation: A Comprehensive Review on Neural Network Based Human Segmentation and Action Recognition. International Journal of Education and Management Engineering 9(2019) 9. https://doi.org/10.5815/ijeme.2019.01.02
[2]
J. Hariyono, and K.-H. Jo, Centroid based pose ratio for pedestrian action recognition, 2016 IEEE 25th International Symposium on Industrial Electronics (ISIE), IEEE, 2016, pp. 895-900. https://doi.org/10.1109/isie.2016.7745009
[3]
A.T. Schulz, and R. Stiefelhagen, A controlled interactive multiple model filter for combined pedestrian intention recognition and path prediction, 2015 IEEE 18th International Conference on Intelligent Transportation Systems, IEEE, 2015, pp. 173-178. https://doi.org/10.1109/itsc.2015.37
[4]
M. Raza, Z. Chen, S.-U. Rehman, P. Wang, and P. Bao, Appearance based pedestrians’ head pose and body orientation estimation using deep learning. Neurocomputing 272 (2018) 647-659. https://doi.org/10.1016/j.neucom.2017.07.029
[5]
H. Song, I.K. Choi, M.S. Ko, J. Bae, S. Kwak, and J. Yoo, Vulnerable pedestrian detection and tracking using deep learning, 2018 International Conference on Electronics, Information, and Communication (ICEIC), IEEE, 2018, pp. 1-2. https://doi.org/10.23919/elinfocom.2018.8330547
[6]
E.J. Lee, B.C. Ko, and J.-Y. Nam, Recognizing pedestrian's unsafe behaviours in far-infrared imagery at night. Infrared Physics & Technology 76 (2016) 261-270. https://doi.org/10.1016/j.infrared.2016.03.006
[7]
R. Quintero, I. Parra, D.F. Llorca, and M. Sotelo, Pedestrian intention and pose prediction through dynamical models and behaviour classification, 2015 IEEE 18th International Conference on Intelligent Transportation Systems, IEEE, 2015, pp. 83-88. https://doi.org/10.1109/itsc.2015.22
[8]
L. Zhang, L. Lin, X. Liang, and K. He, Is faster r-cnn doing well for pedestrian detection?, European conference on computer vision, Springer, 2016, pp. 443-457. https://doi.org/10.1007/978-3-319-46475-6_28
[9]
J. Hariyono, and K.-H. Jo, Pedestrian action recognition using motion type classification, 2015 IEEE 2nd International Conference on Cybernetics (CYBCONF), IEEE, 2015, pp. 129- 132. https://doi.org/10.1109/cybconf.2015.7175919
[10]
R.M. Mueid, C. Ahmed, and M.A.R. Ahad, Pedestrian activity classification using patterns of motion and histogram of oriented gradient. Journal on Multimodal User Interfaces 10 (2016) 299-305. https://doi.org/10.1007/s12193-015-0178-3
[11]
B. Hilsenbeck, D. Münch, A.-K. Grosselfinger, W. Hübner, and M. Arens, Action recognition in the longwave infrared and the visible spectrum using Hough forests, 2016 IEEE International Symposium on Multimedia (ISM), IEEE, 2016, pp. 329-332. https://doi.org/10.1109/ism.2016.0072
[12]
P. Zhang, C. Lan, J. Xing, W. Zeng, J. Xue, and N. Zheng, View adaptive recurrent neural networks for high performance human action recognition from skeleton data, Proceedings of the IEEE International Conference on Computer Vision, 2017, pp. 2117-2126. https://doi.org/10.1109/iccv.2017.233
[13]
C. Li, Z. Cui, W. Zheng, C. Xu, R. Ji, and J. Yang, Action-attending graphic neural network. IEEE Transactions on Image Processing 27 (2018) 3657-3670. https://doi.org/10.1109/tip.2018.2815744
[14]
J.S. Casallas, J.H. Oliver, J.W. Kelly, F. Merienne, and S. Garbaya, Towards a model for predicting intention in 3D moving-target selection tasks, International Conference on Engineering Psychology and Cognitive Ergonomics, Springer, 2013, pp. 13-22. https://doi.org/10.1007/978-3-642-39360-0_2
[15]
A.T. Schulz, and R. Stiefelhagen, A controlled interactive multiple model filter for combined pedestrian intention recognition and path prediction, 2015 IEEE 18th International Conference on Intelligent Transportation Systems, IEEE, 2015, pp. 173-178. https://doi.org/10.1109/itsc.2015.37
[16]
M. Raza, Z. Chen, S.-U. Rehman, P. Wang, and P. Bao, Appearance based pedestrians’ head pose and body orientation estimation using deep learning. Neurocomputing 272 (2018) 647- 659. https://doi.org/10.1016/j.neucom.2017.07.029
[17]
A. Rudenko, L. Palmieri, and K.O. Arras, Joint long-term prediction of human motion using a planning-based social force approach, 2018 IEEE International Conference on Robotics and Automation (ICRA), IEEE, 2018, pp. 1-7. https://doi.org/10.1109/icra.2018.8460527
[18]
I. Batkovic, M. Zanon, N. Lubbe, and P. Falcone, A computationally efficient model for pedestrian motion prediction, 2018 European Control Conference (ECC), IEEE, 2018, pp. 374-379. https://doi.org/10.23919/ecc.2018.8550300
[19]
D.-P. Tran, N.G. Nhu, and V.-D. Hoang, Pedestrian action prediction based on deep features extraction of human posture and traffic scene, Asian Conference on Intelligent Information and Database Systems, Springer, 2018, pp. 563-572. https://doi.org/10.1007/978-3-319-75420-8_53
[20]
H. Kataoka, Y. Satoh, Y. Aoki, S. Oikawa, and Y. Matsui, Temporal and fine-grained pedestrian action recognition on driving recorder database. Sensors 18 (2018) 627. https://doi.org/10.3390/s18020627
[21]
J.-Y. Kwak, B.C. Ko, and J.-Y. Nam, Pedestrian intention prediction based on dynamic fuzzy automata for vehicle driving at nighttime. Infrared Physics & Technology 81 (2017) 41-51. https://doi.org/10.1016/j.infrared.2016.12.014
[22]
R. Mueid, L. Christopher, and R. Tian, Vehicle-pedestrian dynamic interaction through tractography of relative movements and articulated pedestrian pose estimation, 2016 IEEE Applied Imagery Pattern Recognition Workshop (AIPR), IEEE, 2016, pp. 1-6. https://doi.org/10.1109/aipr.2016.8010592
[23]
O. Ghori, R. Mackowiak, M. Bautista, N. Beuter, L. Drumond, F. Diego, and B. Ommer, Learning to forecast pedestrian intention from pose dynamics, 2018 IEEE Intelligent Vehicles Symposium (IV), IEEE, 2018, pp. 1277-1284. https://doi.org/10.1109/ivs.2018.8500657
[24]
K. Nishida, T. Kobayashi, T. Iwamoto, and S. Yamasaki, Pedestrian action prediction using static image feature, 2015 7th International Joint Conference on Computational Intelligence (IJCCI), IEEE, 2015, pp. 99-105. https://doi.org/10.5220/0005593600990105
[25]
J. Qianyin, L. Guoming, Y. Jinwei, and L. Xiying, A model based method of pedestrian abnormal behaviour detection in traffic scene, 2015 IEEE First International Smart Cities Conference (ISC2), IEEE, 2015, pp. 1-6. https://doi.org/10.1109/isc2.2015.7366164
[26]
R.Q. Mínguez, I.P. Alonso, D. Fernández-Llorca, and M.Á. Sotelo, Pedestrian path, pose, and intention prediction through gaussian process dynamical models and pedestrian activity recognition. IEEE Transactions on Intelligent Transportation Systems 20 (2018) 1803-1814. https://doi.org/10.1109/tits.2018.2836305
[27]
J. Almeida, and V. Santos, Pedestrian pose estimation using stereo perception, Robot 2015: Second Iberian Robotics Conference, Springer, 2016, pp. 491-502. https://doi.org/10.1007/978-3-319-27146-0_38
[28]
R. Quintero, I. Parra, D.F. Llorca, and M. Sotelo, Pedestrian intention and pose prediction through dynamical models and behaviour classification, 2015 IEEE 18th International Conference on Intelligent Transportation Systems, IEEE, 2015, pp. 83-88. https://doi.org/10.1109/itsc.2015.22
[29]
J. Hariyono, and K.-H. Jo, Detection of pedestrian crossing road using action classification model, 2015 IEEE International Conference on Advanced Intelligent Mechatronics (AIM), IEEE, 2015, pp. 21-24. https://doi.org/10.1109/aim.2015.7222502
[30]
E.J. Lee, B.C. Ko, and J.-Y. Nam, Recognizing pedestrians's unsafe behaviours in far-infrared imagery at night. Infrared Physics & Technology 76 (2016) 261-270. https://doi.org/10.1016/j.infrared.2016.03.006
[31]
Z. Fang, and A.M. López, Is the pedestrian going to cross? answering by 2d pose estimation, 2018 IEEE Intelligent Vehicles Symposium (IV), IEEE, 2018, pp. 1271-1276. https://doi.org/10.1109/ivs.2018.8500413
[32]
J. Liu, A. Shahroudy, G. Wang, L.-Y. Duan, and A.K. Chichung, Skeleton-based online action prediction using scale selection network. IEEE transactions on pattern analysis and machine intelligence (2019). https://doi.org/10.1109/tpami.2019.2898954
[33]
L. Wang, Y. Xiong, Z. Wang, Y. Qiao, D. Lin, X. Tang, and L. Van Gool, Temporal segment networks for action recognition in videos. IEEE transactions on pattern analysis and machine intelligence (2018). https://doi.org/10.1109/cvpr.2018.00705
[34]
P. Zhang, C. Lan, J. Xing, W. Zeng, J. Xue, and N. Zheng, View adaptive neural networks for high performance skeleton-based human action recognition. IEEE transactions on pattern analysis and machine intelligence (2019). https://doi.org/10.1109/iccv.2017.233
[35]
S. Agahian, F. Negin, and C. Köse, An efficient human action recognition framework with pose-based spatiotemporal features. Engineering Science and Technology, an International Journal (2019). https://doi.org/10.1016/j.jestch.2019.04.014
[36]
N. Jaouedi, N. Boujnah, and M.S. Bouhlel, A new hybrid deep learning model for human action recognition. Journal of King Saud University-Computer and Information Sciences (2019). https://doi.org/10.1016/j.jksuci.2019.09.004
[37]
W. You, J. Guo, K. Shan, and Y. Dai, A Novel Trajectory-VLAD Based Action Recognition Algorithm for Video Analysis. Procedia Computer Science 147 (2019) 165-171. https://doi.org/10.1016/j.procs.2019.01.213
[38]
D. Anisuzzaman, and A.S. Saif, Efficient Framework Using Morphological Modelling for Frequent Iris Movement Investigation towards Questionable Observer Detection. International Journal of Image, Graphics and Signal Processing 10 (2018) 28. https://doi.org/10.5815/ijigsp.2018.11.04
[39]
D. Anisuzzaman, and A.S. Saif, A Study of Activity Recognition and Questionable Observer Detection. International Journal of Computer Applications 975 8887. https://doi.org/10.5120/ijca2018917855
[40]
Z.R. Mahayuddin, and A.S. Saif, Fast and Effective Motion Model for Moving Object Detection Using Aerial Images. International Journal of Computer Vision and Signal Processing (IJCVSP) 1 (2018) 1-11.
[41]
A. Saif, A.S. Prabuwono, and Z.R. Mahayuddin, Moving object detection using dynamic motion modelling from UAV aerial images. The Scientific World Journal 2014 (2014). https://doi.org/10.1155/2014/890619
[42]
A.S. Saif, M.S. Hossain, K.T. Hasan, and M. Rahman, Measurement of Unique Pupillary Distance using Modified Circle Algorithm. International Journal of Computer Applications 975 8887. https://doi.org/10.5120/ijca2018916125
[43]
A.S. Saif, and M.S. Hossain, A Study of Pupil Orientation and Detection of Pupil using Circle Algorithm: A Review. International Journal of Engineering Trends and Technology (IJETT) 54 (2017). https://doi.org/10.14445/22315381/ijett-v54p203
[44]
A.S. Saif, A.G. Garba, J. Awwalu, H. Arshad, and L.Q. Zakaria, Performance Comparison of Min-Max Normalisation on Frontal Face Detection Using Haar Classifiers. Pertanika Journal of Science and Technology 25 (2017) 163-171.
[45]
A.S. Saif, A.S. Prabuwono, and Z.R. Mahayuddin, Moment feature based fast feature extraction algorithm for moving object detection using aerial images. PloS one 10 (2015) e0126212. https://doi.org/10.1371/journal.pone.0126212
[46]
Z.R. Mahayuddin, A.S. Saif, and A.S. Prabuwono, Efficiency measurement of various denoise techniques for moving object detection using aerial images, 2015 International Conference on Electrical Engineering and Informatics (ICEEI), IEEE, 2015, pp. 161-165. https://doi.org/10.1109/iceei.2015.7352488
[47]
A.S. Saif, A.S. Prabuwono, and Z.R. Mahayuddin, Motion analysis for moving object detection from UAV aerial images: A review, 2014 International Conference on Informatics, Electronics & Vision (ICIEV), IEEE, 2014, pp. 1-6. https://doi.org/10.1109/iciev.2014.6850753
[48]
A.S. Saif, A.S. Prabuwono, and Z.R. Mahayuddin, Adaptive motion pattern analysis for machine vision based moving detection from UAV aerial images, International Visual Informatics Conference, Springer, 2013, pp. 104-114. https://doi.org/10.1007/978-3-319-02958-0_10
[49]
D. Nandi, A.S. Saif, P. Prottoy, K.M. Zubair, and S.A. Shubho, Traffic sign detection based on colour segmentation of obscure image candidates: a comprehensive study. International Journal of Modern Education and Computer Science 10 (2018) 35. https://doi.org/10.5815/ijmecs.2018.06.05
[50]
A.S. Saif, A.S. Prabuwono, Z.R. Mahayuddin, and H.T. Himawan, A review of machine vision based on moving objects: object detection from UAV aerial images. International Journal of Advancements in Computing Technology 5 (2013) 57.
[51]
A.S. Saif, A.S. Prabuwono, Z.R. Mahayuddin, and T. Mantoro, Vision-based human face recognition using extended principal component analysis. International Journal of Mobile Computing and Multimedia Communications (IJMCMC) 5 (2013) 82-94. https://doi.org/10.4018/ijmcmc.2013100105
[52]
A.S. Saif, A.S. Prabuwono, and Z.R. Mahayuddin, Real time vision based object detection from UAV aerial images: a conceptual framework, FIRA RoboWorld Congress, Springer, 2013, pp. 265-274. https://doi.org/10.1007/978-3-642-40409-2_23
[53]
E.N. Kajabad, and S.V. Ivanov, People Detection and Finding Attractive Areas by the use of Movement Detection Analysis and Deep Learning Approach. Procedia Computer Science 156 (2019) 327-337. https://doi.org/10.1016/j.procs.2019.08.209
[54]
T. Wang, Z. Miao, Y. Chen, Y. Zhou, G. Shan, and H. Snoussi, AED-Net: An Abnormal Event Detection Network. Engineering (2019). https://doi.org/10.1016/j.eng.2019.02.008
[55]
F. Letsch, D. Jirak, and S. Wermter, Localising salient body motion in multi-person scenes using convolutional neural networks. Neurocomputing 330 (2019) 449-464. https://doi.org/10.1016/j.neucom.2018.11.048
[56]
Z.R. Mahayuddin, and A.S. Saif, A COMPARATIVE STUDY OF THREE CORNER FEATURE BASED MOVING OBJECT DETECTION USING AERIAL IMAGES. Malaysian Journal of Computer Science S.1 (2019) 25-33. https://doi.org/10.22452/mjcs.sp2019no3.2
[57]
Z.R. Mahayuddin and A.S. Saif, Efficiency measurement of various denoise techniques for moving object detection using aerial images, International Visual Informatics Conference, Springer, 2019, pp. 227-236. https://doi.org/10.1007/978-3-030-34032-2_21
[58]
Saif, A. F., Khan, M. A., Hadi, A. M., Karmoker, R. P., & Gomes, J. J. (2021). Silhouette Pose Feature-Based Human Action Classification Using Capsule Network. Journal of Information Technology Research (JITR), 14(2), 106-124. http://doi.org/10.4018/JITR.2021040106
Index Terms
- A Constructive Review on Pedestrian Action Detection, Recognition and Prediction
Recommendations
Pedestrian Action Prediction using Static Image Feature
IJCCI 2015: Proceedings of the 7th International Joint Conference on Computational IntelligenceIn this study, we propose a method to predict how the target object move (run or walk) in the
future by using only appearance-based image features. Such kind of motion prediction significantly
contributes to intelligent braking system in cars; by ...
Human Action Recognition and Prediction: A Survey
AbstractDerived from rapid advances in computer vision and machine learning, video analysis tasks have been moving from inferring the present state to predicting the future state. Vision-based action recognition and prediction from videos are such tasks, ...
Detection of pedestrian crossing road
Detection of pedestrian crossing road is the objective of this work. The model incorporates the pedestrian pose recognition and lateral speed, motion direction and spatial layout of the environment. Pedestrian poses are recognized according to the ...
Comments
Information & Contributors
Information
Published In
March 2022
543 pages
ISBN:9781450397346
DOI:10.1145/3542954
Copyright © 2022 ACM.
Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]
Publisher
Association for Computing Machinery
New York, NY, United States
Publication History
Published: 11 August 2022
Check for updates
Author Tags
Qualifiers
- Research-article
- Research
- Refereed limited
Conference
ICCA 2022
ICCA 2022: 2nd International Conference on Computing Advancements
March 10 - 12, 2022
Dhaka, Bangladesh
Contributors
Other Metrics
Bibliometrics & Citations
Bibliometrics
Article Metrics
- 0Total Citations
- 70Total Downloads
- Downloads (Last 12 months)25
- Downloads (Last 6 weeks)2
Reflects downloads up to 28 Dec 2024
Other Metrics
Citations
View Options
Login options
Check if you have access through your login credentials or your institution to get full access on this article.
Sign inFull Access
View options
View or Download as a PDF file.
PDFeReader
View online with eReader.
eReaderHTML Format
View this article in HTML Format.
HTML Format