Abstract
Fatigue driving is a primary reason for traffic accidents for commercial truck drivers. Using new technology to detect fatigue in advance is very important to improve road safety. Most of the existing research on fatigue detection is based on the facial key-points of drivers, which are prone to inaccurate detection and thus affect the accuracy of fatigue judgment in complex driving environments. This paper first proposes an end-to-end fatigue detection model based on the CNN-LSTM network structure. The model does not detect facial key-points directly but extracts the driver’s fatigue-related high-level features in full video image frames through a CNN network. Then, the LSTM network is used to calculate the fatigue probability of the driver on the time-series fatigue features of the multi-frame images. To alleviate the impact of complex driving environments, an attention mechanism for CNN is introduced to help the network to focus on the region of interest. To further eliminate background disturbance, this paper proposes a comprehensive fatigue detection method (CFDM) combined with facial recognition. The results of the ablation experiments show that the attention mechanism and face recognition can effectively improve the accuracy of the fatigue detection model, increasing the model’s AUC by 6.7% and 3.8%, respectively. And even when the face is occluded or the driver wears a mask, the comprehensive model we propose can still correctly detect the fatigue state, and the AUC of the model reaches 0.92, while the existing fatigue detection algorithms based on facial key-points will basically fail.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Similar content being viewed by others
References
National Highway Traffic Safety Administration. Drowsy Driving 2015 (Crash-Stats Brief Statistical Summary. Report No. DOT HS 812 446). Washington, DC (2017)
Alkinani, M.H., Khan, W.Z., Arshad, Q.: Detecting human driver inattentive and aggressive driving behavior using deep learning: recent advances, requirements and open challenges. IEEE Access 8, 105008–105030 (2020)
de Naurois, C.J., Bourdin, C., Stratulat, A., et al.: Detection and prediction of driver drowsiness using artificial neural network models. Accid. Anal. Prev. 126, 95–104 (2019)
Dinges, D.F., Grace, R.: Perclos: a valid psychophysiological measure of alertness as assessed by psychomotor vigilance. US Department of Transportation, Federal Highway Administration, Publication Number FHWA-MCRT-98-006 (1998)
Alioua, N., Amine, A., Rziza, M.: Driver’s fatigue detection based on yawning extraction. Int. J. Veh. Technol. (2014)
Jo, J., Lee, S.J., Kim, J., et al.: Vision-based method for detecting driver drowsiness and distraction in driver monitoring system. Opt. Eng. 50(12), 127202 (2011)
Dwivedi, K., Biswaranjan, K., Sethi, A.: Drowsy driver detection using representation learning. In: 2014 IEEE International Advance Computing Conference (IACC), pp. 995–999. IEEE (2014)
Han, H., Li, K., Li, Y.: Monitoring driving in a monotonous environment: classification and recognition of driving fatigue based on long short-term memory network. J. Adv. Transp. (2022)
Zhao, Y., Xie, K., Zou, Z., et al.: Intelligent recognition of fatigue and sleepiness based on InceptionV3-LSTM via multi-feature fusion. IEEE Access 8, 144205–144217 (2020)
Hochreiter, S., Schmidhuber, J.: Long short-term memory. Neural Comput. 9(8), 1735–1780 (1997)
Zhang, K., Zhang, Z., Li, Z., et al.: Joint face detection and alignment using multitask cascaded convolutional networks. IEEE Signal Process. Lett. 23(10), 1499–1503 (2016)
Lan, Y., Zhou, D., Zhang, H., et al.: Development of early warning models. In: Early Warning for Infectious Disease Outbreak, pp. 35–74. Academic Press (2017)
Woo, S., Park, J., Lee, J.Y., et al.: Cbam: convolutional block attention module. In: Proceedings of the European Conference on Computer Vision (ECCV), pp. 3–19 (2018)
Abtahi, S., Omidyeganeh, M., Shirmohammadi, S., et al.: YawDD: a yawning detection dataset. In: Proceedings of the 5th ACM Multimedia Systems Conference, pp. 24–28 (2014)
Selvaraju, R.R., Cogswell, M., Das, A., et al.: Grad-cam: visual explanations from deep networks via gradient-based localization. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 618–626 (2017)
Tibrewal, M., Srivastava, A., Kayalvizhi, R.: A deep learning approach to detect driver drowsiness. Int. J. Eng. Res. Technol. 10, 183–189 (2021)
Chen, L., Xin, G., Liu, Y., et al.: Driver fatigue detection based on facial key points and LSTM. Secur. Commun. Netw. (2021)
Acknowledgements
This work was supported by the National Natural Science Foundation of China (No. 71771174) and Natural Science Foundation of Shanghai (No. 23ZR1465300 & No. 21ZR1465100).
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2023 The Author(s), under exclusive license to Springer Nature Switzerland AG
About this paper
Cite this paper
Gao, Z., Chen, X., Xu, J., Yu, R., Zong, J. (2023). A Comprehensive Vision-Based Model for Commercial Truck Driver Fatigue Detection. In: Tanveer, M., Agarwal, S., Ozawa, S., Ekbal, A., Jatowt, A. (eds) Neural Information Processing. ICONIP 2022. Lecture Notes in Computer Science, vol 13625. Springer, Cham. https://doi.org/10.1007/978-3-031-30111-7_16
Download citation
DOI: https://doi.org/10.1007/978-3-031-30111-7_16
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-031-30110-0
Online ISBN: 978-3-031-30111-7
eBook Packages: Computer ScienceComputer Science (R0)