(Department of Information Science and Intelligent Systems, Tokushima University, Japan)(2) Stephen Karungaru
(Department of Information Science and Intelligent Systems,Tokushima University, Japan)(3) Kenji Terada
(Department of Information Science and Intelligent Systems,Tokushima University, Japan)(4) Altangerel Ayush
(Department of Information Technology, Mongolian University of Science and Technology, Mongolia)*corresponding author
AbstractDistracted driving is a leading cause of traffic accidents, and often arises from a lack of visual attention on the road. To enhance road safety, monitoring a driver's visual attention is crucial. Appearance-based gaze estimation using deep learning and Convolutional Neural Networks (CNN) has shown promising results, but it faces challenges when applied to different drivers and environments. In this paper, we propose a domain adaptation-based solution for gaze mapping, which aims to accurately estimate a driver's gaze in diverse drivers and new environments. Our method consists of three steps: pre-processing, facial feature extraction, and gaze region classification. We explore two strategies for input feature extraction, one utilizing the full appearance of the driver and environment and the other focusing on the driver's face. Through unsupervised domain adaptation, we align the feature distributions of the source and target domains using a conditional Generative Adversarial Network (GAN). We conduct experiments on the Driver Gaze Mapping (DGM) dataset and the Columbia Cave-DB dataset to evaluate the performance of our method. The results demonstrate that our proposed method reduces the gaze mapping error, achieves better performance on different drivers and camera positions, and outperforms existing methods. We achieved an average Strictly Correct Estimation Rate (SCER) accuracy of 81.38% and 93.53% and Loosely Correct Estimation Rate (LCER) accuracy of 96.69% and 98.9% for the two strategies, respectively, indicating the effectiveness of our approach in adapting to different domains and camera positions. Our study contributes to the advancement of gaze mapping techniques and provides insights for improving driver safety in various driving scenarios.
Keywordsgaze mapping; domain adaptation; visual attention; gaze regions
|
DOIhttps://doi.org/10.26555/ijain.v10i1.1168 |
Article metricsAbstract views : 572 | PDF views : 109 |
Cite |
Full Text Download
|
References
[1] “Pedestrians, cyclists among main road traffic crash victims,” World Health Organization, 2010. [Online]. Available at: https://www.who.int/news/item/11-12-2010-pedestrians-cyclists-among-main-road-traffic-crash-victims.
[2] Y. Dong, Z. Hu, K. Uchimura, and N. Murayama, “Driver Inattention Monitoring System for Intelligent Vehicles: A Review,” IEEE Trans. Intell. Transp. Syst., vol. 12, no. 2, pp. 596–614, Jun. 2011, doi: 10.1109/TITS.2010.2092770.
[3] I. Dua, A. U. Nambi, C. V. Jawahar, and V. N. Padmanabhan, “Evaluation and Visualization of Driver Inattention Rating From Facial Features,” IEEE Trans. Biometrics, Behav. Identity Sci., vol. 2, no. 2, pp. 98–108, Apr. 2020, doi: 10.1109/TBIOM.2019.2962132.
[4] F. Vicente, Z. Huang, X. Xiong, F. De la Torre, W. Zhang, and D. Levi, “Driver Gaze Tracking and Eyes Off the Road Detection System,” IEEE Trans. Intell. Transp. Syst., vol. 16, no. 4, pp. 2014–2027, Aug. 2015, doi: 10.1109/TITS.2015.2396031.
[5] N. Mizuno, A. Yoshizawa, A. Hayashi, and T. Ishikawa, “Detecting driver’s visual attention area by using vehicle-mounted device,” in 2017 IEEE 16th International Conference on Cognitive Informatics & Cognitive Computing (ICCI*CC), Jul. 2017, pp. 346–352, doi: 10.1109/ICCI-CC.2017.8109772.
[6] L. Yang, K. Dong, A. J. Dmitruk, J. Brighton, and Y. Zhao, “A Dual-Cameras-Based Driver Gaze Mapping System With an Application on Non-Driving Activities Monitoring,” IEEE Trans. Intell. Transp. Syst., vol. 21, no. 10, pp. 4318–4327, Oct. 2020, doi: 10.1109/TITS.2019.2939676.
[7] Y. Wang, X. Ding, G. Yuan, and X. Fu, “Dual-Cameras-Based Driver’s Eye Gaze Tracking System with Non-Linear Gaze Point Refinement,” Sensors, vol. 22, no. 6, p. 2326, Mar. 2022, doi: 10.3390/s22062326.
[8] P. Smith, M. Shah, and N. da Vitoria Lobo, “Determining driver visual attention with one camera,” IEEE Trans. Intell. Transp. Syst., vol. 4, no. 4, pp. 205–218, Dec. 2003, doi: 10.1109/TITS.2003.821342.
[9] J. Jo, “Vision-based method for detecting driver drowsiness and distraction in driver monitoring system,” Opt. Eng., vol. 50, no. 12, p. 127202, Dec. 2011, doi: 10.1117/1.3657506.
[10] S. Guasconi, M. Porta, C. Resta, and C. Rottenbacher, “A low-cost implementation of an eye tracking system for driver’s gaze analysis,” in 2017 10th International Conference on Human System Interactions (HSI), Jul. 2017, pp. 264–269, doi: 10.1109/HSI.2017.8005043.
[11] Z. Guo, Q. Zhou, and Z. Liu, “Appearance-based gaze estimation under slight head motion,” Multimed. Tools Appl., vol. 76, no. 2, pp. 2203–2222, Jan. 2017, doi: 10.1007/s11042-015-3182-4.
[12] X. Zhang, Y. Sugano, and A. Bulling, “Evaluation of Appearance-Based Methods and Implications for Gaze-Based Applications,” in Proceedings of the 2019 CHI Conference on Human Factors in Computing Systems, May 2019, pp. 1–13, doi: 10.1145/3290605.3300646.
[13] P. Kellnhofer, A. Recasens, S. Stent, W. Matusik, and A. Torralba, “Gaze360: Physically Unconstrained Gaze Estimation in the Wild,” in 2019 IEEE/CVF International Conference on Computer Vision (ICCV), Oct. 2019, vol. 2019-Octob, pp. 6911–6920, doi: 10.1109/ICCV.2019.00701.
[14] J. Araluce et al., “Gaze Focalization System for Driving Applications Using OpenFace 2.0 Toolkit with NARMAX Algorithm in Accidental Scenarios,” Sensors 2021, Vol. 21, Page 6262, vol. 21, no. 18, p. 6262, Sep. 2021, doi: 10.3390/S21186262.
[15] T. Baltrusaitis, A. Zadeh, Y. C. Lim, and L.-P. Morency, “OpenFace 2.0: Facial Behavior Analysis Toolkit,” in 2018 13th IEEE International Conference on Automatic Face & Gesture Recognition (FG 2018), May 2018, pp. 59–66, doi: 10.1109/FG.2018.00019.
[16] Z. Hu, S. Li, C. Zhang, K. Yi, G. Wang, and D. Manocha, “DGaze: CNN-Based Gaze Prediction in Dynamic Scenes,” IEEE Trans. Vis. Comput. Graph., vol. 26, no. 5, pp. 1902–1911, May 2020, doi: 10.1109/TVCG.2020.2973473.
[17] R. Naqvi, M. Arsalan, G. Batchuluun, H. Yoon, and K. Park, “Deep Learning-Based Gaze Detection System for Automobile Drivers Using a NIR Camera Sensor,” Sensors, vol. 18, no. 2, p. 456, Feb. 2018, doi: 10.3390/s18020456.
[18] L. Fridman, J. Lee, B. Reimer, and T. Victor, “‘Owl’ and ‘Lizard’: patterns of head pose and eye pose in driver gaze classification,” IET Comput. Vis., vol. 10, no. 4, pp. 308–314, Jun. 2016, doi: 10.1049/iet-cvi.2015.0296.
[19] In-Ho Choi, Sung Kyung Hong, and Yong-Guk Kim, “Real-time categorization of driver’s gaze zone using the deep learning techniques,” in 2016 International Conference on Big Data and Smart Computing (BigComp), Jan. 2016, pp. 143–148, doi: 10.1109/BIGCOMP.2016.7425813.
[20] S. J. Lee, J. Jo, H. G. Jung, K. R. Park, and J. Kim, “Real-Time Gaze Estimator Based on Driver’s Head Orientation for Forward Collision Warning System,” IEEE Trans. Intell. Transp. Syst., vol. 12, no. 1, pp. 254–267, Mar. 2011, doi: 10.1109/TITS.2010.2091503.
[21] U. Sonom-Ochir, S. Karungaru, K. Terada, and A. Ayush, “Detection of Driver’S Visual Distraction Using Dual Cameras,” Int. J. Innov. Comput. Inf. Control, vol. 18, no. 5, pp. 1445–1461, 2022. [Online]. Available at: https://repo.lib.tokushima-u.ac.jp/en/118053.
[22] S. Vora, A. Rangesh, and M. M. Trivedi, “Driver Gaze Zone Estimation Using Convolutional Neural Networks: A General Framework and Ablative Analysis,” IEEE Trans. Intell. Veh., vol. 3, no. 3, pp. 254–265, Sep. 2018, doi: 10.1109/TIV.2018.2843120.
[23] U. Sonom-Ochir, S. Karungaru, K. Terada, and A. Ayush, “Appearance-based Driver’s Gaze Mapping Using a Dash Camera,” in 2022 Joint 12th International Conference on Soft Computing and Intelligent Systems and 23rd International Symposium on Advanced Intelligent Systems (SCIS&ISIS), Nov. 2022, pp. 1–5, doi: 10.1109/SCISISIS55246.2022.10001875.
[24] K. Wang, R. Zhao, H. Su, and Q. Ji, “Generalizing Eye Tracking With Bayesian Adversarial Learning,” in 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), Jun. 2019, vol. 2019-June, pp. 11899–11908, doi: 10.1109/CVPR.2019.01218.
[25] Y. Cheng, Y. Bao, and F. Lu, “PureGaze: Purifying Gaze Feature for Generalizable Gaze Estimation,” Proc. AAAI Conf. Artif. Intell., vol. 36, no. 1, pp. 436–443, Jun. 2022, doi: 10.1609/aaai.v36i1.19921.
[26] Y. Liu, R. Liu, H. Wang, and F. Lu, “Generalizing Gaze Estimation with Outlier-guided Collaborative Adaptation,” in 2021 IEEE/CVF International Conference on Computer Vision (ICCV), Oct. 2021, pp. 3815–3824, doi: 10.1109/ICCV48922.2021.00381.
[27] Y. Bao, Y. Liu, H. Wang, and F. Lu, “Generalizing Gaze Estimation with Rotation Consistency,” in 2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), Jun. 2022, vol. 2022-June, pp. 4197–4206, doi: 10.1109/CVPR52688.2022.00417.
[28] Z. Guo, Z. Yuan, C. Zhang, W. Chi, Y. Ling, and S. Zhang, “Domain Adaptation Gaze Estimation by Embedding with Prediction Consistency,” in Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics), vol. 12626 LNCS, Springer Science and Business Media Deutschland GmbH, 2021, pp. 292–307, doi: 10.1007/978-3-030-69541-5_18.
[29] B. A. Smith, Q. Yin, S. K. Feiner, and S. K. Nayar, “Gaze locking,” in Proceedings of the 26th annual ACM symposium on User interface software and technology, Oct. 2013, pp. 271–280, doi: 10.1145/2501988.2501994.
[30] J. Luo, J. Huang, and H. Li, “A case study of conditional deep convolutional generative adversarial networks in machine fault diagnosis,” J. Intell. Manuf., vol. 32, no. 2, pp. 407–425, Feb. 2021, doi: 10.1007/s10845-020-01579-w.
[31] K. He, X. Zhang, S. Ren, and J. Sun, “Deep Residual Learning for Image Recognition,” in 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), Jun. 2016, pp. 770–778, doi: 10.1109/CVPR.2016.90.
[32] S. M. Shah, Z. Sun, K. Zaman, A. Hussain, M. Shoaib, and L. Pei, “A Driver Gaze Estimation Method Based on Deep Learning,” Sensors, vol. 22, no. 10, p. 3959, May 2022, doi: 10.3390/s22103959.
This work is licensed under a Creative Commons Attribution-ShareAlike 4.0 International License.
___________________________________________________________
International Journal of Advances in Intelligent Informatics
ISSN 2442-6571 (print) | 2548-3161 (online)
Organized by UAD and ASCEE Computer Society
Published by Universitas Ahmad Dahlan
W: http://ijain.org
E: info@ijain.org (paper handling issues)
andri.pranolo.id@ieee.org (publication issues)
This work is licensed under a Creative Commons Attribution-ShareAlike 4.0