Abstract
Hand gesture recognition has recently emerged as a critical component of the human-computer interaction (HCI) concept, allowing computers to capture and interpret hand gestures. In addition to their use in many medical applications, communication between the hearing impaired, device automation, and robot control, hand gestures are of particular importance as a form of nonverbal communication. So far, hand gesture recognition has taken two approaches and relied on a variety of technologies; the first on sensor technology and the second on computer vision. Given the importance of hand gesture recognition applications and technology development today, the importance of the research lies in shedding light on the latest techniques used in the recognition and interpretation of hand gestures. A survey on the techniques used from 2017–2022 has been presented in this research, with a focus on the computer vision approach. The survey was carried out as follows: the first part dealt with research based on artificial intelligence techniques for hand gesture recognition, and the second part focused on research that used artificial neural networks and deep learning for hand gesture recognition.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Similar content being viewed by others
References
Pisharady, K., Saerbeck, M.: Recent methods and databases in vision-based hand gesture recognition: a review. Comput. Vis. Image Underst. 141, 152–165 (2015)
Yasen, M., Jusoh, S.: A systematic review on hand gesture recognition techniques challenges and applications. Peerj Comput. Sci. 5, e218 (2019)
Camgoz, N.C.: Sign language transformers: joint end-to-end sign language recognition and translation. In: Proceedings of the IEEE/CVF Conference on Computer Vision And Pattern Recognition, pp. 10023–10033 (2020).
Tian, L.: Video big data in smart city: background construction and optimization for surveillance video processing. Future Gener. Comput. Syst. 86, pp. 1371–1382 (2018).
Mei, T., Zhang, C.: Deep learning for intelligent video analysis. In: Proceedings of the 25th ACM International Conference on Multimedia, pp. 1955–1956 (2017).
Sarma, D., Bhuyan, M.K.: Methods, databases and recent advancement of vision-based hand gesture recognition for HCI systems: a review. SN Comput. Sci. 2(6), 1–40 (2021)
Alsaffar, M., et al.: Human-computer interaction using manual hand gestures in real time. Comput. Intell. Neurosci. (2021)
Aumgartl, H., Sauter, D., Schenk, C., Atik, C., Buettner, R.: Vision-based hand gesture recognition for human-computer interaction using MobileNetV2. In: IEEE 45th Annual Computers. Software, and Applications Conference (COMPSAC), pp. 3–15. IEEE, Spain (2018)
Noroozi, F.: Survey on emotional body gesture recognition. IEEE Trans. Affect. Comput. 12(2), 505–523 (2018)
Hisham, B., Hamouda, A.: Supervised learning classifiers for arabic gestures recognition using kinect V2. Appl. Sci. 1(7), 1–21 (2019)
Premaratne, P., Nguyen, Q., Premaratne, M.: Human computer interaction using hand gestures. In: Huang, D.-S., McGinnity, M., Heutte, L., Zhang, X.-P. (eds.) ICIC 2010. CCIS, vol. 93, pp. 381–386. Springer, Heidelberg (2010). https://doi.org/10.1007/978-3-642-14831-6_51
Thórisson, K.R.: Simulated perceptual grouping: an application to human-computer interaction. In: Proceedings of the Sixteenth Annual Conference of the Cognitive Science Society, pp. 876–881. Routledge (2019)
Yun, Y., Ma, D., Yang, M.: Human–computer interaction-based decision support system with applications in data mining. Futur. Gener. Comput. Syst. 114, 285–289 (2021)
Yuanyuan, S.H.I., Yunan, L.I., Xiaolong, F.U., Kaibin, M.I.A.O., Qiguang, M.I.A.O.: Review of dynamic gesture recognition. Virtual Reality Intell. Hardware 3(3), 183–206 (2021)
Oudah, M., Al-Naji, A., Chahl, J.: Hand Gesture recognition based on computer vision: a review of techniques. J. Imaging 6(8), 73 (2020)
Ahmed, M.A.: A review on systems-based sensory gloves for sign language recognition state of the art between 2007 and 2017. Sensors 18(7), 2208 (2018)
Escalera, S., Athitsos, V., Guyon, I.: Challenges in multi-modal gesture recognition. Gesture Recogn., 1–60 (2017)
Pisharady, P.K., Saerbeck, M.: Recent methods and databases in vision-based hand gesture recognition: a review. Comput. Vis. Image Underst. 141(152), 165 (2015)
Wang, P.: RGB-D-based human motion recognition with deep learning: a survey. Comput. Vis. Image Underst. 171(118), 139 (2018)
Chalasani, T., Smolic, A.: Simultaneous segmentation and recognition: towards more accurate ego gesture recognition. In: Proceedings of the IEEE/CVF International Conference on Computer Vision Workshops, p. 0. IEEE (2019)
Hernández, I.: Automatic Irish Sign Language Recognition. University of Dublin, Trinity College (2018)
Shaik, K.B.: Comparative study of skin color detection and segmentation in HSV and YCbCr color space. Procedia Comput. Sci. 57(41), 48 (2017)
Biswas, A.: Finger detection for hand gesture recognition using circular hough transform. In: Advances in Communication. Devices and Networking, pp. 651–660. Springer, Singapore (2018)
Perimal, M., Basah, S.N., Safar, M.J.A., Yazid, H.: Hand-gesture recognition-algorithm based on finger counting. J. Telecommun. Electron Comput. 10(19), 24 (2018)
Huang, H.: Hand gesture recognition with skin detection and deep learning method. J. Phys. Conf. Ser. 1213(2), 022001 (2019)
ljawaryy, A., Malallah, L.: Real-time numerical 0–5 counting based on hand-finger gestures recognition. J. Theor. Appl. Inf. Technol. 95(13), 34–41 (2017)
Prakash, J., Gautam, U.K.: Hand gesture recognition. Int. J. Recent Technol. Eng. (7), 54–59 (2019)
Zhang, F., et al.: Mediapipe hands: on-device real-time hand tracking. arXiv preprint arXiv (2006). 10214 (2020)
Karbasi, M., et al.: A hybrid method using Kinect depth and color data stream for hand blobs segmentation. Sci. Int. 71, 515–519 (2017)
Ma, X., Peng, J.: Kinect sensor-based long-distance hand gesture recognition and fingertip detection with depth information. J. Sens. (2018)
Kim, M.S., Lee, C.H.: Hand gesture recognition for Kinect V2 sensor in the near distance where depth data are not provided. Int. J. Softw. Eng. Appl. (10), 407–418 (2016)
Tekin, B., Bogo, F., Pollefeys, M.: Unified Egocentric recognition of 3d hand-object poses and interactions. In: Proceedings of the IEEE Conference on Computer Vision And Pattern Recognition, pp. 4511–4520. IEEE, USA (2019)
Wan, C., Probst, T., Van Gool, L., Yao, A.: Self-supervised 3D hand pose estimation through training by fitting. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 10853–10862. IEEE, USA (2019)
Ge, L., et al.: 3D hand shape and pose estimation from a single RGB image. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 10833–10842. IEEE, USA (2019)
Taylor, J., et al.: Efficient and precise interactive hand tracking through joint, continuous optimization of pose and correspondences. ACM Trans. Graph. 35, 1–12 (2017)
Tsoli, A., Argyros, A. A.: Joint 3D tracking of a deformable object in interaction with a hand. In: Ferrari, V., Hebert, M., Sminchisescu, C., Weiss, Y. (eds.) Computer Vision – ECCV 2018. LNCS, vol. 11218, pp. 504–520. Springer, Cham (2018). https://doi.org/10.1007/978-3-030-01264-9_30
Chen, Y., Tu, Z., Ge, L., Zhang, D., Chen, R., So-Handnet, Y.J.: Self-organizing network for 3D hand pose estimation with semi-supervised learning. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 6961–6970. IEEE, Korea (2019)
Ge, L., Ren, Z., Yuan, J.: Point-to-point regression PointNet for 3D hand pose estimation. In: Ferrari, V., Hebert, M., Sminchisescu, C., Weiss, Y. (eds.) ECCV 2018. LNCS, vol. 11217, pp. 489–505. Springer, Cham (2018). https://doi.org/10.1007/978-3-030-01261-8_29
Wu, Xiaokun, Finnegan, Daniel, O’Neill, Eamonn, Yang, Yong-Liang.: HandMap: robust hand pose estimation via intermediate dense guidance map supervision. In: Ferrari, V., Hebert, M., Sminchisescu, C., Weiss, Y. (eds.) ECCV 2018. LNCS, vol. 11220, pp. 246–262. Springer, Cham (2018). https://doi.org/10.1007/978-3-030-01270-0_15
Cai, Y., Ge, L., Cai, J., Yuan, J.: Weakly-supervised 3D hand pose estimation from monocular RGB images. In: Ferrari, V., Hebert, M., Sminchisescu, C., Weiss, Y. (eds.) ECCV 2018. LNCS, vol. 11210, pp. 678–694. Springer, Cham (2018). https://doi.org/10.1007/978-3-030-01231-1_41
Ahmed, K.H., Al-Asadi, A.H.: Survey of hand gesture recognition systems. J. Phys. Conf. Ser., 042003(2019)
Abdul, R., Hafız, M.: CMSWVHG-control Ms windows via hand gesture. In: 2017 International Multi-Topic Conference (INMIC), pp. 1–7. IEEE. Pakistan (2017)
Zhou, W.: Real-time implementation of vision-based unmarked static hand gesture recognition with neural networks based on FPGAS. In: IEEE International Conference on Robotics and Biomimetics (ROBIO), pp. 1026–103. IEEE, Macao (2017)
Liu, X.., Li, C.: Tian, L.: Hand gesture recognition based on wavelet invariant moments. In: IEEE International Symposium on Multimedia (ISM), pp. 459–464. IEEE, Taiwan (2017)
Salunke, T.P., Bharkad, S.D.: Power point control using hand gesture recognition based on hog feature extraction and K-NN classification. In: International Conference on Computing Methodologies and Communication (ICCMC), pp. 1151–1155. IEEE, India (2017)
Saha, H.N., Tapadar, S., Ray, S., Chatterjee, S.K., Saha, S.: A machine learning based approach for hand gesture recognition using distinctive feature extraction. In: 8th Annual Computing and Communication Workshop and Conference (CCWC), pp. 91–98. IEEE, USA (2018)
Zhang, Q.: Segmentation of hand gesture based on dark channel prior in projector-camera system. In: International Conference on Communications in China (ICCC), pp. 1–6. IEEE, China (2017)
Žemgulys, J.: Recognition of basketball referee signals from videos using histogram of oriented gradients (Hog) and support vector machine (Svm). Procedia Comput. Sci. 130, 953–960 (2018)
Al-Hammadi, M.: Hand gesture recognition for sign language using 3DCNN. IEEE Access 8, 79491–79509 (2020)
Bulugu, I., Ye, Zhongfu., B.J.: Higher-order local autocorrelation feature extraction methodology for hand gestures recognition. In: 2nd International Conference on Multimedia And Image Processing (ICMIP), pp. 83–87. IEEE, China (2017)
Ansar, H., Jalal, A., Gochoo, M., Kim, K.: Hand gesture recognition based on auto-landmark localization and reweighted genetic algorithm for healthcare muscle activities. Sustainability 13(5), 2961 (2021)
Wu, X.Y.: A hand gesture recognition algorithm based on Dc-CNN. Multimedia Tools Appl. 79(13), 9193–9205 (2020)
Molina, J., Pajuelo, J.A., Martínez, J.M.: Real-time motion-based hand gestures recognition from time-of-flight video. J. Sig. Process Syst. 86, 17–25 (2017)
Xi, C., Chen, J., Zhao, C., Pei, Q., Liu, L.: Real-time hand tracking using Kinect. In: Proceedings of the 2nd International Conference on Digital Signal Processing, pp. 37–42. Tokyo, Japan (2018)
Devineau, G., Moutarde, F., Xi, W., Yang, J.: Deep learning for hand gesture recognition on skeletal data. In: Proceedings of the 13th IEEE International Conference on Automatic Face Gesture Recognition, pp. 106–113. IEEE, China (2018)
Mujahid, A., et al.: Real-time hand gesture recognition based on deep learning YOLOv3 model. Appl. Sci. 11(9), 4164 (2021)
Mahmood, M.R., Abdulazeez, A.M., Orman, Z.: Dynamic hand gesture recognition system for Kurdish sign language using two lines of features. In: International Conference on Advanced Science and Engineering (ICOASE), pp. 42–47. IEEE, Iraq (2018)
Ganokratanaa, T., Pumrin, S.: Hand gesture recognition algorithm for smart cities based on wireless sensor. Int. J. Online Eng. 13(6), 58–75 (2017)
Augustauskas, R,. Lipnickas, A.: Robust hand detection using arm segmentation from depth data and static palm gesture recognition. In: 9th IEEE International Conference on Intelligent Data Acquisition and Advanced Computing Systems: Technology and Applications (IDAACS), pp. 664–667. IEEE, Romania (2017)
Alksasbeh, M.Z., et al.: Smart hand gestures recognition using K-NN based algorithm for video annotation purposes. Indonesia J. Electron. Eng. Comput. Sci. 21(1), 242–252 (2021)
Liao, S., et al.: Multi-object intergroup gesture recognition combined with fusion feature and KNN algorithm. J. Intell. Fuzzy Syst. 38(3), 2725–2735 (2020)
Benalcázar, M.E., Motoche, C., Zea, J.A., Jaramillo, A.G.: Real-time hand gesture recognition using the MYO armband and muscle activity detection. In: Second Ecuador Technical Chapters Meeting ETCM, pp. 1–6. IEEE, Ecuador (2017)
Lian, K.Y., Chiu, C.C., Hong, Y.J., Sung, W.T.: Wearable armband for real time hand gesture recognition. In: International Conference on Systems Man and Cybernetics (SMC), pp. 2992–2995. IEEE, Canada (2017)
Zhu, Y., Jiang, S., Shull, B.: Wrist-worn hand gesture recognition based on barometric pressure sensing. In: 15th International Conference on Wearable and Implantable Body Sensor Networks (BSN), pp. 181–184. IEEE, USA (2018)
Sugiura, Y., Nakamura, F., Kawai, W., Kikuchi, T., Sugimoto, M.: Behind the palm: hand gesture recognition through measuring skin deformation on back of hand by using optical sensors. In: 56th Annual Conference of the Society of Instrument and Control Engineers of Japan (SICE), pp. 1082–1087. IEEE, Japan (2017)
Reshna, S., Jayaraju, M.: Spotting and recognition of hand gesture for Indian sign language recognition system with skin segmentation and SVM. In: International Conference on Wireless Communications, Signal Processing and Networking (WISPNET), pp. 386–390. IEEE, Indian (2017)
Tian, Z.: WICATCH: a Wi-Fi based hand gesture recognition system. IEEE Access (6), 16911–16923 (2018)
Sapienza, S.: On-line event-driven hand gesture recognition based on surface electromyographic signals. In: International Symposium on Circuits and Systems (ISCAS), pp. 1–5. IEEE, Italy (2018)
Shengchang, L., Haoyu, T., Wenshuang, Y.: A hand gesture recognition system based on 24 Ghz radars. In: International Symposium on Antennas and Propagation ISAP, pp. 1–2. IEEE, India (2017)
Zhao, J., Mao, J., Wang, G., Yang, H., Zhao, B.: A miniaturized wearable wireless hand gesture recognition system employing deep-forest classifier. In: Biomedical Circuits and Systems Conference, pp. 1–4. IEEE, Italy (2017)
Rishabh, S., Nutan, V., Prachi, R.: Interactive projector screen with hand detection using gestures. In: International Conference on Automatic Control and Dynamic Optimization Techniques ICACDOT, pp. 574–577. IEEE, India (2016)
Yang, J., Pan, J., Li, J.: sEMG-based continuous hand gesture recognition using Gmm-Hmm and threshold model. In: IEEE International Conference on Robotics and Biomimetics (ROBIO), pp. 1509–1514. IEEE, Macao (2017)
Chen, S.H., Hernawan, A., Lee, Y.S., Wang, J.C.: Hand gesture recognition based on Bayesian sensing hidden Markov models and Bhattacharyya divergence. In: International Conference on Image Processing (ICIP), pp. 3535–3539. IEEE, China (2017)
Molina, J., Pajuelo, J.A., Martínez, J.M.: Real-time motion-based hand gestures recognition from time-of-flight video. J. Sig. Process. Syst. 86(1), 17–25 (2017)
Konstantinidis, D., Dimitropoulos, K., Daras, P.: Sign language recognition based on hand and body skeletal data. In: Conference: the True Vision-Capture, Transmission and Display of 3D Video (3DTV-CON), pp. 1–4. IEEE, Finland (2018)
De Smedt, Q., Wannous, H., Vandeborre, J.P., Guerry, J., Saux, B.: 3D hand gesture recognition using a depth and skeletal dataset: Shrec’17 Track. In: Proceedings of the Workshop on 3D Object Retrieval, pp. 33–38. Eurographics Association, Germany (2017)
Karbasi, M.: A hybrid method using Kinect depth and color data stream for hand blobs segmentation. Sci. Int. 29, 515–519 (2017)
Guo, Y., He, Z., Xie, Q., Chen, K., Ni, W., Zou, E.: Development and application of gesture recognition system for intelligent robot. IOP Conf. Ser. Mater. Sci. Eng. 452(042172) (2018)
Oliveira, M., Chatbri, H., Yarlapati, N., O’Connor, N. E., Sutherland, A.: Hand orientation redundancy filter applied to hand-shapes dataset. In: Proceedings of the 2nd International Conference on Applications of Intelligent Systems, pp. 1–5. Association for Computing Machinery, USA (2019)
De Smedt, Q., Wannous, H., Vandeborre, P.: Heterogeneous hand gesture recognition using 3D dynamic skeletal data. Comput. Visi. Image Underst. 181, 60–72 (2019)
Oliveira, M., Chatbri, H., Little, S., Ferstl, Y., O’Connor, N. E., Sutherland, A.: Irish sign language recognition using principal component analysis and convolutional neural networks. In: International Conference on Digital Image Computing: Techniques and Applications (DICTA), pp. 1–8. IEEE, Australia (2017)
Benitez, G., Olivares, J., Sanchez, G., Yanai, K.: IPN hand: a video dataset and benchmark for real-time continuous hand gesture recognition. In: 25th International Conference on Pattern Recognition (ICPR), pp. 4340–4347. IEEE, Macao (2021)
Malik, J., Elhayek, A., Stricker, D.: Structure-aware 3D hand pose regression from a single depth image. In: Bourdot, P., Cobb, S., Interrante, V., kato, H., Stricker, D. (eds.) EuroVR 2018. LNCS, vol. 11162, pp. 3–17. Springer, Cham (2018). https://doi.org/10.1007/978-3-030-01790-3_1
Chen, J., Meng, J., Wang, X., Yuan, J.: Dynamic graph CNN for event-camera based gesture recognition. In: International Symposium on Circuits and Systems (ISCAS), pp. 1–5. IEEE, India (2020)
Sun, H., Ji, T., Zhang, B., Yang, K., Ji, R.: Research on the hand gesture recognition based on deep learning. In: 12th International Symposium on Antennas, Propagation and EM Theory (ISAPE), pp. 1–4. IEEE, China (2018)
Shin, S., Sung, W.: Dynamic hand gesture recognition for wearable devices with low complexity recurrent neural networks. In: International Symposium on Circuits and Systems (ISCAS), pp. 2274–2277. IEEE, Canada (2020)
Chen, X., Guo, H., Wang, G., Zhang, L.: Motion feature augmented recurrent neural network for skeleton-based dynamic hand gesture recognition. In: International Conference on Image Processing (ICIP), pp. 2881–2885. IEEE, China (2017)
lnujaim, I., Alali, H., Khan, F., Kim, Y.: Hand gesture recognition using input impedance variation of two antennas with transfer learning. IEEE Sens. J., 4129–4135 (2018)
Nguyen, S., Brun. L., Lézoray, O., Bougleux, S.: A neural network based on SPD manifold learning for skeleton-based hand gesture recognition. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 12036–12045. IEEE, USA (2019)
Alnaim, N., Abbod, M., Albar, A.: Hand gesture recognition using convolutional neural network for people who have experienced a stroke. In: Proceedings of the 3rd international symposium on multidisciplinary studies and innovative technologies (ISMSIT), pp.1–6. Ankara, Turkey (2019)
Chung, H., Chung, Y., Tsai, W.: An efficient hand gesture recognition system based on deep CNN. In: Proceedings of the International Conference on Industrial Technology (ICIT), pp. 853–858. IEEE, Australia (2019)
Bao, P., Maqueda, A., Del-Blanco, R., García, N.: Tiny hand gesture recognition without localization via a deep convolutional network. IEEE Trans. Consum. Electron. 63(3), 251–257 (2017)
Li, G., et al.: Hand gesture recognition based on convolution neural network. Cluster Comput. 22(2), 2719–2729 (2019)
Wu, X.Y.: A hand gesture recognition algorithm based on Dc-CNN. Multimedia Tools Appl. 79(13), 1–13 (2019)
Lai, K., Yanushkevich, S.N.: CNN+RNN depth and skeleton based dynamic hand gesture recognition. In: International Conference on Pattern Recognition (ICPR), pp. 3451–3456. IEEE. China (2018)
Choi, W., Ryu, S., Kim, J.: Short-range radar based real-time hand gesture recognition using Lstm encoder. IEEE Access 7 (2019)
Nunez, J.C., Cabido, R., Pantrigo, J.J., Montemayor, A.S., Velez, J.F.: Convolutional neural networks and long short-term memory for skeleton-based human activity and hand gesture recognition. Pattern Recogn. 76, 80–94 (2018)
Zhu, G., Zhang, L., Shen, P., Song, J., Shah, S.A., Bennamoun, M.: Continuous gesture segmentation and recognition using 3DCNN and convolutional LSTM. IEEE Trans. Multimedia 21(4), 1011–1021 (2018)
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2023 The Author(s), under exclusive license to Springer Nature Switzerland AG
About this paper
Cite this paper
Fadel, N., Kareem, E.I.A. (2023). Computer Vision Techniques for Hand Gesture Recognition: Survey. In: Al-Bakry, A.M., et al. New Trends in Information and Communications Technology Applications. NTICT 2022. Communications in Computer and Information Science, vol 1764. Springer, Cham. https://doi.org/10.1007/978-3-031-35442-7_4
Download citation
DOI: https://doi.org/10.1007/978-3-031-35442-7_4
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-031-35441-0
Online ISBN: 978-3-031-35442-7
eBook Packages: Computer ScienceComputer Science (R0)