Abstract
Identification of human subject in different environments plays a significant role in many fields like security and health care. The identification can be performed by using different sensory metrics, often named “biometric”. Traditional biometric technologies are based mainly on fingerprint, retina, voice, and face. In this study, the spontaneous use of skeleton, facial, and vocal metrics is being investigated. For this, a Microsoft Kinect (“Kinect”) system, which was mainly built to estimate human subject kinematic features are deployed. Kinect is affordable, non-wearable, and has the potential to assess joints location, voice, and facial properties simultaneously. A set of skeletal, facial, and vocal features is extracted, and create a “Kinect Signature” that is used to identify different subjects in the scene. The methods were verified by a set of four experiments simulating common realistic scenarios. The experiments indicate that the skeleton, facial, and vocal metrics derived from the Kinect can differentiate between different subjects. The results of this work indicate that while skeletal based metrics are usually more accessible compared to facial and vocal metrics, facial and vocal metrics are more accurate. Aggregation of all data streams improves biometric system performance and their continuity in different environments and times. Such systems can be a base for an affordable, accurate real-time biometric system, that can be deployed at home, and public facilities like hospitals.
Y. Lavi, D. Birnbaum and O. Shabaty—Equal contribution.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Similar content being viewed by others
References
Prabhakar, S., Pankanti, S., Jain, A.K.: Biometric recognition: security and privacy concerns. IEEE Secur. Priv. 1(2), 33–42 (2003)
Mishra, D., Mukhopadhyay, S., Kumari, S., Hurram Khan, M.K., Chaturvedi, A.: Security enhancement of a biometric based authentication scheme for telecare medicine information systems with nonce. J. Med. Syst. 38(5), 41 (2014)
Gorodnichy, D.O.: Evolution and evaluation of biometric systems. In: IEEE Symposium on Computer and Intelligence for Security and Defense Applications, CISDA 2009 (2009)
Jain, A.K., Ross, A., Prabhakar, S.: An introduction to biometric recognition. IEEE Trans. Circ. Syst. Video Technol. 14(1), 4–20 (2004)
Wildes, R.P.: Iris recognition: an emerging biometric technology. Proc. IEEE 85(9), 1348–1363 (1997)
Jain, A.K., Hong, L., Kulkarni, Y.: A multimodal biometric system using fingerprint, face, and speech. In: International Conference on Audio- and Video-Based Biometric Person Authentication (AVBPA), pp. 182–187 (1999)
Iwashita, Y., Uchino, K., Kurazume, R.: Gait-based person identification robust to changes in appearance. Sens. (Switz.) 13(6), 7884–7901 (2013)
Blumrosen, G., Fishman, B., Yovel, Y.: Noncontact wideband sonar for human activity detection and classification. IEEE Sens. J. 14(11), 4043–4054 (2014)
Springer, S., Seligmann, G.Y.: Validity of the kinect for gait assessment: a focused review. Sens. (Switz.) 16(2), 1–13 (2016)
Zhang, Z.: Microsoft kinect sensor and its effect. IEEE Multimed. 19(2), 4–10 (2012)
Sung, J., Ponce, C., Selman, B., Saxena, A.: Human activity detection from RGBD images. In: IEEE International Conference on Robotics and Automation, pp. 842–849 (2011)
Min, R., Kose, N., Dugelay, J.L.: KinectFaceDB: a kinect face database for face recognition. IEEE Trans. Syst. Man Cybern. Syst. 44(11), 1534–1548 (2013)
Mehta, D., Sridhar, S., Sotnychenko, O., Rhodin, H., Shafiei, H., Seidel, H., Xu, W., Casas, D.A.N., Theobalt, C., May, C.V.: VNect: real-time 3D human pose estimation with a single RGB camera. In: Siggraph, pp. 1–13 (2017)
Da Luz, L., Masek, M., Lam, C.P.: Activities of daily living classification using depth features. In: IEEE International Conference on IEEE Region 10, TENCON 2013, pp. 1–4 (2013)
Galna, B., Barry, G., Jackson, D., Mhiripiri, D., Olivier, P., Rochester, L.: Accuracy of the Microsoft Kinect sensor for measuring movement in people with Parkinson’s disease. Gait Post. 39(4), 1062–1068 (2014)
Goswami, G., Vatsa, M., Singh, R.: Face recognition with RGB-D images using kinect. In: Bourlai, T. (ed.) Face Recognition Across the Imaging Spectrum, pp. 281–303. Springer, Cham (2016)
Ouellet, S., Grondin, F., Leconte, F., Michaud, F.: Multimodal biometric identification system for mobile robots combining human metrology to face recognition and speaker identification. In: Proceedings of IEEE International Workshop on Robot and Human Interactive Communication, vol. 2014, pp. 323–328, October 2014
Sinha, A., Chakravarty, K., Bhowmick, B.: Person identification using skeleton information from kinect. In: Sixth International Conference on Advances in Computer-Human Interactions, ACHI 2013, pp. 101–108 (2013)
Blumrosen, G., Miron, Y., Plotnik, M., Intrator, N.: Towards a real time kinect signature based human activity assessment at home. In: 2015 IEEE 12th International Conference on Wearable and Implantable Body Sensor Networks (BSN), pp. 1–6 (2015)
Blumrosen, G., Miron, Y., Intrator, N., Plotnik, M.: A real-time kinect signature-based patient home monitoring system. Sensors 16(11), 1965 (2016)
Blumrosen, G., Uziel, M., Rubinsky, B., Porrat, D.: Noncontact tremor characterization using low-power wideband radar technology. IEEE Trans. Biomed. Eng. 59(c), 674–686 (2012)
Shotton, J., Sharp, T., Kipman, A., Fitzgibbon, A., Finocchio, M., Blake, A., Cook, M., Moore, R.: Real-time human pose recognition in parts from single depth images. Commun. ACM 56(1), 116–124 (2013)
Microsoft Inc. (2015). http://www.microsoft.com/en-us/kinectforwindows/
Daniel, H.J.K.: Joint depth and color camera calibration with distortion correction. IEEE Trans. Med. Imaging 34(10), 2058–2064 (2012)
Microsoft: No Title. Report. https://msdn.microsoft.com/en-us/library/dn785525.aspx
Gkalelis, N., Tefas, A., Pitas, I.: Human identification from human movements. In: 2009 16th IEEE International Conference on Image Processing (ICIP), pp. 2585–2588 (2009)
Sinha, A., Chakravarty, K., Bhowmick, B.: Person identification using skeleton information from kinect. In: Proceedings of International Conference on Advances in Computer-Human Interactions, no. c, pp. 101–108 (2013)
Calderita, L.V., Bandera, J.P., Bustos, P., Skiadopoulos, A.: Model-based reinforcement of kinect depth data for human motion capture applications. Sensors 13(7), 8835–8855 (2013)
Donath, L., Faude, O., Lichtenstein, E., Nüesch, C., Mündermann, A.: Validity and reliability of a portable gait analysis system for measuring spatiotemporal gait characteristics: comparison to an instrumented treadmill. J. Neuroeng. Rehabil. 13(1), 1–9 (2016)
Huang, H.Y., Chang, S.H.: A skeleton-occluded repair method from kinect. In: 2014 International Symposium on Computer, Consumer and Control (IS3C), pp. 264–267 (2014)
Guyon, I., Elisseeff, A.: An introduction to variable and feature selection. J. Mach. Learn. Res. 3(Mar), 1157–1182 (2003)
Breiman, L.E.O.: Random Forests, pp. 5–32 (2001)
Aronowitz, H., Aronowitz, V.: Efficient score normalization for speaker recognition, pp. 4402–4405 (2010)
Garcia-romero, D., Espy-wilson, C.Y.: Analysis of I-vector length normalization in speaker recognition systems
Wang, J., Zhang, J., Honda, K., Wei, J., Dang, J.: Audio - visual speech recognition integrating 3D lip information obtained from the Kinect. Multimed. Syst. 22(3), 315–323 (2016)
Blumrosen, G., Miron, Y., Plotnik, M., Intrator, N.: Towards a real-time kinect signature based human activity assessment at home. In: Body Sensor Network (2015)
Kittler, J., Society, I.C., Hatef, M., Duin, R.P.W., Matas, J.: On combining classifiers. IEEE Trans. Pattern Anal. Mach. Intell. 20(3), 226–239 (1998)
Córdova-Esparza, D.-M., Terven, J.R., Jiménez-Hernández, H., Herrera-Navarro, A.-M.: A multiple camera calibration and point cloud fusion tool for Kinect V2. Sci. Comput. Program. 143, 1–8 (2016)
Terven, J.R., Córdova-Esparza, D.M.: Kin2. A Kinect 2 toolbox for MATLAB. Sci. Comput. Program. 130, 97–106 (2016)
Acknowledgment
We would like to thanks the participating in the test sets. Special thanks to Dr. Hagai Aronowitz from IBM research, for referring the authors to papers in the field of vocal identification and to related python package that assisted in extracting the vocal features, and last to Prof. Alex Bronstein, for his help in supervising the students, asking challenging questions in their final examination, and in contributing from his wise comments to improve the paper quality.
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2019 Springer Nature Switzerland AG
About this paper
Cite this paper
Lavi, Y., Birnbaum, D., Shabaty, O., Blumrosen, G. (2019). Biometric System Based on Kinect Skeletal, Facial and Vocal Features. In: Arai, K., Bhatia, R., Kapoor, S. (eds) Proceedings of the Future Technologies Conference (FTC) 2018. FTC 2018. Advances in Intelligent Systems and Computing, vol 880. Springer, Cham. https://doi.org/10.1007/978-3-030-02686-8_66
Download citation
DOI: https://doi.org/10.1007/978-3-030-02686-8_66
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-030-02685-1
Online ISBN: 978-3-030-02686-8
eBook Packages: Intelligent Technologies and RoboticsIntelligent Technologies and Robotics (R0)