Abstract
This paper presents a method to accurately segment the hand over the face. The similarity of colours and the important variability of the hand shape make it challenging. We propose a method based on the combination of two features: pixel colour and edges orientation. First, a specific skin model is used to find, before occlusion, the face position and the face template. Then, during occlusion the face template is registered using local gradient orientations to track the face position. Colour information is extracted from changes on pixel colours and edges are classified as belonging to the hand or to the face by mapping edges orientation to the face template. Finally by merging both features and by using an hysteresis threshold, which considers connectivity, a robust hand segmentation is reached. Experiments were performed using the Dicta-Sign corpus and showed the versatility of the proposed approach.
Chapter PDF
Similar content being viewed by others
References
Imagawa, I., Matsuo, H., Taniguchi, R., Arita, D., Lu, S., Igi, S.: Recognition of local features for camera-based sign language recognition system. In: Proc. 15th International Conference on Pattern Recognition, vol. 4, pp. 849–853 (2000)
Liang, R., Ouhyoung, M.: A real-time continuous gesture recognition system for sign language. In: Proceedings of the Third IEEE International Conference on Automatic Face and Gesture Recognition, pp. 558–567 (1998)
Habili, N., Lim, C., Moini, A.: Segmentation of the face and hands in sign language video sequences using color and motion cues. IEEE Transactions on Circuits and Systems for Video Technology 14, 1086–1097 (2004)
Wittenburg, P., Brugman, H., Russel, A., Klassmann, A., Sloetjes, H.: Elan: a professional framework for multimodality research. In: Proc. of the 5th International Conference on Language Resources and Evaluation, LREC 2006, pp. 1556–1559 (2006)
Kipp, M.: Anvil - a generic annotation tool for multimodal dialogue. In: Proc. of 7th European Conference on Speech Communication and Technology, Eurospeech, pp. 1367–1370 (2001)
Hanke, T.: ilex - a tool for sign language lexicography and corpus analysis. In: Proc. of 3rd International Conference on Language Resources and Evaluation, LREC 2002, Las Palmas de Gran Canaria, Spain, pp. 923–926 (2002)
Hanke, T., Storz, J.: ilex - a database tool for integrating sign language corpus linguistics and sign language lexicography. In: Proc. of 6th International Conference on Language Resources and Evaluation, LREC 2008, Marrakesh, pp. W25-64–W25-67 (2008)
Braffort, A., Choisier, A., Collet, C., Dalle, P., Gianni, F., Lenseigne, B., Segouat, J.: Toward an annotation software for video of sign language, including image processing tools and signing space modelling. In: Proc. of 4th International Conference on Language Resources and Evaluation, LREC 2004, Lisbon, Portugal, vol. 1, pp. 201–203 (2004)
Collet, C., Gonzalez, M., Milachon, F.: Distributed system architecture for assisted annotation of video corpora. In: International Workshop on the Representation and Processing of Sign Languages: Corpora and Sign Language Technologies, LREC, Valletta, Malte, pp. 49–52 (2010)
Gianni, F., Collet, C., Dalle, P.: Robust Tracking for Processing of Videos of Communication’s Gestures. In: Sales Dias, M., Gibet, S., Wanderley, M.M., Bastos, R. (eds.) GW 2007. LNCS (LNAI), vol. 5085, pp. 93–101. Springer, Heidelberg (2009)
Hamada, Y., Shimada, N., Shirai, Y.: Hand shape estimation using sequence of multi-ocular images based on transition network. In: Proceedings of the International Conference on Vision Interface (2002)
Ramamoorthy, A., Vaswani, N., Chaudhury, S., Banerjee, S.: Recognition of dynamic hand gestures. Pattern Recognition 36, 2069–2081 (2003)
Ahmad, T., Taylor, C., Lanitis, A., Cootes, T.: Tracking and recognising hand gestures, using statistical shape models. Image and Vision Computing 15, 345–352 (1997)
Holden, E., Lee, G., Owens, R.: Australian sign language recognition. Machine Vision and Applications 16, 312–320 (2005)
Tanibata, N., Shimada, N., Shirai, Y.: Extraction of hand features for recognition of sign language words. In: International Conference on Vision Interface, pp. 391–398 (2002)
Smith, P., da Vitoria Lobo, N., Shah, M.: Resolving hand over face occlusion. Image and Vision Computing 25, 1432–1448 (2007)
Matthes, S., Hanke, T., Storz, J., Efthimiou, E., Dimiou, N., Karioris, P., Braffort, A., Choisier, A., Pelhate, J., Safar, E.: Elicitation tasks and materials designed for dicta-sign’s multi-lingual corpus. In: International Workshop on the Representation and Processing of Sign Languages: Corpora and Sign Language Technologies, LREC, Valletta, Malte, pp. 158–163 (2010)
Vassili, V.V., Sazonov, V., Andreeva, A.: A survey on pixel-based skin color detection techniques. In: Proc. Graphicon 2003, pp. 85–92 (2003)
Kovac, J., Peer, P., Solina, F.: Human skin color clustering for face detection. In: EUROCON International Conference on Computer as a Tool, vol. 2, pp. 144–148 (2003)
Author information
Authors and Affiliations
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2012 Springer-Verlag Berlin Heidelberg
About this paper
Cite this paper
Gonzalez, M., Collet, C., Dubot, R. (2012). Head Tracking and Hand Segmentation during Hand over Face Occlusion in Sign Language. In: Kutulakos, K.N. (eds) Trends and Topics in Computer Vision. ECCV 2010. Lecture Notes in Computer Science, vol 6553. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-642-35749-7_18
Download citation
DOI: https://doi.org/10.1007/978-3-642-35749-7_18
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-642-35748-0
Online ISBN: 978-3-642-35749-7
eBook Packages: Computer ScienceComputer Science (R0)