Vision-based typing interface for chording input with keystroke detection using kalman filter
Pages 310 - 317
Abstract
In this paper, we present a vision-based typing interface implemented for chording input. Chording input is an interaction technique which uses combinations of finger touches to enter characters or commands. With this technique, we can obtain 127 combinations by using 7 fingers which is greater than the number of keys on a standard keyboard. But chording input requires identification of the fingers by some means. We describe the prototype system built which performs real-time recognition of fingertips and detection of keystrokes for a bi-manual typing interface using a single web cam. By imposing a few feasible constraints, we were able to use basic image processing for the detection and labeling of the fingertips. We also implemented a novel method for keystroke detection by velocity sensing using Kalman filter. Multi-touch detection is also feasible in the system and a Hindi/Devanagari text entry application has been implemented by encoding Unicode representations of the Devanagari character set into combinations of typed fingertips. The system achieved reliable fingertip recognition and keystroke detection with a mean latency of 80 ms.
References
[1]
Ahmad, F., and Musilek, P. A keystroke and pointer control input interface for wearable computers. In Pervasive Computing and Communications, 2006. PerCom 2006. Fourth Annual IEEE International Conference on (march 2006), 10 pp. --11.
[2]
Au, O. K.-C., and Tai, C.-L. Multitouch finger registration and its applications. In Proceedings of the 22nd Conference of the Computer-Human Interaction Special Interest Group of Australia on Computer-Human Interaction, OZCHI '10, ACM (New York, NY, USA, 2010), 41--48.
[3]
Bowman, D. A., Wingrave, C. A., Campbell, J. M., and Ly, V. Q. Using pinch gloves for both natural and abstract interaction... In Proc. HCI International 2001 (2001), 629--633.
[4]
Buchmann, V., Violich, S., Billinghurst, M., and Cockburn, A. FingARtips: gesture based direct manipulation in augmented reality. In Proceedings of the 2nd international conference on Computer graphics and interactive techniques in Australasia and South East Asia, GRAPHITE '04, ACM (New York, NY, USA, 2004), 212--221.
[5]
Dadgostar, F., and Sarrafzadeh, A. An adaptive real-time skin detector based on hue thresholding: A comparison on two motion tracking methods. Pattern Recogn. Lett. 27, 12 (Sept. 2006), 1342--1352.
[6]
Fukumoto, M., and Tonomura, Y. "Body coupled FingerRing": wireless wearable keyboard. In Proceedings of the SIGCHI conference on Human factors in computing systems, CHI '97, ACM (New York, NY, USA, 1997), 147--154.
[7]
Groves, P. Principles of gnss, inertial, and multi-sensor integrated navigation systems.
[8]
Imura, M., Fujimoto, M., Yasumuro, Y., Manabe, Y., and Chihara, K. Airgrabber: virtual keyboard using miniature infrared camera and tilt sensor. In Proceedings of the 2005 international conference on Augmented tele-existence, ICAT '05, ACM (New York, NY, USA, 2005), 277--277.
[9]
Kato, H., and Kato, T. A marker-less augmented reality based on fast fingertip detection for smart phones. In Consumer Electronics (ICCE), 2011 IEEE International Conference on (jan. 2011), 127--128.
[10]
Kerdvibulvech, C., and Saito, H. Vision-based guitarist fingering tracking using a bayesian classifier and particle filters. In Proceedings of the 2nd Pacific Rim conference on Advances in image and video technology, PSIVT'07, Springer-Verlag (Berlin, Heidelberg, 2007), 625--638.
[11]
Kessler, G. D., Hodges, L. F., and Walker, N. Evaluation of the cyberglove as a whole-hand input device. ACM Trans. Comput.-Hum. Interact. 2, 4 (Dec. 1995), 263--283.
[12]
Klsch, M., and Turk, M. Keyboards without keyboards: A survey of virtual keyboards. Tech. rep., In: Proceedings of Sensing and Input for Media-centric Systems, 2002.
[13]
Lepinski, G. J., Grossman, T., and Fitzmaurice, G. The design and evaluation of multitouch marking menus. In Proceedings of the 28th international conference on Human factors in computing systems, CHI '10, ACM (New York, NY, USA, 2010), 2233--2242.
[14]
Letessier, J., and Bérard, F. Visual tracking of bare fingers for interactive surfaces. In Proceedings of the 17th annual ACM symposium on User interface software and technology, UIST '04, ACM (New York, NY, USA, 2004), 119--122.
[15]
Malik, S., and Laszlo, J. Visual touchpad: a two-handed gestural input device. In Proceedings of the 6th international conference on Multimodal interfaces, ICMI '04, ACM (New York, NY, USA, 2004), 289--296.
[16]
Naimark, L., and Foxlin, E. Circular data matrix fiducial system and robust image processing for a wearable vision-inertial self-tracker. In Proceedings of the 1st International Symposium on Mixed and Augmented Reality, IEEE Computer Society (2002), 27.
[17]
Oka, K., Sato, Y., and Koike, H. Real-time fingertip tracking and gesture recognition. IEEE Comput. Graph. Appl. 22, 6 (Nov. 2002), 64--71.
[18]
Terajima, K., Komuro, T., and Ishikawa, M. Fast finger tracking system for in-air typing interface. In Proceedings of the 27th international conference extended abstracts on Human factors in computing systems, CHI EA '09, ACM (New York, NY, USA, 2009), 3739--3744.
[19]
Vardy, A., Robinson, J., and Cheng, L.-T. The wristcam as input device. In Wearable Computers, 1999. Digest of Papers. The Third International Symposium on (oct. 1999), 199--202.
[20]
von Hardenberg, C., and Bérard, F. Bare-hand human-computer interaction. In Proceedings of the 2001 workshop on Perceptive user interfaces, PUI '01, ACM (New York, NY, USA, 2001), 1--8.
[21]
Welch, G., and Bishop, G. An introduction to the kalman filter. Tech. rep., Chapel Hill, NC, USA, 1995.
[22]
yu Zhang, Q., yi Zhang, M., and qiang Hu, J. Hand gesture contour tracking based on skin color probability and state estimation model. Journal of Multimedia 4, 6 (2009), 349--355.
[23]
Zimmerman, T. G., Lanier, J., Blanchard, C., Bryson, S., and Harvill, Y. A hand gesture interface device. In Proceedings of the SIGCHI/GI conference on Human factors in computing systems and graphics interface, CHI '87, ACM (New York, NY, USA, 1987), 189--192.
Index Terms
- Vision-based typing interface for chording input with keystroke detection using kalman filter
Recommendations
Finger identification and hand gesture recognition techniques for natural user interface
APCHI '13: Proceedings of the 11th Asia Pacific Conference on Computer Human InteractionThe natural user interface using hand gesture have been popular field in Human-Computer-Interaction(HCI). Many research papers have been proposed in this field. They proposed vision-based, glove-based and depth-based approach for hand gesture ...
Multi-scenario gesture recognition using Kinect
CGAMES '12: Proceedings of the 2012 17th International Conference on Computer Games: AI, Animation, Mobile, Interactive Multimedia, Educational & Serious Games (CGAMES)Hand gesture recognition (HGR) is an important research topic because some situations require silent communication with sign languages. Computational HGR systems assist silent communication, and help people learn a sign language. In this article, a ...
Comparing two methods for gesture based short text input using chording
CHI EA '07: CHI '07 Extended Abstracts on Human Factors in Computing SystemsWe report on the design and initial evaluation of two methods for short text input in wearable computing applications using hand gestures. A wireless data glove able to recognize 4 basic gestures is used together with the chording principle. We present ...
Comments
Information & Contributors
Information
Published In
September 2013
420 pages
Copyright © 2013 ACM.
Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than the author(s) must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected].
Sponsors
Publisher
Association for Computing Machinery
New York, NY, United States
Publication History
Published: 24 September 2013
Check for updates
Author Tags
Qualifiers
- Research-article
Conference
APCHI '13
Sponsor:
APCHI '13: 11th Asia Pacific Conference on Computer Human Interaction
September 24 - 27, 2013
Bangalore, India
Upcoming Conference
CHI 2025
- Sponsor:
- sigchi
Contributors
Other Metrics
Bibliometrics & Citations
Bibliometrics
Article Metrics
- 0Total Citations
- 136Total Downloads
- Downloads (Last 12 months)1
- Downloads (Last 6 weeks)0
Reflects downloads up to 01 Mar 2025
Other Metrics
Citations
View Options
Login options
Check if you have access through your login credentials or your institution to get full access on this article.
Sign in