Article

Multimodal error correction for continuous handwriting recognition in pen-based user interfaces

Authors:

Guozhong DaiAuthors Info & Claims

IUI '06: Proceedings of the 11th international conference on Intelligent user interfaces

Pages 324 - 326

https://doi.org/10.1145/1111449.1111525

Published: 29 January 2006 Publication History

Get Access

Abstract

In this paper, we describe a multimodal error correction mechanism. It allows the user to correct errors in continuous handwriting recognition naturally by simultaneously using pen gesture and speech. A multimodal fusion algorithm is designed to enhance recognition accuracies of handwriting and speech through cross-modal influence. We have performed preliminary evaluation experiments and the results show that this multimodal mechanism can efficiently correct the errors in continuous handwriting recognition.

References

[1]

Coen M. H. Coen. Multimodal integration a biological view. In Proceedings of 17th International Joint Conferenceon Artificial Intelligence, pages 1417--1424. Seattle, Washington, USA, August 4-10, 2001.

Digital Library

Google Scholar

[2]

C. Frankish, D. Jones and K. Hapeshi. Decline in accuracy of automatic speech recognition as function of time on task: fatigue or voice drift. International Journal of Man-Machine Studies, 36(6):797--816, 1992.

Digital Library

Google Scholar

[3]

C. Halverson, D.B. Horn, C. Karat and J. Karat. The beauty of errors: patterns of error correction in desktop speech systems. In Proceedings of INTERACT '99, pages 133--140, IOS Press. Edinburgh, September 1999.

Google Scholar

[4]

M. Johnston and S. Bangalore. Finite-State multimodal parsing and understanding. In Proceedings of the 18th conference on Computational linguistics, vol.1, pages 369--375, Association for Computational Linguistics Morristown. Saarbruecken, Germany, 2000.

Digital Library

Google Scholar

[5]

J. Mankoff and G. Abowd. Error correction techniques for handwriting, speech, and other ambiguous or error prone systems. GVU Technical Report Number: GIT-GVU-99-18, 1999.

Google Scholar

[6]

J. Mankoff, G. Abowd and S. Hudson. OOPS: a toolkit supporting mediation techniques for resolving ambiguity in recognition-based interfaces. Computers and Graphics, 24: 819--834, 2000.

Crossref

Google Scholar

[7]

S. Oviatt, A. Deangell and K.Kuhn. Integration and synchronization of input modes during multimodal human-computer interaction. In Proceedings of the ACM Conference on Human Factors in Computing Systems, CHI'97, (S. Pemberton, Ed.) pages 415--422,ACM Press. Atlanta, GA, March 22-27, 1997.

Digital Library

Google Scholar

[8]

S. Oviatt. Taming recognition errors with a multimodal interface. Communication of the ACM, 43(9): 45--51, 2000.

Digital Library

Google Scholar

[9]

J.C. Read, S. MacFarlane and C.Casey. Oops! Silly me! Errors in a handwriting recognition based text entry interface for children. In Proceedings of the second Nordic conference on Human-computer interaction, pages 35--40, ACM Press. Aarhus, Denmark, 2002.

Digital Library

Google Scholar

[10]

B. Suhm, B. Myers and A. Waibel. Multimodal error correction for speech user interfaces. ACM Transactions on Computer-Human Interaction, 8(1):60--98, 2001.

Digital Library

Google Scholar

[11]

J. Sturm and L. Boves. Effective error recovery strategies for multimodal form-filling applications. Speech Communication, 45:289--303, 2005.

Crossref

Google Scholar

[12]

M. Zajicek and J. Hewitt. An investigation into the use of error recovery dialogues in a user interface management system for speech recognition. In Proceedings of 3rd IFIP International Conference on Human-Computer Interaction, IFIP INTERACT'90, pages 755--760, North-Holland. Cambridge, UK, August 27-31, 1990.

Digital Library

Google Scholar

Cited By

View all

Chen XLu YLiu J(2020)Stroke-speech: A Multi-channel Input Method for Chinese CharactersProceedings of the Eighth International Workshop of Chinese CHI10.1145/3403676.3403687(69-72)Online publication date: 26-Apr-2020
https://dl.acm.org/doi/10.1145/3403676.3403687
Leiva LAlabau VRomero VToselli AVidal E(2014)Context-Aware Gestures for Mixed-Initiative Text Editing UIsInteracting with Computers10.1093/iwc/iwu01927:6(675-696)Online publication date: 11-Jun-2014
https://doi.org/10.1093/iwc/iwu019
Stedman RTerry MLank EEpps JChen FOviatt SMase KSears AJokinen KSchuller B(2013)Aiding human discovery of handwriting recognition errorsProceedings of the 15th ACM on International conference on multimodal interaction10.1145/2522848.2522882(295-302)Online publication date: 9-Dec-2013
https://dl.acm.org/doi/10.1145/2522848.2522882
Show More Cited By

Index Terms

Multimodal error correction for continuous handwriting recognition in pen-based user interfaces

Recommendations

Crossmodal error correction of continuous handwriting recognition by speech
IUI '07: Proceedings of the 12th international conference on Intelligent user interfaces

In recognition-based user interface, users' satisfaction is determined not only by recognition accuracy but also by effort to correct recognition errors. In this paper, we introduce a crossmodal error correction technique, which allows users to correct ...
Multimodal error correction for speech user interfaces

Although commercial dictation systems and speech-enabled telephone voice user interfaces have become readily available, speech recognition errors remain a serious problem in the design and implementation of speech user interfaces. Previous work ...
Word graph based speech rcognition error correction by handwriting input
ICMI '06: Proceedings of the 8th international conference on Multimodal interfaces

We propose a convenient handwriting user interface for correcting speech recognition errors efficiently. Via the proposed hand-marked correction on the displayed recognition result, substitution, deletion and insertion errors can be corrected ...

Comments

Information & Contributors

Information

Published In

IUI '06: Proceedings of the 11th international conference on Intelligent user interfaces

January 2006

392 pages

ISBN:1595932879

DOI:10.1145/1111449

General Chairs:
Ernest Edmonds
University of Technology Sydney, Australia
,
Doug Riecken
IBM Watson Research, USA
,
Program Chairs:
Cécile L. Paris
CSIRO ICT Centre, Australia
,
Candace L. Sidner
MERL, USA

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 29 January 2006

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Article

Conference

IUI06

Sponsor:

IUI06: 11th International Conference on Intelligent User Interfaces

January 29 - February 1, 2006

Sydney, Australia

Acceptance Rates

Overall Acceptance Rate 746 of 2,811 submissions, 27%

Upcoming Conference

IUI '25

Sponsor:
sigai
sigai

30th International Conference on Intelligent User Interfaces

March 24 - 27, 2025

Cagliari , Italy

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

8
Total Citations
View Citations
536
Total Downloads

Downloads (Last 12 months)6
Downloads (Last 6 weeks)0

Reflects downloads up to 15 Oct 2024

Other Metrics

View Author Metrics

Citations

Cited By

View all

Chen XLu YLiu J(2020)Stroke-speech: A Multi-channel Input Method for Chinese CharactersProceedings of the Eighth International Workshop of Chinese CHI10.1145/3403676.3403687(69-72)Online publication date: 26-Apr-2020
https://dl.acm.org/doi/10.1145/3403676.3403687
Leiva LAlabau VRomero VToselli AVidal E(2014)Context-Aware Gestures for Mixed-Initiative Text Editing UIsInteracting with Computers10.1093/iwc/iwu01927:6(675-696)Online publication date: 11-Jun-2014
https://doi.org/10.1093/iwc/iwu019
Stedman RTerry MLank EEpps JChen FOviatt SMase KSears AJokinen KSchuller B(2013)Aiding human discovery of handwriting recognition errorsProceedings of the 15th ACM on International conference on multimodal interaction10.1145/2522848.2522882(295-302)Online publication date: 9-Dec-2013
https://dl.acm.org/doi/10.1145/2522848.2522882
Jiang YTian FZhang XDai GWang H(2011)Understanding, Manipulating and Searching Hand-Drawn Concept MapsACM Transactions on Intelligent Systems and Technology10.1145/2036264.20362753:1(1-21)Online publication date: 1-Oct-2011
https://dl.acm.org/doi/10.1145/2036264.2036275
Zhang XFu YZhang K(2009)Adaptive Correction of Errors from Recognized Chinese Ink Texts Based on ContextProceedings of the 2009 International Conference on Information Technology and Computer Science - Volume 0110.1109/ITCS.2009.71(314-320)Online publication date: 25-Jul-2009
https://dl.acm.org/doi/10.1109/ITCS.2009.71
Ren XZhou X(2009)The Optimal Size of Handwriting Character Input Boxes on PDAsInternational Journal of Human-Computer Interaction10.1080/1044731090302551125:8(762-784)Online publication date: 3-Dec-2009
https://doi.org/10.1080/10447310903025511
Jiang YWang XTian FAo XDai GWang HStaab S(2008)Multimodal Chinese text entry with speech and keypad on mobile devicesProceedings of the 13th international conference on Intelligent user interfaces10.1145/1378773.1378825(341-344)Online publication date: 13-Jan-2008
https://dl.acm.org/doi/10.1145/1378773.1378825
Ao XWang XTian FDai GWang HChin DZhou MLau TPuerta A(2007)Crossmodal error correction of continuous handwriting recognition by speechProceedings of the 12th international conference on Intelligent user interfaces10.1145/1216295.1216339(243-250)Online publication date: 28-Jan-2007
https://dl.acm.org/doi/10.1145/1216295.1216339

View Options

Get Access

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Cited By

Index Terms

Recommendations

Crossmodal error correction of continuous handwriting recognition by speech

Multimodal error correction for speech user interfaces

Word graph based speech rcognition error correction by handwriting input