research-article

Move it there, or not?: the design of voice commands for gaze with speech

Authors:

Monika Elepfandt,

Martin GrundAuthors Info & Claims

Gaze-In '12: Proceedings of the 4th Workshop on Eye Gaze in Intelligent Human Machine Interaction

Article No.: 12, Pages 1 - 3

https://doi.org/10.1145/2401836.2401848

Published: 26 October 2012 Publication History

Abstract

This paper presents an experiment that was conducted to investigate gaze combined with voice commands. There has been very little research about the design of voice commands for this kind of input. It is not known yet if users prefer longer sentences like in natural dialogues or short commands. In the experiment three different voice commands are compared during a simple task in which participants had to drag & drop, rotate, and resize objects. It turned out that the shortness of a voice command -- in terms of number of words -- is more important than it being absolutely natural. Participants preferred the voice command with the fewest words and the fewest syllables. For the voice commands which had the same number of syllables, the users also preferred the one with the fewest words, even though there were no big differences in time and errors.

References

[1]

Elepfandt, M., & Süünderhauf, M. 2011. Multimodal, Touchless Interaction in Spatial Augmented Reality Environments. In Proceedings of the HCII Orlando, 263--271.

Digital Library

[2]

Laqua, S., Bandara, S. U., & Sasse, M. A. 2007. GazeSpace: eye gaze controlled content spaces. In Proceedings of the 21st British HCI Group Annual Conference on People and Computers: HCI...but not as we know it - Volume 2 (BCSHCI '07), Vol. 2. British Computer Society, Swinton, UK, UK, 55--58.

Digital Library

[3]

Kumar, M., Garfinkel, T., Boneh, D., & Winograd, T. 2007. Reducing shoulder-surfing by using gaze-based password entry. In Proceedings of the 3rd symposium on Usable privacy and security (SOUPS '07), Pittsburgh, PA, USA, July 2007. 13--19.

Digital Library

[4]

Nilsson, S., Gustafsson, T., & Carleberg, P. 2009. Hands Free Interaction with Virtual Information in a Real Environment: Eye Gaze as an Interaction Tool in an Augmented Reality System. PsychNology Journal, 7(2), 175--196.

[5]

Latif, H. O., Sherkat, N., & Lotfi, A. 2009. Teleoperation through eye gaze (TeleGaze): a multimodal approach. In Proceedings of the 2009 international conference on Robotics and biomimetics (ROBIO'09). IEEE Press, Piscataway, NJ, USA, 711--716.

Digital Library

[6]

Jacob, R. J. K. 1990. What you look at is what you get: eye movement-based interaction techniques. In Proceedings of the SIGCHI conference on Human factors in computing systems: Empowering people (CHI '90), Jane Carrasco Chew and John Whiteside (Eds.). ACM, New York, NY, USA, 11--18.

Digital Library

[7]

Drewes, H., Hußmann, H, Schmidt, A. 2007. Blickgesten als Fernbedienung. In: Mensch & Computer 2007. Oldenburg-Verlag, Weimar, 79--88.

[8]

Beelders, T. R., & Blignaut, P. J. (2011). The Usability of Speech and Eye Gaze as a Multimodal Interface for a Word Processor, Speech Technologies, Ivo Ipsic (Ed.), InTech.

[9]

Kaur, M., Tremaine, M., Huang, N., Wilder, J., Gacovski, Z., Flippo, F., & Mantravadi, C. S. 2003. Where is "it"? Event Synchronization in Gaze-Speech Input Systems. In Proceedings of the 5th international conference on Multimodal interfaces (ICMI '03). ACM, New York, NY, USA, 151--158.

Digital Library

[10]

Castellina, E., Corno, F., & Pellegrino, P. 2008. Integrated speech and gaze control for realistic desktop environments. In Proceedings of the 2008 symposium on Eye tracking research 38; applications (ETRA '08). ACM, New York, NY, USA, 79--82.

Digital Library

[11]

Miniotas, D., Špakov, O., Tugoy, I., & MacKenzie, I. S. 2006. Speech-augmented eye gaze interaction with small closely spaced targets. In Proceedings of the 2006 symposium on Eye tracking research\& applications (ETRA '06). ACM, New York, NY, USA, 67--72.

Digital Library

[12]

Bolt, R. A. 1980. 'Put-that-there': Voice and gesture at the graphics interface. In: 7th annual conference on Computer graphics and interactive techniques, ACM Press, 262--270.

Digital Library

[13]

Irawati S, Green S, Billinghurst M, Duenser A, & Ko H. 2006. "Move the couch where?": developing an augmented reality multimodal interface. IEEE/ACM International Symposium on Mixed and Augmented Reality, 183--186.

Digital Library

[14]

Cohen, P. R., Oviatt, S. L. 1995. The role of voice input for human-machine communication. In: National Academy of Sciences of the United States of America '92, 9921--9927.

[15]

Oviatt, S., DeAngeli A., & Kuhn K. 1997. Integration and synchronization of input modes during multimodal human-computer interaction. Proceedings of the SIGCHI conference on Human factors in computing systems - CHI '97, 415--422.

Digital Library

Cited By

Cai ZMa YLu F(2024)Robust Dual-Modal Speech Keyword Spotting for XR HeadsetsIEEE Transactions on Visualization and Computer Graphics10.1109/TVCG.2024.337209230:5(2507-2516)Online publication date: 5-Mar-2024
https://dl.acm.org/doi/10.1109/TVCG.2024.3372092
Lin GLi JFazly APavlovic VTruong K(2023)Identifying Multimodal Context Awareness Requirements for Supporting User Interaction with Procedural VideosProceedings of the 2023 CHI Conference on Human Factors in Computing Systems10.1145/3544548.3581006(1-17)Online publication date: 19-Apr-2023
https://dl.acm.org/doi/10.1145/3544548.3581006
Lee JRodriguez SNatarrajan RChen JDeep HKirlik A(2021)What’s This? A Voice and Touch Multimodal Approach for Ambiguity Resolution in Voice AssistantsProceedings of the 2021 International Conference on Multimodal Interaction10.1145/3462244.3479902(512-520)Online publication date: 18-Oct-2021
https://dl.acm.org/doi/10.1145/3462244.3479902
Show More Cited By

Index Terms

Move it there, or not?: the design of voice commands for gaze with speech
1. Human-centered computing
  1. Human computer interaction (HCI)
    1. Interaction devices
      1. Touch screens

Recommendations

Pointing and speech: comparison of various voice commands
NordiCHI '12: Proceedings of the 7th Nordic Conference on Human-Computer Interaction: Making Sense Through Design

This poster presents an experiment that was conducted to investigate pointing combined with voice commands. Three different voice commands are compared with each other during a simple task where subjects had to drag & drop, rotate and resize objects. It ...
Demonstrating Proxemic Cursor Input for Touchless Displays
SUI '23: Proceedings of the 2023 ACM Symposium on Spatial User Interaction

Touchless gesture interfaces often use cursor-based interactions, where widgets are targeted by a movable cursor and activated with a mid-air gesture. Proxemic cursor interactions are a novel alternative that facilitate faster selection without the need ...
Elicitation and evaluation of zoom gestures for touchless interaction with desktop displays
PerDis '19: Proceedings of the 8th ACM International Symposium on Pervasive Displays

Touchless gestural interaction has been widely studied and adopted in many contexts. Furthermore, the growing availability of low-cost enabling devices, such as Kinect or Leap Motion, boosted up the interest in such interaction both for commercial and ...

Comments

Information & Contributors

Information

Published In

cover image ACM Conferences

Gaze-In '12: Proceedings of the 4th Workshop on Eye Gaze in Intelligent Human Machine Interaction

October 2012

88 pages

ISBN:9781450315166

DOI:10.1145/2401836

Copyright © 2012 ACM.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

Sponsors

SIGCHI: ACM Special Interest Group on Computer-Human Interaction

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 26 October 2012

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Research-article

Conference

ICMI '12

Sponsor:

SIGCHI

ICMI '12: INTERNATIONAL CONFERENCE ON MULTIMODAL INTERACTION

October 26, 2012

California, Santa Monica

Acceptance Rates

Overall Acceptance Rate 19 of 21 submissions, 90%

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

9
Total Citations
View Citations
287
Total Downloads

Downloads (Last 12 months)22
Downloads (Last 6 weeks)4

Reflects downloads up to 15 Oct 2024

Other Metrics

View Author Metrics

Citations

Cited By

Cai ZMa YLu F(2024)Robust Dual-Modal Speech Keyword Spotting for XR HeadsetsIEEE Transactions on Visualization and Computer Graphics10.1109/TVCG.2024.337209230:5(2507-2516)Online publication date: 5-Mar-2024
https://dl.acm.org/doi/10.1109/TVCG.2024.3372092
Lin GLi JFazly APavlovic VTruong K(2023)Identifying Multimodal Context Awareness Requirements for Supporting User Interaction with Procedural VideosProceedings of the 2023 CHI Conference on Human Factors in Computing Systems10.1145/3544548.3581006(1-17)Online publication date: 19-Apr-2023
https://dl.acm.org/doi/10.1145/3544548.3581006
Lee JRodriguez SNatarrajan RChen JDeep HKirlik A(2021)What’s This? A Voice and Touch Multimodal Approach for Ambiguity Resolution in Voice AssistantsProceedings of the 2021 International Conference on Multimodal Interaction10.1145/3462244.3479902(512-520)Online publication date: 18-Oct-2021
https://dl.acm.org/doi/10.1145/3462244.3479902
Ataya AKim WElsharkawy AKim S(2020)Gaze-Head Input: Examining Potential Interaction with Immediate Experience Sampling in an Autonomous VehicleApplied Sciences10.3390/app1024901110:24(9011)Online publication date: 17-Dec-2020
https://doi.org/10.3390/app10249011
Zhao DKarikov NMelnichuk EVelichkovsky BShishkin S(2020)Voice as a Mouse Click: Usability and Effectiveness of Simplified Hands-Free Gaze-Voice SelectionApplied Sciences10.3390/app1024879110:24(8791)Online publication date: 9-Dec-2020
https://doi.org/10.3390/app10248791
Paudyal BCreed CFrutos-Pascual MWilliams IWakkary RAndersen KOdom WDesjardins APetersen M(2020)VoiceyeProceedings of the 2020 ACM Designing Interactive Systems Conference10.1145/3357236.3395553(21-33)Online publication date: 3-Jul-2020
https://dl.acm.org/doi/10.1145/3357236.3395553
Wang ZYu HWang HWang ZLu F(2020)Comparing Single-modal and Multimodal Interaction in an Augmented Reality System2020 IEEE International Symposium on Mixed and Augmented Reality Adjunct (ISMAR-Adjunct)10.1109/ISMAR-Adjunct51615.2020.00052(165-166)Online publication date: Nov-2020
https://doi.org/10.1109/ISMAR-Adjunct51615.2020.00052
Minakata KHansen JMacKenzie IBækgaard PRajanna VKrejtz KSharif B(2019)Pointing by gaze, head, and foot in a head-mounted displayProceedings of the 11th ACM Symposium on Eye Tracking Research & Applications10.1145/3317956.3318150(1-9)Online publication date: 25-Jun-2019
https://dl.acm.org/doi/10.1145/3317956.3318150
Qvarfordt P(2017)Gaze-informed multimodal interactionThe Handbook of Multimodal-Multisensor Interfaces10.1145/3015783.3015794(365-402)Online publication date: 24-Apr-2017
https://dl.acm.org/doi/10.1145/3015783.3015794

View Options

Get Access

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Media

Figures

Other

Tables

View Table of Contents