research-article

Exploring the Role of Conversational Cues in Guided Task Support with Virtual Assistants

Authors:

Alexandra Vtyurina,

Adam FourneyAuthors Info & Claims

CHI '18: Proceedings of the 2018 CHI Conference on Human Factors in Computing Systems

Paper No.: 208, Pages 1 - 7

https://doi.org/10.1145/3173574.3173782

Published: 21 April 2018 Publication History

Abstract

Voice-based conversational assistants are growing in popularity on ubiquitous mobile and stationary devices. Cortana, as well as Google Home, Amazon Echo, and others, can provide support for various tasks from managing reminders to booking a hotel. However, with few exceptions, user input is limited to explicit queries or commands. In this work, we explore the role of implicit conversational cues in guided task completion scenarios. In a Wizard of Oz study, we found that, for the task of cooking a recipe, nearly one-quarter of all user-assistant exchanges were initiated from implicit conversational cues rather than from plain questions. Given that these implicit cues occur in such high frequency, we conclude by presenting a set of design implications for the design of guided task experiences in contemporary conversational assistants.

Supplementary Material

MP4 File (pn2304.mp4)

Download
151.56 MB

References

[1]

2017. Alexa, Say What?! Voice-Enabled Speaker Usage to Grow Nearly 130% This Year. https://www.emarketer.com/Article/Alexa-Say-WhatVoice-Enabled-Speaker-Usage-Grow-Nearly-130-ThisYear/1015812. (2017). {Online; retrieved 5-Jan-2018}.

[2]

2017. Alexa Skills Kit. https://developer.amazon.com/alexa-skills-kit. (2017). {Online; retrieved 5-Jan-2018}.

[3]

2017. The Cortana Skills Kit. https://docs.microsoft.com/en-us/cortana/getstarted. (2017). {Online; retrieved 5-Jan-2018}.

[4]

G. Aist, J. Dowding, B. A. Hockey, M. Rayner, J. Hieronymus, D. Bohus, B. Boven, N. Blaylock, E. Campana, S. Early, G. Gorrell, and S. Phan. 2003. Talking Through Procedures: An Intelligent Space Station Procedure Assistant. (2003), 187--190.

Digital Library

[5]

Dan Bohus and Eric Horvitz. 2014. Managing human-robot engagement with forecasts and... um... hesitations. In Proceedings of the 16th international conference on multimodal interaction. ACM, 2--9.

Digital Library

[6]

Dan Bohus and Alexander Rudnicky. 2005. LARRI: A language-based maintenance and repair assistant. Spoken multimodal human-computer dialogue in mobile environments (2005), 203--218.

[7]

Dan Bohus and Alexander I. Rudnicky. 2003. RavenClaw: Dialog management using hierarchical task decomposition and an expectation agenda. In Proceedings of the 8th European Conference on Speech Communication and Technology, EUROSPEECH 2003 INTERSPEECH 2003. ISCA, 203--218.

[8]

John Brooke and others. 1996. SUS -- A quick and dirty usability scale. Usability evaluation in industry 189, 194 (1996), 4--7.

[9]

Herbert H. Clark and Edward F. Schaefer. 1989. Contributing to discourse. Cognitive science 13, 2 (1989), 259--294.

[10]

Fergus I.M. Craik and Robert S. Lockhart. 1972. Levels of processing: A framework for memory research. Journal of verbal learning and verbal behavior 11, 6 (1972), 671--684.

[11]

Florian Eyben, Felix Weninger, Lucas Paletta, and Björn W. Schuller. 2013. The acoustics of eye contact: detecting visual attention from conversational audio cues. In Proceedings of the 6th workshop on Eye gaze in intelligent human machine interaction: gaze in multimodal interaction. ACM, 7--12.

Digital Library

[12]

David Graus, Paul N. Bennett, Ryen W. White, and Eric Horvitz. 2016. Analyzing and Predicting Task Reminders. In Proceedings of the 2016 Conference on User Modeling Adaptation and Personalization. ACM, 7--15.

Digital Library

[13]

Agustín Gravano, Julia Hirschberg, and Štefan Beňuš. 2012. Affirmative cue words in task-oriented dialogue. Computational Linguistics 38, 1 (2012), 1--39.

Digital Library

[14]

Barbara J. Grosz and Candace L. Sidner. 1986. Attention, Intentions, and the Structure of Discourse. Comput. Linguist. 12, 3 (July 1986), 175--204.

Digital Library

[15]

Sandra G. Hart and Lowell E. Staveland. 1988. Development of NASA-TLX (Task Load Index): Results of empirical and theoretical research. Advances in psychology 52 (1988), 139--183.

[16]

Julia Kiseleva, Kyle Williams, Ahmed Hassan Awadallah, Aidan C. Crook, Imed Zitouni, and Tasos Anastasakos. 2016. Predicting user satisfaction with intelligent assistants. In Proceedings of the 39th International ACM SIGIR conference on Research and Development in Information Retrieval. ACM, 45--54.

Digital Library

[17]

Ewa Luger and Abigail Sellen. 2016. Like having a really bad PA: the gulf between user expectation and experience of conversational agents. In Proceedings of the 2016 CHI Conference on Human Factors in Computing Systems. ACM, 5286--5297.

Digital Library

[18]

Filipe Martins, Joana Paulo Pardal, Luís Franqueira, Pedro Arez, and Nuno J. Mamede. 2008. Starting to cook a tutoring dialogue system. In Spoken Language Technology Workshop, 2008. SLT 2008. IEEE. IEEE, 145--148.

[19]

Masahiro Mizukami, Koichiro Yoshino, Graham Neubig, David R. Traum, and Satoshi Nakamura. 2016. Analyzing the Effect of Entrainment on Dialogue Acts. In SIGDIAL Conference. 310--318.

[20]

Kellie Morrissey and Jurek Kirakowski. 2013. "Realness" in Chatbots: Establishing Quantifiable Criteria. In International Conference on Human-Computer Interaction. Springer, 87--96.

Digital Library

[21]

Kevin Murnane. 2017. IFTTT Survey Provides Insight Into What People Do With Amazon's Echo And Google's Home. https://www.forbes.com/sites/kevinmurnane/2017/ 07/12/ifttt-survey-provides-insight-into-what-peopledo-with-voice-controlled-assistants. (2017). {Online; retrieved 12-July-2017}.

[22]

Duyen T. Nguyen and Susan R. Fussell. 2016. Effects of Conversational Involvement Cues on Understanding and Emotions in Instant Messaging Conversations. Journal of Language and Social Psychology 35, 1 (2016), 28--55.

[23]

Sheryl Ong and Aaron Suplizio. 2016. Unpacking the Breakout Success of the Amazon Echo. https://www.experian.com/innovation/thoughtleadership/amazon-echo-consumer-survey.jsp. (2016). {Online; retrieved 20-October-2016}.

[24]

Emma Persky. 2017. Now we're cooking -- the Assistant on Google Home is your secret ingredient. https: //www.blog.google/products/assistant/cooking-with-theassistant-google-home-your-secret-ingredient/. (2017). {Online; retrieved 12-July-2017}.

[25]

Martin Porcheron, Joel E. Fischer, and Sarah Sharples. 2017. "Do Animals Have Accents?": Talking with Agents in Multi-Party Conversation. (2017), 207--219.

Digital Library

[26]

Antoine Raux, Brian Langner, Alan W. Black, and Maxine Eskenazi. 2003. Let's go: Improving spoken dialog systems for the elderly and non-natives. In Proceedings of the 8th European Conference on Speech Communication and Technology, EUROSPEECH 2003 INTERSPEECH 2003. ISCA, 753--756.

[27]

Harvey Sacks, Emanuel A Schegloff, and Gail Jefferson. 1974. A simplest systematics for the organization of turn-taking for conversation. Language (1974), 696--735.

[28]

AllRecipes Staff. 2016. Introducing a Cool New Way to Cook: Allrecipes on Amazon Alexa. http://dish.allrecipes.com/introducing-allrecipes-onamazon-alexa/. (2016). {Online; retrieved 5-Jan-2018}.

[29]

Andreas Stolcke, Klaus Ries, Noah Coccaro, Elizabeth Shriberg, Rebecca Bates, Daniel Jurafsky, Paul Taylor, Rachel Martin, Carol Van Ess-Dykema, and Marie Meteer. 2000. Dialogue act modeling for automatic tagging and recognition of conversational speech. Computational linguistics 26, 3 (2000), 339--373.

Digital Library

[30]

David R. Traum and Elizabeth A. Hinkelman. 1992. Conversation Acts in Task-Oriented Spoken Dialogue. Computational intelligence 8, 3 (1992), 575--599.

[31]

Wayne Xiong, Jasha Droppo, Xuedong Huang, Frank Seide, Mike Seltzer, Andreas Stolcke, Dong Yu, and Geoffrey Zweig. 2016. Achieving human parity in conversational speech recognition. arXiv preprint arXiv:1610.05256 (2016).

[32]

Lei Yu, Karl Moritz Hermann, Phil Blunsom, and Stephen Pulman. 2014. Deep learning for answer sentence selection. arXiv preprint arXiv:1412.1632 (2014).

Cited By

Jaddoh ALoizides FRana OSyed Y(2024)Interacting with Smart Virtual Assistants for Individuals with Dysarthria: A Comparative Study on Usability and User PreferencesApplied Sciences10.3390/app1404140914:4(1409)Online publication date: 8-Feb-2024
https://doi.org/10.3390/app14041409
Chang FSheng LGu Z(2024)Investigating the Integration and the Long-Term Use of Smart Speakers in Older Adults’ Daily Practices: Qualitative StudyJMIR mHealth and uHealth10.2196/4747212(e47472)Online publication date: 12-Feb-2024
https://doi.org/10.2196/47472
Seaborn KSawa YWatanabe M(2024)Coimagining the Future of Voice Assistants with Cultural SensitivityHuman Behavior and Emerging Technologies10.1155/2024/32387372024(1-21)Online publication date: 25-Mar-2024
https://doi.org/10.1155/2024/3238737
Show More Cited By

Index Terms

Exploring the Role of Conversational Cues in Guided Task Support with Virtual Assistants
1. Human-centered computing

Recommendations

Conversational UX Design
CHI EA '17: Proceedings of the 2017 CHI Conference Extended Abstracts on Human Factors in Computing Systems

From Siri to Alexa to Cortana, conversational interfaces are hitting the mainstream and becoming ubiquitous in our daily lives. However, user experiences with such applications remain disappointing. Although it is easy to get a system to produce words, ...
Statistical Dialog Tracking and Management for Task-Oriented Conversational Systems
Hybrid Artificial Intelligent Systems
Abstract
Conversational interfaces offer users a natural way to interact with a range of applications and devices. Human-machine interaction using these systems involves different components that mimic the mechanisms used by humans when using language and ...
Exploring Conversational Search With Humans, Assistants, and Wizards
CHI EA '17: Proceedings of the 2017 CHI Conference Extended Abstracts on Human Factors in Computing Systems

Chatbots and conversational assistants are becoming increasingly popular. However, for information seeking scenarios, these systems still have very limited conversational abilities, and primarily serve as proxies to existing web search engines. In this ...

Comments

Information & Contributors

Information

Published In

cover image ACM Conferences

CHI '18: Proceedings of the 2018 CHI Conference on Human Factors in Computing Systems

April 2018

8489 pages

ISBN:9781450356206

DOI:10.1145/3173574

General Chairs:
Regan Mandryk
University of Saskatchewan, Canada
,
Mark Hancock
University of Waterloo, Canada
,
Program Chairs:
Mark Perry
Brunel University London, UK
,
Anna Cox
University College London, UK

Copyright © 2018 ACM.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than the author(s) must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected].

Sponsors

SIGCHI: ACM Special Interest Group on Computer-Human Interaction

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 21 April 2018

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Research-article

Conference

CHI '18

Sponsor:

SIGCHI

CHI '18: CHI Conference on Human Factors in Computing Systems

April 21 - 26, 2018

Montreal QC, Canada

Acceptance Rates

CHI '18 Paper Acceptance Rate 666 of 2,590 submissions, 26%;

Overall Acceptance Rate 6,199 of 26,314 submissions, 24%

Upcoming Conference

CHI 2025

Sponsor:
sigchi

ACM CHI Conference on Human Factors in Computing Systems

April 26 - May 1, 2025

Yokohama , Japan

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

62
Total Citations
View Citations
1,358
Total Downloads

Downloads (Last 12 months)72
Downloads (Last 6 weeks)4

Reflects downloads up to 24 Dec 2024

Other Metrics

View Author Metrics

Citations

Cited By

Jaddoh ALoizides FRana OSyed Y(2024)Interacting with Smart Virtual Assistants for Individuals with Dysarthria: A Comparative Study on Usability and User PreferencesApplied Sciences10.3390/app1404140914:4(1409)Online publication date: 8-Feb-2024
https://doi.org/10.3390/app14041409
Chang FSheng LGu Z(2024)Investigating the Integration and the Long-Term Use of Smart Speakers in Older Adults’ Daily Practices: Qualitative StudyJMIR mHealth and uHealth10.2196/4747212(e47472)Online publication date: 12-Feb-2024
https://doi.org/10.2196/47472
Seaborn KSawa YWatanabe M(2024)Coimagining the Future of Voice Assistants with Cultural SensitivityHuman Behavior and Emerging Technologies10.1155/2024/32387372024(1-21)Online publication date: 25-Mar-2024
https://doi.org/10.1155/2024/3238737
Arakawa RLehman JGoel M(2024)PrISM-Q&A: Step-Aware Voice Assistant on a Smartwatch Enabled by Multimodal Procedure Tracking and Large Language ModelsProceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies10.1145/36997598:4(1-26)Online publication date: 21-Nov-2024
https://dl.acm.org/doi/10.1145/3699759
Frummet ASpeggiorin AElsweiler DLeuski ADalton J(2024)Cooking with Conversation: Enhancing User Engagement and Learning with a Knowledge-Enhancing AssistantACM Transactions on Information Systems10.1145/364950042:5(1-29)Online publication date: 29-Apr-2024
https://dl.acm.org/doi/10.1145/3649500
Trippas JGallagher LMackenzie JSerra ESpezzano F(2024)Re-evaluating the Command-and-Control Paradigm in Conversational Search InteractionsProceedings of the 33rd ACM International Conference on Information and Knowledge Management10.1145/3627673.3679588(2260-2270)Online publication date: 21-Oct-2024
https://dl.acm.org/doi/10.1145/3627673.3679588
Montoya MQiao HSasikumar PElvitigala DPell SNanayakkara SMueller F(2024)Exploring an Extended Reality Floatation Tank Experience to Reduce the Fear of Being in WaterProceedings of the 2024 CHI Conference on Human Factors in Computing Systems10.1145/3613904.3642285(1-17)Online publication date: 11-May-2024
https://dl.acm.org/doi/10.1145/3613904.3642285
Schuß MPizzoni LRiener A(2024)Human or robot? Exploring different avatar appearances to increase perceived security in shared automated vehiclesJournal on Multimodal User Interfaces10.1007/s12193-024-00436-xOnline publication date: 30-Aug-2024
https://doi.org/10.1007/s12193-024-00436-x
Esau-Held MKrauß VEssing B(2024)Digitale GestaltungVerbraucherinformatik10.1007/978-3-662-68706-2_6(261-300)Online publication date: 25-Mar-2024
https://doi.org/10.1007/978-3-662-68706-2_6
Porto AFurtado M(2024)Framework to Specify Dialogues for Natural Interaction with Conversational Assistants Applied in Prompt EngineeringIntelligent Systems and Applications10.1007/978-3-031-66336-9_17(231-253)Online publication date: 1-Aug-2024
https://doi.org/10.1007/978-3-031-66336-9_17
Show More Cited By

View Options

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Media

Figures

Other

Tables

View Table of Contents