demonstration

Interact: Tightly-coupling Multimodal Dialog with an Interactive Virtual Assistant

Authors:

Ethan Selfridge,

Michael JohnstonAuthors Info & Claims

ICMI '15: Proceedings of the 2015 ACM on International Conference on Multimodal Interaction

Pages 381 - 382

https://doi.org/10.1145/2818346.2823301

Published: 09 November 2015 Publication History

Get Access

Abstract

Interact is a mobile virtual assistant that uses multimodal dialog to enable an interactive concierge experience over multiple application domains including hotel, restaurants, events, and TV search. Interact demonstrates how multi- modal interaction combined with conversational dialog en- ables a richer and more natural user experience. This demonstration will highlight incremental recognition and under- standing, multimodal speech and gesture input, context track- ing over multiple simultaneous domains, and the use of multimodal interface techniques to enable disambiguation of erors and online personalization.

References

[1]

P. R. Cohen, M. Johnston, D. McGee, S. Oviatt, J. Pittman, I. Smith, L. Chen, and J. Clow. Quickset: Multimodal interaction for distributed applications. In ACM Multimedia, 1997.

Digital Library

Google Scholar

[2]

P. Ehlen and M. Johnston. Multimodal dialogue in mobile local search. In ICMI 2012, 2012.

Digital Library

Google Scholar

[3]

P. Heeman. Combining reinforcement learning with information-state update rules. In ACL, 2007.

Google Scholar

[4]

M. Johnston, J. Chen, P. Ehlen, H. Jung, J. Lieske, A. Reddy, E. Selfridge, S. Stoyanchev, B. Vasilieff, and J. Wilpon. Mva: The multimodal virtual assistant. In SIGdial 2014, 2014.

Google Scholar

[5]

E. Selfridge, I. Arizmendi, P. Heeman, and J. Williams. Stability and accuracy in incremental speech recognition. In SIGdial 2011, 2011.

Digital Library

Google Scholar

[6]

G. Skantze and D. Schlangen. Incremental dialogue processing in a micro-domain. In EACL, 2009.

Digital Library

Google Scholar

Cited By

View all

Filep SKondja AWong CWeber KMoyle BSkavronskaya L(2023)The role of technology in users’ wellbeing: Conceptualizing digital wellbeing in hospitality and future research directionsJournal of Hospitality Marketing & Management10.1080/19368623.2023.229062633:5(583-601)Online publication date: 10-Dec-2023
https://doi.org/10.1080/19368623.2023.2290626
Hutchens MKrishnaswamy NCochran BPustejovsky J(2020)Jarvis: A Multimodal Visualization Tool for Bioinformatic DataHCI International 2020 – Late Breaking Papers: Interaction, Knowledge and Social Media10.1007/978-3-030-60152-2_9(104-116)Online publication date: 27-Sep-2020
https://doi.org/10.1007/978-3-030-60152-2_9
Johnston M(2019)Multimodal integration for interactive conversational systemsThe Handbook of Multimodal-Multisensor Interfaces10.1145/3233795.3233798(21-76)Online publication date: 1-Jul-2019
https://dl.acm.org/doi/10.1145/3233795.3233798
Show More Cited By

Index Terms

Interact: Tightly-coupling Multimodal Dialog with an Interactive Virtual Assistant
1. Computing methodologies
  1. Artificial intelligence
    1. Natural language processing

Recommendations

Annotating non-verbal behaviours in informal interactions
COST'10: Proceedings of the 2010 international conference on Analysis of Verbal and Nonverbal Communication and Enactment

This paper deals with the annotations of non-verbal behaviours in a Danish multimodal corpus of naturally occurring interactions between people who are well-acquainted. The main goal of this work is to provide formally annotated data for describing and ...
When do we interact multimodally?: cognitive load and multimodal communication patterns
ICMI '04: Proceedings of the 6th international conference on Multimodal interfaces

Mobile usage patterns often entail high and fluctuating levels of difficulty as well as dual tasking. One major theme explored in this research is whether a flexible multimodal interface supports users in managing cognitive load. Findings from this ...
Ah, Alright, Okay! Communicating Understanding in Conversational Product Search
CUI '23: Proceedings of the 5th International Conference on Conversational User Interfaces

When talking about products, people often express their needs in vague terms with vocabulary that does not necessarily overlap with product descriptions written by retailers. This poses a problem for chatbots in online shops, as the vagueness and ...

Comments

Information & Contributors

Information

Published In

ICMI '15: Proceedings of the 2015 ACM on International Conference on Multimodal Interaction

November 2015

678 pages

ISBN:9781450339124

DOI:10.1145/2818346

General Chairs:
Zhengyou Zhang
Microsoft Research, USA
,
Phil Cohen
VoiceBox Technologies, USA
,
Program Chairs:
Dan Bohus
Microsoft Research, USA
,
Radu Horaud
INRIA Grenoble Rhone-Alpes, France
,
Helen Meng
Chinese University of Hong Kong, China

Permission to make digital or hard copies of part or all of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for third-party components of this work must be honored. For all other uses, contact the Owner/Author.

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 09 November 2015

Check for updates

Author Tags

Qualifiers

Demonstration

Conference

ICMI '15

Sponsor:

SIGCHI

ICMI '15: INTERNATIONAL CONFERENCE ON MULTIMODAL INTERACTION

November 9 - 13, 2015

Washington, Seattle, USA

Acceptance Rates

ICMI '15 Paper Acceptance Rate 52 of 127 submissions, 41%;

Overall Acceptance Rate 453 of 1,080 submissions, 42%

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

5
Total Citations
View Citations
276
Total Downloads

Downloads (Last 12 months)1
Downloads (Last 6 weeks)0

Reflects downloads up to 27 Feb 2025

Other Metrics

View Author Metrics

Citations

Cited By

View all

Filep SKondja AWong CWeber KMoyle BSkavronskaya L(2023)The role of technology in users’ wellbeing: Conceptualizing digital wellbeing in hospitality and future research directionsJournal of Hospitality Marketing & Management10.1080/19368623.2023.229062633:5(583-601)Online publication date: 10-Dec-2023
https://doi.org/10.1080/19368623.2023.2290626
Hutchens MKrishnaswamy NCochran BPustejovsky J(2020)Jarvis: A Multimodal Visualization Tool for Bioinformatic DataHCI International 2020 – Late Breaking Papers: Interaction, Knowledge and Social Media10.1007/978-3-030-60152-2_9(104-116)Online publication date: 27-Sep-2020
https://doi.org/10.1007/978-3-030-60152-2_9
Johnston M(2019)Multimodal integration for interactive conversational systemsThe Handbook of Multimodal-Multisensor Interfaces10.1145/3233795.3233798(21-76)Online publication date: 1-Jul-2019
https://dl.acm.org/doi/10.1145/3233795.3233798
Fischer MCampagna GXu SLam MBailie LOliver N(2018)BrassauProceedings of the 20th International Conference on Human-Computer Interaction with Mobile Devices and Services10.1145/3229434.3229481(1-12)Online publication date: 3-Sep-2018
https://dl.acm.org/doi/10.1145/3229434.3229481
Kennington CHan TSchlangen DLank EVinciarelli AHoggan ESubramanian SBrewster S(2017)Temporal alignment using the incremental unit frameworkProceedings of the 19th ACM International Conference on Multimodal Interaction10.1145/3136755.3136769(297-301)Online publication date: 3-Nov-2017
https://dl.acm.org/doi/10.1145/3136755.3136769

View Options

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

View options

PDF

View or Download as a PDF file.

PDF

eReader

View online with eReader.

eReader

Abstract

References

Cited By

Index Terms

Recommendations

Annotating non-verbal behaviours in informal interactions

When do we interact multimodally?: cognitive load and multimodal communication patterns

Ah, Alright, Okay! Communicating Understanding in Conversational Product Search

Comments

Information

Published In

Sponsors

Publisher

Publication History

Check for updates

Author Tags

Qualifiers

Conference

Acceptance Rates

Contributors

Other Metrics

Bibliometrics

Article Metrics

Other Metrics

Citations

Cited By

Login options

Full Access

View options

PDF

eReader

Share

Share this Publication link

Share on social media

Affiliations