research-article

Open access

Toward a Unified Metadata Schema for Ecological Momentary Assessment with Voice-First Virtual Assistants

Authors:

Kemeberly Charles,

Michael Hogarth,

Emilia FarcasAuthors Info & Claims

CUI '21: Proceedings of the 3rd Conference on Conversational User Interfaces

Article No.: 31, Pages 1 - 6

https://doi.org/10.1145/3469595.3469626

Published: 27 July 2021 Publication History

All formats PDF

Abstract

Ecological momentary assessment (EMA) is used to evaluate subjects’ behaviors and moods in their natural environments, yet collecting real-time and self-report data with EMA is challenging due to user burden. Integrating voice into EMA data collection platforms through today’s intelligent virtual assistants (IVAs) is promising due to hands-free and eye-free nature. However, efficiently managing conversations and EMAs is non-trivial and time consuming due to the ambiguity of the voice input. We approach this problem by rethinking the data modeling of EMA questions and what is needed to deploy them on voice-first user interfaces. We propose a unified metadata schema that models EMA questions and the necessary attributes to effectively and efficiently integrate voice as a new EMA modality. Our schema allows user experience researchers to write simple rules that can be rendered at run-time, instead of having to edit the source code. We showcase an example EMA survey implemented with our schema, which can run on multiple voice-only and voice-first devices. We believe that our work will accelerate the iterative prototyping and design process of real-world voice-based EMA data collection platforms.

References

[1]

Amazon. 2021. Data Type Supported by Amazon DynamoDB. https://docs.aws.amazon.com/amazondynamodb/latest/APIReference/API_Types_Amazon_DynamoDB.html

[2]

Amazon APL. 2021. Understand Alexa Presentation Language (APL). https://developer.amazon.com/en-US/docs/alexa/alexa-presentation-language/understand-apl.html

[3]

Joan Palmiter Bajorek. 2018. Voice First Versus the Multimodal User Interfaces of the Future. https://www.uxmatters.com/mt/archives/2018/10/voice-first-versus-the-multimodal-user-interfaces-of-the-future.php

[4]

Birgit Brüggemeier and Philip Lalone. 2019. WoS - Open Source Wizard of Oz for Speech Systems. In IUI Workshops.

[5]

Chen Chen, Janet G. Johnson, Alice Charles, Kemeberly abd Lee, Ella T. Lifset, Michael Hogarth, Alison A. Moore, Emilia Farcas, and Nadir Weibel. 2021. Understanding Barriers and Design Opportunities to Improve Healthcare and QOL for Older Adults through Voice Assistants. In The 23rd International ACM SIGACCESS Conference on Computers and Accessibility(Virtual Event, USA) (ASSETS ’21). Association for Computing Machinery, Virtual Event, USA. https://doi.org/10.1145/3441852.3471218

Digital Library

[6]

Michael H. Cohen, James P. Giangola, and Jennifer Balogh. 2004. Voice User Interface Design. Addison-Wesley, Boston, MA.

Digital Library

[7]

Kevin Doherty, Andreas Balaskas, and Gavin Doherty. 2020. The Design of Ecological Momentary Assessment Technologies. 32, 3 (2020), 257–278.

[8]

Denzil Ferreira, Jorge Goncalves, Vassilis Kostakos, Louise Barkhuus, and Anind K. Dey. 2014. Contextual Experience Sampling of Mobile Application Micro-Usage. In Proc. MobileHCI ’14. 91–100.

Digital Library

[9]

Jennifer Holland. 2021. The Challenges & Advantages of Conversational User Interfaces in 2021. https://lform.com/blog/post/the-challenges-advantages-of-conversational-user-interfaces-in-2021/

[10]

Stephen Intille, Caitlin Haynes, Dharam Maniar, Aditya Ponnada, and Justin Manjourides. 2016. μEMA: Microinteraction-Based Ecological Momentary Assessment (EMA) Using a Smartwatch. In Proc. UbiComp ’16. 1124–1128.

Digital Library

[11]

Janet Johnson, Khalil Mrini, Michael Hogarth, Alison Moore, Nadpa Nakashole, Nadir Weibel, and Emilia Farcas. 2020. Voice-Based Conversational Agents for Older Adults. In Adjunct Proc. CHI ’20.

[12]

Jaclyn P. Maher, Amanda L. Rebar, and Genevieve F. Dunton. 2018. Ecological Momentary Assessment Is a Feasible and Valid Methodological Tool to Measure Older Adults’ Physical Activity and Sedentary Behavior. Frontiers in Psychology 9 (2018), 1485.

[13]

Khalil Mrini, Chen Chen, Ndapa Nakashole, Nadir Weibel, and Emilia Farcas. 2021. Medical Question Understanding and Answering for Older Adults. (2021). http://voli.ucsd.edu/pdfs/2021_VOLI_SoCal_NLP.pdf

[14]

Kristine S. Nagel, James M. Hudson, and Gregory D. Abowd. 2004. Predictors of Availability in Home Life Context-Mediated Communication. In Proc. CSCW ’04. 497–506.

Digital Library

[15]

Aditya Ponnada, Caitlin Haynes, Dharam Maniar, Justin Manjourides, and Stephen Intille. 2017. Microinteraction Ecological Momentary Assessment Response Rates: Effect of Microinteractions or the Smartwatch?Proc. ACM Interact. Mob. Wearable Ubiquitous Technol. 1, 3, Article 92 (2017).

[16]

Aditya Ponnada, Binod Thapa-Chhetry, Justin Manjourides, and Stephen Intille. 2021. Measuring Criterion Validity of Microinteraction Ecological Momentary Assessment (Micro-EMA): Exploratory Pilot Study With Physical Activity Measurement. JMIR Mhealth Uhealth 9, 3 (10 Mar 2021), e23391.

[17]

QuestionPro. 2021. What is Survey Skip Logic and Branching?https://www.questionpro.com/features/branching.html

[18]

Sergio Sayago, Barbara Barbosa Neves, and Benjamin R Cowan. 2019. Voice Assistants and Older People: Some Open Issues. In Proc. CUI ’19. Article 7.

Digital Library

[19]

Arthur A Stone and Saul Shiffman. 1994. Ecological Momentary Assessment (EMA) in Behavorial Medicine. Annals of Behavioral Medicine(1994).

[20]

Ke Sun, Chen Chen, and Xinyu Zhang. 2020. ”Alexa, Stop Spying on Me!”: Speech Privacy Protection against Voice Assistants. In Proc. SenSys ’20. 298–311.

[21]

SurveyMonkey. 2021. Make Surveys More Engaging When You Do These 5 Things. https://www.surveymonkey.com/curiosity/make-surveys-more-engaging-with-5-things/

[22]

UCSD VOLI. 2021. VOLI: Voice Assistant for Quality of Life and Healthcare Improvement in Aging Populations. http://voli.ucsd.edu

[23]

Wit.ai. 2021. Building Natural Language Experieces. https://wit.ai

[24]

Josh Ziegler. 2018. Communication is Hard - or How Our First Approach to Converastion Management Caused as Many Problems as It Solved.https://medium.com/navigating-the-conversation/communication-is-hard-part-2-1b7529398cc2

Cited By

Lifset ECharles KFarcas EWeibel NHogarth MChen CJohnson JDraper MNguyen AMoore A(2023)Ascertaining Whether an Intelligent Voice Assistant Can Meet Older Adults’ Health-Related Needs in the Context of a Geriatrics 5Ms FrameworkGerontology and Geriatric Medicine10.1177/233372142312011389Online publication date: 30-Sep-2023
https://doi.org/10.1177/23337214231201138
Chen CLifset EHan YRoy AHogarth MMoore AFarcas EWeibel N(2023)Screen or No Screen? Lessons Learnt from a Real-World Deployment Study of Using Voice Assistants With and Without Touchscreen for Older AdultsProceedings of the 25th International ACM SIGACCESS Conference on Computers and Accessibility10.1145/3597638.3608378(1-21)Online publication date: 22-Oct-2023
https://dl.acm.org/doi/10.1145/3597638.3608378
Chen CLifset EHan YRoy AHogarth MMoore AFarcas EWeibel N(2023)How do Older Adults Set Up Voice Assistants? Lessons Learned from a Deployment Experience for Older Adults to Set Up Standalone Voice AssistantsCompanion Publication of the 2023 ACM Designing Interactive Systems Conference10.1145/3563703.3596640(164-168)Online publication date: 10-Jul-2023
https://dl.acm.org/doi/10.1145/3563703.3596640
Show More Cited By

Recommendations

A digital metadata schema repository
TELE-INFO'08: Proceedings of the 7th WSEAS International Conference on Telecommunications and Informatics

The metadata schema of a digital archive describes the structure and attributes of metadata. Analysis and definition of metadata schema for a new digital archive must be carefully performed at the first stage. To ease the task, we implement a metadata ...
Web-based metadata schema repository

The metadata schema of a digital archive describes the structure and attributes of metadata. Analysis and definition of metadata schema for a new digital archive must be carefully carried out and determined at the first stage of development. To ease the ...
Students' Experiences with Ecological Momentary Assessment Tools to Report on Emotional Well-being

Ecological Momentary Assessment (EMA) methods have emerged as an approach that enhances the ecological validity of data collected for the study of human behavior and experience. In particular, EMA methods are used to capture individuals' experiences (...

Comments

Information & Contributors

Information

Published In

cover image ACM Other conferences

CUI '21: Proceedings of the 3rd Conference on Conversational User Interfaces

July 2021

262 pages

ISBN:9781450389983

DOI:10.1145/3469595

Copyright © 2021 Owner/Author.

This work is licensed under a Creative Commons Attribution-NonCommercial International 4.0 License.

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 27 July 2021

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Research-article
Research
Refereed limited

Conference

CUI '21

CUI '21: CUI 2021 - 3rd Conference on Conversational User Interfaces

July 27 - 29, 2021

Bilbao (online), Spain

Acceptance Rates

Overall Acceptance Rate 34 of 100 submissions, 34%

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

4
Total Citations
View Citations
730
Total Downloads

Downloads (Last 12 months)202
Downloads (Last 6 weeks)24

Reflects downloads up to 25 Jan 2025

Other Metrics

View Author Metrics

Citations

Cited By

Lifset ECharles KFarcas EWeibel NHogarth MChen CJohnson JDraper MNguyen AMoore A(2023)Ascertaining Whether an Intelligent Voice Assistant Can Meet Older Adults’ Health-Related Needs in the Context of a Geriatrics 5Ms FrameworkGerontology and Geriatric Medicine10.1177/233372142312011389Online publication date: 30-Sep-2023
https://doi.org/10.1177/23337214231201138
Chen CLifset EHan YRoy AHogarth MMoore AFarcas EWeibel N(2023)Screen or No Screen? Lessons Learnt from a Real-World Deployment Study of Using Voice Assistants With and Without Touchscreen for Older AdultsProceedings of the 25th International ACM SIGACCESS Conference on Computers and Accessibility10.1145/3597638.3608378(1-21)Online publication date: 22-Oct-2023
https://dl.acm.org/doi/10.1145/3597638.3608378
Chen CLifset EHan YRoy AHogarth MMoore AFarcas EWeibel N(2023)How do Older Adults Set Up Voice Assistants? Lessons Learned from a Deployment Experience for Older Adults to Set Up Standalone Voice AssistantsCompanion Publication of the 2023 ACM Designing Interactive Systems Conference10.1145/3563703.3596640(164-168)Online publication date: 10-Jul-2023
https://dl.acm.org/doi/10.1145/3563703.3596640
Chen CJohnson JCharles KLee ALifset EHogarth MMoore AFarcas EWeibel N(2021)Understanding Barriers and Design Opportunities to Improve Healthcare and QOL for Older Adults through Voice AssistantsProceedings of the 23rd International ACM SIGACCESS Conference on Computers and Accessibility10.1145/3441852.3471218(1-16)Online publication date: 17-Oct-2021
https://dl.acm.org/doi/10.1145/3441852.3471218

View Options

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

HTML Format

View this article in HTML Format.

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

Figures

Tables

Media

View Table of Conten