poster

SAD: A Stress Annotated Dataset for Recognizing Everyday Stressors in SMS-like Conversational Systems

Authors:

Matthew Louis Mauriello,

Thierry Lincoln,

Dan Jurafsky, and

Pablo ParedesAuthors Info & Claims

CHI EA '21: Extended Abstracts of the 2021 CHI Conference on Human Factors in Computing Systems

May 2021

Article No.: 399, Pages 1 - 7

https://doi.org/10.1145/3411763.3451799

Published: 08 May 2021 Publication History

Abstract

There is limited infrastructure for providing stress management services to those in need. To address this problem, chatbots are viewed as a scalable solution. However, one limiting factor is having clear definitions and examples of daily stress on which to build models and methods for routing appropriate advice during conversations. We developed a dataset of 6850 SMS-like sentences that can be used to classify input using a scheme of 9 stressor categories derived from: stress management literature, live conversations from a prototype chatbot system, crowdsourcing, and targeted web scraping from an online repository. In addition to releasing this dataset, we show results that are promising for classification purposes. Our contributions include: (i) a categorization of daily stressors, (ii) a dataset of SMS-like sentences, (iii) an analysis of this dataset that demonstrates its potential efficacy, and (iv) a demonstration of its utility for implementation via a simulation of model response times.

References

[1]

David M Almeida. 2005. Resilience and Vulnerability to Daily Stressors Assessed via Diary Methods. Current Directions in Psychological Science 14, 2 (2005), 64–68. https://doi.org/10.1111/j.0963-7214.2005.00336.x

[2]

Timothy W Bickmore, Ha Trinh, Stefan Olafsson, Teresa K O’Leary, Reza Asadi, Nathaniel M Rickles, and Ricardo Cruz. 2018. Patient and consumer safety risks when using conversational assistants for medical information: an observational study of Siri, Alexa, and Google Assistant. Journal of medical Internet research 20, 9 (2018), e11510.

[3]

Tom B. Brown, Benjamin Mann, Nick Ryder, Melanie Subbiah, Jared Kaplan, Prafulla Dhariwal, Arvind Neelakantan, Pranav Shyam, Girish Sastry, Amanda Askell, Sandhini Agarwal, Ariel Herbert-Voss, Gretchen Krueger, Tom Henighan, Rewon Child, Aditya Ramesh, Daniel M. Ziegler, Jeffrey Wu, Clemens Winter, Christopher Hesse, Mark Chen, Eric Sigler, Mateusz Litwin, Scott Gray, Benjamin Chess, Jack Clark, Christopher Berner, Sam McCandlish, Alec Radford, Ilya Sutskever, and Dario Amodei. 2020. Language Models are Few-Shot Learners. arxiv:2005.14165 [cs.CL]

[4]

Jeffrey Dastin. 2018. ’Kill your foster parents’: Amazon’s Alexa talks murder, sex in AI experiment. Reuters (Dec 2018). https://www.reuters.com/article/us-amazon-com-alexa-insight/kill-your-foster-parents-amazons-alexa-talks-murder-sex-in-ai-experiment-idUSKCN1OK1AJ

[5]

Ryan Daws. 2020. Medical chatbot using OpenAI’s GPT-3 told a fake patient to kill themselves. AI News (Oct 2020). https://artificialintelligence-news.com/2020/10/28/medical-chatbot-openai-gpt3-patient-kill-themselves/

[6]

Jacob Devlin, Ming-Wei Chang, Kenton Lee, and Kristina Toutanova. 2018. Bert: Pre-training of deep bidirectional transformers for language understanding. arXiv preprint arXiv:1810.04805(2018).

[7]

Emily Dinan, Angela Fan, Adina Williams, Jack Urbanek, Douwe Kiela, and Jason Weston. 2020. Queens are powerful too: Mitigating gender bias in dialogue generation.

[8]

Elissa S Epel, Elizabeth H Blackburn, Jue Lin, Firdaus S Dhabhar, Nancy E Adler, Jason D Morrow, and Richard M Cawthon. 2004. Accelerated telomere shortening in response to life stress.Proceedings of the National Academy of Sciences of the United States of America 101, 49 (2004), 17312–17315. https://doi.org/10.1073/pnas.0407162101

[9]

Kathleen Kara Fitzpatrick, Alison Darcy, and Molly Vierhile. 2017. Delivering Cognitive Behavior Therapy to Young Adults With Symptoms of Depression and Anxiety Using a Fully Automated Conversational Agent (Woebot): A Randomized Controlled Trial. JMIR Mental Health 4, 2 (2017). https://doi.org/10.2196/mental.7785

[10]

Luciano Floridi and Massimo Chiriatti. 2020. GPT-3: Its nature, scope, limits, and consequences. Minds and Machines (2020), 1–14.

[11]

Russell Fulmer, Angela Joerin, Breanna Gentile, Lysanne Lakerink, and Michiel Rauws. 2018. Using Psychological Artificial Intelligence (Tess) to Relieve Symptoms of Depression and Anxiety: Randomized Controlled Trial. JMIR Ment Health 5, 4 (dec 2018), e64. https://doi.org/10.2196/mental.9782

[12]

Jennifer Golbeck, Matthew Mauriello, Brooke Auxier, Keval H. Bhanushali, Christopher Bonk, Mohamed Amine Bouzaghrane, Cody Buntain, Riya Chanduka, Paul Cheakalos, Jennine B. Everett, Waleed Falak, Carl Gieringer, Jack Graney, Kelly M. Hoffman, Lindsay Huth, Zhenya Ma, Mayanka Jha, Misbah Khan, Varsha Kori, Elo Lewis, George Mirano, William T. Mohn IV, Sean Mussenden, Tammie M. Nelson, Sean Mcwillie, Akshat Pant, Priya Shetye, Rusha Shrestha, Alexandra Steinheimer, Aditya Subramanian, and Gina Visnansky. 2018. Fake News vs Satire: A Dataset and Analysis. In Proceedings of the 10th ACM Conference on Web Science (Amsterdam, Netherlands) (WebSci ’18). Association for Computing Machinery, New York, NY, USA, 17–21. https://doi.org/10.1145/3201064.3201100

Digital Library

[13]

Kotaro Hara, Vicki Le, and Jon Froehlich. 2013. Combining Crowdsourcing and Google Street View to Identify Street-Level Accessibility Problems. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (Paris, France) (CHI ’13). Association for Computing Machinery, New York, NY, USA, 631–640. https://doi.org/10.1145/2470654.2470744

Digital Library

[14]

Becky Inkster, Shubhankar Sarda, and Vinod Subramanian. 2018. An empathy-driven, conversational artificial intelligence agent (Wysa) for digital mental well-being: real-world data evaluation mixed-methods study. JMIR mHealth and uHealth 6, 11 (2018), e12106.

[15]

Gierad Laput, Walter S. Lasecki, Jason Wiese, Robert Xiao, Jeffrey P. Bigham, and Chris Harrison. 2015. Zensors: Adaptive, Rapidly Deployable, Human-Intelligent Sensor Feeds. In Proceedings of the 33rd Annual ACM Conference on Human Factors in Computing Systems(Seoul, Republic of Korea) (CHI ’15). Association for Computing Machinery, New York, NY, USA, 1935–1944. https://doi.org/10.1145/2702123.2702416

Digital Library

[16]

Haochen Liu, Jamell Dacon, Wenqi Fan, Hui Liu, Zitao Liu, and Jiliang Tang. 2020. Does Gender Matter? Towards Fairness in Dialogue Systems. In Proceedings of the 28th International Conference on Computational Linguistics. 4403–4416.

[17]

B S McEwen. 1998. Stress, adaptation, and disease. Allostasis and allostatic load.Annals of the New York Academy of Sciences 840 (1998), 33–44. https://doi.org/10.1111/j.1749-6632.1998.tb09546.x

[18]

Adam S Miner, Arnold Milstein, Stephen Schueller, Roshini Hegde, Christina Mangurian, and Eleni Linos. 2016. Smartphone-Based Conversational Agents and Responses to Questions About Mental Health, Interpersonal Violence, and Physical Health. JAMA Internal Medicine 176, 5 (2016), 619. https://doi.org/10.1001/jamainternmed.2016.0400 arxiv:15334406

[19]

Peter A Noone. 2017. The Holmes–Rahe Stress Inventory. Occupational Medicine 67, 7 (2017), 581–582.

[20]

Pablo Paredes, Ana Rufino Ferreira, Cory Schillaci, Gene Yoo, Pierre Karashchuk, Dennis Xing, Coye Cheshire, and John Canny. 2017. Inquire: Large-Scale Early Insight Discovery for Qualitative Research. In Proceedings of the 2017 ACM Conference on Computer Supported Cooperative Work and Social Computing(Portland, Oregon, USA) (CSCW ’17). Association for Computing Machinery, New York, NY, USA, 1562–1575. https://doi.org/10.1145/2998181.2998363

Digital Library

[21]

Pablo E Paredes, Nantanick Tantivasadakarn, Grace Hon, Emmanuel Thierry Lincoln, Nikhil Gowda, Marco A Mora-Mendoza, and Matthew Louis Mauriello. 2020. Towards PopBots: A Suite of Conversational Agents for Daily Stress. CH 2020 Workshop on Conversational Agents for Health and Wellbeing (2020). https://www.researchgate.net/publication/344771066_Towards_PopBots_A_Suite_of_Conversational_Agents_for_Daily_Stress/

[22]

F. Pedregosa, G. Varoquaux, A. Gramfort, V. Michel, B. Thirion, O. Grisel, M. Blondel, P. Prettenhofer, R. Weiss, V. Dubourg, J. Vanderplas, A. Passos, D. Cournapeau, M. Brucher, M. Perrot, and E. Duchesnay. 2011. Scikit-learn: Machine Learning in Python. Journal of Machine Learning Research 12 (2011), 2825–2830.

Digital Library

[23]

Jeffrey Pennington, Richard Socher, and Christopher Manning. 2014. GloVe: Global Vectors for Word Representation. In Proceedings of the 2014 Conference on Empirical Methods in Natural Language Processing (EMNLP). Association for Computational Linguistics, Doha, Qatar, 1532–1543. https://doi.org/10.3115/v1/D14-1162

[24]

Jennifer R Piazza, Susan T Charles, Martin J Sliwinski, Jacqueline Mogle, and David M Almeida. 2013. Affective reactivity to daily stressors and long-term risk of reporting a chronic physical health condition. Annals of behavioral medicine : a publication of the Society of Behavioral Medicine (feb 2013). https://doi.org/10.1007/s12160-012-9423-0

[25]

Hannah Rashkin, Eric Michael Smith, Margaret Li, and Y-Lan Boureau. 2019. Towards Empathetic Open-domain Conversation Models: A New Benchmark and Dataset. arXiv preprint arXiv:1811.00207(2019), 5370–5381. https://www.aclweb.org/anthology/P19-1534

[26]

Nils Reimers and Iryna Gurevych. 2019. Sentence-BERT: Sentence Embeddings using Siamese BERT-Networks. In Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing. Association for Computational Linguistics. https://arxiv.org/abs/1908.10084

[27]

Han-Chin Shing, Suraj Nair, Ayah Zirikly, Meir Friedenberg, Hal Daumé III, and Philip Resnik. 2018. Expert, crowdsourced, and machine assessment of suicide risk via online postings. In Proceedings of the Fifth Workshop on Computational Linguistics and Clinical Psychology: From Keyboard to Clinic. 25–36.

[28]

Elsbeth Turcan and Kathleen McKeown. 2019. Dreaddit: A Reddit dataset for stress analysis in social media. arXiv preprint arXiv:1911.00133(2019).

[29]

Joseph Weizenbaum. 1966. ELIZA—a Computer Program for the Study of Natural Language Communication between Man and Machine. Commun. ACM 9, 1 (jan 1966), 36–45. https://doi.org/10.1145/365153.365168

Digital Library

[30]

Ze Yang, Linjun Shou, Ming Gong, Wutao Lin, and Daxin Jiang. 2020. Model Compression with Two-Stage Multi-Teacher Knowledge Distillation for Web Question Answering System. In Proceedings of the 13th International Conference on Web Search and Data Mining (Houston, TX, USA) (WSDM ’20). Association for Computing Machinery, New York, NY, USA, 690–698. https://doi.org/10.1145/3336191.3371792

Digital Library

[31]

Ayah Zirikly, Philip Resnik, Özlem Uzuner, and Kristy Hollingshead. 2019. CLPsych 2019 Shared Task: Predicting the Degree of Suicide Risk in Reddit Posts. In Proceedings of the Sixth Workshop on Computational Linguistics and Clinical Psychology(Minneapolis).

Cited By

Saleem MKim J(2024)Intent aware data augmentation by leveraging generative AI for stress detection in social media textsPeerJ Computer Science10.7717/peerj-cs.215610(e2156)Online publication date: 8-Jul-2024
https://doi.org/10.7717/peerj-cs.2156
Xu XYao BDong YGabriel SYu HHendler JGhassemi MDey AWang D(2024)Mental-LLMProceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies10.1145/36435408:1(1-32)Online publication date: 6-Mar-2024
https://dl.acm.org/doi/10.1145/3643540
Oryngozha NShamoi PIgali A(2024)Detection and Analysis of Stress-Related Posts in Reddit’s Acamedic CommunitiesIEEE Access10.1109/ACCESS.2024.335766212(14932-14948)Online publication date: 2024
https://doi.org/10.1109/ACCESS.2024.3357662
Show More Cited By

Index Terms

SAD: A Stress Annotated Dataset for Recognizing Everyday Stressors in SMS-like Conversational Systems

Index terms have been assigned to the content through auto-classification.

Recommendations

Conversational Agents: Acting on the Wave of Research and Development
CHI EA '19: Extended Abstracts of the 2019 CHI Conference on Human Factors in Computing Systems

In the last five years, work on software that interacts with people via typed or spoken natural language, called chatbots, intelligent assistants, social bots, virtual companions, non-human players, and so on, increased dramatically. Chatbots burst into ...
Read More
IT-supported Stress Management: Mediating between Sensor Data and Individual Appraisal
iWOAR '18: Proceedings of the 5th International Workshop on Sensor-based Activity Recognition and Interaction

People are increasingly exposed to stress these days. There are numerous possible triggers for stress, e.g. multitasking or frequent interruptions. Although stress cannot be considered fundamentally negative, high stress levels can lead to negative ...
Read More
How Personification and Interactivity Influence Stress-Related Disclosures to Conversational Agents
CSCW '18 Companion: Companion of the 2018 ACM Conference on Computer Supported Cooperative Work and Social Computing

In this exploratory study, we examine how personification and interactivity may influence people's disclosures around sensitive topics, such as psychological stressors. Participants (N=441) shared a recent stressful experience with one of three agent ...
Read More

Comments

Information & Contributors

Information

Published In

cover image ACM Conferences

CHI EA '21: Extended Abstracts of the 2021 CHI Conference on Human Factors in Computing Systems

May 2021

2965 pages

ISBN:9781450380959

DOI:10.1145/3411763

Editors:
Yoshifumi Kitamura
Tohoku University, Japan
,
Aaron Quigley
University of New South Wales, Australia
,
Katherine Isbister
University of California Santa Cruz, USA
,
Takeo Igarashi
The University of Tokyo, Japan

Copyright © 2021 ACM.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

Sponsors

SIGCHI: ACM Special Interest Group on Computer-Human Interaction

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 08 May 2021

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Poster
Research
Refereed limited

Conference

CHI '21

Sponsor:

SIGCHI

CHI '21: CHI Conference on Human Factors in Computing Systems

May 8 - 13, 2021

Yokohama, Japan

Acceptance Rates

Overall Acceptance Rate 6,164 of 23,696 submissions, 26%

Upcoming Conference

CHI PLAY '24

Sponsor:
sigchi

The Annual Symposium on Computer-Human Interaction in Play

October 14 - 17, 2024

Tampere , Finland

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

6
Total Citations
View Citations
498
Total Downloads

Downloads (Last 12 months)208
Downloads (Last 6 weeks)34

Other Metrics

View Author Metrics

Citations

Cited By

Saleem MKim J(2024)Intent aware data augmentation by leveraging generative AI for stress detection in social media textsPeerJ Computer Science10.7717/peerj-cs.215610(e2156)Online publication date: 8-Jul-2024
https://doi.org/10.7717/peerj-cs.2156
Xu XYao BDong YGabriel SYu HHendler JGhassemi MDey AWang D(2024)Mental-LLMProceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies10.1145/36435408:1(1-32)Online publication date: 6-Mar-2024
https://dl.acm.org/doi/10.1145/3643540
Oryngozha NShamoi PIgali A(2024)Detection and Analysis of Stress-Related Posts in Reddit’s Acamedic CommunitiesIEEE Access10.1109/ACCESS.2024.335766212(14932-14948)Online publication date: 2024
https://doi.org/10.1109/ACCESS.2024.3357662
Kulkarni DGhosh AGirdhari ALiu SVance LUnruh MSarkar J(2024)Enhancing pre-trained contextual embeddings with triplet loss as an effective fine-tuning method for extracting clinical features from electronic health record derived mental health clinical notesNatural Language Processing Journal10.1016/j.nlp.2023.1000456(100045)Online publication date: Mar-2024
https://doi.org/10.1016/j.nlp.2023.100045
Lamsal RRead MKarunasekera S(2024)CrisisTransformers: Pre-trained language models and sentence encoders for crisis-related social media textsKnowledge-Based Systems10.1016/j.knosys.2024.111916296(111916)Online publication date: Jul-2024
https://doi.org/10.1016/j.knosys.2024.111916
Hamdi EMabrouk MRady SAref M(2023)Data Augmentation Impact on Deep learning Performance for Stress Detection2023 Eleventh International Conference on Intelligent Computing and Information Systems (ICICIS)10.1109/ICICIS58388.2023.10391179(160-165)Online publication date: 21-Nov-2023
https://doi.org/10.1109/ICICIS58388.2023.10391179

View Options

Get Access

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

HTML Format

View this article in HTML Format.

Media

Figures

Other

Tables

View Table of Contents