research-article

Unveiling Social Anxiety: Analyzing Acoustic and Linguistic Traits in Impromptu Speech within a Controlled Study

Authors:

Nilesh Kumar Sahu,

Haroon R LoneAuthors Info & Claims

ACM Journal on Computing and Sustainable Societies, Volume 2, Issue 2

Article No.: 24, Pages 1 - 19

https://doi.org/10.1145/3657245

Published: 20 June 2024 Publication History

Abstract

Early detection and treatment of Social Anxiety Disorder (SAD) is crucial. However, current diagnostic methods have several drawbacks, including being time consuming for clinical interviews, susceptible to emotional bias for self-reports, and inconclusive for physiological measures. Our research focuses on a digital approach using acoustic and linguistic features extracted from participants’ “speech” for diagnosing SAD. Our methodology involves identifying correlations between extracted features and SAD severity, selecting the effective features, and comparing classical machine learning and deep learning methods for predicting SAD. Our results demonstrate that both acoustic and linguistic features outperform deep learning approaches when considered individually. Logistic Regression proves effective for acoustic features, whereas Random Forest excels with linguistic features, achieving the highest accuracy of 85.71%. Our findings pave the way for non-intrusive SAD diagnosing that can be used conveniently anywhere, facilitating early detection.

References

[1]

Psychology Tools. 2022. Social Phobia Inventory (SPIN). Retrieved August 25, 2023 from https://psychology-tools.com/test/spin

[2]

Philippe Albouy, Samuel A. Mehr, Roxane S. Hoyer, Jeremie Ginzburg, and Robert J. Zatorre. 2023. Spectro-temporal acoustical markers differentiate speech from song across cultures. bioRxiv. Published Online, January 29, 2023.

[3]

Or Alon-Ronen, Yosi Shrem, Yossi Keshet, and Eva Gilboa-Schechtman. 2022. The vocal signature of social anxiety: Exploration using hypothesis-testing and machine-learning approaches. arXiv preprint arXiv:2207.08534 (2022).

[4]

American Psychological Association. 2024. Anxiety. Retrieved September 3, 2023 from https://www.apa.org/topics/anxiety/

[5]

Jeffrey A. Brooks, Panagiotis Tzirakis, Alice Baird, Lauren Kim, Michael Opara, Xia Fang, Dacher Keltner, Maria Monroy, Rebecca Corona, Jacob Metrick, and Alan S. Cowen. 2023. Deep learning reveals what vocal bursts express in different cultures. Nature Human Behaviour 7, 2 (2023), 240–250.

[6]

William Clapp, Charlotte Vaughn, and Meghan Sumner. 2023. The episodic encoding of talker voice attributes across diverse voices. Journal of Memory and Language 128 (2023), 104376.

[7]

Kathryn M. Connor, Jonathan R. T. Davidson, L. Erik Churchill, Andrew Sherwood, Richard H. Weisler, and Edna Foa. 2000. Psychometric properties of the Social Phobia Inventory (SPIN): New self-rating scale. British Journal of Psychiatry 176, 4 (2000), 379–386.

[8]

Daniel Di Matteo, Kathryn Fotinos, Sachinthya Lokuge, Geneva Mason, Tia Sternat, Martin A. Katzman, and Jonathan Rose. 2021. Automated screening for social anxiety, generalized anxiety, and depression from objective smartphone-collected data: Cross-sectional study. Journal of Medical Internet Research 23, 8 (2021), e28918.

[9]

Daniel Di Matteo, Wendy Wang, Kathryn Fotinos, Sachinthya Lokuge, Julia Yu, Tia Sternat, Martin A. Katzman, and Jonathan Rose. 2021. Smartphone-detected ambient speech and self-reported measures of anxiety and depression: Exploratory observational study. JMIR Formative Research 5, 1 (2021), e22723.

[10]

Zhengyin Du, Weixin Li, Di Huang, and Yunhong Wang. 2018. Bipolar disorder recognition via multi-scale discriminative audio temporal representation. In Proceedings of the 2018 Audio/Visual Emotion Challenge and Workshop. 23–30.

Digital Library

[11]

Ayman Elfaki, Ani Liza Asnawi, Ahmad Zamani Jusoh, Ahmad Fadzil Ismail, Siti Noorjannah Ibrahim, Nor Fadhillah Mohamed Azmin, and Nik Nur Wahidah Binti Nik Hashim. 2021. Using the short-time Fourier transform and resnet to diagnose depression from speech data. In Proceedings of the 2021 IEEE International Conference on Computing (ICOCO ’21). IEEE, 372–376.

[12]

Lior Galili, Ofer Amir, and Eva Gilboa-Schechtman. 2013. Acoustic properties of dominance and request utterances in social anxiety. Journal of Social and Clinical Psychology 32, 6 (2013), 651–673.

[13]

Maria J. Garcia-Rubio, Laura Espin, Vanesa Hidalgo, Alicia Salvador, and Jesus Gomez-Amor. 2017. Autonomic markers associated with generalized social phobia symptoms: Heart rate variability and salivary alpha-amylase. Stress 20, 1 (2017), 61–68.

[14]

Shikha Gupta, Jafreezal Jaafar, W. F. Wan Ahmad, and Arpit Bansal. 2013. Feature extraction using MFCC. Signal & Image Processing: An International Journal 4, 4 (2013), 101–108.

[15]

A. Harrewijn, M. J. W. Van der Molen, B. Verkuil, S. W. Sweijen, J. J. Houwing-Duistermaat, and P. M. Westenberg. 2018. Heart rate variability as candidate endophenotype of social anxiety: A two-generation family study. Journal of Affective Disorders 237 (2018), 47–55.

[16]

Mohammad Hossin and Md Nasir Sulaiman. 2015. A review on evaluation metrics for data classification evaluations. International Journal of Data Mining & Knowledge Management Process 5, 2 (2015), 1.

[17]

Philip Jefferies and Michael Ungar. 2020. Social anxiety in young people: A prevalence study in seven countries. PLoS One 15, 9 (2020), e0239133.

[18]

Christin Kirchhübel, Georgina Brown, and Paul Foulkes. 2023. What does method validation look like for forensic voice comparison by a human expert? Science & Justice 63, 2 (2023), 251–257.

[19]

Kimaya Lecamwasam, Samantha Gutierrez Arango, Nikhil Singh, Neska Elhaouij, Max Addae, and Rosalind Picard. 2023. Investigating the physiological and psychological effect of an interactive musical interface for stress and anxiety reduction. In Extended Abstracts of the 2023 CHI Conference on Human Factors in Computing Systems. 1–9.

Digital Library

[20]

Hailiang Long, Zhenghao Guo, Xia Wu, Bin Hu, Zhenyu Liu, and Hanshu Cai. 2017. Detecting depression in speech: Comparison and combination between different speech types. In Proceedings of the 2017 IEEE International Conference on Bioinformatics and Biomedicine (BIBM ’17). IEEE, 1052–1058.

[21]

Daniel M. Low, Kate H. Bentley, and Satrajit S. Ghosh. 2020. Automated assessment of psychiatric disorders using speech: A systematic review. Laryngoscope Investigative Otolaryngology 5, 1 (2020), 96–116.

[22]

Xingchen Ma, Hongyu Yang, Qiang Chen, Di Huang, and Yunhong Wang. 2016. DepAudioNet: An efficient deep model for audio based depression classification. In Proceedings of the 6th International Workshop on Audio/Visual Emotion Challenge. 35–42.

Digital Library

[23]

Brian McFee, Colin Raffel, Dawen Liang, Daniel P. Ellis, Matt McVicar, Eric Battenberg, and Oriol Nieto. 2015. Librosa: Audio and music signal analysis in Python. In Proceedings of the 14th Python in Science Conference, Vol. 8. 18–25.

[24]

Ellen W. McGinnis, Steven P. Anderau, Jessica Hruschak, Reed D. Gurchiek, Nestor L. Lopez-Duran, Kate Fitzgerald, Katherine L. Rosenblum, Maria Muzik, and Ryan S. McGinnis. 2019. Giving voice to vulnerable children: Machine learning analysis of speech detects anxiety and depression in early childhood. IEEE Journal of Biomedical and Health Informatics 23, 6 (2019), 2294–2301.

[25]

Ellen W. McGinnis, Steven P. Anderau, Jessica Hruschak, Reed D. Gurchiek, Nestor L. Lopez-Duran, Kate Fitzgerald, Katherine L. Rosenblum, Maria Muzik, and Ryan S. McGinnis. 2019. Giving voice to vulnerable children: Machine learning analysis of speech detects anxiety and depression in early childhood. IEEE Journal of Biomedical and Health Informatics 23, 6 (2019), 2294–2301. DOI:

[26]

Amit Meghanani, C. S. Anoop, and A. G. Ramakrishnan. 2021. An exploration of log-mel spectrogram and MFCC features for Alzheimer’s dementia recognition from spontaneous speech. In Proceedings of the 2021 IEEE Spoken Language Technology Workshop (SLT ’21). IEEE, 670–677.

[27]

Darién Miranda, Marco Calderón, and Jesus Favela. 2014. Anxiety detection using wearable monitoring. In Proceedings of the 5th Mexican Conference on Human-Computer Interaction. 34–41.

Digital Library

[28]

James C. Mundt, Peter J. Snyder, Michael S. Cannizzaro, Kara Chappie, and Dayna S. Geralts. 2007. Voice acoustic measures of depression severity and treatment response collected via interactive voice response (IVR) technology. Journal of Neurolinguistics 20, 1 (2007), 50–64.

[29]

Nilesh K. Sahu, Manjeet Yadav, and Haroon R. Lone. 2024. Social Anxiety Disorder—Participants Audio Data Set. Retrieved April 19, 2024 from https://osf.io/b8uyg/

[30]

Turgut Özseven, Muharrem Düğenci, Ali Doruk, and Hilal I. Kahraman. 2018. Voice traces of anxiety: Acoustic parameters affected by anxiety disorder. Archives of Acoustics 43, 4 (2018), 625–636.

[31]

Dimitri Palaz, Ronan Collobert, et al. 2015. Analysis of CNN-Based Speech Recognition System using Raw Speech as Input. Technical Report. Idiap Research Institute.

[32]

James W. Pennebaker, Martha E. Francis, and Roger J. Booth. 2001. Linguistic Inquiry and Word Count: LIWC 2001. Lawrence Erlbaum Associates, Mahwah, NJ.

[33]

Haroon Rashid, Sanjana Mendu, Katharine E. Daniel, Miranda L. Beltzer, Bethany A. Teachman, Mehdi Boukhechba, and Laura E. Barnes. 2020. Predicting subjective measures of social anxiety from sparsely collected mobile sensor data. Proceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies 4, 3 (2020), 1–24.

Digital Library

[34]

Asif Salekin, Jeremy W. Eberle, Jeffrey J. Glenn, Bethany A. Teachman, and John A. Stankovic. 2018. A weakly supervised learning framework for detecting social anxiety and depression. Proceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies 2, 2 (2018), 1–26.

Digital Library

[35]

Vered Silber-Varod, Hamutal Kreiner, Ronen Lovett, Yossi Levi-Belz, and Noam Amir. 2016. Do social anxiety individuals hesitate more? The prosodic profile of hesitation disfluencies in social anxiety disorder individuals. Speech Prosody 2016 (2016), 1211–1215.

[36]

N. S. Srimadhur and S. Lalitha. 2020. An end-to-end model for detection and assessment of depression levels using speech. Procedia Computer Science 171 (2020), 12–21.

[37]

Michael Stein, Benjamin M. Schubert, Matthias Gruhne, Gabriel Gatzsche, and Markus Mehnert. 2009. Evaluation and comparison of audio chroma feature extraction methods. In Audio Engineering Society Convention 126. Audio Engineering Society.

[38]

Murray B. Stein and Dan J. Stein. 2008. Social anxiety disorder. Lancet 371, 9618 (2008), 1115–1125.

[39]

Zhaodong Sun, Alexander Vedernikov, Virpi-Liisa Kykyri, Mikko Pohjola, Miriam Nokia, and Xiaobai Li. 2022. Estimating stress in online meetings by remote physiological signal and behavioral features. In Adjunct Proceedings of the 2022 ACM International Joint Conference on Pervasive and Ubiquitous Computing and the 2022 ACM International Symposium on Wearable Computers. 216–220.

Digital Library

[40]

Bazen Gashaw Teferra, Sophie Borwein, Danielle D. DeSouza, and Jonathan Rose. 2022. Screening for generalized anxiety disorder from acoustic and linguistic features of impromptu speech: Prediction model evaluation study. JMIR Formative Research 6, 10 (2022), e39998.

[41]

Bazen Gashaw Teferra, Sophie Borwein, Danielle D. DeSouza, William Simpson, Ludovic Rheault, and Jonathan Rose. 2022. Acoustic and linguistic features of impromptu speech and their association with anxiety: Validation study. JMIR Mental Health 9, 7 (2022), e36828.

[42]

Bazen Gashaw Teferra and Jonathan Rose. 2023. Predicting generalized anxiety disorder from impromptu speech transcripts using context-aware transformer-based neural networks: Model evaluation study. JMIR Mental Health 10, 1 (2023), e44325.

[43]

Zhiyuan Wang, Mingyue Tang, Maria A. Larrazabal, Emma R. Toner, Mark Rucker, Congyu Wu, Bethany A. Teachman, Mehdi Boukhechba, and Laura E. Barnes. 2023. Personalized state anxiety detection: An empirical study with linguistic biomarkers and a machine learning pipeline. arXiv preprint arXiv:2304.09928 (2023).

[44]

Shreyasi Watve, Mangal Patil, and Arundhati Shinde. 2023. Review of features and classification for spoken Indian language recognition using deep learning and machine learning techniques. In Proceedings of the 2023 International Conference on Emerging Smart Computing and Informatics (ESCI ’23). IEEE, 1–6.

[45]

Justin W. Weeks, Akanksha Srivastav, Ashley N. Howell, and Andrew R. Menatti. 2016. “Speaking more than words”: Classifying men with social anxiety disorder via vocal acoustic analyses of diagnostic interviews. Journal of Psychopathology and Behavioral Assessment 38 (2016), 30–41.

[46]

WHO. 2022. Mental Disorders. Retrieved September 3, 2023 from https://www.who.int/news-room/fact-sheets/detail/mental-disorders

[47]

Yuni Zeng, Hua Mao, Dezhong Peng, and Zhang Yi. 2019. Spectrogram based multi-task audio classification. Multimedia Tools and Applications 78 (2019), 3705–3722.

Digital Library

Cited By

Sahu NGupta SLone H(2024)Wearable Technology Insights: Unveiling Physiological Responses During Three Different Socially Anxious ActivitiesACM Journal on Computing and Sustainable Societies10.1145/36636712:2(1-23)Online publication date: 20-Jun-2024
https://dl.acm.org/doi/10.1145/3663671

Index Terms

Unveiling Social Anxiety: Analyzing Acoustic and Linguistic Traits in Impromptu Speech within a Controlled Study
1. Applied computing
  1. Law, social and behavioral sciences
    1. Sociology
  2. Operations research
    1. Forecasting
2. Human-centered computing
  1. Human computer interaction (HCI)
    1. Empirical studies in HCI
    2. HCI design and evaluation methods
      1. Laboratory experiments
      2. User studies
  2. Ubiquitous and mobile computing
    1. Empirical studies in ubiquitous and mobile computing
    2. Ubiquitous and mobile devices
      1. Mobile devices
      2. Mobile phones

Recommendations

Wearable Technology Insights: Unveiling Physiological Responses During Three Different Socially Anxious Activities
Wearable technology holds promise for monitoring and managing Social Anxiety Disorder (SAD), yet the absence of clear biomarkers specific to SAD hampers its effectiveness. This article explores this issue by presenting a study investigating variances in ...
Music genre classification based on auditory image, spectral and acoustic features
Abstract
Music genre is one of the conventional ways to describe music content, and also is one of the important labels of music information retrieval. Therefore, the effective and precise music genre classification method becomes an urgent need for ...
Internet administration of self-report measures commonly used in research on social anxiety disorder: A psychometric evaluation

The Internet has become increasingly popular as a way to administer self-report questionnaires, especially in the field of Internet delivered psychological treatments. Collecting questionnaire data over the Internet has advantages, such as ease of ...

Comments

Information & Contributors

Information

Published In

cover image ACM Journal on Computing and Sustainable Societies

ACM Journal on Computing and Sustainable Societies Volume 2, Issue 2

June 2024

421 pages

EISSN:2834-5533

DOI:10.1145/3613748

Editor:
Lakshminarayanan Subramanian
New York University, United States

Issue’s Table of Contents

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than the author(s) must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected].

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 20 June 2024

Online AM: 12 April 2024

Accepted: 31 March 2024

Revised: 28 March 2024

Received: 26 September 2023

Published in ACMJCSS Volume 2, Issue 2

Check for updates

Author Tags

Qualifiers

Research-article

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

1
Total Citations
View Citations
212
Total Downloads

Downloads (Last 12 months)212
Downloads (Last 6 weeks)53

Reflects downloads up to 01 Sep 2024

Other Metrics

View Author Metrics

Citations

Cited By

Sahu NGupta SLone H(2024)Wearable Technology Insights: Unveiling Physiological Responses During Three Different Socially Anxious ActivitiesACM Journal on Computing and Sustainable Societies10.1145/36636712:2(1-23)Online publication date: 20-Jun-2024
https://dl.acm.org/doi/10.1145/3663671

View Options

Get Access

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Article

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Full Text

View this article in Full Text.

Media

Figures

Other

Tables

View full text|Download PDF

View Issue’s Table of Contents