tutorial

Speech stress assessment using physiological and psychological measures

Authors:

Mariana Kaiseler,

Traian E. Abrudan,

Pedro R. AlmeidaAuthors Info & Claims

UbiComp '13 Adjunct: Proceedings of the 2013 ACM conference on Pervasive and ubiquitous computing adjunct publication

Pages 921 - 930

https://doi.org/10.1145/2494091.2497346

Published: 08 September 2013 Publication History

Abstract

Emotional stress is commonly experienced while speaking in public, producing changes to the various speech productions subsystems, affecting the speech signal in predictable ways and being easily conveyed to listeners. Speech stress indicators, however, are typically studied under laboratory settings, allowing little generalization to real life settings. To bridge this gap, we propose an interdisciplinary approach to assess speech stress during public speaking events, based on a platform that records speech simultaneously annotated with physiological and psychological measures. This approach enables the collection of a large corpus of annotated speech in ecological settings, i.e. in objectively stressing situations. We also propose and implement a methodology to assess listeners evaluation of stress including psychologists, and overall public.

The platform has been in use for the past 5 months, and we have collected 13 complete samples after the initial iterative development procedure. Preliminary results indicate that the proposed user-friendly platform is an accurate and robust method to collect annotated speech under ecological settings that can be processed to obtain speech stress indicators. The findings will be used primarily in the design of computer and mobile assisted voice coaching applications, but the outreach extends to mobile emotion sensing for individuals and crowds.

References

[1]

Allen, M., Boquet, A., and Shelley, K. Cluster analyses of cardiovascular responsivity to three laboratory stressors. Psychosomatic Medicine 53 (1991), 272--288.

[2]

Batista, F., Moniz, H., Trancoso, I., and Mamede, N. Bilingual experiments on automatic recovery of capitalization and punctuation of automatic speech transcripts. Trans. Audio, Speech and Lang. Proc. 20, 2 (Feb. 2012), 474--485.

Digital Library

[3]

Berntson, G., and Cacioppo, J. Dynamic Electrocardiography. Future, 2004, ch. Heart rate variability: Stress and psychiatric conditions, 57--64.

[4]

Eyben, F., Wöllmer, M., and Schuller, B. Opensmile: the munich versatile and fast open-source audio feature extractor. In Proceedings of the international conference on Multimedia, MM '10, ACM (New York, NY, USA, 2010), 1459--1462.

Digital Library

[5]

Hall, M., Frank, E., Holmes, G., Pfahringer, B., Reutemann, P., and Witten, I. H. The weka data mining software: an update. SIGKDD Explor. Newsl. 11, 1 (Nov. 2009), 10--18.

Digital Library

[6]

Hall, M. A. Correlation-based Feature Subset Selection for Machine Learning. PhD thesis, University of Waikato, Hamilton, New Zealand, 1998.

[7]

Ho, T. K. The random subspace method for constructing decision forests. IEEE Trans. Pattern Anal. Mach. Intell. 20, 8 (Aug. 1998), 832--844.

Digital Library

[8]

Kaiseler, M., Polman, R., and Nicholls, A. Think aloud: Gender differences in appraisal and coping with stress during the execution of a complex motor task. International Journal of Sport and Exercise Psychology 10, 4 (2012), 1--15.

[9]

Mehrabian, A. Silent messages: Implicit communication of emotions and attitudes. Belmont: Wadsworth, 1981.

[10]

Miller, T., and Stone, D. Public speaking apprehension (psa),motivation, and affect among accounting majors: A proof-of-concept intervention. Issues in Accounting Education (2009).

[11]

of The European Society of Cardiology, T. F., of Pacing, T. N. A. S., and Electrophysiology. Heart rate variability: Standards of measurement, physiological interpretation, and clinical use. European Heart Journal 17 (1996), 354--381.

[12]

Ponciano, E., Loureiro, L., Pereira, A., and Spielberger, C. Características psicométricas e estrutura factorial do tai de spielberger em estudantes universitários. In Actas do Congresso Nacional Acçãao Social e Aconselhamento Psicológico no Ensino Superior: Investigação e Intervenção, A. P. E. Motta, Ed. (2005), 315--322.

[13]

Rachuri, K. K., Musolesi, M., Mascolo, C., Rentfrow, P. J., Longworth, C., and Aucinas, A. Emotionsense: a mobile phones based adaptive platform for experimental social psychology research. In Proceedings of the 12th ACM international conference on Ubiquitous computing, Ubicomp '10, ACM (2010), 281--290.

Digital Library

[14]

Scherer, K. Vocal communication of emotion: A review of research paradigms. Speech Communication 40 (2003), 227--256.

Digital Library

[15]

Schuller, B., Steidl, S., Batliner, A., Nöth, E., Vinciarelli, A., Burkhardt, F., van Son, R., Weninger, F., Eyben, F., Bocklet, T., Mohammadi, G., and Weiss, B. The INTERSPEECH 2012 Speaker Trait Challenge. In Proceedings of the 13th Annual Conference of the International Speech Communication Association (INTERSPEECH 2012) (2012).

[16]

Spielberger, C. D., Gorsuch, R. L., Lushene, R., Vagg, P. R., and Jacobs, G. A. Manual for the State-Trait Anxiety Inventory. Consulting Psychologists Press, 1983.

[17]

Stone, A., and Shiffman, S. Capturing momentary, self- report data: A proposal for reporting guidelines. Annals of Behavior Medicine 24, 3 (2002), 236--243.

[18]

Ververidis, D., and Kotropoulos, C. Emotional speech recognition: Resources, features, and methods. Speech Communication 48, 9 (2006), 1162--1181.

[19]

Wilhelm, F., and Grossman, P. Emotions beyond the laboratory: Theoretical fundaments, study design, and analytic strategies for advanced ambulatory assessment. Biological Psychology 84 (2010), 552--569.

[20]

Yang, N., Muraleedharan, R., Kohl, J., Demirkol, I., Heinzelman, W., and Sturge-Apple, M. Speech-based emotion classification using multiclass svm with hybrid kernel and thresholding fusion. In Proceedings of the 4th IEEE Workshop on Spoken Language Technology (Dec 2012).

[21]

Zanstra, Y., and Johnston, D. Cardiovascular reactivity in real life settings: Measurement, mechanisms and meaning. Biological Psychology 86 (2011), 98--105.

Cited By

Zhao LNiu XWang LNiu JZhu XDai Z(2023)Stress Detection via Multimodal Multitemporal-Scale Fusion: A Hybrid of Deep Learning and Handcrafted Feature ApproachIEEE Sensors Journal10.1109/JSEN.2023.331471823:22(27817-27827)Online publication date: 15-Nov-2023
https://doi.org/10.1109/JSEN.2023.3314718
Yadav MSakib MNirjhar EFeng KBehzadan AChaspari T(2022)Exploring Individual Differences of Public Speaking Anxiety in Real-Life and Virtual PresentationsIEEE Transactions on Affective Computing10.1109/TAFFC.2020.304829913:3(1168-1182)Online publication date: 1-Jul-2022
https://doi.org/10.1109/TAFFC.2020.3048299
Ding XWei SGui XGu NZhang PKitamura YQuigley AIsbister KIgarashi TBjørn PDrucker S(2021)Data Engagement Reconsidered: A Study of Automatic Stress Tracking Technology in UseProceedings of the 2021 CHI Conference on Human Factors in Computing Systems10.1145/3411764.3445763(1-13)Online publication date: 6-May-2021
https://dl.acm.org/doi/10.1145/3411764.3445763
Show More Cited By

Index Terms

Speech stress assessment using physiological and psychological measures
1. Applied computing
  1. Law, social and behavioral sciences
    1. Psychology
2. Human-centered computing
  1. Human computer interaction (HCI)

Recommendations

Modern Standard Arabic speech disorders corpus for digital speech processing applications
Abstract
Digital speech processing applications including automatic speech recognition (ASR), speaker recognition, speech translation, and others, essentially require large volumes of speech data for training and testing purposes. Although there are ...
A new speech corpus of super-elderly Japanese for acoustic modeling
Abstract
The development of accessible speech recognition technology will allow the elderly to more easily access electronically stored information. However, the necessary level of recognition accuracy for elderly speech has not yet been ...
Highlights
- The acoustic characteristics of elderly speech differ from those of younger speakers.
Development and Analysis of Speech Recognition Systems for Assamese Language Using HTK

Language analysis is very important for the native speaker to connect with the digital world. Assamese is a relatively unexplored language. In this report, we analyze different aspects of speech-to-text processing, starting from building a speech corpus,...

Comments

Information & Contributors

Information

Published In

cover image ACM Conferences

UbiComp '13 Adjunct: Proceedings of the 2013 ACM conference on Pervasive and ubiquitous computing adjunct publication

September 2013

1608 pages

ISBN:9781450322157

DOI:10.1145/2494091

General Chairs:
Friedemann Mattern
ETH Zurich, CH
,
Silvia Santini
TU Darmstadt, DE
,
Program Chairs:
John F. Canny
UC Berkeley, US
,
Marc Langheinrich
Università della Svizzera italiana, CH
,
Jun Rekimoto
University of Tokyo, JP

Copyright © 2013 ACM.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than the author(s) must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected].

Sponsors

SIGMOBILE: ACM Special Interest Group on Mobility of Systems, Users, Data and Computing
University of Florida: University of Florida
SIGCHI: ACM Special Interest Group on Computer-Human Interaction

In-Cooperation

SIGSPATIAL: ACM Special Interest Group on Spatial Information

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 08 September 2013

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Tutorial

Conference

UbiComp '13

Sponsor:

SIGMOBILE
University of Florida
SIGCHI

UbiComp '13: The 2013 ACM International Joint Conference on Pervasive and Ubiquitous Computing

September 8 - 12, 2013

Zurich, Switzerland

Acceptance Rates

UbiComp '13 Adjunct Paper Acceptance Rate 254 of 399 submissions, 64%;

Overall Acceptance Rate 764 of 2,912 submissions, 26%

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

8
Total Citations
View Citations
298
Total Downloads

Downloads (Last 12 months)28
Downloads (Last 6 weeks)3

Reflects downloads up to 15 Oct 2024

Other Metrics

View Author Metrics

Citations

Cited By

Zhao LNiu XWang LNiu JZhu XDai Z(2023)Stress Detection via Multimodal Multitemporal-Scale Fusion: A Hybrid of Deep Learning and Handcrafted Feature ApproachIEEE Sensors Journal10.1109/JSEN.2023.331471823:22(27817-27827)Online publication date: 15-Nov-2023
https://doi.org/10.1109/JSEN.2023.3314718
Yadav MSakib MNirjhar EFeng KBehzadan AChaspari T(2022)Exploring Individual Differences of Public Speaking Anxiety in Real-Life and Virtual PresentationsIEEE Transactions on Affective Computing10.1109/TAFFC.2020.304829913:3(1168-1182)Online publication date: 1-Jul-2022
https://doi.org/10.1109/TAFFC.2020.3048299
Ding XWei SGui XGu NZhang PKitamura YQuigley AIsbister KIgarashi TBjørn PDrucker S(2021)Data Engagement Reconsidered: A Study of Automatic Stress Tracking Technology in UseProceedings of the 2021 CHI Conference on Human Factors in Computing Systems10.1145/3411764.3445763(1-13)Online publication date: 6-May-2021
https://dl.acm.org/doi/10.1145/3411764.3445763
Sidorova JKarlsson SRosander OBerthier MMoreno-Torres I(2021)Towards Disorder-Independent Automatic Assessment of Emotional Competence in Neurological Patients with a Classical Emotion Recognition System: Application in Foreign Accent SyndromeIEEE Transactions on Affective Computing10.1109/TAFFC.2019.290836512:4(962-973)Online publication date: 1-Oct-2021
https://doi.org/10.1109/TAFFC.2019.2908365
Roldan-Rojo TRendon-Velez ECarrizosa S(2021)Stressors and Algorithms Used for Stress Detection: a Review2021 9th International Conference on Affective Computing and Intelligent Interaction (ACII)10.1109/ACII52823.2021.9597456(1-8)Online publication date: 28-Sep-2021
https://doi.org/10.1109/ACII52823.2021.9597456
von Ebers MNirjhar EBehzadan AChaspari TTruong KHeylen DCzerwinski MBerthouze NChetouani MNakano M(2020)Predicting the Effectiveness of Systematic Desensitization Through Virtual Reality for Mitigating Public Speaking AnxietyProceedings of the 2020 International Conference on Multimodal Interaction10.1145/3382507.3418883(670-674)Online publication date: 21-Oct-2020
https://dl.acm.org/doi/10.1145/3382507.3418883
Alberdi AAztiria ABasarab A(2016)Towards an automatic early stress recognition system for office environments based on multimodal measurementsJournal of Biomedical Informatics10.1016/j.jbi.2015.11.00759:C(49-75)Online publication date: 1-Feb-2016
https://dl.acm.org/doi/10.1016/j.jbi.2015.11.007
Julião MSilva JAguiar AMoniz HBatista F(2015)Speech Features for Discriminating Stress Using Branch and Bound Wrapper SearchLanguages, Applications and Technologies10.1007/978-3-319-27653-3_1(3-14)Online publication date: 24-Dec-2015
https://doi.org/10.1007/978-3-319-27653-3_1

View Options

Get Access

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Media

Figures

Other

Tables

View Table of Contents