Abstract
This article describes a general framework for detecting accident-prone fatigue states based on prosody, articulation and speech quality related speech characteristics. The advantages of this real-time measurement approach are that obtaining speech data is non obtrusive, and free from sensor application and calibration efforts. The main part of the feature computation is the combination of frame level based speech features and high level contour descriptors resulting in over 8,500 features per speech sample. In general the measurement process follows the speech adapted steps of pattern recognition: (a) recording speech, (b) preprocessing (segmenting speech units of interest), (c) feature computation (using perceptual and signal processing related features, as e.g. fundamental frequency, intensity, pause patterns, formants, cepstral coefficients), (d) dimensionality reduction (filter and wrapper based feature subset selection, (un-)supervised feature transformation), (e) classification (e.g. SVM, K-NN classifier), and (f) evaluation (e.g. 10-fold cross validation). The validity of this approach is briefly discussed by summarizing the empirical results of a sleep deprivation study.
This is a preview of subscription content, log in via an institution to check access.
Preview
Unable to display preview. Download preview PDF.
Similar content being viewed by others
References
MacLean, A.W.: Sleepiness and Driving. Sleep Medicine Reviews 7, 507–521 (2003)
Melamed, S.: Excessive Daytime Sleepiness and Risk of Occupational Injuries in Non-Shift Daytime Workers. Sleep 25(3), 315–322 (2002)
Wright, N., McGown, A.: Vigilance on the Civil Flight Deck: Incidence of Sleepiness and Sleep during Long-Haul Flights and Associated Changes in Physiological Parameters. Ergonomics 44, 82–106 (2001)
Durmer, J.S., Dinges, D.F.: Neurocognitive Consequences of Sleep Deprivation. Seminars in Neurology 25, 117–129 (2005)
Nilsson, J.P., Soderstrom, M., Karlsson, A.U., Lekander, M., Akerstedt, T., Lindroth, N.E., Axelsson, J.: Less Effective Executive Functioning after one Night´s Sleep Deprivation. Journal of Sleep Research 14, 1–6 (2005)
Cañas, J.J.: Technology for special needs. An Interdisciplinary. Journal on Humans in ICT Environments 2, 4–7 (2006)
Kollias, S., Amir, N., Kim, J., Grandjean, D.: Description of Potential Exemplars: Signals and Signs of Emotion. HUMAINE Human-Machine Interaction Network on Emotions (2004)
Caffier, P.P.: The Spontaneous Eye-Blink as Sleepiness Indicator in Patients with Obstructive Sleep Apnoea Syndrome-a Pilot Study. Sleep Medicine 2, 155–162 (2002)
Sommer, D., Chen, M., Golz, M., Trunschel, U., Mandic, D.: Fusion of State Space and Frequency Domain Features for Improved Microsleep Detection. In: Duch, W., Kacprzyk, J., Oja, E., Zadrożny, S. (eds.) ICANN 2005. LNCS, vol. 3697, pp. 753–759. Springer, Heidelberg (2005)
Vöhringer-Kuhnt, T., Baumgarten, T., Karrer, K., Briest, S.: Wierwille’s Method of Driver Drowsiness Evaluation Revisited. In: Proceeding of International Conference on Traffic & Transport Psychology (2004)
Schuller, B.: Automatische Emotionserkennung aus sprachlicher und manueller Interaktion. [Automatic Emotion Recognition from verbal and manual Interaction]. Dissertation, Technische Universität München (2006)
Schuller, B., Batliner, A., Seppi, D., Steidl, S., Vogt, T., Wagner, J., Devillers, L., Vidrascu, L., Amir, N., Kessous, L., Aharonson, V.: The Relevance of Feature Type for the Automatic Classification of Emotional User States: Low Level Descriptors and Functionals. In: Proceedings of Interspeech, pp. 2253–2256 (2007)
Vlasenko, B., Schuller, B., Wendemuth, A., Rigoll, G.: Combining Frame and Turn-Level Information for Robust Recognition of Emotions within Speech. In: Proceedings of Interspeech, pp. 2249–2252 (2007)
Batliner, A., Steidl, S., Schuller, B., Seppi, D., Laskowski, K., Vogt, T., Devillers, L., Vidrascu, L., Amir, N., Kessous, L., Aharonson, V.: Combining Efforts for Improving Automatic Classification of Emotional User States. In: Erjavec, T., Gros, J.Z. (eds.) Language Technologies, IS-LTC 2006, Ljubljana, Slovenia, pp. 240–245 (2006)
Mierswa, I., Morik, K.: Automatic Feature Extraction for Classifying Audio Data. Kluwe, Amsterdam (2005)
Batliner, A., Hacker, C., Steidl, S., Noeth, E., D’Arcy, S., Rusell, M., Wong, M.: “You stupid tin box” – Children interacting with the AIBO robot: A crosslinguisitc emotional speech corpus. In: Proceedings of the 4th International Conference of Language Resources and Evaluation LREC 2004 (LREC Lisbon 2004), pp. 171–174 (2004)
Steidl, S., Hacker, C., Ruff, C., Batliner, A., Noeth, E., Haas, J.: Looking at the Last Two Turns, I’d Say This Dialogue is Doomed – Measuring Dialogue Success. In: Proceedings TSD (Text, Speech and Dialog), pp. 629–636 (2004)
Burkhardt, F., Paeschke, A., Rolfes, M., Sendlmeier, W., Weiss, B.: A Database of German Emotional Speech. In: Proceedings of Interspeech 2005, Lisboa, Portugal, pp. 1517–1520 (2005)
Schiel, F.: MAUS Goes Iterative. In: Proc. of the IV. International Conference on Language Resources and Evaluation, Lisbon, Portugal, pp. 1015–1018 (2004)
Rabiner, L., Schafer, R.W.: Digital Processing of Speech Signals. Prentice-Hall, Upper Saddle River (1978)
Scherer, K.R.: Vocal affect expression: A review and a model for future research. Psychological Bulletin 99, 143–165 (1986)
Kienast, M., Sendlmeier, W.F.: Acoustical analysis of spectral and temporal changes in emotional speech. Speech Emotion, 92–97 (2000)
Tartter, V.C.: Happy talk - Perceptual and acoustic effects of smiling on speech. Perception and Psychophysics 27(1), 24–27 (1980)
Nwe, T.L., Li, H., Dong, M.: Analysis and Detection of Speech under Sleep Deprivation. In: Proceeding of Interspeech, pp. 17–21 (2006)
Krajewski, J., Kröger, B.: Using prosodic and spectral characteristics for sleepiness detection. In: Interspeech Proceedings, pp. 1841–1844 (2007)
Author information
Authors and Affiliations
Editor information
Rights and permissions
Copyright information
© 2008 Springer-Verlag Berlin Heidelberg
About this paper
Cite this paper
Krajewski, J., Wieland, R., Batliner, A. (2008). An Acoustic Framework for Detecting Fatigue in Speech Based Human-Computer-Interaction. In: Miesenberger, K., Klaus, J., Zagler, W., Karshmer, A. (eds) Computers Helping People with Special Needs. ICCHP 2008. Lecture Notes in Computer Science, vol 5105. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-540-70540-6_7
Download citation
DOI: https://doi.org/10.1007/978-3-540-70540-6_7
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-540-70539-0
Online ISBN: 978-3-540-70540-6
eBook Packages: Computer ScienceComputer Science (R0)