short-paper

Fusing active orientation models and mid-term audio features for automatic depression estimation

Authors:

Christos Smailis,

Nikolaos Sarafianos,

Theodoros Giannakopoulos, and

Stavros PerantonisAuthors Info & Claims

PETRA '16: Proceedings of the 9th ACM International Conference on PErvasive Technologies Related to Assistive Environments

June 2016

Article No.: 39, Pages 1 - 4

https://doi.org/10.1145/2910674.2935856

Published: 29 June 2016 Publication History

Get Access

Abstract

In this paper, we predict a human's depression level in the BDI-II scale, using facial and voice features. Active orientation models (AOM) and several voice features were extracted from the video and audio modalities. Long-term and mid-term features were computed and a fusion is performed in the feature space. Videos from the Depression Recognition Sub-Challenge of the 2014 Audio-Visual Emotion Challenge and Workshop (AVEC 2014) were used and support vector regression models were trained to predict the depression level. We demonstrated that the fusion of AOMs with audio features leads to better performance compared to individual modalities. The obtained regression results indicate the robustness of the proposed technique, under different settings, as well as an RMSE improvement compared to the AVEC 2014 video baseline.

References

[1]

World Health Organization. Depression - a hidden burden. www.who.int/mediacentre/factsheets/fs369. Last accessed: 05.01.2016.

Google Scholar

[2]

Almaev, T. R., and Valstar, M. F. Local gabor binary patterns from three orthogonal planes for automatic facial expression recognition. In ACII (2013).

Digital Library

Google Scholar

[3]

Beck, A. T., Steer, R. A., Ball, R., and Ranieri, W. Comparison of Beck Depression Inventories -IA and -II in psychiatric outpatients. Journal of Personality Assessment (1996).

Google Scholar

[4]

Belhumeur, P. N., Jacobs, D. W., Kriegman, D. J., and Kumar, N. Localizing parts of faces using a consensus of exemplars. TPAMI (2013).

Digital Library

Google Scholar

[5]

Cohn, J. F., Kruez, T. S., Matthews, I., Yang, Y., Nguyen, M. H., Padilla, M. T., Zhou, F., and La Torre, F. D. Detecting depression from facial actions and vocal prosody. In ACII Workshops (2009).

Crossref

Google Scholar

[6]

Gupta, R., Malandrakis, N., Xiao, B., Guha, T., Van Segbroeck, M., Black, M., Potamianos, A., and Narayanan, S. Multimodal prediction of affective dimensions and depression in human-computer interactions. In ACM International AVEC Workshop (2014).

Digital Library

Google Scholar

[7]

Jain, V., Crowley, J. L., Dey, A. K., and Lux, A. Depression estimation using audiovisual features and fisher vector encoding. In ACM International AVEC Workshop (2014).

Digital Library

Google Scholar

[8]

Jan, A., Meng, H., Gaus, Y. F. A., Zhang, F., and Turabzadeh, S. Automatic depression scale prediction using facial expression dynamics and regression. In ACM International AVEC Workshop (2014).

Digital Library

Google Scholar

[9]

Kaya, H., Eyben, F., Salah, A. A., and Schuller, B. CCA based feature selection with application to continuous depression recognition from acoustic speech features. In ICASSP (2014).

Google Scholar

[10]

Pérez Espinosa, H., Escalante, H. J., Villaseñor Pineda, L., Montes-y Gómez, M., Pinto-Avedaño, D., and Reyez-Meza, V. Fusing affective dimensions and audio-visual features from segmented video for depression recognition: INAOE-BUAP's. In ACM International AVEC Workshop (2014).

Digital Library

Google Scholar

[11]

Senoussaoui, M., Sarria-Paja, M., Santos, J. a. F., and Falk, T. H. Model fusion for multimodal depression classification and level detection. In ACM International AVEC Workshop (2014).

Digital Library

Google Scholar

[12]

Tzimiropoulos, G., Alabort-i Medina, J., Zafeiriou, S., and Pantic, M. Generic active appearance models revisited. In ACCV. 2013.

Digital Library

Google Scholar

[13]

Valstar, M., Schuller, B., Smith, K., Almaev, T., Eyben, F., Krajewski, J., Cowie, R., and Pantic, M. AVEC 2014: 3D dimensional affect and depression recognition challenge. In ACM International AVEC Workshop (2014).

Digital Library

Google Scholar

[14]

Williamson, J. R., Quatieri, T. F., Helfer, B. S., Ciccarelli, G., and Mehta, D. D. Vocal and facial biomarkers of depression based on motor incoordination and timing. In ACM International AVEC Workshop (2014).

Digital Library

Google Scholar

Cited By

View all

Pampouchidou ASimos PMarias KMeriaudeau FYang FPediaditis MTsiknakis M(2019)Automatic Assessment of Depression Based on Visual Cues: A Systematic ReviewIEEE Transactions on Affective Computing10.1109/TAFFC.2017.272403510:4(445-470)Online publication date: 1-Oct-2019
https://doi.org/10.1109/TAFFC.2017.2724035
Bernin AMüller LGhose Svon Luck KGrecos CWang QVogt F(2017)Towards More Robust Automatic Facial Expression Recognition in Smart EnvironmentsProceedings of the 10th International Conference on PErvasive Technologies Related to Assistive Environments10.1145/3056540.3056546(37-44)Online publication date: 21-Jun-2017
https://dl.acm.org/doi/10.1145/3056540.3056546

Fusing active orientation models and mid-term audio features for automatic depression estimation
1. Computing methodologies
  1. Artificial intelligence

Recommendations

Speech Enhancement and Recognition in Meetings With an Audio–Visual Sensor Array

This paper addresses the problem of distant speech acquisition in multiparty meetings, using multiple microphones and cameras. Microphone array beamforming techniques present a potential alternative to close-talking microphones by providing speech ...
Read More
Realtime depression estimation using mid-term audio features
AIAM'14: Proceedings of the 3rd International Conference on Artificial Intelligence and Assistive Medicine - Volume 1213

This paper presents a method towards estimating a clinical depression-specific score, namely the Beck Depression Inventory (BDI) score, based on analysis of mid-term audio features. A combination of support vector machines and semi-supervised learning ...
Read More
Fusing short term and long term features for improved speaker diarization
ICASSP '09: Proceedings of the 2009 IEEE International Conference on Acoustics, Speech and Signal Processing

The following article shows how a state-of-the-art speaker diarization system can be improved by combining traditional short-term features (MFCCs) with prosodic and other long-term features. First, we present a framework to study the speaker ...
Read More

Comments

Information & Contributors

Information

Published In

PETRA '16: Proceedings of the 9th ACM International Conference on PErvasive Technologies Related to Assistive Environments

June 2016

455 pages

ISBN:9781450343374

DOI:10.1145/2910674

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 29 June 2016

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Short-paper
Research
Refereed limited

Conference

PETRA '16

PETRA '16: 9th ACM International Conference on PErvasive Technologies Related to Assistive Environments

June 29 - July 1, 2016

Island, Corfu, Greece

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

2
Total Citations
View Citations
84
Total Downloads

Downloads (Last 12 months)4
Downloads (Last 6 weeks)0

Other Metrics

View Author Metrics

Citations

Cited By

View all

Pampouchidou ASimos PMarias KMeriaudeau FYang FPediaditis MTsiknakis M(2019)Automatic Assessment of Depression Based on Visual Cues: A Systematic ReviewIEEE Transactions on Affective Computing10.1109/TAFFC.2017.272403510:4(445-470)Online publication date: 1-Oct-2019
https://doi.org/10.1109/TAFFC.2017.2724035
Bernin AMüller LGhose Svon Luck KGrecos CWang QVogt F(2017)Towards More Robust Automatic Facial Expression Recognition in Smart EnvironmentsProceedings of the 10th International Conference on PErvasive Technologies Related to Assistive Environments10.1145/3056540.3056546(37-44)Online publication date: 21-Jun-2017
https://dl.acm.org/doi/10.1145/3056540.3056546

View Options

Get Access

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Cited By

Recommendations

Speech Enhancement and Recognition in Meetings With an Audio–Visual Sensor Array

Realtime depression estimation using mid-term audio features

Fusing short term and long term features for improved speaker diarization

Comments

Published In

Publisher

Publication History

Permissions

Check for updates

Author Tags

Qualifiers

Conference

Other Metrics

Article Metrics

Other Metrics

Cited By

Login options

Full Access

PDF

eReader

Abstract

References

Cited By

Recommendations

Speech Enhancement and Recognition in Meetings With an Audio–Visual Sensor Array

Realtime depression estimation using mid-term audio features

Fusing short term and long term features for improved speaker diarization

Comments

Information

Published In

Publisher

Publication History

Permissions

Check for updates

Author Tags

Qualifiers

Conference

Contributors

Other Metrics

Bibliometrics

Article Metrics

Other Metrics

Citations

Cited By

Get Access

Login options

Full Access

View options

PDF

eReader

Figures

Other

Share

Share this Publication link

Share on social media

Affiliations