SHAP-based Prediction of Mother's History of Depression to Understand the Influence on Child Behavior
Authors: 
Maneesh Bilalpur, Saurabh Hinduja, Laura Cariola, Lisa Sheeber, Nicholas Allen, Louis-Philippe Morency, 
Jeffrey F. CohnAuthors Info & Claims
Maneesh Bilalpur, Saurabh Hinduja, Laura Cariola, Lisa Sheeber, Nicholas Allen, Louis-Philippe Morency, Jeffrey F. CohnAuthors Info & ClaimsPages 537 - 544
Abstract
Depression strongly impacts parents’ behavior. Does parents’ depression strongly affect the behavior of their children as well? To investigate this question, we compared dyadic interactions between 73 depressed and 75 non-depressed mothers and their adolescent child. Families were of low income and 84% were white. Child behavior was measured from audio-video recordings using manual annotation of verbal and nonverbal behavior by expert coders and by multimodal computational measures of facial expression, face and head dynamics, prosody, speech behavior, and linguistics. For both sets of measures, we used Support Vector Machines. For computational measures, we investigated the relative contribution of single versus multiple modalities using a novel approach to SHapley Additive exPlanations (SHAP). Computational measures outperformed manual ratings by human experts. Among individual computational measures, prosody was the most informative. SHAP reduction resulted in a four-fold decrease in the number of features and highest performance (77% accuracy; positive and negative agreements at 75% and 76%, respectively). These findings suggest that maternal depression strongly impacts the behavior of adolescent children; differences are most revealed in prosody; multimodal features together with SHAP reduction are most powerful.
References
[1]
Sharifa Alghowinem, Roland Goecke, Julien Epps, Michael Wagner, and Jeffrey F Cohn. 2016. Cross-cultural depression recognition from vocal biomarkers. In Interspeech. 1943–1947.
[2]
Sharifa Mohammed Alghowinem, Tom Gedeon, Roland Goecke, Jeffrey Cohn, and Gordon Parker. 2020. Interpretation of depression detection models via feature selection methods. IEEE Transactions on Affective Computing (2020).
[3]
Pınar Baki, Heysem Kaya, Elvan Çiftçi, Hüseyin Güleç, and Albert Ali Salah. 2022. A Multimodal Approach for Mania Level Prediction in Bipolar Disorder. IEEE Transactions on Affective Computing 13, 4 (2022), 2119–2131.
[4]
Lisa Feldman Barrett, Ralph Adolphs, Stacy Marsella, Aleix M Martinez, and Seth D Pollak. 2019. Emotional expressions reconsidered: Challenges to inferring emotion from human facial movements. Psychological science in the public interest 20, 1 (2019), 1–68.
[5]
Maneesh Bilalpur, Saurabh Hinduja, Laura A Cariola, Lisa B Sheeber, Nick Allen, László A Jeni, Louis-Philippe Morency, and Jeffrey F Cohn. 2023. Multimodal Feature Selection for Detecting Mothers’ Depression in Dyadic Interactions with their Adolescent Offspring. (2023).
[6]
Laura A Cariola, Saurabh Hinduja, Maneesh Bilalpur, Lisa B Sheeber, Nicholas Allen, Louis-Philippe Morency, and Jeffrey F Cohn. 2022. Language Use in Mother-Adolescent Dyadic Interaction: Preliminary Results. In ACII. IEEE.
[7]
James C Coyne. 1976. Depression and the response of others.Journal of abnormal psychology 85, 2 (1976), 186.
[8]
Nicholas Cummins, Stefan Scherer, Jarek Krajewski, Sebastian Schnieder, Julien Epps, and Thomas F Quatieri. 2015. A review of depression and suicide risk assessment using speech analysis. Speech communication 71 (2015), 10–49.
[9]
Paul Ekman, Wallace V Friesen, and Joseph C Hager. 2002. Facial Action Coding System: Facial action coding system: the manual: on CD-ROM. Research Nexus.
[10]
Karen A Ertel, Janet W Rich-Edwards, and Karestan C Koenen. 2011. Maternal depression in the United States: Nationally representative rates and risks. Journal of women’s health 20, 11 (2011), 1609–1617.
[11]
Itir Onal Ertugrul, Jeffrey F Cohn, László A Jeni, Zheng Zhang, Lijun Yin, and Qiang Ji. 2020. Crossing domains for au coding: Perspectives, approaches, and measures. IEEE transactions on biometrics, behavior, and identity science 2, 2 (2020), 158–171.
[12]
Florian Eyben, Klaus R Scherer, Björn W Schuller, Johan Sundberg, Elisabeth André, Carlos Busso, Laurence Y Devillers, Julien Epps, Petri Laukka, Shrikanth S Narayanan, 2015. The Geneva minimalistic acoustic parameter set (GeMAPS) for voice research and affective computing. IEEE transactions on affective computing 7, 2 (2015), 190–202.
[13]
Florian Eyben, Martin Wöllmer, and Björn Schuller. 2010. Opensmile: the munich versatile and fast open-source audio feature extractor. In ACM Multimedia. 1459–1462.
[14]
Jeffrey M Girard, Wen-Sheng Chu, László A Jeni, and Jeffrey F Cohn. 2017. Sayette group formation task (GFT) spontaneous facial expression database. In FG. IEEE, 581–588.
[15]
Sherryl H Goodman, Hannah FM Simon, Amanda L Shamblaw, and Christine Youngwon Kim. 2020. Parenting as a mediator of associations between depression in mothers and children’s functioning: A systematic review and meta-analysis. Clinical Child and Family Psychology Review 23 (2020), 427–460.
[16]
Jonathan Gratch, Ron Artstein, Gale Lucas, Giota Stratou, Stefan Scherer, Angela Nazarian, Rachel Wood, Jill Boberg, David DeVault, Stacy Marsella, 2014. The distress analysis interview corpus of human and computer interviews. Technical Report. UNIVERSITY OF SOUTHERN CALIFORNIA LOS ANGELES.
[17]
Hyman Hops, Betsy Davis, and Nancy Longoria. 1995. Methodological issues in direct observation: Illustrations with the Living in Familial Environments (LIFE) coding system. Journal of Clinical Child Psychology 24, 2 (1995), 193–203.
[18]
Joseph Jaffe, Beatrice Beebe, Stanley Feldstein, Cynthia L Crown, Michael D Jasnow, Philippe Rochat, and Daniel N Stern. 2001. Rhythms of dialogue in infancy: Coordinated timing in development. Monographs of the society for research in child development (2001), i–149.
[19]
Anis Kacem, Zakia Hammal, Mohamed Daoudi, and Jeffrey Cohn. 2018. Detecting depression severity by interpretable representations of motion dynamics. In FG. IEEE, 739–745.
[20]
Scott M Lundberg and Su-In Lee. 2017. A unified approach to interpreting model predictions. NeurIPS 30 (2017).
[21]
Wilson E Marcílio and Danilo M Eler. 2020. From explanations to feature selection: assessing SHAP values as feature selection mechanism. In 2020 33rd SIBGRAPI conference on Graphics, Patterns and Images (SIBGRAPI). Ieee, 340–347.
[22]
Shahan Ali Memon. 2020. Acoustic Correlates of the Voice Qualifiers: A Survey. arXiv preprint arXiv:2010.15869 (2020).
[23]
Benjamin W Nelson, Lisa Sheeber, Jennifer Pfeifer, and Nicholas B Allen. 2021. Psychobiological markers of allostatic load in depressed and nondepressed mothers and their adolescent offspring. Journal of Child Psychology and Psychiatry 62, 2 (2021), 199–211.
[24]
Benjamin W Nelson, Lisa Sheeber, Jennifer H Pfeifer, and Nicholas B Allen. 2021. Affective and Autonomic Reactivity During Parent–Child Interactions in Depressed and Non-Depressed Mothers and Their Adolescent Offspring. Research on Child and Adolescent Psychopathology 49, 11 (2021), 1513–1526.
[25]
Itir Onal Ertugrul, László A Jeni, Wanqiao Ding, and Jeffrey F Cohn. 2019. AFAR: A Deep Learning Based Tool for Automated Facial Affect Recognition. In FG. IEEE.
[26]
Asli Ozdas, Richard G Shiavi, Stephen E Silverman, Marilyn K Silverman, and D Mitchell Wilkes. 2004. Investigation of vocal jitter and glottal flow spectrum as possible cues for depression and near-term suicidal risk. IEEE transactions on Biomedical engineering 51, 9 (2004), 1530–1540.
[27]
James W Pennebaker, Ryan L Boyd, Kayla Jordan, and Kate Blackburn. 2015. The development and psychometric properties of LIWC2015. Technical Report.
[28]
Anupama Ray, Siddharth Kumar, Rutvik Reddy, Prerana Mukherjee, and Ritu Garg. 2019. Multi-level attention network using text, audio and video for depression prediction. In 9th AVEC challenge. 81–88.
[29]
Klaus R Scherer. 1979. Nonlinguistic vocal indicators of emotion and psychopathology. Emotions in personality and psychopathology (1979), 493–529.
[30]
Orli S Schwartz, Paul Dudgeon, Lisa B Sheeber, Marie BH Yap, Julian G Simmons, and Nicholas B Allen. 2012. Parental behaviors during family interactions predict changes in depression and anxiety symptoms during adolescence. Journal of abnormal child psychology 40, 1 (2012), 59–71.
[31]
Nadee Seneviratne and Carol Espy-Wilson. 2022. Multimodal depression classification using articulatory coordination features and hierarchical attention based text embeddings. In ICASSP. IEEE, 6252–6256.
[32]
Jarne Verhaeghe, Jeroen Van Der Donckt, Femke Ongenae, and Sofie Van Hoecke. 2022. Powershap: A power-full shapley feature selection method. In Joint European Conference on Machine Learning and Knowledge Discovery in Databases. Springer, 71–87.
[33]
Torsten Wörtwein, Lisa B Sheeber, Nicholas Allen, Jeffrey F Cohn, and Louis-Philippe Morency. 2021. Human-Guided Modality Informativeness for Affective States. In ACM ICMI. 728–734.
[34]
Ying Yang, Catherine Fairbairn, and Jeffrey F Cohn. 2012. Detecting depression severity from vocal prosody. IEEE transactions on affective computing 4, 2 (2012), 142–150.
Index Terms
- SHAP-based Prediction of Mother's History of Depression to Understand the Influence on Child Behavior
Recommendations
Media use and Internet addiction in adult depression
The present case-control study explored tendencies of Internet addiction in a group of depressive patients compared with a control group of healthy persons. Standardized questionnaires were used to assess the extent of Internet addiction (ISS), ...
Gender differences in the relationship between internet addiction and depression
The present study explored the role of gender in the association between Internet addiction and depression. Three-wave longitudinal panel data were collected from self-reported questionnaires that were completed by 1715 adolescents in grades 6-8 in ...
Comments
Information & Contributors
Information
Published In
October 2023
858 pages
ISBN:9798400700552
DOI:10.1145/3577190
Copyright © 2023 ACM.
Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than the author(s) must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected].
Sponsors
Publisher
Association for Computing Machinery
New York, NY, United States
Publication History
Published: 09 October 2023
Check for updates
Author Tags
Qualifiers
- Research-article
- Research
- Refereed limited
Funding Sources
Conference
ICMI '23
Sponsor:
Acceptance Rates
Overall Acceptance Rate 453 of 1,080 submissions, 42%
Contributors
Other Metrics
Bibliometrics & Citations
Bibliometrics
Article Metrics
- 0Total Citations
- 373Total Downloads
- Downloads (Last 12 months)341
- Downloads (Last 6 weeks)57
Reflects downloads up to 12 Nov 2024
Other Metrics
Citations
View Options
View options
View or Download as a PDF file.
PDFeReader
View online with eReader.
eReaderHTML Format
View this article in HTML Format.
HTML FormatGet Access
Login options
Check if you have access through your login credentials or your institution to get full access on this article.
Sign in