Location via proxy:   [ UP ]  
[Report a bug]   [Manage cookies]                
skip to main content
research-article
Public Access

EarlyScreen: Multi-scale Instance Fusion for Predicting Neural Activation and Psychopathology in Preschool Children

Published: 07 July 2022 Publication History

Abstract

Emotion dysregulation in early childhood is known to be associated with a higher risk of several psychopathological conditions, such as ADHD and mood and anxiety disorders. In developmental neuroscience research, emotion dysregulation is characterized by low neural activation in the prefrontal cortex during frustration. In this work, we report on an exploratory study with 94 participants aged 3.5 to 5 years, investigating whether behavioral measures automatically extracted from facial videos can predict frustration-related neural activation and differentiate between low- and high-risk individuals. We propose a novel multi-scale instance fusion framework to develop EarlyScreen - a set of classifiers trained on behavioral markers during emotion regulation. Our model successfully predicts activation levels in the prefrontal cortex with an area under the receiver operating characteristic (ROC) curve of 0.85, which is on par with widely-used clinical assessment tools. Further, we classify clinical and non-clinical subjects based on their psychopathological risk with an area under the ROC curve of 0.80. Our model's predictions are consistent with standardized psychometric assessment scales, supporting its applicability as a screening procedure for emotion regulation-related psychopathological disorders. To the best of our knowledge, EarlyScreen is the first work to use automatically extracted behavioral features to characterize both neural activity and the diagnostic status of emotion regulation-related disorders in young children. We present insights from mental health professionals supporting the utility of EarlyScreen and discuss considerations for its subsequent deployment.

Supplementary Material

kalanadhabhatta (kalanadhabhatta.zip)
Supplemental movie, appendix, image and software files for, EarlyScreen: Multi-scale Instance Fusion for Predicting Neural Activation and Psychopathology in Preschool Children

References

[1]
Saeed Abdullah and Tanzeem Choudhury. 2018. Sensing technologies for monitoring serious mental illnesses. IEEE MultiMedia 25, 1 (2018), 61--75.
[2]
Saeed Abdullah, Mark Matthews, Ellen Frank, Gavin Doherty, Geri Gay, and Tanzeem Choudhury. 2016. Automatic detection of social rhythms in bipolar disorder. Journal of the American Medical Informatics Association 23, 3 (2016), 538--543.
[3]
Thomas M Achenbach and Leslie A Rescorla. 2000. Manual for the ASEBA preschool forms and profiles. Vol. 30. Burlington, VT: University of Vermont, Research center for children, youth, & families.
[4]
Nadav Aharony, Wei Pan, Cory Ip, Inas Khayal, and Alex Pentland. 2011. Social fMRI: Investigating and shaping social mechanisms in the real world. Pervasive and Mobile Computing 7, 6 (2011), 643--659.
[5]
Mawulolo K Ameko, Miranda L Beltzer, Lihua Cai, Mehdi Boukhechba, Bethany A Teachman, and Laura E Barnes. 2020. Offline contextual multi-armed bandits for mobile health interventions: A case study on emotion regulation. In Fourteenth ACM Conference on Recommender Systems. 249--258.
[6]
Stuart Andrews, Ioannis Tsochantaridis, and Thomas Hofmann. 2002. Support Vector Machines for Multiple-Instance Learning. In NIPS, Vol. 2. Citeseer, 561--568.
[7]
Inge Antrop, Herbert Roeyers, Paulette Van Oost, and Ann Buysse. 2000. Stimulation seeking and hyperactivity in children with ADHD. Journal of Child Psychology and Psychiatry 41, 2 (2000), 225--231.
[8]
Shelli Avenevoli, Joseph C Blader, and Ellen Leibenluft. 2015. Irritability in Youth: An Update. Journal of the American Academy of Child and Adolescent Psychiatry 54, 11 (2015), 881.
[9]
Roger Azevedo, Michelle Taub, Nicholas V Mudrick, Garrett C Millar, Amanda E Bradbury, and Megan J Price. 2017. Using data visualizations to foster emotion regulation during self-regulated learning with advanced learning technologies. In Informational environments. Springer, 225--247.
[10]
Tadas Baltrušaitis, Marwa Mahmoud, and Peter Robinson. 2015. Cross-dataset learning and person-specific normalisation for automatic action unit detection. In 2015 11th IEEE International Conference and Workshops on Automatic Face and Gesture Recognition (FG), Vol. 6. IEEE, 1--6.
[11]
Tadas Baltrusaitis, Amir Zadeh, Yao Chong Lim, and Louis-Philippe Morency. 2018. Openface 2.0: Facial behavior analysis toolkit. In 2018 13th IEEE International Conference on Automatic Face & Gesture Recognition (FG 2018). IEEE, 59--66.
[12]
Jeffrey W Barker, Ardalan Aarabi, and Theodore J Huppert. 2013. Autoregressive model based algorithm for correcting motion and serially correlated errors in fNIRS. Biomedical optics express 4, 8 (2013), 1366--1379.
[13]
Russell A Barkley. 1997. ADHD and the nature of self-control. Guilford Press.
[14]
Ian Barnett, John Torous, Patrick Staples, Luis Sandoval, Matcheri Keshavan, and Jukka-Pekka Onnela. 2018. Relapse prediction in schizophrenia through digital phenotyping: a pilot study. Neuropsychopharmacology 43, 8 (2018), 1660--1666.
[15]
Lisa Feldman Barrett, Ralph Adolphs, Stacy Marsella, Aleix M Martinez, and Seth D Pollak. 2019. Emotional expressions reconsidered: Challenges to inferring emotion from human facial movements. Psychological science in the public interest 20, 1 (2019), 1--68.
[16]
Jennifer Beecham. 2014. Annual research review: child and adolescent mental health interventions: a review of progress in economic studies across different disorders. Journal of Child Psychology and Psychiatry 55, 6 (2014), 714--732.
[17]
Ashkan Beheshti, Mira-Lynn Chavanon, and Hanna Christiansen. 2020. Emotion dysregulation in adults with attention deficit hyperactivity disorder: a meta-analysis. BMC psychiatry 20, 1 (2020), 1--11.
[18]
Dror Ben-Zeev, Christopher J Brenner, Mark Begale, Jennifer Duffecy, David C Mohr, and Kim T Mueser. 2014. Feasibility, acceptability, and preliminary efficacy of a smartphone intervention for schizophrenia. Schizophrenia bulletin 40, 6 (2014), 1244--1253.
[19]
Yoav Benjamini and Yosef Hochberg. 1995. Controlling the false discovery rate: a practical and powerful approach to multiple testing. Journal of the Royal statistical society: series B (Methodological) 57, 1 (1995), 289--300.
[20]
Vinay Bettadapura. 2012. Face expression recognition and analysis: the state of the art. arXiv preprint arXiv:1203.6722 (2012).
[21]
J Biederman, MC Monuteaux, E Kendrick, KL Klein, and SV Faraone. 2005. The CBCL as a screen for psychiatric comorbidity in paediatric patients with ADHD. Archives of Disease in Childhood 90, 10 (2005), 1010--1015.
[22]
Robert JR Blair. 2016. The neurobiology of impulsive aggression. Journal of child and adolescent psychopharmacology 26, 1 (2016), 4--9.
[23]
Tibor Bosse and Frank PJ De Lange. 2008. Estimating emotion regulation capabilities. In Proceedings of the 1st international conference on PErvasive Technologies Related to Assistive Environments. 1--8.
[24]
Elizabeth H Bringewatt and Elizabeth T Gershoff. 2010. Falling through the cracks: Gaps and barriers in the mental health system for America's disadvantaged children. Children and Youth Services Review 32, 10 (2010), 1291--1299.
[25]
Rebecca A Burwell and Stephen R Shirk. 2007. Subtypes of rumination in adolescence: Associations between brooding, reflection, depressive symptoms, and coping. Journal of Clinical Child and Adolescent Psychology 36, 1 (2007), 56--65.
[26]
Susan D Calkins, Susan E Dedmon, Kathryn L Gill, Laura E Lomax, and Laura M Johnson. 2002. Frustration in infancy: Implications for emotion regulation, physiological processes, and temperament. Infancy 3, 2 (2002), 175--197.
[27]
Luca Canzian and Mirco Musolesi. 2015. Trajectories of depression: unobtrusive monitoring of depressive states by means of smartphone mobility traces analysis. In Proceedings of the 2015 ACM international joint conference on pervasive and ubiquitous computing. 1293--1304.
[28]
Marc-André Carbonneau, Veronika Cheplygina, Eric Granger, and Ghyslain Gagnon. 2018. Multiple instance learning: A survey of problem characteristics and applications. Pattern Recognition 77 (2018), 329--353.
[29]
Yixin Chen, Jinbo Bi, and James Ze Wang. 2006. MILES: Multiple-instance learning via embedded instance selection. IEEE Transactions on Pattern Analysis and Machine Intelligence 28, 12 (2006), 1931--1947.
[30]
Veronika Cheplygina, David MJ Tax, and Marco Loog. 2015. Multiple instance learning with bag dissimilarities. Pattern recognition 48, 1 (2015), 264--275.
[31]
Chul Woo Cho, Ji Woo Lee, Kwang Yong Shin, Eui Chul Lee, Kang Ryoung Park, Heekyung Lee, and Jihun Cha. 2012. Gaze Detection by Wearable Eye-Tracking and NIR LED-Based Head-Tracking Device Based on SVR. Etri Journal 34, 4 (2012), 542--552.
[32]
Emil F Coccaro, Chandra Sekhar Sripada, Rachel N Yanowitch, and K Luan Phan. 2011. Corticolimbic function in impulsive aggressive behavior. Biological psychiatry 69, 12 (2011), 1153--1159.
[33]
Pamela M Cole. 1986. Children's spontaneous control of facial expression. Child development (1986), 1309--1321.
[34]
Pamela M Cole, Carolyn Zahn-Waxler, Nathan A Fox, Barbara A Usher, and Jean D Welsh. 1996. Individual differences in emotion regulation and behavior problems in preschool children. Journal of Abnormal Psychology 105, 4 (1996), 518.
[35]
Jean Costa, Alexander T Adams, Malte F Jung, François Guimbretière, and Tanzeem Choudhury. 2016. EmotionCheck: leveraging bodily signals and false feedback to regulate our emotions. In Proceedings of the 2016 ACM International Joint Conference on Pervasive and Ubiquitous Computing. 758--769.
[36]
Ana Cubillo, Rozmin Halari, Anna Smith, Eric Taylor, and Katya Rubia. 2012. A review of fronto-striatal and fronto-cortical brain abnormalities in children and adults with Attention Deficit Hyperactivity Disorder (ADHD) and new evidence for dysfunction in adults with ADHD during motivation and attention. cortex 48, 2 (2012), 194--215.
[37]
Richard J Davidson, Paul Ekman, Clifford D Saron, Joseph A Senulis, and Wallace V Friesen. 1990. Approach-withdrawal and cerebral asymmetry: emotional expression and brain physiology: I. Journal of personality and social psychology 58, 2 (1990), 330.
[38]
Munmun De Choudhury, Michael Gamon, Scott Counts, and Eric Horvitz. 2013. Predicting depression via social media. Icwsm 13 (2013), 1--10.
[39]
Munmun De Choudhury, Emre Kiciman, Mark Dredze, Glen Coppersmith, and Mrinal Kumar. 2016. Discovering shifts to suicidal ideation from mental health content in social media. In Proceedings of the 2016 CHI conference on human factors in computing systems. 2098--2110.
[40]
Nuria de la Osa, Roser Granero, Esther Trepat, Josep Maria Domenech, and Lourdes Ezpeleta. 2016. The discriminative capacity of CBCL/11/2-5-DSM5 scales to identify disruptive and internalizing disorders in preschool children. European child & adolescent psychiatry 25, 1 (2016), 17--23.
[41]
Minet de Wied, Anton van Boxtel, Ruud Zaalberg, Paul P Goudena, and Walter Matthys. 2006. Facial EMG responses to dynamic emotional facial expressions in boys with disruptive behavior disorders. Journal of Psychiatric research 40, 2 (2006), 112--121.
[42]
Thomas G Dietterich, Richard H Lathrop, and Tomás Lozano-Pérez. 1997. Solving the multiple instance problem with axis-parallel rectangles. Artificial intelligence 89, 1-2 (1997), 31--71.
[43]
Hanna Drimalla, Niels Landwehr, Irina Baskow, Behnoush Behnia, Stefan Roepke, Isabel Dziobek, and Tobias Scheffer. 2018. Detecting autism by analyzing a simulated social interaction. In Joint European Conference on Machine Learning and Knowledge Discovery in Databases. Springer, 193--208.
[44]
George J DuPaul, Robert Reid, Arthur D Anastopoulos, Matthew C Lambert, Marley W Watkins, and Thomas J Power. 2016. Parent and teacher ratings of attention-deficit/hyperactivity disorder symptoms: Factor structure and normative data. Psychological Assessment 28, 2 (2016), 214.
[45]
Damien Dupré, Eva G Krumhuber, Dennis Küster, and Gary J McKeown. 2020. A performance comparison of eight commercially available automatic classifiers for facial affect recognition. Plos one 15, 4 (2020), e0231968.
[46]
Nancy Eisenberg and Richard A Fabes. 1992. Emotion, regulation, and the development of social competence. (1992).
[47]
Paul Ekman, Richard J Davidson, and Wallace V Friesen. 1990. The Duchenne smile: emotional expression and brain physiology: II. Journal of personality and social psychology 58, 2 (1990), 342.
[48]
P Ekman and WV Friesen. 1978. Facial Action Coding System (FACS): Manual. Palo Alto.
[49]
Paul Ekman and Wallace V Friesen. 1971. Constants across cultures in the face and emotion. Journal of personality and social psychology 17, 2 (1971), 124.
[50]
Jon D Elhai, Jason C Levine, and Brian J Hall. 2019. The relationship between anxiety symptom severity and problematic smartphone use: A review of the literature and conceptual frameworks. Journal of Anxiety Disorders 62 (2019), 45--52.
[51]
Charles Fage. 2015. An emotion regulation app for school inclusion of children with ASD: design principles and preliminary results for its evaluation. ACM SIGACCESS Accessibility and Computing 112 (2015), 8--15.
[52]
Tobias Fischer, Hyung Jin Chang, and Yiannis Demiris. 2018. Rt-gene: Real-time eye gaze estimation in natural environments. In Proceedings of the European Conference on Computer Vision (ECCV). 334--352.
[53]
Thomas B Fitzpatrick. 1988. The validity and practicality of sun-reactive skin types I through VI. Archives of dermatology 124, 6 (1988), 869--871.
[54]
E Friesen and Paul Ekman. 1978. Facial action coding system: a technique for the measurement of facial movement. Palo Alto 3 (1978).
[55]
Thomas Gärtner, Peter A Flach, Adam Kowalczyk, and Alexander J Smola. 2002. Multi-instance kernels. In ICML, Vol. 2. 7.
[56]
Miles Gilliom, Daniel S Shaw, Joy E Beck, Michael A Schonberg, and JoElla L Lukon. 2002. Anger regulation in disadvantaged preschool boys: Strategies, antecedents, and the development of self-control. Developmental psychology 38, 2 (2002), 222.
[57]
Adam S Grabell, Theodore J Huppert, Frank A Fishburn, Yanwei Li, Christina O Hlutkowsky, Hannah M Jones, Lauren S Wakschlag, and Susan B Perlman. 2019. Neural correlates of early deliberate emotion regulation: Young children's responses to interpersonal scaffolding. Developmental cognitive neuroscience 40 (2019), 100708.
[58]
Adam S Grabell, Theodore J Huppert, Frank A Fishburn, Yanwei Li, Hannah M Jones, Aimee E Wilett, Lisa M Bemis, and Susan B Perlman. 2018. Using facial muscular movements to understand young children's emotion regulation and concurrent neural activation. Developmental science 21, 5 (2018), e12628.
[59]
Adam S Grabell, Yanwei Li, Jeff W Barker, Lauren S Wakschlag, Theodore J Huppert, and Susan B Perlman. 2018. Evidence of non-linear associations between frustration-related prefrontal cortex activation and the normal: abnormal spectrum of irritability in young children. Journal of abnormal child psychology 46, 1 (2018), 137--147.
[60]
Paulo A Graziano, Rachael D Reavis, Susan P Keane, and Susan D Calkins. 2007. The role of emotion regulation in children's early academic success. Journal of school psychology 45, 1 (2007), 3--19.
[61]
James J Gross. 2014. Emotion regulation: Conceptual and empirical foundations. (2014).
[62]
Helen Harris and Clifford Nass. 2011. Emotion regulation for frustrating driving contexts. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems. 749--752.
[63]
Caroline Heary, Eilis Hennessy, Lorraine Swords, and Patrick Corrigan. 2017. Stigma towards mental health problems during childhood and adolescence: Theory, research and intervention approaches. Journal of Child and Family Studies 26, 11 (2017), 2949--2959.
[64]
Aaron S Heller, Regina C Lapate, Kaitlyn E Mayer, and Richard J Davidson. 2014. The face of negative affect: trial-by-trial corrugator responses to negative pictures are positively associated with amygdala and negatively associated with ventromedial prefrontal cortex activity. Journal of Cognitive Neuroscience 26, 9 (2014), 2102--2110.
[65]
Stefan G Hofmann, Alice T Sawyer, Angela Fang, and Anu Asnaani. 2012. Emotion dysregulation model of mood and anxiety disorders. Depression and anxiety 29, 5 (2012), 409--416.
[66]
Tahera Hossain, Md Shafiqul Islam, Md Atiqur Rahman Ahad, and Sozo Inoue. 2019. Human activity recognition using earable device. In Adjunct Proceedings of the 2019 ACM International Joint Conference on Pervasive and Ubiquitous Computing and Proceedings of the 2019 ACM International Symposium on Wearable Computers. 81--84.
[67]
Yu Huang, Haoyi Xiong, Kevin Leach, Yuyan Zhang, Philip Chow, Karl Fua, Bethany A Teachman, and Laura E Barnes. 2016. Assessing social anxiety using gps trajectories and point-of-interest data. In Proceedings of the 2016 ACM International Joint Conference on Pervasive and Ubiquitous Computing. 898--903.
[68]
Jeremy F Huckins, Alex W DaSilva, Rui Wang, Weichen Wang, Elin L Hedlund, Eilis I Murphy, Richard B Lopez, Courtney Rogers, Paul E Holtzheimer, William M Kelley, et al. 2019. Fusing mobile phone sensing and brain imaging to assess depression in college students. Frontiers in Neuroscience 13 (2019), 248.
[69]
James J Hudziak, William Copeland, Catherine Stanger, and Martha Wadsworth. 2004. Screening for DSM-IV externalizing disorders with the Child Behavior Checklist: a receiver-operating characteristic analysis. Journal of child psychology and psychiatry 45, 7 (2004), 1299--1307.
[70]
Daren C Jackson, Corrina J Mueller, Isa Dolski, Kim M Dalton, Jack B Nitschke, Heather L Urry, Melissa A Rosenkranz, Carol D Ryff, Burton H Singer, and Richard J Davidson. 2003. Now you feel it, now you don't: Frontal brain electrical asymmetry and individual differences in emotion regulation. Psychological science 14, 6 (2003), 612--617.
[71]
Steven L Jacques. 2013. Optical properties of biological tissues: a review. Physics in Medicine & Biology 58, 11 (2013), R37.
[72]
Sue Jamison-Powell, Conor Linehan, Laura Daley, Andrew Garbett, and Shaun Lawson. 2012. " I can't get no sleep" discussing# insomnia on twitter. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems. 1501--1510.
[73]
Herbert H Jasper. 1958. The ten-twenty electrode system of the International Federation. Electroencephalogr. Clin. Neurophysiol. 10 (1958), 370--375.
[74]
Amanda Jensen-Doss and Kristin M Hawley. 2010. Understanding barriers to evidence-based assessment: Clinician attitudes toward standardized assessment tools. Journal of Clinical Child & Adolescent Psychology 39, 6 (2010), 885--896.
[75]
Bihan Jiang, Michel F Valstar, and Maja Pantic. 2011. Action unit detection using sparse appearance descriptors in space-time video volumes. In Face and Gesture 2011. IEEE, 314--321.
[76]
Joohwan Kim, Michael Stengel, Alexander Majercik, Shalini De Mello, David Dunn, Samuli Laine, Morgan McGuire, and David Luebke. 2019. Nvgaze: An anatomically-informed dataset for low-latency, near-eye gaze estimation. In Proceedings of the 2019 CHI Conference on Human Factors in Computing Systems. 1--12.
[77]
M Justin Kim, Rebecca A Loucks, Amy L Palmer, Annemarie C Brown, Kimberly M Solomon, Ashley N Marchante, and Paul J Whalen. 2011. The structural and functional connectivity of the amygdala: from normal emotion to pathological anxiety. Behavioural brain research 223, 2 (2011), 403--410.
[78]
Michael Koenigs and Jordan Grafman. 2009. The functional neuroanatomy of depression: distinct roles for ventromedial and dorsolateral prefrontal cortex. Behavioural brain research 201, 2 (2009), 239--243.
[79]
Yubo Kou and Xinning Gui. 2020. Emotion Regulation in eSports Gaming: A Qualitative Study of League of Legends. Proceedings of the ACM on Human-Computer Interaction 4, CSCW2 (2020), 1--25.
[80]
Maria Kovacs, Joel Sherrill, Charles J George, Myrna Pollock, Rameshwari V Tumuluru, and Vincent Ho. 2006. Contextual emotion-regulation therapy for childhood depression: Description and pilot testing of a new intervention. Journal of the American Academy of Child & Adolescent Psychiatry 45, 8 (2006), 892--903.
[81]
Ellen Leibenluft, Dennis S Charney, and Daniel S Pine. 2003. Researching the pathophysiology of pediatric bipolar disorder. Biological Psychiatry 53, 11 (2003), 1009--1020.
[82]
Astar Lev, Yoram Braw, Tomer Elbaum, Michael Wagner, and Yuri Rassovsky. 2020. Eye Tracking During a Continuous Performance Test: Utility for Assessing ADHD Patients. Journal of Attention Disorders (2020), 1087054720972786.
[83]
Valentina Levantini, Pietro Muratori, Emanuela Inguaggiato, Gabriele Masi, Annarita Milone, Elena Valente, Alessandro Tonacci, and Lucia Billeci. 2020. EYES are the window to the mind: Eye-tracking technology as a novel approach to study clinical characteristics of ADHD. Psychiatry Research 290 (2020), 113135.
[84]
Deborah L Levy, Anne B Sereno, Diane C Gooding, and Gilllian A O'Driscoll. 2010. Eye tracking dysfunction in schizophrenia: characterization and pathophysiology. In Behavioral Neurobiology of Schizophrenia and Its Treatment. Springer, 311--347.
[85]
Yan Li, David MJ Tax, Robert PW Duin, and Marco Loog. 2013. Multiple-instance learning as a classifier combining problem. Pattern recognition 46, 3 (2013), 865--874.
[86]
Hong Lu, Denise Frauendorfer, Mashfiqui Rabbi, Marianne Schmid Mast, Gokul T Chittaranjan, Andrew T Campbell, Daniel Gatica-Perez, and Tanzeem Choudhury. 2012. Stresssense: Detecting stress in unconstrained acoustic environments using smartphones. In Proceedings of the 2012 ACM conference on ubiquitous computing. 351--360.
[87]
Diana MacLean, Asta Roseway, and Mary Czerwinski. 2013. MoodWings: a wearable biofeedback device for real-time stress intervention. In Proceedings of the 6th international conference on PErvasive Technologies Related to Assistive Environments. 1--8.
[88]
Eric J Mash and John Hunsley. 2005. Evidence-based assessment of child and adolescent disorders: Issues and challenges. Journal of clinical child and adolescent psychology 34, 3 (2005), 362--379.
[89]
Mark Matthews, Saeed Abdullah, Elizabeth Murnane, Stephen Voida, Tanzeem Choudhury, Geri Gay, and Ellen Frank. 2016. Development and evaluation of a smartphone-based measure of social rhythms for bipolar disorder. Assessment 23, 4 (2016), 472--483.
[90]
Addison Mayberry, Pan Hu, Benjamin Marlin, Christopher Salthouse, and Deepak Ganesan. 2014. iShadow: design of a wearable, real-time mobile gaze tracker. In Proceedings of the 12th annual international conference on Mobile systems, applications, and services. 82--94.
[91]
Katie A McLaughlin, Matthew Peverill, Andrea L Gold, Sonia Alves, and Margaret A Sheridan. 2015. Child maltreatment and neural systems underlying emotion regulation. Journal of the American Academy of Child & Adolescent Psychiatry 54, 9 (2015), 753--762.
[92]
Douglas S Mennin, Richard G Heimberg, Cynthia L Turk, and David M Fresco. 2002. Applying an emotion regulation framework to integrative approaches to generalized anxiety disorder. Clinical Psychology: Science and Practice 9, 1 (2002), 85--90.
[93]
Margaret Mitchell, Simone Wu, Andrew Zaldivar, Parker Barnes, Lucy Vasserman, Ben Hutchinson, Elena Spitzer, Inioluwa Deborah Raji, and Timnit Gebru. 2019. Model cards for model reporting. In Proceedings of the conference on fairness, accountability, and transparency. 220--229.
[94]
Philipp Mock, Maike Tibus, Ann-Christine Ehlis, Harald Baayen, and Peter Gerjets. 2018. Predicting ADHD risk from touch interaction data. In Proceedings of the 20th ACM International Conference on Multimodal Interaction. 446--454.
[95]
David C Mohr, Mi Zhang, and Stephen M Schueller. 2017. Personal sensing: understanding mental health using ubiquitous sensors and machine learning. Annual review of clinical psychology 13 (2017), 23--47.
[96]
Amir Muaremi, Franz Gravenhorst, Agnes Grünerbl, Bert Arnrich, and Gerhard Tröster. 2014. Assessing bipolar episodes using speech cues derived from phone calls. In International Symposium on Pervasive Computing Paradigms for Mental Health. Springer, 103--114.
[97]
Åsa Nilsonne. 1987. Acoustic analysis of speech variables during depression and after improvement. Acta Psychiatrica Scandinavica 76, 3 (1987), 235--245.
[98]
Mikio Obuchi, Jeremy F Huckins, Weichen Wang, Alex daSilva, Courtney Rogers, Eilis Murphy, Elin Hedlund, Paul Holtzheimer, Shayan Mirjafari, and Andrew Campbell. 2020. Predicting Brain Functional Connectivity Using Mobile Sensing. Proceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies 4, 1 (2020), 1--22.
[99]
Averil Overton, Susan Selway, Kenneth Strongman, and Michelle Houston. 2005. Eating disorders---The regulation of positive as well as negative emotion experience. Journal of Clinical Psychology in Medical Settings 12, 1 (2005), 39--56.
[100]
Asli Ozdas, Richard G Shiavi, Stephen E Silverman, Marilyn K Silverman, and D Mitchell Wilkes. 2004. Investigation of vocal jitter and glottal flow spectrum as possible cues for depression and near-term suicidal risk. IEEE Transactions on Biomedical Engineering 51, 9 (2004), 1530--1540.
[101]
Maja Pantic and Leon J. M. Rothkrantz. 2000. Automatic analysis of facial expressions: The state of the art. IEEE Transactions on pattern analysis and machine intelligence 22, 12 (2000), 1424--1445.
[102]
Pablo Paredes and Matthew Chan. 2011. CalmMeNow: exploratory research and design of stress mitigating mobile interventions. In CHI'11 Extended Abstracts on Human Factors in Computing Systems. 1699--1704.
[103]
Susan B Perlman, Beatriz Luna, Tyler C Hein, and Theodore J Huppert. 2014. fNIRS evidence of prefrontal regulation of frustration in early childhood. Neuroimage 85 (2014), 326--334.
[104]
Skyler Place, Danielle Blanch-Hartigan, Channah Rubin, Cristina Gorrostieta, Caroline Mead, John Kane, Brian P Marx, Joshua Feast, Thilo Deckersbach, Andrew Nierenberg, et al. 2017. Behavioral indicators on a mobile sensing platform predict clinically validated psychiatric symptoms of mood and anxiety disorders. Journal of medical Internet research 19, 3 (2017), e75.
[105]
Joseph S Raiker, Andrew J Freeman, Guillermo Perez-Algorta, Thomas W Frazier, Robert L Findling, and Eric A Youngstrom. 2017. Accuracy of Achenbach scales in the screening of attention-deficit/hyperactivity disorder in a community mental health clinic. Journal of the American Academy of Child & Adolescent Psychiatry 56, 5 (2017), 401--409.
[106]
Michael P Reiman, Adam P Goode, Eric J Hegedus, Chad E Cook, and Alexis A Wright. 2013. Diagnostic accuracy of clinical tests of the hip: a systematic review with meta-analysis. British journal of sports medicine 47, 14 (2013), 893--902.
[107]
Soha Rostaminia, Alexander Lamson, Subhransu Maji, Tauhidur Rahman, and Deepak Ganesan. 2019. W! NCE: Unobtrusive Sensing of Upper Facial Action Units with EOG-based Eyewear. Proceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies 3, 1 (2019), 1--26.
[108]
Carolyn Saarni. 1979. Children's understanding of display rules for expressive behavior. Developmental psychology 15, 4 (1979), 424.
[109]
Sohrab Saeb, Emily G Lattie, Stephen M Schueller, Konrad P Kording, and David C Mohr. 2016. The relationship between mobile phone location sensor data and depressive symptom severity. PeerJ 4 (2016), e2537.
[110]
Akane Sano and Rosalind W Picard. 2013. Stress recognition using wearable sensors and mobile phones. In 2013 Humaine Association Conference on Affective Computing and Intelligent Interaction. IEEE, 671--676.
[111]
Evangelos Sariyanidi, Hatice Gunes, and Andrea Cavallaro. 2014. Automatic analysis of facial affect: A survey of registration, representation, and recognition. IEEE transactions on pattern analysis and machine intelligence 37, 6 (2014), 1113--1133.
[112]
Jocelyn Scheirer, Raul Fernandez, and Rosalind W Picard. 1999. Expression glasses: a wearable device for facial expression recognition. In CHI'99 Extended Abstracts on Human Factors in Computing Systems. 262--263.
[113]
Jessica Schroeder, Chelsey Wilkes, Kael Rowan, Arturo Toledo, Ann Paradiso, Mary Czerwinski, Gloria Mark, and Marsha M Linehan. 2018. Pocket skills: A conversational mobile web app to support dialectical behavioral therapy. In Proceedings of the 2018 CHI Conference on Human Factors in Computing Systems. 1--15.
[114]
Philip Shaw, Argyris Stringaris, Joel Nigg, and Ellen Leibenluft. 2014. Emotion dysregulation in attention deficit hyperactivity disorder. American Journal of Psychiatry 171, 3 (2014), 276--293.
[115]
TO Smith, T Back, AP Toms, and CB Hing. 2011. Diagnostic accuracy of ultrasound for rotator cuff tears in adults: a systematic review and meta-analysis. Clinical radiology 66, 11 (2011), 1036--1048.
[116]
David MJ Tax, Marco Loog, Robert PW Duin, Veronika Cheplygina, and Wan-Jui Lee. 2011. Bag dissimilarities for multiple instance learning. In International Workshop on Similarity-Based Pattern Recognition. Springer, 222--234.
[117]
Alina Trifan, Maryse Oliveira, and José Luís Oliveira. 2019. Passive sensing of health outcomes through smartphones: systematic review of current solutions and possible limitations. JMIR mHealth and uHealth 7, 8 (2019), e12649.
[118]
Talia Tron, Abraham Peled, Alexander Grinsphoon, and Daphna Weinshall. 2015. Automated facial expressions analysis in schizophrenia: A continuous dynamic approach. In International Symposium on Pervasive Computing Paradigms for Mental Health. Springer, 72--81.
[119]
Heather L Urry, Carien M Van Reekum, Tom Johnstone, Ned H Kalin, Marchell E Thurow, Hillary S Schaefer, Cory A Jackson, Corrina J Frye, Lawrence L Greischar, Andrew L Alexander, et al. 2006. Amygdala and ventromedial prefrontal cortex are inversely coupled during regulation of negative affect and predict the diurnal pattern of cortisol secretion among older adults. Journal of Neuroscience 26, 16 (2006), 4415--4425.
[120]
Carien M van Reekum, Tom Johnstone, Heather L Urry, Marchell E Thurow, Hillary S Schaefer, Andrew L Alexander, and Richard J Davidson. 2007. Gaze fixations predict brain activation during the voluntary regulation of picture-induced negative affect. Neuroimage 36, 3 (2007), 1041--1055.
[121]
Lauren S Wakschlag, Seung W Choi, Alice S Carter, Heide Hullsiek, James Burns, Kimberly McCarthy, Ellen Leibenluft, and Margaret J Briggs-Gowan. 2012. Defining the developmental parameters of temper loss in early childhood: implications for developmental psychopathology. Journal of Child Psychology and Psychiatry 53, 11 (2012), 1099--1108.
[122]
Lauren S Wakschlag, Ryne Estabrook, Amelie Petitclerc, David Henry, James L Burns, Susan B Perlman, Joel L Voss, Daniel S Pine, Ellen Leibenluft, and Margaret L Briggs-Gowan. 2015. Clinical implications of a dimensional approach: the normal: abnormal spectrum of early irritability. Journal of the American Academy of Child & Adolescent Psychiatry 54, 8 (2015), 626--634.
[123]
Lauren S Wakschlag, Patrick H Tolan, and Bennett L Leventhal. 2010. Research Review:'Ain't misbehavin': Towards a developmentally-specified nosology for preschool disruptive behavior. Journal of Child Psychology and Psychiatry 51, 1 (2010), 3--22.
[124]
Sebastian Walther, Katharina Stegmayer, Helge Horn, Nadja Razavi, Thomas J Müller, and Werner Strik. 2015. Physical activity in schizophrenia is higher in the first episode than in subsequent ones. Frontiers in psychiatry 5 (2015), 191.
[125]
Jun Wang and Jean-Daniel Zucker. 2000. Solving multiple-instance problem: A lazy learning approach. (2000).
[126]
Rui Wang, Min SH Aung, Saeed Abdullah, Rachel Brian, Andrew T Campbell, Tanzeem Choudhury, Marta Hauser, John Kane, Michael Merrill, Emily A Scherer, et al. 2016. CrossCheck: toward passive sensing and detection of mental health changes in people with schizophrenia. In Proceedings of the 2016 ACM International Joint Conference on Pervasive and Ubiquitous Computing. 886--897.
[127]
Rui Wang, Weichen Wang, Alex DaSilva, Jeremy F Huckins, William M Kelley, Todd F Heatherton, and Andrew T Campbell. 2018. Tracking depression dynamics in college students using mobile phone and wearable sensing. Proceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies 2, 1 (2018), 1--26.
[128]
Erin M Warnick, Michael B Bracken, and Stanislav Kasl. 2008. Screening efficiency of the Child Behavior Checklist and Strengths and Difficulties Questionnaire: A systematic review. Child and Adolescent Mental Health 13, 3 (2008), 140--147.
[129]
Sarah Watts, Anna Mackenzie, Cherian Thomas, Al Griskaitis, Louise Mewton, Alishia Williams, and Gavin Andrews. 2013. CBT for depression: a pilot RCT comparing mobile phone vs. computer. BMC psychiatry 13, 1 (2013), 49.
[130]
Nils Weidmann, Eibe Frank, and Bernhard Pfahringer. 2003. A two-level learning method for generalized multi-instance problems. In European Conference on Machine Learning. Springer, 468--479.
[131]
Daniel G Whitney and Mark D Peterson. 2019. US national and state-level prevalence of mental health disorders and disparities of mental health care use in children. JAMA pediatrics 173, 4 (2019), 389--391.
[132]
C Winograd-Gurvich, Nellie Georgiou-Karistianis, Paul Bernard Fitzgerald, Lynette Millist, and Owen B White. 2006. Ocular motor differences between melancholic and non-melancholic depression. Journal of affective disorders 93, 1-3 (2006), 193--203.
[133]
Jia Wu, Shirui Pan, Xingquan Zhu, Chengqi Zhang, and Xindong Wu. 2018. Multi-instance learning with discriminative bag mapping. IEEE Transactions on Knowledge and Data Engineering 30, 6 (2018), 1065--1080.
[134]
Zhefan Ye, Yin Li, Alireza Fathi, Yi Han, Agata Rozga, Gregory D Abowd, and James M Rehg. 2012. Detecting eye contact using wearable eye-tracking glasses. In Proceedings of the 2012 ACM conference on ubiquitous computing. 699--704.
[135]
JungKyoon Yoon, Shuran Li, Yu Hao, and Chajoong Kim. 2019. Towards Emotional Well-Being by Design: 17 Opportunities for Emotion Regulation for User-Centered Healthcare Design. In Proceedings of the 13th EAI International Conference on Pervasive Computing Technologies for Healthcare. 351--355.
[136]
Anis Zaman, Boyu Zhang, Vincent Silenzio, Ehsan Hoque, and Henry Kautz. 2020. Estimating Anxiety based on individual level engagements on YouTube & Google Search Engine. arXiv preprint arXiv:2007.00613 (2020).
[137]
Janice Zeman and Judy Garber. 1996. Display rules for anger, sadness, and pain: It depends on who is watching. Child development 67, 3 (1996), 957--973.
[138]
Cha Zhang, John Platt, and Paul Viola. 2005. Multiple instance boosting for object detection. Advances in neural information processing systems 18 (2005), 1417--1424.
[139]
Qi Zhang and Sally A Goldman. 2001. EM-DD: An improved multiple-instance learning technique. In Advances in neural information processing systems. 1073--1080.
[140]
Zhi-Hua Zhou and Min-Ling Zhang. 2007. Solving multi-instance problems with classifier ensemble based on constructive clustering. Knowledge and information systems 11, 2 (2007), 155--170.

Cited By

View all
  • (2024)Multi-stakeholder Perspectives on Mental Health Screening Tools for ChildrenProceedings of the CHI Conference on Human Factors in Computing Systems10.1145/3613904.3642604(1-15)Online publication date: 11-May-2024
  • (2024)Developing Multi-Scale Patch Representations for Low Energy Data Aggregation Using Unit Block Building Model2024 International Conference on Optimization Computing and Wireless Communication (ICOCWC)10.1109/ICOCWC60930.2024.10470838(1-7)Online publication date: 29-Jan-2024
  • (2024)A training and assessment system for human-computer interaction combining fNIRS and eye-tracking dataAdvanced Engineering Informatics10.1016/j.aei.2024.10276562(102765)Online publication date: Oct-2024
  • Show More Cited By

Recommendations

Comments

Information & Contributors

Information

Published In

cover image Proceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies
Proceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies  Volume 6, Issue 2
July 2022
1551 pages
EISSN:2474-9567
DOI:10.1145/3547347
Issue’s Table of Contents
Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 07 July 2022
Published in IMWUT Volume 6, Issue 2

Permissions

Request permissions for this article.

Check for updates

Author Tags

  1. affective computing
  2. computer vision
  3. mental health
  4. neuroscience

Qualifiers

  • Research-article
  • Research
  • Refereed

Funding Sources

Contributors

Other Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

  • Downloads (Last 12 months)280
  • Downloads (Last 6 weeks)38
Reflects downloads up to 13 Sep 2024

Other Metrics

Citations

Cited By

View all
  • (2024)Multi-stakeholder Perspectives on Mental Health Screening Tools for ChildrenProceedings of the CHI Conference on Human Factors in Computing Systems10.1145/3613904.3642604(1-15)Online publication date: 11-May-2024
  • (2024)Developing Multi-Scale Patch Representations for Low Energy Data Aggregation Using Unit Block Building Model2024 International Conference on Optimization Computing and Wireless Communication (ICOCWC)10.1109/ICOCWC60930.2024.10470838(1-7)Online publication date: 29-Jan-2024
  • (2024)A training and assessment system for human-computer interaction combining fNIRS and eye-tracking dataAdvanced Engineering Informatics10.1016/j.aei.2024.10276562(102765)Online publication date: Oct-2024
  • (2023)Towards Accurate and Scalable Mental Health Screening Technologies for Young ChildrenAdjunct Proceedings of the 2023 ACM International Joint Conference on Pervasive and Ubiquitous Computing & the 2023 ACM International Symposium on Wearable Computing10.1145/3594739.3610763(225-229)Online publication date: 8-Oct-2023
  • (2022)Extracting Multimodal Embeddings via Supervised Contrastive Learning for Psychological Screening2022 10th International Conference on Affective Computing and Intelligent Interaction (ACII)10.1109/ACII55700.2022.9953836(1-8)Online publication date: 18-Oct-2022

View Options

View options

PDF

View or Download as a PDF file.

PDF

eReader

View online with eReader.

eReader

Get Access

Login options

Full Access

Media

Figures

Other

Tables

Share

Share

Share this Publication link

Share on social media