research-article

Open access

Semantic Gap in Predicting Mental Wellbeing through Passive Sensing

Authors:

Vedant Das Swain,

Stephen M. Mattingly,

Gregory D. Abowd, and

Munmun De ChoudhuryAuthors Info & Claims

CHI '22: Proceedings of the 2022 CHI Conference on Human Factors in Computing Systems

April 2022

Article No.: 374, Pages 1 - 16

https://doi.org/10.1145/3491102.3502037

Published: 29 April 2022 Publication History

All formats PDF

Abstract

When modeling passive data to infer individual mental wellbeing, a common source of ground truth is self-reports. But these tend to represent the psychological facet of mental states, which might not align with the physiological facet of that state. Our paper demonstrates that when what people “feel” differs from what people “say they feel”, we witness a semantic gap that limits predictions. We show that predicting mental wellbeing with passive data (offline sensors or online social media) is related to how the ground-truth is measured (objective arousal or self-report). Features with psycho-social signals (e.g., language) were better at predicting self-reported anxiety and stress. Conversely, features with behavioral signals (e.g., sleep), were better at predicting stressful arousal. Regardless of the source of ground truth, integrating both signals boosted prediction. To reduce the semantic gap, we provide recommendations to evaluate ground truth measures and adopt parsimonious sensing.

Supplementary Material

MP4 File (3491102.3502037-talk-video.mp4)

Talk Video

Download
11.53 MB

References

[1]

Leona S Aiken and Stephen G West. 1990. Invalidity of true experiments: Self-report pretest biases. Evaluation review 14, 4 (1990), 374–390.

[2]

Antti Airola, Tapio Pahikkala, Willem Waegeman, Bernard De Baets, and Tapio Salakoski. 2011. An experimental comparison of cross-validation techniques for estimating the area under the ROC curve. Computational Statistics & Data Analysis 55, 4 (2011), 1828–1844.

Digital Library

[3]

Saima Aman and Stan Szpakowicz. 2008. Using roget’s thesaurus for fine-grained emotion recognition. In Proceedings of the Third International Joint Conference on Natural Language Processing: Volume-I. Asian Federation of Natural Language Processing, 312–318.

[4]

Silvio Amir, Glen Coppersmith, Paula Carvalho, Mario J Silva, and Byron C Wallace. 2017. Quantifying mental health from social media with neural user embeddings. arXiv preprint arXiv:1705.00335(2017).

[5]

Aris Anagnostopoulos, Ravi Kumar, and Mohammad Mahdian. 2008. Influence and correlation in social networks. In Proceedings of the 14th ACM SIGKDD international conference on Knowledge discovery and data mining. ACM, 7–15.

Digital Library

[6]

Mark H Anshel. 1996. Effect of chronic aerobic exercise and progressive relaxation on motor performance and affect following acute stress. Behavioral Medicine 21, 4 (1996), 186–196.

[7]

Activity Recognition API. 2018. https://developers.google.com/location-context/activity-recognition/. Accessed: 2018-11-01.

[8]

Garmin Health API. 2018. http://developer.garmin.com/health-api/overview/. Accessed: 2018-11-01.

[9]

Manager REST API. 2018. https://docs.gimbal.com/rest.html. Accessed: 2018-11-01.

[10]

Dror Ben-Zeev, Emily A Scherer, Rui Wang, Haiyi Xie, and Andrew T Campbell. 2015. Next-generation psychiatric assessment: Using smartphone sensors to monitor behavior and mental health.Psychiatric rehabilitation journal 38, 3 (2015), 218.

[11]

Stuart JH Biddle and Kenneth R Fox. 2003. The way forward for physical activity and the promotion of psychological well-being. In Physical activity and psychological well-being. Routledge, 166–173.

[12]

Mehrab Bin Morshed, Koustuv Saha, Richard Li, Sidney K. D’Mello, Munmun De Choudhury, Gregory D Abowd, and Thomas Plötz. 2019. Prediction of Mood Instability with Passive Sensing. Proceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies 3, 3 (2019).

Digital Library

[13]

Daniel Bone, Matthew S Goodwin, Matthew P Black, Chi-Chun Lee, Kartik Audhkhasi, and Shrikanth Narayanan. 2015. Applying machine learning to facilitate autism diagnostics: pitfalls and promises. Journal of autism and developmental disorders 45, 5 (2015), 1121–1136.

[14]

Mehdi Boukhechba, Yu Huang, Philip Chow, Karl Fua, Bethany A Teachman, and Laura E Barnes. 2017. Monitoring social anxiety from mobility and communication patterns. In Proceedings of the 2017 ACM International Joint Conference on Pervasive and Ubiquitous Computing and Proceedings of the 2017 ACM International Symposium on Wearable Computers. ACM, 749–753.

Digital Library

[15]

Danah Boyd and Kate Crawford. 2012. Critical questions for big data: Provocations for a cultural, technological, and scholarly phenomenon. Information, communication & society 15, 5 (2012), 662–679.

[16]

Louis Bucciarelli. 2003. Engineering philosophy. DUP Satellite; an imprint of Delft University Press.

[17]

Michelle Nicole Burns, Mark Begale, Jennifer Duffecy, Darren Gergle, Chris J Karr, Emily Giangrande, and David C Mohr. 2011. Harnessing context sensing to develop a mobile intervention for depression. Journal of medical Internet research 13, 3 (2011), e55.

[18]

Andrew T Campbell, Shane B Eisenman, Nicholas D Lane, Emiliano Miluzzo, and Ronald A Peterson. 2006. People-centric urban sensing. In Proceedings of the 2nd annual international workshop on Wireless internet. ACM, 18.

Digital Library

[19]

Luca Canzian and Mirco Musolesi. 2015. Trajectories of depression: unobtrusive monitoring of depressive states by means of smartphone mobility traces analysis. In Proceedings of the 2015 ACM international joint conference on pervasive and ubiquitous computing. ACM, 1293–1304.

Digital Library

[20]

Larry Chan, Vedant Das Swain, Christina Kelley, Kaya de Barbaro, Gregory D Abowd, and Lauren Wilcox. 2018. Students’ Experiences with Ecological Momentary Assessment Tools to Report on Emotional Well-being. Proceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies 2, 1 (2018), 3.

Digital Library

[21]

Tianqi Chen, Tong He, Michael Benesty, Vadim Khotilovich, and Yuan Tang. 2015. Xgboost: extreme gradient boosting. R package version 0.4-2(2015), 1–4.

[22]

Farhan Asif Chowdhury, Yozen Liu, Koustuv Saha, Nicholas Vincent, Leonardo Neves, Neil Shah, and Maarten W Bos. 2021. Modeling Cyclic and Ephemeral User Behavior on Social Platforms. In Proceedings of the International AAAI Conference on Web and Social Media.

[23]

Matteo Ciman and Katarzyna Wac. 2016. Individuals’ stress assessment using human-smartphone interaction analysis. IEEE Transactions on Affective Computing 9, 1 (2016), 51–65.

[24]

Glen Coppersmith, Craig Harman, and Mark Dredze. 2014. Measuring post traumatic stress disorder in Twitter. In Eighth international AAAI conference on weblogs and social media.

[25]

Sauvik Das and Adam Kramer. 2013. Self-censorship on Facebook. In Seventh international AAAI conference on weblogs and social media.

[26]

Vedant Das Swain, Manikanta D. Reddy, Kari Anne Nies, Louis Tay, Munmun De Choudhury, and Gregory D. Abowd. 2019. Birds of a Feather Clock Together: A Study of Person–Organization Fit Through Latent Activity Routines. Proc. ACM Hum.-Comput. InteractCSCW (2019).

[27]

Vedant Das Swain, Koustuv Saha, Gregory D Abowd, and Munmun De Choudhury. 2020. Social Media and Ubiquitous Technologies for Remote Worker Wellbeing and Productivity in a Post-Pandemic World. In 2020 IEEE Second International Conference on Cognitive Machine Intelligence (CogMI). IEEE, 121–130.

[28]

Heather M Davey, Alexandra L Barratt, Phyllis N Butow, and Jonathan J Deeks. 2007. A one-item question with a Likert or Visual Analog Scale adequately measured current anxiety. Journal of clinical epidemiology 60, 4 (2007), 356–360.

[29]

Firstbeat Analytics All day Stress & Recovery. 2021. https://www.firstbeatanalytics.com/en/features/all-day-stress-recovery/. (2021).

[30]

Munmun De Choudhury, Michael Gamon, Scott Counts, and Eric Horvitz. 2013. Predicting depression via social media. In Seventh international AAAI conference on weblogs and social media.

[31]

Norman K Denzin. 2012. Triangulation 2.0. Journal of mixed methods research 6, 2 (2012), 80–88.

[32]

Trinh Minh Tri Do and Daniel Gatica-Perez. 2014. Where and what: Using smartphones to predict next locations and applications in daily life. Pervasive and Mobile Computing 12 (2014), 79–91.

[33]

Pedro Domingos. 1998. Occam’s two razors: The sharp and the blunt. In KDD. 37–43.

[34]

Jon D Elhai, Robert D Dvorak, Jason C Levine, and Brian J Hall. 2017. Problematic smartphone use: A conceptual overview and systematic review of relations with anxiety and depression psychopathology. Journal of affective disorders 207 (2017), 251–259.

[35]

Jane Elith, John R Leathwick, and Trevor Hastie. 2008. A working guide to boosted regression trees. Journal of Animal Ecology 77, 4 (2008), 802–813.

[36]

Sindhu Kiranmai Ernala, Michael L Birnbaum, Kristin A Candan, Asra F Rizvi, William A Sterling, John M Kane, and Munmun De Choudhury. 2019. Methodological gaps in predicting mental health states from social media: Triangulating diagnostic signals. In Proceedings of the 2019 CHI Conference on Human Factors in Computing Systems. ACM, 134.

Digital Library

[37]

Sindhu Kiranmai Ernala, Tristan Labetoulle, Fred Bane, Michael L Birnbaum, Asra F Rizvi, John M Kane, and Munmun De Choudhury. 2018. Characterizing audience engagement and assessing its impact on social media disclosures of mental illnesses. In Twelfth International AAAI Conference on Web and Social Media.

[38]

Sindhu Kiranmai Ernala, Asra F Rizvi, Michael L Birnbaum, John M Kane, and Munmun De Choudhury. 2017. Linguistic markers indicating therapeutic outcomes of social media disclosures of schizophrenia. Proceedings of the ACM on Human-Computer Interaction 1, CSCW(2017), 43.

Digital Library

[39]

Emre Ertin, Nathan Stohs, Santosh Kumar, Andrew Raij, Mustafa Al’Absi, and Siddharth Shah. 2011. AutoSense: unobtrusively wearable sensor suite for inferring the onset, causality, and consequences of stress in the field. In Proceedings of the 9th ACM Conference on Embedded Networked Sensor Systems. 274–287.

Digital Library

[40]

Deborah Estrin. 2014. Small data, where n= me. Commun. ACM 57, 4 (2014), 32–34.

Digital Library

[41]

George S Everly and Jeffrey M Lating. 2019. The anatomy and physiology of the human stress response. In A clinical guide to the treatment of the human stress response. Springer, 19–56.

[42]

Raihana Ferdous, Venet Osmani, and Oscar Mayora. 2018. Smartphone apps usage patterns as a predictor of perceived stress levels at workplace. arXiv preprint arXiv:1803.03863(2018).

[43]

Kenneth R Foster, Robert Koprowski, and Joseph D Skufca. 2014. Machine learning, medical diagnosis, and biomedical engineering research-commentary. Biomedical engineering online 13, 1 (2014), 94.

[44]

Asma Ghandeharioun and Rosalind Picard. 2017. BrightBeat: effortlessly influencing breathing for cultivating calmness and focus. In Proceedings of the 2017 CHI Conference Extended Abstracts on Human Factors in Computing Systems. ACM, 1624–1631.

Digital Library

[45]

Martin Gjoreski, Mitja Luštrek, Matjaž Gams, and Hristijan Gjoreski. 2017. Monitoring stress with a wrist device using context. Journal of biomedical informatics 73 (2017), 159–170.

Digital Library

[46]

Erving Goffman 1978. The presentation of self in everyday life. Harmondsworth London.

[47]

Mathew J Gregoski, Alexey Vertegel, Aleksey Shaporev, and Frank A Treiber. 2013. Tension Tamer: delivering meditation with objective heart rate acquisition for adherence monitoring using a smart phone platform. The Journal of Alternative and Complementary Medicine 19, 1 (2013), 17–19.

[48]

Robert M Groves and Emilia Peytcheva. 2008. The impact of nonresponse rates on nonresponse bias: a meta-analysis. Public opinion quarterly 72, 2 (2008), 167–189.

[49]

Gabriella M Harari, Nicholas D Lane, Rui Wang, Benjamin S Crosier, Andrew T Campbell, and Samuel D Gosling. 2016. Using smartphones to collect behavioral data in psychological science: Opportunities, practical considerations, and challenges. Perspectives on Psychological Science 11, 6 (2016), 838–854.

[50]

Jonathon S Hare, Paul H Lewis, Peter GB Enser, and Christine J Sandom. 2006. Mind the gap: another look at the problem of the semantic gap in image retrieval. In Multimedia Content Analysis, Management, and Retrieval 2006, Vol. 6073. International Society for Optics and Photonics, 607309.

[51]

Juliane Hellhammer and Melanie Schubert. 2012. The physiological response to Trier Social Stress Test relates to subjective measures of stress during but not before or after the test. Psychoneuroendocrinology 37, 1 (2012), 119–124.

[52]

Karen Hovsepian, Mustafa al’Absi, Emre Ertin, Thomas Kamarck, Motohiro Nakajima, and Santosh Kumar. 2015. cStress: towards a gold standard for continuous stress assessment in the mobile environment. In Proceedings of the 2015 ACM international joint conference on pervasive and ubiquitous computing. ACM, 493–504.

Digital Library

[53]

John A Johnson. 1981. The” self-disclosure” and” self-presentation” views of item response dynamics and personality scale validity.Journal of Personality and Social Psychology 40, 4(1981), 761.

[54]

Timothy P Johnson and Fons JR Van de Vijver. 2003. Social desirability in cross-cultural research. Cross-cultural survey methods 325 (2003), 195–204.

[55]

Victor Jupp. 2006. The Sage dictionary of social research methods. Sage.

[56]

Alexandra Kavushansky, Dorit Ben-Shachar, Gal Richter-Levin, and Ehud Klein. 2009. Physical stress differs from psychosocial stress in the pattern and time-course of behavioral responses, serum corticosterone and expression of plasticity-related genes in the rat. Stress 12, 5 (2009), 412–425.

[57]

Gina Leigh Kraft and Rachel A Roberts. 2017. Validation of the Garmin Forerunner 920 XT Fitness Watch VO 2 peak Test. Int. J. Innov. Educ. Res 5 (2017), 62–67.

[58]

Ivar Krumpal. 2013. Determinants of social desirability bias in sensitive surveys: a literature review. Quality & Quantity 47, 4 (2013), 2025–2047.

[59]

Vivienne MHCJ Lahaut, Harrie AM Jansen, Dike Van de Mheen, and Henk FL Garretsen. 2002. Non-response bias in a sample survey on alcohol consumption. Alcohol and Alcoholism 37, 3 (2002), 256–260.

[60]

David Lazer and Jason Radford. 2017. Data ex machina: introduction to big data. Annual Review of Sociology 43 (2017), 19–39.

[61]

Eun-Hyun Lee. 2012. Review of the psychometric evidence of the perceived stress scale. Asian nursing research 6, 4 (2012), 121–127.

[62]

Uichin Lee, Joonwon Lee, Minsam Ko, Changhun Lee, Yuhwan Kim, Subin Yang, Koji Yatani, Gahgene Gweon, Kyong-Mee Chung, and Junehwa Song. 2014. Hooked on smartphones: an exploratory study on smartphone overuse among college students. In Proceedings of the 32nd annual ACM conference on Human factors in computing systems. ACM, 2327–2336.

Digital Library

[63]

Aubrey Lewis. 1970. The ambiguous word” anxiety”.International journal of psychiatry 9 (1970), 62–79.

[64]

Andy Liaw, Matthew Wiener, 2002. Classification and regression by randomForest. R news 2, 3 (2002), 18–22.

[65]

Huijie Lin, Jia Jia, Quan Guo, Yuanyuan Xue, Qi Li, Jie Huang, Lianhong Cai, and Ling Feng. 2014. User-level psychological stress detection from social media using deep neural network. In Proceedings of the 22nd ACM international conference on Multimedia. ACM, 507–516.

Digital Library

[66]

Christopher Manning, Mihai Surdeanu, John Bauer, Jenny Finkel, Steven Bethard, and David McClosky. 2014. The Stanford CoreNLP natural language processing toolkit. In Proceedings of 52nd annual meeting of the association for computational linguistics: system demonstrations. 55–60.

[67]

Gloria Mark, Shamsi Iqbal, Mary Czerwinski, and Paul Johns. 2014. Capturing the mood: facebook and face-to-face encounters in the workplace. In Proceedings of the 17th ACM conference on Computer supported cooperative work & social computing. ACM, 1082–1094.

Digital Library

[68]

Abraham H Maslow. 1966. The psychology of science a reconnaissance. (1966).

[69]

Stephen M. Mattingly, Julie M. Gregg, Pino Audia, Ayse Elvan Bayraktaroglu, Andrew T. Campbell, Nitesh V. Chawla, Vedant Das Swain, Munmun De Choudhury, Sidney K. D’Mello, Anind K. Dey, Ge Gao, Krithika Jagannath, Kaifeng Jiang, Suwen Lin, Qiang Liu, Gloria Mark, Gonzalo J. Martinez, Kizito Masaba, Shayan Mirjafari, Edward Moskal, Raghu Mulukutla, Kari Nies, Manikanta D. Reddy, Pablo Robles-Granda, Koustuv Saha, Anusha Sirigiri, and Aaron Striegel. 2019. The Tesserae Project: Large-Scale, Longitudinal, In Situ, Multimodal Sensing of Information Workers. In Extended Abstracts of the 2019 CHI Conference on Human Factors in Computing Systems(CHI EA ’19). 8 pages. https://doi.org/10.1145/3290607.3299041

Digital Library

[70]

Abhinav Mehrotra and Mirco Musolesi. 2018. Using autoencoders to automatically extract mobility features for predicting depressive states. Proceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies 2, 3 (2018), 127.

Digital Library

[71]

Abhinav Mehrotra, Fani Tsapeli, Robert Hendley, and Mirco Musolesi. 2017. MyTraces: Investigating correlation and causation between users’ emotional states and mobile phone interaction. Proceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies 1, 3 (2017), 83.

Digital Library

[72]

Shayan Mirjafari, Kizito Masaba, Ted Grover, Weichen Wang, Pino Audia, Andrew T Campbell, Nitesh V Chawla, Vedant Das Swain, Munmun De Choudhury, Anind K Dey, 2019. Differentiating Higher and Lower Job Performers in the Workplace Using Mobile Sensing. Proceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies 3, 2 (2019), 37.

Digital Library

[73]

Larry W Morris and Robert M Liebert. 1970. Relationship of cognitive and emotional components of test anxiety to physiological arousal and academic performance.Journal of consulting and clinical psychology 35, 3(1970), 332.

[74]

Mehrab Bin Morshed, Koustuv Saha, Richard Li, Sidney K D’Mello, Munmun De Choudhury, Gregory D Abowd, and Thomas Plötz. 2019. Prediction of Mood Instability with Passive Sensing. Proceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies 3, 3 (2019), 75.

Digital Library

[75]

Amir Muaremi, Bert Arnrich, and Gerhard Tröster. 2013. Towards measuring stress with smartphones and wearable devices during workday and sleep. BioNanoScience 3, 2 (2013), 172–183.

[76]

Karla Klein Murdock. 2013. Texting while stressed: Implications for students’ burnout, sleep, and well-being.Psychology of Popular Media Culture 2, 4 (2013), 207.

[77]

MD Nefzger and James Drasgow. 1957. The needless assumption of normality in Pearson’s r.American Psychologist 12, 10 (1957), 623.

[78]

Jukka-Pekka Onnela and Scott L Rauch. 2016. Harnessing smartphone-based digital phenotyping to enhance behavioral and mental health. Neuropsychopharmacology 41, 7 (2016), 1691.

[79]

James W Pennebaker, Matthias R Mehl, and Kate G Niederhoffer. 2003. Psychological aspects of natural language use: Our words, our selves. Annual review of psychology 54, 1 (2003), 547–577.

[80]

Thomas Plötz. 2021. Applying Machine Learning for Sensor Data Analysis in Interactive Systems: Common Pitfalls of Pragmatic Use and Ways to Avoid Them. ACM Computing Surveys (CSUR) 54, 6 (2021), 1–25.

Digital Library

[81]

Rachael Purta, Stephen Mattingly, Lixing Song, Omar Lizardo, David Hachen, Christian Poellabauer, and Aaron Striegel. 2016. Experiences measuring sleep and physical activity patterns across a large college cohort with fitbits. In Proceedings of the 2016 ACM international symposium on wearable computers. ACM, 28–35.

Digital Library

[82]

Mashfiqui Rabbi, Shahid Ali, Tanzeem Choudhury, and Ethan Berke. 2011. Passive and in-situ assessment of mental and physical well-being using mobile sensors. In Proceedings of the 13th international conference on Ubiquitous computing. ACM, 385–394.

Digital Library

[83]

Ulrike Rimmele, Roland Seiler, Bernard Marti, Petra H Wirtz, Ulrike Ehlert, and Markus Heinrichs. 2009. The level of physical activity affects adrenal and cardiovascular reactivity to psychosocial stress. Psychoneuroendocrinology 34, 2 (2009), 190–198.

[84]

John Rooksby, Alistair Morrison, and Dave Murray-Rust. 2019. Student Perspectives on Digital Phenotyping: The Acceptability of Using Smartphone Data to Assess Mental Health. In Proceedings of the 2019 CHI Conference on Human Factors in Computing Systems. ACM, 425.

Digital Library

[85]

James A Russell. 1980. A circumplex model of affect.Journal of personality and social psychology 39, 6(1980), 1161.

[86]

Avi Sadeh, Giora Keinan, and Keren Daon. 2004. Effects of stress on sleep: the moderating role of coping style.Health Psychology 23, 5 (2004), 542.

[87]

Koustuv Saha, Ayse E. Bayraktaroglu, Andrew T. Campbell, Nitesh V. Chawla, Munmun De Choudhury, Sidney K. D’Mello, Anind K. Dey, Ge Gao, Julie M. Gregg, Krithika Jagannath, Gloria Mark, Gonzalo J. Martinez, Stephen M. Mattingly, Edward Moskal, Anusha Sirigiri, Aaron Striegel, and Dong Whi Yoo. 2019. Social Media As a Passive Sensor in Longitudinal Studies of Human Behavior and Wellbeing. In Extended Abstracts of the 2019 CHI Conference on Human Factors in Computing Systems(CHI EA ’19). ACM, Article CS12, 8 pages. https://doi.org/10.1145/3290607.3299065

Digital Library

[88]

Koustuv Saha, Larry Chan, Kaya De Barbaro, Gregory D Abowd, and Munmun De Choudhury. 2017. Inferring mood instability on social media by leveraging ecological momentary assessments. Proceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies 1, 3 (2017), 95.

Digital Library

[89]

Koustuv Saha, Ted Grover, Stephen M. Mattingly, Vedant Das swain, Pranshu Gupta, Gonzalo J. Martinez, Pablo Robles-Granda, Gloria Mark, Aaron Striegel, and Munmun De Choudhury. 2021. Person-Centered Predictions of Psychological Constructs with Social Media Contextualized by Multimodal Sensing. Proc. ACM Interact. Mob. Wearable Ubiquitous Technol. 5, 1, Article 32 (mar 2021), 32 pages. https://doi.org/10.1145/3448117

Digital Library

[90]

Koustuv Saha, Manikanta D Reddy, Stephen M Mattingly, Edward Moskal, Anusha Sirigiri, and Munmun De Choudhury. 2019. LibRA : On LinkedIn based Role Ambiguity and Its Relationship with Wellbeing and Job Performance. Proc. ACM Hum.-Comput. Interact.CSCW (2019).

Digital Library

[91]

Koustuv Saha, Manikanta D Reddy, Vedant Das Swain, Julie M Gregg, Ted Grover, Suwen Lin, Gonzalo J Martinez, Stephen M Mattingly, Shayan Mirjafari, Raghu Mulukutla, [n.d.]. Imputing Missing Social Media Data Stream in Multisensor Studies of Human Behavior. linguistic analysis 19, 29 ([n. d.]), 43.

[92]

Virginia Sandulescu, Sally Andrews, David Ellis, Nicola Bellotto, and Oscar Martinez Mozos. 2015. Stress detection using wearable physiological sensors. In International Work-Conference on the Interplay Between Natural and Artificial Computation. Springer, 526–532.

[93]

Akane Sano and Rosalind W Picard. 2013. Stress recognition using wearable sensors and mobile phones. In 2013 Humaine Association Conference on Affective Computing and Intelligent Interaction. IEEE, 671–676.

Digital Library

[94]

Hillol Sarker, Matthew Tyburski, Md Mahbubur Rahman, Karen Hovsepian, Moushumi Sharmin, David H Epstein, Kenzie L Preston, C Debra Furr-Holden, Adam Milam, Inbal Nahum-Shani, 2016. Finding significant stress episodes in a discontinuous time series of rapidly varying mobile sensor data. In Proceedings of the 2016 CHI conference on human factors in computing systems. ACM, 4489–4501.

Digital Library

[95]

Hans Selye. 1976. Stress without distress. In Psychopathology of human adaptation. Springer, 137–146.

[96]

Moushumi Sharmin, Andrew Raij, David Epstien, Inbal Nahum-Shani, J Gayle Beck, Sudip Vhaduri, Kenzie Preston, and Santosh Kumar. 2015. Visualization of time-series sensor data to inform the design of just-in-time adaptive stress interventions. In Proceedings of the 2015 ACM International Joint Conference on Pervasive and Ubiquitous Computing. ACM, 505–516.

Digital Library

[97]

Graciela E Silva, James L Goodwin, Duane L Sherrill, Jean L Arnold, Richard R Bootzin, Terry Smith, Joyce A Walsleben, Carol M Baldwin, and Stuart F Quan. 2007. Relationship between reported and measured sleep times: the sleep heart health study (SHHS). Journal of Clinical Sleep Medicine 3, 06 (2007), 622–630.

[98]

Meredith M Skeels and Jonathan Grudin. 2009. When social networks cross boundaries: a case study of workplace use of facebook and linkedin. In Proceedings of the ACM 2009 international conference on Supporting group work. ACM, 95–104.

Digital Library

[99]

Arnold WM Smeulders, Marcel Worring, Simone Santini, Amarnath Gupta, and Ramesh Jain. 2000. Content-based image retrieval at the end of the early years. IEEE Transactions on Pattern Analysis & Machine Intelligence12 (2000), 1349–1380.

Digital Library

[100]

Jasper Snoek, Hugo Larochelle, and Ryan P Adams. 2012. Practical bayesian optimization of machine learning algorithms. In Advances in neural information processing systems. 2951–2959.

[101]

Marcantonio M Spada, Ana V Nikčević, Giovanni B Moneta, and Adrian Wells. 2008. Metacognition, perceived stress, and negative emotion. Personality and Individual Differences 44, 5 (2008), 1172–1181.

[102]

Charles D Spielberger. 2013. The effects of anxiety on complex learning. Anxiety and behavior(2013), 361–397.

[103]

Firstbeat Technologies Stress and Recovery White Paper. 2014. https://assets.firstbeat.com/firstbeat/uploads/2015/11/Stress-and-recovery_white-paper_20145.pdf. Accessed: 2019-04-01.

[104]

John Torous, Patrick Staples, Meghan Shanahan, Charlie Lin, Pamela Peck, Matcheri Keshavan, and Jukka-Pekka Onnela. 2015. Utilizing a personal smartphone custom app to assess the patient health questionnaire-9 (PHQ-9) depressive symptoms in patients with major depressive disorder. JMIR mental health 2, 1 (2015), e8.

[105]

Gert Van Dijck and Marc M Van Hulle. 2006. Speeding up the wrapper feature subset selection in regression by mutual information relevance and redundancy analysis. In International Conference on Artificial Neural Networks. Springer, 31–40.

Digital Library

[106]

Stephen Voida, Donald J Patterson, and Shwetak N Patel. 2014. Sensor data streams. In Ways of Knowing in HCI. Springer, 291–321.

[107]

Rui Wang, Min SH Aung, Saeed Abdullah, Rachel Brian, Andrew T Campbell, Tanzeem Choudhury, Marta Hauser, John Kane, Michael Merrill, Emily A Scherer, 2016. CrossCheck: toward passive sensing and detection of mental health changes in people with schizophrenia. In Proceedings of the 2016 ACM International Joint Conference on Pervasive and Ubiquitous Computing. 886–897.

Digital Library

[108]

Rui Wang, Fanglin Chen, Zhenyu Chen, Tianxing Li, Gabriella Harari, Stefanie Tignor, Xia Zhou, Dror Ben-Zeev, and Andrew T Campbell. 2014. StudentLife: assessing mental health, academic performance and behavioral trends of college students using smartphones. In Ubicomp. ACM, 3–14.

[109]

Wenbo Wang, Lu Chen, Krishnaprasad Thirunarayan, and Amit P Sheth. 2012. Harnessing twitter” big data” for automatic emotion identification. In 2012 International Conference on Privacy, Security, Risk and Trust and 2012 International Confernece on Social Computing. IEEE, 587–592.

Digital Library

[110]

Travis A Wearne, Abbie Lucien, Emily M Trimmer, Jodie A Logan, JacquelineA Rushby, Emily Wilson, Michaela Filipčíková, and Skye McDonald. 2019. Anxiety sensitivity moderates the subjective experience but not the physiological response to psychosocial stress. International Journal of Psychophysiology 141 (2019), 76–83.

[111]

Daniel A Weinberger, Gary E Schwartz, and Richard J Davidson. 1979. Low-anxious, high-anxious, and repressive coping styles: psychometric patterns and behavioral and physiological responses to stress.Journal of abnormal psychology 88, 4 (1979), 369.

[112]

Howard M Weiss. 2002. Deconstructing job satisfaction: Separating evaluations, beliefs and affective experiences. Human resource management review 12, 2 (2002), 173–194.

[113]

J Mark G Williams, Thorsten Barnhofer, Catherine Crane, Dirk Herman, Filip Raes, Ed Watkins, and Tim Dalgleish. 2007. Autobiographical memory specificity and emotional disorder.Psychological bulletin 133, 1 (2007), 122.

[114]

Firstbeat Technologies Automated Fitness Level (VO2max) Estimation with Heart Rate and Speed Data. 2014. https://hubloq.net/cloud/runlog/diverse/white_paper_VO2max_11-11-20142.pdf. (2014).

[115]

Svante Wold, Kim Esbensen, and Paul Geladi. 1987. Principal component analysis. Chemometrics and intelligent laboratory systems 2, 1-3 (1987), 37–52.

[116]

Xuhai Xu, Prerna Chikersal, Afsaneh Doryab, Daniella K Villalba, Janine M Dutcher, Michael J Tumminia, Tim Althoff, Sheldon Cohen, Kasey G Creswell, J David Creswell, 2019. Leveraging routine behavior and contextually-filtered features for depression detection among college students. Proceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies 3, 3 (2019), 1–33.

Digital Library

[117]

Cha Zhang and Yunqian Ma. 2012. Ensemble machine learning: methods and applications. Springer.

[118]

Rhong Zhao and William I Grosky. 2002. Bridging the semantic gap in image retrieval. In Distributed multimedia databases: Techniques and applications. IGI Global, 14–36.

Cited By

Khoo LLim MChong CMcNaney R(2024)Machine Learning for Multimodal Mental Health Detection: A Systematic Review of Passive Sensing ApproachesSensors10.3390/s2402034824:2(348)Online publication date: 6-Jan-2024
https://doi.org/10.3390/s24020348
Das Swain VYe JRamesh SMondal AAbowd GDe Choudhury M(2024)Leveraging Social Media to Predict COVID-19–Induced Disruptions to Mental Well-Being Among University Students: Modeling StudyJMIR Formative Research10.2196/523168(e52316)Online publication date: 25-Jun-2024
https://doi.org/10.2196/52316
Evans HRyu MHsieh TZhou JXu KAkers KSherrill AArriaga R(2024)Using Sensor-Captured Patient-Generated Data to Support Clinical Decision-making in PTSD TherapyProceedings of the ACM on Human-Computer Interaction10.1145/36374268:CSCW1(1-28)Online publication date: 26-Apr-2024
https://dl.acm.org/doi/10.1145/3637426
Show More Cited By

Index Terms

Semantic Gap in Predicting Mental Wellbeing through Passive Sensing
1. Applied computing
  1. Law, social and behavioral sciences
    1. Psychology
2. Human-centered computing
  1. Ubiquitous and mobile computing
    1. Empirical studies in ubiquitous and mobile computing

Recommendations

Social Media as a Passive Sensor in Longitudinal Studies of Human Behavior and Wellbeing
CHI EA '19: Extended Abstracts of the 2019 CHI Conference on Human Factors in Computing Systems

Social media serves as a platform to share thoughts and connect with others. The ubiquitous use of social media also enables researchers to study human behavior as the data can be collected in an inexpensive and unobtrusive way. Not only does social ...
Read More
Designing for mental wellbeing: towards a more holistic approach in the treatment and prevention of mental illness
British HCI '15: Proceedings of the 2015 British HCI Conference

To date, HCI for mental health has primarily responded to challenges in the treatment of mental illness, with a focus on therapy access and engagement. However, approaches to improving and protecting people's mental wellbeing have received less ...
Read More
A school survey on use and knowledge of apps and websites for promoting mental wellbeing
HCI '18: Proceedings of the 32nd International BCS Human Computer Interaction Conference

Mental wellbeing is the combination of feeling good and functioning well. Digital technology widens the opportunities for promoting mental wellbeing, particularly among those young people for whom technology is an ordinary part of life. This paper ...
Read More

Comments

Information & Contributors

Information

Published In

cover image ACM Conferences

CHI '22: Proceedings of the 2022 CHI Conference on Human Factors in Computing Systems

April 2022

10459 pages

ISBN:9781450391573

DOI:10.1145/3491102

Editors:
Simone Barbosa
PUC-Rio, Brazil
,
Cliff Lampe
University of Michigan, USA
,
Caroline Appert
Université Paris-Saclay, France
,
David A. Shamma
Toyota Research Institute, USA
,
Steven Drucker
Microsoft Research, USA
,
Julie Williamson
University of Glasgow, UK
,
Koji Yatani
University of Tokyo, Japan

Copyright © 2022 ACM.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

Sponsors

SIGCHI: ACM Special Interest Group on Computer-Human Interaction

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 29 April 2022

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Badges

Honorable Mention

Author Tags

Qualifiers

Research-article
Research
Refereed limited

Funding Sources

Intelligence Advanced Research Projects Activity

Conference

CHI '22

Sponsor:

SIGCHI

CHI '22: CHI Conference on Human Factors in Computing Systems

April 29 - May 5, 2022

LA, New Orleans, USA

Acceptance Rates

Overall Acceptance Rate 6,199 of 26,314 submissions, 24%

Upcoming Conference

CHI PLAY '24

Sponsor:
sigchi

The Annual Symposium on Computer-Human Interaction in Play

October 14 - 17, 2024

Tampere , Finland

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

13
Total Citations
View Citations
1,094
Total Downloads

Downloads (Last 12 months)391
Downloads (Last 6 weeks)34

Other Metrics

View Author Metrics

Citations

Cited By

Khoo LLim MChong CMcNaney R(2024)Machine Learning for Multimodal Mental Health Detection: A Systematic Review of Passive Sensing ApproachesSensors10.3390/s2402034824:2(348)Online publication date: 6-Jan-2024
https://doi.org/10.3390/s24020348
Das Swain VYe JRamesh SMondal AAbowd GDe Choudhury M(2024)Leveraging Social Media to Predict COVID-19–Induced Disruptions to Mental Well-Being Among University Students: Modeling StudyJMIR Formative Research10.2196/523168(e52316)Online publication date: 25-Jun-2024
https://doi.org/10.2196/52316
Evans HRyu MHsieh TZhou JXu KAkers KSherrill AArriaga R(2024)Using Sensor-Captured Patient-Generated Data to Support Clinical Decision-making in PTSD TherapyProceedings of the ACM on Human-Computer Interaction10.1145/36374268:CSCW1(1-28)Online publication date: 26-Apr-2024
https://dl.acm.org/doi/10.1145/3637426
Pendse SMassachi TMahdavimoghaddam JButler JSuh JCzerwinski M(2024)Towards Inclusive Futures for Worker WellbeingProceedings of the ACM on Human-Computer Interaction10.1145/36374148:CSCW1(1-32)Online publication date: 26-Apr-2024
https://dl.acm.org/doi/10.1145/3637414
Das Swain VGao LMondal AAbowd GDe Choudhury M(2024)Sensible and Sensitive AI for Worker Wellbeing: Factors that Inform Adoption and Resistance for Information WorkersProceedings of the CHI Conference on Human Factors in Computing Systems10.1145/3613904.3642716(1-30)Online publication date: 11-May-2024
https://dl.acm.org/doi/10.1145/3613904.3642716
Nghiem JAdler DEstrin DLivesey CChoudhury T(2023)Understanding Mental Health Clinicians’ Perceptions and Concerns Regarding Using Passive Patient-Generated Health Data for Clinical Decision-Making: Qualitative Semistructured Interview StudyJMIR Formative Research10.2196/473807(e47380)Online publication date: 10-Aug-2023
https://doi.org/10.2196/47380
Kawakami AChowdhary SIqbal SLiao QOlteanu ASuh JSaha K(2023)Sensing Wellbeing in the Workplace, Why and For Whom? Envisioning Impacts with Organizational StakeholdersProceedings of the ACM on Human-Computer Interaction10.1145/36102077:CSCW2(1-33)Online publication date: 4-Oct-2023
https://dl.acm.org/doi/10.1145/3610207
Chancellor SFeuston JChang J(2023)Contextual Gaps in Machine Learning for Mental Illness Prediction: The Case of Diagnostic DisclosuresProceedings of the ACM on Human-Computer Interaction10.1145/36101817:CSCW2(1-27)Online publication date: 4-Oct-2023
https://dl.acm.org/doi/10.1145/3610181
Jörke MSefidgar YMassachi TSuh JRamos G(2023)Pearl: A Technology Probe for Machine-Assisted Reflection on Personal DataProceedings of the 28th International Conference on Intelligent User Interfaces10.1145/3581641.3584054(902-918)Online publication date: 27-Mar-2023
https://dl.acm.org/doi/10.1145/3581641.3584054
Bhattacharjee APang JLiu AMariakakis AWilliams J(2023)Design Implications for One-Way Text Messaging Services that Support Psychological WellbeingACM Transactions on Computer-Human Interaction10.1145/356988830:3(1-29)Online publication date: 10-Jun-2023
https://dl.acm.org/doi/10.1145/3569888
Show More Cited By

View Options

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

HTML Format

View this article in HTML Format.

Get Access

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

Media

Figures

Other

Tables

View Table of Contents