research-article

Deep Hierarchical Attention Active Learning for Mental Disorder Unlabeled Data in AIoMT

Authors:

Jerry Chun-Wei Lin,

Gautam SrivastavaAuthors Info & Claims

ACM Transactions on Sensor Networks, Volume 19, Issue 3

Article No.: 49, Pages 1 - 18

https://doi.org/10.1145/3519304

Published: 01 March 2023 Publication History

Abstract

In the Artificial Intelligence of Medical Things (AIoMT), Internet-Delivered Psychological Treatment (IDPT) effectively improves the quality of mental health treatments. With the advent of COVID-19, psychological tasks have become overloaded and complicated for medical professionals due to the overlap of sentimental values. The development of an AIoMT tool requires labeling of data to achieve clinical-level performance. Text data requires an appropriate set of linguistic features for vector latent representation and segmentation. Emotional biases could lead to incorrect segmentation of patient-authorized texts, and labeling emotional data is time-consuming. In this article, we propose an assistant tool for psychologists to assist them in mental health treatment and note-taking. We first extend the word and emotion lexicon and then apply a hierarchical attention method to support data labeling. The learned latent representation uses word position prediction and sentence-level attention to create a semantic framework. The augmented vector representation helps in highlighting words and classifying nine different symptoms from the text written by the patient. Our experimental results show that the emotion lexicon helps to increase the accuracy by 5% without affecting the overall results, and that the hierarchical attention method achieves an F1 score of 0.89.

References

[1]

Usman Ahmed, Jerry Chun-Wei Lin, and Gautam Srivastava. 2021. Fuzzy explainable attention-based deep active learning on mental-health data. In IEEE International Conference on Fuzzy Systems. 1–6.

Digital Library

[2]

Usman Ahmed, Suresh Kumar Mukhiya, Gautam Srivastava, Yngve Lamo, and Jerry Chun-Wei Lin. 2021. Attention-based deep entropy active learning using lexical algorithm for mental health treatment. Frontiers in Psychology 12 (2021), 471.

[3]

Xiangmao Chang, Gangkai Li, Guoliang Xing, Kun Zhu, and Linlin Tu. 2021. DeepHeart: A deep learning approach for accurate heart rate estimation from PPG signals. ACM Transactions on Sensor Networks 17, 2 (2021), 1–18.

Digital Library

[4]

Emily Chen, Kristina Lerman, and Emilio Ferrara. 2020. Tracking social media discourse about the COVID-19 pandemic: Development of a public coronavirus Twitter data set. JMIR Public Health and Surveillance 6, 2 (2020), e19273.

[5]

Xiangyi Chen, Zhiwei Steven Wu, and Mingyi Hong. 2020. Understanding gradient clipping in private SGD: A geometric perspective. In Advances in Neural Information Processing Systems 33: Annual Conference on Neural Information Processing Systems.

[6]

Kyunghyun Cho, Bart van Merrienboer, Çaglar Gülçehre, Dzmitry Bahdanau, Fethi Bougares, Holger Schwenk, and Yoshua Bengio. 2014. Learning phrase representations using RNN encoder-decoder for statistical machine translation. In The Conference on Empirical Methods in Natural Language Processing, Alessandro Moschitti, Bo Pang, and Walter Daelemans (Eds.). 1724–1734.

[7]

Munmun De Choudhury, Michael Gamon, Scott Counts, and Eric Horvitz. 2013. Predicting depression via social media. In The International Conference on Weblogs and Social Media.

[8]

Karthik Dinakar, Emily Weinstein, Henry Lieberman, and Robert Louis Selman. 2014. Stacked generalization learning to analyze teenage distress. In Proceedings of the 8th International Conference on Weblogs and Social Media.

[9]

Grant Van Horn, Oisin Mac Aodha, Yang Song, Yin Cui, Chen Sun, Alexander Shepard, Hartwig Adam, Pietro Perona, and Serge J. Belongie. 2018. The INaturalist species classification and detection dataset. In IEEE Conference on Computer Vision and Pattern Recognition. 8769–8778.

[10]

Yingren Huang, Jiaojiao Chen, Shaomin Zheng, Yun Xue, and Xiaohui Hu. 2021. Hierarchical multi-attention networks for document classification. International Journal of Machine Learning and Cybernetics 12, 6 (2021), 1639–1647.

[11]

Spencer L. James, Degu Abate, Kalkidan Hassen Abate, Solomon M. Abay, Cristiana Abbafati, Nooshin Abbasi, Hedayat Abbastabar, Foad Abd-Allah, Jemal Abdela, Ahmed Abdelalim, et al. 2018. Global, regional, and national incidence, prevalence, and years lived with disability for 354 diseases and injuries for 195 countries and territories, 1990–2017: A systematic analysis for the global burden of disease study 2017. The Lancet 392, 10159 (2018), 1789–1858.

[12]

Diederik P. Kingma and Jimmy Ba. 2014. Adam: A method for stochastic optimization. arXiv Preprint arXiv:1412.6980 (2014).

[13]

Kurt Kroenke, Robert L. Spitzer, and Janet B. W. Williams. 2001. The PHQ-9: Validity of a brief depression severity measure. Journal of General Internal Medicine 16, 9 (2001), 606–613.

[14]

Kurt Kroenke, Robert L. Spitzer, and Janet B. W. Williams. 2002. The PHQ-15: Validity of a new measure for evaluating the severity of somatic symptoms. Psychosomatic Medicine 64, 2 (2002), 258–266.

[15]

Tim M. H. Li, Michael Chau, Paul W. C. Wong, and Paul S. F. Yip. 2012. A hybrid system for online detection of emotional distress. In Intelligence and Security Informatics. 73–80.

[16]

Huijie Lin, Jia Jia, Quan Guo, Yuanyuan Xue, Qi Li, Jie Huang, Lianhong Cai, and Ling Feng. 2014. User-level psychological stress detection from social media using deep neural network. In ACM International Conference on Multimedia. ACM Press.

Digital Library

[17]

David E. Losada and Pablo Gamallo. 2018. Evaluating and improving lexical resources for detecting signs of depression in text. Language Resources and Evaluation 54, 1 (2018), 1–24.

[18]

Daniel M. Low, Laurie Rumker, Tanya Talkar, John Torous, Guillermo Cecchi, and Satrajit S. Ghosh. 2020. Natural language processing reveals vulnerable mental health support groups and heightened health anxiety on Reddit during COVID-19: Observational study. Journal of Medical Internet Research 22, 10 (2020), e22635.

[19]

Thang Luong, Hieu Pham, and Christopher D. Manning. 2015. Effective approaches to attention-based neural machine translation. In The Conference on Empirical Methods in Natural Language Processing, Lluís Màrquez, Chris Callison-Burch, Jian Su, Daniele Pighin, and Yuval Marton (Eds.). 1412–1421.

[20]

Mario Gennaro Mazza, Rebecca De Lorenzo, Caterina Conte, Sara Poletti, Benedetta Vai, Irene Bollettini, Elisa Maria Teresa Melloni, Roberto Furlan, Fabio Ciceri, Patrizia Rovere-Querini, and Francesco Benedetti. 2020. Anxiety and depression in COVID-19 survivors: Role of inflammatory and clinical predictors. Brain, Behavior, and Immunity 89 (2020), 594–600.

[21]

Michelle McDonnell, Jason Edward Owen, and Erin O'Carroll Bantum. 2020. Identification of emotional expression with cancer survivors: Validation of linguistic inquiry and word count. JMIR Formative Research 4, 10 (2020), e18246.

[22]

Armand Joulin, Edouard Grave, Piotr Bojanowski, and Tomas Mikolov. 2017. Bag of Tricks for Efficient Text Classification. In Proceedings of the 15th Conference of the European Chapter of the Association for Computational Linguistics: Volume 2, Short Papers. 427–431.

[23]

George Miller, Christiane Fellbaum, Judy Kegl, and Katherine Miller. 2009. WordNet: An electronic lexical reference system based on theories of lexical memory. Revue québécoise de linguistique 17, 2 (2009), 181–212.

[24]

Julia Mühleck, Sigrid Borse, Eva Wunderer, Bernhard Strauß, and Uwe Berger. 2019. Online-Befragung zur Bekanntheit von Angeboten zur Aufklärung, Prävention, Beratung und Nachsorge bei Essstörungen. Prävention und Gesundheitsförderung 15, 1 (2019), 73–79.

[25]

Suresh Kumar Mukhiya, Usman Ahmed, Fazle Rabbi, Ka I. Pun, and Yngve Lamo. 2020. Adaptation of IDPT system based on patient-authored text data using NLP. In IEEE International Symposium on Computer-based Medical Systems. 226–232.

[26]

Suresh Kumar Mukhiya, Jo Dugstad Wake, Yavuz Inal, Ka I. Pun, and Yngve Lamo. 2020. Adaptive elements in internet-delivered psychological treatment systems: Systematic review. Journal of Medical Internet Research 22, 11 (2020), e21066.

[27]

Yair Neuman, Yohai Cohen, Dan Assaf, and Gabbi Kedma. 2012. Proactive screening for depression through metaphorical and automatic text analysis. Artificial Intelligence in Medicine 56, 1 (2012), 19–25.

Digital Library

[28]

Antoine Neuraz, Ivan Lerner, William Digan, Nicolas Paris, Rosy Tsopra, Alice Rogier, David Baudoin, Kevin Bretonnel Cohen, Anita Burgun, Nicolas Garcelon, et al. 2020. Natural language processing for rapid response to emergent diseases: Case study of calcium channel blockers and hypertension in the COVID-19 pandemic. Journal of Medical Internet Research 22, 8 (2020), e20773.

[29]

Giang Nguyen, Stefan Dlugolinsky, Martin Bobák, Viet D. Tran, Álvaro López García, Ignacio Heredia, Peter Malík, and Ladislav Hluchý. 2019. Machine learning and deep learning frameworks and libraries for large-scale data mining: A survey. Artificial Intelligence Review 52, 1 (2019), 77–124.

Digital Library

[30]

World Health Organization et al. 1993. The ICD-10 Classification of Mental and Behavioural Disorders: Diagnostic Criteria for Research, Vol. 2.

[31]

Jeffrey Pennington, Richard Socher, and Christopher D. Manning. 2014. Glove: Global vectors for word representation. In The Conference on Empirical Methods in Natural Language Processing. 1532–1543.

[32]

Pranav Rajpurkar, Jeremy Irvin, Kaylie Zhu, Brandon Yang, Hershel Mehta, Tony Duan, Daisy Yi Ding, Aarti Bagul, Curtis Langlotz, Katie S. Shpanskaya, Matthew P. Lungren, and Andrew Y. Ng. 2017. CheXNet: Radiologist-level pneumonia detection on chest X-Rays with deep learning. CoRR abs/1711.05225 (2017).

[33]

Christopher Schröder and Andreas Niekler. 2020. A survey of active learning for text classification using deep neural networks. CoRR abs/2008.07267 (2020). arXiv:2008.07267. https://arxiv.org/abs/2008.07267.

[34]

Vivienne Sze, Yu Hsin Chen, Tien Ju Yang, and Joel S. Emer. 2017. Efficient processing of deep neural networks: A tutorial and survey. Proceedings of the IEEE 105, 12 (2017), 2295–2329.

[35]

Emily A. Troyer, Jordan N. Kohn, and Suzi Hong. 2020. Are we facing a crashing wave of neuropsychiatric sequelae of COVID-19? Neuropsychiatric symptoms and potential immunologic mechanisms. Brain, Behavior, and Immunity 87 (2020), 34–39.

[36]

Michael Wainberg, Daniele Merico, Andrew Delong, and Brendan J. Frey. 2018. Deep learning in biomedicine. Nature Biotechnology 36, 9 (2018), 829–838.

[37]

Kelvin Xu, Jimmy Ba, Ryan Kiros, Kyunghyun Cho, Aaron C. Courville, Ruslan Salakhutdinov, Richard S. Zemel, and Yoshua Bengio. 2015. Show, attend and tell: Neural image caption generation with visual attention. In Proceedings of the 32nd International Conference on Machine Learning (ICML’15)(JMLR Workshop and Conference Proceedings, Vol. 37). JMLR.org, 2048–2057.

[38]

Lingling Xu, Ruyi Jin, Feifei Huang, Yanhui Zhou, Zonglong Li, and Minqiang Zhang. 2020. Development of computerized adaptive testing for emotion regulation. Frontiers in Psychology 11 (2020), 3340.

[39]

Yifan Zhang and Xinglin Zhang. 2021. Price learning-based incentive mechanism for mobile crowd sensing. ACM Transactions on Sensor Networks 17, 2 (2021), 1–24.

Digital Library

Cited By

Shafik WTufail ALiyanage CApong R(2024)Medical Robotics and AI-Assisted Diagnostics Challenges for Smart Sustainable HealthcareAI-Driven Innovations in Digital Healthcare10.4018/979-8-3693-3218-4.ch016(304-323)Online publication date: 5-Jan-2024
https://doi.org/10.4018/979-8-3693-3218-4.ch016
Liu HZhang YChen XZhang DLi RQin T(2024)Self-Supervised EEG Representation Learning for Robust Emotion RecognitionACM Transactions on Sensor Networks10.1145/367497520:5(1-22)Online publication date: 5-Jul-2024
https://dl.acm.org/doi/10.1145/3674975
Abbas SSampedro GAlsubai SOjo SAlmadhor AHejaili AStrazovska L(2024)Advancing Healthcare and Elderly Activity Recognition: Active Machine and Deep Learning for Fine- Grained Heterogeneity Activity RecognitionIEEE Access10.1109/ACCESS.2024.338043212(44949-44959)Online publication date: 2024
https://doi.org/10.1109/ACCESS.2024.3380432
Show More Cited By

Index Terms

Deep Hierarchical Attention Active Learning for Mental Disorder Unlabeled Data in AIoMT
1. Applied computing
  1. Life and medical sciences
    1. Health care information systems
2. Computing methodologies
  1. Machine learning
    1. Machine learning approaches
      1. Neural networks

Recommendations

Learning Instance Weighted Naive Bayes from labeled and unlabeled data

In real-world data mining applications, it is often the case that unlabeled instances are abundant, while available labeled instances are very limited. Thus, semi-supervised learning, which attempts to benefit from large amount of unlabeled data ...
Cost‐effective multi‐instance multilabel active learning
Abstract
Multi‐instance multi‐label (MIML) Active Learning (M2AL) aims to improve the learner while reducing the cost as much as possible by querying informative labels of complex bags composed of diverse instances. Existing M2AL solutions suffer high ...
Transfer active learning
CIKM '11: Proceedings of the 20th ACM international conference on Information and knowledge management

Active learning traditionally assumes that labeled and unlabeled samples are subject to the same distributions and the goal of an active learner is to label the most informative unlabeled samples. In reality, situations may exist that we may not have ...

Comments

Information & Contributors

Information

Published In

cover image ACM Transactions on Sensor Networks

ACM Transactions on Sensor Networks Volume 19, Issue 3

August 2023

597 pages

ISSN:1550-4859

EISSN:1550-4867

DOI:10.1145/3584865

Editor:
Yunhao Liu
Tsinghua University, China

Issue’s Table of Contents

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected].

Publisher

Association for Computing Machinery

New York, NY, United States

Journal Family

ACM Journals for the Design of Smart and Connected Systems

Publication History

Published: 01 March 2023

Online AM: 04 March 2022

Accepted: 15 February 2022

Revised: 18 January 2022

Received: 29 September 2021

Published in TOSN Volume 19, Issue 3

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Research-article

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

5
Total Citations
View Citations
406
Total Downloads

Downloads (Last 12 months)106
Downloads (Last 6 weeks)20

Reflects downloads up to 12 Sep 2024

Other Metrics

View Author Metrics

Citations

Cited By

Shafik WTufail ALiyanage CApong R(2024)Medical Robotics and AI-Assisted Diagnostics Challenges for Smart Sustainable HealthcareAI-Driven Innovations in Digital Healthcare10.4018/979-8-3693-3218-4.ch016(304-323)Online publication date: 5-Jan-2024
https://doi.org/10.4018/979-8-3693-3218-4.ch016
Liu HZhang YChen XZhang DLi RQin T(2024)Self-Supervised EEG Representation Learning for Robust Emotion RecognitionACM Transactions on Sensor Networks10.1145/367497520:5(1-22)Online publication date: 5-Jul-2024
https://dl.acm.org/doi/10.1145/3674975
Abbas SSampedro GAlsubai SOjo SAlmadhor AHejaili AStrazovska L(2024)Advancing Healthcare and Elderly Activity Recognition: Active Machine and Deep Learning for Fine- Grained Heterogeneity Activity RecognitionIEEE Access10.1109/ACCESS.2024.338043212(44949-44959)Online publication date: 2024
https://doi.org/10.1109/ACCESS.2024.3380432
Chai XLong GGan ZZhang Y(2024)TPE-MM: Thumbnail preserving encryption scheme based on Markov model for JPEG imagesApplied Intelligence10.1007/s10489-024-05318-z54:4(3429-3447)Online publication date: 23-Feb-2024
https://dl.acm.org/doi/10.1007/s10489-024-05318-z
Zhang Z(2023)Early warning model of adolescent mental health based on big data and machine learningSoft Computing - A Fusion of Foundations, Methodologies and Applications10.1007/s00500-023-09422-z28:1(811-828)Online publication date: 29-Nov-2023
https://dl.acm.org/doi/10.1007/s00500-023-09422-z

View Options

Get Access

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Article

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Full Text

View this article in Full Text.

HTML Format

View this article in HTML Format.

Media

Figures

Other

Tables

View full text|Download PDF

View Issue’s Table of Contents