Abstract
The increasing number of mental disorders is a severe problem in the modern world and can even lead to suicide if left untreated. In the age of digitalization, we move part of our lives to social media, where we share both the good and the bad moments. This allows for the early detection of mental disorders (such as depression, excessive stress, or social phobia) of which the user may even be unaware. We propose to modify large language models, such as PHI-2, Mistral, Flan-T5, or LLaMA 2, to classify mental disorders and to add appropriate layers. This gives a better prediction performance than zero-shot/few-shot for LLMs and classification by BERT-based models. Using such an architecture makes it possible to return a label rather than text, thus allowing the output of the LLM model to be freely modified.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Similar content being viewed by others
References
Brown, T.B., et al.: Language models are few-shot learners (2020)
Demner-Fushman, D., Chapman, W.W., McDonald, C.J.: What can natural language processing do for clinical decision support? J. Biomed. Inform. 42(5), 760–772 (2009). https://doi.org/10.1016/j.jbi.2009.08.007, https://www.sciencedirect.com/science/article/pii/S1532046409001087. biomedical Natural Language Processing
Devlin, J., Chang, M.W., Lee, K., Toutanova, K.: Bert: pre-training of deep bidirectional transformers for language understanding (2019)
Garg, M., et al.: Cams: an annotated corpus for causal analysis of mental health issues in social media posts. arXiv preprint arXiv:2207.04674 (2022)
Garg, M., Shahbandegan, A., Chadha, A., Mago, V.: An annotated dataset for explainable interpersonal risk factors of mental disturbance in social media posts (2023)
Gunasekar, S., et al.: Textbooks are all you need (2023)
Hu, E.J.,et al.: Lora: low-rank adaptation of large language models. CoRR abs/2106.09685 (2021). https://arxiv.org/abs/2106.09685
Ji, S., Zhang, T., Ansari, L., Fu, J., Tiwari, P., Cambria, E.: Mentalbert: publicly available pretrained language models for mental healthcare (2021)
Li, J., Sun, A., Han, J., Li, C.: A survey on deep learning for named entity recognition. IEEE Trans. Knowl. Data Eng. 34(1), 50–70 (2022). https://doi.org/10.1109/tkde.2020.2981314
Lin, L.Y., et al.: Association between social media use and depression among us young adults. Depress. Anxiety 33(4), 323–331 (2016)
Mauriello, M.L., Lincoln, T., Hon, G., Simon, D., Jurafsky, D., Paredes, P.: Sad: a stress annotated dataset for recognizing everyday stressors in sms-like conversational systems. In: Extended Abstracts of the 2021 CHI Conference on Human Factors in Computing Systems. CHI EA 2021, Association for Computing Machinery, New York, NY, USA (2021). https://doi.org/10.1145/3411763.3451799
OpenAI: Gpt-4 technical report (2023)
Pestian, J.P., Nasrallah, H.A., Matykiewicz, P., Bennett, A.J., Leenaars, A.A.: Suicide note classification using natural language processing: A content analysis. Biomed. Inform. Insights 3 (2010). https://api.semanticscholar.org/CorpusID:7779743
Radford, A., Wu, J., Child, R., Luan, D., Amodei, D., Sutskever, I., et al.: Language models are unsupervised multitask learners. OpenAI blog 1(8), 9 (2019)
Raffel, C., et al.: Exploring the limits of transfer learning with a unified text-to-text transformer (2023)
Sun, C., Qiu, X., Xu, Y., Huang, X.: How to fine-tune bert for text classification? (2020)
Touvron, H., et al.: Llama: open and efficient foundation language models (2023)
Turcan, E., McKeown, K.: Dreaddit: a reddit dataset for stress analysis in social media. arXiv preprint arXiv:1911.00133 (2019)
Vaswani, A., et al.: Attention is all you need (2023)
Wei, J., et al.: Finetuned language models are zero-shot learners (2022)
Windfuhr, K., Kapur, N.: Suicide and mental illness: a clinical review of 15 years findings from the UK national confidential inquiry into suicide. Br. Med. Bull. 100(1), 101–121 (2011)
Yang, K., Zhang, T., Kuang, Z., Xie, Q., Ananiadou, S., Huang, J.: Mentallama: interpretable mental health analysis on social media with large language models (2023)
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2024 The Author(s), under exclusive license to Springer Nature Switzerland AG
About this paper
Cite this paper
Nowacki, A., Sitek, W., Rybiński, H. (2024). LLMental: Classification of Mental Disorders with Large Language Models. In: Appice, A., Azzag, H., Hacid, MS., Hadjali, A., Ras, Z. (eds) Foundations of Intelligent Systems. ISMIS 2024. Lecture Notes in Computer Science(), vol 14670. Springer, Cham. https://doi.org/10.1007/978-3-031-62700-2_4
Download citation
DOI: https://doi.org/10.1007/978-3-031-62700-2_4
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-031-62699-9
Online ISBN: 978-3-031-62700-2
eBook Packages: Computer ScienceComputer Science (R0)