Location via proxy:   [ UP ]  
[Report a bug]   [Manage cookies]                
Skip to main content
Springer Nature Link
Log in

A Comparative Analysis of Metaphorical Cognition in ChatGPT and Human Minds

  • Research
  • Published:
Cognitive Computation Aims and scope Submit manuscript

Abstract

ChatGPT represents a significant advancement in the field of Artificial Intelligence (AI), showcasing the development of a robust AI system capable of multitasking and generating human-like language. At present, many scholars have done evaluations on ChatGPT in terms of language, reasoning, and scientific knowledge abilities, based on benchmarks or well-crafted questions. However, to the best of our knowledge, there is currently no existing comparative analysis from a cognitive perspective that directly assesses ChatGPT alongside humans. Metaphor, serving as a manifestation of linguistic creativity, provides a valuable avenue for examining cognition. This is due to the mapping relationship it establishes between the target and source conceptual domains, reflecting distinct cognitive patterns. In this paper, we use a metaphor processing tool, MetaPro, to analyze the cognitive differences between ChatGPT and humans through the metaphorical expressions in ChatGPT- and human-generated text. We illustrate the preferences in metaphor usage, concept mapping, and cognitive pattern variances across different domains. The methodology utilized in this study makes a valuable contribution to the task-agnostic evaluation of AI systems and cognitive research. The insights garnered from this research prove instrumental in comprehending the cognitive distinctions between ChatGPT and humans, facilitating the identification of potential cognitive biases within ChatGPT.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Subscribe and save

Springer+ Basic
$34.99 /Month
  • Get 10 units per month
  • Download Article/Chapter or eBook
  • 1 Unit = 1 Article or 1 Chapter
  • Cancel anytime
Subscribe now

Buy Now

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4

Similar content being viewed by others

Explore related subjects

Discover the latest articles, news and stories from top researchers in related subjects.

Data Availability

The data used in this study will be made available upon request.

Notes

  1. Ecological validity pertains to how well the design or evaluation setup aligns with the authentic work context of the user. It focuses on the accuracy with which the design or evaluation mirrors the pertinent characteristics of the interaction’s ecology, capturing its context in the real world or environment [6].

  2. We define metaphorical cognition as the reflection of cognition through metaphors, encompassing elements such as the cognition of target concepts, source concepts, and their mappings.

  3. While dictionaries may contain the meanings of numerous conventional metaphors, their mere inclusion is not a feature to identify the metaphoricity of a lexical unit. According to Metaphor Identification Procedure [12], a metaphor is identified through the semantic contrast between its contextual and basic meanings. The basic meaning of a metaphor is typically more concrete, related to bodily action, more precise, and historically older.

  4. Italics denote metaphors; small capital words denote concepts.

  5. In this work, the representation of a concept mapping takes the form of “a target concept is a source concept”.

  6. https://metapro.ruimao.tech

  7. A word association test involves the presentation of a stimulus word to a participant, who subsequently provides the initial word that comes to mind in response.

  8. The thematic apperception test is a projective psychological evaluation that requires individuals to furnish interpretations for scenes characterized by ambiguity.

  9. The Rorschach test is a projective technique used in psychological assessment, involving the individual’s task of describing their interpretations of ten inkblots. These inkblots consist of a combination of black or gray elements and others featuring patches of color.

  10. Questions in the dataset were generated by humans only. Thus, we did not parse questions.

  11. https://nlp.stanford.edu/projects/glove/

References

  1. Shidiq M. The use of artificial intelligence-based Chat-GPT and its challenges for the world of education; from the viewpoint of the development of creative writing skills. In: Proceeding of International Conference on Education, Society and Humanity; 2023;1:353–357

  2. Méndez G, Gervás P. Using ChatGPT for story sifting in narrative generation. In: Proceedings of The 14th International Conference on Computational Creativity; 2023

  3. Qin C, Zhang A, Zhang Z, Chen J, Yasunaga M, Yang D. Is ChatGPT a general-purpose natural language processing task solver? 2023 arXiv:2302.06476

  4. Soni M, Wade V Comparing abstractive summaries generated by ChatGPT to real summaries through blinded reviewers and text classification algorithms. 2023 arXiv:2303.17650

  5. Mao R, Chen G, Zhang X, Guerin F, Cambria E. GPTEval: a survey on assessments of ChatGPT and GPT-4. In: Proceedings of the 2024 Joint International Conference on Computational Linguistics, Language Resources and Evaluation (LREC-COLING), Torino, Italia; 2024. pp. 7844–7866

  6. Hartson R, Pyla PS. Chapter 14 - rigorous empirical evaluation: preparation. In: Hartson R, Pyla PS (eds.) The UX Book; 2012. pp. 503–536. Morgan Kaufmann, Boston

  7. Rapaport D, Gill M, Schafer R. Diagnostic psychological testing: the theory, statistical evaluation, and diagnostic application of a battery of tests. 1946:2

  8. Crossley SA, Kyle K, McNamara DS. Sentiment analysis and social cognition engine (SEANCE): an automatic tool for sentiment, social cognition, and social-order analysis. Behav Res Methods. 2017;49:803–21.

    Article  Google Scholar 

  9. Naskar D, Singh S.R, Kumar D, Nandi S, Rivaherrera EOdl. Emotion dynamics of public opinions on Twitter. ACM Trans Inf Syst (TOIS). 2020;38(2):1–24

  10. Orabi AH, Buddhitha P, Orabi MH, Inkpen D. Deep learning for depression detection of Twitter users. In: Proceedings of the Fifth Workshop on Computational Linguistics and Clinical Psychology: from Keyboard to Clinic; 2018. pp. 88–97

  11. Priva UC, Austerweil JL. Analyzing the history of cognition using topic models. Cognit. 2015;135:4–9.

    Article  Google Scholar 

  12. Pragglejaz G. MIP: a method for identifying metaphorically used words in discourse. Metaphor Symb. 2007;22(1):1–39.

    Article  Google Scholar 

  13. Lakoff G, Johnson M. Metaphors we live by; 1980

  14. Lakoff G. The contemporary theory of metaphor. Metaphor and thought; 1993:202–251

  15. Mao R, Li X, He K, Ge M, Cambria E. MetaPro online: a computational metaphor processing online system. In: Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics (System Demonstrations); 2023;3:127–135

  16. Guo B, Zhang X, Wang Z, Jiang M, Nie J, Ding Y, Yue J, Wu Y. How close is ChatGPT to human experts? Comparison corpus, evaluation, and detection. 2023 arXiv:2301.07597

  17. Ge M, Mao R, Cambria E. A survey on computational metaphor processing techniques: from identification, interpretation, generation to application. Artif Intell Rev. 2023;56:1829–95.

    Article  Google Scholar 

  18. Mao R, Liu Q, He K, Li W, Cambria E. The biases of pre-trained language models: an empirical study on prompt-based sentiment analysis and emotion detection. IEEE Trans Affect Comput. 2023;14(3):1743–53.

    Article  Google Scholar 

  19. Fan C, Lin J, Mao R, Cambria E. Fusing pairwise modalities for emotion recognition in conversations. Inf Fusion. 2024;106:102306.

  20. William D, Suhartono D. Text-based depression detection on social media posts: a systematic literature review. Procedia Comput Sci. 2021;179:582–9.

    Article  Google Scholar 

  21. Ji S, Pan S, Li X, Cambria E, Long G, Huang Z. Suicidal ideation detection: a review of machine learning methods and applications. IEEE Trans Comput Soc Syst. 2020;8(1):214–26.

    Article  Google Scholar 

  22. Karami A, Zhu M, Goldschmidt B, Boyajieff HR, Najafabadi MM. COVID-19 vaccine and social media in the US: exploring emotions and discussions on Twitter. Vaccines. 2021;9(10):1059.

    Article  Google Scholar 

  23. Xia E, Yue H, Liu H. Tweet sentiment analysis of the 2020 US presidential election. In: Companion Proceedings of the Web Conference 2021; 2021. pp. 367–371

  24. Duong C, Liu Q, Mao R, Cambria E. Saving earth one tweet at a time through the lens of artificial intelligence. In: 2022 International Joint Conference on Neural Networks (IJCNN); 2022. pp. 1–9

  25. Duong C, Raghuram VC, Lee A, Mao R, Mengaldo G, Cambria E. Neurosymbolic AI for mining public opinions about wildfires. Cognit Comput. 2023;16:1531–53.

    Article  Google Scholar 

  26. Garcia MB, Cunanan-Yabut A. Public sentiment and emotion analyses of Twitter data on the 2022 Russian invasion of Ukraine. In: 2022 9th International Conference on Information Technology, Computer, and Electrical Engineering (ICITACEE); 2022. pp. 242–247. IEEE

  27. Guntuku SC, Preotiuc-Pietro D, Eichstaedt JC, Ungar LH. What twitter profile and posted images reveal about depression and anxiety. In: Proceedings of the International AAAI Conference on Web and Social Media; 2019;13:236–246

  28. Wu X, Pan F, Nguyen T, Feng Y, Liu C, Nguyen CD, Luu AT. On the affinity, rationality, and diversity of hierarchical topic modeling. In: Proceedings of the AAAI Conference on Artificial Intelligence; 2024;38:19261–19269

  29. Wu X, Dong X, Nguyen TT, Luu AT. Effective neural topic modeling with embedding clustering regularization. In: International Conference on Machine Learning; 2023. pp. 37335–37357. PMLR

  30. Taleb NN. The black swan: the impact of the highly improbable. 2007:2

  31. Glette-Iversen I, Aven T. On the meaning of and relationship between dragon-kings, black swans and related concepts. Reliab Eng Syst Saf. 2021;211:107625.

  32. Arrese Á. The use of ‘bubble’ as an economic metaphor in the news: the case of the ‘real estate bubble’ in Spain. Lang Commun. 2021;78:100–8.

    Article  Google Scholar 

  33. Ruane E, Farrell S, Ventresque A. User perception of text-based chatbot personality. In: Chatbot Research and Design: 4th International Workshop; 2021. pp. 32–47. Springer

  34. Liu B, Sundar SS. Should machines express sympathy and empathy? Experiments with a health advice chatbot. Cyberpsychology Behav Soc Netw. 2018;21(10):625–36.

    Article  Google Scholar 

  35. Santo L.E, Cardoso A, Wiggins G. Theoretical learning creators and creative scientists. In: 13th International Conference on Computational Creativity. 2022. Association for Computational Creativity

  36. Moghaddam SR, Honey CJ. Boosting theory-of-mind performance in large language models via prompting. 2023. arXiv:2304.11490

  37. Hutson M. How does ChatGPT ‘think’? Psychology and neuroscience crack open AI large language models. Nat. 2024;629(8014):986–8.

    Article  Google Scholar 

  38. Mao R, He K, Ong CB, Liu Q, Cambria E. MetaPro 2.0: computational metaphor processing on the effectiveness of anomalous language modeling. In: Findings of the Association for Computational Linguistics: ACL; 2024. pp. 9891–9908. Association for Computational Linguistics, Bangkok, Thailand

  39. Manro R, Mao R, Dahiya L, Ma Y, Cambria E. A cognitive analysis of CEO speeches and their effects on stock markets. In: Proceedings of the 5th International Conference on Financial Technology (ICFT), 2024. Singapore

  40. Prosser J, Cohen LJ, Steinfeld M, Eisenberg D, London ED, Galynker II. Neuropsychological functioning in opiate-dependent subjects receiving and following methadone maintenance treatment. Drug Alcohol Depend. 2006;84(3):240–7.

    Article  Google Scholar 

  41. Wiggins JS. Paradigms of personality assessment. 2003

  42. De Vos GA. Boyer LB. Symbolic analysis cross-culturally: the rorschach test; 2021.

    Google Scholar 

  43. Han S, Mao R, Cambria E. Hierarchical attention network for explainable depression detection on Twitter aided by metaphor concept mappings. In: Proceedings of the 29th International Conference on Computational Linguistics (COLING); 2022. pp. 94–104

  44. Mao R, Zhang T, Liu Q, Hussain A, Cambria E. Unveiling diplomatic narratives: analyzing United Nations Security Council debates through metaphorical cognition. In: Proceedings of the Annual Meeting of the Cognitive Science Society (CogSci); 2024. pp. 1709–1716. Rotterdam, the Netherlands

  45. Mao R, Li X. Bridging towers of multi-task learning with a gating mechanism for aspect-based sentiment analysis and sequential metaphor identification. Proceed AAAI Conf Artif Intell. 2021;35(15):13534–42.

    Google Scholar 

  46. Mao R, Li X, Ge M, Cambria E. MetaPro: a computational metaphor processing model for text pre-processing. Inf Fusion. 2022;86–87:30–43.

    Article  Google Scholar 

  47. Ge M, Mao R, Cambria E. Explainable metaphor identification inspired by conceptual metaphor theory. Proceed AAAI Conf Artif Intell. 2022;36(10):10681–9.

    Google Scholar 

  48. Steen GJ, Dorst AG, Herrmann JB, Kaal A, Krennmayr T, Pasma T. A method for linguistic metaphor identification: from MIP to MIPVU. 2010;2010:14.

    Google Scholar 

  49. Fellbaum C. WordNet: an electronic lexical database. 1998

  50. Liu Y, Ott M, Goyal N, Du J, Joshi M, Chen D, Levy O, Lewis M, Zettlemoyer L, Stoyanov V. RoBERTa: a robustly optimized BERT pretraining approach. 2019. arXiv:1907.11692

  51. Sackett DL. Evidence-based medicine. In: Seminars in Perinatology; 1997;21:3–5. Elsevier

  52. Mao R, Du K, Ma Y, Zhu L, Cambria E. Discovering the cognition behind language: financial metaphor analysis with MetaPro. In: 2023 IEEE International Conference on Data Mining (ICDM); 2023. pp. 1211–1216. IEEE

  53. Jamrozik A, McQuire M, Cardillo ER, Chatterjee A. Metaphor: bridging embodiment to abstraction. Psychon Bull Rev. 2016;23:1080–9.

    Article  Google Scholar 

  54. Sperber D, Wilson D. Relevance: communication and cognition. 2nd ed. 1995

  55. Pennington J, Socher R, Manning CD. GloVe: global vectors for word representation. In: Proceedings of the 2014 conference on empirical methods in natural language processing; 2014. pp. 1532–1543

  56. Lloyd S. Least squares quantization in PCM. IEEE Trans Inf Theory. 1982;28(2):129–37.

    Article  MathSciNet  Google Scholar 

  57. Cambria E, Mao R, Chen M, Wang Z, Ho SB. Seven pillars for the future of artificial intelligence. IEEE Intell Syst. 2023;38(6):62–9.

    Article  Google Scholar 

  58. Zhu L, Li W, Mao R, Pandelea V, Cambria E. PAED: zero-shot persona attribute extraction in dialogues. In: Proceedings of the 61st annual meeting of the association for Computational Linguistics (ACL); 2023;1:9771–9787

  59. Zhu L, Mao R, Cambria E, Jansen BJ. Neurosymbolic AI for personalized sentiment analysis. In: Proceedings of international conference on human-computer Interaction (HCII); 2024. Washington DC, USA

Download references

Funding

This research/project is supported by the Ministry of Education, Singapore, under its MOE Academic Research Fund Tier 2 (STEM RIE2025 Award MOE-T2EP20123-0005). Guanyi Chen is supported by the Hubei Provincial Key Laboratory of Artificial Intelligence and Smart Learning and the National Language Resources Monitoring and Research Center for Network Media of Central China Normal University in Wuhan, China.

Author information

Authors and Affiliations

  1. Nanyang Technological University, Singapore, Singapore

    Rui Mao, Mengshi Ge & Erik Cambria

  2. Central China Normal University, Wuhan, China

    Guanyi Chen

  3. University of Aberdeen, Aberdeen, UK

    Xiao Li

Authors
  1. Rui Mao
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Guanyi Chen
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Xiao Li
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Mengshi Ge
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Erik Cambria
    View author publications

    You can also search for this author in PubMed Google Scholar

Contributions

Rui Mao: conceptualization, methodology, software, formal analysis, visualization, writing—original draft, writing—review and editing. Guanyi Chen: formal analysis, visualization, writing—review and editing. Xiao Li: investigation, validation, writing—review and editing. Mengshi Ge: investigation, validation, writing—review and editing. Erik Cambria: conceptualization, data curation, writing—review and editing, supervision.

Corresponding author

Correspondence to Erik Cambria.

Ethics declarations

Conflict of Interest

The authors declare no competing interests.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Mao, R., Chen, G., Li, X. et al. A Comparative Analysis of Metaphorical Cognition in ChatGPT and Human Minds. Cogn Comput 17, 35 (2025). https://doi.org/10.1007/s12559-024-10393-y

Download citation

  • Received:

  • Accepted:

  • Published:

  • DOI: https://doi.org/10.1007/s12559-024-10393-y

Keywords

Search

Navigation