Abstract
This paper addresses new challenges of detecting campaigns in social media, which emerged with the rise of Large Language Models (LLMs). LLMs particularly challenge algorithms focused on the temporal analysis of topical clusters. Simple similarity measures can no longer capture and map campaigns that were previously broadly similar in content. Herein, we analyze whether the classification of messages over time can be profitably used to rediscover poorly detectable campaigns at the content level. Thus, we evaluate classical classifiers and a new method based on siamese neural networks. Our results show that campaigns can be detected despite the limited reliability of the classifiers as long as they are based on a large amount of simultaneously spread artificial content.
The authors acknowledge support by the European Research Center in Information Systems (ERCIS) and by the project HybriD (FKZ: 16KIS1531K) funded by the German Federal Ministry of Education and Research.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Similar content being viewed by others
References
Alamleh, H., Al Qahtani, A., ElSaid, A.: Distinguishing human-written and ChatGPT-generated text using machine learning. In: Systems and Information Engineering Design Symposium, pp. 154–158. IEEE, Charlottesville, USA (2023)
Antoun, W., Mouilleron, V., Sagot, B., Seddah, D.: Towards a Robust Detection of Language Model Generated Text. arXiv 2306.05871 (2023)
Assenmacher, D., Adam, L., Trautmann, H., Grimme, C.: Towards real-time and unsupervised campaign detection in social media. In: Proceedings of the Florida Artificial Intelligence Research Society Conference. AAAI Press, Florida, USA (2020)
Assenmacher, D., Clever, L., Pohl, J.S., Trautmann, H., Grimme, C.: A two-phase framework for detecting manipulation campaigns in social media. In: Meiselwitz, G. (ed.) HCII 2020. LNCS, vol. 12194, pp. 201–214. Springer, Cham (2020). https://doi.org/10.1007/978-3-030-49570-1_14
Assenmacher, D., et al.: Demystifying social bots: on the intelligence of automated social media actors. Soc. Med. Soc. 6(3), 1–14 (2020)
Assenmacher, D., Trautmann, H.: Textual one-pass stream clustering with automated distance threshold adaption. In: Nguyen, N.T., Tran, T.K., Tukayev, U., Hong, TP., Trawiń,ski B., Szczerbicki, E. (eds.) Intelligent Information and Database Systems. ACIIDS 2022. LNCS, vol. 13757, pp. 3–16. Springer, Cham (2022). https://doi.org/10.1007/978-3-031-21743-2_1
Bellutta, D., Carley, K.M.: Investigating coordinated account creation using burst detection and network analysis. J. Big Data 10(1), 1–17 (2023)
Bromley, J., Guyon, I., LeCun, Y., Säckinger, E., Shah, R.: Signature verification using a siamese time delay neural network. In: Cowan, J.D., Tesauro, G., Alspector, J. (eds.) Advances in Neural Information Processing Systems, vol. 6, NIPS, pp. 737–744. Morgan Kaufmann (1993)
Chakraborty, S., Bedi, A.S., Zhu, S., An, B., Manocha, D., Huang, F.: On the Possibilities of AI-Generated Text Detection (2023). arXiv:2304.04736
Cinelli, M., Cresci, S., Quattrociocchi, W., Tesconi, M., Zola, P.: Coordinated inauthentic behavior and information spreading on Twitter. Decis. Support Syst. 160, 1–28 (2022). https://doi.org/10.1016/j.dss.2022.113819
Cresci, S., Di Pietro, R., Petrocchi, M., Spognardi, A., Tesconi, M.: DNA-inspired online behavioral modeling and its application to spambot detection. IEEE Intell. Syst. 31(5), 58–64 (2016)
Crothers, E., Japkowicz, N., Viktor, H.: Machine Generated Text: A Comprehensive Survey of Threat Models and Detection Methods (2023). arXiv:2210.07321
Davis, C.A., Varol, O., Ferrara, E., Flammini, A., Menczer, F.: BotOrNot: a system to evaluate social bots. In: Proceedings of the 25th International Conference Companion on World Wide Web, pp. 273–274 (2016)
Devlin, J., Chang, M.W., Lee, K., Toutanova, K.: BERT: pre-training of deep bidirectional transformers for language understanding (2018). arXiv:1810.04805
EleutherAI: GPT Neo - An Implementation of Model & Data Parallel GPT3-Like Models Using the Mesh-Tensorflow Library (2022). https://github.com/EleutherAI/gpt-neo. Accessed 02 July 2023
Erhardt, K., Albassam, D.: Detecting the hidden dynamics of networked actors using temporal correlations. In: Companion Proceedings of the ACM Web Conference 2023, pp. 1214–1217. WWW 2023 Companion, ACM, Austin, TX, USA (2023)
Fagni, T., Falchi, F., Gambini, M., Martella, A., Tesconi, M.: TweepFake: about detecting deepfake Tweets. PLOS ONE. 16(5), e0251415 (2021)
Fernández-Delgado, M., Cernadas, E., Barro, S., Amorim, D.: Do we need hundreds of classifiers to solve real world classification problems? J. Mach. Learn. Res. 15(1), 3133–3181 (2014)
Ferrara, E.: Social bot detection in the age of ChatGPT: challenges and opportunities. First Monday. 28(6), 1–30 (2023). https://doi.org/10.5210/fm.v28i6.13185
Gao, L., et al.: The Pile: An 800 GB Dataset of Diverse Text for Language Modeling (2020). arXiv:2101.00027
Grimme, C., Preuss, M., Adam, L., Trautmann, H.: Social bots: human-like by means of human control? Big Data 5(4), 279–293 (2017)
Grimme, C., Assenmacher, D., Adam, L.: Changing perspectives: is it sufficient to detect social bots? In: Proceedings of the International Conference on Human-Computer Interaction. Las Vegas, United States of America (2018)
Grimme, C., Pohl, J., Cresci, S., Lüling, R., Preuss, M.: New automation for social bots: from trivial behavior to AI-powered communication. In: Spezzano, F., Amaral, A., Ceolin, D., Fazio, L., Serra, E. (eds.) Disinformation in Open Online Media. MISDOOM 2022. LNCS, vol. 13545, pp. 79–99. Springer, Cham (2022). https://doi.org/10.1007/978-3-031-18253-2_6
Guo, B., et al.: How Close is ChatGPT to Human Experts? Comparison Corpus, Evaluation, and Detection (2023). arXiv:2301.07597
Ippolito, D., Duckworth, D., Callison-Burch, C., Eck, D.: Automatic detection of generated text is easiest when humans are fooled. In: Proceedings of the 58th Annual Meeting of the ACL, pp. 1808–1822. ACL, Online (2020)
James, G., Witten, D., Hastie, T., Tibshirani, R.: An Introduction to Statistical Learning, 2nd edn. Springer, New York (2021). https://doi.org/10.1007/978-1-4614-7138-7
Kirchner, J.H., Ahmad, L., Aaronson, S., Leike, J.: New AI classifier for indicating AI-written text. https://openai.com/blog/new-ai-classifier-for-indicating-ai-written-text (2022). Accessed 30 June 2023
Kumarage, T., Garland, J., Bhattacharjee, A., Trapeznikov, K., Ruston, S., Liu, H.: Stylometric Detection of AI-Generated Text in Twitter Timelines (2023). arXiv:2303.03697
Michail, D., Kanakaris, N., Varlamis, I.: Detection of fake news campaigns using graph convolutional networks. Int. J. Inf. Manage. Data Insights 2(2), 100104 (2022)
Mitchell, E., Lee, Y., Khazatsky, A., Manning, C.D., Finn, C.: DetectGPT: Zero-Shot Machine-Generated Text Detection using Probability Curvature (2023). arXiv:2301.11305
Mitrović, S., Andreoletti, D., Ayoub, O.: ChatGPT or Human? Detect and Explain. Explaining Decisions of Machine Learning Model for Detecting Short ChatGPT-generated Text (2023). arXiv:2301.13852
Pohl, J., Assenmacher, D., Seiler, M., Trautmann, H., Grimme, C.: Artificial social media campaign creation for benchmarking and challenging detection approaches. In: Workshop Proceedings of the 16\(^{th}\) International Conference on Web and Social Media. AAAI Press, Atlanta, GA, USA (2022)
Radford, A., Narasimhan, K., Salimans, T., Sutskever, I.: Improving Language Understanding by Generative Pre-Training. Technical report, OpenAI (2018)
Rauchfleisch, A., Kaiser, J.: The false positive problem of automatic bot detection in social science research. PLoS ONE 15(10), e0241045 (2020)
Sadasivan, V.S., Kumar, A., Balasubramanian, S., Wang, W., Feizi, S.: Can AI-Generated Text be Reliably Detected? (2023). arXiv:2303.11156
Schroff, F., Kalenichenko, D., Philbin, J.: FaceNet: a unified embedding for face recognition and clustering. In: 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 815–823 (2015)
Singh, A.: A comparison study on AI language detector. In: 2023 IEEE 13th Annual Computing and Communication Workshop and Conference (CCWC), pp. 489–493. IEEE, Las Vegas, NV, USA (2023)
Tang, R., Chuang, Y.N., Hu, X.: The Science of Detecting LLM-Generated Texts (2023). arXiv:2303.07205
Vaswani, A., et al.: Attention is all you need. In: Guyon, I., et al. (eds.) Advances in Neural Information Processing Systems. vol. 30. Curran Associates, Inc. (2017)
Verma, V., Fleisig, E., Tomlin, N., Klein, D.: Ghostbuster: Detecting Text Ghostwritten by Large Language Models (2023). arXiv:2305.15047
Weber, D., Neumann, F.: Amplifying influence through coordinated behaviour in social networks. Soc. Netw. Anal. Min. 11(1), 1–42 (2021). https://doi.org/10.1007/s13278-021-00815-2
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2023 The Author(s), under exclusive license to Springer Nature Switzerland AG
About this paper
Cite this paper
Grimme, B., Pohl, J., Winkelmann, H., Stampe, L., Grimme, C. (2023). Lost in Transformation: Rediscovering LLM-Generated Campaigns in Social Media. In: Ceolin, D., Caselli, T., Tulin, M. (eds) Disinformation in Open Online Media. MISDOOM 2023. Lecture Notes in Computer Science, vol 14397. Springer, Cham. https://doi.org/10.1007/978-3-031-47896-3_6
Download citation
DOI: https://doi.org/10.1007/978-3-031-47896-3_6
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-031-47895-6
Online ISBN: 978-3-031-47896-3
eBook Packages: Computer ScienceComputer Science (R0)