IMKG: The Internet Meme Knowledge Graph
Pages 354 - 371
Abstract
Internet Memes (IMs) are creative media that combine text and vision modalities that people use to describe their situation by reusing an existing, familiar situation. Prior work on IMs has focused on analyzing their spread over time or high-level classification tasks like hate speech detection, while a principled analysis of their stratified semantics is missing. Hypothesizing that Semantic Web technologies are appropriate to help us bridge this gap, we build the first Internet Meme Knowledge Graph (IMKG): an explicit representation with 2 million edges that capture the semantics encoded in the text, vision, and metadata of thousands of media frames and their adaptations as memes. IMKG is designed to fulfil seven requirements derived from the inherent characteristics of IMs. IMKG is based on a comprehensive semantic model, it is populated with data from representative IM sources, and enriched with entities extracted from text and vision connected through background knowledge from Wikidata. IMKG integrates its knowledge both in RDF and as a labelled property graph. We provide insights into the structure of IMKG, analyze its central concepts, and measure the effect of knowledge enrichment from different information modalities. We demonstrate its ability to support novel use cases, like querying for IMs that are based on films, and we provide insights into the signal captured by the structure and the content of its nodes. As a novel publicly available resource, IMKG opens the possibility for further work to study the semantics of IMs, develop novel reasoning tasks, and improve its quality.
References
[1]
Akram U, Irvine K, Allen SF, Stevenson JC, Ellis JG, and Drabble J Internet memes related to the COVID-19 pandemic as a potential coping mechanism for anxiety Sci. Rep. 2021 11 1 22305
[2]
Auer S, Bizer C, Kobilarov G, Lehmann J, Cyganiak R, Ives Z, et al. Aberer K et al. DBpedia: a nucleus for a web of open data The Semantic Web 2007 Heidelberg Springer 722-735
[3]
Bollacker, K., Evans, C., Paritosh, P., Sturge, T., Taylor, J.: Freebase: a collaboratively created graph database for structuring human knowledge. In: ACM SIGMOD International Conference on Management of Data (2008)
[4]
Chen Y-C et al. Vedaldi A, Bischof H, Brox T, Frahm J-M, et al. UNITER: UNiversal image-TExt representation learning Computer Vision – ECCV 2020 2020 Cham Springer 104-120
[5]
Dancygier B and Vandelanotte L Internet memes as multimodal constructions Cogn. Linguist. 2017 28 3 565-598
[6]
Davison, P.: The Language of Internet Memes, pp. 120–134. New York University Press (2012)
[7]
Dawkins R The Selfish Gene 1976 Oxford Oxford University Press
[8]
Dimou, A., Sande, M.V., Colpaert, P., Verborgh, R., Mannens, E., de Walle, R.V.: RML: a generic language for integrated RDF mappings of heterogeneous data. In: Bizer, C., Heath, T., Auer, S., Berners-Lee, T. (eds.) Proceedings of the Workshop on Linked Data on the Web co-located with the 23rd International World Wide Web Conference (WWW 2014), Seoul, Korea, 8 April 2014 (2014)
[9]
Färber M, Bartscherer F, Menne C, and Rettinger A Linked data quality of dbpedia, freebase, opencyc, wikidata, and yago Semant. Web 2018 9 1 77-129
[10]
Fersini, E., et al.: Semeval-2022 task 5: multimedia automatic misogyny identification. In: Proceedings of the 16th International Workshop on Semantic Evaluation (SemEval-2022), pp. 533–549 (2022)
[11]
Fillmore, C.J., Baker, C.F.: A frames approach to semantic analysis. In: The Oxford Handbook of Linguistic Analysis, pp. 313–339. Oxford University Press (2012)
[12]
Ilievski F, et al., et al. Pan JZ, et al., et al. KGTK: a toolkit for large knowledge graph manipulation and analysis The Semantic Web – ISWC 2020 2020 Cham Springer 278-293
[13]
Johann M and Bülow L One does not simply create a meme: conditions for the diffusion of internet memes Int. J. Commun. 2019 13 23
[14]
Kiela, D., et al.: The hateful memes challenge: detecting hate speech in multimodal memes (2021)
[15]
Kirk, H.R., et al.: Memes in the wild: assessing the generalizability of the hateful memes challenge dataset. arXiv preprint arXiv:2107.04313 (2021)
[16]
Lenzerini, M.: data integration: a theoretical perspective. In: Popa, L., Abiteboul, S., Kolaitis, P.G. (eds.) Proceedings of the Twenty-first ACM SIGACT-SIGMOD-SIGART Symposium on Principles of Database Systems, 3–5 June, Madison, Wisconsin, USA, pp. 233–246. ACM (2002).
[17]
Liang, P.P., Zadeh, A., Morency, L.: Foundations and recent trends in multimodal machine learning: Principles, challenges, and open questions. CoRR abs/2209.03430 (2022). https://doi.org/10.48550/arXiv.2209.03430
[18]
Ling, C., AbuHilal, I., Blackburn, J., De Cristofaro, E., Zannettou, S., Stringhini, G.: Dissecting the meme magic: Understanding indicators of virality in image memes. Proc. ACM Hum.-Comput. Interact. (CSCW) (2021)
[19]
Lou A Multimodal simile: the “when” meme in social media discourse Engl. Text Constr. 2017 10 1 106-131
[20]
Lu, J., Batra, D., Parikh, D., Lee, S.: Vilbert: pretraining task-agnostic visiolinguistic representations for vision-and-language tasks. In: Advances in Neural Information Processing Systems, vol. 32 (2019)
[21]
Lynch MP Memes, misinformation, and political meaning South. J. Philos. 2022 60 1 38-56
[22]
Marino, G.: Semiotics of spreadability: a systematic approach to internet memes and virality (2015)
[23]
McNamara, A.: Can we measure memes? Front. Evol. Neurosci. (2011)
[24]
Mendes, P.N., Jakob, M., García-Silva, A., Bizer, C.: Dbpedia spotlight: shedding light on the web of documents. In: Proceedings of the 7th International Conference on Semantic Systems, pp. 1–8 (2011)
[25]
Nieubuurt JT Internet memes: leaflet propaganda of the digital age Front. Commun. 2021 5
[26]
Radford, A., et al.: Learning transferable visual models from natural language supervision. In: International Conference on Machine Learning, pp. 8748–8763. PMLR (2021)
[27]
Reimers, N., Gurevych, I.: Sentence-bert: Sentence embeddings using Siamese Bert-networks. arXiv preprint arXiv:1908.10084 (2019)
[28]
Sakr S et al. The future is big graphs: a community view on graph processing systems Commun. ACM 2021 64 9 62-71
[29]
Sambasivan, N., Kapania, S., Highfill, H., Akrong, D., Paritosh, P.K., Aroyo, L.: “Everyone wants to do the model work, not the data work”: data cascades in high-stakes AI. In: Kitamura, Y., Quigley, A., Isbister, K., Igarashi, T., Bjørn, P., Drucker, S.M. (eds.) CHI 2021: CHI Conference on Human Factors in Computing Systems, Virtual Event / Yokohama, Japan, pp. 39:1–39:15. ACM (2021)
[30]
Sherratt, V.: Towards contextually sensitive analysis of memes: Meme genealogy and knowledge base (2022)
[31]
Taecharungroj, V., Nueangjamnong, P.: The effect of humour on virality: The study of internet memes on social media. In: 7th International Forum on Public Relations and Advertising Media Impacts on Culture and Social Communication. Bangkok, August (2014)
[32]
Thakur, A.K., Ilievski, F., Sandlin, H.Â., Mermoud, A., Sourati, Z., Luceri, L., Tommasini, R.: Multimodal and explainable internet meme classification. arXiv preprint arXiv:2212.05612 (2022)
[33]
Tommasini R, Bonte P, Spiga F, and Della Valle E Streaming Linked Data 2023 Cham Springer
[34]
Vrandečić D and Krötzsch M Wikidata: a free collaborative knowledgebase Commun. ACM 2014 57 10 78-85
[35]
Wilkinson MD et al. The fair guiding principles for scientific data management and stewardship Sci. Data 2016 3 1 1-9
[36]
Zenner, E., Geeraerts, D.: One does not simply process memes: Image macros as multimodal constructions, pp. 167–194. De Gruyter, Berlin (2018)
Recommendations
Dissecting the Meme Magic: Understanding Indicators of Virality in Image Memes
CSCWDespite the increasingly important role played by image memes, we do not yet have a solid understanding of the elements that might make a meme go viral on social media. In this paper, we investigate what visual elements distinguish image memes that are ...
Contextualizing Internet Memes Across Social Media Platforms
WWW '24: Companion Proceedings of the ACM Web Conference 2024Internet memes have emerged as a novel format for communication and expressing ideas on the web. Their fluidity and creative nature are reflected in their widespread use, often across platforms and occasionally for unethical or harmful purposes. While ...
MemeGraphs: Linking Memes to Knowledge Graphs
Document Analysis and Recognition - ICDAR 2023AbstractMemes are a popular form of communicating trends and ideas in social media and on the internet in general, combining the modalities of images and text. They can express humor and sarcasm but can also have offensive content. Analyzing and ...
Comments
Information & Contributors
Information
Published In
May 2023
741 pages
ISBN:978-3-031-33454-2
DOI:10.1007/978-3-031-33455-9
© The Author(s), under exclusive license to Springer Nature Switzerland AG 2023.
Publisher
Springer-Verlag
Berlin, Heidelberg
Publication History
Published: 28 May 2023
Author Tags
Qualifiers
- Article
Contributors
Other Metrics
Bibliometrics & Citations
Bibliometrics
Article Metrics
- 0Total Citations
- 0Total Downloads
- Downloads (Last 12 months)0
- Downloads (Last 6 weeks)0
Reflects downloads up to 03 Oct 2024
Other Metrics
Citations
View Options
View options
Get Access
Login options
Check if you have access through your login credentials or your institution to get full access on this article.
Sign in