Automatic and Semi-automatic Augmentation of Migration Related Semantic Concepts for Visual Media Retrieval
Authors: 
Damianos Galanopoulos, Erick Elejalde, Alexandros Pournaras, Claudia Niederée, Vasileios MezarisAuthors Info & Claims
Damianos Galanopoulos, Erick Elejalde, Alexandros Pournaras, Claudia Niederée, Vasileios MezarisAuthors Info & ClaimsPages 33 - 38
Abstract
Understanding the factors related to migration, such as perceptions about routes and target countries, is critical for border agencies and society altogether. A systematic analysis of communication and news channels, such as social media, can improve our understanding of such factors. Videos and images play a critical role in social media as they have significant impact on perception manipulation and misinformation campaigns. However, more research is needed in the identification of semantically relevant visual content for specific queried concepts. Furthermore, an important problem to overcome in this area is the lack of annotated datasets that could be used to create and test accurate models. A recent study proposed a novel video representation and retrieval approach that effectively bridges the gap between a substantiated domain understanding - encapsulated into textual descriptions of Migration Related Semantic Concepts (MRSCs) - and the expression of such concepts in a video. In this work, we build on this approach and propose an improved procedure for the crucial step of the concept labels' textual augmentation, which contributes towards the full automation of the pipeline. We assemble the first, to the best of our knowledge, migration-related videos and images dataset and we experimentally assess our method on it.
References
[1]
George Awad, Asad A. Butt, Keith Curtis, Jonathan Fiscus, et almbox. 2020. TRECVID 2020: A comprehensive campaign for evaluating video retrieval tasks across multiple application domains. In Proceedings of TRECVID 2020. NIST, USA.
[2]
Gerhard Backfried, Christian Schmidt, Mark Pfeiffer, Gerald Quirchmayr, and Johannes Göllner. 2015. Open Source Intelligence for Traditional- and Social Media Sources - The Sail Labs Media Mining System for OSINT . The 10th International Conference iNCEB2015 (2015).
[3]
Jacob Devlin, Ming-Wei Chang, Kenton Lee, and Kristina Toutanova. 2018. BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding. arXiv preprint arXiv:1810.04805 .
[4]
Jianfeng Dong, Xirong Li, and Cees G. M. Snoek. 2018. Predicting visual features from text for image and video caption retrieval. IEEE Transactions on Multimedia (TMM), Vol. 20, 12 (Dec 2018), 3377--3388.
[5]
Erick Elejalde, Damianos Galanopoulos, Claudia Niederée, and Vasileios Mezaris. 2021. Migration-Related Semantic Concepts for the Retrieval of Relevant Video Content. In Intelligent Technologies and Applications: Third International Conference, INTAP 2020. Springer International Publishing, 404--416.
[6]
Katie Eriksson. 2010. Concept determination as part of the development of knowledge in caring science. Scandinavian Journal of Caring Sciences, Vol. 24 (2010), 2--11.
[7]
Damianos Galanopoulos and Vasileios Mezaris. 2020. Attention Mechanisms, Signal Encodings and Fusion Strategies for Improved Ad-hoc Video Search with Dual Encoding Networks. In Proceedings of the ACM International Conference on Multimedia Retrieval (Dublin, Ireland) ((ICMR '20)). ACM.
[8]
Yuncheng Li, Yale Song, Liangliang Cao, et almbox. 2016. TGIF: A new dataset and benchmark on animated GIF description. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR) . 4641--4650.
[9]
Foteini Markatopoulou, Anastasia Ioannidou, Christos Tzelepis, et almbox. 2015. ITI-CERTH participation to TRECVID 2015. In Proceedings of TRECVID 2015. NIST, USA.
[10]
Tomas Mikolov, Kai Chen, Greg Corrado, and Jeffrey Dean. 2013. Efficient estimation of word representations in vector space. In 1st International Conference on Learning Representations, Workshop Track Proceedings (ICLR '13).
[11]
Paul Over, Jonathan Fiscus, David Joy, et almbox. 2015. TRECVID 2015 -- An Overview of the Goals, Tasks, Data, Evaluation Mechanisms and Metrics. In Proceedings of TRECVID 2015. NIST, USA.
[12]
Nils Reimers and Iryna Gurevych. 2019. Sentence-BERT: Sentence Embeddings using Siamese BERT-Networks. In Proceedings of the 2019 Conference EMNLP. Association for Computational Linguistics.
[13]
Jun Xu, Tao Mei, Ting Yao, and Yong Rui. 2016. MSR-VTT: A Large Video Description Dataset for Bridging Video and Language. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR) . 5288--5296.
[14]
Emine Yilmaz, Evangelos Kanoulas, and Javed A. Aslam. 2008. A simple and efficient sampling method for estimating AP and NDCG. In Proceedings of the 31st International ACM SIGIR 2008. 603--610.
Index Terms
- Automatic and Semi-automatic Augmentation of Migration Related Semantic Concepts for Visual Media Retrieval
Recommendations
Visual ontology construction for digitized art image retrieval
Current investigations on visual information retrieval are generally content-based methods. The significant difference between similarity in low-level features and similarity in high-level semantic meanings is still a major challenge in the area of ...
Information retrieval from relational databases using semantic queries
CIKM '06: Proceedings of the 15th ACM international conference on Information and knowledge managementRelational databases are widely used today as a mechanism for providing access to structured data. They, however, are not suitable for typical information finding tasks of end users. There is often a semantic gap between the queries users want to ...
Semi-automatic ontology construction for visual media web service
ICUIMC '08: Proceedings of the 2nd international conference on Ubiquitous information management and communicationOntology is a description of the concepts and relationships that can exist for an agent or a community of agents. Ontologies are a key enabling technology for the Semantic Web. Previously, ontologies have been built manually by knowledge engineers using ...
Comments
Information & Contributors
Information
Published In
October 2021
44 pages
Copyright © 2021 ACM.
Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than the author(s) must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected].
Sponsors
Publisher
Association for Computing Machinery
New York, NY, United States
Publication History
Published: 28 October 2021
Check for updates
Author Tags
Qualifiers
- Research-article
Funding Sources
Conference
Contributors
Other Metrics
Bibliometrics & Citations
Bibliometrics
Article Metrics
- 0Total Citations
- 40Total Downloads
- Downloads (Last 12 months)8
- Downloads (Last 6 weeks)0
Reflects downloads up to 17 Oct 2024
Other Metrics
Citations
View Options
Get Access
Login options
Check if you have access through your login credentials or your institution to get full access on this article.
Sign in