research-article

Reviving the Context: Camera Trap Species Classification as Link Prediction on Multimodal Knowledge Graphs

Authors:

Vardaan Pahuja,

Tanya Berger-Wolf,

Charles Stewart,

Yu SuAuthors Info & Claims

CIKM '24: Proceedings of the 33rd ACM International Conference on Information and Knowledge Management

Pages 1825 - 1835

https://doi.org/10.1145/3627673.3679545

Published: 21 October 2024 Publication History

Abstract

Camera traps are important tools in animal ecology for biodiversity monitoring and conservation. However, their practical application is limited by issues such as poor generalization to new and unseen locations. Images are typically associated with diverse forms of context, which may exist in different modalities. In this work, we exploit the structured context linked to camera trap images to boost out-of-distribution generalization for species classification tasks in camera traps. For instance, a picture of a wild animal could be linked to details about the time and place it was captured, as well as structured biological knowledge about the animal species. While often overlooked by existing studies, incorporating such context offers several potential benefits for better image understanding, such as addressing data scarcity and enhancing generalization. However, effectively incorporating such heterogeneous context into the visual domain is a challenging problem. To address this, we propose a novel framework that transforms species classification as link prediction in a multimodal knowledge graph (KG). This framework enables the seamless integration of diverse multimodal contexts for visual recognition. We apply this framework for out-of-distribution species classification on the iWildCam2020-WILDS and Snapshot Mountain Zebra datasets and achieve competitive performance with state-of-the-art approaches. Furthermore, our framework enhances sample efficiency for recognizing under-represented species.

References

[1]

Jorge A Ahumada, Eric Fegraus, Tanya Birch, Nicole Flores, Roland Kays, Timothy G O'Brien, Jonathan Palmer, Stephanie Schuttler, Jennifer Y Zhao, Walter Jetz, et al. 2020. Wildlife insights: A platform to maximize the potential of camera trap and other passive sensor wildlife data for the planet. Environmental Conservation, Vol. 47, 1 (2020), 1--6.

[2]

Rosamund EA Almond, Monique Grooten, and T Peterson. 2020. Living Planet Report 2020-Bending the curve of biodiversity loss. World Wildlife Fund.

[3]

Bilal Alsallakh, Amin Jourabloo, Mao Ye, Xiaoming Liu, and Liu Ren. 2018. Do Convolutional Neural Networks Learn Class Hierarchy? IEEE Trans. Vis. Comput. Graph., Vol. 24, 1 (2018), 152--162. https://doi.org/10.1109/TVCG.2017.2744683

[4]

Elissa Aminoff, Nurit Gronau, and Moshe Bar. 2007. The parahippocampal cortex mediates spatial and nonspatial associations. Cerebral cortex, Vol. 17, 7 (2007), 1493--1503.

[5]

Moshe Bar. 2004. Visual objects in context. Nature Reviews Neuroscience, Vol. 5, 8 (2004), 617--629.

[6]

Suchet Bargoti and James Underwood. 2016. Image classification with orchard metadata. In 2016 IEEE International Conference on Robotics and Automation (ICRA). IEEE, 5164--5170.

Digital Library

[7]

Sara Beery, Elijah Cole, and Arvi Gjoka. 2020. The iWildCam 2020 Competition Dataset. arxiv: 2004.10340 [cs.CV] https://arxiv.org/abs/2004.10340

[8]

Sara Beery, Grant van Horn, Oisin Mac Aodha, and Pietro Perona. 2019. The iWildCam 2018 Challenge Dataset. arxiv: 1904.05986 [cs.CV] https://arxiv.org/abs/1904.05986

[9]

Sara Beery, Grant Van Horn, and Pietro Perona. 2018. Recognition in terra incognita. In Proceedings of the European conference on computer vision (ECCV). 456--473.

Digital Library

[10]

Luca Bertinetto, Romain Müller, Konstantinos Tertikas, Sina Samangooei, and Nicholas A. Lord. 2020. Making Better Mistakes: Leveraging Class Hierarchies With Deep Networks. In 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition, CVPR 2020, Seattle, WA, USA, June 13--19, 2020. Computer Vision Foundation / IEEE, 12503--12512. https://doi.org/10.1109/CVPR42600.2020.01252

[11]

Anil Bhattacharyya. 1946. On a measure of divergence between two multinomial populations. Sankhy=a: the indian journal of statistics (1946), 401--406.

[12]

Antoine Bordes, Nicolas Usunier, Alberto García-Durán, Jason Weston, and Oksana Yakhnenko. 2013. Translating Embeddings for Modeling Multi-relational Data. In Advances in Neural Information Processing Systems 26: 27th Annual Conference on Neural Information Processing Systems 2013. Proceedings of a meeting held December 5--8, 2013, Lake Tahoe, Nevada, United States, Christopher J. C. Burges, Léon Bottou, Zoubin Ghahramani, and Kilian Q. Weinberger (Eds.). 2787--2795. https://proceedings.neurips.cc/paper/2013/hash/1cecc7a77928ca8133fa24680a88d2f9-Abstract.html

Digital Library

[13]

Ludwig Bothmann, Lisa Wimmer, Omid Charrakh, Tobias Weber, Hendrik Edelhoff, Wibke Peters, Hien Nguyen, Caryl Benjamin, and Annette Menzel. 2023. Automated wildlife image classification: An active learning tool for ecological applications. Ecol. Informatics, Vol. 77 (2023), 102231. https://doi.org/10.1016/J.ECOINF.2023.102231

[14]

Ohio Supercomputer Center. 1987. Ohio Supercomputer Center. http://osc.edu/ark:/19495/f5s1ph73

[15]

Sanxing Chen, Xiaodong Liu, Jianfeng Gao, Jian Jiao, Ruofei Zhang, and Yangfeng Ji. 2021. HittER: Hierarchical Transformers for Knowledge Graph Embeddings. In Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing. Association for Computational Linguistics, Online and Punta Cana, Dominican Republic, 10395--10407. https://doi.org/10.18653/v1/2021.emnlp-main.812

[16]

Grace Chu, Brian Potetz, Weijun Wang, Andrew Howard, Yang Song, Fernando Brucher, Thomas Leung, and Hartwig Adam. 2019. Geo-Aware Networks for Fine-Grained Recognition. In 2019 IEEE/CVF International Conference on Computer Vision Workshops, ICCV Workshops 2019, Seoul, Korea (South), October 27--28, 2019. IEEE, 247--254. https://doi.org/10.1109/ICCVW.2019.00033

[17]

Tim Dettmers, Pasquale Minervini, Pontus Stenetorp, and Sebastian Riedel. 2018. Convolutional 2d knowledge graph embeddings. In Thirty-second AAAI conference on artificial intelligence.

[18]

Qishuai Diao, Yi Jiang, Bin Wen, Jia Sun, and Zehuan Yuan. 2022. MetaFormer: A Unified Meta Framework for Fine-Grained Recognition. arxiv: 2203.02751 [cs.CV] https://arxiv.org/abs/2203.02751

[19]

Sandra Díaz, Josef Settele, Eduardo S Brondízio, Hien T Ngo, John Agard, Almut Arneth, Patricia Balvanera, Kate A Brauman, Stuart HM Butchart, Kai MA Chan, et al. 2019. Pervasive human-driven decline of life on Earth points to the need for transformative change. Science, Vol. 366, 6471 (2019), eaax3100.

[20]

Xin Dong, Evgeniy Gabrilovich, Geremy Heitz, Wilko Horn, Ni Lao, Kevin Murphy, Thomas Strohmann, Shaohua Sun, and Wei Zhang. 2014. Knowledge vault: A web-scale approach to probabilistic knowledge fusion. In Proceedings of the 20th ACM SIGKDD international conference on Knowledge discovery and data mining. 601--610.

Digital Library

[21]

Jeffrey S Ellen, Casey A Graff, and Mark D Ohman. 2019. Improving plankton image classification using context metadata. Limnology and Oceanography: Methods, Vol. 17, 8 (2019), 439--461.

[22]

Alberto García-Durán and Mathias Niepert. 2018. KBlrn: End-to-End Learning of Knowledge Base Representations with Latent, Relational, and Numerical Features. In Conference on Uncertainty in Artificial Intelligence. http://auai.org/uai2018/proceedings/papers/149.pdf

[23]

Paul Glover-Kapfer, Carolina A Soto-Navarro, and Oliver R Wearn. 2019. Camera-trapping version 3.0: current constraints and future priorities for development. Remote Sensing in Ecology and Conservation, Vol. 5, 3 (2019), 209--223.

[24]

Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun. 2016. Deep Residual Learning for Image Recognition. In 2016 IEEE Conference on Computer Vision and Pattern Recognition, CVPR 2016, Las Vegas, NV, USA, June 27--30, 2016. IEEE Computer Society, 770--778. https://doi.org/10.1109/CVPR.2016.90

[25]

Grant Van Horn, Oisin Mac Aodha, Yang Song, Yin Cui, Chen Sun, Alexander Shepard, Hartwig Adam, Pietro Perona, and Serge J. Belongie. 2018. The INaturalist Species Classification and Detection Dataset. In 2018 IEEE Conference on Computer Vision and Pattern Recognition, CVPR 2018, Salt Lake City, UT, USA, June 18--22, 2018. Computer Vision Foundation / IEEE Computer Society, 8769--8778. https://doi.org/10.1109/CVPR.2018.00914

[26]

Weihua Hu, Gang Niu, Issei Sato, and Masashi Sugiyama. 2018. Does Distributionally Robust Supervised Learning Give Robust Classifiers?. In Proceedings of the 35th International Conference on Machine Learning, ICML 2018, Stockholmsmässan, Stockholm, Sweden, July 10--15, 2018 (Proceedings of Machine Learning Research, Vol. 80), Jennifer G. Dy and Andreas Krause (Eds.). PMLR, 2034--2042. http://proceedings.mlr.press/v80/hu18a.html

[27]

Mirantha Jayathilaka, Tingting Mu, and Uli Sattler. 2021. Ontology-based n-ball Concept Embeddings Informing Few-shot Image Classification. In Machine Learning with Symbolic Methods and Knowledge Graphs co-located with European Conference on Machine Learning and Principles and Practice of Knowledge Discovery in Databases (ECML PKDD 2021), Virtual, September 17, 2021 (CEUR Workshop Proceedings, Vol. 2997), Mehwish Alam, Mehdi Ali, Paul Groth, Pascal Hitzler, Jens Lehmann, Heiko Paulheim, Achim Rettinger, Harald Sack, Afshin Sadeghi, and Volker Tresp (Eds.). CEUR-WS.org. https://ceur-ws.org/Vol-2997/paper1.pdf

[28]

Justin Johnson, Lamberto Ballan, and Li Fei-Fei. 2015. Love Thy Neighbors: Image Annotation by Exploiting Image Metadata. In 2015 IEEE International Conference on Computer Vision, ICCV 2015, Santiago, Chile, December 7--13, 2015. IEEE Computer Society, 4624--4632. https://doi.org/10.1109/ICCV.2015.525

Digital Library

[29]

Diederik P. Kingma and Jimmy Ba. 2015. Adam: A Method for Stochastic Optimization. In 3rd International Conference on Learning Representations, ICLR 2015, San Diego, CA, USA, May 7--9, 2015, Conference Track Proceedings, Yoshua Bengio and Yann LeCun (Eds.). http://arxiv.org/abs/1412.6980

[30]

Pang Wei Koh, Shiori Sagawa, Henrik Marklund, Sang Michael Xie, Marvin Zhang, Akshay Balsubramani, Weihua Hu, Michihiro Yasunaga, Richard Lanas Phillips, Irena Gao, Tony Lee, Etienne David, Ian Stavness, Wei Guo, Berton Earnshaw, Imran S. Haque, Sara M. Beery, Jure Leskovec, Anshul Kundaje, Emma Pierson, Sergey Levine, Chelsea Finn, and Percy Liang. 2021. WILDS: A Benchmark of in-the-Wild Distribution Shifts. In Proceedings of the 38th International Conference on Machine Learning, ICML 2021, 18--24 July 2021, Virtual Event (Proceedings of Machine Learning Research, Vol. 139), Marina Meila and Tong Zhang (Eds.). PMLR, 5637--5664. http://proceedings.mlr.press/v139/koh21a.html

[31]

Ranjay Krishna, Yuke Zhu, Oliver Groth, Justin Johnson, Kenji Hata, Joshua Kravitz, Stephanie Chen, Yannis Kalantidis, Li-Jia Li, David A. Shamma, Michael S. Bernstein, and Li Fei-Fei. 2017. Visual Genome: Connecting Language and Vision Using Crowdsourced Dense Image Annotations. Int. J. Comput. Vis., Vol. 123, 1 (2017), 32--73. https://doi.org/10.1007/S11263-016-0981--7

Digital Library

[32]

Wen Li, Li Niu, and Dong Xu. 2014. Exploiting Privileged Information from Web Data for Image Categorization. In Computer Vision - ECCV 2014 - 13th European Conference, Zurich, Switzerland, September 6--12, 2014, Proceedings, Part V (Lecture Notes in Computer Science, Vol. 8693), David J. Fleet, Tomás Pajdla, Bernt Schiele, and Tinne Tuytelaars (Eds.). Springer, 437--452. https://doi.org/10.1007/978--3--319--10602--1_29

[33]

Xinhang Li, Xiangyu Zhao, Jiaxing Xu, Yong Zhang, and Chunxiao Xing. 2023. IMF: Interactive Multimodal Fusion Model for Link Prediction. In Proceedings of the ACM Web Conference 2023. 2572--2580.

Digital Library

[34]

Yankai Lin, Zhiyuan Liu, Maosong Sun, Yang Liu, and Xuan Zhu. 2015. Learning Entity and Relation Embeddings for Knowledge Graph Completion. In Proceedings of the Twenty-Ninth AAAI Conference on Artificial Intelligence, January 25--30, 2015, Austin, Texas, USA, Blai Bonet and Sven Koenig (Eds.). AAAI Press, 2181--2187. http://www.aaai.org/ocs/index.php/AAAI/AAAI15/paper/view/9571

Digital Library

[35]

Chengjiang Long, Roddy Collins, Eran Swears, and Anthony Hoogs. 2019. Deep Neural Networks in Fully Connected CRF for Image Labeling with Social Network Metadata. In IEEE Winter Conference on Applications of Computer Vision, WACV 2019, Waikoloa Village, HI, USA, January 7--11, 2019. IEEE, 1607--1615. https://doi.org/10.1109/WACV.2019.00176

[36]

Kenneth Marino, Ruslan Salakhutdinov, and Abhinav Gupta. 2017. The More You Know: Using Knowledge Graphs for Image Classification. In 2017 IEEE Conference on Computer Vision and Pattern Recognition, CVPR 2017, Honolulu, HI, USA, July 21--26, 2017. IEEE Computer Society, 20--28. https://doi.org/10.1109/CVPR.2017.10

[37]

Sean L Maxwell, Richard A Fuller, Thomas M Brooks, and James EM Watson. 2016. Biodiversity: The ravages of guns, nets and bulldozers. Nature, Vol. 536, 7615 (2016), 143--145.

[38]

Julian J. McAuley and Jure Leskovec. 2012. Image Labeling on a Network: Using Social-Network Metadata for Image Classification. In Computer Vision - ECCV 2012 - 12th European Conference on Computer Vision, Florence, Italy, October 7--13, 2012, Proceedings, Part IV (Lecture Notes in Computer Science, Vol. 7575), Andrew W. Fitzgibbon, Svetlana Lazebnik, Pietro Perona, Yoichi Sato, and Cordelia Schmid (Eds.). Springer, 828--841. https://doi.org/10.1007/978--3--642--33765--9_59

[39]

Zhongqi Miao, Kaitlyn M Gaynor, Jiayun Wang, Ziwei Liu, Oliver Muellerklein, Mohammad Sadegh Norouzzadeh, Alex McInturff, Rauri CK Bowie, Ran Nathan, Stella X Yu, et al. 2019. Insights and approaches using deep learning to classify wildlife. Scientific reports, Vol. 9, 1 (2019), 8137.

[40]

George A Miller. 1995. WordNet: a lexical database for English. Commun. ACM, Vol. 38, 11 (1995), 39--41.

Digital Library

[41]

Dai Quoc Nguyen, Tu Dinh Nguyen, Dat Quoc Nguyen, and Dinh Phung. 2018. A Novel Embedding Model for Knowledge Base Completion Based on Convolutional Neural Network. In Proceedings of the 2018 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 2 (Short Papers). Association for Computational Linguistics, New Orleans, Louisiana, 327--333. https://doi.org/10.18653/v1/N18--2053

[42]

Maximilian Nickel, Volker Tresp, and Hans-Peter Kriegel. 2011. A Three-Way Model for Collective Learning on Multi-Relational Data. In Proceedings of the 28th International Conference on Machine Learning, ICML 2011, Bellevue, Washington, USA, June 28 - July 2, 2011, Lise Getoor and Tobias Scheffer (Eds.). Omnipress, 809--816. https://icml.cc/2011/papers/438_icmlpaper.pdf

Digital Library

[43]

Mohammad Sadegh Norouzzadeh, Anh Nguyen, Margaret Kosmala, Alexandra Swanson, Meredith S Palmer, Craig Packer, and Jeff Clune. 2018. Automatically identifying, counting, and describing wild animals in camera-trap images with deep learning. Proceedings of the National Academy of Sciences, Vol. 115, 25 (2018), E5716--E5725.

[44]

Allan F O'Connell, James D Nichols, and K Ullas Karanth. 2011. Camera traps in animal ecology: methods and analyses. Vol. 271. Springer.

[45]

Aude Oliva and Antonio Torralba. 2007. The role of context in object recognition. Trends in cognitive sciences, Vol. 11, 12 (2007), 520--527.

[46]

OpenTreeofLife, Karen A. Cranston, Benjamin Redelings, Luna Luisa Sanchez Reyes, Jim Allman, Emily Jane McTavish, and Mark T. Holder. 2019. Open Tree of Life Taxonomy. https://doi.org/10.5281/zenodo.3937751

[47]

Vardaan Pahuja, Boshi Wang, Hugo Latapie, Jayanth Srinivasa, and Yu Su. 2023. A Retrieve-and-Read Framework for Knowledge Graph Link Prediction. In Proceedings of the 32nd ACM International Conference on Information and Knowledge Management (Birmingham, United Kingdom) (CIKM '23). Association for Computing Machinery, New York, NY, USA, 1992--2002. https://doi.org/10.1145/3583780.3614769

Digital Library

[48]

Lain E Pardo, Sara Bombaci, Sarah E Huebner, Michael J Somers, Herve Fritz, Colleen Downs, Abby Guthmann, Robyn S Hetem, Mark Keith, Aliza le Roux, et al. 2021. Snapshot Safari: A large-scale collaborative to monitor Africa's remarkable biodiversity. South African Journal of Science, Vol. 117, 1--2 (2021), 1--4.

[49]

Pouya Pezeshkpour, Liyan Chen, and Sameer Singh. 2018. Embedding Multimodal Relational Data for Knowledge Base Completion. In Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing. Association for Computational Linguistics, Brussels, Belgium, 3208--3218. https://doi.org/10.18653/v1/D18--1359

[50]

Qi Qi, Yi Xu, Wotao Yin, Rong Jin, and Tianbao Yang. 2023. Attentional-Biased Stochastic Gradient Descent. Transactions on Machine Learning Research (2023). https://openreview.net/forum?id=B0WYWvVA2r

[51]

Daniel Ruffinelli, Samuel Broscheit, and Rainer Gemulla. 2020. You CAN Teach an Old Dog New Tricks! On Training Knowledge Graph Embeddings. In International Conference on Learning Representations. https://openreview.net/forum?id=BkxSmlBFvr

[52]

Mohammad Sadegh Norouzzadeh, Dan Morris, Sara Beery, Neel Joshi, Nebojsa Jojic, and Jeff Clune. 2020. A Deep Active Learning System for Species Identification and Counting in Camera Trap Images. Methods in Ecology and Evolution, Vol. 12, 1 (December 2020), 150--161. https://www.microsoft.com/en-us/research/publication/a-deep-active-learning-system-for-species-identification-and-counting-in-camera-trap-images/

[53]

Apoorv Saxena, Adrian Kochsiek, and Rainer Gemulla. 2022. Sequence-to-Sequence Knowledge Graph Completion and Question Answering. In Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers). Association for Computational Linguistics, Dublin, Ireland, 2814--2828. https://doi.org/10.18653/v1/2022.acl-long.201

[54]

Michael Schlichtkrull, Thomas N Kipf, Peter Bloem, Rianne Van Den Berg, Ivan Titov, and Max Welling. 2018. Modeling relational data with graph convolutional networks. In European Semantic Web Conference. Springer, 593--607.

Digital Library

[55]

Stefan Schneider, Saul Greenberg, Graham W Taylor, and Stefan C Kremer. 2020. Three critical factors affecting automated image species recognition performance for camera traps. Ecology and evolution, Vol. 10, 7 (2020), 3503--3517.

[56]

Stefan Schneider, Graham W. Taylor, and Stefan C. Kremer. 2018. Deep Learning Object Detection Methods for Ecological Camera Trap Data. In 15th Conference on Computer and Robot Vision, CRV 2018, Toronto, ON, Canada, May 8--10, 2018. IEEE Computer Society, 321--328. https://doi.org/10.1109/CRV.2018.00052

[57]

Hatem Mousselly Sergieh, Teresa Botschen, Iryna Gurevych, and Stefan Roth. 2018. A Multimodal Translation-Based Approach for Knowledge Graph Representation Learning. In Proceedings of the Seventh Joint Conference on Lexical and Computational Semantics, *SEM@NAACL-HLT 2018, New Orleans, Louisiana, USA, June 5--6, 2018, Malvina Nissim, Jonathan Berant, and Alessandro Lenci (Eds.). Association for Computational Linguistics, 225--234. https://doi.org/10.18653/v1/s18--2027

[58]

Yuge Shi, Jeffrey Seely, Philip Torr, Siddharth N, Awni Hannun, Nicolas Usunier, and Gabriel Synnaeve. 2022. Gradient Matching for Domain Generalization. In International Conference on Learning Representations. https://openreview.net/forum?id=vDwBW49HmO

[59]

Karen Simonyan and Andrew Zisserman. 2015. Very Deep Convolutional Networks for Large-Scale Image Recognition. In 3rd International Conference on Learning Representations, ICLR 2015, San Diego, CA, USA, May 7--9, 2015, Conference Track Proceedings, Yoshua Bengio and Yann LeCun (Eds.). http://arxiv.org/abs/1409.1556

[60]

Samuel Stevens, Jiaman Wu, Matthew J Thompson, Elizabeth G Campolongo, Chan Hee Song, David Edward Carlyn, Li Dong, Wasila M Dahdul, Charles Stewart, Tanya Berger-Wolf, Wei-Lun Chao, and Yu Su. 2024. BioCLIP: A Vision Foundation Model for the Tree of Life. In 2024 IEEE Conference on Computer Vision and Pattern Recognition, CVPR 2024, Seattle, WA, USA, June 17--21, 2024. Computer Vision Foundation / IEEE Computer Society, 19412--19424. https://openaccess.thecvf.com/content/CVPR2024/html/Stevens_BioCLIP_A_Vision_Foundation_Model_for_the_Tree_of_Life_CVPR_2024_paper.html

[61]

Baochen Sun and Kate Saenko. 2016. Deep CORAL: Correlation Alignment for Deep Domain Adaptation. In Computer Vision - ECCV 2016 Workshops - Amsterdam, The Netherlands, October 8--10 and 15--16, 2016, Proceedings, Part III (Lecture Notes in Computer Science, Vol. 9915), Gang Hua and Hervé Jégou (Eds.). 443--450. https://doi.org/10.1007/978--3--319--49409--8_35

[62]

Zhiqing Sun, Shikhar Vashishth, Soumya Sanyal, Partha Talukdar, and Yiming Yang. 2020. A Re-evaluation of Knowledge Graph Completion Methods. In Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics, Dan Jurafsky, Joyce Chai, Natalie Schluter, and Joel Tetreault (Eds.). Association for Computational Linguistics, Online, 5516--5522. https://doi.org/10.18653/v1/2020.acl-main.489

[63]

Michael A Tabak, Mohammad S Norouzzadeh, David W Wolfson, Erica J Newton, Raoul K Boughton, Jacob S Ivan, Eric A Odell, Eric S Newkirk, Reesa Y Conrey, Jennifer Stenglein, et al. 2020. Improving the accessibility and transferability of machine learning algorithms for identification of animals in camera trap images: MLWIC2. Ecology and evolution, Vol. 10, 19 (2020), 10374--10383.

[64]

Michael A Tabak, Mohammad S Norouzzadeh, David W Wolfson, Steven J Sweeney, Kurt C VerCauteren, Nathan P Snow, Joseph M Halseth, Paul A Di Salvo, Jesse S Lewis, Michael D White, et al. 2019. Machine learning to classify animal species in camera trap images: Applications in ecology. Methods in Ecology and Evolution, Vol. 10, 4 (2019), 585--590.

[65]

Shikhar Vashishth, Soumya Sanyal, Vikram Nitin, and Partha P. Talukdar. 2020. Composition-based Multi-Relational Graph Convolutional Networks. In 8th International Conference on Learning Representations, ICLR 2020, Addis Ababa, Ethiopia, April 26--30, 2020. OpenReview.net. https://openreview.net/forum?id=BylA_C4tPr

[66]

Jean-Christophe Vié, Craig Hilton-Taylor, Caroline Pollock, James Ragle, Jane Smart, Simon N Stuart, and Rashila Tong. 2009. The IUCN Red List: a key conservation tool. Wildlife in a changing world--An analysis of the 2008 IUCN Red List of Threatened Species (2009), 1.

[67]

OR Wearn and P Glover-Kapfer. 2017. Camera-trapping for conservation: a guide to best-practices. WWF conservation technology series, Vol. 1, 1 (2017), 181.

[68]

Ben G Weinstein. 2018. A computer vision for animal ecology. Journal of Animal Ecology, Vol. 87, 3 (2018), 533--545.

[69]

Xander Wilcke, Peter Bloem, Victor de Boer, and Rein van 't Veer. 2021. End-to-End Learning on Multimodal Knowledge Graphs. Semantic Web -- Interoperability, Usability, Applicability an IOS Press Journal (2021). https://www.semantic-web-journal.net/content/end-end-learning-multimodal-knowledge-graphs

[70]

Bishan Yang, Wen-tau Yih, Xiaodong He, Jianfeng Gao, and Li Deng. 2015. Embedding Entities and Relations for Learning and Inference in Knowledge Bases. In 3rd International Conference on Learning Representations, ICLR 2015, San Diego, CA, USA, May 7--9, 2015, Conference Track Proceedings, Yoshua Bengio and Yann LeCun (Eds.). http://arxiv.org/abs/1412.6575

[71]

Liang Yao, Chengsheng Mao, and Yuan Luo. 2019. KG-BERT: BERT for Knowledge Graph Completion. arxiv: 1909.03193 [cs.CL] https://arxiv.org/abs/1909.03193

[72]

Donghan Yu, Yiming Yang, Ruohong Zhang, and Yuexin Wu. 2021. Knowledge Embedding Based Graph Convolutional Network. In WWW '21: The Web Conference 2021, Virtual Event / Ljubljana, Slovenia, April 19--23, 2021, Jure Leskovec, Marko Grobelnik, Marc Najork, Jie Tang, and Leila Zia (Eds.). ACM / IW3C2, 1619--1628. https://doi.org/10.1145/3442381.3449925

Digital Library

[73]

Shu Zhang, Ran Xu, Caiming Xiong, and Chetan Ramaiah. 2022. Use All The Labels: A Hierarchical Multi-Label Contrastive Learning Framework. In IEEE/CVF Conference on Computer Vision and Pattern Recognition, CVPR 2022, New Orleans, LA, USA, June 18--24, 2022. IEEE, 16639--16648. https://doi.org/10.1109/CVPR52688.2022.01616

Cited By

Chalmers CFergus PWich SLongmore SWalsh NOliver LWarrington JQuinlan JAppleby K(2025)AI-Driven Real-Time Monitoring of Ground-Nesting Birds: A Case Study on Curlew Detection Using YOLOv10Remote Sensing10.3390/rs1705076917:5(769)Online publication date: 23-Feb-2025
https://doi.org/10.3390/rs17050769

Index Terms

Reviving the Context: Camera Trap Species Classification as Link Prediction on Multimodal Knowledge Graphs
1. Computing methodologies
  1. Artificial intelligence
    1. Computer vision
      1. Computer vision problems
        Object recognition
    2. Knowledge representation and reasoning
      1. Semantic networks

Recommendations

Fast face detection and species classification of African great apes
AVSS '11: Proceedings of the 2011 8th IEEE International Conference on Advanced Video and Signal Based Surveillance

Decline of biodiversity has become a serious problem on a global scale. Therefore environmental groups try to protect the endangered species populations. To this end, autonomous video traps are powerful instruments to monitor population sizes in the ...
Multi-species comparative genomics: synteny and genome rearrangements
MKVSE: Multimodal Knowledge Enhanced Visual-semantic Embedding for Image-text Retrieval
Image-text retrieval aims to take the text (image) query to retrieve the semantically relevant images (texts), which is fundamental and critical in the search system, online shopping, and social network. Existing works have shown the effectiveness of ...

Comments

Information & Contributors

Information

Published In

cover image ACM Conferences

CIKM '24: Proceedings of the 33rd ACM International Conference on Information and Knowledge Management

October 2024

5705 pages

ISBN:9798400704369

DOI:10.1145/3627673

General Chairs:
Edoardo Serra
Boise State University, USA
,
Francesca Spezzano
Boise State University, USA

Copyright © 2024 ACM.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than the author(s) must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected].

Sponsors

SIGIR: ACM Special Interest Group on Information Retrieval

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 21 October 2024

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Research-article

Funding Sources

Conference

CIKM '24

Sponsor:

SIGIR

CIKM '24: The 33rd ACM International Conference on Information and Knowledge Management

October 21 - 25, 2024

ID, Boise, USA

Acceptance Rates

Overall Acceptance Rate 1,861 of 8,427 submissions, 22%

Upcoming Conference

CIKM '25

Sponsor:
sigir
sigir

The 34th ACM International Conference on Information and Knowledge Management

November 10 - 14, 2025

Seoul , Republic of Korea

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

1
Total Citations
View Citations
115
Total Downloads

Downloads (Last 12 months)115
Downloads (Last 6 weeks)26

Reflects downloads up to 07 Mar 2025

Other Metrics

View Author Metrics

Citations

Cited By

Chalmers CFergus PWich SLongmore SWalsh NOliver LWarrington JQuinlan JAppleby K(2025)AI-Driven Real-Time Monitoring of Ground-Nesting Birds: A Case Study on Curlew Detection Using YOLOv10Remote Sensing10.3390/rs1705076917:5(769)Online publication date: 23-Feb-2025
https://doi.org/10.3390/rs17050769

View Options

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Figures

Tables

Media

View Table of Conten