Abstract
Machine-assisted photo identification processes require significant amounts of data for each member of a population of interest but offer the possibility to alleviate a significant amount of manual effort. Gathering such data is time consuming and opportunistic, leading to imbalanced datasets ill-suited for traditional machine (deep) learning efforts. Incomplete classifiers, trained on a subset of classes in a population, can be initially useful to identify the most commonly seen individuals. This study investigates the use of incomplete classifiers trained on a subset of often-observed individual killer whales to generate latent space representations of the larger population containing unseen individuals. These semantically relevant representations are subsequently clustered to investigate the efficacy of this method as a secondary identification mechanism. This method proves to be robust to a significant amount of noise while being able to isolate individuals unknown to the classifier when applying limited expert knowledge to the approximate size of the population.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Similar content being viewed by others
References
Bendale, A., Boult, T.E.: Towards open set deep networks. In: 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 1563–1572 (2016). https://doi.org/10.1109/CVPR.2016.173
Bergler, C., et al.: FIN-PRINT a fully-automated multi-stage deep-learning-based framework for the individual recognition of killer whales. Sci. Rep. 11, 1–16 (2021). https://doi.org/10.1038/s41598-021-02506-6
Bigg, M., Olesiuk, P., Ellis, G.M., Ford, J., Balcomb, K.C.: Social organization and genealogy of resident killer whales (orcinus orca) in the coastal waters of British Columbia and washington state. Report Int. Whaling Commission 12, 383–405 (1990)
Bogucki, R., Cygan, M., Khan, C.B., Klimek, M., Milczek, J.K., Mucha, M.: Applying deep learning to right whale photo identification. Conserv. Biol. 33(3), 676–684 (Nov 2018). https://doi.org/10.1111/cobi.13226
Caruana, R., Elhawary, M., Nguyen, N., Smith, C.: Meta clustering. In: Sixth International Conference on Data Mining (ICDM’06). pp. 107–118 (2006). https://doi.org/10.1109/ICDM.2006.103
Ford, J.K., Ellis, G.M., Balcomb, K.C.: Killer whales: the natural history and genealogy of Orcinus orca in British Columbia and Washington. UBC press (1994)
Gómez Blas, N., de Mingo López, L.F., Arteta Albert, A., Martínez Llamas, J.: Image classification with convolutional neural networks using gulf of maine humpback whale catalog. Electronics 9(5) (2020). https://doi.org/10.3390/electronics9050731, https://www.mdpi.com/2079-9292/9/5/731
Hassen, M., Chan, P.K.: Learning a neural-network-based representation for open set recognition, pp. 154–162. https://doi.org/10.1137/1.9781611976236.18, https://epubs.siam.org/doi/abs/10.1137/1.9781611976236.18
Hu, M., You, F.: Research on animal image classification based on transfer learning. In: Proceedings of the 2020 4th International Conference on Electronic Information Technology and Computer Engineering, pDp. 756–761. EITCE ’20, Association for Computing Machinery, New York, NY, USA (2021). https://doi.org/10.1145/3443467.3443849
Jin, X., et al.: Meta clustering learning for large-scale unsupervised person re-identification. In: Proceedings of the 30th ACM International Conference on Multimedia. ACM (Oct 2022). https://doi.org/10.1145/3503161.3547900, https://doi.org/10.1145/3503161.3547900
Kirillov, A., et al.: Segment anything (2023). https://doi.org/10.48550/ARXIV.2304.02643, https://arxiv.org/abs/2304.02643
Konovalov, D.A., Hillcoat, S., Williams, G., Birtles, R.A., Gardiner, N., Curnock, M.I.: Individual minke whale recognition using deep learning convolutional neural networks. J. Geosci. Environ. Protect. 6, 25–36 (2018)
Maglietta, R., et al.: DolFin: an innovative digital platform for studying risso’s dolphins in the northern ionian sea (north-eastern central mediterranean). Sci. Reports 8(1) (Nov 2018). https://doi.org/10.1038/s41598-018-35492-3
Patton, P.T., et al.: A deep learning approach to photo–identification demonstrates high performance on two dozen cetacean species. Methods in Ecology and Evolution (Jul 2023). https://doi.org/10.1111/2041-210x.14167
Renò, V., et al.: A sift-based software system for the photo-identification of the risso’s dolphin. Ecol. Inform. 50, 95–101 (2019). https://doi.org/10.1016/j.ecoinf.2019.01.006, https://www.sciencedirect.com/science/article/pii/S1574954118301377
Rosenberg, A., Hirschberg, J.: V-measure: A conditional entropy-based external cluster evaluation measure. In: Eisner, J. (ed.) EMNLP-CoNLL 2007, Proceedings of the 2007 Joint Conference on Empirical Methods in Natural Language Processing and Computational Natural Language Learning, June 28-30, 2007, Prague, Czech Republic, pp. 410–420. ACL (2007), https://aclanthology.org/D07-1043/
Scheirer, W.J., de Rezende Rocha, A., Sapkota, A., Boult, T.E.: Toward open set recognition. IEEE Trans. Pattern Anal. Mach. Intell. 35(7), 1757–1772 (2013). https://doi.org/10.1109/TPAMI.2012.256
Thompson, J.W., et al.: finfindr: Automated recognition and identification of marine mammal dorsal fins using residual convolutional neural networks. Marine Mammal Sci. 38(1), 139–150 (2022). https://doi.org/10.1111/mms.12849, https://onlinelibrary.wiley.com/doi/abs/10.1111/mms.12849
Towers, J.R. et al.: Photo-identification catalogue, population status, and distribution of bigg’s killer whales known from coastal waters of British Columbia, Canada. Canadian Tech. Report Fisheries Aquatic Sci. 3311 (07 2019)
Wang, H., Zhu, X., Xiang, T., Gong, S.: Towards unsupervised open-set person re-identification. In: 2016 IEEE International Conference on Image Processing (ICIP), pp. 769–773 (2016). https://doi.org/10.1109/ICIP.2016.7532461
Wheeldon, A., Serb, A.: A study on the clusterability of latent representations in image pipelines. Front. Neuroinform. 17 (Feb 2023). https://doi.org/10.3389/fninf.2023.1074653, https://doi.org/10.3389/fninf.2023.1074653
Zeiler, M.D., Fergus, R.: Visualizing and understanding convolutional networks. In: Fleet, D., Pajdla, T., Schiele, B., Tuytelaars, T. (eds.) Computer Vision - ECCV 2014, pp. 818–833. Springer International Publishing, Cham (2014)
Zhang, Y., Kang, B., Hooi, B., Yan, S., Feng, J.: Deep long-tailed learning: A survey. IEEE Transactions on Pattern Analysis and Machine Intelligence (2023)
Zheng, X., Kellenberger, B., Gong, R., Hajnsek, I., Tuia, D.: Self-supervised pretraining and controlled augmentation improve rare wildlife recognition in uav images. In: Proceedings of the IEEE/CVF International Conference on Computer Vision (ICCV) Workshops, pp. 732–741 (October 2021)
Acknowledgements
The authors would like to thank Ellyne Hamran with Ocean Sounds e.V. for the Norwegian killer whale photos used as comparison in this work, Lorenzo von Fersen with the Nuremburg Zoo for allowing the use of the bottlenose dolphin photos, and Gary Sutton and Tasli Shaw for their efforts in annotating the Bigg’s killer whale data.
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2025 The Author(s), under exclusive license to Springer Nature Switzerland AG
About this paper
Cite this paper
Barnhill, A., Towers, J.R., Nöth, E., Maier, A., Bergler, C. (2025). Utilizing Deep Incomplete Classifiers to Implement Semantic Clustering for Killer Whale Photo Identification Data. In: Antonacopoulos, A., Chaudhuri, S., Chellappa, R., Liu, CL., Bhattacharya, S., Pal, U. (eds) Pattern Recognition. ICPR 2024. Lecture Notes in Computer Science, vol 15301. Springer, Cham. https://doi.org/10.1007/978-3-031-78107-0_22
Download citation
DOI: https://doi.org/10.1007/978-3-031-78107-0_22
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-031-78106-3
Online ISBN: 978-3-031-78107-0
eBook Packages: Computer ScienceComputer Science (R0)