Abstract
In the task of Fine-Grained Image Recognition (FGIR), the overall difference between different types of images is slight, so locating the representative local region in the image is the key to improving the classification accuracy. This idea of FGIR has been widely used in previous work, and has achieved good results on the benchmark dataset. Recently, the proposal of the Vision Transformer (ViT) method, provides a new method for the field of computer vision. Compared with the previous work based on Convolutional Neural Network (CNN), it has achieved better performance. ViT performs well in general image recognition tasks. However, when applied to FGIR tasks, it only pays attention to the global information and does not pay enough attention to the local features with discrimination. In order to make the model pay more attention to differentiated local regions, we propose an attention-based local region merging method Group Attention Transformer (GA-Trans), which evaluates the importance of each patch by using the self-attention weight inside the Transformer, and then aggregates adjacent high weight attention blocks into groups, then randomly select groups for image crop and drop. Through the weight sharing encoder, the global and local regions of the image are classified after obtaining the features respectively, which is convenient to realize the end-to-end training. Comprehensive experiments show that GA-Trans can achieve state-of-the-art performance on multiple benchmark datasets.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Similar content being viewed by others
References
Wei, X., Xie, C., Wu, J.: Mask-CNN: localizing parts and selecting descriptors for fine-grained image recognition. arXiv:1605.06878 (2016)
Xie, L., et al.: Hierarchical part matching for fine-grained visual categorization. In: Proceedings of the IEEE International Conference on Computer Vision (2013)
Branson, S., et al.: Bird species categorization using pose normalized deep convolutional nets. arXiv:1406.2952 (2014)
Huang, S., et al.: Part-stacked CNN for fine-grained visual categorization. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (2016)
Simonyan, K., Zisserman A.: Very deep convolutional networks for large-scale image recognition. arXiv:1409.1556 (2014)
He, K., et al.: Deep residual learning for image recognition. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (2016)
Dosovitskiy, A., et al.: An image is worth 16x16 words: transformers for image recognition at scale. arXiv:2010.11929 (2020)
He, J., et al.: TransFG: a transformer architecture for fine-grained recognition. arXiv:2103.07976 (2021)
Lin, T., Aruni, R., Subhransu, M.: Bilinear CNN models for fine-grained visual recognition. In: Proceedings of the IEEE International Conference on Computer Vision (2015)
Chen, Y., et al.: Destruction and construction learning for fine-grained image recognition. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (2019)
Jiang, S., et al.: Multi-scale multi-view deep feature aggregation for food recognition. IEEE Trans. Image Process. 29, 265–276 (2019)
Ge, W., Lin, X., Yu, Y.: Weakly supervised complementary parts models for fine-grained image classification from the bottom up. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 3034–3043 (2019)
Hu, T., et al.: See better before looking closer: weakly supervised data augmentation network for fine-grained visual classification. arXiv:1901.09891 (2019)
Vaswani, A., et al.: Attention is all you need. In: Advances in Neural Information Processing Systems (2017)
Devlin, J., et al.: BERT: pre-training of deep bidirectional transformers for language understanding. arXiv:1810.04805 (2018)
Yang, Z., et al.: XLNet: generalized autoregressive pretraining for language understanding. In: Advances in Neural Information Processing Systems (2019)
Carion, N., Massa, F., Synnaeve, G., Usunier, N., Kirillov, A., Zagoruyko, S.: End-to-end object detection with transformers. In: Vedaldi, A., Bischof, H., Brox, T., Frahm, J.-M. (eds.) ECCV 2020. LNCS, vol. 12346, pp. 213–229. Springer, Cham (2020). https://doi.org/10.1007/978-3-030-58452-8_13
Zheng, S., et al.: Rethinking semantic segmentation from a sequence-to-sequence perspective with transformers. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (2021)
Chen, J., et al.: TransUNet: transformers make strong encoders for medical image segmentation. arXiv:2102.04306 (2021)
Xie, E., et al.: Trans2Seg: transparent object segmentation with transformer. arXiv:2101.08461 (2021)
Sun, P., et al.: TransTrack: multiple-object tracking with transformer. arXiv:2012.15460 (2020)
Yang, F., et al.: Learning texture transformer network for image super-resolution. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (2020)
Girdhar, R., et al.: Video action transformer network. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (2019)
Zhuang, P., Wang, Y., Qiao, Y.: Learning attentive pairwise interaction for fine-grained classification. In: Proceedings of the AAAI Conference on Artificial Intelligence, vol. 34, no. 07 (2020)
Dubey, A., et al.: Maximum-entropy fine-grained classification. arXiv:1809.05934 (2018)
Wang, Y., Vlad, I., Morariu, L., Davis, S.: Learning a discriminative filter bank within a CNN for fine-grained recognition. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (2018)
Yang, Z., et al.: Learning to navigate for fine-grained classification. In: Proceedings of the European Conference on Computer Vision (ECCV) (2018)
Luo, W., et al.: Cross-X learning for fine-grained visual categorization. In: Proceedings of the IEEE/CVF International Conference on Computer Vision (2019)
Gao, Y., et al.: Channel interaction networks for fine-grained image categorization. In: Proceedings of the AAAI Conference on Artificial Intelligence, vol. 34, no. 07 (2020)
Zheng, H., et al.: Learning deep bilinear transformation for fine-grained image representation. arXiv:1911.03621 (2019)
Ji, R., et al.: Attention convolutional binary neural tree for fine-grained visual categorization. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (2020)
Ding, Y., et al.: Selective sparse sampling for fine-grained image recognition. In: Proceedings of the IEEE/CVF International Conference on Computer Vision (2019)
Liu, C., et al.: Filtration and distillation: enhancing region attention for fine-grained visual categorization. In: Proceedings of the AAAI Conference on Artificial Intelligence, vol. 34, no. 07 (2020)
Du, R., et al.: Fine-grained visual classification via progressive multi-granularity training of jigsaw patches. In: Vedaldi, A., Bischof, H., Brox, T., Frahm, J.-M. (eds.) ECCV 2020. LNCS, vol. 12365, pp. 153–168. Springer, Cham (2020). https://doi.org/10.1007/978-3-030-58565-5_10
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2022 The Author(s), under exclusive license to Springer Nature Switzerland AG
About this paper
Cite this paper
Yan, B., Wang, S., Zhu, E., Liu, X., Chen, W. (2022). Group-Attention Transformer for Fine-Grained Image Recognition. In: Sun, X., Zhang, X., Xia, Z., Bertino, E. (eds) Advances in Artificial Intelligence and Security. ICAIS 2022. Communications in Computer and Information Science, vol 1587. Springer, Cham. https://doi.org/10.1007/978-3-031-06761-7_4
Download citation
DOI: https://doi.org/10.1007/978-3-031-06761-7_4
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-031-06760-0
Online ISBN: 978-3-031-06761-7
eBook Packages: Computer ScienceComputer Science (R0)