Semantic Filtering Global and Local Embeddings Fusion for Few-Shot Image Classification
Authors: 
Luyu Nong, Min WuAuthors Info & Claims
Luyu Nong, Min WuAuthors Info & ClaimsPages 440 - 447
Abstract
Neural networks have demonstrated superior performance in image classification tasks, however, acquiring sufficient labeled data for training them in real-world scenarios remains challenging. Few-shot learning (FSL) was proposed to address this challenge in low-data scenarios and has attracted considerable attention in recent years. In few-shot image classification tasks, most of the existing FSL methods focus on designing discriminative network structures for exploring image features, either neglecting semantic information that is crucial for accurate classification, or tending to prioritize either global or local image features. To address this issue, we propose in this work a novel model Semantic Filtering Global and Local embeddings for fusion (SFGL) for few-shot image classification. SFGL effectively integrates both global and local features while leveraging semantic information. Extensive experiments conducted on benchmark datasets demonstrate that our proposed SFGL model outperforms state-of-the-art methods in FSL tasks.
References
[1]
Antreas Antoniou, Amos Storkey, and Harrison Edwards. 2017. Data augmentation generative adversarial networks. arXiv preprint arXiv:1711.04340 (2017).
[2]
Luca Bertinetto, Joao F Henriques, Philip HS Torr, 2018. Meta-learning with differentiable closed-form solvers. arXiv preprint arXiv:1805.08136 (2018).
[3]
Irving Biederman. 1987. Recognition-by-components: a theory of human image understanding.Psychological review 94, 2 (1987), 115.
[4]
Zitian Chen, Yanwei Fu, Yinda Zhang, 2019. Multi-level semantic feature augmentation for one-shot learning. IEEE Transactions on Image Processing 28, 9 (2019), 4594–4605.
[5]
Chelsea Finn, Pieter Abbeel, and Sergey Levine. 2017. Model-agnostic meta-learning for fast adaptation of deep networks. In International conference on machine learning. PMLR, 1126–1135.
[6]
Hang Gao, Zheng Shou, Alireza Zareian, 2018. Low-shot learning via covariance-preserving adversarial augmentation networks. Advances in Neural Information Processing Systems 31 (2018).
[7]
Spyros Gidaris and Nikos Komodakis. 2018. Dynamic few-shot visual learning without forgetting. In Proceedings of the IEEE conference on computer vision and pattern recognition. 4367–4375.
[8]
Fusheng Hao, Fengxiang He, Jun Cheng, 2019. Collect and select: Semantic alignment metric learning for few-shot learning. In Proceedings of the IEEE/CVF international Conference on Computer Vision. 8460–8469.
[9]
Fusheng Hao, Fengxiang He, Jun Cheng, Lei Wang, Jianzhong Cao, and Dacheng Tao. 2019. Collect and Select: Semantic Alignment Metric Learning for Few-Shot Learning. In 2019 IEEE/CVF International Conference on Computer Vision, ICCV 2019, Seoul, Korea (South), October 27 - November 2, 2019. IEEE, 8459–8468. https://doi.org/10.1109/ICCV.2019.00855
[10]
Ruibing Hou, Hong Chang, Bingpeng Ma, 2019. Cross attention network for few-shot classification. Advances in neural information processing systems 32 (2019).
[11]
Muhammad Abdullah Jamal and Guo-Jun Qi. 2019. Task agnostic meta-learning for few-shot learning. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. 11719–11727.
[12]
Xiang Jiang, Mohammad Havaei, Farshid Varno, 2018. Learning to learn with conditional class dependencies. In international conference on learning representations.
[13]
Gregory Koch, Richard Zemel, Ruslan Salakhutdinov, 2015. Siamese neural networks for one-shot image recognition. In ICML deep learning workshop, Vol. 2. Lille.
[14]
Aoxue Li, Weiran Huang, Xu Lan, 2020. Boosting few-shot learning with adaptive margin loss. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition. 12576–12584.
[15]
Zechao Li, Hao Tang, Zhimao Peng, 2023. Knowledge-guided semantic transfer network for few-shot image recognition. IEEE Transactions on Neural Networks and Learning Systems (2023).
[16]
Yann Lifchitz, Yannis Avrithis, Sylvaine Picard, 2019. Dense classification and implanting for few-shot learning. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition. 9258–9267.
[17]
Akshay Mehrotra and Ambedkar Dukkipati. 2017. Generative adversarial residual pairwise networks for one shot learning. arXiv preprint arXiv:1703.08033 (2017).
[18]
Alex Nichol, Joshua Achiam, and John Schulman. 2018. On first-order meta-learning algorithms. arXiv preprint arXiv:1803.02999 (2018).
[19]
Jeffrey Pennington, Richard Socher, and Christopher D Manning. 2014. Glove: Global vectors for word representation. In Proceedings of the 2014 conference on empirical methods in natural language processing (EMNLP). 1532–1543.
[20]
Alec Radford, Jong Wook Kim, Chris Hallacy, 2021. Learning transferable visual models from natural language supervision. In International conference on machine learning. PMLR, 8748–8763.
[21]
Tiago Ramalho and Marta Garnelo. 2019. Adaptive posterior learning: few-shot learning with a surprise-based memory module. arXiv preprint arXiv:1902.02527 (2019).
[22]
Mengye Ren, Eleni Triantafillou, Sachin Ravi, 2018. Meta-learning for semi-supervised few-shot classification. arXiv preprint arXiv:1803.00676 (2018).
[23]
Adam Santoro, Sergey Bartunov, Matthew Botvinick, 2016. Meta-learning with memory-augmented neural networks. In International conference on machine learning. PMLR, 1842–1850.
[24]
Boyao Shi, Wenbin Li, Jing Huo, 2023. Global-and local-aware feature augmentation with semantic orthogonality for few-shot image classification. Pattern Recognition 142 (2023), 109702.
[25]
Jake Snell, Kevin Swersky, and Richard Zemel. 2017. Prototypical networks for few-shot learning. Advances in neural information processing systems 30 (2017).
[26]
Jae Woong Soh, Sunwoo Cho, and Nam Ik Cho. 2020. Meta-transfer learning for zero-shot super-resolution. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. 3516–3525.
[27]
Mingze Sun, Weizhi Ma, and Yang Liu. 2022. Global and Local Feature Interaction with Vision Transformer for Few-shot Image Classification. In Proceedings of the 31st ACM International Conference on Information & Knowledge Management. 4530–4534.
[28]
Qianru Sun, Yaoyao Liu, Tat-Seng Chua, 2019. Meta-transfer learning for few-shot learning. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition. 403–412.
[29]
Flood Sung, Yongxin Yang, Li Zhang, 2018. Learning to compare: Relation network for few-shot learning. In Proceedings of the IEEE conference on computer vision and pattern recognition. 1199–1208.
[30]
Pinzhuo Tian, Wenbin Li, and Yang Gao. 2021. Consistent meta-regularization for better meta-knowledge in few-shot learning. IEEE Transactions on Neural Networks and Learning Systems 33, 12 (2021), 7277–7288.
[31]
Oriol Vinyals, Charles Blundell, Timothy Lillicrap, 2016. Matching networks for one shot learning. Advances in neural information processing systems 29 (2016).
[32]
Catherine Wah, Steve Branson, Peter Welinder, 2011. The caltech-ucsd birds-200-2011 dataset. (2011).
[33]
Chen Xing, Negar Rostamzadeh, Boris Oreshkin, 2019. Adaptive cross-modal few-shot learning. Advances in Neural Information Processing Systems 32 (2019).
[34]
Fengyuan Yang, Ruiping Wang, and Xilin Chen. 2022. SEGA: Semantic guided attention on visual prototype for few-shot learning. In Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision. 1056–1066.
[35]
Fengyuan Yang, Ruiping Wang, and Xilin Chen. 2023. Semantic Guided Latent Parts Embedding for Few-Shot Learning. In Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision. 5447–5457.
[36]
Chi Zhang, Yujun Cai, Guosheng Lin, 2020. Deepemd: Few-shot image classification with differentiable earth mover’s distance and structured classifiers. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition. 12203–12213.
Index Terms
- Semantic Filtering Global and Local Embeddings Fusion for Few-Shot Image Classification
Recommendations
Global and Local Feature Interaction with Vision Transformer for Few-shot Image Classification
CIKM '22: Proceedings of the 31st ACM International Conference on Information & Knowledge ManagementImage classification is a classical machine learning task and has been widely used. Due to the high costs of annotation and data collection in real scenarios, few-shot learning has become a vital technique to improve image classification performances. ...
Dual class representation learning for few-shot image classification
AbstractFew-shot learning (FSL) models are trained on base classes that have many training examples and evaluated on novel classes that have very few training examples. Since these models cannot be properly fine-tuned on the novel classes ...
Highlights- Proposes dual class representation learning (DCRL) for few-shot image classification.
Few-shot image classification with composite rotation based self-supervised auxiliary task
Highlights- Proposes composite rotation that is composed of inner and outer rotations of image.
AbstractMany real-life problem settings have classes of data with very few examples for training. Deep learning networks do not perform well for such few-shot classes. In order to perform well in this setting, the networks should learn to ...
Comments
Information & Contributors
Information
Published In
October 2023
589 pages
ISBN:9798400707988
DOI:10.1145/3633637
Copyright © 2023 ACM.
Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than the author(s) must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected].
Publisher
Association for Computing Machinery
New York, NY, United States
Publication History
Published: 28 February 2024
Check for updates
Author Tags
Qualifiers
- Research-article
- Research
- Refereed limited
Conference
ICCPR 2023
ICCPR 2023: 2023 12th International Conference on Computing and Pattern Recognition
October 27 - 29, 2023
Qingdao, China
Contributors
Other Metrics
Bibliometrics & Citations
Bibliometrics
Article Metrics
- 0Total Citations
- 24Total Downloads
- Downloads (Last 12 months)24
- Downloads (Last 6 weeks)5
Reflects downloads up to 25 Dec 2024
Other Metrics
Citations
View Options
Login options
Check if you have access through your login credentials or your institution to get full access on this article.
Sign inFull Access
View options
View or Download as a PDF file.
PDFeReader
View online with eReader.
eReaderHTML Format
View this article in HTML Format.
HTML Format