Location via proxy:   [ UP ]  
[Report a bug]   [Manage cookies]                
Skip to main content

Advertisement

Springer Nature Link
Log in

Image classification with consistency-regularized bad semi-supervised generative adversarial networks: a visual data analysis and synthesis

  • Research
  • Published:
The Visual Computer Aims and scope Submit manuscript

Abstract

Semi-supervised learning, which entails training a model with manually labeled images and pseudo-labels for unlabeled images, has garnered considerable attention for its potential to improve image classification performance. Nevertheless, incorrect decision boundaries of classifiers and wrong pseudo-labels for beneficial unlabeled images below the confidence threshold increase the generalization error in semi-supervised learning. This study proposes a novel framework for semi-supervised learning termed consistency-regularized bad generative adversarial network (CRBSGAN) through a new loss function. The proposed model comprises a discriminator, a bad generator, and a classifier that employs data augmentation and consistency regularization. Local augmentation is created to compensate for data scarcity and boost bad generators. Moreover, label consistency regularization is considered for bad fake images, real labeled images, unlabeled images, and latent space for the discriminator and bad generator. In the adversarial game between the discriminator and the bad generator, feature space is better captured under these conditions. Furthermore, local consistency regularization for good-augmented images applied to the classifier strengthens the bad generator in the generator–classifier adversarial game. The consistency-regularized bad generator produces informative fake images similar to the support vectors located near the correct classification boundary. In addition, the pseudo-label error is reduced for low-confidence unlabeled images used in training. The proposed method reduces the state-of-the-art error rate from 6.44 to 4.02 on CIFAR-10, 2.06 to 1.56 on MNIST, and 6.07 to 3.26 on SVHN using 4000, 3000, and 500 labeled training images, respectively. Furthermore, it achieves a reduction in the error rate on the CINIC-10 dataset from 19.38 to 15.32 and on the STL-10 dataset from 27 to 16.34 when utilizing 1000 and 500 labeled images per class, respectively. Experimental results and visual synthesis indicate that the CRBSGAN algorithm is more efficient than the methods proposed in previous works. The source code is available at https://github.com/ms-iraji/CRBSGAN ↗.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Subscribe and save

Springer+ Basic
$34.99 /Month
  • Get 10 units per month
  • Download Article/Chapter or eBook
  • 1 Unit = 1 Article or 1 Chapter
  • Cancel anytime
Subscribe now

Buy Now

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Algorithm 1
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11
Fig. 12
Fig. 13
Fig. 14
Fig. 15
Fig. 16
Fig. 17
Fig. 18
Fig. 19
Fig. 20
Fig. 21

Similar content being viewed by others

Explore related subjects

Discover the latest articles, news and stories from top researchers in related subjects.

Data availability

Data will be made available on request.

References

  1. Qin, Y., et al.: GuideRender: large-scale scene navigation based on multi-modal view frustum movement prediction. Vis. Comput. 39(8), 3597–3607 (2023)

    Google Scholar 

  2. Sheng, B., et al.: Accelerated robust Boolean operations based on hybrid representations. Comput. Aided Geom. Des. 62, 133–153 (2018)

    MathSciNet  Google Scholar 

  3. Jiang, J., et al.: Real-time hair simulation with heptadiagonal decomposition on mass spring system. Graph. Models 111, 101077 (2020)

    Google Scholar 

  4. Ertugrul, E., et al.: Embedding 3D models in offline physical environments. Comput. Anim. Virtual Worlds 31(4–5), e1959 (2020)

    Google Scholar 

  5. Huo, X., et al.: Attention regularized semi-supervised learning with class-ambiguous data for image classification. Pattern Recogn. 129, 108727 (2022)

    Google Scholar 

  6. Jian, C., Yang, K., Ao, Y.: Industrial fault diagnosis based on active learning and semi-supervised learning using small training set. Eng. Appl. Artif. Intell. 104, 104365 (2021)

    Google Scholar 

  7. Chang, J.-H., Weng, H.-C.: Fully used reliable data and attention consistency for semi-supervised learning. Knowl.-Based Syst. 249, 108837 (2022)

    Google Scholar 

  8. Ren, Q., et al.: A framework of active learning and semi-supervised learning for lithology identification based on improved naive Bayes. Expert Syst. Appl. 202, 117278 (2022)

    Google Scholar 

  9. Gu, X.: A self-training hierarchical prototype-based approach for semi-supervised classification. Inf. Sci. 535, 204–224 (2020)

    MathSciNet  Google Scholar 

  10. Lu, L., et al.: Uncertainty-aware pseudo-label and consistency for semi-supervised medical image segmentation. Biomed. Signal Process. Control 79, 104203 (2023)

    Google Scholar 

  11. Zhang, Y., et al.: Multi-view classification with semi-supervised learning for SAR target recognition. Signal Process. 183, 108030 (2021)

    Google Scholar 

  12. Emadi, M., et al.: A selection metric for semi-supervised learning based on neighborhood construction. Inf. Process. Manage. 58(2), 102444 (2021)

    Google Scholar 

  13. Wei, X., et al.: FMixCutMatch for semi-supervised deep learning. Neural Netw. 133, 166–176 (2021)

    Google Scholar 

  14. Zhang, B., et al.: Flexmatch: Boosting semi-supervised learning with curriculum pseudo labeling. Adv. Neural. Inf. Process. Syst. 34, 18408–18419 (2021)

    Google Scholar 

  15. Arantes, R.B., Vogiatzis, G., Faria, D.R.: Learning an augmentation strategy for sparse datasets. Image Vis. Comput. 117, 104338 (2022)

    Google Scholar 

  16. Xiu, Y., et al.: FreMix: Frequency-based mixup for data augmentation. Wirel. Commun. Mob. Comput. 2022 (2022)

  17. Gan, Y., et al.: Deep semi-supervised learning with contrastive learning and partial label propagation for image data. Knowl.-Based Syst. 245, 108602 (2022)

    Google Scholar 

  18. Miyato, T., et al.: Virtual adversarial training: a regularization method for supervised and semi-supervised learning. IEEE Trans. Pattern Anal. Mach. Intell. 41(8), 1979–1993 (2018)

    Google Scholar 

  19. Gangwar, A., et al.: Triple-BigGAN: Semi-supervised generative adversarial networks for image synthesis and classification on sexual facial expression recognition. Neurocomputing 528, 200–216 (2023)

    Google Scholar 

  20. He, R., et al.: Generative adversarial network-based semi-supervised learning for real-time risk warning of process industries. Expert Syst. Appl. 150, 113244 (2020)

    Google Scholar 

  21. Liu, Y., et al.: Regularizing discriminative capability of CGANs for semi-supervised generative learning. In: Proceedings of the IEEE/CVF conference on computer vision and pattern recognition (2020)

  22. Li, Y., et al.: The theoretical research of generative adversarial networks: an overview. Neurocomputing 435, 26–41 (2021)

    Google Scholar 

  23. Dai, Z., et al.: Good semi-supervised learning that requires a bad gan. Adv, Neural Inf. Process. Syst. 30 (2017)

  24. Yun, S., et al.: Cutmix: Regularization strategy to train strong classifiers with localizable features. In: Proceedings of the IEEE/CVF international conference on computer vision. (2019)

  25. Goodfellow, I., et al.: Generative adversarial networks. Commun. ACM 63(11), 139–144 (2020)

    MathSciNet  Google Scholar 

  26. Wang, R., et al.: Better pseudo-label: Joint domain-aware label and dual-classifier for semi-supervised domain generalization. Pattern Recogn. 133, 108987 (2023)

    Google Scholar 

  27. Kim, D., et al.: Multi-co-training for document classification using various document representations: TF–IDF, LDA, and Doc2Vec. Inf. Sci. 477, 15–29 (2019)

    Google Scholar 

  28. Yu, K., et al.: A consistency regularization based semi-supervised learning approach for intelligent fault diagnosis of rolling bearing. Measurement 165, 107987 (2020)

    Google Scholar 

  29. Liu, L., Tan, R.T.: Certainty driven consistency loss on multi-teacher networks for semi-supervised learning. Pattern Recogn. 120, 108140 (2021)

    Google Scholar 

  30. Ke, Z., et al.: Dual student: Breaking the limits of the teacher in semi-supervised learning. In: Proceedings of the IEEE/CVF International Conference on Computer Vision. (2019)

  31. Deng, W., et al.: Deep ladder reconstruction-classification network for unsupervised domain adaptation. Pattern Recogn. Lett. 152, 398–405 (2021)

    Google Scholar 

  32. Xiao, H., et al.: Semi-supervised semantic segmentation with cross teacher training. Neurocomputing 508, 36–46 (2022)

    Google Scholar 

  33. Li, B., Pi, D., Lin, Y.: Learning ladder neural networks for semi-supervised node classification in social network. Expert Syst. Appl. 165, 113957 (2021)

    Google Scholar 

  34. Chen, J., Yang, M., Ling, J.: Attention-based label consistency for semi-supervised deep learning based image classification. Neurocomputing 453, 731–741 (2021)

    Google Scholar 

  35. Meel, P., Vishwakarma, D.K.: A temporal ensembling based semi-supervised ConvNet for the detection of fake news articles. Expert Syst. Appl. 177, 115002 (2021)

    Google Scholar 

  36. Ding, W., Abdel-Basset, M., Hawash, H.: RCTE: A reliable and consistent temporal-ensembling framework for semi-supervised segmentation of COVID-19 lesions. Inf. Sci. 578, 559–573 (2021)

    MathSciNet  Google Scholar 

  37. Wang, J., et al.: Adversarial attacks and defenses in deep learning for image recognition: A survey. Neurocomputing 514, 162–181 (2022)

    Google Scholar 

  38. Berthelot, D., et al.: Remixmatch: Semi-supervised learning with distribution alignment and augmentation anchoring. Int. Conf. Learn. Represent. (ICLR), (2020)

  39. Sohn, K., et al.: Fixmatch: Simplifying semi-supervised learning with consistency and confidence. Adv. Neural. Inf. Process. Syst. 33, 596–608 (2020)

    Google Scholar 

  40. Yang, M., et al.: Discriminative semi-supervised learning via deep and dictionary representation for image classification. Pattern Recogn. 140, 109521 (2023)

    Google Scholar 

  41. Xu, H., et al.: Semi-supervised learning with pseudo-negative labels for image classification. Knowl.-Based Syst. 260, 110166 (2023)

    Google Scholar 

  42. Li, X., et al.: Feature-aware conditional GAN for category text generation. Neurocomputing 547, 126352 (2023)

    Google Scholar 

  43. Rubin, M., et al.: TOP-GAN: Stain-free cancer cell classification using deep learning with a small training set. Med. Image Anal. 57, 176–185 (2019)

    Google Scholar 

  44. Mao, J., et al.: Pseudo-labeling generative adversarial networks for medical image classification. Comput. Biol. Med. 147, 105729 (2022)

    Google Scholar 

  45. Chen, Z., Ramachandra, B., Vatsavai, R.R.: Consistency regularization with generative adversarial networks for semi-supervised learning (2020). arXiv preprint arXiv:2007.03844

  46. Verma, V., et al.: Interpolation consistency training for semi-supervised learning. Neural Netw. 145, 90–106 (2022)

    Google Scholar 

  47. Zhao, Z. et al.: Improved consistency regularization for gans. In: Proceedings of the AAAI Conference on Artificial Intelligence (2021)

  48. Li, C. et al.: Triple generative adversarial nets. Adv. Neural Inf. Process. Syst. 30 (2017)

  49. Gan, Y. et al.: Generative adversarial networks with adaptive learning strategy for noise-to-image synthesis. Neural Comput. Appl. 35(8), 6197–6206 (2022)

    Google Scholar 

  50. Wang, L., Sun, Y., Wang, Z.: CCS-GAN: A semi-supervised generative adversarial network for image classification. Vis. Comput. 38(6), 2009–2021 (2022)

    Google Scholar 

  51. Dong, J., Lin, T.: MarginGAN: Adversarial training in semi-supervised learning. Adv. Neural Inf. Process. Syst. 32 (2019)

  52. Gu, X., Angelov, P.P.: Semi-supervised deep rule-based approach for image classification. Appl. Soft Comput. 68, 53–68 (2018)

    Google Scholar 

  53. Zhang, H. et al.: Consistency regularization for generative adversarial networks. Proc. Int. Conf. Learn. Represent. (2020)

  54. Yang, M., et al.: Deep neural networks with L1 and L2 regularization for high dimensional corporate credit risk prediction. Expert Syst. Appl. 213, 118873 (2023)

    Google Scholar 

  55. Yang, S. et al.: Sample efficiency of data augmentation consistency regularization. In: International Conference on Artificial Intelligence and Statistics. PMLR (2023)

  56. Feng, W., et al.: New margin-based subsampling iterative technique in modified random forests for classification. Knowl.-Based Syst. 182, 104845 (2019)

    Google Scholar 

  57. Lee, D.-H.: Pseudo-label: The simple and efficient semi-supervised learning method for deep neural networks. In: Workshop on challenges in representation learning, ICML. (2013)

  58. Liu, Z., et al.: Dual-feature-embeddings-based semi-supervised learning for cognitive engagement classification in online course discussions. Knowl.-Based Syst. 259, 110053 (2023)

    Google Scholar 

  59. Li, W., et al.: Tackling mode collapse in multi-generator GANs with orthogonal vectors. Pattern Recogn. 110, 107646 (2021)

    Google Scholar 

  60. LeCun, Y., et al.: Gradient-based learning applied to document recognition. Proc. IEEE 86(11), 2278–2324 (1998)

    Google Scholar 

  61. Netzer, Y. et al.: Reading digits in natural images with unsupervised feature learning. In: NIPS workshop on deep learning and unsupervised feature learning. 2011, Granada, Spain.

  62. Darlow, L.N. et al.: Cinic-10 is not imagenet or cifar-10 (2018). arXiv preprint arXiv:1810.03505

  63. Krizhevsky, A., Hinton, G.: Learning multiple layers of features from tiny images (2009)

  64. Coates, A., Ng, A., Lee, H.: An analysis of single-layer networks in unsupervised feature learning. In: Proceedings of the fourteenth international conference on artificial intelligence and statistics. JMLR Workshop and Conference Proceedings. (2011)

  65. Qiu, S., et al.: Adversarial attack and defense technologies in natural language processing: A survey. Neurocomputing 492, 278–307 (2022)

    Google Scholar 

  66. Zoppi, T., Ceccarelli, A.: Detect adversarial attacks against deep neural networks with GPU monitoring. IEEE Access 9, 150579–150591 (2021)

    Google Scholar 

  67. Bao, J. et al.: CVAE-GAN: fine-grained image generation through asymmetric training. In: Proceedings of the IEEE international conference on computer vision. (2017)

  68. Wu, Y.-H. et al.: P2T: Pyramid pooling transformer for scene understanding. IEEE Trans. Pattern Anal. Mach. Intell. (2022)

  69. Jiang, Y., Chang, S., Wang, Z.: Transgan: Two pure transformers can make one strong gan, and that can scale up. Adv. Neural. Inf. Process. Syst. 34, 14745–14758 (2021)

    Google Scholar 

  70. Weston, J., Ratle, F., Collobert, R.: Deep learning via semi-supervised embedding. In: Proceedings of the 25th international conference on Machine learning. (2008)

  71. Salakhutdinov, R., Hinton, G.: Learning a nonlinear embedding by preserving class neighbourhood structure. In: Artificial Intelligence and Statistics. PMLR (2007)

  72. Ranzato, M.A. et al.: Unsupervised learning of invariant feature hierarchies with applications to object recognition. In: 2007 IEEE conference on computer vision and pattern recognition. IEEE (2007)

  73. Rifai, S. et al.: The manifold tangent classifier. Adv. Neural Inf. Process. Syst. 24 (2011)

  74. Li, C., et al.: Triple generative adversarial networks. IEEE Trans. Pattern Anal. Mach. Intell. 44(12), 9629–9640 (2021)

    Google Scholar 

  75. Mayer, C., Paul, M., Timofte, R.: Adversarial feature distribution alignment for semi-supervised learning. Comput. Vis. Image Underst. 202, 103109 (2021)

    Google Scholar 

  76. Rasmus, A. et al.: Semi-supervised learning with ladder networks. Adv. Neural Inf. Process. Syst. 28 (2015)

  77. Springenberg, J.T.: Unsupervised and semi-supervised learning with categorical generative adversarial networks. Proceedings of International Conference on Learning Representations (ICLR), (2016)

  78. Salimans, T. et al.: Improved techniques for training gans. Adv. Neural Inf. Process. Syst. 29 (2016)

  79. Deng, Z. et al.: Structured generative adversarial networks. Adv. Neural Inf. Process. Syst. 30 (2017)

  80. Tarvainen, A., Valpola, H.: Mean teachers are better role models: Weight-averaged consistency targets improve semi-supervised deep learning results. Adv. Neural Inf. Process. Syst. 30 (2017)

  81. Wu, Y.-H., et al.: MobileSal: Extremely efficient RGB-D salient object detection. IEEE Trans. Pattern Anal. Mach. Intell. 44(12), 10261–10269 (2021)

    Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Department of Computer Engineering, Faculty of Electrical and Computer Engineering, University of Tabriz, Tabriz, Iran

    Mohammad Saber Iraji, Jafar Tanha, Mohammad-Ali Balafar & Mohammad-Reza Feizi-Derakhshi

Authors
  1. Mohammad Saber Iraji
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Jafar Tanha
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Mohammad-Ali Balafar
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Mohammad-Reza Feizi-Derakhshi
    View author publications

    You can also search for this author in PubMed Google Scholar

Contributions

Iraji and Tanha proposed the Consistency-Regularized Bad Semi-Supervised Generative Adversarial Networks approach. Iraji executed the approach and analyzed the results. Iraji, Tanha, Balafar, and Feizi-Derakhshi were responsible for the manuscript's conceptualization, validation, resources, and editing. All authors read and authorized the final manuscript.

Corresponding author

Correspondence to Jafar Tanha.

Ethics declarations

Competing interest

The authors declare that they have no known competing financial interests or personal relationships that could have appeared to influence the work reported in this paper.

Ethical and informed consent

This article does not contain any studies with human participants or animals performed by any of the authors. The datasets used in the manuscript are derived from publicly available data sets and may be obtained from the appropriate authors upon reasonable request.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Iraji, M.S., Tanha, J., Balafar, MA. et al. Image classification with consistency-regularized bad semi-supervised generative adversarial networks: a visual data analysis and synthesis. Vis Comput 40, 6843–6865 (2024). https://doi.org/10.1007/s00371-024-03360-z

Download citation

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s00371-024-03360-z

Keywords

Search

Navigation