Abstract
Hyperspectral Imaging (HSI) has revolutionized earth observation through advanced remote sensing technology, providing rich spectral and spatial information across multiple bands. However, this wealth of data introduces significant challenges for classification, including high spectral correlation, the curse of dimensionality due to limited labeled data, the need to model long-term dependencies, and the impact of sample input on deep learning performance. These challenges are further exacerbated by the costly and complex acquisition of HSI data, resulting in limited availability of labeled samples and class imbalances. To address these critical issues, our study proposes a novel approach for generating high-quality synthetic hyperspectral data cubes using an advanced Generative Adversarial Network (GAN) integrated with the Wasserstein loss and gradient penalty phenomenon (WGAN-GP). This approach aims to augment real-world data, mitigating the scarcity of labeled samples that has long been a bottleneck in hyperspectral image analysis and classification. To fully leverage both the synthetic and real data, we introduce a novel Convolutional LSTM classifier designed to process the intricate spatial and spectral correlations inherent in hyperspectral data. This classifier excels in modeling multi-dimensional relationships within the data, effectively capturing long-term dependencies and improving feature extraction and classification accuracy. The performance of our proposed model, termed 3D-ACWGAN-ConvLSTM, is rigorously validated using benchmark hyperspectral datasets, demonstrating its effectiveness in augmenting real-world data and enhancing classification performance. This research contributes to addressing the critical need for robust data augmentation techniques in hyperspectral imaging, potentially opening new avenues for applications in areas constrained by limited data availability and complex spectral-spatial relationships.
This is a preview of subscription content, log in via an institution to check access.
Similar content being viewed by others
Explore related subjects
Discover the latest articles, news and stories from top researchers in related subjects.Data Availability
The data was collected from https://www.ehu.eus/ccwintco/index.php/Hyperspectral_Remote_Sensing_Scenes.
References
Arjovsky M, Chintala S, Bottou L (2017) Wasserstein generative adversarial networks. In: Int Conf Mach Learn, pp 214–223. PMLR
Audebert N, Le Saux B, Lefèvre S (2018) Generative adversarial networks for realistic synthesis of hyperspectral samples. In: IGARSS 2018-2018 IEEE Int Geosci Remote Sens Symp, pp 4359–4362. IEEE
Bera S, Shrivastava VK (2020) Analysis of various optimizers on deep convolutional neural network model in the application of hyperspectral remote sensing image classification. Int J Remote Sens 41(7):2664–2683
Chen Y, Jiang H, Li C, Jia X, Ghamisi P (2016) Deep feature extraction and classification of hyperspectral images based on convolutional neural networks. IEEE Trans Geosci Remote Sens 54(10):6232–6251
Chen L, Wei Y, Yao Z, Chen E, Zhang X (2022) Data augmentation in prototypical networks for forest tree species classification using airborne hyperspectral images. IEEE Trans Geosci Remote Sens 60:1–16
Dhok S, Bhurane A, Kothari A (2019) Automated hyperspectral image classification using spatial-spectral features. In: 2019 6th Int Conf Signal Process Integr Netw (SPIN), pp 184–189 (2019). IEEE
Du B, Zhang L, Tao D, Zhang D (2013) Unsupervised transfer learning for target detection from hyperspectral images. Neurocomputing 120:72–82
Farooque G, Xiao L, Yang J, Sargano AB (2021) Hyperspectral image classification via a novel spectral-spatial 3d convlstm-cnn. Remote Sensing. 13(21):4348
Feng J, Feng X, Chen J, Cao X, Zhang X, Jiao L, Yu T (2020) Generative adversarial networks based on collaborative learning and attention mechanism for hyperspectral image classification. Remote Sensing. 12(7):1149
Goodfellow I, Pouget-Abadie J, Mirza M, Xu B, Warde-Farley D, Ozair S, Courville A, Bengio Y (2014) Generative adversarial nets. Adv Neural Inform Process Syst 27
Goodfellow I, Pouget-Abadie J, Mirza M, Xu B, Warde-Farley D, Ozair S, Courville A, Bengio Y (2020) Generative adversarial networks. Commun ACM 63(11):139–144
Hang R, Zhou F, Liu Q, Ghamisi P (2020) Classification of hyperspectral images via multitask generative adversarial networks. IEEE Trans Geosci Remote Sens 59(2):1424–1436
Haut JM, Paoletti ME, Plaza J, Plaza A, Li J (2019) Hyperspectral image classification using random occlusion data augmentation. IEEE Geosci Remote Sens Lett 16(11):1751–1755
Hu WS, Li HC, Pan L, Li W, Tao R, Du Q (2020) Spatial-spectral feature extraction via deep convlstm neural networks for hyperspectral image classification. IEEE Trans Geosci Remote Sens 58(6):4237–4250
Hu W, Huang Y, Wei L, Zhang F, Li H (2015) Deep convolutional neural networks for hyperspectral image classification. Journal of Sensors 2015(1):258619
Jia S, Jiang S, Lin Z, Li N, Xu M, Yu S (2021) A survey: deep learning for hyperspectral image classification with few labeled samples. Neurocomputing 448:179–204
Koumoutsou D, Siolas G, Charou E, Stamou G (2022) Generative adversarial networks for data augmentation in hyperspectral image classification. In: Generative adversarial learning: architectures and applications, pp 115–144. Springer, ???
Liao C, Dong M (2022) Acwgan: an auxiliary classifier wasserstein gan-based oversampling approach for multi-class imbalanced learning. Int J Innov Comput Inf Control 18(3):703–721
Mirza M, Osindero S (2014) Conditional generative adversarial nets. arXiv preprint arXiv:1411.1784
Odena A, Olah C, Shlens J (2017) Conditional image synthesis with auxiliary classifier gans. In: Int Conf Mach Learn, pp 2642–2651. PMLR
Pande S, Banerjee B, Pižurica A (2019) Class reconstruction driven adversarial domain adaptation for hyperspectral image classification. In: Iberian conference on pattern recognition and image analysis, pp 472–484. Springer
Paul A, Bhoumik S (2022) Classification of hyperspectral imagery using spectrally partitioned hyperunet. Neural Comput Appl 34(3):2073–2082
Paul A, Bhoumik S, Chaki N (2021) Ssnet: an improved deep hybrid network for hyperspectral image classification. Neural Comput Appl 33:1575–1585
Radford A, Metz L, Chintala S (2015) Unsupervised representation learning with deep convolutional generative adversarial networks. arXiv preprint arXiv:1511.06434
Ranjan P, Kumar R, Girdhar A: A 3d-convolutional-autoencoder embedded siamese-attention-network for classification of hyperspectral images. Neural Computing and Applications, 1–20 (2024)
Ranjan P, Girdhar A (2022) A comprehensive systematic review of deep learning methods for hyperspectral images classification. Int J Remote Sens 43(17):6221–6306
Ranjan P, Girdhar A (2022) Xcep-dense: a novel lightweight extreme inception model for hyperspectral image classification. Int J Remote Sens 43(14):5204–5230
Ranjan P, Girdhar A (2024) Deep siamese network with handcrafted feature extraction for hyperspectral image classification. Multimedia Tools and Applications. 83(1):2501–2526
Ranjan P, Gupta G (2023) A cross-domain semi-supervised zero-shot learning model for the classification of hyperspectral images. Journal of the Indian Society of Remote Sensing. 51(10):1991–2005
Roy SK, Haut JM, Paoletti ME, Dubey SR, Plaza A (2021) Generative adversarial minority oversampling for spectral-spatial hyperspectral image classification. IEEE Trans Geosci Remote Sens 60:1–15
Seydgar M, Alizadeh Naeini A, Zhang M, Li W, Satari M (2019) 3-d convolution-recurrent networks for spectral-spatial classification of hyperspectral images. Remote Sensing. 11(7):883
Shi X, Chen Z, Wang H, Yeung DY, Wong WK, Woo Wc (2015) Convolutional lstm network: A machine learning approach for precipitation nowcasting. Adv Neural Inform Process Syst 28
Shoemake K (1985) Animating rotation with quaternion curves. In: Proceedings of the 12th annual conference on computer graphics and interactive techniques, pp 245–254
Song A, Choi J, Han Y, Kim Y (2018) Change detection in hyperspectral images using recurrent 3d fully convolutional networks. Remote Sensing. 10(11):1827
Srivastava A, Valkov L, Russell C, Gutmann MU, Sutton C (2017) Veegan: reducing mode collapse in gans using implicit variational learning. Adv Neural Inform Process Syst 30 (2017)
Sun Q, Bourennane S (2020) Hyperspectral image classification with unsupervised feature extraction. Remote Sensing Letters. 11(5):475–484
Wambugu N, Chen Y, Xiao Z, Tan K, Wei M, Liu X, Li J (2021) Hyperspectral image classification on insufficient-sample and feature learning using deep neural networks: a review. Int J Appl Earth Observ Geoinform 105:102603
Wang WY, Li HC, Deng YJ, Shao LY, Lu XQ, Du Q (2020) Generative adversarial capsule network with convlstm for hyperspectral image classification. IEEE Geosci Remote Sens Lett 18(3):523–527
Wang J, Gao F, Dong J, Du Q (2020) Adaptive dropblock-enhanced generative adversarial networks for hyperspectral image classification. IEEE Trans Geosci Remote Sens 59(6):5040–5053
Wang J, Guo S, Huang R, Li L, Zhang X, Jiao L (2021) Dual-channel capsule generation adversarial network for hyperspectral image classification. IEEE Trans Geosci Remote Sens 60:1–16
Wang X, Tan K, Du Q, Chen Y, Du P (2019) Caps-triplegan: Gan-assisted capsnet for hyperspectral image classification. IEEE Trans Geosci Remote Sens 57(9):7232–7245
Wu H, Prasad S (2017) Convolutional recurrent neural networks for hyperspectral data classification. Remote Sensing. 9(3):298
Xia X, Pan X, Li N, He X, Ma L, Zhang X, Ding N (2022) Gan-based anomaly detection: a review. Neurocomputing 493:497–535
Xue Z (2020) A general generative adversarial capsule network for hyperspectral image spectral-spatial classification. Remote Sens Lett 11(1):19–28
Zhang F, Bai J, Zhang J, Xiao Z, Pei C (2020) An optimized training method for gan-based hyperspectral image classification. IEEE Geosci Remote Sens Lett 18(10):1791–1795
Zhou F, Hang R, Liu Q, Yuan X (2019) Hyperspectral image classification using spectral-spatial lstms. Neurocomputing 328:39–47
Zhu JY, Park T, Isola P, Efros AA (2017) Unpaired image-to-image translation using cycle-consistent adversarial networks. In: Proceedings of the IEEE Int Conf Comput Vis, pp 2223–2232
Zhu L, Chen Y, Ghamisi P, Benediktsson JA (2018) Generative adversarial networks for hyperspectral image classification. IEEE Trans Geosci Remote Sens 56(9):5046–5063
Acknowledgements
Thanks to the Data Processing Teams who have shared data on the website https://www.ehu.eus/ccwintco/index.php/Hyperspectral_Remote_Sensing_Scenes.
Funding
No funding was received for this study.
Author information
Authors and Affiliations
Corresponding author
Ethics declarations
Conflict of Interest/Competing Interests
No conflict of interest. The authors declare that they have no known competing financial interests or personal relationships that could have appeared to influence the work reported in this paper.
Additional information
Communicated by: Hassan Babaie.
Publisher's Note
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Rights and permissions
Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.
About this article
Cite this article
Ranjan, P., Girdhar, A., Ankur et al. A novel spectral-spatial 3D auxiliary conditional GAN integrated convolutional LSTM for hyperspectral image classification. Earth Sci Inform 17, 5251–5271 (2024). https://doi.org/10.1007/s12145-024-01451-y
Received:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s12145-024-01451-y