Multi-Scale Spectral-Spatial Attention Network for Hyperspectral Image Classification Combining 2D Octave and 3D Convolutional Neural Networks
Abstract
:1. Introduction
- Based on 2D octave, a multi-scale 2D octave convolution sub-network is proposed to capture spatial feature information. It can not only reduce the spatial feature information redundancy, but also extract complex spatial structure information adequately.
- A multi-scale DenseNet based on 3D CNNs is exploited to adequately explore the discriminative spectral signatures at various scales, while fusing the spectral signatures in both shallow and deep convolutional layers to enhance the reuse of spectral features.
- Two types of attention models are utilized to highlight the features containing important information to boost the spectral-spatial feature capture capability of the model. Furthermore, in order to address the problem of sample imbalance, a sample balancing strategy based on the WCL is employed to achieve the balance of the weight probabilities for each category, resulting in the fact that the model focuses more on categories with scarce training samples.
2. Materials and Methods
2.1. 2D Octave Convolution
2.2. 3D DenseNet
2.3. Attention Mechanism Module
2.3.1. Spectral Attention Mechanism
2.3.2. Channel Attention Mechanism
2.4. Balanced Sampling Strategy
3. Experimental Results and Discussion
3.1. Experimental Datasets Description
3.2. Parameter Setting
3.3. Experimental Setup
3.4. Classification Maps and Results
3.5. Discussion
3.5.1. Performance under Different Numbers of Training Samples
3.5.2. Ablation Experiment
4. Conclusions
Author Contributions
Funding
Conflicts of Interest
Abbreviations
CNN | convolutional neural network |
HSI | hyperspectral image |
MLR | multi-nomial logistic regression |
MRF | Markov random field |
DL | deep learning |
3Doc-conv | 3D octave convolution |
MOCNN | multi-scale spectral-spatial attention network |
framework combining 2D octave and 3D CNNs | |
multi-scale 3D DenseNet | multi-scale DenseNet based on 3D CNNs |
multi-scale 2D octave | multi-scale 2D octave convolution network |
LF | low frequency |
HF | high frequency |
PCA | principal component analysis |
BAM | band attention mechanism |
ECA | efficient channel attention mechanism |
WCL | weighted cross-entropy loss function |
AP | averaging pooling |
BN | batch normalization |
2Doc-conv | 2D octave convolution |
OA | overall accuracy |
AA | average accuracy |
Ka | Kappa cofficient |
UP | Pavia University dataset |
SV | Salinas Valley dataset |
IN | Indian Pines dataset |
ZY | Zaoyuan dataset |
SR | spatial resolution |
References
- He, L.; Li, J.; Liu, C.; Li, S. Recent Advances on Spectral–Spatial Hyperspectral Image Classification: An Overview and New Guidelines. IEEE Trans. Geosci. Remote Sens. 2018, 56, 1579–1597. [Google Scholar] [CrossRef]
- Li, S.; Song, W.; Fang, L.; Chen, Y.; Ghamisi, P.; Benediktsson, J.A. Deep Learning for Hyperspectral Image Classification: An Overview. IEEE Trans. Geosci. Remote Sens. 2019, 57, 6690–6709. [Google Scholar] [CrossRef] [Green Version]
- Thoreau, R.; Achard, V.; Risser, L.; Berthelot, B.; Briottet, X. Active Learning for Hyperspectral Image Classification: A comparative review. IEEE Trans. Geosci. Remote Sens. Mag. 2022, 10, 256–278. [Google Scholar] [CrossRef]
- Yang, X.; Yu, Y. Estimating Soil Salinity Under Various Moisture Conditions: An Experimental Study. IEEE Trans. Geosci. Remote Sens. 2017, 55, 2525–2533. [Google Scholar] [CrossRef]
- Liang, L.; Di, L.; Zhang, L.; Deng, M.; Qin, Z.; Zhao, S.; Lin, H. Estimation of crop LAI using hyperspectral vegetation indices and a hybrid inversion method. Remote Sens. Environ. 2015, 165, 123–134. [Google Scholar] [CrossRef]
- Makki, I.; Younes, R.; Francis, C.; Bianchi, T.; Zucchetti, M. A survey of landmine detection using hyperspectral imaging. ISPRS J. Photogramm. Remote Sens. 2017, 124, 40–53. [Google Scholar] [CrossRef]
- Tao, R.; Zhao, X.; Li, W.; Li, H.C.; Du, Q. Hyperspectral Anomaly Detection by Fractional Fourier Entropy. IEEE J. Sel. Topics Appl. Earth Observ. Remote Sens. 2019, 12, 4920–4929. [Google Scholar] [CrossRef]
- Melgani, F.; Bruzzone, L. Classification of hyperspectral remote sensing images with support vector machines. IEEE Trans. Geosci. Remote Sens. 2004, 42, 1778–1790. [Google Scholar] [CrossRef] [Green Version]
- Brodley, C.; Friedl, M. Decision tree classification of land cover from remotely sensed data. Remote Sens. Environ. 1997, 61, 399–409. [Google Scholar]
- Li, J.; Bioucas-Dias, J.M.; Plaza, A. Semisupervised Hyperspectral Image Segmentation Using Multinomial Logistic Regression with Active Learning. IEEE Trans. Geosci. Remote Sens. 2004, 48, 4085–4098. [Google Scholar] [CrossRef] [Green Version]
- Li, J.; Bioucas-Dias, J.M.; Plaza, A. Spectral–Spatial Hyperspectral Image Segmentation Using Subspace Multinomial Logistic Regression and Markov Random Fields. IEEE Trans. Geosci. Remote Sens. 2011, 50, 809–823. [Google Scholar] [CrossRef]
- Tariq, A.; Shu, H. CA-Markov Chain Analysis of Seasonal Land Surface Temperature and Land Use Land Cover Change Using Optical Multi-Temporal Satellite Data of Faisalabad, Pakistan. Remote Sens. 2020, 12, 3402. [Google Scholar] [CrossRef]
- Dimov, I.; Maire, S.; Todorov, V. An unbiased Monte Carlo method to solve linear Volterra equations of the second kind. Neural. Comput. Appl. 2022, 34, 1527–1540. [Google Scholar] [CrossRef]
- Shen, L.; Jia, S. Three-dimensional Gabor wavelets for pixel-based hyperspectral imagery classification. IEEE Trans. Geosci. Remote Sens. 2011, 49, 5039–5046. [Google Scholar] [CrossRef]
- Duan, P.; Kang, X.; Li, S.; Ghamisi, P.; Benediktsson, J.A. Fusion of Multiple Edge-Preserving Operations for Hyperspectral Image Classification. IEEE Trans. Geosci. Remote Sens. 2019, 57, 10336–10349. [Google Scholar] [CrossRef]
- Zhong, P.; Wang, R. Modeling and classifying hyperspectral imagery by CRFs with sparse higher order potentials. IEEE Trans. Geosci. Remote Sens. 2011, 49, 688–705. [Google Scholar] [CrossRef]
- Mura, M.D.; Villa, A.; Benediktsson, J.A.; Chanussot, J.; Bruzzone, L. Classification of hyperspectral images by using extended morphological attribute profiles and independent component analysis. IEEE Trans. Geosci. Remote Sens. Lett. 2011, 8, 542–546. [Google Scholar] [CrossRef] [Green Version]
- Fang, L.; Li, S.; Duan, W.; Ren, J.; Benediktsson, J.A. Classification of hyperspectral images by exploiting spectral–spatial information of superpixel via multiple kernels. IEEE Trans. Geosci. Remote Sens. 2015, 53, 6663–6674. [Google Scholar] [CrossRef] [Green Version]
- Ghamisi, P.; Souza, R.; Benediktsson, J.A.; Zhu, X.; Rittner, L.; Lotufo, R.A. Extinction profiles for the classification of remote sensing data. IEEE Trans. Geosci. Remote Sens. 2016, 54, 5631–5645. [Google Scholar] [CrossRef]
- Audebert, N.; Saux, B.L.; Lefevre, S. Deep Learning for Classification of Hyperspectral Data: A Comparative Review. IEEE Trans. Geosci. Remote Sens. Mag. 2019, 7, 159–173. [Google Scholar] [CrossRef] [Green Version]
- Chen, Y.; Lin, Z.; Zhao, X.; Wang, G.; Gu, Y. Deep learning-based classification of hyperspectral data. IEEE J. Sel. Topics Appl. Earth Observ. Remote Sens. 2014, 7, 2094–2107. [Google Scholar] [CrossRef]
- Mou, L.; Ghamisi, P.; Zhu, X.X. Deep recurrent neural networks for hyperspectral image classification. IEEE Trans. Geosci. Remote Sens. 2017, 55, 3639–3655. [Google Scholar] [CrossRef] [Green Version]
- Chen, Y.; Zhao, X.; Jia, X. Spectral–Spatial classification of hyperspectral data based on deep belief network. IEEE J. Sel. Topics Appl. Earth Observ. Remote Sens. 2015, 8, 2381–2392. [Google Scholar] [CrossRef]
- Mei, X.; Pan, E.; Ma, Y.; Dai, X.; Huang, J.; Fan, F.; Du, Q.; Zheng, H.; Ma, J. Spectral-Spatial Attention Networks for Hyperspectral Image Classification. Remote Sens. 2019, 11, 963. [Google Scholar] [CrossRef] [Green Version]
- Hang, R.; Qian, X.; Liu, Q. Cross-Modality Contrastive Learning for Hyperspectral Image Classification. IEEE Trans. Geosci. Remote Sens. 2022, 60, 5532812. [Google Scholar] [CrossRef]
- Chen, Y.; Jiang, H.; Li, C.; Jia, X.; Ghamisi, P. Deep feature extraction and classification of hyperspectral images based on convolutional neural networks. IEEE Trans. Geosci. Remote Sens. 2016, 54, 6232–6251. [Google Scholar] [CrossRef] [Green Version]
- Hang, R.; Li, Z.; Ghamisi, P.; Bhattacharyya, S.S. Hyperspectral Image Classification With Attention-Aided CNNs. IEEE Trans. Geosci. Remote Sens. 2021, 59, 2281–2293. [Google Scholar] [CrossRef]
- He, K.; Zhang, X.; Ren, S.; Sun, J. Identity mappings in deep residual networks. In Proceedings of the European Conference on Computer Vision, Amsterdam, The Netherlands, 11–16 October 2016; pp. 630–645. [Google Scholar]
- Huang, G.; Liu, Z.; Maaten, L.v.d.; Weinberger, K.Q. Densely connected convolutional networks. In Proceedings of the IEEE International Conference on Computer Vision and Pattern Recognition, Honolulu, HI, USA, 22–25 July 2017; pp. 4700–4708. [Google Scholar]
- Lee, H.; Kwon, H. Going Deeper With Contextual CNN for Hyperspectral Image Classification. IEEE Trans. Image Process. 2017, 26, 4843–4855. [Google Scholar] [CrossRef] [Green Version]
- Wang, W.; Dou, S.; Sun, L.; Jiang, Z. A Fast Dense Spectral–Spatial Convolution Network Framework for Hyperspectral Images Classification. Remote Sens. 2018, 10, 1068. [Google Scholar] [CrossRef] [Green Version]
- Li, Z.; Wang, T.; Li, W.; Du, Q.; Wang, C.; Liu, C.; Shi, X. Deep Multilayer Fusion Dense Network for Hyperspectral Image Classification. IEEE J. Sel. Topics Appl. Earth Observ. Remote Sens. 2020, 13, 1258–1270. [Google Scholar] [CrossRef]
- Liang, L.; Zhang, S.; Li, J. Multiscale DenseNet Meets With Bi-RNN for Hyperspectral Image Classification. IEEE J. Sel. Topics Appl. Earth Observ. Remote Sens. 2022, 15, 5401–5415. [Google Scholar] [CrossRef]
- Ma, W.; Yang, Q.; Wu, Y.; Zhao, W.; Zhang, X. Double-Branch Multi-Attention Mechanism Network for Hyperspectral Image Classification. Remote Sens. 2019, 11, 1307. [Google Scholar] [CrossRef] [Green Version]
- Li, R.; Zheng, S.; Duan, C.; Yang, Y.; Wang, X. Classification of hyperspectral image based on double-branch dual-attention mechanism network. Remote Sens. 2020, 12, 582. [Google Scholar] [CrossRef] [Green Version]
- Chen, Y.; Fan, H.; Xu, B.; Yan, Z.; Kalantidis, Y.; Rohrbach, M.; Yan, S.; Feng, J. Drop an octave: Reducing spatial redundancy in convolutional neural networks with octave convolution. In Proceedings of the IEEE/CVF International Conference on Computer Vision, Seoul, Republic of Korea, 27 October–2 November 2019; pp. 3435–3444. [Google Scholar]
- Tang, X.; Meng, F.; Zhang, X.; Cheung, Y.M.; Ma, J.; Liu, F.; Jiao, L. Hyperspectral Image Classification Based on 3-D Octave Convolution With Spatial–Spectral Attention Network. IEEE Trans. Geosci. Remote Sens. 2021, 59, 2430–2447. [Google Scholar] [CrossRef]
- Xu, Q.; Xiao, Y.; Luo, B.; Wang, D. CSA-MSO3DCNN: Multiscale octave 3D CNN with channel and spatial attention for hyperspectral image classification. Remote Sens. 2020, 12, 188. [Google Scholar] [CrossRef] [Green Version]
- Xu, Q.; Wang, D.; Luo, B. Faster Multiscale Capsule Network With Octave Convolution for Hyperspectral Image Classification. IEEE Geosci. Remote Sens. Lett. 2021, 18, 361–365. [Google Scholar] [CrossRef]
- Roy, S.K.; Krishna, G.; Dubey, S.R.; Chaudhuri, B.B. HybridSN: Exploring 3-D–2-D CNN Feature Hierarchy for Hyperspectral Image Classification. IEEE Geosci. Remote Sens. Lett. 2020, 17, 277–281. [Google Scholar] [CrossRef] [Green Version]
- Yu, D.; Li, Q.; Wang, X.; Xu, C.; Zhou, Y. A Cross-Level Spectral–Spatial Joint Encode Learning Framework for Imbalanced Hyperspectral Image Classification. IEEE Trans. Geosci. Remote Sens. 2022, 60, 5411717. [Google Scholar] [CrossRef]
- Zhu, Q.; Deng, W.; Zheng, Z.; Zhong, Y.; Guan, Q.; Lin, W.; Zhang, L.; Li, D. A Spectral-Spatial-Dependent Global Learning Framework for Insufficient and Imbalanced Hyperspectral Image Classification. IEEE Trans. Cybern. 2021, 52, 11709–11723. [Google Scholar] [CrossRef]
- Liang, L.; Zhang, S.; Li, J.; Cui, Z. Hyperspectral Image Classification Via Double-Branch Multi-Scale Spectral-Spatial Convolution Network. In Proceedings of the IIEEE International Geoscience and Remote Sensing Symposium, Kuala Lumpur, Malaysia, 17–22 July 2022; pp. 3588–3591. [Google Scholar]
- Dong, H.; Zhang, L.; Zou, B. Band Attention Convolutional Networks for Hyperspectral Image Classification. arXiv 2019, arXiv:1906.04379. [Google Scholar]
- Wang, Q.; Wu, B.; Zhu, P.; Li, P.; Zuo, W.; Hu, Q. ECA-Net: Efficient Channel Attention for Deep Convolutional Neural Networks. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Washington, DC, USA, 16–20 June 2020; pp. 11534–11542. [Google Scholar]
- Shang, R.; Ren, J.; Zhu, S.; Zhang, W.; Feng, J.; Li, Y.; Jiao, L. Hyperspectral Image Classification Based on Pyramid Coordinate Attention and Weighted Self-Distillation. IEEE Trans. Geosci. Remote Sens. 2022, 60, 5544316. [Google Scholar] [CrossRef]
- Tu, B.; He, W.; He, W.; Qu, X.; Plaza, A. Hyperspectral Classification via Global-Local Hierarchical Weighting Fusion Network. IEEE J. Sel. Topics Appl. Earth Observ. Remote Sens. 2022, 15, 184–200. [Google Scholar] [CrossRef]
- Alkhatib, M.Q.; Al-Saad, M.; Aburaed, N.; Almansoori, S.; Zabalza, J.; Marshall, S.; Al-Ahmad, H. Tri-CNN: A Three Branch Model for Hyperspectral Image Classification. Remote Sens. 2023, 15, 316. [Google Scholar] [CrossRef]
- Xue, Z.; Yu, X.; Liu, B.; Tan, X.; Wei, X. HResNetAM: Hierarchical Residual Network With Attention Mechanism for Hyperspectral Image Classification. IEEE J. Sel. Topics Appl. Earth Obs. Remote Sens. 2021, 14, 3566–3580. [Google Scholar] [CrossRef]
Category | Total | Training | Validation | Test |
---|---|---|---|---|
C1 | 6631 | 100 | 100 | 6431 |
C2 | 18,649 | 280 | 280 | 18,089 |
C3 | 2099 | 32 | 32 | 2035 |
C4 | 3064 | 46 | 46 | 2972 |
C5 | 1345 | 21 | 21 | 1303 |
C6 | 5029 | 76 | 76 | 4877 |
C7 | 1330 | 20 | 20 | 1290 |
C8 | 3682 | 56 | 56 | 3570 |
C9 | 947 | 15 | 15 | 917 |
Total | 42,776 | 646 | 646 | 41,484 |
Category | Total | Training | Validation | Test |
---|---|---|---|---|
C1 | 2009 | 21 | 21 | 1967 |
C2 | 3726 | 38 | 38 | 3650 |
C3 | 1976 | 20 | 20 | 1936 |
C4 | 1394 | 14 | 14 | 1366 |
C5 | 2678 | 27 | 27 | 2624 |
C6 | 3959 | 40 | 40 | 3879 |
C7 | 3579 | 36 | 36 | 3507 |
C8 | 11,271 | 113 | 113 | 11,045 |
C9 | 6203 | 63 | 63 | 6077 |
C10 | 3278 | 33 | 33 | 3212 |
C11 | 1068 | 11 | 11 | 1046 |
C12 | 1927 | 20 | 20 | 1887 |
C13 | 916 | 10 | 10 | 896 |
C14 | 1070 | 11 | 11 | 1048 |
C15 | 7268 | 73 | 73 | 7122 |
C16 | 1807 | 19 | 19 | 1769 |
Total | 54,129 | 549 | 549 | 53,031 |
Category | Total | Training | Validation | Test |
---|---|---|---|---|
C1 | 46 | 6 | 6 | 30 |
C2 | 1428 | 172 | 172 | 1084 |
C3 | 830 | 100 | 100 | 630 |
C4 | 237 | 29 | 29 | 179 |
C5 | 483 | 27 | 27 | 429 |
C6 | 730 | 58 | 58 | 614 |
C7 | 28 | 4 | 4 | 20 |
C8 | 478 | 58 | 58 | 362 |
C9 | 20 | 3 | 3 | 14 |
C10 | 972 | 117 | 117 | 738 |
C11 | 2455 | 295 | 295 | 1865 |
C12 | 593 | 72 | 72 | 449 |
C13 | 205 | 25 | 25 | 155 |
C14 | 1265 | 152 | 152 | 961 |
C15 | 386 | 47 | 47 | 292 |
C16 | 93 | 12 | 12 | 69 |
Total | 10,249 | 1238 | 1238 | 7773 |
Category | Total | Training | Validation | Test |
---|---|---|---|---|
C1 | 2625 | 53 | 53 | 2519 |
C2 | 1302 | 27 | 27 | 1248 |
C3 | 3442 | 69 | 69 | 3304 |
C4 | 10,243 | 205 | 205 | 9833 |
C5 | 1425 | 29 | 29 | 1367 |
C6 | 1484 | 30 | 30 | 1424 |
C7 | 1808 | 37 | 37 | 1734 |
C8 | 1492 | 30 | 30 | 1432 |
Total | 23,821 | 480 | 480 | 22,861 |
Model | Type/Layer | Filter/Operation | Configuration |
---|---|---|---|
2D octave | 2Doc-conv1 | (, ), 32 | stride:1, padding:1, BN+mish |
2Doc-conv2 | (, ), 64 | ||
2Doc-conv3 | (, ), 32 | ||
2Doc-conv4 | (, ), 16 | ||
3D DenseNet | conv1 | (1, 1, ), 32 | stride:2, padding:0, BN+mish |
conv2 | (1, 1, ), 16 | stride:1, padding:1, BN+mish | |
conv3 | (1, 1, ), 16 | ||
concat1 | concat(conv2, conv3), 32 | BN+mish | |
conv4 | (1, 1, ), 16 | stride:1, padding:1, BN+mish | |
concat2 | concat(conv2, conv3, conv4), 48 | BN+mish | |
conv5 | (1, 1, ), 16 | stride:1, padding:1, BN+mish | |
concat3 | concat(conv2, conv3, conv4, conv5), 64 | BN+mish | |
conv6 | (1, 1, ), 16 | stride:1, padding:0, BN+mish |
Class | Learning Rate | Dropout | Batch Size | Epoch |
---|---|---|---|---|
UP | 0.00005 | 0.5 | 32 | 250 |
SV | 0.0001 | 0.5 | 32 | 400 |
IN | 0.00005 | 0.5 | 32 | 400 |
ZY | 0.00005 | 0.5 | 64 | 600 |
Category | CDCNN | FDSSC | DBMA | DBDA | SSAN | TriCNN | 3DOC-CNN | HRAM | MOCNN |
---|---|---|---|---|---|---|---|---|---|
C1 | 92.19 | 99.35 | 97.67 | 98.90 | 92.80 | 89.86 | 95.20 | 98.75 | 99.21 |
C2 | 96.37 | 99.70 | 99.30 | 99.72 | 99.45 | 99.71 | 100.00 | 99.68 | 99.95 |
C3 | 72.28 | 97.03 | 93.10 | 95.97 | 69.73 | 82.95 | 89.19 | 92.87 | 97.25 |
C4 | 97.94 | 96.61 | 96.62 | 97.53 | 98.15 | 96.71 | 97.91 | 97.36 | 97.98 |
C5 | 99.19 | 99.70 | 99.38 | 99.51 | 97.01 | 99.87 | 94.55 | 99.29 | 95.32 |
C6 | 92.04 | 99.66 | 99.07 | 98.55 | 93.66 | 95.15 | 97.99 | 99.89 | 100.00 |
C7 | 90.47 | 99.95 | 99.14 | 99.27 | 85.43 | 80.53 | 82.56 | 99.26 | 96.20 |
C8 | 83.18 | 90.79 | 94.30 | 92.63 | 93.17 | 92.93 | 97.62 | 91.85 | 95.52 |
C9 | 99.08 | 98.15 | 97.03 | 97.59 | 90.73 | 98.79 | 98.80 | 97.72 | 100.00 |
OA | 92.86 ± 2.15 | 98.33 ± 0.93 | 98.00 ± 0.39 | 98.37 ± 0.49 | 94.94 ± 0.61 | 95.41 ± 0.86 | 97.39 ± 0.53 | 98.17 ± 0.81 | 98.92 ± 0.55 |
AA | 91.41 ± 3.40 | 97.88 ± 0.91 | 97.29 ± 0.53 | 97.74 ± 0.75 | 91.13 ± 1.35 | 92.95 ± 0.92 | 94.87 ± 0.47 | 97.41 ± 1.01 | 97.94 ±0.71 |
Ka | 90.48 ± 2.91 | 97.79 ± 1.23 | 97.35 ± 0.52 | 97.85 ± 0.65 | 93.27 ± 0.72 | 93.91 ± 0.95 | 96.54 ± 0.65 | 97.57 ± 1.33 | 98.57 ± 0.41 |
Category | CDCNN | FDSSC | DBMA | DBDA | SSAN | TriCNN | 3DOC-CNN | HRAM | MOCNN |
---|---|---|---|---|---|---|---|---|---|
C1 | 49.99 | 100.00 | 100.00 | 97.48 | 86.32 | 99.98 | 99.08 | 100.00 | 100.00 |
C2 | 79.97 | 100.00 | 99.92 | 99.96 | 99.53 | 99.92 | 99.78 | 98.72 | 100.00 |
C3 | 92.96 | 93.64 | 97.76 | 98.20 | 92.51 | 99.49 | 94.16 | 95.99 | 99.70 |
C4 | 93.95 | 97.55 | 94.23 | 97.13 | 99.78 | 99.50 | 97.66 | 96.78 | 98.66 |
C5 | 89.59 | 96.73 | 97.51 | 98.46 | 96.84 | 95.58 | 99.28 | 99.47 | 99.78 |
C6 | 98.18 | 99.81 | 98.77 | 99.88 | 100.00 | 100.00 | 97.68 | 99.99 | 98.70 |
C7 | 98.02 | 99.99 | 99.80 | 99.91 | 99.03 | 99.96 | 99.60 | 99.93 | 100.00 |
C8 | 80.45 | 96.25 | 94.66 | 96.99 | 90.88 | 95.81 | 95.70 | 93.49 | 99.83 |
C9 | 98.20 | 99.72 | 99.77 | 99.40 | 98.47 | 99.88 | 98.68 | 99.44 | 95.78 |
C10 | 87.17 | 99.40 | 96.47 | 98.31 | 99.66 | 97.61 | 92.28 | 98.60 | 99.67 |
C11 | 76.08 | 97.00 | 97.34 | 96.88 | 95.32 | 99.91 | 99.71 | 98.16 | 100.00 |
C12 | 91.28 | 99.88 | 99.71 | 99.70 | 89.51 | 96.23 | 97.14 | 99.98 | 98.28 |
C13 | 97.01 | 99.87 | 98.15 | 99.37 | 97.43 | 95.87 | 97.43 | 99.47 | 98.62 |
C14 | 97.35 | 98.13 | 98.40 | 98.46 | 89.60 | 95.85 | 90.36 | 97.64 | 99.89 |
C15 | 52.71 | 96.11 | 87.93 | 87.12 | 89.96 | 86.58 | 94.02 | 95.26 | 96.47 |
C16 | 99.02 | 100.00 | 98.13 | 99.96 | 96.10 | 99.12 | 97.06 | 99.93 | 99.35 |
OA | 83.77 ± 5.41 | 97.83 ± 1.10 | 95.91 ± 1.58 | 96.49 ± 2.10 | 94.67 ± 0.53 | 96.59 ± 0.75 | 96.69 ± 0.41 | 97.21 ± 1.12 | 98.69 ± 0.32 |
AA | 86.31 ± 6.21 | 98.38 ± 0.90 | 97.41 ± 0.95 | 97.95 ± 0.91 | 95.06 ± 0.47 | 97.58 ± 0.71 | 96.85 ± 0.55 | 98.30 ± 0.74 | 99.05 ± 0.55 |
Ka | 81.95 ± 2.20 | 97.58 ± 1.23 | 95.46 ± 2.05 | 96.10 ± 2.33 | 94.07 ± 0.65 | 96.20 ± 0.84 | 96.31 ± 0.72 | 96.88 ± 1.26 | 98.54 ± 0.71 |
Category | CDCNN | FDSSC | DBMA | DBDA | SSAN | TriCNN | 3DOC-CNN | HRAM | MOCNN |
---|---|---|---|---|---|---|---|---|---|
C1 | 48.27 | 96.07 | 97.35 | 100.00 | 91.18 | 100.00 | 100.00 | 74.24 | 97.06 |
C2 | 80.85 | 98.26 | 95.51 | 96.64 | 96.76 | 94.59 | 97.48 | 95.76 | 97.42 |
C3 | 78.45 | 96.97 | 99.13 | 99.10 | 92.72 | 98.35 | 97.99 | 98.24 | 98.57 |
C4 | 74.12 | 98.45 | 96.23 | 98.46 | 95.63 | 95.45 | 100.00 | 99.04 | 100.00 |
C5 | 96.44 | 97.99 | 97.64 | 98.77 | 92.53 | 97.81 | 94.13 | 97.03 | 95.10 |
C6 | 96.27 | 97.41 | 98.69 | 98.92 | 99.30 | 99.69 | 97.36 | 98.36 | 98.74 |
C7 | 44.91 | 95.89 | 81.70 | 82.30 | 100.00 | 99.20 | 75.00 | 70.70 | 95.00 |
C8 | 89.30 | 100.00 | 99.82 | 99.95 | 100.00 | 100.00 | 100.00 | 98.49 | 100.00 |
C9 | 42.50 | 95.63 | 95.82 | 97.20 | 100.00 | 63.33 | 100.00 | 60.00 | 100.00 |
C10 | 78.95 | 96.16 | 95.92 | 94.41 | 96.31 | 97.66 | 98.15 | 96.92 | 97.97 |
C11 | 78.02 | 99.03 | 97.10 | 99.18 | 99.16 | 98.27 | 97.86 | 99.15 | 98.98 |
C12 | 63.40 | 98.68 | 94.92 | 96.49 | 96.31 | 92.27 | 94.79 | 95.18 | 98.22 |
C13 | 98.45 | 98.69 | 99.79 | 98.78 | 94.97 | 99.56 | 99.37 | 100.00 | 100.00 |
C14 | 93.75 | 99.12 | 98.69 | 99.32 | 99.29 | 98.99 | 99.19 | 99.30 | 99.90 |
C15 | 86.86 | 98.13 | 95.94 | 97.90 | 94.33 | 94.99 | 94.67 | 98.69 | 100.00 |
C16 | 97.64 | 95.98 | 93.18 | 91.66 | 87.32 | 95.12 | 94.37 | 94.00 | 95.65 |
OA | 81.76 ± 3.71 | 98.16 ± 0.80 | 97.04 ± 1.08 | 97.88 ± 0.54 | 97.14 ± 0.52 | 97.36 ± 0.49 | 97.60 ± 0.32 | 97.79 ± 1.50 | 98.58 ± 0.41 |
AA | 78.01 ± 5.81 | 97.65 ± 1.04 | 96.09 ± 0.84 | 96.82 ± 0.49 | 95.99 ± 1.12 | 95.33 ± 1.53 | 96.27 ± 0.91 | 92.19 ± 3.35 | 98.29 ± 0.67 |
Ka | 79.08 ± 4.36 | 97.91 ± 1.01 | 96.62 ± 1.24 | 97.58 ± 1.07 | 96.74 ± 0.81 | 96.99 ± 0.55 | 97.27 ± 0.55 | 97.48 ± 1.01 | 98.39 ± 0.51 |
Category | CDCNN | FDSSC | DBMA | DBDA | SSAN | TriCNN | 3DOC-CNN | HRAM | MOCNN |
---|---|---|---|---|---|---|---|---|---|
1 | 80.19 | 97.34 | 96.86 | 96.88 | 99.88 | 95.28 | 96.10 | 95.67 | 97.66 |
2 | 85.53 | 97.63 | 96.64 | 97.03 | 90.46 | 97.75 | 99.28 | 97.58 | 97.04 |
3 | 81.97 | 98.10 | 97.06 | 98.09 | 97.31 | 95.52 | 92.05 | 98.01 | 99.49 |
4 | 89.22 | 98.38 | 98.52 | 98.46 | 96.68 | 97.66 | 98.66 | 98.47 | 97.67 |
5 | 55.62 | 95.81 | 92.46 | 96.10 | 94.15 | 90.49 | 91.21 | 97.41 | 99.93 |
6 | 88.00 | 96.37 | 95.75 | 95.13 | 94.03 | 96.27 | 91.20 | 97.09 | 98.03 |
7 | 94.75 | 96.27 | 98.87 | 98.49 | 99.60 | 97.36 | 98.44 | 97.62 | 97.98 |
8 | 61.69 | 94.29 | 94.31 | 94.54 | 86.59 | 92.19 | 95.10 | 95.33 | 98.95 |
OA | 84.14 ± 3.73 | 97.43 ± 0.83 | 96.96 ± 0.44 | 97.50 ± 0.55 | 96.06 ± 0.95 | 95.98 ± 0.66 | 96.31 ± 0.45 | 97.59 ± 0.36 | 98.16 ± 0.63 |
AA | 86.31 ± 6.21 | 96.77 ± 0.62 | 95.93 ± 0.68 | 96.84 ± 0.73 | 94.84 ± 1.02 | 95.31 ± 0.81 | 95.25 ± 0.41 | 97.14 ± 0.41 | 98.34 ± 0.62 |
Ka | 81.95 ± 2.20 | 96.63 ± 1.12 | 96.02 ± 0.57 | 96.72 ± 0.72 | 94.83 ± 1.80 | 94.74 ± 0.85 | 95.15 ± 0.77 | 96.84 ± 0.47 | 97.59 ± 1.04 |
Disclaimer/Publisher’s Note: The statements, opinions and data contained in all publications are solely those of the individual author(s) and contributor(s) and not of MDPI and/or the editor(s). MDPI and/or the editor(s) disclaim responsibility for any injury to people or property resulting from any ideas, methods, instructions or products referred to in the content. |
© 2023 by the authors. Licensee MDPI, Basel, Switzerland. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC BY) license (https://creativecommons.org/licenses/by/4.0/).
Share and Cite
Liang, L.; Zhang, S.; Li, J.; Plaza, A.; Cui, Z. Multi-Scale Spectral-Spatial Attention Network for Hyperspectral Image Classification Combining 2D Octave and 3D Convolutional Neural Networks. Remote Sens. 2023, 15, 1758. https://doi.org/10.3390/rs15071758
Liang L, Zhang S, Li J, Plaza A, Cui Z. Multi-Scale Spectral-Spatial Attention Network for Hyperspectral Image Classification Combining 2D Octave and 3D Convolutional Neural Networks. Remote Sensing. 2023; 15(7):1758. https://doi.org/10.3390/rs15071758
Chicago/Turabian StyleLiang, Lianhui, Shaoquan Zhang, Jun Li, Antonio Plaza, and Zhi Cui. 2023. "Multi-Scale Spectral-Spatial Attention Network for Hyperspectral Image Classification Combining 2D Octave and 3D Convolutional Neural Networks" Remote Sensing 15, no. 7: 1758. https://doi.org/10.3390/rs15071758