A trainable spectral-spatial sparse coding model for hyperspectral image restoration
Article No.: 415, Pages 5430 - 5442
Abstract
Hyperspectral imaging offers new perspectives for diverse applications, ranging from the monitoring of the environment using airborne or satellite remote sensing, precision farming, food safety, planetary exploration, or astrophysics. Unfortunately, the spectral diversity of information comes at the expense of various sources of degradation, and the lack of accurate ground-truth "clean" hyperspectral signals acquired on the spot makes restoration tasks challenging. In particular, training deep neural networks for restoration is difficult, in contrast to traditional RGB imaging problems where deep models tend to shine. In this paper, we advocate instead for a hybrid approach based on sparse coding principles that retains the in-terpretability of classical techniques encoding domain knowledge with handcrafted image priors, while allowing to train model parameters end-to-end without massive amounts of data. We show on various denoising benchmarks that our method is computationally efficient and significantly outperforms the state of the art.
Supplementary Material
Supplemental material.
- Download
- 3.26 MB
References
[1]
T. Adão, J. Hruška, L. Pádua, J. Bessa, E. Peres, R. Morais, and J. J. Sousa. Hyperspectral imaging: A review on UAV-Based sensors, data processing and applications for agriculture and forestry. Remote Sensing, 9(11), 2017.
[2]
H. Akbari, L. Halig, D. M. Schuster, B. Fei, A. Osunkoya, V. Master, P. Nieh, and G. Chen. Hyperspectral imaging and quantitative analysis for prostate cancer detection. Journal of Biomedical Optics, 17(7):1–11, 2012.
[3]
L. Alparone, L. Wald, J. Chanussot, C. Thomas, P. Gamba, and L. M. Bruce. Comparison of pansharpening algorithms: Outcome of the 2006 grs-s data-fusion contest. IEEE Transactions on Geoscience and Remote Sensing, 45(10):3012–3021, 2007.
[4]
B. Arad and O. Ben-Shahar. Sparse recovery of hyperspectral signal from natural RGB images. In Proc. European Conference on Computer Vision (ECCV), 2016.
[5]
N. Audebert, B. Saux, and S. Lefèvre. Deep learning for classification of hyperspectral data: A comparative review. IEEE Geoscience and Remote Sensing Magazine, 7(2):159–173, 2019.
[6]
A. G. Baydin, B. A. Pearlmutter, A. A. Radul, and J. M. Siskind. Automatic differentiation in machine learning: a survey. Journal of Machine Learning Research (JMLR), 18:1–43, 2018.
[7]
J. M. Bioucas-Dias and J. M. P. Nascimento. Hyperspectral subspace identification. IEEE Transactions on Geoscience and Remote Sensing, 46(8):2435–2445, 2008.
[8]
J. M. Bioucas-Dias, A. Plaza, G. Camps-Valls, P. Scheunders, N. Nasrabadi, and J. Chanussot. Hyperspec-tral remote sensing data analysis and future challenges. IEEE Geoscience and Remote Sensing Magazine, 1(2):6–36, 2013.
[9]
A. Buades, B. Coll, and J.-M. Morel. A non-local algorithm for image denoising. In Proc. Conference on Computer Vision and Pattern Recognition (CVPR), 2005.
[10]
A. Chakrabarti and T. Zickler. Statistics of real-world hyperspectral images. In Proc. Conference on Computer Vision and Pattern Recognition (CVPR), 2011.
[11]
Y. Chang, L. Yan, and S. Zhong. Hyper-laplacian regularized unidirectional low-rank tensor recovery for multispectral image denoising. In Proc. Conference on Computer Vision and Pattern Recognition (CVPR), 2017.
[12]
K. Dabov, A. Foi, V. Katkovnik, and K. Egiazarian. Image Denoising by Sparse 3-D Transform-Domain Collaborative Filtering. IEEE Transactions on Image Processing, 16(8):2080–2095, 2007.
[13]
C. F. Dantas, J. E. Cohen, and R. Gribonval. Hyperspectral image denoising using dictionary learning. In 2019 10th Workshop on Hyperspectral Imaging and Signal Processing: Evolution in Remote Sensing (WHISPERS), 2019.
[14]
Q. Du, N. H. Younan, R. King, and V. P. Shah. On the performance evaluation of pan-sharpening techniques. IEEE Geoscience and Remote Sensing Letters, 4(4):518–522, 2007.
[15]
M. Elad and M. Aharon. Image denoising via sparse and redundant representations over learned dictionaries. IEEE Transactions on Image Processing, 15(12):3736–3745, 2006.
[16]
G. Elmasry, M. Kamruzzaman, D.-W. Sun, and Paul Allen. Principles and applications of hyperspectral imaging in quality evaluation of agro-food products: A review. Critical Reviews in Food Science and Nutrition, 52(11):999–1023, 2012.
[17]
H. Fan, C. Li, Y. Guo, G. Kuang, and J. Ma. Spatial–Spectral total variation regularized low-rank tensor decomposition for hyperspectral image denoising. IEEE Transactions on Geoscience and Remote Sensing, 56(10):6196–6213, 2018.
[18]
B. Fei. Chapter 3.6 - Hyperspectral imaging in medical applications. In J. M. Amigo, editor, Hyperspectral Imaging, volume 32 of Data Handling in Science and Technology, pages 523–565. Elsevier, 2020.
[19]
Y.-Z. Feng and Da-Wen Sun. Application of hyperspectral imaging in food safety inspection and control: A review. Critical Reviews in Food Science and Nutrition, 52(11):1039–1058, 2012.
[20]
M. Figueiredo and R. Nowak. An EM algorithm for wavelet-based image restoration. IEEE Transactions on Image Processing, 12(8):906–916, 2003.
[21]
Y. Fu, A. Lam, I. Sato, and Y. Sato. Adaptive spatial-spectral dictionary learning for hyperspectral image denoising. In Proc. International Conference on Computer Vision (ICCV), 2015.
[22]
A. F. Goetz. Three decades of hyperspectral remote sensing of the Earth: A personal view. Remote Sensing of Environment, 113:S5–S16, 2009.
[23]
X. Gong, W. Chen, and J. Chen. A low-rank tensor dictionary learning method for hyperspectral image denoising. IEEE Transactions on Signal Processing, 68:1168–1180, 2020.
[24]
C. A. G. Gonzalez, O. Absil, and M. van Droogenbroeck. Supervised detection of exoplanets in high-contrast imaging sequences. Astronomy & Astrophysics, 613:A71, May 2018.
[25]
A. A. Gowen, Y. Feng, E. Gaston, and V. Valdramidis. Recent applications of hyperspectral imaging in microbiology. Talanta, 137:43–54, 2015.
[26]
K. Gregor and Y. LeCun. Learning fast approximations of sparse coding. In Proc. International Conference on Machine Learning (ICML), 2010.
[27]
W. He, Q. Yao, C. Li, N. Yokoya, and Q. Zhao. Non-local meets global: An integrated paradigm for hyperspectral denoising. In Proc. Conference on Computer Vision and Pattern Recognition (CVPR), 2019.
[28]
K. I. Itten, F. Dell'Endice, A. Hueni, M. Kneubühler, D. Schläpfer, D. Odermatt, F. Seidel, S. Huber, J. Schopfer, T. Kellenberger, et al. Apex-the hyperspectral esa airborne prism experiment. Sensors, 8(10):6235–6259, 2008.
[29]
J. Kerekes and J. Baum. Hyperspectral imaging system modeling. Lincoln Laboratory Journal, 14(1):117–130, 2003.
[30]
N. Keshava and J. F. Mustard. Spectral unmixing. IEEE signal processing magazine, 19(1):44–57, 2002.
[31]
Z. Kong, X. Yang, and L. He. A comprehensive comparison of multi-dimensional image denoising methods, 2020.
[32]
S. Laine, T. Karras, J. Lehtinen, and T. Aila. High-quality self-supervised deep image denoising. In Adv. Neural Information Processing Systems (NeurIPS), 2019.
[33]
B. Lecouat, J. Ponce, and J. Mairal. Designing and learning trainable priors with non-cooperative games. In Adv. Neural Information Processing Systems (NeurIPS), 2020.
[34]
B. Lecouat, J. Ponce, and J. Mairal. Fully trainable and interpretable non-local sparse models for image restoration. In Proc. European Conference on Computer Vision (ECCV), 2020.
[35]
Y. Liu, H. Pu, and D.-W. Sun. Hyperspectral imaging technique for evaluating food quality and safety during various processes: A review of recent applications. Trends in Food Science & Technology, 69:25–35, 2017.
[36]
B. Lu, P. D. Dao, J. Liu, Y. He, and J. Shang. Recent advances of hyperspectral imaging technology and applications in agriculture. Remote Sensing, 12(16), 2020.
[37]
G. Lu and B. Fei. Medical hyperspectral imaging: A review. Journal of Biomedical Optics, 19(1):1–24, 2014.
[38]
A. Maffei, J. M. Haut, M. E. Paoletti, J. Plaza, L. Bruzzone, and A. Plaza. A single model CNN for hyperspectral image denoising. IEEE Transactions on Geoscience and Remote Sensing, 58(4):2516–2529, 2020.
[39]
M. Maggioni, V. Katkovnik, K. Egiazarian, and A. Foi. Nonlocal Transform-Domain Filter for Volumetric Data Denoising and Reconstruction. IEEE Transactions on Image Processing, 22(1):119–133, 2013.
[40]
S. Mahesh, D. Jayas, J. Paliwal, and N. White. Hyperspectral imaging to classify and monitor quality of agricultural materials. Journal of Stored Products Research, 61:17–26, 2015.
[41]
J. Mairal, F. Bach, and J. Ponce. Task-driven dictionary learning. IEEE Transactions on Pattern Analysis and Machine Intelligence (PAMI), 34(4):791–804, 2012.
[42]
J. Mairal, F. Bach, J. Ponce, et al. Sparse modeling for image and vision processing. Foundations and Trends in Computer Graphics and Vision, 8(2-3):85–283, 2014.
[43]
J. Mairal, M. Elad, and G. Sapiro. Sparse representation for color image restoration. IEEE Transactions on image processing, 17(1):53–69, 2007.
[44]
Y. Mäkinen, L. Azzari, and A. Foi. Exact transform-domain noise variance for collaborative filtering of stationary correlated noise. In IEEE International Conference on Image Processing (ICIP), 2019.
[45]
Y. Mäkinen, L. Azzari, and A. Foi. Collaborative filtering of correlated noise: Exact transform-domain variance for improved shrinkage and patch matching. IEEE Transactions on Image Processing, 29:8339–8354, 2020.
[46]
D. G. Manolakis, R. B. Lockwood, and T. W. Cooley. Hyperspectral Imaging Remote Sensing: Physics, Sensors, and Algorithms. Cambridge University Press, Cambridge, 2016.
[47]
B. A. Olshausen and D. J. Field. Emergence of simple-cell receptive field properties by learning a sparse code for natural images. Nature, 381(6583):607–609, 1996.
[48]
H. Othman and S.-E. Qian. Noise reduction of hyperspectral imagery using hybrid spatial-spectral derivative-domain wavelet shrinkage. IEEE Transactions on Geoscience and Remote Sensing, 44(2):397–408, 2006.
[49]
Y. Peng, D. Meng, Z. Xu, C. Gao, Y. Yang, and B. Zhang. Decomposable nonlocal tensor dictionary learning for multispectral image denoising. In Proc. Conference on Computer Vision and Pattern Recognition (CVPR), 2014.
[50]
N. Qi, Y. Shi, X. Sun, and B. Yin. TenSR: Multi-dimensional tensor sparse representation. In IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2016.
[51]
B. Rasti, P. Scheunders, P. Ghamisi, G. Licciardi, and J. Chanussot. Noise Reduction in Hyperspectral Imagery: Overview and Application. Remote Sensing, 10(3):482, Mar. 2018.
[52]
B. Rasti, J. R. Sveinsson, M. O. Ulfarsson, and J. A. Benediktsson. Hyperspectral image denoising using first order spectral roughness penalty in wavelet domain. IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing, 7(6):2458–2467, 2014.
[53]
B. Rasti, M. O. Ulfarsson, and P. Ghamisi. Automatic hyperspectral image restoration using sparse and low-rank modeling. IEEE Geoscience and Remote Sensing Letters, 14(12):2335–2339, 2017.
[54]
Q. Shi, X. Tang, T. Yang, R. Liu, and L. Zhang. Hyperspectral image denoising using a 3-D attention denoising network. IEEE Transactions on Geoscience and Remote Sensing, 2021.
[55]
D. Simon and M. Elad. Rethinking the CSC model for natural images. In Adv. Neural Information Processing Systems (NeurIPS), 2019.
[56]
V. Studer, J. Bobin, M. Chahid, H. S. Mousavi, E. Candes, and M. Dahan. Compressive fluorescence microscopy for biological and hyperspectral imaging. Proceedings of the National Academy of Sciences, 109(26):E1679–E1687, 2012.
[57]
L. Tucker. Some mathematical notes on three-mode factor analysis. Psychometrika, 31(3):279–311, 1966.
[58]
G. R. L. (U.S.). Hypercube sample data set: Hydice sensor imagery urban.
[59]
M. Wang, Q. Wang, J. Chanussot, and D. Hong. L0-l1 hybrid total variation regularization and its applications on hyperspectral image mixed noise removal and compressed sensing. IEEE Transactions on Geoscience and Remote Sensing, 2021.
[60]
M. Wang, Q. Wang, J. Chanussot, and D. Li. Hyperspectral image mixed noise removal based on multidirectional low-rank modeling and spatial-spectral total variation. IEEE Transactions on Geoscience and Remote Sensing, 59(1):488–507, 2021.
[61]
Z. Wang, A. Bovik, H. Sheikh, and E. Simoncelli. Image quality assessment: from error visibility to structural similarity. IEEE Transactions on Image Processing, 13(4):600–612, 2004.
[62]
K. Wei, Y. Fu, and H. Huang. 3-D quasi-recurrent neural network for hyperspectral image denoising. IEEE Transactions on Neural Networks and Learning Systems, 2020.
[63]
F. Xiong, J. Zhou, J. Lu, and Y. Qian. SMDS-Net: Model Guided Spectral-Spatial Network for Hyperspec-tral Image Denoising. arXiv:2012.01829, 2020.
[64]
F. Yasuma, T. Mitsunaga, D. Iso, and S. Nayar. Generalized Assorted Pixel Camera: Post-Capture Control of Resolution, Dynamic Range and Spectrum. Technical report, Nov 2008.
[65]
Q. Yuan, L. Zhang, and H. Shen. Hyperspectral image denoising with a Spatial–Spectral view fusion strategy. IEEE Transactions on Geoscience and Remote Sensing, 52(5):2314–2325, 2014.
[66]
Q. Yuan, Q. Zhang, J. Li, H. Shen, and L. Zhang. Hyperspectral Image Denoising Employing a Spatial–Spectral Deep Residual Convolutional Neural Network. IEEE Transactions on Geoscience and Remote Sensing, 57(2):1205–1218, Feb. 2019.
[67]
H. Zhang, W. He, L. Zhang, H. Shen, and Q. Yuan. Hyperspectral image restoration using low-rank matrix recovery. IEEE Transactions on Geoscience and Remote Sensing, 52(8):4729–4743, 2014.
[68]
K. Zhang, W. Zuo, Y. Chen, D. Meng, and L. Zhang. Beyond a Gaussian denoiser: Residual learning of deep CNN for image denoising. IEEE Transactions on Image Processing, 26(7):3142–3155, 2017.
[69]
L. Zhang, L. Zhang, X. Mou, and D. Zhang. Fsim: A feature similarity index for image quality assessment. IEEE Transactions on Image Processing, 20(8):2378–2386, 2011.
[70]
Y.-Q. Zhao and J. Yang. Hyperspectral image denoising via sparse representation and low-rank constraint. IEEE Transactions on Geoscience and Remote Sensing, 53(1):296–308, 2015.
[71]
L. Zhuang and J. M. Bioucas-Dias. Hyperspectral image denoising based on global and non-local low-rank factorizations. In IEEE International Conference on Image Processing (ICIP), 2017.
Index Terms
- A trainable spectral-spatial sparse coding model for hyperspectral image restoration
Index terms have been assigned to the content through auto-classification.
Recommendations
Adaptive Spatial-Spectral Dictionary Learning for Hyperspectral Image Restoration
Hyperspectral imaging is beneficial in a diverse range of applications from diagnostic medicine, to agriculture, to surveillance to name a few. However, hyperspectral images often suffer from degradation such as noise and low resolution. In this paper, ...
Spatial-Spectral Deep Residual Network for Hyperspectral Image Super-Resolution
AbstractRecently, single hyperspectral image super-resolution (SR) methods based on deep learning have been extensively studied. However, there has been limited technical development focusing on single hyperspectral image super-resolution due to the high-...
Hierarchical spatio-spectral fusion for hyperspectral image super resolution via sparse representation and pre-trained deep model
AbstractFusing a hyperspectral image (HSI) with a high resolution multispectral image (MSI) has been a highly attractive and effective approach for improving the spatial resolution of HSIs. However, most existing spatio-spectral fusion methods ...
Comments
Information & Contributors
Information
Published In
December 2021
30517 pages
ISBN:9781713845393
Copyright © 2021 Neural Information Processing Systems Foundation, Inc.
Publisher
Curran Associates Inc.
Red Hook, NY, United States
Publication History
Published: 06 December 2021
Qualifiers
- Research-article
- Research
- Refereed limited
Contributors
Other Metrics
Bibliometrics & Citations
Bibliometrics
Article Metrics
- 0Total Citations
- 0Total Downloads
- Downloads (Last 12 months)0
- Downloads (Last 6 weeks)0
Reflects downloads up to 21 Jan 2025