research-article

Multi-Channel Deep Networks for Block-Based Image Compressive Sensing

Authors:

Jianming ZhangAuthors Info & Claims

IEEE Transactions on Multimedia, Volume 23

Pages 2627 - 2640

https://doi.org/10.1109/TMM.2020.3014561

Published: 01 January 2021 Publication History

Abstract

Incorporating deep neural networks in image compressive sensing (CS) receives intensive attentions in multimedia technology and applications recently. As deep network approaches learn the inverse mapping directly from the CS measurements, the reconstruction speed is significantly faster than the conventional CS algorithms. However, for existing network-based approaches, a CS sampling procedure has to map a separate network model. This may potentially degrade the performance of image CS with block-wise sampling because of blocking artifacts, especially when multiple sampling rates are assigned to different blocks within an image. In this paper, we develop a multi-channel deep network for block-based image CS by exploiting inter-block correlation with performance significantly exceeding the current state-of-the-art methods. The significant performance improvement is attributed to block-wise approximation but full-image removal of blocking artifacts. Specifically, with our multi-channel structure, the image blocks with a variety of sampling rates can be reconstructed in a single model. The initially reconstructed blocks are then capable of being reassembled into a full image to improve the recovered images by unrolling a hand-designed block-based CS recovery algorithm. Experimental results demonstrate that the proposed method outperforms the state-of-the-art CS methods by a large margin in terms of objective metrics and subjective visual image quality. Our source codes are available at <uri>https://github.com/siwangzhou/DeepBCS</uri>.

References

[1]

D. Donoho, “Compressed sensing,”IEEE Trans. Inform. Theory, vol. 52, no. 4, pp. 1289–1306, Apr.2006.

Digital Library

[2]

M. Elad, “Optimized projections for compressed sensing,”IEEE Trans. Signal Process., vol. 55, no. 12, pp. 5695–5702, Dec.2007.

Digital Library

[3]

Q. Jiang, S. Li, Z. Zhu, H. Bai, X. He, and R. C. de Lamare, “Design of compressed sensing system with probability-based prior information,”IEEE Trans. Multimedia, vol. 22, no. 3, pp. 594–609, Mar.2020.

[4]

M. F. Duarteet al., “Single-pixel imaging via compressive sampling,”IEEE Signal Process. Mag., vol. 25, no. 2, pp. 83–91, Mar.2008.

[5]

M. Lustig, D. L. Donoho, J. M. Santos, and J. M. Pauly, “Compressed sensing MRI,”IEEE Signal Process. Mag., vol. 25, no. 2, pp. 72–82, Mar.2008.

[6]

Z. Chen, X. Hou, X. Qian, and C. Gong, “Efficient and robust image coding and transmission based on scrambled block compressive sensing,”IEEE Trans. Multimedia, vol. 20, no. 7, pp. 1610–1621, Jul.2017.

[7]

S. Zheng, X.-P. Zhang, J. Chen, and Y. Kuo, “A high-efficiency compressed sensing based terminal-to-cloud video transmission system,”IEEE Trans. Multimedia, vol. 21, no. 8, pp. 1905–1920, Aug.2019.

[8]

C. Li, Y. Zhang, and E. Y. Xie, “When an attacker meets a cipher-image in 2018: A Year in Review,”J. Inform. Secur. Appl., vol. 48, 2019, Art. no. .

[9]

L. Gan, “Block compressed sensing of natural images,” in Proc. Int. Conf. Digital Signal Process., 2007, pp. 403–406.

[10]

J. Bigot, C. Boyer, and P. Weiss, “An analysis of block sampling strategies in compressed sensing,”IEEE Trans. Inf. Theory, vol. 62, no. 4, pp. 2125–2139, Apr.2016.

Digital Library

[11]

J. Fowler, Sungkwang Mun, and E. Tramel, “Block-based compressed sensing of images and video,”Foundations Trends Signal Process., vol. 4, no. 4, pp. 297–416, 2010.

Digital Library

[12]

Y. Yu, B. Wang, and L. Zhang, “Saliency-based compressive sampling for image signals,”IEEE Signal Process. Lett., vol. 17, no. 11, pp. 973–976, Nov.2010.

[13]

C. Zhao, J. Zhang, S. Ma, X. Fan, Y. Zhang, and W. Gao, “Reducing image compression artifacts by structural sparse representation and quantization constraint prior,”IEEE Trans. Circuits Syst. for Video Technol., vol. 27, no. 10, pp. 2057–2071, Oct.2017.

Digital Library

[14]

A. Krizhevsky, I. Sutskever, and G. E. Hinton, “Imagenet classification with deep convolutional neural networks,”Commn. ACM, vol. 60, no. 6, pp. 84–90, 2017.

Digital Library

[15]

L. Galteri, L. Seidenari, M. Bertini, and A. Del Bimbo, “Deep universal generative adversarial compression artifact removal,”IEEE Trans. Multimedia, vol. 21, no. 8, pp. 2131–2145, Aug.2019.

[16]

K. Kulkarni, S. Lohit, P. Turaga, R. Kerviche, and A. Ashok, “Reconnet: Non-iterative reconstruction of images from compressively sensed measurements,” in Proc. IEEE Conf. Comput. Vision Pattern Recognit., 2016, pp. 449–458.

[17]

C. Metzler, A. Mousavi, and R. Baraniuk, “Learned D-AMP: Principled neural network based compressive image recovery,” in Proc. 31st Int. Conf. Neural Inform. Process. Syst., 2017, pp. 1770–1781.

[18]

S. Lohit, K. Kulkarni, R. Kerviche, P. Turaga, and A. Ashok, “Convolutional neural networks for non-iterative reconstruction of compressively sensed images,”IEEE Trans. Comput. Imag., vol. 4, no. 3, pp. 326–340, Sep.2018.

[19]

J. Zhang and B. Ghanem, “Ista-net: Interpretable optimization-inspired deep network for image compressive sensing,” in Proc. IEEE Conf. Comput. Vision Pattern Recognit., 2018, pp. 1828–1837.

[20]

K. Zhang, W. Zuo, Y. Chen, D. Meng, and L. Zhang, “Beyond a gaussian denoiser: Residual learning of deep CNN for image denoising,”IEEE Trans. Image Process., vol. 26, no. 7, pp. 3142–3155, Jul.2017.

Digital Library

[21]

S. S. Chen, D. L. Donoho, and M. A. Saunders, “Atomic decomposition by basis pursuit,”SIAM Rev., vol. 43, no. 1, pp. 129–159, 2001.

Digital Library

[22]

M. Yang and F. de Hoog, “Orthogonal matching pursuit with thresholding and its application in compressive sensing,”IEEE Trans. Signal Process., vol. 63, no. 20, pp. 5479–5486, Oct.2015.

Digital Library

[23]

J. Zhang, D. Zhao, and W. Gao, “Group-based sparse representation for image restoration,”IEEE Trans. Image Process., vol. 23, no. 8, pp. 3336–3351, Aug.2014.

[24]

W. Dong, G. Shi, X. Li, Y. Ma, and F. Huang, “Compressive sensing via nonlocal low-rank regularization,”IEEE Trans. Image Process., vol. 23, no. 8, pp. 3618–3632, Aug.2014.

[25]

C. A. Metzler, A. Maleki, and R. G. Baraniuk, “From denoising to compressed sensing,”IEEE Trans. Inf. Theory, vol. 62, no. 9, pp. 5117–5144, Sep.2016.

Digital Library

[26]

Z. Zhu, G. Li, J. Ding, Q. Li, and X. He, “On collaborative compressive sensing systems: The framework, design, and algorithm,”SIAM J. Imag. Sci., vol. 11, no. 2, pp. 1717–1758, 2018.

[27]

S. Zheng, J. Chen, and Y. Kuo, “A multi-level residual reconstruction based image compressed sensing recovery scheme,”Multimedia Tools Appl., vol. 78, pp. 25 101–25 119, 2019.

[28]

J. Chen, Y. Gao, C. Ma, and Y. Kuo, “Compressive sensing image reconstruction based on multiple regulation constraints,”Circuits Syst. Signal Process., vol. 36, no. 4, pp. 1621–1638, 2017.

Digital Library

[29]

S. Mun and J. E. Fowler, “Block compressed sensing of images using directional transforms,” in Proc. Int. Conf. Image Process., 2009, pp. 2985–2988.

[30]

A. Akbari, D. Mandache, M. Trocan, and B. Granado, “Adaptive saliency-based compressive sensing image reconstruction,” in Proc. IEEE Int. Conf. Multimedia Expo Workshops, 2016, pp. 1–6.

[31]

R. G. Baraniuk, T. Goldstein, A. C. Sankaranarayanan, C. Studer, A. Veeraraghavan, and M. B. Wakin, “Compressive video sensing: Algorithms, architectures, and applications,”IEEE Signal Process. Mag., vol. 34, no. 1, pp. 52–66, Jan.2017.

[32]

S. Zhou, S. Xiang, X. Liu, and H. Li, “Asymmetric block based compressive sensing for image signals,” in Proc. IEEE Int. Conf. Multimedia Expo, 2018, pp. 1–6.

[33]

X. Xie, C. Wang, J. Du, and G. Shi, “Full image recover for block-based compressive sensing,” in Proc. IEEE Int. Conf. Multimedia Expo, 2018, pp. 1–6.

[34]

W. Shi, F. Jiang, S. Zhang, and D. Zhao, “Deep networks for compressed image sensing,” in Proc. IEEE Int. Conf. Multimedia Expo, 2017, pp. 877–882.

[35]

B. Lim, S. Son, H. Kim, S. Nah, and K. M. Lee, “Enhanced deep residual networks for single image super-resolution,” in Proc. IEEE Conf. Comput. Vision Pattern Recognit. Workshops, 2017, pp. 1132–1140.

[36]

W. Shi, F. Jiang, S. Liu, and D. Zhao, “Multi-scale deep networks for image compressed sensing,” in Proc. IEEE Int. Conf. Image Process., 2018, pp. 46–50.

[37]

Y. Cheng, D. Wang, P. Zhou, and T. Zhang, “Model compression and acceleration for deep neural networks: The principles, progress, and challenges,”IEEE Signal Process. Mag., vol. 35, no. 1, pp. 126–136, Jan.2018.

[38]

K. Dabov, A. Foi, V. Katkovnik, and K. Egiazarian, “Image denoising by sparse 3-D transform-domain collaborative filtering,”IEEE Trans. Image Process., vol. 16, no. 8, pp. 2080–2095, Aug.2007.

[39]

M. Abadiet al., “Tensorflow: A system for large-scale machine learning,” in Proc. 12th USENIX Symp. Operating Syst. Design Implementation, 2016, pp. 265–283.

[40]

P. Arbelaez, M. Maire, C. Fowlkes, and J. Malik, “Contour detection and hierarchical image segmentation,”IEEE Trans. Pattern Anal. Mach. Intell., vol. 33, no. 5, pp. 898–916, May 2011.

Digital Library

[41]

J. Justin, A. Alexandre, and F. Li, “Perceptual losses for real-time style transfer and super-resolution,” in Proc. Eur. Conf. Comput. Vision, 2016, pp. 694–711.

Cited By

Ullah FUllah IKhan KKhan SAmin F(2025)Advances in deep neural network-based hyperspectral image classification and feature learning with limited samples: a surveyApplied Intelligence10.1007/s10489-024-06139-w55:6Online publication date: 1-Apr-2025
https://dl.acm.org/doi/10.1007/s10489-024-06139-w
Zhang JWang ZYi YKuang LZhang J(2025)PSFE-YOLO: a traffic sign detection algorithm with pixel-wise spatial feature enhancementPattern Analysis & Applications10.1007/s10044-024-01406-528:1Online publication date: 1-Mar-2025
https://dl.acm.org/doi/10.1007/s10044-024-01406-5
Song HGong JMeng HLai YWooldridge MDy JNatarajan S(2024)Multi-cross sampling and frequency-division reconstruction for image compressed sensingProceedings of the Thirty-Eighth AAAI Conference on Artificial Intelligence and Thirty-Sixth Conference on Innovative Applications of Artificial Intelligence and Fourteenth Symposium on Educational Advances in Artificial Intelligence10.1609/aaai.v38i5.28294(4909-4917)Online publication date: 20-Feb-2024
https://dl.acm.org/doi/10.1609/aaai.v38i5.28294
Show More Cited By

Recommendations

Block compressive sensing

Iterative re-weighted l1 norm minimization reconstruction algorithm.Joint reconstruction algorithm for multiview images and video frames.The correlation between multiple images is exploited through joint sparsity model. Compressive sensing provides ...
Image compressive sensing via Truncated Schatten-p Norm regularization

Low-rank property as a useful image prior has attracted much attention in image processing communities. Recently, a nonlocal low-rank regularization (NLR) approach toward exploiting low-rank property has shown the state-of-the-art performance in ...
Image Block Compressive Sensing Reconstruction via Group-Based Sparse Representation and Nonlocal Total Variation

Compressive sensing (CS) has recently drawn considerable attentions in signal and image processing communities as a joint sampling and compression approach. Generally, the image CS reconstruction can be formulated as an optimization problem with a ...

Comments

Information & Contributors

Information

Published In

cover image IEEE Transactions on Multimedia

IEEE Transactions on Multimedia Volume 23, Issue

2021

1967 pages

ISSN:1520-9210

Issue’s Table of Contents

1520-9210 © 2020 IEEE. Personal use is permitted, but republication/redistribution requires IEEE permission. See https://www.ieee.org/publications/rights/index.html for more information.

Publisher

IEEE Press

Publication History

Published: 01 January 2021

Qualifiers

Research-article

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

37
Total Citations
View Citations
0
Total Downloads

Downloads (Last 12 months)0
Downloads (Last 6 weeks)0

Reflects downloads up to 08 Feb 2025

Other Metrics

View Author Metrics

Citations

Cited By

Ullah FUllah IKhan KKhan SAmin F(2025)Advances in deep neural network-based hyperspectral image classification and feature learning with limited samples: a surveyApplied Intelligence10.1007/s10489-024-06139-w55:6Online publication date: 1-Apr-2025
https://dl.acm.org/doi/10.1007/s10489-024-06139-w
Zhang JWang ZYi YKuang LZhang J(2025)PSFE-YOLO: a traffic sign detection algorithm with pixel-wise spatial feature enhancementPattern Analysis & Applications10.1007/s10044-024-01406-528:1Online publication date: 1-Mar-2025
https://dl.acm.org/doi/10.1007/s10044-024-01406-5
Song HGong JMeng HLai YWooldridge MDy JNatarajan S(2024)Multi-cross sampling and frequency-division reconstruction for image compressed sensingProceedings of the Thirty-Eighth AAAI Conference on Artificial Intelligence and Thirty-Sixth Conference on Innovative Applications of Artificial Intelligence and Fourteenth Symposium on Educational Advances in Artificial Intelligence10.1609/aaai.v38i5.28294(4909-4917)Online publication date: 20-Feb-2024
https://dl.acm.org/doi/10.1609/aaai.v38i5.28294
Han KWang JShi YCai HLing NYin B(2024)WTDUN: Wavelet Tree-Structured Sampling and Deep Unfolding Network for Image Compressed SensingACM Transactions on Multimedia Computing, Communications, and Applications10.1145/370173121:1(1-22)Online publication date: 26-Oct-2024
https://dl.acm.org/doi/10.1145/3701731
Han KWang JShi YLing NYin BCai JKankanhalli MPrabhakaran BBoll SSubramanian RZheng LSingh VCesar PXie LXu D(2024)D3U-Net: Dual-Domain Collaborative Optimization Deep Unfolding Network for Image Compressive SensingProceedings of the 32nd ACM International Conference on Multimedia10.1145/3664647.3681532(9952-9960)Online publication date: 28-Oct-2024
https://dl.acm.org/doi/10.1145/3664647.3681532
Hui CZhu HYan SLiu SJiang FZhao DCai JKankanhalli MPrabhakaran BBoll SSubramanian RZheng LSingh VCesar PXie LXu D(2024)S2-CSNet: Scale-Aware Scalable Sampling Network for Image Compressive SensingProceedings of the 32nd ACM International Conference on Multimedia10.1145/3664647.3681323(2515-2524)Online publication date: 28-Oct-2024
https://dl.acm.org/doi/10.1145/3664647.3681323
Zhu HGao ZWang JZhou YLi C(2024)Few-Shot Fine-Grained Image Classification via Multi-Frequency Neighborhood and Double-Cross ModulationIEEE Transactions on Multimedia10.1109/TMM.2024.340571326(10264-10278)Online publication date: 1-Jan-2024
https://dl.acm.org/doi/10.1109/TMM.2024.3405713
Ping PGuo BBloh OMao YXu F(2024)Hiding Multiple Images into a Single Image Using Up-SamplingIEEE Transactions on Multimedia10.1109/TMM.2023.332231626(4401-4415)Online publication date: 1-Jan-2024
https://dl.acm.org/doi/10.1109/TMM.2023.3322316
Cui WFan XZhang JZhao D(2024)Deep Unfolding Network for Image Compressed Sensing by Content-Adaptive Gradient Updating and Deformation-Invariant Non-Local ModelingIEEE Transactions on Multimedia10.1109/TMM.2023.332142426(4012-4027)Online publication date: 1-Jan-2024
https://dl.acm.org/doi/10.1109/TMM.2023.3321424
Kong XChen YHe Z(2024)When Channel Correlation Meets Sparse Prior: Keeping Interpretability in Image Compressive SensingIEEE Transactions on Multimedia10.1109/TMM.2023.330582826(2953-2965)Online publication date: 1-Jan-2024
https://dl.acm.org/doi/10.1109/TMM.2023.3305828
Show More Cited By

View Options

View options

Figures

Tables

Media

View Issue’s Table of Contents