Deep Network for Image Compressed Sensing Coding Using Local Structural Sampling
Authors: 
Wenxue Cui, Xingtao Wang, Xiaopeng Fan, Shaohui Liu, Xinwei Gao, Debin ZhaoAuthors Info & Claims
Wenxue Cui, Xingtao Wang, Xiaopeng Fan, Shaohui Liu, Xinwei Gao, Debin ZhaoAuthors Info & ClaimsArticle No.: 212, Pages 1 - 22
Abstract
Existing image compressed sensing (CS) coding frameworks usually solve an inverse problem based on measurement coding and optimization-based image reconstruction, which still exist the following two challenges: (1) the widely used random sampling matrix, such as the Gaussian Random Matrix (GRM), usually leads to low measurement coding efficiency, and (2) the optimization-based reconstruction methods generally maintain a much higher computational complexity. In this article, we propose a new convolutional neural network based image CS coding framework using local structural sampling (dubbed CSCNet) that includes three functional modules: local structural sampling, measurement coding, and Laplacian pyramid reconstruction. In the proposed framework, instead of GRM, a new local structural sampling matrix is first developed, which is able to enhance the correlation between the measurements through a local perceptual sampling strategy. Besides, the designed local structural sampling matrix can be jointly optimized with the other functional modules during the training process. After sampling, the measurements with high correlations are produced, which are then coded into final bitstreams by the third-party image codec. Last, a Laplacian pyramid reconstruction network is proposed to efficiently recover the target image from the measurement domain to the image domain. Extensive experimental results demonstrate that the proposed scheme outperforms the existing state-of-the-art CS coding methods while maintaining fast computational speed.
References
[1]
E. Agustsson, M. Tschannen, F. Mentzer, R. Timofte, and L. Van Gool. 2019. Generative adversarial networks for extreme learned image compression. In Proceedings of the 2019 IEEE/CVF International Conference on Computer Vision (ICCV ’19). 221–231. DOI:
[2]
Emmanuel J. Candès and Michael B. Wakin. 2008. An introduction to compressive sampling. IEEE Signal Processing Magazine 25, 2 (2008), 21–30.
[3]
Bin Chen and Jian Zhang. 2022. Content-aware scalable deep compressed sensing. IEEE Transactions on Image Processing 31 (2022), 5412–5426.
[4]
T. Chen, H. Liu, Z. Ma, Q. Shen, X. Cao, and Y. Wang. 2021. End-to-end learnt image compression via non-local attention optimization and improved context modeling. IEEE Transactions on Image Processing 30 (2021), 3179–3191. DOI:
[5]
Zan Chen, Xingsong Hou, Xueming Qian, and Chen Gong. 2018. Efficient and robust image coding and transmission based on scrambled block compressive sensing. IEEE Transactions on Multimedia 20, 7 (2018), 1610–1621. DOI:
[6]
Z. Chen, X. Hou, L. Shao, C. Gong, X. Qian, Y. Huang, and S. Wang. 2020. Compressive sensing multi-layer residual coefficients for image coding. IEEE Transactions on Circuits and Systems for Video Technology 30, 4 (2020), 1109–1120. DOI:
[7]
Wonwoo Cho and Nam Yul Yu. 2020. Secure and efficient compressed sensing-based encryption with sparse matrices. IEEE Transactions on Information Forensics and Security 15 (2020), 1999–2011. DOI:
[8]
Wenxue Cui, Feng Jiang, Xinwei Gao, Shengping Zhang, and Debin Zhao. 2018. An efficient deep quantized compressed sensing coding framework of natural images. In Proceedings of the ACM International Conference on Multimedia (MM ’18). 1777–1785. DOI:
[9]
Ingrid Daubechies, Michel Defrise, and Christine De Mol. 2004. An iterative thresholding algorithm for linear inverse problems with a sparsity constraint. Communications on Pure and Applied Mathematics: A Journal Issued by the Courant Institute of Mathematical Sciences 57, 11 (2004), 1413–1457.
[10]
K. Q. Dinh and B. Jeon. 2017. Iterative weighted recovery for block-based compressive sensing of image/video at a low subrate. IEEE Transactions on Circuits and Systems for Video Technology 27, 11 (2017), 2294–2308. DOI:
[11]
Khanh Quoc Dinh, Hiuk Jae Shim, and Byeungwoo Jeon. 2013. Measurement coding for compressive imaging using a structural measurement matrix. In Proceedings of the 2013 IEEE International Conference on Image Processing. 10–13. DOI:
[12]
Weisheng Dong, Guangming Shi, Xin Li, Yi Ma, and Feng Huang. 2014. Compressive sensing via nonlocal low-rank regularization. IEEE Transactions on Image Processing 23, 8 (2014), 3618–3632.
[13]
David L. Donoho. 2006. Compressed sensing. IEEE Transactions on Information Theory 52, 4 (2006), 1289–1306.
[14]
Marco F. Duarte, Mark A. Davenport, Dharmpal Takhar, Jason N. Laska, Ting Sun, Kevin F. Kelly, and Richard G. Baraniuk. 2008. Single-pixel imaging via compressive sampling. IEEE Signal Processing Magazine 25, 2 (2008), 83–91. DOI:
[15]
Mansoor Ebrahim and Wai Chong Chia. 2015. Multiview image block compressive sensing with joint multiphase decoding for visual sensor network. ACM Transactions on Multimedia Computing, Communications, and Applications 12, 2 (Oct. 2015), Article 30, 23 pages. DOI:
[16]
Mark Everingham and John Winn. 2011. The PASCAL Visual Object Classes Challenge 2012 (VOC2012) Development Kit. Retrieved March 5, 2024 from http://host.robots.ox.ac.uk/pascal/VOC/voc2012/htmldoc/index.html
[17]
Lu Gan. 2007. Block compressed sensing of natural images. In Proceedings of the International Conference on Digital Signal Processing. 403–406.
[18]
Xinwei Gao, Jian Zhang, Wenbin Che, Xiaopeng Fan, and Debin Zhao. 2015. Block-based compressive sensing coding of natural images by local structural measurement matrix. In Proceedings of the IEEE Data Compression Conference (DCC ’15). 133–142.
[19]
Zhifan Gao, Yifeng Guo, Jiajing Zhang, Tieyong Zeng, and Guang Yang. 2023. Hierarchical perception adversarial learning framework for compressed sensing MRI. IEEE Transactions on Medical Imaging 42, 6 (2023), 1859–1874. DOI:
[20]
Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun. 2015. Delving deep into rectifiers: Surpassing human-level performance on ImageNet classification. In Proceedings of the IEEE International Conference on Computer Vision (ICCV ’15). 1026–1034.
[21]
Gang Huang, Hong Jiang, Kim Matthews, and Paul Wilford. 2013. Lensless imaging by compressive sensing. In Proceedings of the 2013 IEEE International Conference on Image Processing. 2101–2105. DOI:
[22]
Laurent Jacques, David K. Hammond, and Jalal M. Fadili. 2011. Dequantizing compressed sensing: When oversampling and non-Gaussian constraints combine. IEEE Transactions on Information Theory 57, 1 (2011), 559–571.
[23]
C. Jia, X. Zhang, S. Wang, S. Wang, and S. Ma. 2019. Light field image compression using generative adversarial network-based view synthesis. IEEE Journal on Emerging and Selected Topics in Circuits and Systems 9, 1 (2019), 177–189. DOI:
[24]
F. Jiang, W. Tao, S. Liu, J. Ren, X. Guo, and D. Zhao. 2018. An end-to-end compression framework based on convolutional neural networks. IEEE Transactions on Circuits and Systems for Video Technology 28, 10 (2018), 3007–3018. DOI:
[25]
C. Li, W. Yin, and Y. Zhang. 2013. TVAL3: TV Minimization by Augmented Lagrangian and Alternating Direction Algorithms. Retrieved March 5, 2024 from https://nuit-blanch/blogspot.com/2009/06/cs-tval3-tv-minimization-by-augmented.html
[26]
M. Li, K. Ma, J. You, D. Zhang, and W. Zuo. 2020. Efficient and effective context-based convolutional entropy modeling for image compression. IEEE Transactions on Image Processing 29 (2020), 5900–5911. DOI:
[27]
M. Li, W. Zuo, S. Gu, J. You, and D. Zhang. 2020. Learning content-weighted deep image compression. IEEE Transactions on Pattern Analysis and Machine Intelligence. Published Online, March 30, 2020. DOI:
[28]
Y. Li, D. Liu, H. Li, L. Li, Z. Li, and F. Wu. 2019. Learning a convolutional neural network for image compact-resolution. IEEE Transactions on Image Processing 28, 3 (2019), 1092–1107. DOI:
[29]
Y. Li, D. Liu, H. Li, L. Li, F. Wu, H. Zhang, and H. Yang. 2018. Convolutional neural network-based block up-sampling for intra frame coding. IEEE Transactions on Circuits and Systems for Video Technology 28, 9 (2018), 2316–2330. DOI:
[30]
Yinghua Li, Bin Song, Rong Cao, Yue Zhang, and Hao Qin. 2016. Image encryption based on compressive sensing and scrambled index for secure multimedia transmission. ACM Transactions on Multimedia Computing, Communications, and Applications 12, 4s (2016), 1–22.
[31]
J. Lin, D. Liu, H. Yang, H. Li, and F. Wu. 2019. Convolutional neural network-based block up-sampling for HEVC. IEEE Transactions on Circuits and Systems for Video Technology 29, 12 (2019), 3701–3715. DOI:
[32]
Xianming Liu, Deming Zhai, Jiantao Zhou, Xinfeng Zhang, Debin Zhao, and Wen Gao. 2016. Compressive sampling-based image coding for resource-deficient visual communication. IEEE Transactions on Image Processing 25, 6 (2016), 2844–2855.
[33]
Zhi Liu, Shuyuan Yang, Zhixi Feng, Min Wang, and Zhifan Yu. 2022. Deep compressive imaging with meta-learning. IEEE Transactions on Instrumentation and Measurement 72 (2022), 1–9.
[34]
Michael Lustig, David L. Donoho, Juan M. Santos, and John M. Pauly. 2008. Compressed sensing MRI. IEEE Signal Processing Magazine 25, 2 (2008), 72–82.
[35]
Stéphane G. Mallat and Zhifeng Zhang. 1993. Matching pursuits with time-frequency dictionaries. IEEE Transactions on Signal Processing 41, 12 (1993), 3397–3415.
[36]
Christopher A. Metzler, Arian Maleki, and Richard G. Baraniuk. 2016. From denoising to compressed sensing. IEEE Transactions on Information Theory 62, 9 (2016), 5117–5144.
[37]
Ali Mousavi, Ankit B. Patel, and Richard G. Baraniuk. 2015. A deep learning approach to structured signal recovery. In Proceedings of the IEEE Allerton Conference on Communication, Control, and Computing (Allerton ’15). 1336–1343.
[38]
Sungkwang Mun and James E. Fowler. 2012. DPCM for quantized blockbased compressed sensing of images. In Proceedings of the IEEE Signal Processing Conference. 1424–1428.
[39]
Wuzhen Shi, Feng Jiang, Shaohui Liu, and Debin Zhao. 2019. Scalable convolutional neural network for image compressed sensing. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. 12290–12299.
[40]
Wuzhen Shi, Feng Jiang, Shaohui Liu, and Debin Zhao. 2020. Image compressed sensing using convolutional neural network. IEEE Transactions on Image Processing 29, 1 (2020), 375–388.
[41]
W. Shi, S. Liu, F. Jiang, and D. Zhao. 2021. Video compressed sensing using a convolutional neural network. IEEE Transactions on Circuits and Systems for Video Technology 31, 2 (2021), 425–438. DOI:
[42]
Wuzhen Shi, Fei Tao, and Yang Wen. 2023. Structure-aware deep networks and pixel-level generative adversarial training for single image super-resolution. IEEE Transactions on Instrumentation and Measurement 72 (2023), 1–14. DOI:
[43]
Jiechong Song, Bin Chen, and Jian Zhang. 2021. Memory-augmented deep unfolding network for compressive sensing. In Proceedings of the 29th ACM International Conference on Multimedia. 4249–4258.
[44]
Gary J. Sullivan, Jens Ohm, Woo Jin Han, and Thomas Wiegand. 2012. Overview of the high efficiency video coding (HEVC) standard. IEEE Transactions on Circuits and Systems for Video Technology 22, 12 (2012), 1649–1668.
[45]
Yubao Sun, Jiwei Chen, Qingshan Liu, Bo Liu, and Guodong Guo. 2020. Dual-path attention network for compressed sensing image reconstruction. IEEE Transactions on Image Processing 29 (2020), 9482–9495. DOI:
[46]
Chaoqing Tang, Guiyun Tian, Said Boussakta, and Jianbo Wu. 2019. Feature-supervised compressed sensing for microwave imaging systems. IEEE Transactions on Instrumentation and Measurement 69, 8 (2019), 5287–5297.
[47]
Thuy T. T. Tran, Jirayu Peetakul, Chi D. K. Pham, and Jinjia Zhou. 2020. Bi-directional intra prediction based measurement coding for compressive sensing images. In Proceedings of the 2020 IEEE 22nd International Workshop on Multimedia Signal Processing (MMSP ’20). 1–6. DOI:
[48]
Joel A. Tropp and Anna C. Gilbert. 2007. Signal recovery from random measurements via orthogonal matching pursuit. IEEE Transactions on Information Theory 53, 12 (2007), 4655–4666.
[49]
Rentao Wan, Jinjia Zhou, Bowen Huang, Hui Zeng, and Yibo Fan. 2022. APMC: Adjacent pixels based measurement coding system for compressively sensed images. IEEE Transactions on Multimedia 24 (2022), 3558–3569. DOI:
[50]
Huake Wang, Ziang Li, and Xingsong Hou. 2023. Versatile denoising-based approximate message passing for compressive sensing. IEEE Transactions on Image Processing 32 (2023), 2761–2775. DOI:
[51]
Thomas Wiegand, Gary J. Sullivan, Gisle Bjøntegaard, and Ajay Luthra. 2003. Overview of the H. 264/AVC video coding standard. IEEE Transactions on Circuits and Systems for Video Technology 13, 7 (2003), 560–576.
[52]
Stephen J. Wright, Robert D. Nowak, and Mário A. T. Figueiredo. 2009. Sparse reconstruction by separable approximation. IEEE Transactions on Signal Processing 57, 7 (2009), 2479–2493.
[53]
Dapeng Wu, Boran Yang, Honggang Wang, Chonggang Wang, and Ruyan Wang. 2016. Privacy-preserving multimedia big data aggregation in large-scale wireless sensor networks. ACM Transactions on Multimedia Computing, Communications, and Applications 12, 4s (2016), 1–19.
[54]
Hongwei Xu, Ning Fu, Liyan Qiao, Wei Yu, and Xiyuan Peng. 2014. Compressive blind mixing matrix estimation of audio signals. IEEE Transactions on Instrumentation and Measurement 63, 5 (2014), 1253–1261.
[55]
Kai Xu, Zhikang Zhang, and Fengbo Ren. 2018. LAPRAN: A scalable Laplacian pyramid reconstructive adversarial network for flexible compressive sensing reconstruction. In Proceedings of the European Conference on Computer Vision (ECCV ’18). 491–507.
[56]
Wenjie Yan, Qiang Wang, and Yi Shen. 2014. Shrinkage-based alternating projection algorithm for efficient measurement matrix construction in compressive sensing. IEEE Transactions on Instrumentation and Measurement 63, 5 (2014), 1073–1084.
[57]
Peihao Yang, Linghe Kong, Meikang Qiu, Xue Liu, and Guihai Chen. 2021. Compressed imaging reconstruction with sparse random projection. ACM Transactions on Multimedia Computing, Communications, and Applications 17, 1 (April 2021), Article 26, 25 pages. DOI:
[58]
Zai Yang, Lihua Xie, and Cishen Zhang. 2013. Variational Bayesian algorithm for quantized compressed sensing. IEEE Transactions on Signal Processing 61, 11 (2013), 2815–2824.
[59]
Hantao Yao, Feng Dai, Dongming Zhang, Yike Ma, Shiliang Zhang, Yongdong Zhang, and Qi Tian. 2017. DR2-Net: Deep residual reconstruction network for image compressive sensing. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR ’17).
[60]
Di You, Jian Zhang, Jingfen Xie, Bin Chen, and Siwei Ma. 2021. COAST: COntrollable Arbitrary-Sampling neTwork for compressive sensing. IEEE Transactions on Image Processing 30 (2021), 6066–6080. DOI:
[61]
Xin Yuan and Raziel Haimi-Cohen. 2020. Image compression based on compressive sensing: End-to-end comparison with JPEG. IEEE Transactions on Multimedia 22, 11 (2020), 2889–2904. DOI:
[62]
Jian Zhang and Bernard Ghanem. 2018. ISTA-Net: Interpretable optimization-inspired deep network for image compressive sensing. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR ’18). 1828–1837.
[63]
Jian Zhang, Chen Zhao, and Wen Gao. 2020. Optimization-inspired compact deep compressive sensing. IEEE Journal of Selected Topics in Signal Processing 14, 4 (2020), 765–774. DOI:
[64]
Jian Zhang, Debin Zhao, and Wen Gao. 2014. Group-based sparse representation for image restoration. IEEE Transactions on Image Processing 23, 8 (2014), 3336–3351.
[65]
Jian Zhang, Debin Zhao, and Feng Jiang. 2013. Spatially directional predictive coding for block-based compressive sensing of natural images. In Proceedings of the IEEE International Conference on Image Processing (ICIP ’13). 1021–1025.
[66]
Jian Zhang, Debin Zhao, Chen Zhao, Ruiqin Xiong, Siwei Ma, and Wen Gao. 2012. Compressed sensing recovery via collaborative sparsity. In Proceedings of the Data Compression Conference (DCC ’12). IEEE, 287–296.
[67]
Zhonghao Zhang, Yipeng Liu, Jiani Liu, Fei Wen, and Ce Zhu. 2021. AMP-Net: Denoising-based deep unfolding for compressive image sensing. IEEE Transactions on Image Processing 30 (2021), 1487–1500. DOI:
[68]
Chen Zhao, Jian Zhang, Siwei Ma, and Wen Gao. 2016. Nonconvex lp nuclear norm based ADMM framework for compressed sensing. In Proceedings of the 2016 Data Compression Conference (DCC ’16). IEEE, 161–170.
[69]
L. Zhao, H. Bai, A. Wang, and Y. Zhao. 2019. Multiple description convolutional neural networks for image compression. IEEE Transactions on Circuits and Systems for Video Technology 29, 8 (2019), 2494–2508. DOI:
[70]
S. Zhu, C. Cui, R. Xiong, Y. Guo, and B. Zeng. 2019. Efficient chroma sub-sampling and luma modification for color image compression. IEEE Transactions on Circuits and Systems for Video Technology 29, 5 (2019), 1559–1563. DOI:
Index Terms
- Deep Network for Image Compressed Sensing Coding Using Local Structural Sampling
Recommendations
An Efficient Deep Quantized Compressed Sensing Coding Framework of Natural Images
MM '18: Proceedings of the 26th ACM international conference on MultimediaTraditional image compressed sensing (CS) coding frameworks solve an inverse problem that is based on the measurement coding tools (prediction, quantization, entropy coding, etc.) and the optimization based image reconstruction method. These CS coding ...
Adaptive sampling for compressed sensing based image compression
A jointly-reweighted block-based compressed sensing scheme.A generic measurement allocation algorithm to assign CS-measurements.Statistical parameters as allocation factors.Two solutions to implement the adaptive measurement allocation.Remarkable ...
Comments
Information & Contributors
Information
Published In
July 2024
973 pages
ISSN:1551-6857
EISSN:1551-6865
DOI:10.1145/3613662
- Editor:
- Abdulmotaleb El Saddik
Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than the author(s) must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected].
Publisher
Association for Computing Machinery
New York, NY, United States
Publication History
Published: 16 May 2024
Online AM: 26 February 2024
Accepted: 16 February 2024
Revised: 08 January 2024
Received: 10 April 2023
Published in TOMM Volume 20, Issue 7
Check for updates
Author Tags
Qualifiers
- Research-article
Funding Sources
- National Key R&D Program of China
- National Natural Science Foundation of China (NSFC)
Contributors
Other Metrics
Bibliometrics & Citations
Bibliometrics
Article Metrics
- 0Total Citations
- 205Total Downloads
- Downloads (Last 12 months)205
- Downloads (Last 6 weeks)24
Reflects downloads up to 27 Jul 2024
Other Metrics
Citations
View Options
Get Access
Login options
Check if you have access through your login credentials or your institution to get full access on this article.
Sign in