research-article

Video Compressive Sensing Reconstruction via Reweighted Residual Sparsity

Authors:

Wen GaoAuthors Info & Claims

IEEE Transactions on Circuits and Systems for Video Technology, Volume 27, Issue 6

Pages 1182 - 1195

https://doi.org/10.1109/TCSVT.2016.2527181

Published: 01 June 2017 Publication History

Abstract

The compressive sensing (CS) theory indicates that robust reconstruction of signals can be obtained from far fewer measurements than those required by the Nyquist–Shannon theorem. Thus, CS has great potential in video acquisition and processing, considering that it makes the subsequent complex data compression unnecessary. In this paper, we propose a novel algorithm for effectively reconstructing videos from CS measurements. The algorithm comprises double phases, of which the first phase exploits intra-frame correlation and provides good initial recovery for each frame, and the second phase iteratively enhances reconstruction quality by alternating interframe multihypothesis (MH) prediction and sparsity modeling of residuals in a weighted manner. The weights of residual coefficients are updated in each iteration using a statistical method based on the MH predictions. These procedures are performed in the unit of overlapped patches such that potential blocking artifacts can be effectively suppressed through averaging. In addition, we devise an effective scheme based on the split Bregman iteration algorithm to solve the formulated weighted <inline-formula> <tex-math notation="LaTeX">${\ell }_{1}$ </tex-math></inline-formula> minimization problem. The experimental results demonstrate that the proposed algorithm outperforms the state-of-the-art methods in both objective and subjective reconstruction quality.

References

[1]

J. Y. Park and M. B. Wakin, “A multiscale framework for compressive sensing of video,” in Proc. IEEE Picture Coding Symp., May 2009, pp. 1–4.

[2]

J. Prades-Nebot, Y. Ma, and T. Huang, “Distributed video coding using compressive sampling,” in Proc. IEEE Picture Coding Symp., May 2009, pp. 1–4.

[3]

D. L. Donoho, “Compressed sensing,” IEEE Trans. Inf. Theory, vol. 52, no. 4, pp. 1289–1306, Apr. 2006.

Digital Library

[4]

Y. C. Eldar and G. Kutyniok, Compressed Sensing: Theory and Applications. Cambridge, U.K.: Cambridge Univ. Press, 2012.

[5]

E. J. Candès, J. Romberg, and T. Tao, “Robust uncertainty principles: Exact signal reconstruction from highly incomplete frequency information,” IEEE Trans. Inf. Theory, vol. 52, no. 2, pp. 489–509, Feb. 2006.

Digital Library

[6]

C. Zhao, J. Zhang, S. Ma, and W. Gao, “Compressive-sensed image coding via stripe-based DPCM,” Snowbird, UT, USA, in Proc. IEEE Data Compress. Conf. (DCC), Mar. 2016.

[7]

C. Zhao, S. Ma, and W. Gao, “Video compressive sensing via structured Laplacian modelling,” in Proc. IEEE Vis. Commun. Image Process. Conf. (VCIP), Dec. 2014, pp. 402–405.

[8]

C. Zhao, J. Zhang, S. Ma, R. Xiong, and W. Gao, “A dual structured-sparsity model for compressive-sensed video reconstruction,” in Proc. IEEE Int. Conf. Vis. Commun. Image Process. (VCIP), Singapore, Dec. 2015.

[9]

J. E. Fowler, S. Mun, and E. W. Tramel, “Block-based compressed sensing of images and video,” Found. Trends Signal Process., vol. 4, no. 4, pp. 297–416, Apr. 2012.

Digital Library

[10]

L.-W. Kang and C.-S. Lu, “Distributed compressive video sensing,” in Proc. IEEE Int. Conf. Acoust., Speech Signal Process., Apr. 2009, pp. 1169–1172.

[11]

J. Y. Park and M. B. Wakin, “Multiscale algorithm for reconstructing videos from streaming compressive measurements,” J. Electron. Imag., vol. 22, no. 2, p. 021001, Mar. 2013.

[12]

S. Pudlewski, A. Prasanna, and T. Melodia, “Compressed-sensing-enabled video streaming for wireless multimedia sensor networks,” IEEE Trans. Mobile Comput., vol. 11, no. 6, pp. 1060–1072, Jun. 2012.

Digital Library

[13]

R. Robucci, J. D. Gray, L. K. Chiu, J. Romberg, and P. Hasler, “Compressive sensing on a CMOS separable-transform image sensor,” Proc. IEEE, vol. 98, no. 6, pp. 1089–1101, Jun. 2010.

[14]

A. C. Sankaranarayanan, P. K. Turaga, R. Chellappa, and R. G. Baraniuk, “Compressive acquisition of linear dynamical systems,” SIAM J. Imag. Sci., vol. 6, no. 4, pp. 2109–2133, 2013.

[15]

N. Vaswani, “LS-CS-residual (LS-CS): Compressive sensing on least squares residual,” IEEE Trans. Signal Process., vol. 58, no. 8, pp. 4108–4120, Aug. 2010.

Digital Library

[16]

N. Vaswani and W. Lu, “Modified-CS: Modifying compressive sensing for problems with partially known support,” IEEE Trans. Signal Process., vol. 58, no. 9, pp. 4595–4607, Sep. 2010.

Digital Library

[17]

M. Wakin et al., “Compressive imaging for video representation and coding,” in Proc. Picture Coding Symp., vol. 1, 2006, pp. 13–18.

[18]

A. C. Sankaranarayanan, C. Studer, and R. G. Baraniuk, “CS-MUVI: Video compressive sensing for spatial-multiplexing cameras,” in Proc. IEEE Int. Conf. Comput. Photogr. (ICCP), Apr. 2012, pp. 1–10.

[19]

H. Jung, K. Sung, K. S. Nayak, E. Y. Kim, and J. C. Ye, “k-t FOCUSS: A general compressed sensing framework for high resolution dynamic MRI,” Magn. Reson. Med., vol. 61, no. 1, pp. 103–116, 2009.

[20]

D. Reddy, A. Veeraraghavan, and R. Chellappa, “P2C2: Programmable pixel compressive camera for high speed imaging,” in Proc. IEEE Conf. Comput. Vis. Pattern Recognit. (CVPR), Jun. 2011, pp. 329–336.

[21]

Y. Hitomi, J. Gu, M. Gupta, T. Mitsunaga, and S. K. Nayar, “Video from a single coded exposure photograph using a learned over-complete dictionary,” in Proc. IEEE Int. Conf. Comput. Vis. (ICCV), Nov. 2011, pp. 287–294.

[22]

A. Veeraraghavan, D. Reddy, and R. Raskar, “Coded strobing photography: Compressive sensing of high speed periodic videos,” IEEE Trans. Pattern Anal. Mach. Intell., vol. 33, no. 4, pp. 671–686, Apr. 2011.

Digital Library

[23]

J. Holloway, A. C. Sankaranarayanan, A. Veeraraghavan, and S. Tambe, “Flutter shutter video camera for compressive sensing of videos,” in Proc. IEEE Int. Conf. Comput. Photogr. (ICCP), Apr. 2012, pp. 1–9.

[24]

P. Llull et al., “Coded aperture compressive temporal imaging,” Opt. Exp., vol. 21, no. 9, pp. 10526–10545, 2013.

[25]

C. Li, W. Yin, and Y. Zhang, “User’s guide for TVAL3: TV minimization by augmented Lagrangian and alternating direction algorithms,” Dept. CAAM, Rice Univ., Houston, TX, USA, Tech. Rep., 2009.

[26]

S. Mun and J. E. Fowler, “Block compressed sensing of images using directional transforms,” in Proc. 16th IEEE Int. Conf. Image Process. (ICIP), Nov. 2009, pp. 3021–3024.

[27]

C. Zhao, J. Zhang, S. Ma, and W. Gao, “Nonconvex Lp nuclear norm based ADMM framework for compressed sensing,” in Proc. IEEE Data Compress. Conf. (DCC), Mar. 2016.

[28]

J. Zhang, R. Xiong, C. Zhao, Y. Zhang, S. Ma, and W. Gao, “CONCOLOR: Constrained non-convex low-rank model for image deblocking,” IEEE Trans. Image Process., vol. 25, no. 3, pp. 1246–1259, Mar. 2016.

Digital Library

[29]

J. Zhang, D. Zhao, and W. Gao, “Group-based sparse representation for image restoration,” IEEE Trans. Image Process., vol. 23, no. 8, pp. 3336–3351, Aug. 2014.

[30]

J. Zhang, D. Zhao, C. Zhao, R. Xiong, S. Ma, and W. Gao, “Image compressive sensing recovery via collaborative sparsity,” IEEE Emerg. Sel. Topics Circuits Syst., vol. 2, no. 3, pp. 380–391, Sep. 2012.

[31]

Y. Kim, M. S. Nadar, and A. Bilgin, “Compressed sensing using a Gaussian scale mixtures model in wavelet domain,” in Proc. 17th IEEE Int. Conf. Image Process. (ICIP), Sep. 2010, pp. 3365–3368.

[32]

L. He and L. Carin, “Exploiting structure in wavelet-based Bayesian compressive sensing,” IEEE Trans. Signal Process., vol. 57, no. 9, pp. 3488–3497, Sep. 2009.

Digital Library

[33]

C. Zhao, S. Ma, and W. Gao, “Image compressive-sensing recovery using structured Laplacian sparsity in DCT domain and multi-hypothesis prediction,” in Proc. IEEE Int. Conf. Multimedia Expo (ICME), Jul. 2014, pp. 1–6.

[34]

J. Zhang, D. Zhao, F. Jiang, and W. Gao, “Structural group sparse representation for image compressive sensing recovery,” in Proc. IEEE Data Compress. Conf. (DCC), Mar. 2013, pp. 331–340.

[35]

J. Zhang, C. Zhao, D. Zhao, and W. Gao, “Image compressive sensing recovery using adaptively learned sparsifying basis via L0 minimization,” Signal Process., vol. 103, pp. 114–126, Oct. 2014.

Digital Library

[36]

S. Mun and J. E. Fowler, “Residual reconstruction for block-based compressed sensing of video,” in Proc. IEEE Data Compress. Conf. (DCC), Mar. 2011, pp. 183–192.

[37]

C. Chen, E. W. Tramel, and J. E. Fowler, “Compressed-sensing recovery of images and video using multihypothesis predictions,” in Proc. Conf. Rec. 45th Asilomar Conf. Signals, Syst. Comput., Nov. 2011, pp. 1193–1198.

[38]

E. W. Tramel and J. E. Fowler, “Video compressed sensing with multihypothesis,” in Proc. IEEE Data Compress. Conf. (DCC), Mar. 2011, pp. 193–202.

[39]

E. J. Candès and T. Tao, “Decoding by linear programming,” IEEE Trans. Inf. Theory, vol. 51, no. 12, pp. 4203–4215, Dec. 2005.

Digital Library

[40]

R. G. Baraniuk, V. Cevher, M. F. Duarte, and C. Hegde, “Model-based compressive sensing,” IEEE Trans. Inf. Theory, vol. 56, no. 4, pp. 1982–2001, Apr. 2010.

Digital Library

[41]

M. A. Khajehnejad, W. Xu, A. S. Avestimehr, and B. Hassibi, “Analyzing weighted $\ell _{1}$ minimization for sparse recovery with nonuniform sparse models,” IEEE Trans. Signal Process., vol. 59, no. 5, pp. 1985–2001, May 2011.

Digital Library

[42]

E. J. Candès, M. B. Wakin, and S. P. Boyd, “Enhancing sparsity by reweighted L1 minimization,” J. Fourier Anal. Appl., vol. 14, nos. 5–6, pp. 877–905, 2008.

[43]

G. J. Sullivan, J.-R. Ohm, W.-J. Han, and T. Wiegand, “Overview of the High Efficiency Video Coding (HEVC) standard,” IEEE Trans. Circuits Syst. Video Technol., vol. 22, no. 12, pp. 1649–1668, Dec. 2012.

Digital Library

[44]

T. Wiegand, G. J. Sullivan, G. Bjøntegaard, and A. Luthra, “Overview of the H.264/AVC video coding standard,” IEEE Trans. Circuits Syst. Video Technol., vol. 13, no. 7, pp. 560–576, Jul. 2003.

Digital Library

[45]

E. Y. Lam and J. W. Goodman, “A mathematical analysis of the DCT coefficient distributions for images,” IEEE Trans. Image Process., vol. 9, no. 10, pp. 1661–1666, Oct. 2000.

[46]

T. Goldstein and S. Osher, “The split Bregman method for L1-regularized problems,” SIAM J. Imag. Sci., vol. 2, no. 2, pp. 323–343, 2009.

Digital Library

[47]

J.-F. Cai, S. Osher, and Z. Shen, “Split Bregman methods and frame based image restoration,” Multiscale Model. Simul., vol. 8, no. 2, pp. 337–369, 2010.

[48]

P. Deift and X. Zhou, “A steepest descent method for oscillatory Riemann–Hilbert problems. Asymptotics for the MKdV equation,” Ann. Math., vol. 137, no. 2, pp. 295–368, 1993.

[49]

D. L. Donoho, “De-noising by soft-thresholding,” IEEE Trans. Inf. Theory, vol. 41, no. 3, pp. 613–627, May 1995.

Digital Library

[50]

J. Zhang, D. Zhao, R. Xiong, S. Ma, and W. Gao, “Image restoration using joint statistical modeling in a space-transform domain,” IEEE Trans. Circuits Syst. Video Technol., vol. 24, no. 6, pp. 915–928, Jun. 2014.

[51]

S. G. Lingala, Y. Hu, E. DiBella, and M. Jacob, “Accelerated dynamic MRI exploiting sparsity and low-rank structure: k-t SLR,” IEEE Trans. Med. Imag., vol. 30, no. 5, pp. 1042–1054, May 2011.

[52]

T. T. Do, Y. Chen, D. T. Nguyen, N. Nguyen, L. Gan, and T. D. Tran, “Distributed compressed video sensing,” in Proc. 16th IEEE Int. Conf. Image Process. (ICIP), Nov. 2009, pp. 1393–1396.

[53]

J. Skorupa et al., “Efficient low-delay distributed video coding,” IEEE Trans. Circuits Syst. Video Technol., vol. 22, no. 4, pp. 530–544, Apr. 2012.

Digital Library

[54]

T. Maugey, J. Gauthier, M. Cagnazzo, and B. Pesquet-Popescu, “Evaluation of side information effectiveness in distributed video coding,” IEEE Trans. Circuits Syst. Video Technol., vol. 23, no. 12, pp. 2116–2126, Dec. 2013.

Digital Library

[55]

S. Mun and J. E. Fowler, “Motion-compensated compressed-sensing reconstruction for dynamic MRI,” in Proc. 20th IEEE Int. Conf. Image Process. (ICIP), Sep. 2013, pp. 1006–1010.

[56]

J. Aelterman, H. Q. Luong, B. Goossens, A. Pizurica, and W. Philips, “COMPASS: A joint framework for parallel imaging and compressive sensing in MRI,” in Proc. 17th IEEE Int. Conf. Image Process. (ICIP), Sep. 2010, pp. 1653–1656.

Cited By

Chen BZhang XLiu SZhang YZhang J(2025)Self-supervised Scalable Deep Compressed SensingInternational Journal of Computer Vision10.1007/s11263-024-02209-1133:2(688-723)Online publication date: 1-Feb-2025
https://dl.acm.org/doi/10.1007/s11263-024-02209-1
Han KWang JShi YCai HLing NYin B(2024)WTDUN: Wavelet Tree-Structured Sampling and Deep Unfolding Network for Image Compressed SensingACM Transactions on Multimedia Computing, Communications, and Applications10.1145/370173121:1(1-22)Online publication date: 26-Oct-2024
https://dl.acm.org/doi/10.1145/3701731
Zhong YZhang CYang XWang S(2024)Video Compressed Sensing Reconstruction via an Untrained Network with Low-Rank RegularizationIEEE Transactions on Multimedia10.1109/TMM.2023.332449026(4590-4601)Online publication date: 1-Jan-2024
https://dl.acm.org/doi/10.1109/TMM.2023.3324490
Show More Cited By

Index Terms

Video Compressive Sensing Reconstruction via Reweighted Residual Sparsity

Index terms have been assigned to the content through auto-classification.

Recommendations

Image compressive sensing via Truncated Schatten-p Norm regularization

Low-rank property as a useful image prior has attracted much attention in image processing communities. Recently, a nonlocal low-rank regularization (NLR) approach toward exploiting low-rank property has shown the state-of-the-art performance in ...
Image/video compressive sensing recovery using joint adaptive sparsity measure

Compressive sensing (CS) is a recently emerging technique and an extensively studied problem in signal and image processing, which enables joint sampling and compression into a unified approach. Recently, local smoothness and nonlocal self-similarity ...
Compressive sensing via nonlocal low-rank tensor regularization

The aim of Compressing sensing (CS) is to acquire an original signal, when it is sampled at a lower rate than Nyquist rate previously. In the framework of CS, the original signal is often assumed to be sparse and correlated in some domain. Recently, ...

Comments

Information & Contributors

Information

Published In

cover image IEEE Transactions on Circuits and Systems for Video Technology

IEEE Transactions on Circuits and Systems for Video Technology Volume 27, Issue 6

June 2017

220 pages

ISSN:1051-8215

Issue’s Table of Contents

1051-8215 © 2016 IEEE. Personal use is permitted, but republication/redistribution requires IEEE permission. See http://www.ieee.org/publications_standards/publications/rights/index.html for more information.

Publisher

IEEE Press

Publication History

Published: 01 June 2017

Qualifiers

Research-article

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

25
Total Citations
View Citations
0
Total Downloads

Downloads (Last 12 months)0
Downloads (Last 6 weeks)0

Reflects downloads up to 11 Feb 2025

Other Metrics

View Author Metrics

Citations

Cited By

Chen BZhang XLiu SZhang YZhang J(2025)Self-supervised Scalable Deep Compressed SensingInternational Journal of Computer Vision10.1007/s11263-024-02209-1133:2(688-723)Online publication date: 1-Feb-2025
https://dl.acm.org/doi/10.1007/s11263-024-02209-1
Han KWang JShi YCai HLing NYin B(2024)WTDUN: Wavelet Tree-Structured Sampling and Deep Unfolding Network for Image Compressed SensingACM Transactions on Multimedia Computing, Communications, and Applications10.1145/370173121:1(1-22)Online publication date: 26-Oct-2024
https://dl.acm.org/doi/10.1145/3701731
Zhong YZhang CYang XWang S(2024)Video Compressed Sensing Reconstruction via an Untrained Network with Low-Rank RegularizationIEEE Transactions on Multimedia10.1109/TMM.2023.332449026(4590-4601)Online publication date: 1-Jan-2024
https://dl.acm.org/doi/10.1109/TMM.2023.3324490
Cui WFan XZhang JZhao D(2024)Deep Unfolding Network for Image Compressed Sensing by Content-Adaptive Gradient Updating and Deformation-Invariant Non-Local ModelingIEEE Transactions on Multimedia10.1109/TMM.2023.332142426(4012-4027)Online publication date: 1-Jan-2024
https://dl.acm.org/doi/10.1109/TMM.2023.3321424
Zhao YZhang JChen YWang ZLi X(2024)RCUMP: Residual Completion Unrolling With Mixed Priors for Snapshot Compressive ImagingIEEE Transactions on Image Processing10.1109/TIP.2024.337409333(2347-2360)Online publication date: 18-Mar-2024
https://dl.acm.org/doi/10.1109/TIP.2024.3374093
Li WChen BLiu SZhao SDu BZhang YZhang J(2024)D³C²-Net: Dual-Domain Deep Convolutional Coding Network for Compressive SensingIEEE Transactions on Circuits and Systems for Video Technology10.1109/TCSVT.2024.339701234:10_Part_1(9341-9355)Online publication date: 1-Oct-2024
https://dl.acm.org/doi/10.1109/TCSVT.2024.3397012
Yue HGuo JYin XZhang YWen BLi C(2024)Salient Object Detection Toward Single-Pixel ImagingIEEE Transactions on Circuits and Systems for Video Technology10.1109/TCSVT.2023.328370534:1(235-247)Online publication date: 1-Jan-2024
https://dl.acm.org/doi/10.1109/TCSVT.2023.3283705
Zha ZWen BYuan XZhang JZhou JZhu C(2024)Structured residual sparsity for video compressive sensing reconstructionSignal Processing10.1016/j.sigpro.2024.109513222:COnline publication date: 1-Sep-2024
https://dl.acm.org/doi/10.1016/j.sigpro.2024.109513
Wang HLi HJiang X(2024)DMFNet: deep matrix factorization network for image compressed sensingMultimedia Systems10.1007/s00530-024-01380-230:4Online publication date: 26-Jun-2024
https://dl.acm.org/doi/10.1007/s00530-024-01380-2
Nezhad VAzghani MMarvasti F(2024)Compressed Video Sensing Based on Deep Generative Adversarial NetworkCircuits, Systems, and Signal Processing10.1007/s00034-024-02672-843:8(5048-5064)Online publication date: 1-Aug-2024
https://dl.acm.org/doi/10.1007/s00034-024-02672-8
Show More Cited By

View Options

View options

Figures

Tables

Media

View Issue’s Table of Contents