Location via proxy:   [ UP ]  
[Report a bug]   [Manage cookies]                
Skip to main content
SpringerLink
Log in

Half quadratic splitting method combined with convolution neural network for blind image deblurring

  • Published:
Multimedia Tools and Applications Aims and scope Submit manuscript

Abstract

Blind image deblurring is the process of recovering the original image from a degraded image under unknown point spread function, and it is the solution to an ill-posed inverse problem. In this paper, the blurry image is firstly divided into skeleton image and blur kernel, aiming to achieve accurate blur kernel estimation. Then the advantages of model-based optimization method and discriminative learning method are integrated through variable splitting technique. Finally, a trained convolutional neural network (CNN) is used as a module to be inserted into a model-based optimization method to solve the problem of blind image deblurring more effectively. By comparing visual and quantitative experimental data, the network proposed in this paper can provide powerful prior information for blind image deblurring and the restoration effects can approximate or exceed those of some representative algorithms.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Subscribe and save

Springer+ Basic
$34.99 /Month
  • Get 10 units per month
  • Download Article/Chapter or eBook
  • 1 Unit = 1 Article or 1 Chapter
  • Cancel anytime
Subscribe now

Buy Now

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6

Similar content being viewed by others

Explore related subjects

Find the latest articles, discoveries, and news in related topics.

References

  1. Bai Y, Cheung G, Liu X et al (2018) Graph-based blind image Deblurring from a single photograph[J]. IEEE Trans Image Process 28(3):1404–1418

    Article  MathSciNet  Google Scholar 

  2. Boyd S, Parikh N, Chu E et al (2010) Distributed Optimization and Statistical Learning via the Alternating Direction Method of Multipliers[J]. Found Trends® Mach Learn 3(1):1–122

    Article  Google Scholar 

  3. Brifman A, Romano Y, Elad M (2016) Turning a denoiser into a super-resolver using plug and play priors[C]// 2016 IEEE International Conference on Image Processing (ICIP). IEEE

  4. Cai JF, Ji H, Liu C, Shen Z (2012) Framelet-based blind motion Deblurring from a single image. IEEE Trans Image Process 21(2):562–572

    Article  MathSciNet  Google Scholar 

  5. Chambolle A, De Vore RA, Lee N-Y et al (1998) Nonlinear wavelet image processing: Variational problems, compression, and noise removal through wavelet shrinkage. IEEE Trans Image Process 7(3):319–335

    Article  MathSciNet  Google Scholar 

  6. Chan TF, Wong CK (1998) Total variation blind deconvolution. IEEE Trans Image Process 7(3):370–375

    Article  Google Scholar 

  7. Chan SH, Wang X, Elgendy OA (2016) Plug-and-play ADMM for image restoration: fixed point convergence and applications[J]. IEEE Trans Comput Imaging 3(1):84–98

    Article  MathSciNet  Google Scholar 

  8. Chen Y, Pock T (2017) Trainable nonlinear reaction diffusion: a flexible framework for fast and effective image restoration[J]. IEEE Trans Patt Analysis Mach Intell 39(6):1256–1272

    Article  Google Scholar 

  9. Cho S, Lee S (2009) Fast motion deblurring. ACM Trans Graph 28(5):1–8

  10. Deng J, Dong W, Socher R, Li L-J, Li K, FeiFei L (2009) Imagenet: a large-scale hierarchical image database. In IEEE Conference on Computer Vision and Pattern Recognition, p. 248–255

  11. Fang H, Yan L, Liu H, Chang Y (2013) Blind Poissonian images deconvolution with framelet regularization. Opt Lett 38(4):389–391

    Article  Google Scholar 

  12. Fereus R, Sineh B, Hertzmann A, Roweis ST, Freeman WT (2006) Removine camera shake from a single photograph. ACM Trans Graph 25(3):787–794

    Article  Google Scholar 

  13. Fergus R, Singh B, Hertzmann A, Roweis ST, Freeman WT (2006) Removing camera shake from a single photograph. ACM Trans Graph 25(3):787–794

    Article  Google Scholar 

  14. Figueiredo MAT, Nowak RD (2003) An EM algorithm for wavelet-based image restoration. IEEE Trans Image Process 12(8):906–916

    Article  MathSciNet  Google Scholar 

  15. Gao H, Tao X, Shen X, Jia J (2019) Dynamic Scene Deblurring With Parameter Selective Sharing and Nested Skip Connections. 3843–3851. https://doi.org/10.1109/CVPR.2019.00397

  16. Geman D, Yang C (1994) Nonlinear image recovery with half-quadratic regularization[J]. IEEE Trans Image Process 4(7):932

    Article  Google Scholar 

  17. Gonzalez RC, Woods RE (2007) Digital Image Processing (3rd Edition) [M]. Prentice-Hall, Inc

  18. Hirsch M, Schuler CJ, Harmeling S, Schölkopf B (2011) Fast removal of non-uniform camera shake. 2011 International Conference on Computer Vision, p 463–470

  19. Ioffe S, Szegedy C (2015) Batch normalization: accelerating deep network training by reducing internal covariate shift[C]// International Conference on International Conference on Machine Learning. JMLR.org

  20. Kingma DP, Ba J (2014) Adam: a method for stochastic optimization[J]. Comp Sci

  21. Krizhevsky A, Sutskever I, Hinton G (2012) ImageNet classification with deep convolutional neural networks[C]// NIPS. Curran Associates Inc

  22. Kundur D, Hatzinakos D (1996) Blind image deconvolution. IEEE Signal Process Mag 13(3):43–64

    Article  Google Scholar 

  23. Kundur D, Hatzinakos D (2013) Blind image deconvolution[J]. Signal Process Mag IEEE 13(3):43–64

    Article  Google Scholar 

  24. Kupyn O, Budzan V, Mykhailych M et al (2018) DeblurGAN: blind motion Deblurring using conditional adversarial networks[C]// 2018 IEEE/CVF conference on computer vision and pattern recognition (CVPR). IEEE 8183–8192

  25. Kupyn O, Martyniuk T, Wu J, Wang Z (2019) DeblurGAN-v2: Deblurring (Orders-of-Magnitude) Faster and Better. 2019 IEEE/CVF International Conference on Computer Vision (ICCV), 8877–8886

  26. Levin A, Weiss Y, Durand F, Freeman WT (2011) Understanding blind Deconvolution algorithms. IEEE Trans Patt Analysis Mach Intell 33(12):2354–2367

    Article  Google Scholar 

  27. Levin A, Weiss Y, Durand F, Freeman WT (2011) Effificient marginal likelihood optimization in blind deconvolution. In CVPR 2011, p 2657–2664

  28. Li L, Pan J, Lai WS, Gao C, Sang N, Yang MH (2019) Blind image Deblurring via deep discriminative priors[J]. Int J Comput Vis 127:1025–1043

    Article  Google Scholar 

  29. Lu B, Chen J-C, Chellappa R (2019) Unsupervised domain-specific deblurring via disentangled representations. 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) 10217–10226

  30. Lu X, Wang W, Ma C et al (2019) See more, know more: unsupervised video object segmentation with co-attention Siamese networks[C]// CVPR19. 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) 3618–3627

  31. Lu X, Wang W, Shen J, Tai Y-W, Crandall D, Hoi S (2020) Learning video object segmentation from unlabeled videos. 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) 8957–8967

  32. Ma K, Duanmu Z, Wu Q, Wang Z, Yong H, Li H, Zhang L (2017) Waterloo exploration database: new challenges for image quality assessment models. IEEE Trans Image Process 26(2):1004–1016

    Article  MathSciNet  Google Scholar 

  33. Malczewski K, Stasinski R (2008) Toeplitz-based iterative image fusion scheme for MRI[C]// IEEE International Conference on Image Processing. IEEE

  34. Pan Z, Yu J et al (2013) Super-resolution based on compressive sensing and structural self-similarity for remote sensing images[J]. IEEE Trans Geoence Remote Sens 51(9):4864–4876

    Article  Google Scholar 

  35. Pan J, Sun D, Pfister H et al. (2016) Blind image Deblurring using Dark Channel prior// 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR). IEEE

  36. Parikh N, Boyd SP (2014) Proximal algorithms. Found Trends R Optim 1:123–231

    Google Scholar 

  37. Romano Y, Elad M, Milanfar P (2016) The little engine that could regularization by denoising (RED). arXiv preprint arXiv:1611.02862

  38. Rond A, Giryes R, Elad M (2016) Poisson inverse problems by the plug-and-play scheme. J Vis Commun Image Represent 41:96–108

    Article  Google Scholar 

  39. Schuler CJ, Hirsch M, Harmeling S et al (2014) Learning to Deblur[J]. IEEE Trans Patt Analysis Mach Intell 38(7):1439–1451

    Article  Google Scholar 

  40. Shan Q, Jia J, Agarwala A (2008) High-quality motion deblurring from a single image. ACM Trans Graph 27(3)73

  41. Shen Z, Wang W, Lu X et al (2019) Human-aware motion Deblurring. 2019 IEEE/CVF International Conference on Computer Vision (ICCV), 5571–5580

  42. Sreehari S, Venkatakrishnan SV, Wohlberg B et al (2017) Plug-and-play priors for bright field Electron tomography and sparse interpolation[J]. IEEE Trans Comput Imaging 2(4):408–423

    MathSciNet  Google Scholar 

  43. Tao X, Gao H, Wang Y et al. (2018) Scale-recurrent network for deep image Deblurring[C]// 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition. IEEE

  44. Teodoro AM, Bioucas-Dias JM, Figueiredo MA (2016) Image restoration and reconstruction using variable splitting and class-adapted image priors. 2016 IEEE International Conference on Image Processing (ICIP), 3518–3522

  45. Vedaldi A, Lenc K (2015) MatConvNet: convolutional neural networks for matlab. In ACM Conference on Multimedia Conference, p 689–692

  46. Whyte O, Sivic J, Zisserman A, Ponce J (2010) Non-uniform deblurring for shaken images. Int J Comput Vis 98:168–186

  47. Xu L, Jia J (2010) Two-phase kernel estimation for robust motion deblurring[C]// Computer Vision - ECCV 2010, 11th European Conference on Computer Vision, Heraklion, Crete, Greece, September 5-11, 2010, Proceedings, Part I. DBLP

  48. Yu F, Koltun V (2015) Multi-scale context aggregation by dilated convolutions[J]

  49. Zhang J, Pan J, Lai WS et al. (2017) Learning fully convolutional networks for iterative non-blind Deconvolution[J]. 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 6969–6977

  50. Zhang K, Zuo W, Chen Y, Meng D, Zhang L (2017) Beyond a Gaussian Denoiser: residual learning of deep CNN for image Denoising. IEEE Trans Image Process 26:3142–3155

    Article  MathSciNet  Google Scholar 

  51. Zhang K, Zuo W, Zhang L (2018) Learning a single convolutional super-resolution network for multiple degradations. 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition, 3262–3271

  52. Zhang Y, Li K, Li K, Wang L, Zhong B, Fu Y (2018) Image super-resolution using very deep Residual Channel attention networks. ArXiv, abs/1807.02758

  53. Zhou S, Zhang J, Zuo W, Xie H, Pan J, Ren J (2019) DAVANet: stereo deblurring with view aggregation. 2019 IEEE/CVF Conference on Computer Vision and PatternRecognition (CVPR), 10988–10997

Download references

Acknowledgements

We thank Southwest Jiaotong University Photoelectric Engineering Institute for their support in this experiment. This work was supported by grants from National Natural Science Foundation of China (Grant No. 61471304).

Author information

Authors and Affiliations

  1. School of Physical Science and Technology, Southwest Jiaotong University, Chengdu, 610031, China

    Jiaqi Bao, Lin Luo, Yu Zhang, Kai Yang, Chaoyong Peng, Jianping Peng & Ran Li

Authors
  1. Jiaqi Bao
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Lin Luo
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Yu Zhang
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Kai Yang
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Chaoyong Peng
    View author publications

    You can also search for this author in PubMed Google Scholar

  6. Jianping Peng
    View author publications

    You can also search for this author in PubMed Google Scholar

  7. Ran Li
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Yu Zhang.

Additional information

Publisher’s note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Bao, J., Luo, L., Zhang, Y. et al. Half quadratic splitting method combined with convolution neural network for blind image deblurring. Multimed Tools Appl 80, 3489–3504 (2021). https://doi.org/10.1007/s11042-020-09821-6

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11042-020-09821-6

Keywords

Search

Navigation