Location via proxy:   [ UP ]  
[Report a bug]   [Manage cookies]                
Skip to main content
Springer Nature Link
Log in

Impact of visual saliency on multi-distorted blind image quality assessment using deep neural architecture

  • Published:
Multimedia Tools and Applications Aims and scope Submit manuscript

    We’re sorry, something doesn't seem to be working properly.

    Please try refreshing the page. If that doesn't work, please contact support so we can address the problem.

A Correction to this article was published on 03 May 2022

This article has been updated

Abstract

No-referenceimage quality assessment (NR-IQA) techniques try to assess the quality of images without anyinformation regarding the pristine version of the image. NR-IQA becomes more challenging for images affected by multiple distortions and images taken in real-world scenarios. Most convolutional neural network (CNN) based NR-IQA techniques do not use visual saliency (VS) and the ones that do require a reference image for computation of VS map. This paper proposes a completely blind end-to-end NR-IQA methodology based on VS and convolutional neural network (CNN). It utilizes VS models to identify the significant region of the image that can be used for quality assessment. The VS methodology used in the proposed approach does not require a reference image for the extraction of VS map. The regions proposed by the VS model are then utilized by a CNN to predict the quality score. The proposed methodology is tested on three publicly available multi-distorted image quality assessment databases i.e., live multiply distorted image quality (MDIQ) database, multiply distorted image database (MDID), and the live in the wild challenge (CLIVE) database. The proposed methodology is also tested over three VS models extract VS maps without the reference image. The experimental results show that using VS models improve the performance of CNN for predicting the image quality score.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Subscribe and save

Springer+ Basic
$34.99 /Month
  • Get 10 units per month
  • Download Article/Chapter or eBook
  • 1 Unit = 1 Article or 1 Chapter
  • Cancel anytime
Subscribe now

Buy Now

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6

Similar content being viewed by others

Change history

References

  1. Bianco S, Celona L, Napoletano P, Schettini R (2018) On the use of deep learning for blind image quality assessment. Signal Image Vid Process 12 (2):355–362

    Article  Google Scholar 

  2. Bosse S, Maniry D, Müller K-R, Wiegand T, Samek W (2017) Deep neural networks for no-reference and full-reference image quality assessment. IEEE Trans Image Process 27(1):206–219

    Article  MathSciNet  MATH  Google Scholar 

  3. Bosse S, Maniry D, Müller K-R, Wiegand T, Samek W (2018) Deep neural networks for no-reference and full-reference image quality assessment. IEEE Trans Image Process 27(1):206–219

    Article  MathSciNet  MATH  Google Scholar 

  4. Canziani A, Paszke A, Culurciello E An analysis of deep neural network models for practical applications, arXiv:1605.07678

  5. Chatfield K, Simonyan K, Vedaldi A, Zisserman A Return of the devil in the details: delving deep into convolutional nets. arXiv:1405.3531

  6. Deng J, Dong W, Socher R, Li L-J, Li K, Fei-Fei L (2009) Imagenet: a large-scale hierarchical image database. In: 2009 IEEE Conference on computer vision and pattern recognition, 2009. CVPR. IEEE, pp 248–255

  7. Doermann D, Kang L, Kumar J, Ye P (2012) Unsupervised feature learning framework for no-reference image quality assessment. In: 2012 IEEE Conference on computer vision and pattern recognition. IEEE, pp 1098–1105

  8. Freitas PG, Akamine WY, Farias MC (2018) No-reference image quality assessment using orthogonal color planes patterns. IEEE Trans Multimed 20(12):3353–3360

    Article  Google Scholar 

  9. Gao F, Wang Y, Li P, Tan M, Yu J, Zhu Y (2017) Deepsim: deep similarity for image quality assessment. Neurocomputing 257:104–114

    Article  Google Scholar 

  10. Ghadiyaram D, Bovik AC (2016) Massive online crowdsourced study of subjective and objective picture quality. IEEE Trans Image Process 25(1):372–387

    Article  MathSciNet  MATH  Google Scholar 

  11. Golestaneh S, Karam LJ (2016) Reduced-reference quality assessment based on the entropy of dwt coefficients of locally weighted gradient magnitudes. IEEE Trans Image Process 25(11):5293–5303

    Article  MathSciNet  MATH  Google Scholar 

  12. Gu K, Zhai G, Lin W, Yang X, Zhang W (2015) No-reference image sharpness assessment in autoregressive parameter space. IEEE Trans Image Process 24(10):3218–3231

    Article  MathSciNet  MATH  Google Scholar 

  13. Guan J, Yi S, Zeng X, Cham W-K, Wang X (2017) Visual importance and distortion guided deep image quality assessment framework. IEEE Trans Multimed 19(11):2505–2520

    Article  Google Scholar 

  14. Guan J, Zhang W, Gu J, Ren H (2015) No-reference blur assessment based on edge modeling. J Vis Commun Image Represent 29:1–7

    Article  Google Scholar 

  15. Harel J, Koch C, Perona P (2007) Graph-based visual saliency. In: Advances in neural information processing systems, pp 545–552

  16. Hou W, Gao X, Tao D, Li X (2015) Blind image quality assessment via deep learning. IEEE Trans Neural Netw Learn Syst 26(6):1275–1286

    Article  MathSciNet  Google Scholar 

  17. Hou X, Zhang L (2007) Saliency detection: a spectral residual approach. In: 2007 IEEE Conference on computer vision and pattern recognition. IEEE, pp 1–8

  18. Itti L, Koch C, Niebur E (1998) A model of saliency-based visual attention for rapid scene analysis. IEEE Trans Pattern Anal Mach Intell 20(11):1254–1259

    Article  Google Scholar 

  19. Jayaraman D, Mittal A, Moorthy AK, Bovik AC (2012) Objective quality assessment of multiply distorted images. In: 2012 Conference record of the forty sixth asilomar conference on signals, systems and computers (ASILOMAR). IEEE, pp 1693–1697

  20. Jiang B, He J, Yang S, Fu H, Li T, Song H, He D (2019) Fusion of machine vision technology and alexnet-cnns deep learning network for the detection of postharvest apple pesticide residues. Artif Intell Agri 1:1–8

    Google Scholar 

  21. Kang L, Ye P, Li Y, Doermann D (2014) Convolutional neural networks for no-reference image quality assessment. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 1733–1740

  22. Kim J, Lee S (2017) Fully deep blind image quality predictor. IEEE J Selected Topics Signal Process 11(1):206–220

    Article  Google Scholar 

  23. Kim J, Lee S (2017) Fully deep blind image quality predictor. IEEE J Selected Topics Signal Process 11(1):206–220

    Article  Google Scholar 

  24. Kim J, Zeng H, Ghadiyaram D, Lee S, Zhang L, Bovik AC (2017) Deep convolutional neural models for picture-quality prediction: Challenges and solutions to data-driven image quality assessment. IEEE Signal Process Mag 34 (6):130–141

    Article  Google Scholar 

  25. Krizhevsky A, Sutskever I, Hinton GE (2012) Imagenet classification with deep convolutional neural networks. In: Advances in neural information processing systems, pp 1097–1105

  26. Li S, Zhang F, Ma L, Ngan KN (2011) Image quality assessment by separately evaluating detail losses and additive impairments. IEEE Trans Multimed 13(5):935–949

    Article  Google Scholar 

  27. Li L, Lin W, Wang X, Yang G, Bahrami K, Kot AC (2016) No-reference image blur assessment based on discrete orthogonal moments. IEEE Trans Cybern 46(1):39–50

    Article  Google Scholar 

  28. Li S, Ren W, Wang F, Araujo IB, Tokuda EK, Junior RH, Cesar-Jr RM, Wang Z, Cao X (2021) A comprehensive benchmark analysis of single image deraining: current challenges and future perspectives. Int J Comput Vis 129(4):1301–1322

    Article  Google Scholar 

  29. Lin K-Y, Wang G (2018) Hallucinated-iqa: no-reference image quality assessment via adversarial learning. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 732–741

  30. Liu A, Lin W, Narwaria M (2012) Image quality assessment based on gradient similarity. IEEE Trans Image Process 21(4):1500–1512

    Article  MathSciNet  MATH  Google Scholar 

  31. Liu L, Hua Y, Zhao Q, Huang H, Bovik AC (2016) Blind image quality assessment by relative gradient statistics and adaboosting neural network. Signal Process Image Commun 40:1–15

    Article  Google Scholar 

  32. Liu H, Klomp N, Heynderickx I (2010) A no-reference metric for perceived ringing artifacts in images. IEEE Trans Circ Syst Video Technol 20(4):529–539

    Article  Google Scholar 

  33. Liu T-J, Liu K-H (2018) No-reference image quality assessment by wide-perceptual-domain scorer ensemble method. IEEE Trans Image Process 27(3):1138–1151

    Article  MathSciNet  MATH  Google Scholar 

  34. Liu L, Liu B, Huang H, Bovik AC (2014) No-reference image quality assessment based on spatial and spectral entropies. Signal Process Image Commun 29(8):856–863

    Article  Google Scholar 

  35. Lu Y, Li W, Ning X, Dong X, Zhang L, Sun L, Cheng C (2021) Blind image quality assessment based on the multiscale and dual-domains features fusion. Concurrency and Computation: Practice and Experience, e6177

  36. Ma X, Jiang X (2020) Multimedia image quality assessment based on deep feature extraction. Multimed Tools Applic 79(47):35209–35220

    Article  Google Scholar 

  37. Ma L, Li S, Zhang F, Ngan KN (2011) Reduced-reference image quality assessment using reorganized dct-based image representation. IEEE Trans Multimed 13(4):824–829

    Article  Google Scholar 

  38. Ma K, Liu W, Liu T, Wang Z, Tao D (2017) dipiq: blind image quality assessment by learning-to-rank discriminable image pairs. IEEE Trans Image Process 26(8):3951–3964

    Article  MathSciNet  MATH  Google Scholar 

  39. Ma K, Liu W, Zhang K, Duanmu Z, Wang Z, Zuo W (2017) End-to-end blind image quality assessment using deep neural networks. IEEE Trans Image Process 27(3):1202–1213

    Article  MathSciNet  MATH  Google Scholar 

  40. Ma K, Liu W, Zhang K, Duanmu Z, Wang Z, Zuo W (2018) End-to-end blind image quality assessment using deep neural networks. IEEE Trans Image Process 27(3):1202–1213

    Article  MathSciNet  MATH  Google Scholar 

  41. Min X, Gu K, Zhai G, Liu J, Yang X, Chen CW (2018) Blind quality assessment based on pseudo-reference image. IEEE Trans Multimed 20 (8):2049–2062

    Article  Google Scholar 

  42. Min X, Zhai G, Gu K, Liu Y, Yang X (2018) Blind image quality estimation via distortion aggravation. IEEE Trans Broadcast 64(2):508–517

    Article  Google Scholar 

  43. Mittal A, Moorthy AK, Bovik AC (2012) No-reference image quality assessment in the spatial domain. IEEE Trans Image Process 21(12):4695–4708

    Article  MathSciNet  MATH  Google Scholar 

  44. Mittal A, Soundararajan R, Bovik AC (2013) Making a “completely blind” image quality analyzer. IEEE Signal Process Lett 20(3):209–212

    Article  Google Scholar 

  45. Mittal A, Soundararajan R, Bovik AC (2013) Making a “completely blind” image quality analyzer. IEEE Signal Process Lett 20(3):209–212

    Article  Google Scholar 

  46. Moorthy AK, Bovik AC (2010) A two-step framework for constructing blind image quality indices. IEEE Signal Process Lett 17(5):513–516

    Article  Google Scholar 

  47. Moorthy AK, Bovik AC (2011) Blind image quality assessment: from natural scene statistics to perceptual quality. IEEE Trans Image Process 20 (12):3350–3364

    Article  MathSciNet  MATH  Google Scholar 

  48. Moorthy AK, Bovik AC (2011) Blind image quality assessment: from natural scene statistics to perceptual quality. IEEE Trans Image Process 20 (12):3350–3364

    Article  MathSciNet  MATH  Google Scholar 

  49. Oszust M (2017) No-reference image quality assessment using image statistics and robust feature descriptors. IEEE Signal Process Lett 24(11):1656–1660

    Article  Google Scholar 

  50. Pan F, Lin X, Rahardja S, Lin W, Ong E, Yao S, Lu Z, Yang X (2004) A locally-adaptive algorithm for measuring blocking artifacts in images and videos. In: 2004 IEEE International symposium on circuits and systems (IEEE Cat. No. 04CH37512), vol 3. IEEE, pp III–925

  51. Reisenhofer R, Bosse S, Kutyniok G, Wiegand T (2018) A haar wavelet-based perceptual similarity index for image quality assessment. Signal Process Image Commun 61:33–43

    Article  Google Scholar 

  52. Rehman A, Wang Z (2012) Reduced-reference image quality assessment by structural similarity estimation. IEEE Trans Image Process 21(8):3378–3389

    Article  MathSciNet  MATH  Google Scholar 

  53. Saad MA, Bovik AC, Charrier C (2012) Blind image quality assessment: a natural scene statistics approach in the dct domain. IEEE Trans Image Process 21(8):3339–3352

    Article  MathSciNet  MATH  Google Scholar 

  54. Saad MA, Bovik AC, Charrier C (2012) Blind image quality assessment: a natural scene statistics approach in the dct domain. IEEE Trans Image Process 21(8):3339–3352

    Article  MathSciNet  MATH  Google Scholar 

  55. Sun W, Zhou F, Liao Q (2017) Mdid: a multiply distorted image database for image quality assessment. Pattern Recogn 61:153–168

    Article  Google Scholar 

  56. Talebi H, Milanfar P (2018) Nima: neural image assessment. IEEE Trans Image Process 27(8):3998–4011

    Article  MathSciNet  MATH  Google Scholar 

  57. Wang Z, Li Q (2011) Information content weighting for perceptual image quality assessment. IEEE Trans Image Process 20(5):1185–1198

    Article  MathSciNet  MATH  Google Scholar 

  58. Wang Z, Bovik AC, Sheikh HR, Simoncelli EP (2004) Image quality assessment: from error visibility to structural similarity. IEEE Trans Image Process 13(4):600–612

    Article  Google Scholar 

  59. Wu Q, Li H, Meng F, Ngan KN, Luo B, Huang C, Zeng B (2016) Blind image quality assessment based on multichannel feature fusion and label transfer. IEEE Trans Circ Syst Video Technol 26(3):425– 440

    Article  Google Scholar 

  60. Wu Q, Li H, Ngan KN, Ma K (2018) Blind image quality assessment using local consistency aware retriever and uncertainty aware evaluator. IEEE Trans Circ Syst Video Technol 28(9):2078–2089

    Article  Google Scholar 

  61. Xue W, Zhang L, Mou X, Bovik AC (2014) Gradient magnitude similarity deviation: a highly efficient perceptual image quality index. IEEE Trans Image Process 23(2):684–695

    Article  MathSciNet  MATH  Google Scholar 

  62. Xue W, Mou X, Zhang L, Bovik AC, Feng X (2014) Blind image quality assessment using joint statistics of gradient magnitude and Laplacian features. IEEE Trans Image Process 23(11):4850–4862

    Article  MathSciNet  MATH  Google Scholar 

  63. Yang X, Li F, Liu H (2020) Deep feature importance awareness based no-reference image quality prediction. Neurocomputing 401:209–223

    Article  Google Scholar 

  64. Ye P, Kumar J, Kang L, Doermann D (2012) Unsupervised feature learning framework for no-reference image quality assessment. In: 2012 IEEE conference on computer vision and pattern recognition. IEEE, pp 1098–1105

  65. Zhang L, Tong MH, Marks TK, Shan H, Cottrell GW (2008) Sun: a Bayesian framework for saliency using natural statistics. J Vis 8(7):32–32

    Article  Google Scholar 

  66. Zhang L, Zhang L, Mou X, Zhang D, et al. (2011) Fsim: a feature similarity index for image quality assessment. IEEE Trans Image Process 20(8):2378–2386

    Article  MathSciNet  MATH  Google Scholar 

  67. Zhang L, Shen Y, Li H (2014) Vsi: a visual saliency-induced index for perceptual image quality assessment. IEEE Trans Image Process 23(10):4270–4281

    Article  MathSciNet  MATH  Google Scholar 

  68. Zhang W, Ma K, Yan J, Deng D, Wang Z Blind image quality assessment using a deep bilinear convolutional neural network. IEEE Transactions on Circuits and Systems for Video Technology

Download references

Author information

Authors and Affiliations

  1. Department of Electrical Engineering, Bahria University, Islamabad, Pakistan

    Imran Fareed Nizami & Asad Waqar

  2. Department of Electronics and Information Engineering, Jeonbuk National University, Jeonju, 54896, Korea

    Mobeen ur Rehman

  3. Institute of Avionics and Aeronautics (IAA), Air University, Islamabad, 44000, Pakistan

    Mobeen ur Rehman

  4. Department of Computer Engineering, University of Engineering and Technology, Taxila, Pakistan

    Muhammad Majid

Authors
  1. Imran Fareed Nizami
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Mobeen ur Rehman
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Asad Waqar
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Muhammad Majid
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Imran Fareed Nizami.

Additional information

Publisher’s note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

The original online version of this article was revised: The affiliations were incorrectly assigned to all authors.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Nizami, I.F., Rehman, M.u., Waqar, A. et al. Impact of visual saliency on multi-distorted blind image quality assessment using deep neural architecture. Multimed Tools Appl 81, 25283–25300 (2022). https://doi.org/10.1007/s11042-022-12060-6

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11042-022-12060-6

Keywords

Search

Navigation