Location via proxy:   [ UP ]  
[Report a bug]   [Manage cookies]                
Skip to main content

Advertisement

Springer Nature Link
Log in

NNCD-IQA: A new neural networks based compressed database for image quality assessment

  • Published:
Multimedia Tools and Applications Aims and scope Submit manuscript

Abstract

Objective and subjective quality assessment is still a challenging problem in various image processing tasks. For instance, in the context of image compression, most of the conducted studies have focused on image datasets encoded using standard algorithms such as JPEG and JPEG2000. In this paper, we propose to further investigate the quality assessment issue in the presence of neural networks-based compressed images. More precisely, a new database of compressed images has been firstly built using JPEG2000 standard as well as four recent neural networks based coding schemes. Then, subjective experiments are performed to obtain the mean opinion scores of the generated distorted images. Finally, an extensive evaluation and analysis of objective image quality assessment metrics is achieved. For instance, in addition to conventional and machine learning metrics, we have considered different deep learning based models, which have been trained on our database. The new subjective database with its associated mean opinion scores as well as the learned models are publicly available at https://github.com/zakopz/NNCD-IQA-Database. The obtained results show the interest of deep learning based metrics in the context of neural networks-based compressed images.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Subscribe and save

Springer+ Basic
$34.99 /Month
  • Get 10 units per month
  • Download Article/Chapter or eBook
  • 1 Unit = 1 Article or 1 Chapter
  • Cancel anytime
Subscribe now

Buy Now

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11

Similar content being viewed by others

References

  1. Agustsson E, Tschannen M, Mentzer F, Timofte R, Luc VG (2019) Generative adversarial networks for extreme learned image compression. In: International conference on learning representations. Louisiana, USA, pp 1–31

  2. Ahanonu E, Marcellin M, Bilgin A (2018) Lossless image compression using reversible integer wavelet transforms and convolutional neural networks. In: Data compression conference. UT, USA, p 1

  3. Amirkhani D, Bastanfard A (2021) An objective method to evaluate exemplar-based inpainted images quality using Jaccard index. Multimed Tools Appl 80:26199–26212

    Article  Google Scholar 

  4. Ballé J, Laparra V, Simoncelli EP (2017) End-to-end optimized image compression. In: International conference on learning representations. Toulon, France, pp 1–27

  5. Ballé J, Minnen D, Singh S, Hwang SJ, Johnston N (2018) Variational image compression with a scale hyperprior. In: International conference on learning representations. Vancouver, Canada, pp 1–47

  6. Bosse S, Maniry D, Muller KR, Wiegand T, Samek W (2018) Deep neural networks for no-reference and full-reference image quality assessment. IEEE Trans Image Process 27(1):206–219

    Article  MathSciNet  MATH  Google Scholar 

  7. Cheng Z, Akyazi P, Sun H, Katto J, Ebrahimi T (2019) Perceptual quality study on deep learning based image compression. In: IEEE International conference on image processing. Taipei, Taiwan, pp 719–723

  8. Cheng Z, Sun H, Takeuchi M, Katto J (2020) Learned image compression with discretized gaussian mixture likelihoods and attention modules. In: IEEE International conference on computer vision and pattern recognition, pp 7936–7945

  9. Corchs S, Gasparini F, Schettini R (2014) No reference image quality classification for JPEG-distorted images. Digit Signal Process 30:86–100

    Article  Google Scholar 

  10. Dardouri T, Kaaniche M, Benazza-Benyahia A, Pesquet JC (2020) Optimized lifting scheme based on a dynamical fully connected network for image coding. In: International conference on image processing. Abu Dhabi, United Arab Emirates, pp 1–5

  11. Dardouri T, Kaaniche M, Benazza-Benyahia A, Pesquet JC (2021) Dynamic neural network for lossy-to-lossless image coding. IEEE Trans Image Process 31:569–584

    Article  Google Scholar 

  12. Ding K, Ma K, Wang S, Simoncelli EP (2022) Image quality assessment: Unifying structure and texture similarity. IEEE Trans Pattern Anal Mach Intell 44(5):2567–2581

    Google Scholar 

  13. Dumas T, Roumy A, Guillemot C (2019) Context-adaptive neural network-based prediction for image compression. IEEE Trans Image Process 29(1):679–693

    MathSciNet  MATH  Google Scholar 

  14. Garcia-Lucas D, Cebrian-Márquez G, Cuenca P (2020) Rate-distortion/complexity analysis of HEVC, VVC and AV1 video codecs. Multimed Tools Appl 79:29621–29638

    Article  Google Scholar 

  15. Hajihashemi V, Najafabadi HE, Gharahbagh AA, Leung H, Yousefan M, Tavares JMRS (2021) A novel high-efficiency holography image compression method, based on HEVC, wavelet, and nearest-neighbor interpolation. Multimed Tools Appl 80:31953–31966

    Article  Google Scholar 

  16. Hammou D, Fezza SA, Hamidouche W (2021) EGB: Image Quality assessment based on ensemble of gradient boosting. In: IEEE International conference on computer vision and pattern recognition, pp 541–549

  17. Methodology for the subjective assessment of the quality of television pictures. Recommendation, ITU-R BT. 500–13 (2012)

  18. Jin L, Lin JY, Hu S, Wang H, Wang P, Katsavounidis I, Aaron A, Kuo CCJ (2016) Statistical study on perceived JPEG image quality via MCL-JCI dataset construction and analysis. Electron Imaging 2016(13):1–9

    Article  Google Scholar 

  19. Kaaniche M, Benazza-Benyahia A, Pesquet-Popescu B, Pesquet JC (2011) Non separable lifting scheme with adaptive update step for still and stereo image coding. Elsevier Signal Processing: Special issue on Advances in Multirate Filter Bank Structures and Multiscale Representations 91(12):2767–2782

    Article  MATH  Google Scholar 

  20. Kim J, Lee S (2017) Deep learning of human visual sensitivity in image quality assessment framework. In: IEEE Conference on computer vision and pattern recognition, pp 1676–1684

  21. Kim J, Lee S (2017) Fully deep blind image quality predictor. IEEE J Select Top Signal Process 11(1):206–220

    Article  Google Scholar 

  22. Korshunov P, Hanhart P, Richter T, Artusi A, Mantiuk R, Ebrahimi T (2015) Subjective quality assessment database of HDR images compressed with JPEG XT. In: International conference on quality of multimedia experience (qoMEX), pp 1–6

  23. Laparra V, Ballé J, Berardino A, Simoncelli EP (2016) Perceptual image quality assessment using a normalized laplacian pyramid. Electron Imaging 2016(16):1–6

    Article  Google Scholar 

  24. Larson EC, Chandler DM (2010) Most apparent distortion: full-reference image quality assessment and the role of strategy. J Electron Imaging 19(1):011006

    Article  Google Scholar 

  25. Lee J, Cho S, Beack SK (2019) Context adaptive entropy model for end-to-end optimized image compression. In: International conference on learning representations. Louisiana, USA, pp 1–20

  26. Li J, Li B, Xu J, Xiong R, Gao W (2018) Fully connected network-based intra prediction for image coding. IEEE Trans Image Process 27(7):3236–3247

    Article  MathSciNet  MATH  Google Scholar 

  27. Li M, Zuo W, Gu S, Zhao D, Zhang D (2018) Learning convolutional networks for content-weighted image compression. In: IEEE International conference on computer vision and pattern recognition. UT, USA, pp 3214–3223

  28. Liu D, Li Y, Lin J, Li H, Wu F (2020) Deep learning-based video coding: a review and a case study. ACM Comput Surv 53(1):1–35

    Article  Google Scholar 

  29. Liu D, Ma H, Xiong Z, Wu F (2018) CNN-Based DCT-like transform for image compression. In: International conference on multimedia modeling. Bangkok, Thailand, pp 61–72

  30. Ma H, Liu D, Xiong R, Wu F (2020) iWave: CNN-based wavelet-like transform for image compression. IEEE Trans Multimedia 22(7):1667–1679

    Article  Google Scholar 

  31. Ma S, Zhang X, Jia C, Zhao Z, Wang S, Wang S (2020) Image and video compression with neural networks: a review. IEEE Trans Circuits Syst Video Technol 30(6):1683–1698

    Article  Google Scholar 

  32. Minnen D, Toderici G, Covell M, Chinen T, Johnston N, Shor J, Hwang SJ, Vincent D, Singh S (2017) Spatially adaptive image compression using a tiled deep network. In: IEEE International conference on image processing. Beijing, China, pp 1–5

  33. Mittal A, Moorthy AK, Bovik AC (2012) No-reference image quality assessmentin the spatial domain. IEEE Trans Image Process 21(12):4695–4708

    Article  MathSciNet  MATH  Google Scholar 

  34. Mittal A, Soundararajan R, Bovik AC (2013) Making a completely blind image quality analyzer. IEEE Signal Process Lett 20(3):909–912

    Article  Google Scholar 

  35. Moorthy AK, Bovik AC (2011) Blind image quality assessment: From natural scene statistics to perceptual quality. IEEE Trans Image Process 20 (12):3350–3364

    Article  MathSciNet  MATH  Google Scholar 

  36. Ni Z, Zeng H, Ma L, Hou J, Chen J, Ma KK (2018) A gabor feature-based quality assessment model for the screen content images. IEEE Trans Image Process 27(9):4516–4528

    Article  MathSciNet  MATH  Google Scholar 

  37. Ponomarenko N, Jin L, Ieremeiev O, Lukin V, Egiazarian K, Astola J, Vozel B, Chehdi K, Carli M, Battisti F et al (2015) Image database TID2013: peculiarities, results and perspectives. Signal Processing: Image communication 30:57–77

    Google Scholar 

  38. Ponomarenko N, Lukin V, Zelensky A, Egiazarian K, Carli M, Battisti F (2009) TID2008-A database for evaluation of full-reference visual quality assessment metrics. Adv Mod Radioelectron 10(4):30–45

    Google Scholar 

  39. Prashnani E, Cai H, Mostofi Y, Sen P (2018) PieAPP: Perceptual image-error assessment through pairwise preference. In: IEEE Conference on computer vision and pattern recognition. UT, USA, pp 1808–1817

  40. Rippel O, Bourdev L (2017) Real-time adaptive image compression. In: International conference on machine learning. Sydney, Australia, pp 1–9

  41. Saad MA, Bovik AC, Charrier C (2012) Blind image quality assessment: a natural scene statistics approach in the DCT domain. IEEE Trans Image Process 21(8):3339–3352

    Article  MathSciNet  MATH  Google Scholar 

  42. Schiopu I, Munteanu A (2018) Macro-pixel prediction based on convolutional neural networks for lossless compression of light field images. In: International conference on image processing. Athens, Greece, pp 445–449

  43. Sheikh HR, Bovik AC (2006) Image information and visual quality. IEEE Trans Image Process 15(2):430–444

    Article  Google Scholar 

  44. Sheikh HR, Bovik AC, Cormack L (2005) No-reference quality assessment using natural scene statistics: JPEG2000. IEEE Trans Image Process 14 (11):1918–1927

    Article  Google Scholar 

  45. Sheikh HR, Sabir MF, Bovik AC (2006) A statistical evaluation of recent full reference image quality assessment algorithms. IEEE Trans Image Process 15(11):3440–3451

    Article  Google Scholar 

  46. Shi S, Bai Q, Cao M, Xia W, Wang J, Chen Y, Yang Y (2018) Region-adaptive deformable network for image quality assessment. In: IEEE Conference on computer vision and pattern recognition, pp 324–333

  47. Sun H, Cheng Z, Takeuchi M, Katto J (2020) End-to-end learned image compression with fixed point weight quantization. In: International conference on image processing. Abu Dhabi, United Arab Emirates, pp 1–5

  48. Sun W, Zhou F, Liao Q (2017) MDID: A multiply distorted image database for image quality assessment. Pattern Recogn 61:153–168

    Article  Google Scholar 

  49. Sweldens W (1996) The lifting scheme: a custom-design construction of biorthogonal wavelets. Appl Comput Harmon Anal 3(2):186–200

    Article  MathSciNet  MATH  Google Scholar 

  50. Taubman D (2000) High performance scalable image compression with EBCOT. IEEE Trans Image Process 9(7):1158–1170

    Article  MathSciNet  Google Scholar 

  51. Taubman D, Marcellin M (2002) JPEG2000: Image Compression fundamentals, standards and practice. Kluwer academic publishers, norwell, MA USA

  52. Testolina M, Upenik E, Ascenso J, Pereira F, Ebrahimi T (2021) Performance evaluation of objective image quality metrics on conventional and learning-based compression artifacts. In: International conference on quality of multimedia experience (qoMEX), pp 1–6

  53. Theis L, Shi W, Cunningham A, Huszar F (2017) Lossy image compression with compressive autoencoders. In: International conference on learning representation. Toulon, France, pp 1–19

  54. Toderici G, Vincent D, Johnston N, Hwang SJ, Minnen D, Shor J, Covell M (2017) Full resolution image compression with recurrent neural networks. In: IEEE International conference on computer vision and pattern recognition. Hawai, USA, pp 5306–5314

  55. Valenzise G, Purica A, Hulusic V, Cagnazzo M (2018) Quality assessment of deep learning-based image compression. In: International workshop on multimedia signal processing. Vancouver, Canada, pp 1–6

  56. Wang Z, Bovik AC, Sheikh HR, Simoncelli EP (2004) Image quality assessment: from error visibility to structural similarity. IEEE Trans Image Process 13(4):600–612

    Article  Google Scholar 

  57. Wang Z, Sheikh HR, Bovik AC (2002) No-reference perceptual quality assessment of JPEG compressed images. In: IEEE International conference on image processing. Rochester, USA, pp 477–480

  58. Wang Z, Wang W, Li Z, Wang J, Lin W (2012) No-reference image quality assessment for compressed images based on DCT coefficient distribution and PSNR estimation. In: Asia pacific signal and information processing association annual summit and conference. CA, USA, pp 1–4

  59. Zhang C, Cheng W, Hirakawa K (2019) Corrupted reference image quality assessment of denoised images. IEEE Trans Image Process 28(4):1732–1747

    Article  MathSciNet  Google Scholar 

  60. Zhang X, Lin W, Wang S, Liu J, Ma S, Gao W (2018) Fine-grained quality assessment for compressed images. IEEE Trans Image Process 28 (3):1163–1175

    Article  MathSciNet  Google Scholar 

  61. Zhang X, Lin W, Wang S, Liu J, Ma S, Gao W (2019) Fine-grained quality assessment for compressed images. IEEE Trans Image Process 23 (3):1163–1175

    Article  MathSciNet  Google Scholar 

Download references

Funding

No funds, grants, or other support was received.

Author information

Author notes

    Authors and Affiliations

    1. L2S, CentraleSupélec, Université Paris Saclay, 91190, Gif-sur-Yvette, France

      Zohaib Amjad Khan

    2. L2TI, UR 3043, Université Sorbonne Paris Nord, F-93430, Villetaneuse, France

      Tassnim Dardouri, Mounir Kaaniche & Gabriel Dauphin

    Authors
    1. Zohaib Amjad Khan
      View author publications

      You can also search for this author in PubMed Google Scholar

    2. Tassnim Dardouri
      View author publications

      You can also search for this author in PubMed Google Scholar

    3. Mounir Kaaniche
      View author publications

      You can also search for this author in PubMed Google Scholar

    4. Gabriel Dauphin
      View author publications

      You can also search for this author in PubMed Google Scholar

    Corresponding author

    Correspondence to Tassnim Dardouri.

    Ethics declarations

    Conflict of Interests

    The authors have no competing interests to declare that are relevant to the content of this article.

    Additional information

    Publisher’s note

    Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

    Zohaib Amjad Khan and Tassnim Dardouri contributed equally.

    Rights and permissions

    Springer Nature or its licensor holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.

    Reprints and permissions

    About this article

    Check for updates. Verify currency and authenticity via CrossMark

    Cite this article

    Khan, Z.A., Dardouri, T., Kaaniche, M. et al. NNCD-IQA: A new neural networks based compressed database for image quality assessment. Multimed Tools Appl 82, 13951–13971 (2023). https://doi.org/10.1007/s11042-022-13842-8

    Download citation

    • Received:

    • Revised:

    • Accepted:

    • Published:

    • Issue Date:

    • DOI: https://doi.org/10.1007/s11042-022-13842-8

    Keywords

    Search

    Navigation