Location via proxy:   [ UP ]  
[Report a bug]   [Manage cookies]                
Skip to main content

Advertisement

Springer Nature Link
Log in

Accuracy and generalization improvement for image quality assessment of authentic distortion by semi-supervised learning

  • Published:
Applied Intelligence Aims and scope Submit manuscript

Abstract

Image quality assessment of authentically distorted images constitutes a indispensable part of numerous computer vision tasks. Despite the substantial progress in recent years, accuracy and generalization performance is still unsatisfactory. These challenges are primarily attributed to the scarcity of labeled images. In order to increase the amount of images for training, we use semi-supervised learning to combine labeled images and specifically selected unlabeled images. In our new training paradigm, denominated Selected Data Retrain under Regularization, the selection criteria of unlabeled images is based on the supposition that an image and a certain of its patches ought to have approximate image quality scores. Unlabeled images that meets the aforementioned criteria, named as Highly Credible Unlabeled Images, mitigate the problem of scarcity, thus, improve accuracy. However generalization may be compromised due to selection procedure’s reliance on labeled images and presence of coherent variance existed between labeled images and unlabeled images. Therefore we incorporate a sorting loss function to reduce variation within the new dataset of labeled images and specifically selected unlabeled images, and thus achieve better generalization. The effectiveness of our proposed paradigm is empirically validated using public datasets. Codes are available at https://github.com/dvstter/SDRR_IQA.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Subscribe and save

Springer+ Basic
$34.99 /Month
  • Get 10 units per month
  • Download Article/Chapter or eBook
  • 1 Unit = 1 Article or 1 Chapter
  • Cancel anytime
Subscribe now

Buy Now

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Algorithm 1
Fig. 4
Fig. 5
Fig. 6
Fig. 7

Similar content being viewed by others

Explore related subjects

Discover the latest articles, news and stories from top researchers in related subjects.

Data Availability

The data generated during and/or analysed during the current study is available from the first author or corresponding author on reasonable request.

References

  1. Narvekar ND, Karam LJ (2011) A no-reference image blur metric based on the cumulative probability of blur detection (cpbd). IEEE Trans Image Process 20(9):2678–2683

    Article  MathSciNet  Google Scholar 

  2. Manap RA, Shao L (2015) Non-distortion-specific no-reference image quality assessment: a survey. Inf Sci 301:141–160

    Article  Google Scholar 

  3. Mittal A, Moorthy AK, Bovik AC (2012) No-reference image quality assessment in the spatial domain. IEEE Trans Image Process 21(12):4695–4708

    Article  MathSciNet  Google Scholar 

  4. Ghadiyaram D, Bovik AC (2015) Massive online crowdsourced study of subjective and objective picture quality. IEEE Trans Image Process 25(1):372–387

    Article  MathSciNet  Google Scholar 

  5. Hosu V, Lin H, Sziranyi T, Saupe D (2020) Koniq-10k: an ecologically valid database for deep learning of blind image quality assessment. IEEE Trans Image Process 29:4041–4056

  6. Fang Y, Zhu H, Zeng Y, Ma K, Wang Z (2020) Perceptual quality assessment of smartphone photography. In: Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pp 3677–3686

  7. Moorthy AK, Bovik AC (2010) A two-step framework for constructing blind image quality indices. IEEE Signal Process Lett 17(5):513–516

    Article  Google Scholar 

  8. Saad MA, Bovik AC, Charrier C (2010) A dct statistics-based blind image quality index. IEEE Signal Process Lett 17(6):583–586

    Article  Google Scholar 

  9. Saad MA, Bovik AC, Charrier C (2012) Blind image quality assessment: a natural scene statistics approach in the dct domain. IEEE Trans Image Process 21(8):3339–3352

    Article  MathSciNet  Google Scholar 

  10. Nizami IF, Majid M, Khurshid K (2018) New feature selection algorithms for no-reference image quality assessment. Appl Intell 48:3482–3501

    Article  Google Scholar 

  11. Luo Z, Tang Z, Jiang L, Ma G (2022) A referenceless image degradation perception method based on the under water imaging model. Appl Intell 52:6522–6538

  12. Lecun Y, Bottou L, Bengio Y, Haffner P (1998) Gradient-based learning applied to document recognition. Proc IEEE 86(11):2278–2324

    Article  Google Scholar 

  13. He K, Zhang X, Ren S, Sun J (2016) Deep residual learning for image recognition. In: 2016 IEEE Conference on computer vision and pattern recognition (CVPR), IEEE, pp 770–778

  14. Krizhevsky A, Sutskever I, Hinton GE (2017) ImageNet Classification with Deep Convolutional Neural Networks. Assoc Computing Machinery, NEW YORK

    Book  Google Scholar 

  15. Radhika T, Chandrasekar A, Vijayakumar V, Zhu Q (2023) Analysis of markovian jump stochastic cohen–grossberg bam neural networks with time delays for exponential input-to-state stability. Neural Process Lett 55(8):11055–11072

    Article  Google Scholar 

  16. Cao Y, Chandrasekar A, Radhika T, Vijayakumar V (2024) Input-to-state stability of stochastic markovian jump genetic regulatory networks. Math Comput Simul 222:174–187. https://doi.org/10.1016/j.matcom.2023.08.007. International Conference on Mathematical Modeling 2023

  17. Kang L, Ye P, Li Y, Doermann D (2014) Convolutional neural networks for no-reference image quality assessment. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 1733–1740

  18. Bosse S, Maniry D, Wiegand T, Samek W (2016) A deep neural network for image quality assessment. In: IEEE International conference on image processing, IEEE, pp 3773–3777

  19. Zhu P, Liu S, Liu Y, Yap P-T (2023) Meter: multi-task efficient transformer for no-reference image quality assessment. Appl Intell pp 1–17

  20. Zhang W, Ma K, Yan J, Deng D, Wang Z (2018) Blind image quality assessment using a deep bilinear convolutional neural network. IEEE Trans Circuits Syst Video Technol 30(1):36–47

    Article  Google Scholar 

  21. Amirshahi SA, Pedersen M, Yu SX (2016) Image quality assessment by comparing cnn features between images. J Imaging Sci Technol 60(6):60410–1

    Article  Google Scholar 

  22. Fang Y, Yan J, Liu X, Wang J (2019) Stereoscopic image quality assessment by deep convolutional neural network. J Vis Commun Image Represent 58:400–406

    Article  Google Scholar 

  23. Yang D, Peltoketo V-T, Kamarainen J-K (2019) Cnn-based cross-dataset no-reference image quality assessment. In: Proceedings of the IEEE/CVF international conference on computer vision workshops, pp 0–0

  24. You J, Korhonen J (2021) Transformer for image quality assessment. In: IEEE International conference on image processing, pp 1389–1393

  25. Cheon M, Yoon S-J, Kang B, Lee J (2021) Perceptual image quality assessment with transformers. In: Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pp 433–442

  26. Yang S, Wu T, Shi S, Lao S, Gong Y, Cao M, Wang J, Yang Y (2022) Maniqa: multi-dimension attention network for no-reference image quality assessment. In: Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pp 1191–1200

  27. Yue G, Cheng D, Li L, Zhou T, Liu H, Wang T (2022) Semi-supervised authentically distorted image quality assessment with consistency-preserving dual-branch convolutional neural network. IEEE Trans Multimed pp 6499–6511

  28. Cao Y, Wan Z, Ren D, Yan Z, Zuo W (2022) Incorporating semi-supervised and positive-unlabeled learning for boosting full reference image quality assessment. In: Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pp 5851–5861

  29. Cao J, Wang R, Jia Y, Zhang X, Wang S, Kwong S (2021) No-reference image quality assessment for contrast-changed images via a semi-supervised robust pca model. Inf Sci 574:640–652

    Article  MathSciNet  Google Scholar 

  30. Shu W, Yan Z, Yu J, Qian W (2023) Information gain-based semi-supervised feature selection for hybrid data. Appl Intell 53:7310–7325

    Article  Google Scholar 

  31. Song R, Giunchiglia F, Zhao K, Xu H (2023) Topological enhanced graph neural networks for semi-supervised node classification. Appl Intell 53:23538–23552

    Article  Google Scholar 

  32. Xiao Y, Chen C, Fu X, Wang L, Yu J, Zou Y (2023) A novel multi-task semi-supervised medical image segmentation method based on multi-branch cross pseudo supervision. Appl Intell pp 0–16

  33. Cozman FG, Cohen I, Cirelo MC et al (2003) Semi-supervised learning of mixture models. In: The international conference on machine learning, vol 4, p 24

  34. Triguero I, García S, Herrera F (2015) Self-labeled techniques for semi-supervised learning: taxonomy, software and empirical study. Knowl Inf Syst 42:245–284

    Article  Google Scholar 

  35. Triguero I, García S, Herrera F (2014) Seg-ssc: A framework based on synthetic examples generation for self-labeled semi-supervised classification. IEEE Transactions on Cybernetics 45(4):622–634

    Article  Google Scholar 

  36. Tang H, Joshi N, Kapoor A (2014) Blind image quality assessment using semi-supervised rectifier networks. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 2877–2884

  37. Wang Z, Li D, Ma K (2021) Semi-supervised deep ensembles for blind image quality assessment. arXiv preprint arXiv:2106.14008, pp 1–6

  38. Lu W, Mei N, Gao F, He L, Gao X (2015) Blind image quality assessment via semi-supervised learning and fuzzy inference. In: Applied informatics, SpringerOpen, vol 2, pp 1–20

  39. Zhao X, Li Y, Wang S (2019) Face quality assessment via semi-supervised learning. In: Proceedings of the 2019 8th international conference on computing and pattern recognition, pp 288–293

  40. Guo L-Z, Zhang Z-Y, Jiang Y, Li Y-F, Zhou Z-H (2020) Safe deep semi-supervised learning for unseen-class unlabeled data. In: International conference on machine learning, PMLR, pp 3897–3906

  41. Shah R, Kumar V (2021) Rrl: resnet as representation for reinforcement learning. arXiv preprint arXiv:2107.03380

  42. Kingma DP, Ba J (2014) Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980

  43. He K, Zhang X, Ren S, Sun J (2015) Delving deep into rectifiers: surpassing human-level performance on imagenet classification. In: Proceedings of the IEEE international conference on computer vision, pp 1026–1034

  44. Thomee B, Shamma DA, Friedland G, Elizalde B, Ni K, Poland D, Borth D, Li L-J (2016) Yfcc100m: the new data in multimedia research. Commun ACM 59(2):64–73

    Article  Google Scholar 

  45. Zhang L, Zhang L, Bovik AC (2015) A feature-enriched completely blind image quality evaluator. IEEE Trans Image Process 24(8):2579–2591

    Article  MathSciNet  Google Scholar 

  46. Mittal A, Soundararajan R, Bovik AC (2012) Making a “completely blind’’ image quality analyzer. IEEE Signal Process Lett 20(3):209–212

    Article  Google Scholar 

  47. Li Q, Lin W, Fang Y (2016) No-reference quality assessment for multiply-distorted images in gradient domain. IEEE Signal Process Lett 23(4):541–545

    Article  Google Scholar 

  48. Liu L, Liu B, Huang H, Bovik AC (2014) No-reference image quality assessment based on spatial and spectral entropies. Signal processing: Image Communication 29(8):856–863

    Google Scholar 

  49. Gu K, Zhai G, Yang X, Zhang W (2014) Using free energy principle for blind image quality assessment. IEEE Trans Multimedia 17(1):50–63

    Article  Google Scholar 

  50. Gu K, Zhai G, Liu M, Yang X, Zhang W, Su X, Chen W, Zuo Y (2013) Fisblim: a five-step blind metric for quality assessment of multiply distorted images. In: IEEE Workshop on signal processing systems, IEEE, pp 241–246

  51. Ye P, Kuma J, Kang L, Doermann D (2012) Unsupervised feature learning framework for no-reference image quality assessment. In: 2012 IEEE Conference on computer vision and pattern recognition, IEEE, pp 1098–1105

  52. Bosse S, Maniry D, Müller K-R, Wiegand T, Samek W (2017) Deep neural networks for no-reference and full-reference image quality assessment. IEEE Trans Image Process 27(1):206–219

  53. Ying Z, Niu H, Gupta P, Mahajan D, Ghadiyaram D, Bovik A (2020) From patches to pictures (paq-2-piq): mapping the perceptual space of picture quality. In: Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pp 3575–3585

  54. Zhu H, Li L, Wu J, Dong W, Shi G (2020) Metaiqa: deep meta-learning for no-reference image quality assessment. In: Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pp 14143–14152

  55. Zheng B, Yuan S, Slabaugh G, Leonardis A (2020) Image demoireing with learnable bandpass filters. In: Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pp 3636–3645

  56. Sun S, Yu T, Xu J, Zhou W, Chen Z (2022) Graphiqa: learning distortion graph representations for blind image quality assessment. IEEE Trans Multimed

Download references

Funding

No funding involved.

Author information

Authors and Affiliations

  1. Institute of Fundamental and Frontier Sciences, University of Electronic Science and Technology of China, Chengdu, 610054, Sichuan Province, China

    Hanlin Yang & William Zhu

  2. College of Computer and Data Science, Fuzhou University, Fuzhou, 350116, Fujian Province, China

    Shiping Wang

Authors
  1. Hanlin Yang
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. William Zhu
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Shiping Wang
    View author publications

    You can also search for this author in PubMed Google Scholar

Contributions

Hanlin Yang designed the framework and network architecture, carried out the implementation, performed the experiments and analysed the data. Hanlin Yang and William Zhu wrote the manuscript. Hanlin Yang, William Zhu and Shiping Wang revised the manuscript. Shiping Wang conceived the study and were in charge of overall direction and planning.

Corresponding author

Correspondence to Shiping Wang.

Ethics declarations

Competing of Interest

The authors have no competing interests to declare that are relevant to the content of this article.

Ethical and Informed Consent for Data Used

As all data utilized in this study is sourced from publicly available datasets and academic papers, ethical and informed consent is not applicable.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Yang, H., Zhu, W. & Wang, S. Accuracy and generalization improvement for image quality assessment of authentic distortion by semi-supervised learning. Appl Intell 54, 10948–10961 (2024). https://doi.org/10.1007/s10489-024-05790-7

Download citation

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s10489-024-05790-7

Keywords

Search

Navigation