research-article

A genetic programming-based convolutional neural network for image quality evaluations

Authors:

Huimin JiangAuthors Info & Claims

Neural Computing and Applications, Volume 34, Issue 18

Pages 15409 - 15427

https://doi.org/10.1007/s00521-022-07218-0

Published: 01 September 2022 Publication History

Abstract

Monitoring the perceptual quality of digital images is fundamentally important since digital image transmissions through the Internet continue to increase exponentially. Many automatic image quality evaluation (IQE) metrics have been developed based on image features correlated to image distortions; however, those metrics are only effective on particular image distortion types. In recent years, convolutional neural network (CNNs) have been developed for IQEs. These CNNs first capture image features from distorted images; image qualities are predicted based on the captured image features. Since the CNN weights are randomly initialized and are updated with respect to a loss function, image features which are strongly correlated to image quality are not guaranteed to be captured. In this paper, a hybrid deep neural network (DNN) is proposed by integrating image quality metrics to capture image features which are correlated to image quality; the approach guarantees that significant image features are included to predict image quality. Also, a tree-based classifier namely geometric semantic genetic programming is proposed to perform the overall predictions by incorporating CNN predictions and image features; the approach is simpler than the fully connected network but is able to model the nonlinear image qualities. The performance of the proposed hybrid DNN is evaluated by an image quality database with 3000 distorted images. The mean correlation achieved by the proposed hybrid DNN is 0.57 which is higher than the other tested methods. Experimental results with the t- test, F-test and Tueky’s range tests show that the proposed hybrid DNN achieves more accurate image predictions with a 99.9% confidence level, compared to the state-of-the-art IQE metics and the most recently developed CNN for IQEs.

References

[1]

Kamble V and Bhurchandi K No-reference image quality assessment algorithms: a survey Optik 2015 126 11–12 1090-1097

[2]

Xu SP, Jiang SL, and Min WD No-reference/blind image quality assessment: a survey IETE Tech Rev 2017 34 3 223-245

[3]

Bovik AC Automatic prediction of perceptual image and video quality Proc IEEE 2013 101 9 2008-2024

[4]

Mittal A, Moorthy AK, and Bovik AC No-reference image quality assessment in the spatial domain IEEE Trans Image Process 2012 21 12 4695-4708

[5]

Mittal A, Soundararajan R, and Bovik AC Making a ‘completely blind’ image quality analyzer IEEE Signal Process Lett 2013 20 3 209-212

[6]

Reibman AR, Suthaharan S (2008) A no-reference spatial aliasing measure for digital image resizing. In: IEEE international conference on image processing, pp 1184–1187

[7]

Qian J, Wu D, Li L, Cheng D, and Wang X Image quality assessment based on multi-scale representation of structure Digit Signal Process 2014 33 123-133

[8]

Guha T, Nezhadary E, and Ward RK Sparse representation-based image quality assessment Signal Process: Image Commun 2014 29 10 1138-1148

[9]

Corchs S, Gasparini F, and Schettini R No reference image quality classification for jpeg-distorted images Digit Signal Process 2014 30 86-100

[10]

Saad MA, Bovik AC, and Charrier C A dct statistics- based blind image quality index IEEE Signal Process Lett 2010 17 6 583-586

[11]

Saad MA, Bovik AC, and Charrier C Blind image quality assessment: a natural scene statistics approach in the dct domain IEEE Trans Image Process 2012 21 8 3339-3352

[12]

Chapelle O, Haffner P, and Vapnik V Support vector machines for histogram-based image classification IEEE Trans Neural Netw 1999 10 5 1055-1064

[13]

Chan KY and Engelke U Fuzzy regression for perceptual image quality assessment Eng Appl Artif Intell 2015 43 102-110

[14]

Chan KY, Lam HK, Yiu KFC, and Dillon TS A flexible fuzzy regression for addressing uncertainty on aesthetic quality assessments IEEE Trans Syst Man Cybern: Systems 2017 47 12 2363-2377

[15]

Babu RV, Sureshb S, and Perkis A No-reference jpeg-image quality assessment using gap-rbf Signal Process 2007 87 6 1493-1503

[16]

Engelke U, Zepernick HJ (2007) An artificial neural network for quality assessment in wireless imaging based on extraction of structural information. In: IEEE international conference on acoustics, speech, and signal processing, pp 1249–1252

[17]

Bosse S, Maniry D, Muller TWKR, and Samek W Deep neural networks for no-reference and full-reference image quality assessment IEEE Trans Image Process 2018 27 1 206-219

[18]

Ghadiyaram D and Bovik AC Massive online crowdsourced study of subjective and objective picture quality IEEE Trans Image Process 2016 25 1 372-397

[19]

Ye P, Kumar J, Kang L, Doermann D (2012) Unsupervised feature learning framework for no-reference image quality assessment. In: IEEE proceedings of conference on computer visions, pp 1098–1105

[20]

Kang L, Ye P, Li Y, Doermann D (2014) Convolutional neural networks for no-reference image quality assessment. In: Proceedings of IEEE international conference of conference on computer vision and pattern recognition, IEEE, pp 1733–1740

[21]

Kim J and Lee S Fully deep blind image quality predictor IEEE J Sel Topics Signal Process 2017 11 1 206-220

[22]

Ponomarenkoa N, Jinb L, Ieremeieva O, Lukina V, Egiazarianb K, Astolab J, Vozelc B, Chehdic K, Carlid M, Battistid F, and Kuo CCJ Image database tid2013: peculiarities, resultsand perspectives Signal Process: Image Commun 2015 30 57-77

[23]

Wang Z, Sheikh HR, Bovik AC (2002) No reference perceptual quality assessment of jpeg compressed images. In: IEEE Proceedings of the 15th international conference on image processing, pp 477–480

[24]

Moraglio A, Krawiec K, Johnson CG (2002) A no-reference perceptual blur metric. In: Proceedings of IEEE international conference of image processing, pp 57–60

[25]

Saha S and Vemuri R An analysis on the effect of image features on lossy coding performance IEEE Signal Process Lett 2000 7 5 104-107

[26]

Guan J, Zhang W, Gu J, and Ren H No-reference blur assessment based on edge modeling J Vis Commun Image Represent 2015 29 1-7

[27]

Sang Q, Qi H, Wua X, Li C, and Bovik AC Noreference image blur index based on singular value curve J Vis Commun Image Represent 2014 25 7 1625-1630

[28]

Varadarajan S, Karam L (2008) An improved perceptionbased no reference objective image sharpness metric using iterative edge refinement. In: IEEE Proceedings of the 15th international conference on image processing, pp 401–404

[29]

Ong E, Lin W, Lu Z (2003) A no-reference quality metric for measuring image blur. In: Proceedings of the 7th international symposium on signal processing and its applications, Vol. 1, pp 469–472

[30]

Meesters L and Martens JB A single-ended blockiness measure for jpeg-coded images Signal Process 2002 82 3 369-387

[31]

Chen CH, Bloom JA (2010) A blind reference-free blockiness measure. In: Proceedings of the 11th pacific rim conference on multimedia, pp 112–123

[32]

Tong H, Li M, Zhang H, Zhang C (2004) No-reference quality assessment for jpeg2000 compressed images. In: Proceedings of international conference on image processing, Vol. 5, pp 3539–3542

[33]

Sazzad ZMP, Kawayoke Y, Horita Y (2007) Spatial features based no reference image quality assessment for jpeg2000. In: IEEE Proceedings of international conference on image processing, pp 517–520

[34]

Zhang J, Ong SH, and Le TM Kurtosis-based noreference quality assessment of jpeg2000 images Signal Process: Image Commun 2011 26 1 13-23

[35]

Yuan Y, Guo Q, and Lu X Image quality assessment: a sparse learning way Neurocomputing 2015 159 227-241

[36]

Moorthy AK and Bovik AC Blind image quality assessment: from natural scene statistics to perceptual quality IEEE Trans Image Process 2011 20 12 3350-3364

[37]

Artusi A, Banterle F, Carrara F, and Moreo A Efficient evaluation of image quality via deep-learning approximation of perceptual metrics IEEE Trans Image Process 2020 29 1843-1854

[38]

Sujit SJ, Coronado I, Kamali A, Narayana PA, and Gabr RE Automated image quality evaluation of structural brain mri using an ensemble of deep learning networks J Magn Reson Imaging 2019 50 4 1260-1267

[39]

Zhang HQ, Li S, Li DH (2020) A dual-path deep neural network for sonar image quality evaluation. In: IEEE Proceedings of conference on networking, sensing and control

[40]

Campo FAD, Villeas OOV, Sanchez VGC, Dominguez HDJO, and Nandayapa M Radial basis function neural network for the evaluation of image color quality shown on liquid crystal displays IEEE Access 2021 9 21694-21707

[41]

Yan ZC, Bi Y, Xue B, Zhang MJ (2021) Automatically extracting features using genetic programming for low-quality fish image classification. In: IEEE congress on evolutionary computation, pp 2015–2022

[42]

Bi Y, Xue B, and Zhang MJ Genetic programming-based discriminative feature learning for low-quality image classification IEEE Trans Cybern 2021

[43]

Castelli M, Vanneschi L, and Silva S Semantic search-based genetic programming and the effect of intron deletion IEEE Trans Cybern 2014 44 1 103-113

[44]

Castelli M, Silva S, and Vanneschi L A c++ framework for geometric semantic genetic programming Genet Program Evolvable Mach 2015 16 73-81

[45]

Moraglio A, Krawiec K, Johnson CG (2012) Geometric semantic genetic programming. In: Parallel Problem Solving From Nature, Vol. 7491 of Lecture notes in computer science, pp 21–31

[46]

Ponomarenko N, Lukin V, Zelensky A, Egiazarian K, Carli M, and Battisti F Tid 2008 - a database for evaluation of full-reference visual quality assessment metrics Adv Mod Radioelectron 2009 10 30-45

[47]

Sheikh H, Wang Z, Cormack L, Bovik A (2005) Live image quality assessment database. https://live.ece.utexas.edu/research/Quality/subjective.htm

[48]

Sheikh HR, Sabir MF, and Bovik AC A statistical evaluation of recent full reference image quality assessment algorithms IEEE Trans Image Process 2006 15 11 3440-3451

[49]

Engelkea U, Maederb A, and Zepernic HJ Human observer confidence in image quality assessment IEEE Trans Image Process 2012 27 935-947

[50]

Simonyan K, Zisserman A (2014) Very deep convolutional networks for large-scale image recognition. In: IEEE Proceedings of imagenet challenge, IEEE, pp 1–10

[51]

Zhang P, Zhou W, Wu L, Li H (2015) Som: Semantic obviousness metric for image quality assessment. In: IEEE Proceedings of conference on computer vision and pattern recognition, pp 2394–2402

[52]

Lee SJ, Haralick RM, and Shapiro LG Morphologic edge detection IEEE J Robot Autom RA- 1987 3 2 142-156

[53]

Box GEP, Hunter J, and Hunter W Statistics for Experiments: Design, Innovation, and Discovery 2005 2 New York Wiley

Cited By

Tang ZChen YChen ZLiang XZhang X(2024)Lightweight transformer and multi-head prediction network for no-reference image quality assessmentNeural Computing and Applications10.1007/s00521-023-09188-336:4(1931-1946)Online publication date: 1-Feb-2024
https://dl.acm.org/doi/10.1007/s00521-023-09188-3

Index Terms

A genetic programming-based convolutional neural network for image quality evaluations
1. Computing methodologies
  1. Artificial intelligence
    1. Computer vision
  2. Machine learning
    1. Machine learning approaches
      1. Neural networks

Index terms have been assigned to the content through auto-classification.

Recommendations

Reduced-reference metric design for objective perceptual quality assessment in wireless imaging

The rapid growth of third and development of future generation mobile systems has led to an increase in the demand for image and video services. However, the hostile nature of the wireless channel makes the deployment of such services much more ...
Genetic programming based blind image deconvolution for surveillancesystems

Image acquisition, segmentation, object detection and tracking are essential parts of surveillance systems. Usually, image filtering approaches are employed as preprocessing step to reduce the effect of motion or out-of-focus blur problem. In this paper,...
Edge-preserving image denoising using a deep convolutional neural network
Highlights
- This paper makes use of a deep CNN for image denoising.
- The network is trained ...
Abstract
This paper introduces a novel denoising approach making use of a deep convolutional neural network to preserve image edges. The network is trained by using the edge map obtained from the well-known Canny algorithm and aims at ...

Comments

Information & Contributors

Information

Published In

cover image Neural Computing and Applications

Neural Computing and Applications Volume 34, Issue 18

Sep 2022

1039 pages

ISSN:0941-0643

EISSN:1433-3058

Issue’s Table of Contents

© The Author(s) 2022.

Publisher

Springer-Verlag

Berlin, Heidelberg

Publication History

Published: 01 September 2022

Accepted: 29 March 2022

Received: 27 August 2021

Author Tags

Qualifiers

Research-article

Funding Sources

Curtin University

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

1
Total Citations
View Citations
0
Total Downloads

Downloads (Last 12 months)0
Downloads (Last 6 weeks)0

Reflects downloads up to 26 Jan 2025

Other Metrics

View Author Metrics

Citations

Cited By

Tang ZChen YChen ZLiang XZhang X(2024)Lightweight transformer and multi-head prediction network for no-reference image quality assessmentNeural Computing and Applications10.1007/s00521-023-09188-336:4(1931-1946)Online publication date: 1-Feb-2024
https://dl.acm.org/doi/10.1007/s00521-023-09188-3

View Options

View options

Figures

Tables

Media

View Issue’s Table of Contents