Location via proxy:   [ UP ]  
[Report a bug]   [Manage cookies]                
skip to main content
research-article

Precise No-Reference Image Quality Evaluation Based on Distortion Identification

Published: 15 November 2021 Publication History

Abstract

The difficulty of no-reference image quality assessment (NR IQA) often lies in the lack of knowledge about the distortion in the image, which makes quality assessment blind and thus inefficient. To tackle such issue, in this article, we propose a novel scheme for precise NR IQA, which includes two successive steps, i.e., distortion identification and targeted quality evaluation. In the first step, we employ the well-known Inception-ResNet-v2 neural network to train a classifier that classifies the possible distortion in the image into the four most common distortion types, i.e., Gaussian white noise (WN), Gaussian blur (GB), jpeg compression (JPEG), and jpeg2000 compression (JP2K). Specifically, the deep neural network is trained on the large-scale Waterloo Exploration database, which ensures the robustness and high performance of distortion classification. In the second step, after determining the distortion type of the image, we then design a specific approach to quantify the image distortion level, which can estimate the image quality specially and more precisely. Extensive experiments performed on LIVE, TID2013, CSIQ, and Waterloo Exploration databases demonstrate that (1) the accuracy of our distortion classification is higher than that of the state-of-the-art distortion classification methods, and (2) the proposed NR IQA method outperforms the state-of-the-art NR IQA methods in quantifying the image quality.

References

[1]
Chenggang Yan, Hongtao Xie, Jianjun Chen, Zhengjun Zha, Xinhong Hao, Yongdong Zhang, and Qionghai Dai. 2018. A fast Uyghur text detector for complex background images. IEEE Trans. Multimedia 20, 12 (2018), 3389–3398.
[2]
Chenggang Yan, Liang Li, Chunjie Zhang, Bingtao Liu, Yongdong Zhang, and Qionghai Dai. 2019. Cross-modality bridging and knowledge transferring for image understanding. IEEE Trans. Multimedia 21, 10 (2019), 2675–2685.
[3]
Chenggang Yan, Yunbin Tu, Xingzheng Wang, Yongbing Zhang, Xinhong Hao, Yongdong Zhang, and Qionghai Dai. 2019. STAT: Spatial-temporal attention mechanism for video captioning. IEEE Trans. Multimedia 22, 1 (2019), 229–241.
[4]
Zunjie Zhu, Feng Xu, Chenggang Yan, Xinhong Hao, Xiangyang Ji, Yongdong Zhang, and Qionghai Dai. 2019. Real-time indoor scene reconstruction with RGBD and inertial input. In 2019 IEEE International Conference on Multimedia and Expo (ICME’19). IEEE, 7–12.
[5]
Yi Zhu, Sharath Chandra Guntuku, Weisi Lin, Gheorghita Ghinea, and Judith A. Redi. 2018. Measuring individual video QoE: A survey, and proposal for future directions using social media. ACM Trans. Multimedia Comput. Commun. Appl. 14, 2s (May 2018), 30:1–30:24.
[6]
Stephen R. Gulliver and Gheorghita Ghinea. 2006. Defining user perception of distributed multimedia quality. ACM Trans. Multimedia Comput. Commun. Appl. 2, 4 (Nov. 2006), 241–257.
[7]
Zhou Wang, Alan Conrad Bovik, Hamid Rahim Sheikh, and Eero P. Simoncelli. 2004. Image quality assessment: From error visibility to structural similarity. IEEE Trans. Image Process. 13, 4 (2004), 600–612.
[8]
Y. Liu, G. Zhai, K. Gu, X. Liu, D. Zhao, and W. Gao. 2018. Reduced-reference image quality assessment in free-energy principle and sparse representation. IEEE Trans. Multimedia 20, 2 (2018), 379–391.
[9]
Yutao Liu, Ke Gu, Xiu Li, and Yongbing Zhang. 2020. Blind image quality assessment by natural scene statistics and perceptual characteristics. ACM Trans. Multimedia Comput. Comm. Appl. (TOMM) 16, 3 (2020), 1–91.
[10]
Yutao Liu, Ke Gu, Guangtao Zhai, Xianming Liu, Debin Zhao, and Wen Gao. 2017. Quality assessment for real out-of-focus blurred images. J. Vis. Comm. Image Represent. 46 (2017), 70–80.
[11]
Phong V. Vu and Damon M. Chandler. 2012. A fast wavelet-based algorithm for global and local image sharpness estimation. IEEE Signal Process. Lett. 19, 7 (2012), 423–426.
[12]
Rania Hassen, Zhou Wang, and Magdy M. A. Salama. 2013. Image sharpness assessment based on local phase coherence. IEEE Trans. Image Process. 22, 7 (2013), 2798–2810.
[13]
Daniel Zoran and Yair Weiss. 2009. Scale invariance and noise in natural images. In Proceedings of the IEEE International Conference on Computer Vision.2209–2216.
[14]
X. Liu, M. Tanaka, and M. Okutomi. 2013. Single-image noise level estimation for blind denoising. IEEE Trans. Image Process. 22, 12 (Dec. 2013), 5226–5237.
[15]
Zhou Wang, A. C. Bovik, and B. L. Evan. 2000. Blind measurement of blocking artifacts in images. In Proceedings 2000 International Conference on Image Processing (Cat. No.00CH37101), Vol. 3. 981–984.
[16]
F. Pan, X. Lin, S. Rahardja, W. Lin, E. Ong, S. Yao, Z. Lu, and X. Yang. 2004. A locally-adaptive algorithm for measuring blocking artifacts in images and videos. In 2004 IEEE International Symposium on Circuits and Systems (IEEE Cat. No.04CH37512), Vol. 3. III–925.
[17]
Yutao Liu, Ke Gu, Shiqi Wang, Debin Zhao, and Wen Gao. 2019. Blind quality assessment of camera images based on low-level and high-level statistical features. IEEE Trans. Multimedia 21, 1 (2019), 135–146.
[18]
Ke Gu, Guangtao Zhai, Xiaokang Yang, and Wenjun Zhang. 2015. Using free energy principle for blind image quality assessment. IEEE Trans. Multimedia 17, 1 (2015), 50–63.
[19]
Anush Krishna Moorthy and Alan Conrad Bovik. 2010. A two-step framework for constructing blind image quality indices. IEEE Signal Process. Lett. 17, 5 (2010), 513–516.
[20]
Y. Liu, K. Gu, Y. Zhang, X. Li, G. Zhai, D. Zhao, and W. Gao. 2019. Unsupervised blind image quality evaluation via statistical measurements of structure, naturalness and perception. IEEE Trans. Circuits Syst. Video Technol. 30, 4 (2019), 929–943.
[21]
Lin Zhang, Ying Shen, and Hongyu Li. 2014. VSI: A visual saliency-induced index for perceptual image quality assessment. IEEE Trans. Image Process. 23, 10 (2014), 4270–4281.
[22]
Anush Krishna Moorthy and Alan Conrad Bovik. 2011. Blind image quality assessment: From natural scene statistics to perceptual quality. IEEE Trans. Image Process. 20, 12 (2011), 3350–3364.
[23]
K. Ma, W. Liu, K. Zhang, Z. Duanmu, Z. Wang, and W. Zuo. 2018. End-to-end blind image quality assessment using deep neural networks. IEEE Trans. Image Process. 27, 3 (2018), 1202–1213.
[24]
Christian Szegedy, Sergey Ioffe, Vincent Vanhoucke, and Alexander A. Alemi. 2017. Inception-v4, inception-resnet and the impact of residual connections on learning. In 31st AAAI Conference on Artificial Intelligence.
[25]
Kede Ma, Zhengfang Duanmu, Qingbo Wu, Zhou Wang, Hongwei Yong, Hongliang Li, and Lei Zhang. 2016. Waterloo exploration database: New challenges for image quality assessment models. IEEE Trans. Image Process. 26, 2 (2016), 1004–1016.
[26]
Zhou Wang, H. R. Sheikh, and A. C. Bovik. 2002. No-reference perceptual quality assessment of JPEG compressed images. In Proceedings International Conference on Image Processing.
[27]
C. Perra, F. Massidda, and D. D. Giusto. 2005. Image blockiness evaluation based on Sobel operator. In IEEE International Conference on Image Processing 2005, Vol. 1. I–389.
[28]
X. Min, K. Ma, K. Gu, G. Zhai, Z. Wang, and W. Lin. 2017. Unified blind quality assessment of compressed natural, graphic, and screen content images. IEEE Trans. Image Process. 26, 11 (Nov. 2017), 5462–5474.
[29]
Pina Marziliano, Frederic Dufaux, Stefan Winkler, and Touradj Ebrahimi. 2002. A no-reference perceptual blur metric. In Proceedings of the IEEE International Conference on Image Processing.57–60.
[30]
Rony Ferzli and Lina J. Karam. 2009. A no-reference objective image sharpness metric based on the notion of just noticeable blur (JNB). IEEE Trans. Image Process. 18, 4 (2009), 717–728.
[31]
K. Gu, W. Lin, G. Zhai, X. Yang, W. Zhang, and C. W. Chen. 2017. No-reference quality metric of contrast-distorted images based on information maximization. IEEE Trans. Cybernetics 47, 12 (Dec. 2017), 4559–4565.
[32]
Y. Liu and X. Li. 2020. No-reference quality assessment for contrast-distorted images. IEEE Access 8 (2020), 84105–84115.
[33]
Anish Mittal, Anush Krishna Moorthy, and Alan Conrad Bovik. 2012. No-reference image quality assessment in the spatial domain. IEEE Trans. Image Process. 21, 12 (2012), 4695–4708.
[34]
Michele A. Saad, Alan C. Bovik, and Christophe Charrier. 2012. Blind image quality assessment: A natural scene statistics approach in the DCT domain. IEEE Trans. Image Process. 21, 8 (2012), 3339–3352.
[35]
Anish Mittal, Ravi Soundararajan, and Alan C. Bovik. 2013. Making a completely blind image quality analyzer. IEEE Signal Process. Lett. 20, 3 (2013), 209–212.
[36]
Y. Zhang and D. M. Chandler. 2018. Opinion-unaware blind quality assessment of multiply and singly distorted images via distortion parameter estimation. IEEE Trans. Image Process. 27, 11 (Nov. 2018), 5433–5448.
[37]
L. Kang, P. Ye, Y. Li, and D. Doermann. 2014. Convolutional neural networks for no-reference image quality assessment. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR’14). 1733–1740.
[38]
S. Bosse, D. Maniry, K. Müller, T. Wiegand, and W. Samek. 2018. Deep neural networks for no-reference and full-reference image quality assessment. IEEE Trans. Image Process. 27, 1 (Jan. 2018), 206–219.
[39]
J. Wu, J. Ma, F. Liang, W. Dong, G. Shi, and W. Lin. 2020. End-to-end blind image quality prediction with cascaded deep neural network. IEEE Trans. Image Process. 29 (2020), 7414–7426.
[40]
X. Liu, J. Van De Weijer, and A. D. Bagdanov. 2017. RankIQA: Learning from rankings for no-reference image quality assessment. In 2017 IEEE International Conference on Computer Vision (ICCV’17). 1040–1049.
[41]
J. Kim and S. Lee. 2017. Fully deep blind image quality predictor. IEEE J. Sel. Topics Signal Process. 11, 1 (2017), 206–220. DOI:
[42]
W. Zhang, K. Ma, G. Zhai, and X. Yang. 2021. Uncertainty-aware blind image quality assessment in the laboratory and wild. IEEE Trans. Image Process. 30 (2021), 3474–3486. DOI:
[43]
Q. Yan, D. Gong, and Y. Zhang. 2019. Two stream convolutional networks for blind image quality assessment. IEEE Transactions on Image Processing 28, 5 (2019), 2200–2211.
[44]
Robert M. Haralick, Karthikeyan Shanmugam, and Its’ Hak Dinstein. 1973. Textural features for image classification. IEEE Trans. Syst. Man Cybernetics 6 (1973), 610–621.
[45]
Martin Szummer and Rosalind W. Picard. 1998. Indoor-outdoor image classification. In Proceedings of the 1998 IEEE International Workshop on Content-Based Access of Image and Video Database. IEEE, 42–51.
[46]
Alireza Khotanzad and Yaw Hua Hong. 1990. Invariant image recognition by Zernike moments. IEEE Trans. Pattern Anal. Mach. Intell. 12, 5 (1990), 489–497.
[47]
Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun. 2016. Deep residual learning for image recognition. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. 770–778.
[48]
Christian Szegedy, Vincent Vanhoucke, Sergey Ioffe, Jon Shlens, and Zbigniew Wojna. 2016. Rethinking the inception architecture for computer vision. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. 2818–2826.
[49]
G. Zhai, A. Kaup, J. Wang, and X. Yang. 2013. A dual-model approach to blind quality assessment of noisy images. In 2013 Picture Coding Symposium (PCS’13). 29–32.
[50]
Eero P. Simoncelli and Bruno A. Olshausen. 2001. Natural image statistics and neural representation. Ann. Rev. Neurosci. 24, 1 (2001), 1193–1216.
[51]
M. Concetta Morrone and D. C. Burr. 1988. Feature detection in human vision: A phase-dependent energy model. Proc. Roy. Soc. London B Biol. Sci. 235, 1280 (1988), 221–245.
[52]
M. Concetta Morrone, John Ross, David C. Burr, and Robyn Owens. 1986. Mach bands are phase dependent. Nature 324, 6094 (1986), 250–253.
[53]
Yutao Liu, Guangtao Zhai, Xianming Liu, and Debin Zhao. 2015. Quality assessment for out-of-focus blurred images. In Visual Communications and Image Processing (VCIP’15). IEEE, 1–4.
[54]
Peter Kovesi et al. 1999. Image features from phase congruency. Videre: Journal of Computer Vision Research 1, 3 (1999), 1–26.
[55]
Albert Cohen, Ingrid Daubechies, and J.-C. Feauveau. 1992. Biorthogonal bases of compactly supported wavelets. Comm. Pure Appl. Math. 45, 5 (1992), 485–560.
[56]
Xiongkuo Min, Guangtao Zhai, Ke Gu, Yuming Fang, Xiaokang Yang, Xiaolin Wu, Jiantao Zhou, and Xianming Liu. 2016. Blind quality assessment of compressed images via pseudo structural similarity. In 2016 IEEE International Conference on Multimedia and Expo (ICME’16). IEEE, 1–6.
[57]
Leida Li, Weisi Lin, and Hancheng Zhu. 2014. Learning structural regularity for evaluating blocking artifacts in JPEG images. IEEE Signal Process. Lett. 21, 8 (2014), 918–922.
[58]
Leida Li, Weisi Lin, and Hancheng Zhu. 2014. Learning structural regularity for evaluating blocking artifacts in JPEG images. IEEE Signal Process. Lett. 21, 8 (2014), 918–922.
[59]
Jianbo Shi et al. 1994. Good features to track. In 1994 Proceedings of IEEE Conference on Computer Vision and Pattern Recognition. IEEE, 593–600.
[60]
Phong V. Vu and Damon M. Chandler. 2013. A no-reference quality assessment algorithm for JPEG2000-compressed images based on local sharpness. In Image Quality and System Performance X, Vol. 8653. International Society for Optics and Photonics, 865302.
[61]
Cuong T. Vu, Thien D. Phan, and Damon M. Chandler. 2012. S3: A spectral and spatial measure of local perceived sharpness in natural images. IEEE Trans. Image Process. 21, 3 (2012), 934–945.
[62]
Hamid R. Sheikh, Muhammad F. Sabir, and Alan C. Bovik. 2006. A statistical evaluation of recent full reference image quality assessment algorithms. IEEE Trans. Image Process. 15, 11 (2006), 3440–3451.
[63]
Eric Cooper Larson and Damon Michael Chandler. 2010. Most apparent distortion: full-reference image quality assessment and the role of strategy. Journal of Electron. Imaging 19, 1 (2010), 011006.
[64]
Nikolay Ponomarenko, Lina Jin, Oleg Ieremeiev, Vladimir Lukin, Karen Egiazarian, Jaakko Astola, Benoit Vozel, Kacem Chehdi, Marco Carli, Federica Battisti, et al. 2015. Image database TID2013: Peculiarities, results and perspectives. Signal Process. Image Comm. 30 (2015), 57–77.
[65]
Ann Marie Rohaly, John Libert, Philip Corriveau, Arthur Webster, et al. 2000. Final report from the video quality experts group on the validation of objective models of video quality assessment. ITU-T Standards Contribution COM (2000), 9–80.
[66]
Qingbo Wu, Zhou Wang, and Hongliang Li. 2015. A highly efficient method for blind image quality assessment. In IEEE International Conference on Image Processing.339–343.
[67]
Wufeng Xue, Lei Zhang, and Xuanqin Mou. 2013. Learning without human scores for blind image quality assessment. In Proceedings of the IEEE International Conference on Computer Vision and Pattern Recognition.995–1002.
[68]
Lin Zhang, Lei Zhang, and Alan C. Bovik. 2015. A feature-enriched completely blind image quality evaluator. IEEE Trans. Image Process. 24, 8 (2015), 2579–2591.

Cited By

View all
  • (2025)EM-Trans: Edge-Aware Multimodal Transformer for RGB-D Salient Object DetectionIEEE Transactions on Neural Networks and Learning Systems10.1109/TNNLS.2024.335885836:2(3175-3188)Online publication date: Feb-2025
  • (2025)CRDH: Compatible Reversible Data Hiding With High Capacity and GeneralizationIEEE Transactions on Circuits and Systems for Video Technology10.1109/TCSVT.2024.346544535:1(874-887)Online publication date: Jan-2025
  • (2025)Transformer-Prompted Network: Efficient Audio–Visual Segmentation via Transformer and Prompt LearningIEEE Signal Processing Letters10.1109/LSP.2024.352412032(516-520)Online publication date: 2025
  • Show More Cited By

Index Terms

  1. Precise No-Reference Image Quality Evaluation Based on Distortion Identification

    Recommendations

    Comments

    Information & Contributors

    Information

    Published In

    cover image ACM Transactions on Multimedia Computing, Communications, and Applications
    ACM Transactions on Multimedia Computing, Communications, and Applications  Volume 17, Issue 3s
    October 2021
    324 pages
    ISSN:1551-6857
    EISSN:1551-6865
    DOI:10.1145/3492435
    Issue’s Table of Contents

    Publisher

    Association for Computing Machinery

    New York, NY, United States

    Publication History

    Published: 15 November 2021
    Accepted: 01 May 2021
    Revised: 01 May 2021
    Received: 01 November 2020
    Published in TOMM Volume 17, Issue 3s

    Permissions

    Request permissions for this article.

    Check for updates

    Author Tags

    1. Image quality assessment (IQA)
    2. distortion identification
    3. no-reference (NR)/blind
    4. deep learning
    5. noisiness
    6. sharpness

    Qualifiers

    • Research-article
    • Refereed

    Funding Sources

    • National Key Research and Development Program of China
    • National Natural Science Foundation of China
    • China Postdoctoral Science Foundation
    • Zhejiang Province Natural Science Foundation of China
    • 111 Project
    • Shenzhen Science and Technology Project
    • Guangdong Special Support

    Contributors

    Other Metrics

    Bibliometrics & Citations

    Bibliometrics

    Article Metrics

    • Downloads (Last 12 months)280
    • Downloads (Last 6 weeks)22
    Reflects downloads up to 20 Feb 2025

    Other Metrics

    Citations

    Cited By

    View all
    • (2025)EM-Trans: Edge-Aware Multimodal Transformer for RGB-D Salient Object DetectionIEEE Transactions on Neural Networks and Learning Systems10.1109/TNNLS.2024.335885836:2(3175-3188)Online publication date: Feb-2025
    • (2025)CRDH: Compatible Reversible Data Hiding With High Capacity and GeneralizationIEEE Transactions on Circuits and Systems for Video Technology10.1109/TCSVT.2024.346544535:1(874-887)Online publication date: Jan-2025
    • (2025)Transformer-Prompted Network: Efficient Audio–Visual Segmentation via Transformer and Prompt LearningIEEE Signal Processing Letters10.1109/LSP.2024.352412032(516-520)Online publication date: 2025
    • (2025)Designing adaptive deep denoisers for Plug-and-Play FBS with stable iterationsNeurocomputing10.1016/j.neucom.2025.129710(129710)Online publication date: Feb-2025
    • (2025)A spatio-frequency cross fusion model for deepfake detection and segmentationNeurocomputing10.1016/j.neucom.2025.129683628(129683)Online publication date: May-2025
    • (2025)A Simplified Adaptive Fuzzy Min–Max Neural Network for pattern classificationNeurocomputing10.1016/j.neucom.2025.129668630(129668)Online publication date: May-2025
    • (2025)Long–Short Observation-driven Prediction Network for pedestrian crossing intention prediction with momentary observationNeurocomputing10.1016/j.neucom.2024.128824614(128824)Online publication date: Jan-2025
    • (2025)Method and system for detecting pulmonary function parameters by integrating four-subdivisions impedance informationMeasurement10.1016/j.measurement.2025.116795(116795)Online publication date: Jan-2025
    • (2025)A novel model-driven approach for recovering close-loop hysteresis curve from measurements with liftoffsMeasurement10.1016/j.measurement.2025.116716247(116716)Online publication date: Apr-2025
    • (2025)Heterogeneous hypergraph transformer network with cross-modal future interaction for multi-agent trajectory predictionEngineering Applications of Artificial Intelligence10.1016/j.engappai.2025.110125144(110125)Online publication date: Mar-2025
    • Show More Cited By

    View Options

    Login options

    Full Access

    View options

    PDF

    View or Download as a PDF file.

    PDF

    eReader

    View online with eReader.

    eReader

    Full Text

    View this article in Full Text.

    Full Text

    HTML Format

    View this article in HTML Format.

    HTML Format

    Figures

    Tables

    Media

    Share

    Share

    Share this Publication link

    Share on social media