Abstract
Recently, there has been a growing interest in Ultra High-Definition (UHD) content, which brings a better visual experience for end-users. However, quite a few contents with 4K resolution are upscaled from High-Definition (HD) contents and suffer degradations in quality, such as blur, texture shift, etc. These pseudo 4K contents can not deliver the expected quality of experience (QoE) to end-users while requiring a high transmission bit rate in the meantime, which inevitably results in a waste of bandwidth resources. Hence, we develop a novel deep learning-based no reference (NR) image quality assessment (IQA) model for recognition and quality evaluation of real and fake 4K images. To reduce the computational overhead for a 4K image, we first select three representative patches with high texture complexity by the Grey-Level Co-occurrence Matrix (GLCM) based measure. Next, the convolutional neural network (CNN) is adopted to extract the quality-aware features of three representative patches. Specifically, we extract different levels of features from intermediate layers of CNN and concatenate them into a more effective quality-aware feature representation. Finally, the shallow fully connected (FC) network is utilized to aggregate the features into the quality score and the overall quality score of the 4K image is calculated as the average value of three patches’ quality scores. The experimental results show that the proposed method outperforms all compared NR IQA metrics on the 4K IQA database.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Similar content being viewed by others
Notes
- 1.
Code access: https://github.com/luwei-1998/4K_IQA.
References
Wang, J., Li, S.: Investigation and design of 4K ultra high definition television production and broadcasting system. In: 2020 IEEE 5th Information Technology and Mechatronics Engineering Conference (ITOEC), pp. 579–582 (2020). https://doi.org/10.1109/ITOEC49072.2020.9141900
Shah, R.R., Akundy, V.A., Wang, Z.: Real versus fake 4K-authentic resolution assessment. In: ICASSP 2021–2021 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), pp. 2185–2189. IEEE (2021)
Zhai, G., Sun, W., Min, X., Zhou, J.: Perceptual quality assessment of low-light image enhancement. ACM Trans. Multimed. Comput. Commun. Appl. (TOMM) 17(4), 1–24 (2021)
Sun, W., Min, X., Zhai, G., Gu, K., Ma, S., Yang, X.: Dynamic backlight scaling considering ambient luminance for mobile videos on LCD displays. IEEE Trans. Mob. Comput. 21(1), 110–124 (2022)
Zhai, G., Min, X.: Perceptual image quality assessment: a survey. Sci. China Inf. Sci. 63(11), 1–52 (2020). https://doi.org/10.1007/s11432-019-2757-1
Mittal, A., Moorthy, A.K., Bovik, A.C.: No-reference image quality assessment in the spatial domain. IEEE Trans. Image Process. 21(12), 4695–4708 (2012)
Zhai, G., Wu, X., Yang, X., Lin, W., Zhang, W.: A psychovisual quality metric in free-energy principle. IEEE Trans. Image Process. 21(1), 41–52 (2011)
Xue, W., Zhang, L., Mou, X.: Learning without human scores for blind image quality assessment. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 995–1002 (2013)
Zhu, W., Zhai, G., Min, X., Yang, X., Zhang, X.P.: Perceptual quality assessment for recognizing true and pseudo 4K content. In: ICASSP 2021–2021 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), pp. 2190–2194. IEEE (2021)
Haralick, R.M., Shanmugam, K., Dinstein, I.H.: Textural features for image classification. IEEE Trans. Syst. Man Cybern. 6, 610–621 (1973)
Sun, W., Min, X., Zhai, G., Ma, S.: Blind quality assessment for in-the-wild images via hierarchical feature fusion and iterative mixed database training. arXiv preprint arXiv:2105.14550 (2021)
Sun, W., Wang, T., Min, X., Yi, F., Zhai, G.: Deep learning based full-reference and no-reference quality assessment models for compressed UGC videos. In: 2021 IEEE International Conference on Multimedia & Expo Workshops (ICMEW), pp. 1–6. IEEE (2021)
Simonyan, K., Zisserman, A.: Very deep convolutional networks for large-scale image recognition. arXiv preprint arXiv:1409.1556 (2014)
He, K., Zhang, X., Ren, S., Sun, J.: Deep residual learning for image recognition. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 770–778 (2016)
Zhao, G., Huang, J.: DeepSim: deep learning code functional similarity. In: Proceedings of the 2018 26th ACM Joint Meeting on European Software Engineering Conference and Symposium on the Foundations of Software Engineering, pp. 141–151 (2018)
Deng, J., Dong, W., Socher, R., Li, L.J., Li, K., Fei-Fei, L.: ImageNet: a large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255. IEEE (2009)
Paszke, A., et al.: Automatic differentiation in PyTorch (2017)
Kingma, D.P., Ba, J.: Adam: a method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014)
Mittal, A., Soundararajan, R., Bovik, A.C.: Making a “completely blind’’ image quality analyzer. IEEE Signal Process. Lett. 20(3), 209–212 (2012)
Zhang, L., Zhang, L., Bovik, A.C.: A feature-enriched completely blind image quality evaluator. IEEE Trans. Image Process. 24(8), 2579–2591 (2015)
Wu, Q., Wang, Z., Li, H.: A highly efficient method for blind image quality assessment. In: 2015 IEEE International Conference on Image Processing (ICIP), pp. 339–343. IEEE (2015)
Xu, J., Ye, P., Li, Q., Du, H., Liu, Y., Doermann, D.: Blind image quality assessment based on high order statistics aggregation. IEEE Trans. Image Process. 25(9), 4444–4457 (2016)
Min, X., Gu, K., Zhai, G., Liu, J., Yang, X., Chen, C.W.: Blind quality assessment based on pseudo-reference image. IEEE Trans. Multimed. 20(8), 2049–2062 (2017)
Min, X., Zhai, G., Gu, K., Liu, Y., Yang, X.: Blind image quality estimation via distortion aggravation. IEEE Trans. Broadcast. 64(2), 508–517 (2018)
Gu, K., Zhai, G., Yang, X., Zhang, W.: Using free energy principle for blind image quality assessment. IEEE Trans. Multimed. 17(1), 50–63 (2014)
Xue, W., Mou, X., Zhang, L., Bovik, A.C., Feng, X.: Blind image quality assessment using joint statistics of gradient magnitude and Laplacian features. IEEE Trans. Image Process. 23(11), 4850–4862 (2014)
Narvekar, N.D., Karam, L.J.: A no-reference perceptual image sharpness metric based on a cumulative probability of blur detection. In: 2009 International Workshop on Quality of Multimedia Experience, pp. 87–91 (2009). https://doi.org/10.1109/QOMEX.2009.5246972
Sheikh, H.R., Sabir, M.F., Bovik, A.C.: A statistical evaluation of recent full reference image quality assessment algorithms. IEEE Trans. Image Process. 15(11), 3440–3451 (2006)
Acknowledgements
This work was supported in part by National Key R&D Program of China (No. 2019YFB1405900), and in part by the Open Research Project of the State Key Laboratory of Media Convergence and Communication, Communication University of China, China (No. SKLMCC2020KF003).
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2022 Springer Nature Singapore Pte Ltd.
About this paper
Cite this paper
Lu, W. et al. (2022). A CNN-Based Quality Assessment Method for Pseudo 4K Contents. In: Zhai, G., Zhou, J., Yang, H., An, P., Yang, X. (eds) Digital TV and Wireless Multimedia Communications. IFTC 2021. Communications in Computer and Information Science, vol 1560. Springer, Singapore. https://doi.org/10.1007/978-981-19-2266-4_13
Download citation
DOI: https://doi.org/10.1007/978-981-19-2266-4_13
Published:
Publisher Name: Springer, Singapore
Print ISBN: 978-981-19-2265-7
Online ISBN: 978-981-19-2266-4
eBook Packages: Computer ScienceComputer Science (R0)