Abstract
The intelligent video surveillance system (IVSS) can automatically analyze the content of the surveillance image (SI) and reduce the burden of the manual labour. However, the SIs may suffer quality degradations in the procedure of acquisition, compression, and transmission, which makes IVSS hard to understand the content of SIs. In this paper, we first conduct an example experiment (i.e. the face detection task) to demonstrate that the quality of the SIs has a crucial impact on the performance of the IVSS, and then propose a saliency-based deep neural network for the blind quality assessment of the SIs, which helps IVSS to filter the low-quality SIs and improve the detection and recognition performance. Specifically, we first compute the saliency map of the SI to select the most salient local region since the salient regions usually contain rich semantic information for machine vision and thus have a great impact on the overall quality of the SIs. Next, the convolutional neural network (CNN) is adopted to extract quality-aware features for the whole image and local region, which are then mapped into the global and local quality scores through the fully connected (FC) network respectively. Finally, the overall quality score is computed as the weighted sum of the global and local quality scores. Experimental results on the SI quality database (SIQD) show that the proposed method outperforms all compared state-of-the-art BIQA methods.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Similar content being viewed by others
References
Sreenu, G., Durai, M.S.: Intelligent video surveillance: a review through deep learning techniques for crowd analysis. J. Big Data 6(1), 1–27 (2019)
Zhu, W., Zhai, G., Yao, C., Yang, X.: SIQD: surveillance image quality database and performance evaluation for objective algorithms. In: 2018 IEEE Visual Communications and Image Processing (VCIP). pp. 1–4. IEEE (2018)
Leszczuk, M., Romaniak, P., Janowski, L.: Quality assessment in video surveillance. In: Recent Developments in Video Surveillance. IntechOpen (2012)
Aqqa, M., Mantini, P., Shah, S.K.: Understanding how video quality affects object detection algorithms. In: VISIGRAPP (5: VISAPP), pp. 96–104 (2019)
Sun, W., Min, X., Zhai, G., Ma, S.: Blind quality assessment for in-the-wild images via hierarchical feature fusion and iterative mixed database training. arXiv preprint arXiv:2105.14550 (2021)
Sun, W., Min, X., Zhai, G., Gu, K., Duan, H., Ma, S.: MC360IQA: a multi-channel CNN for blind 360-degree image quality assessment. IEEE J. Sel. Top. Signal Proces. 14(1), 64–77 (2019)
Zhai, G., Sun, W., Min, X., Zhou, J.: Perceptual quality assessment of low-light image enhancement. ACM Trans. Multimedia Comput. Commun. Appl. (TOMM) 17(4), 1–24 (2021)
Zhang, Z., et al.: A no-reference deep learning quality assessment method for super-resolution images based on frequency maps. arXiv preprint arXiv:2206.04289 (2022)
Mittal, A., Moorthy, A.K., Bovik, A.C.: No-reference image quality assessment in the spatial domain. IEEE Trans. Image Process. 21(12), 4695–4708 (2012)
Fang, Y., Ma, K., Wang, Z., Lin, W., Fang, Z., Zhai, G.: No-reference quality assessment of contrast-distorted images based on natural scene statistics. IEEE Signal Process. Lett. 22(7), 838–842 (2014)
Zhang, Z., et al.: A no-reference evaluation metric for low-light image enhancement. In: 2021 IEEE International Conference on Multimedia and Expo (ICME), pp. 1–6. IEEE (2021)
Zhang, Z., Sun, W., Min, X., Wang, T., Lu, W., Zhai, G.: A full-reference quality assessment metric for fine-grained compressed images. In: 2021 International Conference on Visual Communications and Image Processing (VCIP), pp. 1–4. IEEE (2021)
Gu, K., Zhai, G., Yang, X., Zhang, W.: Using free energy principle for blind image quality assessment. IEEE Trans. Multimedia 17(1), 50–63 (2014)
Zhai, G., Wu, X., Yang, X., Lin, W., Zhang, W.: A psychovisual quality metric in free-energy principle. IEEE Trans. Image Process. 21(1), 41–52 (2011)
Min, X., Zhai, G., Gu, K., Liu, Y., Yang, X.: Blind image quality estimation via distortion aggravation. IEEE Trans. Broadcast. 64(2), 508–517 (2018)
Sun, W., Wang, T., Min, X., Yi, F., Zhai, G.: Deep learning based full-reference and no-reference quality assessment models for compressed UGC videos. In: 2021 IEEE International Conference on Multimedia & Expo Workshops (ICMEW), pp. 1–6. IEEE (2021)
Wang, T., Sun, W., Min, X., Lu, W., Zhang, Z., Zhai, G.: A multi-dimensional aesthetic quality assessment model for mobile game images. In: 2021 International Conference on Visual Communications and Image Processing (VCIP), pp. 1–5. IEEE (2021)
Lu, W., et al.: A CNN-based quality assessment method for pseudo 4K contents. In: Zhai, G., Zhou, J., Yang, H., An, P., Yang, X. (eds.) IFTC 2021. Communications in Computer and Information Science, vol. 1560, pp. 164–176. Springer, Cham (2022). https://doi.org/10.1007/978-981-19-2266-4_13
Sun, W., Min, X., Lu, W., Zhai, G.: A deep learning based no-reference quality assessment model for UGC videos. arXiv preprint arXiv:2204.14047 (2022)
Reddy, N., Jain, S., Yarlagadda, P., Gandhi, V.: Tidying deep saliency prediction architectures. In: 2020 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), pp. 10241–10247. IEEE (2020)
Jiang, M., Huang, S., Duan, J., Zhao, Q.: Salicon: saliency in context. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 1072–1080 (2015)
He, K., Zhang, X., Ren, S., Sun, J.: Deep residual learning for image recognition. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 770–778 (2016)
Deng, J., Dong, W., Socher, R., Li, L.J., Li, K., Fei-Fei, L.: Imagenet: a large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255. IEEE (2009)
Zhang, K., Zhang, Z., Li, Z., Qiao, Y.: Joint face detection and alignment using multitask cascaded convolutional networks. IEEE Signal Process. Lett. 23(10), 1499–1503 (2016)
Seshadrinathan, K., Soundararajan, R., Bovik, A.C., Cormack, L.K.: Study of subjective and objective quality assessment of video. IEEE Trans. Image Process. 19(6), 1427–1441 (2010)
Ye, P., Kumar, J., Kang, L., Doermann, D.: Unsupervised feature learning framework for no-reference image quality assessment. In: 2012 IEEE Conference on Computer Vision and Pattern Recognition, pp. 1098–1105. IEEE (2012)
Moorthy, A.K., Bovik, A.C.: Blind image quality assessment: from natural scene statistics to perceptual quality. IEEE Trans. Image Process. 20(12), 3350–3364 (2011)
Xue, W., Mou, X., Zhang, L., Bovik, A.C., Feng, X.: Blind image quality assessment using joint statistics of gradient magnitude and Laplacian features. IEEE Trans. Image Process. 23(11), 4850–4862 (2014)
Xu, J., Ye, P., Li, Q., Du, H., Liu, Y., Doermann, D.: Blind image quality assessment based on high order statistics aggregation. IEEE Trans. Image Process. 25(9), 4444–4457 (2016)
Li, D., Jiang, T., Lin, W., Jiang, M.: Which has better visual quality: the clear blue sky or a blurry animal? IEEE Trans. Multimedia 21(5), 1221–1234 (2018)
Zhang, W., Ma, K., Yan, J., Deng, D., Wang, Z.: Blind image quality assessment using a deep bilinear convolutional neural network. IEEE Trans. Circuits Syst. Video Technol. 30(1), 36–47 (2018)
Su, S., et al.: Blindly assess image quality in the wild guided by a self-adaptive hyper network. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 3667–3676 (2020)
Sheikh, H.R., Sabir, M.F., Bovik, A.C.: A statistical evaluation of recent full reference image quality assessment algorithms. IEEE Trans. Image Process. 15(11), 3440–3451 (2006)
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2022 The Author(s), under exclusive license to Springer Nature Switzerland AG
About this paper
Cite this paper
Lu, W. et al. (2022). Blind Surveillance Image Quality Assessment via Deep Neural Network Combined with the Visual Saliency. In: Fang, L., Povey, D., Zhai, G., Mei, T., Wang, R. (eds) Artificial Intelligence. CICAI 2022. Lecture Notes in Computer Science(), vol 13605. Springer, Cham. https://doi.org/10.1007/978-3-031-20500-2_11
Download citation
DOI: https://doi.org/10.1007/978-3-031-20500-2_11
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-031-20499-9
Online ISBN: 978-3-031-20500-2
eBook Packages: Computer ScienceComputer Science (R0)