Abstract
Face recognition in visual surveillance systems is important for various applications to identify individuals who are behaving defiantly at the time of an event or for investigation purposes. Despite the dramatic improvements in facial recognition technology in recent years, it is difficult to recognize faces from surveillance feeds due to the presence of multiple people of different scales and orientations. This paper solves the task of low-resolution face recognition by combining exemplary techniques for extracting distinct features. This research utilizes the attributes learned by customized and learnable filters and injected in the training process to better match them with human brain functionality. The Gabor transform aims to convolve a facial image using a range of Gabor filter coefficients at various scales and orientations, resulting in scale and rotation invariant features. The tailored architecture with residual stream aims to enhance functional representation and prevent the gradient of the prediction engine from affecting the backbone network functional map. Experimental analysis is performed on the SCface and TinyFace databases and is reported with an accuracy of 89.21% on the SCface database and 56.68% on the TinyFace database.
Similar content being viewed by others
Explore related subjects
Discover the latest articles, news and stories from top researchers in related subjects.References
Hennings-Yeomans, P.H., Baker, S., Kumar, B.V.K.V.: Simultaneous super-resolution and feature extraction for recognition of low-resolution faces. In: 26th IEEE Conference Computer Vision and Pattern Recognition, CVPR. (2008). https://doi.org/10.1109/CVPR.2008.4587810
Gunturk, B.K., Batur, A.U., Altunbasak, Y., Hayes, M.H., Mersereau, R.M.: Eigenface-domain super-resolution for face recognition. IEEE Trans. Image Process. 12, 597–606 (2003). https://doi.org/10.1109/TIP.2003.811513
Fookes, C., Lin, F., Chandran, V., Sridharan, S.: Evaluation of image resolution and super-resolution on face recognition performance. J. Vis. Commun. Image Represent. 23, 75–93 (2012). https://doi.org/10.1016/j.jvcir.2011.06.004
Gupta, S., Thakur, K., Kumar, M.: 2D-human face recognition using SIFT and SURF descriptors of face’s feature regions. Vis. Comput. 37, 447–456 (2021). https://doi.org/10.1007/s00371-020-01814-8
Meedeniya, D.A., Ratnaweera, D.A.A.C.: Enhanced face recognition through variation of Principle component analysis (PCA). In ICIIS 2007 - 2nd International Conference on Industrial and Information Systems, pp. 347–351 (2007). https://doi.org/10.1109/ICIINFS.2007.4579200
Wang, Z., Chang, S., Yang, Y., Liu, D., Huang, T.S.: Studying very low resolution recognition using deep networks. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 4792–4800 (2016). https://doi.org/10.1109/CVPR.2016.518
Zou, W.W.W., Yuen, P.C.: Very low resolution face recognition problem. IEEE Trans. Image Process. 21, 327–340 (2012). https://doi.org/10.1109/TIP.2011.2162423
Obara, K., Yoshimura, H., Nishiyama, M., Iwai, Y.: Low-resolution person recognition using image downsampling. In Proceedings 15th IAPR International Conference on Machine Vision Applications MVA, pp. 478–481 (2017). https://doi.org/10.23919/MVA.2017.7986904
Peng, Y., Spreeuwers, L., Veldhuis, R.: Designing a low-resolution face recognition system for long-range surveillance. In Lecture Notes Informatics (LNI), Proc. - Ser. Gesellschaft fur Inform. P-260, (2016). https://doi.org/10.1109/BIOSIG.2016.7736917
Hennings-Yeomans, P.H., Vijaya Kumar, B.V.K., Baker, S.: Robust low-resolution face identification and verification using high-resolution features. In Proceedings International Conference on Image Processing ICIP, pp. 33–36 (2009). https://doi.org/10.1109/ICIP.2009.5413920
Yang, F., Yang, W., Gao, R., Liao, Q.: Discriminative multidimensional scaling for low-resolution face recognition. IEEE Signal Process. Lett. 25, 388–392 (2018). https://doi.org/10.1109/LSP.2017.2746658
Zou, W., Yuen, P.C.: Very low resolution face recognition problem. In IEEE 4th International Conference on Biometrics: Theory, Applications and Systems BTAS (2010). https://doi.org/10.1109/BTAS.2010.5634490
Jiang, J., Chen, C., Ma, J., Wang, Z., Wang, Z., Hu, R.: SRLSP: a face image super-resolution algorithm using smooth regression with local structure prior. IEEE Trans. Multimed. 19, 27–40 (2017). https://doi.org/10.1109/TMM.2016.2601020
Mudunuri, S.P., Sanyal, S., Biswas, S.: GenLR-Net: Deep framework for very low resolution face and object recognition with generalization to unseen categories. In IEEE Conference on Computer Vision and Pattern Recognition, pp. 602–611 (2018). https://doi.org/10.1109/CVPRW.2018.00090
Zhang, P., Ben, X., Jiang, W., Yan, R., Zhang, Y.: Coupled marginal discriminant mappings for low-resolution face recognition. Optik (Stuttg) 126, 4352–4357 (2015). https://doi.org/10.1016/j.ijleo.2015.08.138
Maity, S., Abdel-Mottaleb, M., Asfour, S.S.: Multimodal low resolution face and frontal gait recognition from surveillance video. Electron (2021). https://doi.org/10.3390/electronics10091013
Mishra, N.K., Dutta, M., Singh, S.K.: Multiscale parallel deep CNN (mpdCNN) architecture for the real low-resolution face recognition for surveillance. Image Vis. Comput. 115, 104290 (2021). https://doi.org/10.1016/J.IMAVIS.2021.104290
Li, C., Huang, Y., Huang, W., Qin, F.: Learning features from covariance matrix of gabor wavelet for face recognition under adverse conditions. Pattern Recognit. (2021). https://doi.org/10.1016/j.patcog.2021.108085
Rouhsedaghat, M., Wang, Y., Hu, S., You, S., Kuo, C.C.J.: Low-resolution face recognition in resource-constrained environments. Pattern Recognit. Lett. 149, 193–199 (2021). https://doi.org/10.1016/j.patrec.2021.05.009
Zangeneh, E., Rahmati, M., Mohsenzadeh, Y.: Low resolution face recognition using a two-branch deep convolutional neural network architecture. Expert Syst. Appl. 139, 112854 (2020). https://doi.org/10.1016/j.eswa.2019.112854
Cheng, Z., Zhu, X., Gong, S.: Low-resolution face recognition (including Subser). In Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics. 11363 LNCS, pp. 605–621 (2019). https://doi.org/10.1007/978-3-030-20893-6_38
Xie, J., Ge, Y., Zhang, J., Huang, S., Chen, F., Wang, H.: Low-resolution assisted three-stream network for person re-identification. Vis. Comput. 38, 2515–2525 (2022). https://doi.org/10.1007/s00371-021-02127-0
He, K., Zhang, X., Ren, S., Sun, J.: Deep residual learning for image recognition. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 770–778 (2016). https://doi.org/10.1109/CVPR.2016.90
Hubel, D.H., Wiesel, T.N.: Receptive fields, binocular interaction and functional architecture in the cat’s visual cortex. J. Physiol. (Lond.) 160, 106–154 (1962)
Grgic, M., Delac, K., Grgic, S.: SCface - surveillance cameras face database. Multimed. Tools Appl. 51, 863–879 (2011). https://doi.org/10.1007/s11042-009-0417-2
Hu, P., Ramanan, D.: Finding tiny faces. In Proceedings 30th IEEE Conference on Computer Vision and Pattern Recognition, pp. 1522–1530 (2017). https://doi.org/10.1109/CVPR.2017.166
X. Yin, Y. Tai, Y. Huang, X. Liu, Fan: Feature adaptation network for surveillance face recognition and normalization. In: Proceedings of the Asian Conference on Computer Vision (ACCV) 2020)
Lu, Z., Jiang, X., Kot, A.: Deep coupled ResNet for low-resolution face recognition. IEEE Signal Process. Lett. 25(4), 526–530 (2018). https://doi.org/10.1109/LSP.2018.2810121
Y. Huang, P. Shen, Y. Tai, S. Li, X. Liu, J. Li, F. Huang, R. Ji, Improving face recognition from hard samples via distribution distillation loss. In: Proceedings of the 13th European Conference on Computer vision (ECCV), (Springer 2020), pp. 138–154.
Song, Y., Tang, H., Meng, F., Wang, C., Wu, M., Shu, Z., Tong, G.: A transformer-based low-resolution face recognition method via on-and-offline knowledge distillation. Neurocomputing 509, 193–205 (2022). https://doi.org/10.1016/j.neucom.2022.08.058
Gao, X., Sun, Y., Xiao, Y., Gu, Y., Chai, S., Chen, B.: Adaptive random down-sampling data augmentation and area attention pooling for low resolution face recognition. Expert Syst. Appl. 209, 118275 (2022). https://doi.org/10.1016/j.eswa.2022.118275
Uzun-Per, M., Gökmen, M.: Face recognition with patch-based local Walsh transform. Signal Process. Image Commun. 61, 85–96 (2018). https://doi.org/10.1016/j.image.2017.11.003
Mudunuri, S.P., Biswas, S.: Low resolution face recognition across variations in pose and illumination. IEEE Trans. Pattern Anal. Mach. Intell. 38(5), 1034–1040 (2015)
Yang, F., Yang, W., Gao, R., Liao, Q.: Discriminative multidimensional scaling for low-resolution face recognition. IEEE Signal Process. Lett. 25(3), 388–392 (2017)
Li, P., Prieto, L., Mery, D., Flynn, P.J.: On low-resolution face recognition in the wild: comparisons and new techniques. IEEE Trans. Inf. Forensics Secur. 14, 2000–2012 (2019). https://doi.org/10.1109/TIFS.2018.2890812
Khalid, S.S., Awais, M., Feng, Z.H., Chan, C.H., Farooq, A., Akbari, A., Kittler, J.: Resolution invariant face recognition using a distillation approach. IEEE Trans. Biometr. Behav. Identity Sci. 2, 410–420 (2020). https://doi.org/10.1109/TBIOM.2020.3007356
Acknowledgements
Authors would like to acknowledge Principal and Management of Mepco Schlenk Engineering College, Sivakasi for their encouragement and technical support for this research work. This Project was funded by the Deanship of scientific Research (DsR) at King Abdulaziz University, Jeddah, under grant no. (J: 1-611-1441). The authors, therefore, acknowledge with thanks DSR for technical and financial support.
Author information
Authors and Affiliations
Corresponding author
Ethics declarations
Conflict of interest
The authors declare that there is no conflict of interest.
Additional information
Publisher's Note
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Rights and permissions
Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.
About this article
Cite this article
Ketab, F., Russel, N.S., Selvaraj, A. et al. Parallel deep learning architecture with customized and learnable filters for low-resolution face recognition. Vis Comput 39, 6699–6710 (2023). https://doi.org/10.1007/s00371-022-02757-y
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s00371-022-02757-y