Location via proxy:   [ UP ]  
[Report a bug]   [Manage cookies]                
Skip to main content
SpringerLink
Log in

Noise robust face super-resolution via learning of spatial attentive features

  • Published:
Multimedia Tools and Applications Aims and scope Submit manuscript

    We’re sorry, something doesn't seem to be working properly.

    Please try refreshing the page. If that doesn't work, please contact support so we can address the problem.

Abstract

Face super-resolution (SR) is a process of restoring the high-resolution (HR) face images from the low-resolution (LR) inputs. Recently, deep learning-based methods have shown excellent performance in the field of image super-resolution. Many face SR methods heavily rely on facial priors, e.g., parsing maps and landmarks to reconstruct the HR images. However, such methods may estimate inaccurate facial priors and cause the generation of poor-quality HR images. Therefore, this paper proposes a face SR framework built on the proposed feature attention unit. An Exigent Feature (ExFeat) block with spatial attention is used to design the proposed feature attention unit. It assists the proposed framework in learning the micro, high-level detailed features and subsequently reducing the noise. Spatial attention helps the framework to focus on specific facial features as well as allowing the convolutional layers to give more attention to crucial face attributes related features and less to the remaining features. The proposed framework repeats the Feature Attention Unit (FAU) to learn the different facial components and informative features which helps in improving the overall quality of the resultant face images. Experimental outcomes performed on CelebAHQ and LFW face datasets exhibit that the proposed framework outperforms over the other competitive methods. The extensive experiments demonstrate an improvement of more than 0.42 dB in PSNR and 0.027 over SSIM compared to other state-of-the-art models.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Subscribe and save

Springer+ Basic
$34.99 /Month
  • Get 10 units per month
  • Download Article/Chapter or eBook
  • 1 Unit = 1 Article or 1 Chapter
  • Cancel anytime
Subscribe now

Buy Now

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9

Similar content being viewed by others

Explore related subjects

Find the latest articles, discoveries, and news in related topics.

Data Availability

Data sharing is not applicable to this article as no datasets were generated.

References

  1. Ajagbe SA, Amuda KA, Oladipupo MA, Oluwaseyi FA, Okesola KI (2021) Multi-classification of alzheimer disease on magnetic resonance images (mri) using deep convolutional neural network (dcnn) approaches. Int J Adv Comput Res 11(53):51

    Article  Google Scholar 

  2. Al-Obaydy WNI, Suandi SA (2020) Automatic pose normalization for open-set single-sample face recognition in video surveillance. Multimed Tools Appl 79(3):2897–2915

    Article  Google Scholar 

  3. Baker S, Kanade T (2000) Hallucinating faces. In: Proceedings Fourth IEEE international conference on automatic face and gesture recognition (Cat. No. PR00580). IEEE, pp 83–88

  4. Bao Q, Gang B, Yang W, Zhou J, Liao Q (2022) Attention-driven graph neural network for deep face super-resolution. IEEE Trans Image Process 31:6455–6470

    Article  Google Scholar 

  5. Bulat A, Tzimiropoulos G (2017) How far are we from solving the 2d & 3d face alignment problem?(and a dataset of 230,000 3d facial landmarks). In: Proceedings of the IEEE international conference on computer vision, pp 1021–1030

  6. Cao Q, Lin L, Shi Y, Liang X, Li G (2017) Attention-aware face hallucination via deep reinforcement learning. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 690–698

  7. Chen C, Gong D, Wang H, Li Z, Wong KYK (2020) Learning spatial attention for face super-resolution. IEEE Trans Image Process 30:1219–1231

    Article  Google Scholar 

  8. Chen L, Pan J, Hu R, Han Z, Liang C, Wu Y (2019) Modeling and optimizing of the multi-layer nearest neighbor network for face image super-resolution. IEEE Trans Circuits Syst Video Technol 30(12):4513–4525

    Article  Google Scholar 

  9. Chen W, Yao P, Gai S, Da F (2022) Multi-scale feature aggregation network for image super-resolution. Appl Intell 52(4):3577–3586

    Article  Google Scholar 

  10. Chen Y, Phonevilay V, Tao J, Chen X, Xia R, Zhang Q, Yang K, Xiong J, Xie J (2021) The face image super-resolution algorithm based on combined representation learning. Multimed Tools Appl 80(20):30839–30861

    Article  Google Scholar 

  11. Chen Y, Tai Y, Liu X, Shen C, Yang J (2018) Fsrnet: end-to-end learning face super-resolution with facial priors. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 2492–2501

  12. Chen Z, Tong Y (2017) Face super-resolution through wasserstein gans. arXiv:1705.02438

  13. Dastmalchi H, Aghaeinia H (2022) Super-resolution of very low-resolution face images with a wavelet integrated, identity preserving, adversarial network. Signal processing: Image Communication pp 116755

  14. Dong C, Loy CC, He K, Tang X (2014) Learning a deep convolutional network for image super-resolution. In: European conference on computer vision. Springer, pp 184–199

  15. Gao G, Xu Z, Li J, Yang J, Zeng T, Qi GJ (2022) Ctcnet: a cnn-transformer cooperation network for face image super-resolution. arXiv:2204.08696

  16. Guo K, Hu M, Ren S, Li F, Zhang J, Guo H, Kui X (2022) Deep illumination-enhanced face super-resolution network for low-light images. ACM Transactions on Multimedia Computing, Communications and Applications (TOMM) 18(3):1–19

    Article  Google Scholar 

  17. Kalarot R, Li T, Porikli F (2020) Component attention guided face super-resolution network: cagface. In: Proceedings of the IEEE/CVF winter conference on applications of computer vision, pp 370–380

  18. Karras T, Aila T, Laine S, Lehtinen J (2017) Progressive growing of gans for improved quality, stability, and variation. arXiv:1710.10196

  19. Kim J, Lee JK, Lee KM (2016) Accurate image super-resolution using very deep convolutional networks. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 1646–1654

  20. Kumar TA, Rajmohan R, Pavithra M, Ajagbe SA, Hodhod R, Gaber T (2022) Automatic face mask detection system in public transportation in smart cities using iot and deep learning. Electronics 11(6):904

    Article  Google Scholar 

  21. Learned-Miller E, Huang GB, RoyChowdhury A, Li H, Hua G (2016) Labeled faces in the wild: a survey. In: Advances in face detection and facial image analysis. Springer, pp 189–248

  22. Ledig C, Theis L, Huszár F, Caballero J, Cunningham A, Acosta A, Aitken A, Tejani A, Totz J, Wang Z et al (2017) Photo-realistic single image super-resolution using a generative adversarial network. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 4681–4690

  23. Li J, Du S, Wu C, Leng Y, Song R, Li Y (2022) Drcr net: dense residual channel re-calibration network with non-local purification for spectral super resolution. In: Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pp 1259–1268

  24. Li L, Tang J, Ye Z, Sheng B, Mao L, Ma L (2021) Unsupervised face super-resolution via gradient enhancement and semantic guidance. Vis Comput 37(9):2855–2867

    Article  Google Scholar 

  25. Li P, Tu S, Xu L (2022) Deep rival penalized competitive learning for low-resolution face recognition. Neural Networks

  26. Li Y, Zhang Z, Chen B, Lu G, Zhang D (2022) Deep margin-sensitive representation learning for cross-domain facial expression recognition. IEEE Transactions on Multimedia

  27. Lim B, Son S, Kim H, Nah S, Mu Lee K (2017) Enhanced deep residual networks for single image super-resolution. In: Proceedings of the IEEE conference on computer vision and pattern recognition workshops, pp 136–144

  28. Liu S, Xiong C, Shi X, Gao Z (2021) Progressive face super-resolution with cascaded recurrent convolutional network. Neurocomputing 449:357–367

    Article  Google Scholar 

  29. Liu X, Chen S, Song L, Woźniak M, Liu S (2022) Self-attention negative feedback network for real-time image super-resolution. Journal of King Saud University-Computer and Information Sciences 34(8):6179–6186

    Article  Google Scholar 

  30. Liu Z, Lu H, Pan X, Xu M, Lan R, Luo X (2022) Diagnosis of alzheimer’s disease via an attention-based multi-scale convolutional neural network. Knowl-Based Syst 238:107942

    Article  Google Scholar 

  31. Lu E, Hu X (2022) Image super-resolution via channel attention and spatial attention. Appl Intell 52(2):2260–2268

    Article  MathSciNet  Google Scholar 

  32. Lu T, Wang Y, Zhang Y, Wang Y, Wei L, Wang Z, Jiang J (2021) Face hallucination via split-attention in split-attention network. In: Proceedings of the 29th ACM International Conference on Multimedia, pp 5501–5509

  33. Luo Y, Cao X, Zhang J, Guo J, Shen H, Wang T, Feng Q (2022) Ce-fpn: enhancing channel information for object detection. Multimed Tools Appl, pp 1–20

  34. Ma C, Jiang Z, Rao Y, Lu J, Zhou J (2020) Deep face super-resolution with iterative collaboration between attentive recovery and landmark estimation. In: Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pp 5569–5578

  35. Ma X, Zhang J, Qi C (2010) Hallucinating face by position-patch. Pattern Recogn 43(6):2224–2236

    Article  Google Scholar 

  36. Mikaeli E, Aghagolzadeh A, Azghani M (2020) Single-image super-resolution via patch-based and group-based local smoothness modeling. Vis Comput 36(8):1573–1589

    Article  Google Scholar 

  37. Molahasani Majdabadi M, Ko SB (2020) Capsule gan for robust face super resolution. Multimed Tools Appl 79(41):31205–31218

    Article  Google Scholar 

  38. Nan F, Zeng Q, Xing Y, Qian Y (2020) Single image super-resolution reconstruction based on the resnext network. Multimed Tools Appl 79 (45):34459–34470

    Article  Google Scholar 

  39. Pandey G, Ghanekar U (2021) Single image super-resolution using multi-scale feature enhancement attention residual network. Optik 231:166359

    Article  Google Scholar 

  40. Rajput SS (2022) Gaussian noise robust face hallucination via average filtering based data fidelity and locality regularization. Appl Intell, pp 1–14

  41. Rajput SS (2022) Mixed gaussian-impulse noise robust face hallucination via noise suppressed low-and-high resolution space-based neighbor representation. Multimed Tools Appl 81(11):15997–16019

    Article  Google Scholar 

  42. Rajput SS, Arya K (2020) A robust face super-resolution algorithm and its application in low-resolution face recognition system. Multimed Tools Appl 79(33):23909–23934

    Article  Google Scholar 

  43. Rakshit RD, Kisku DR, Gupta P, Sing JK (2021) Cross-resolution face identification using deep-convolutional neural network. Multimed Tools Appl 80(14):20733–20758

    Article  Google Scholar 

  44. Serengil SI, Ozpinar A (2020) Lightface: a hybrid deep face recognition framework. In: 2020 Innovations in intelligent systems and applications conference (ASYU). IEEE, pp 1–5

  45. Verma M, Bhui JK, Vipparthi SK, Singh G (2018) Expertnet: exigent features preservative network for facial expression recognition. In: Proceedings of the 11th Indian conference on computer vision, graphics and image processing, pp 1–8

  46. Wang C, Jiang J, Zhong Z, Liu X (2022) Propagating facial prior knowledge for multi-task learning in face super-resolution. IEEE Transactions on Circuits and Systems for Video Technology

  47. Wang H, Hu Q, Wu C, Chi J, Yu X, Wu H (2021) Dclnet: dual closed-loop networks for face super-resolution. Knowl-Based Syst 222:106987

    Article  Google Scholar 

  48. Wang M, Chen Z, Wu Q, Jian M (2020) Improved face super-resolution generative adversarial networks. Mach Vis Appl 31(4):1–12

    Article  Google Scholar 

  49. Wang Y, Li X, Nan F, Liu F, Li H, Wang H, Qian Y (2022) Image super-resolution reconstruction based on generative adversarial network model with feedback and attention mechanisms. Multimed Tools Appl 81(5):6633–6652

    Article  Google Scholar 

  50. Ying L, Dinghua S, Fuping W, Pang LK, Kiang CT, Yi L (2021) Learning wavelet coefficients for face super-resolution. Vis Comput 37 (7):1613–1622

    Article  Google Scholar 

  51. Yu X, Porikli F (2016) Ultra-resolving face images by discriminative generative networks. In: European conference on computer vision. Springer, pp 318–333

  52. Zhang K, Zhang Z, Li Z, Qiao Y (2016) Joint face detection and alignment using multitask cascaded convolutional networks. IEEE Signal Process Lett 23(10):1499–1503

    Article  Google Scholar 

  53. Zhang Y, Sun Y, Liu S (2022) Deformable and residual convolutional network for image super-resolution. Appl Intell 52(1):295–304

    Article  Google Scholar 

Download references

Funding

No funding was received for conducting this work.

Author information

Authors and Affiliations

  1. Multimedia and Information Security Research Group, ABV-Indian Institute of Information Technology and Management, Gwalior, 474015, India

    Anurag Singh Tomar & K. V. Arya

  2. Department of Computer Science & Engineering, National Institute of Technology Patna, Patna, 800005, India

    Shyam Singh Rajput

Authors
  1. Anurag Singh Tomar
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. K. V. Arya
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Shyam Singh Rajput
    View author publications

    You can also search for this author in PubMed Google Scholar

Contributions

All authors have contributed equally.

Corresponding author

Correspondence to Anurag Singh Tomar.

Ethics declarations

Financial interests

None.

Conflict of Interests

I state that there is no conflict of interest.

Additional information

Publisher’s note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Tomar, A.S., Arya, K.V. & Rajput, S.S. Noise robust face super-resolution via learning of spatial attentive features. Multimed Tools Appl 82, 25449–25465 (2023). https://doi.org/10.1007/s11042-023-14472-4

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11042-023-14472-4

Keywords

Search

Navigation