research-article

Improving Face Anti-spoofing via Advanced Multi-perspective Feature Learning

Authors:

Zun LiAuthors Info & Claims

ACM Transactions on Multimedia Computing, Communications and Applications, Volume 19, Issue 6

Article No.: 212, Pages 1 - 18

https://doi.org/10.1145/3575660

Published: 12 July 2023 Publication History

Abstract

Face anti-spoofing (FAS) plays a vital role in securing face recognition systems. Previous approaches usually learn spoofing features from a single perspective, in which only universal cues shared by all attack types are explored. However, such single-perspective-based approaches ignore the differences among various attacks and commonness between certain attacks and bona fides, thus tending to neglect some non-universal cues that contain strong discernibility against certain types. As a result, when dealing with multiple types of attacks, the above approaches may suffer from the uncomprehensive representation of bona fides and spoof faces. In this work, we propose a novel Advanced Multi-Perspective Feature Learning network (AMPFL), in which multiple perspectives are adopted to learn discriminative features, to improve the performance of FAS. Specifically, the proposed network first learns universal cues and several perspective-specific cues from multiple perspectives, then aggregates the above features and further enhances them to perform face anti-spoofing. In this way, AMPFL obtains features that are difficult to be captured by single-perspective-based methods and provides more comprehensive information on bona fides and spoof faces, thus achieving better performance for FAS. Experimental results show that our AMPFL achieves promising results in public databases, and it effectively solves the issues of single-perspective-based approaches.

References

[1]

2016. Biometrics, Information Technology - Biometric Presentation Attack Detection - Part 1: Framework. Standard ISO/IEC JTC 1/SC 37. International Organization for Standardization. Retrieved from https://www.iso.org/obp/ui/iso.

[2]

Akshay Agarwal, Richa Singh, and Mayank Vatsa. 2016. Face anti-spoofing using Haralick features. In IEEE 8th International Conference on Biometrics Theory, Applications and Systems (BTAS). 1–6. DOI:

Digital Library

[3]

Yousef Atoum, Yaojie Liu, Amin Jourabloo, and Xiaoming Liu. 2017. Face anti-spoofing using patch and depth-based CNNs. In IEEE International Joint Conference on Biometrics (IJCB). 319–328. DOI:

Digital Library

[4]

Zinelabidine Boulkenafet, Jukka Komulainen, and Abdenour Hadid. 2015. Face anti-spoofing based on color texture analysis. In IEEE International Conference on Image Processing (ICIP). 2636–2640. DOI:

Digital Library

[5]

Zinelabinde Boulkenafet, Jukka Komulainen, Lei Li, Xiaoyi Feng, and Abdenour Hadid. 2017. OULU-NPU: A mobile face presentation attack database with real-world variations. In 12th IEEE International Conference on Automatic Face Gesture Recognition (FG’17). 612–618. DOI:

Digital Library

[6]

Ivana Chingovska, André Anjos, and Sébastien Marcel. 2012. On the effectiveness of local binary patterns in face anti-spoofing. In International Conference of Biometrics Special Interest Group (BIOSIG). 1–7.

[7]

Tiago de Freitas Pereira, André Anjos, José Mario De Martino, and Sébastien Marcel. 2012. LBP TOP-based countermeasure against face spoofing attacks. In Asian Conference on Computer Vision. 121–132.

[8]

Tiago de Freitas Pereira, André Anjos, José Mario De Martino, and Sébastien Marcel. 2013. Can face anti-spoofing countermeasures work in a real world scenario? In International Conference on Biometrics (ICB). 1–8. DOI:

[9]

Xin Dong, Hao Liu, Weiwei Cai, Pengyuan Lv, and Zekuan Yu. 2021. Open Set Face Anti-spoofing in Unseen Attacks. Association for Computing Machinery, New York, NY, 4082–4090. DOI:

Digital Library

[10]

Nesli Erdogmus and Sébastien Marcel. 2014. Spoofing face recognition with 3D masks. IEEE Trans. Inf. Forens. Secur. 9, 7 (2014), 1084–1097. DOI:

Digital Library

[11]

Litong Feng, Lai-Man Po, Yuming Li, Xuyuan Xu, Fang Yuan, Terence Chun-Ho Cheung, and Kwok-Wai Cheung. 2016. Integration of image quality and motion cues for face anti-spoofing: A neural network approach. J. Vis. Commun. Image Represent. 38 (2016), 451–460. DOI:

Digital Library

[12]

Yao Feng, Fan Wu, Xiaohu Shao, Yanfeng Wang, and Xi Zhou. 2017. Joint 3D face reconstruction and dense alignment with position map regression network. In European Conference on Computer Vision (ECCV). 557–574.

[13]

Anjith George and Sébastien Marcel. 2019. Deep pixel-wise binary supervision for face presentation attack detection. In International Conference on Biometrics (ICB). 1–8. DOI:

[14]

Jianzhu Guo, Xiangyu Zhu, Chenxu Zhao, Dong Cao, Zhen Lei, and Stan Z. Li. 2020. Learning meta face recognition in unseen domains. In IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR). 6162–6171. DOI:

[15]

Amin Jourabloo, Yaojie Liu, and Xiaoming Liu. 2018. Face de-spoofing: Anti-spoofing via noise modeling. In European Conference on Computer Vision. 290–306.

Digital Library

[16]

Klaus Kollreider, Hartwig Fronthaler, Maycel Isaac Faraj, and Josef Bigun. 2007. Real-time face detection and motion analysis with application in “Liveness” assessment. IEEE Trans. Inf. Forens. Secur. 2, 3 (2007), 548–558. DOI:

Digital Library

[17]

Jukka Komulainen, Abdenour Hadid, and Matti Pietikäinen. 2012. Face spoofing detection using dynamic texture. In Asian Conference on Computer Vision. Springer, 146–157.

[18]

Jukka Komulainen, Abdenour Hadid, and Matti Pietikäinen. 2013. Context-based face anti-spoofing. In IEEE 6th International Conference on Biometrics: Theory, Applications and Systems (BTAS). 1–8. DOI:

[19]

Lei Li, Xiaoyi Feng, Zinelabidine Boulkenafet, Zhaoqiang Xia, Mingming Li, and Abdenour Hadid. 2016. An original face anti-spoofing approach using partial convolutional neural network. In 6th International Conference on Image Processing Theory, Tools and Applications (IPTA). 1–6. DOI:

[20]

Sheng Li, Xun Zhu, Guorui Feng, Xinpeng Zhang, and Zhenxing Qian. 2021. Diffusing the Liveness Cues for Face Anti-spoofing. Association for Computing Machinery, New York, NY, 1636–1644. DOI:

[21]

Xiaobai Li, Jukka Komulainen, Guoying Zhao, Pong-Chi Yuen, and Matti Pietikäinen. 2016. Generalized face anti-spoofing by detecting pulse from face videos. In 23rd International Conference on Pattern Recognition (ICPR). 4244–4249. DOI:

[22]

Yidong Li, Wenhua Liu, Yi Jin, and Yuanzhouhan Cao. 2021. SPGAN: Face forgery using spoofing generative adversarial networks. ACM Trans. Multim. Comput. Commun. Appl. 17, 1s (Mar.2021). DOI:

Digital Library

[23]

Siqi Liu, Xiangyuan Lan, and Pong C. Yuen. 2018. Remote photoplethysmography correspondence feature for 3D mask face presentation attack detection. In European Conference on Computer Vision.

Digital Library

[24]

Siqi Liu, Baoyao Yang, Pong C. Yuen, and Guoying Zhao. 2016. A 3D mask face anti-spoofing database with real world variations. In IEEE Conference on Computer Vision and Pattern Recognition Workshops (CVPRW). 1551–1557. DOI:

[25]

Siqi Liu, Pong C. Yuen, Shengping Zhang, and Guoying Zhao. 2016. 3D mask face anti-spoofing with remote photoplethysmography. In European Conference on Computer Vision.

[26]

Yaojie Liu, Amin Jourabloo, and Xiaoming Liu. 2018. Learning deep models for face anti-spoofing: Binary or auxiliary supervision. In IEEE/CVF Conference on Computer Vision and Pattern Recognition. 389–398. DOI:

[27]

Yaojie Liu, Joel Stehouwer, Amin Jourabloo, and Xiaoming Liu. 2019. Deep tree learning for zero-shot face anti-spoofing. In IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR). 4675–4684. DOI:

[28]

Xuesong Niu, Hu Han, Shiguang Shan, and Xilin Chen. 2017. Continuous heart rate measurement from face: A robust rPPG approach with distribution learning. In IEEE International Joint Conference on Biometrics (IJCB). 642–650. DOI:

Digital Library

[29]

Gang Pan, Lin Sun, Zhaohui Wu, and Shihong Lao. 2007. Eyeblink-based anti-spoofing in face recognition from a generic webcamera. In IEEE 11th International Conference on Computer Vision. 1–8. DOI:

[30]

Keyurkumar Patel, Hu Han, and Anil K. Jain. 2016. Secure face unlock: Spoof detection on smartphones. IEEE Trans. Inf. Forens. Secur. 11, 10 (2016), 2268–2283. DOI:

Digital Library

[31]

Rui Shao, Xiangyuan Lan, and Pong C. Yuen. 2017. Deep convolutional dynamic texture learning with adaptive channel-discriminability for 3D mask face anti-spoofing. In IEEE International Joint Conference on Biometrics (IJCB). 748–755. DOI:

Digital Library

[32]

Rui Shao, Xiangyuan Lan, and Pong C. Yuen. 2019. Joint discriminative learning of deep dynamic textures for 3D mask face anti-spoofing. IEEE Trans. Inf. Forens. Secur. 14, 4 (2019), 923–938. DOI:

[33]

Talha Ahmad Siddiqui, Samarth Bharadwaj, Tejas I. Dhamecha, Akshay Agarwal, Mayank Vatsa, Richa Singh, and Nalini Ratha. 2016. Face anti-spoofing with multifeature videolet aggregation. In 23rd International Conference on Pattern Recognition (ICPR). 1035–1040. DOI:

[34]

Wenyun Sun, Yu Song, Changsheng Chen, Jiwu Huang, and Alex C. Kot. 2020. Face spoofing detection based on local ternary label supervision in fully convolutional networks. IEEE Trans. Inf. Forens. Secur. 15 (2020), 3181–3196. DOI:

[35]

Xiaoyang Tan, Yi Li, Jun Liu, and Lin Jiang. 2010. Face liveness detection from a single image with sparse low rank bilinear discriminative model. In European Conference on Computer Vision. Springer, 504–517.

[36]

Laurens van der Maaten and Geoffrey Hinton. 2008. Viualizing data using t-SNE. J. Mach. Learn. Res. 9 (112008), 2579–2605.

[37]

Zhuming Wang, Yaowen Xu, Lifang Wu, Hu Han, Yukun Ma, and Guozhang Ma. 2021. Multi-perspective features learning for face anti-spoofing. In IEEE/CVF International Conference on Computer Vision Workshops (ICCVW). 4099–4105. DOI:

[38]

Zezheng Wang, Zitong Yu, Chenxu Zhao, Xiangyu Zhu, Yunxiao Qin, Qiusheng Zhou, Feng Zhou, and Zhen Lei. 2020. Deep spatial gradient and temporal depth learning for face anti-spoofing. In IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR). 5041–5050. DOI:

[39]

Zezheng Wang, Chenxu Zhao, Yunxiao Qin, Qiusheng Zhou, Guojun Qi, Jun Wan, and Zhen Lei. 2019. Exploiting temporal and depth information for multi-frame face anti-spoofing. arxiv:1811.05118v3.

[40]

Jingtian Xia, Siyang Song, Yan Tang, and Linlin Shen. 2020. Analyzing facial temporal patterns for face anti-spoofing. In 9th International Conference on Computing and Pattern Recognition (ICCPR’20). Association for Computing Machinery, New York, NY, 200–207. DOI:

Digital Library

[41]

Yaowen Xu, Zhuming Wang, Hu Han, Lifang Wu, and Yongluo Liu. 2021. Exploiting non-uniform inherent cues to improve presentation attack detection. In IEEE International Joint Conference on Biometrics (IJCB). 1–8. DOI:

Digital Library

[42]

Zhenqi Xu, Shan Li, and Weihong Deng. 2015. Learning temporal features using LSTM-CNN architecture for face anti-spoofing. In 3rd IAPR Asian Conference on Pattern Recognition (ACPR). 141–145. DOI:

[43]

Xiao Yang, Wenhan Luo, Linchao Bao, Yuan Gao, Dihong Gong, Shibao Zheng, Zhifeng Li, and Wei Liu. 2019. Face anti-spoofing: Model matters, so does data. In IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR). 3502–3511. DOI:

[44]

Chenlin Yao, Shihe Wang, Jialu Zhang, Wentao He, Heshan Du, Jianfeng Ren, Ruibin Bai, and Jiang Liu. 2021. rPPG-based spoofing detection for face mask attack using EfficientNet on weighted spatial-temporal representation. In IEEE International Conference on Image Processing (ICIP).

[45]

Zitong Yu, Jun Wan, Yunxiao Qin, Xiaobai Li, Stan Z. Li, and Guoying Zhao. 2021. NAS-FAS: Static-dynamic central difference network search for face anti-spoofing. IEEE Trans. Pattern Anal. Mach. Intell. 43, 9 (2021), 3005–3023. DOI:

[46]

Zitong Yu, Chenxu Zhao, Zezheng Wang, Yunxiao Qin, Zhuo Su, Xiaobai Li, Feng Zhou, and Guoying Zhao. 2020. Searching central difference convolutional networks for face anti-spoofing. In IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR). 5294–5304. DOI:

[47]

Ke-Yue Zhang, Taiping Yao, Jian Zhang, Ying Tai, Shouhong Ding, Jilin Li, Feiyue Huang, Haichuan Song, and Lizhuang Ma. 2020. Face anti-spoofing via disentangled representation learning. In European Conference on Computer Vision. Springer, 641–657.

Digital Library

[48]

Zhiwei Zhang, Junjie Yan, Sifei Liu, Zhen Lei, Dong Yi, and Stan Z. Li. 2012. A face antispoofing database with diverse attacks. In 5th IAPR International Conference on Biometrics (ICB). 26–31. DOI:

Cited By

Zhang YLin XYang HHe JQing LHe XLi YChen H(2024)A Multi-Attention Feature Distillation Neural Network for Lightweight Single Image Super-ResolutionInternational Journal of Intelligent Systems10.1155/2024/32552332024Online publication date: 15-Feb-2024
https://dl.acm.org/doi/10.1155/2024/3255233
Zhang DZhu WLiao XQi FYang GDing X(2024)Spatiotemporal Inconsistency Learning and Interactive Fusion for Deepfake Video DetectionACM Transactions on Multimedia Computing, Communications, and Applications10.1145/3664654Online publication date: 13-May-2024
https://dl.acm.org/doi/10.1145/3664654
Zhang PLiu MSong XCao DGao ZNie L(2024)Universal Relocalizer for Weakly Supervised Referring Expression GroundingACM Transactions on Multimedia Computing, Communications, and Applications10.1145/365604520:7(1-23)Online publication date: 16-May-2024
https://dl.acm.org/doi/10.1145/3656045
Show More Cited By

Index Terms

Improving Face Anti-spoofing via Advanced Multi-perspective Feature Learning
1. Computing methodologies
  1. Artificial intelligence
    1. Computer vision
      1. Computer vision tasks
        Biometrics

Recommendations

Learning Generalizable and Identity-Discriminative Representations for Face Anti-Spoofing
Survey Paper and Regular Paper

Face anti-spoofing aims to detect presentation attack to face recognition--based authentication systems. It has drawn growing attention due to the high security demand. The widely adopted CNN-based methods usually well recognize the spoofing faces when ...
Integration of image quality and motion cues for face anti-spoofing

A multi-cues integration framework is proposed using a hierarchical neural network.Bottleneck representations are effective in multi-cues feature fusion.Shearlet is utilized to perform face image quality assessment.Motion-based face liveness features ...
Latent Spatial Features Based on Generative Adversarial Networks for Face Anti-spoofing
Biometric Recognition
Abstract
With the wide deployment of the face recognition system, many face attacks, such as print attack, video attack and 3D face mask, have emerged. Face anti-spoofing is very important to protect face recognition system from attack. This paper proposes ...

Comments

Information & Contributors

Information

Published In

cover image ACM Transactions on Multimedia Computing, Communications, and Applications

ACM Transactions on Multimedia Computing, Communications, and Applications Volume 19, Issue 6

November 2023

858 pages

ISSN:1551-6857

EISSN:1551-6865

DOI:10.1145/3599695

Editor:
Abdulmotaleb El Saddik
Mohamed Bin Zayed University of Artificial Intelligence, UAE and University of Ottawa, Canada

Issue’s Table of Contents

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected].

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 12 July 2023

Online AM: 08 December 2022

Accepted: 20 October 2022

Revised: 17 August 2022

Received: 27 February 2022

Published in TOMM Volume 19, Issue 6

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Research-article

Funding Sources

Natural Science Foundation of China
China Postdoctoral Science Foundation
Beijing Postdoctoral Science Foundation

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

16
Total Citations
View Citations
266
Total Downloads

Downloads (Last 12 months)140
Downloads (Last 6 weeks)3

Reflects downloads up to 12 Sep 2024

Other Metrics

View Author Metrics

Citations

Cited By

Zhang YLin XYang HHe JQing LHe XLi YChen H(2024)A Multi-Attention Feature Distillation Neural Network for Lightweight Single Image Super-ResolutionInternational Journal of Intelligent Systems10.1155/2024/32552332024Online publication date: 15-Feb-2024
https://dl.acm.org/doi/10.1155/2024/3255233
Zhang DZhu WLiao XQi FYang GDing X(2024)Spatiotemporal Inconsistency Learning and Interactive Fusion for Deepfake Video DetectionACM Transactions on Multimedia Computing, Communications, and Applications10.1145/3664654Online publication date: 13-May-2024
https://dl.acm.org/doi/10.1145/3664654
Zhang PLiu MSong XCao DGao ZNie L(2024)Universal Relocalizer for Weakly Supervised Referring Expression GroundingACM Transactions on Multimedia Computing, Communications, and Applications10.1145/365604520:7(1-23)Online publication date: 16-May-2024
https://dl.acm.org/doi/10.1145/3656045
Ben HWang SWang MHong RGurrin CKongkachandra RSchoeffmann KDang-Nguyen DRossetto LSatoh SZhou L(2024)Pseudo Content Hallucination for Unpaired Image CaptioningProceedings of the 2024 International Conference on Multimedia Retrieval10.1145/3652583.3658080(320-329)Online publication date: 30-May-2024
https://dl.acm.org/doi/10.1145/3652583.3658080
Antil ADhiman C(2024)MF2ShrT: Multimodal Feature Fusion Using Shared Layered Transformer for Face Anti-spoofingACM Transactions on Multimedia Computing, Communications, and Applications10.1145/364081720:6(1-21)Online publication date: 8-Mar-2024
https://dl.acm.org/doi/10.1145/3640817
Li MZhou THuang ZYang JYang JGong C(2024)Dynamic Weighted Adversarial Learning for Semi-Supervised Classification under Intersectional Class MismatchACM Transactions on Multimedia Computing, Communications, and Applications10.1145/363531020:4(1-24)Online publication date: 11-Jan-2024
https://dl.acm.org/doi/10.1145/3635310
Shi PHu MShi XRen F(2024)Deep Modular Co-Attention Shifting Network for Multimodal Sentiment AnalysisACM Transactions on Multimedia Computing, Communications, and Applications10.1145/363470620:4(1-23)Online publication date: 11-Jan-2024
https://dl.acm.org/doi/10.1145/3634706
Feng ZXu JMa LZhang S(2024)Efficient Video Transformers via Spatial-temporal Token Merging for Action RecognitionACM Transactions on Multimedia Computing, Communications, and Applications10.1145/363378120:4(1-21)Online publication date: 11-Jan-2024
https://dl.acm.org/doi/10.1145/3633781
Li MZhou THan BLiu TLiang XZhao JGong C(2024)Class-Wise Contrastive Prototype Learning for Semi-Supervised Classification Under Intersectional Class MismatchIEEE Transactions on Multimedia10.1109/TMM.2024.337712326(8145-8156)Online publication date: 18-Mar-2024
https://dl.acm.org/doi/10.1109/TMM.2024.3377123
Wang TLi ZLiu RWang YNie L(2024)An Efficient Attribute-Preserving Framework for Face SwappingIEEE Transactions on Multimedia10.1109/TMM.2024.335457326(6554-6565)Online publication date: 16-Jan-2024
https://dl.acm.org/doi/10.1109/TMM.2024.3354573
Show More Cited By

View Options

Get Access

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Article

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Full Text

View this article in Full Text.

Media

Figures

Other

Tables

View full text|Download PDF

View Issue’s Table of Contents