Cited By
View all- Kang HCha SKim S(2024)Security Improvements of JPEG Images Using Image De-IdentificationElectronics10.3390/electronics1307133213:7(1332)Online publication date: 2-Apr-2024
Detection of Adversarial Facial Accessory Presentation Attacks Using Local Face Differential
Authors: 
Fei Peng, Le Qin, Min Long, Jin LiAuthors Info & Claims
Fei Peng, Le Qin, Min Long, Jin LiAuthors Info & ClaimsArticle No.: 192, Pages 1 - 28
Abstract
To counter adversarial facial accessory presentation attacks (PAs), a detection method based on local face differential is proposed in this article. It extracts the local face differential features from a suspected face image and a reference face image, and then adaptively fuses the differential features of different local face regions to detect adversarial facial accessory PAs. Meanwhile, the principle of the proposed method is explained by theoretically investigating the local facial differences between a bona fide presentation and an adversarial facial accessory PA when they are compared with a reference face image. To evaluate the proposed method, this article builds a database with different adversarial examples (AEs), presentation attack instruments (PAIs), illumination conditions, and cameras. The experimental results show that it can effectively distinguish between adversarial facial accessory PAs and bona fide presentations, and it has good generalization ability to unseen AEs, PAIs, illumination conditions, and cameras. Moreover, it outperforms the existing AE detection and presentation attack detection methods in detecting adversarial facial accessory PAs.
References
[1]
Akshay Agarwal, Akarsha Sehwag, Richa Singh, and Mayank Vatsa. 2019. Deceiving face presentation attack detection via image transforms. In 2019 IEEE 5th International Conference on Multimedia Big Data (BigMM). IEEE, 373–382.
[2]
Akshay Agarwal, Richa Singh, and Mayank Vatsa. 2016. Face anti-spoofing using Haralick features. In 2016 IEEE 8th International Conference on Biometrics Theory, Applications and Systems (BTAS). IEEE, 1–6.
[3]
Shervin Rahimzadeh Arashloo. 2023. Unknown face presentation attack detection via localized learning of multiple kernels. IEEE Transactions on Information Forensics and Security 18 (2023), 1421–1432.
[4]
Daniel Benalcazar, Juan E. Tapia, Sebastian Gonzalez, and Christoph Busch. 2023. Synthetic ID card image generation for improving presentation attack detection. IEEE Transactions on Information Forensics and Security 18 (2023), 1814–1824.
[5]
Lokendra Birla, Puneet Gupta, and Shravan Kumar. 2022. SUNRISE: Improving 3D mask face anti-spoofing for short videos using pre-emptive split and merge. IEEE Transactions on Dependable and Secure Computing (2022).
[6]
Zinelabidine Boulkenafet, Jukka Komulainen, and Abdenour Hadid. 2016. Face spoofing detection using colour texture analysis. IEEE Transactions on Information Forensics and Security 11, 8 (2016), 1818–1830.
[7]
Rizhao Cai, Haoliang Li, Shiqi Wang, Changsheng Chen, and Alex C. Kot. 2020. DRL-FAS: A novel framework based on deep reinforcement learning for face anti-spoofing. IEEE Transactions on Information Forensics and Security 16 (2020), 937–951.
[8]
Chih-Chung Chang and Chih-Jen Lin. 2011. LIBSVM: A library for support vector machines. ACM Transactions on Intelligent Systems and Technology (TIST) 2, 3 (2011), 1–27.
[9]
Changsheng Chen, Shuzheng Zhang, Fengbo Lan, and Jiwu Huang. 2021. Domain generalization for document authentication against practical recapturing attacks. arXiv preprint arXiv:2101.01404 (2021).
[10]
Debayan Deb and Anil K. Jain. 2020. Look locally infer globally: A generalizable face anti-spoofing approach. IEEE Transactions on Information Forensics and Security 16 (2020), 1143–1157.
[11]
Jiankang Deng, Jia Guo, Niannan Xue, and Stefanos Zafeiriou. 2019. ARCFACE: Additive angular margin loss for deep face recognition. In IEEE/CVF Conference on Computer Vision and Pattern Recognition. 4690–4699.
[12]
J. Matías Di Martino, Qiang Qiu, and Guillermo Sapiro. 2020. Rethinking shape from shading for spoofing detection. IEEE Transactions on Image Processing 30 (2020), 1086–1099.
[13]
Pawel Drozdowski, Christian Rathgeb, Antitza Dantcheva, Naser Damer, and Christoph Busch. 2020. Demographic bias in biometrics: A survey on an emerging challenge. IEEE Transactions on Technology and Society 1, 2 (2020), 89–103.
[14]
Meiling Fang, Naser Damer, Florian Kirchbuchner, and Arjan Kuijper. 2022. Real masks and spoof faces: On the masked face presentation attack detection. Pattern Recognition 123 (2022), 108398.
[15]
Anjith George and Sébastien Marcel. 2020. Learning one class representations for face presentation attack detection using multi-channel convolutional neural networks. IEEE Transactions on Information Forensics and Security 16 (2020), 361–375.
[16]
Gaurav Goswami, Akshay Agarwal, Nalini Ratha, Richa Singh, and Mayank Vatsa. 2019. Detecting and mitigating adversarial perturbations for robust face recognition. International Journal of Computer Vision 127, 6 (2019), 719–742.
[17]
ICAO. 2015. Doc 9303 Machine Readable Travel Documents Seventh Edition - Part 9: Deployment of Biometric Identification and Electronic Storage of Data in MRTDs. International Civil Aviation Organization.
[18]
ISO/IEC JTC1 SC37 Biometrics. 2017. ISO/IEC 30107-3 Information Technology - Biometric presentation attack detection - Part 3: Testing and Reporting. International Organization for Standardization.
[19]
Phillip Isola, Jun-Yan Zhu, Tinghui Zhou, and Alexei A. Efros. 2017. Image-to-image translation with conditional adversarial networks. In IEEE Conference on Computer Vision and Pattern Recognition. 1125–1134.
[20]
Shan Jia, Xin Li, Chuanbo Hu, Guodong Guo, and Zhengquan Xu. 2020. 3D face anti-spoofing with factorized bilinear coding. IEEE Transactions on Circuits and Systems for Video Technology 31, 10 (2020), 4031–4045.
[21]
Edgar Kaziakhmedov, Klim Kireev, Grigorii Melnikov, Mikhail Pautov, and Aleksandr Petiushko. 2019. Real-world attack on MTCNN face detection system. In 2019 International Multi-Conference on Engineering, Computer and Information Sciences (SIBIRCON). IEEE, 0422–0427.
[22]
Davis E. King. 2009. Dlib-ML: A machine learning toolkit. The Journal of Machine Learning Research 10 (2009), 1755–1758.
[23]
Stepan Komkov and Aleksandr Petiushko. 2019. ADVHAT: Real-world adversarial attack on arcface face ID system. arXiv preprint arXiv:1908.08705 (2019).
[24]
Haoliang Li, Wen Li, Hong Cao, Shiqi Wang, Feiyue Huang, and Alex C. Kot. 2018. Unsupervised domain adaptation for face anti-spoofing. IEEE Transactions on Information Forensics and Security 13, 7 (2018), 1794–1809.
[25]
Haoliang Li, Shiqi Wang, Peisong He, and Anderson Rocha. 2020. Face anti-spoofing with deep neural network distillation. IEEE Journal of Selected Topics in Signal Processing 14, 5 (2020), 933–946.
[26]
Yan Li, Yingjiu Li, Ke Xu, Qiang Yan, and Robert H. Deng. 2018. Empirical study of face authentication systems under OSNFD attacks. IEEE Transactions on Dependable and Secure Computing 15, 2 (2018), 231–245.
[27]
Yan Li, Yingjiu Li, Qiang Yan, Hancong Kong, and Robert H. Deng. 2015. Seeing your face is not enough: An inertial sensor-based liveness detection for face authentication. In 22nd ACM SIGSAC Conference on Computer and Communications Security. 1558–1569.
[28]
Yidong Li, Wenhua Liu, Yi Jin, and Yuanzhouhan Cao. 2021. SPGAN: Face forgery using spoofing generative adversarial networks. ACM Trans. Multimedia Comput. Commun. Appl. 17, 1s, Article 19 (Mar 2021), 20 pages.
[29]
Ajian Liu, Chenxu Zhao, Zitong Yu, Jun Wan, Anyang Su, Xing Liu, Zichang Tan, Sergio Escalera, Junliang Xing, Yanyan Liang, Guodong Guo, Zhen Lei, Stan Z. Li, and Du Zhang. 2022. Contrastive context-aware learning for 3D high-fidelity mask face presentation attack detection. IEEE Transactions on Information Forensics and Security 17 (2022), 2497–2507.
[30]
Si-Qi Liu, Xiangyuan Lan, and Pong C. Yuen. 2021. Multi-channel remote photoplethysmography correspondence feature for 3D mask face presentation attack detection. IEEE Transactions on Information Forensics and Security (2021).
[31]
Weiyang Liu, Yandong Wen, Zhiding Yu, Ming Li, Bhiksha Raj, and Le Song. 2017. Sphereface: Deep hypersphere embedding for face recognition. In IEEE Conference on Computer Vision and Pattern Recognition. 212–220.
[32]
Yaojie Liu, Amin Jourabloo, and Xiaoming Liu. 2018. Learning deep models for face anti-spoofing: Binary or auxiliary supervision. In IEEE Conference on Computer Vision and Pattern Recognition. 389–398.
[33]
Yaojie Liu, Joel Stehouwer, Amin Jourabloo, and Xiaoming Liu. 2019. Deep tree learning for zero-shot face anti-spoofing. In IEEE/CVF Conference on Computer Vision and Pattern Recognition. 4680–4689.
[34]
Puspita Majumdar, Akshay Agarwal, Richa Singh, and Mayank Vatsa. 2019. Evading face recognition via partial tampering of faces. In IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops. 11–20.
[35]
João Baptista Cardia Neto, Claudio Ferrari, Aparecido Nilceu Marana, Stefano Berretti, and Alberto Del Bimbo. 2023. Learning streamed attention network from descriptor images for cross-resolution 3D face recognition. ACM Trans. Multimedia Comput. Commun. Appl. 19, 1s, Article 30 (Jan 2023), 20 pages.
[36]
Dinh-Luan Nguyen, Sunpreet S. Arora, Yuhang Wu, and Hao Yang. 2020. Adversarial light projection attacks on face recognition systems: A feasibility study. In IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops. 814–815.
[37]
Omkar Parkhi, Andrea Vedaldi, and Andrew Zisserman. 2015. Deep face recognition. In BMVC 2015-Proceedings of the British Machine Vision Conference 2015. British Machine Vision Association.
[38]
Fei Peng, Le Qin, and Min Long. 2018. CCoLBP: Chromatic co-occurrence of local binary pattern for face presentation attack detection. In 2018 27th International Conference on Computer Communication and Networks (ICCCN). IEEE, 1–9.
[39]
Fei Peng, Le Qin, and Min Long. 2018. Face presentation attack detection using guided scale texture. Multimedia Tools and Applications 77, 7 (2018), 8883–8909.
[40]
Fei Peng, Le Qin, and Min Long. 2020. Face presentation attack detection based on chromatic co-occurrence of local binary pattern and ensemble learning. Journal of Visual Communication and Image Representation 66 (2020), 102746.
[41]
Allan Pinto, Siome Goldenstein, Alexandre Ferreira, Tiago Carvalho, Helio Pedrini, and Anderson Rocha. 2020. Leveraging shape, reflectance and albedo from shading for face presentation attack detection. IEEE Transactions on Information Forensics and Security 15 (2020), 3347–3358.
[42]
Le Qin, Fei Peng, Min Long, Raghavendra Ramachandra, and Christoph Busch. 2021. Vulnerabilities of unattended face verification systems to facial components-based presentation attacks: An empirical study. ACM Transactions on Privacy and Security 25, 1 (2021), 1–28.
[43]
Le Qin, Fei Peng, Sushma Venkatesh, Raghavendra Ramachandra, Min Long, and Christoph Busch. 2020. Low visual distortion and robust morphing attacks based on partial face image manipulation. IEEE Transactions on Biometrics, Behavior, and Identity Science (2020).
[44]
Raghavendra Ramachandra and Christoph Busch. 2017. Presentation attack detection methods for face recognition systems: A comprehensive survey. ACM Computing Surveys (CSUR) 50, 1 (2017), 1–37.
[45]
Christian Rathgeb, Pawel Drozdowski, and Christoph Busch. 2020. Detection of makeup presentation attacks based on deep face representations. In IEEE International Conference on Pattern Recognition.
[46]
Christian Rathgeb, Pawel Drozdowski, and Christoph Busch. 2020. Makeup presentation attacks: Review and detection performance benchmark. IEEE Access 8 (2020), 224958–224973.
[47]
Sanjay Saha and Terence Sim. 2020. Is face recognition safe from realizable attacks?. In 2020 IEEE International Joint Conference on Biometrics (IJCB). IEEE, 1–8.
[48]
Ulrich Scherhag, Christian Rathgeb, Johannes Merkle, and Christoph Busch. 2020. Deep face representations for differential morphing attack detection. IEEE Transactions on Information Forensics and Security 15 (2020), 3625–3639.
[49]
Florian Schroff, Dmitry Kalenichenko, and James Philbin. 2015. Facenet: A unified embedding for face recognition and clustering. In IEEE Conference on Computer Vision and Pattern Recognition. 815–823.
[50]
Mahmood Sharif, Sruti Bhagavatula, Lujo Bauer, and Michael K. Reiter. 2016. Accessorize to a crime: Real and stealthy attacks on state-of-the-art face recognition. In 2016 ACM SIGSAC Conference on Computer and Communications Security. 1528–1540.
[51]
Mahmood Sharif, Sruti Bhagavatula, Lujo Bauer, and Michael K. Reiter. 2019. A general framework for adversarial examples with objectives. ACM Transactions on Privacy and Security (TOPS) 22, 3 (2019), 1–30.
[52]
Meng Shen, Hao Yu, Liehuang Zhu, Ke Xu, Qi Li, and Jiankun Hu. 2021. Effective and robust physical-world attacks on deep learning face recognition systems. IEEE Transactions on Information Forensics and Security (2021).
[53]
Talha Ahmad Siddiqui, Samarth Bharadwaj, Tejas I. Dhamecha, Akshay Agarwal, Mayank Vatsa, Richa Singh, and Nalini Ratha. 2016. Face anti-spoofing with multifeature videolet aggregation. In 2016 23rd International Conference on Pattern Recognition (ICPR). IEEE, 1035–1040.
[54]
Maneet Singh, Richa Singh, Mayank Vatsa, Nalini K. Ratha, and Rama Chellappa. 2019. Recognizing disguised faces in the wild. IEEE Transactions on Biometrics, Behavior, and Identity Science 1, 2 (2019), 97–108.
[55]
Wenyun Sun, Yu Song, Changsheng Chen, Jiwu Huang, and Alex C. Kot. 2020. Face spoofing detection based on local ternary label supervision in fully convolutional networks. IEEE Transactions on Information Forensics and Security 15 (2020), 3181–3196.
[56]
Di Tang, Zhe Zhou, Yinqian Zhang, and Kehuan Zhang. 2018. Face flashing: A secure liveness detection protocol based on light reflections. In Network and Distributed System Security Symposium (NDSS). 1–15.
[57]
Jinyu Tian, Jiantao Zhou, Yuanman Li, and Jia Duan. 2021. Detecting adversarial examples from sensitivity inconsistency of spatial-transform domain. In AAAI Conference on Artificial Intelligence (AAAI).
[58]
Laurens Van der Maaten and Geoffrey Hinton. 2008. Visualizing data using t-SNE. Journal of Machine Learning Research 9, 11 (2008).
[59]
Dong Wang, Jia Guo, Qiqi Shao, Haochi He, Zhian Chen, Chuanbao Xiao, Ajian Liu, Sergio Escalera, Hugo Jair Escalante, Zhen Lei, Jun Wan, and Jiankang Deng. 2023. Wild face anti-spoofing challenge 2023: Benchmark and results. In IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops. 6379–6390.
[60]
Di Wen, Hu Han, and Anil K. Jain. 2015. Face spoof detection with image distortion analysis. IEEE Transactions on Information Forensics and Security 10, 4 (2015), 746–761.
[61]
Emily Wenger, Josephine Passananti, Arjun Nitin Bhagoji, Yuanshun Yao, Haitao Zheng, and Ben Y. Zhao. 2021. Backdoor attacks against deep learning systems in the physical world. In IEEE/CVF Conference on Computer Vision and Pattern Recognition. 6206–6215.
[62]
Tong Wu, Liang Tong, and Yevgeniy Vorobeychik. 2020. Defending against physically realizable attacks on image classification. In International Conference on Learning Representations (ICLR).
[63]
Weiye Xu, Wenfan Song, Jianwei Liu, Yajie Liu, Xin Cui, Yuanqing Zheng, Jinsong Han, Xinhuai Wang, and Kui Ren. 2022. Mask does not matter: Anti-spoofing face authentication using mmWave without on-site registration. In 28th Annual International Conference on Mobile Computing and Networking. 310–323.
[64]
Zitong Yu, Yunxiao Qin, Hengshuang Zhao, Xiaobai Li, and Guoying Zhao. 2021. Dual-cross central difference network for face anti-spoofing. In International Joint Conference on Artificial Intelligence (IJCAI).
[65]
Zitong Yu, Chenxu Zhao, Zezheng Wang, Yunxiao Qin, Zhuo Su, Xiaobai Li, Feng Zhou, and Guoying Zhao. 2020. Searching central difference convolutional networks for face anti-spoofing. In IEEE/CVF Conference on Computer Vision and Pattern Recognition. 5295–5305.
[66]
Bowen Zhang, Benedetta Tondi, and Mauro Barni. 2020. Adversarial examples for replay attacks against CNN-based face recognition with anti-spoofing capability. Computer Vision and Image Understanding 197 (2020), 102988.
[67]
Ying Zhang, Lilei Zheng, Vrizlynn L. L. Thing, Roger Zimmermann, Bin Guo, and Zhiwen Yu. 2023. FaceLivePlus: A unified system for face liveness detection and face verification. In 2023 ACM International Conference on Multimedia Retrieval. 144–152.
[68]
Zheng Zheng, Qian Wang, Cong Wang, Man Zhou, Yi Zhao, Qi Li, and Chao Shen. 2023. Where are the dots: Hardening face authentication on smartphones with unforgeable eye movement patterns. IEEE Transactions on Information Forensics and Security 18 (2023), 1295–1308.
[69]
Zhe Zhou, Di Tang, Xiaofeng Wang, Weili Han, Xiangyu Liu, and Kehuan Zhang. 2018. Invisible mask: Practical attacks on face recognition with infrared. arXiv preprint arXiv:1803.04683 (2018).
[70]
Jun-Yan Zhu, Taesung Park, Phillip Isola, and Alexei A. Efros. 2017. Unpaired image-to-image translation using cycle-consistent adversarial networks. In IEEE International Conference on Computer Vision. 2223–2232.
[71]
Xun Zhu, Sheng Li, Xinpeng Zhang, Haoliang Li, and Alex C. Kot. 2021. Detection of spoofing medium contours for face anti-spoofing. IEEE Transactions on Circuits and Systems for Video Technology 31, 5 (2021), 2039–2045.
Index Terms
- Detection of Adversarial Facial Accessory Presentation Attacks Using Local Face Differential
Recommendations
Vulnerabilities of Unattended Face Verification Systems to Facial Components-based Presentation Attacks: An Empirical Study
As face presentation attacks (PAs) are realistic threats for unattended face verification systems, face presentation attack detection (PAD) has been intensively investigated in past years, and the recent advances in face PAD have significantly reduced the ...
A Master Key backdoor for universal impersonation attack against DNN-based face verification
Highlights- Face verification system is vulnerable to Master Key backdoor attack even in open-set framework.
AbstractWe introduce a new attack against face verification systems based on Deep Neural Networks (DNN). The attack relies on the introduction into the network of a hidden backdoor, whose activation at test time induces a verification error ...
High-resolution face verification using pore-scale facial features
Face recognition methods, which usually represent face images using holistic or local facial features, rely heavily on alignment. Their performances also suffer a severe degradation under variations in expressions or poses, especially when there is one ...
Comments
Information & Contributors
Information
Published In
July 2024
973 pages
Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than the author(s) must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected].
Publisher
Association for Computing Machinery
New York, NY, United States
Publication History
Published: 27 March 2024
Online AM: 15 February 2024
Accepted: 19 January 2024
Revised: 02 August 2023
Received: 26 October 2022
Published in TOMM Volume 20, Issue 7
Check for updates
Author Tags
Qualifiers
- Research-article
Funding Sources
- National Natural Science Foundation of China
- Natural Science Foundation of Guangdong Province
Contributors
Other Metrics
Bibliometrics & Citations
Bibliometrics
Article Metrics
- View Citations1Total Citations
- 183Total Downloads
- Downloads (Last 12 months)183
- Downloads (Last 6 weeks)18
Reflects downloads up to 10 Nov 2024
Other Metrics
Citations
Cited By
View all- Kang HCha SKim S(2024)Security Improvements of JPEG Images Using Image De-IdentificationElectronics10.3390/electronics1307133213:7(1332)Online publication date: 2-Apr-2024
View Options
Get Access
Login options
Check if you have access through your login credentials or your institution to get full access on this article.
Sign in