research-article

Metric Learning for Anti-Compression Facial Forgery Detection

Authors:

Zhongyuan WangAuthors Info & Claims

MM '21: Proceedings of the 29th ACM International Conference on Multimedia

Pages 1929 - 1937

https://doi.org/10.1145/3474085.3475347

Published: 17 October 2021 Publication History

Abstract

Detecting facial forgery images and videos is an increasingly important topic in multimedia forensics. As forgery images and videos are usually compressed into different formats such as JPEG and H264 when circulating on the Internet, existing forgery-detection methods trained on uncompressed data often suffer from significant performance degradation in identifying them. To solve this problem, we propose a novel anti-compression facial forgery detection framework, which learns a compression-insensitive embedding feature space utilizing both original and compressed forgeries. Specifically, our approach consists of three ideas: (i) extracting compression-insensitive features from both uncompressed and compressed forgeries using an adversarial learning strategy; (ii) learning a robust partition by constructing a metric loss that can reduce the distance of the paired original and compressed images in the embedding space; (iii) improving the accuracy of tampered localization with an attention-transfer module. Experimental results demonstrate that, the proposed method is highly effective in handling both compressed and uncompressed facial forgery images.

References

[1]

Darius Afchar, Vincent Nozick, Junichi Yamagishi, and Isao Echizen. 2018. Mesonet: a compact facial video forgery detection network. In IEEE International Workshop on Information Forensics and Security (WIFS). 1--7.

[2]

Hadar Averbuch-Elor, Daniel Cohen-Or, Johannes Kopf, and Michael F Cohen. 2017. Bringing portraits to life. ACM Transactions on Graphics (TOG), Vol. 36, 6 (2017), 1--13.

Digital Library

[3]

Belhassen Bayar and Matthew C. Stamm. 2016. A Deep Learning Approach to Universal Image Manipulation Detection Using a New Convolutional Layer. In Proceedings of the 4th ACM Workshop on Information Hiding and Multimedia Security. ACM, 5--10.

Digital Library

[4]

Lucy Chai, David Bau, Ser-Nam Lim, and Phillip Isola. 2020. What makes fake images detectable? understanding properties that generalize. In European Conference on Computer Vision. Springer, 103--120.

Digital Library

[5]

Zhuo Chen, Chaoyue Wang, Bo Yuan, and Dacheng Tao. 2020. Puppeteergan: Arbitrary portrait animation with semantic-aware appearance transformation. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR). 13518--13527.

[6]

Francc ois Chollet. 2017. Xception: Deep learning with depthwise separable convolutions. In Proceedings of the IEEE conference on computer vision and pattern recognition (CVPR). 1251--1258.

[7]

Davide Cozzolino, Giovanni Poggi, and Luisa Verdoliva. 2017. Recasting residual-based local descriptors as convolutional neural networks: an application to image forgery detection. In Proceedings of the 5th ACM Workshop on Information Hiding and Multimedia Security. 159--164.

Digital Library

[8]

Hao Dang, Feng Liu, Joel Stehouwer, Xiaoming Liu, and Anil K Jain. 2020. On the detection of digital face manipulation. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR). 5781--5790.

[9]

Jessica Fridrich and Jan Kodovsky. 2012. Rich models for steganalysis of digital images. IEEE Transactions on Information Forensics and Security (TIFS), Vol. 7, 3 (2012), 868--882.

Digital Library

[10]

Ian J. Goodfellow, Jean Pouget-Abadie, Mehdi Mirza, Bing Xu, David Warde-Farley, Sherjil Ozair, Aaron C. Courville, and Yoshua Bengio. 2014. Generative Adversarial Nets. In Advances in Neural Information Processing Systems (NIPS). 2672--2680.

Digital Library

[11]

Tero Karras, Timo Aila, Samuli Laine, and Jaakko Lehtinen. 2018. Progressive Growing of GANs for Improved Quality, Stability, and Variation. In Proceedings of the 6th International Conference on Learning Representations (ICLR).

[12]

Tero Karras, Samuli Laine, and Timo Aila. 2019. A style-based generator architecture for generative adversarial networks. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR). 4401--4410.

[13]

Tero Karras, Samuli Laine, Miika Aittala, Janne Hellsten, Jaakko Lehtinen, and Timo Aila. 2020. Analyzing and improving the image quality of stylegan. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR). 8110--8119.

[14]

Hyeongwoo Kim, Pablo Garrido, Ayush Tewari, Weipeng Xu, Justus Thies, Matthias Niessner, Patrick Pérez, Christian Richardt, Michael Zollhöfer, and Christian Theobalt. 2018. Deep video portraits. ACM Transactions on Graphics (TOG), Vol. 37, 4 (2018), 1--14.

Digital Library

[15]

Diederik P Kingma and Max Welling. 2014. Auto-Encoding Variational Bayes. In 2nd International Conference on Learning Representations (ICLR).

[16]

Iryna Korshunova, Wenzhe Shi, Joni Dambre, and Lucas Theis. 2017. Fast face-swap using convolutional neural networks. In Proceedings of the IEEE international conference on computer vision (ICCV). 3677--3685.

[17]

Akash Kumar, Arnav Bhavsar, and Rajesh Verma. 2020. Detecting deepfakes with metric learning. In 2020 8th International Workshop on Biometrics and Forensics (IWBF). 1--6.

[18]

Lingzhi Li, Jianmin Bao, Hao Yang, Dong Chen, and Fang Wen. 2020 a. Advancing High Fidelity Identity Swapping for Forgery Detection. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR). 5074--5083.

[19]

Lingzhi Li, Jianmin Bao, Ting Zhang, Hao Yang, Dong Chen, Fang Wen, and Baining Guo. 2020 b. Face x-ray for more general face forgery detection. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR). 5001--5010.

[20]

Yuezun Li and Siwei Lyu. 2019. Exposing DeepFake Videos By Detecting Face Warping Artifacts. In IEEE Conference on Computer Vision and Pattern Recognition Workshops (CVPRW). 46--52.

[21]

Yuezun Li, Xin Yang, Pu Sun, Honggang Qi, and Siwei Lyu. 2020 c. Celeb-df: A large-scale challenging dataset for deepfake forensics. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR). 3207--3216.

[22]

Zhengzhe Liu, Xiaojuan Qi, and Philip HS Torr. 2020. Global texture enhancement for fake face detection in the wild. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR). 8060--8069.

[23]

Iacopo Masi, Aditya Killekar, Royston Marian Mascarenhas, Shenoy Pratik Gurudatt, and Wael AbdAlmageed. 2020. Two-branch recurrent network for isolating deepfakes in videos. In Proceedings of the European conference on computer vision (ECCV). 667--684.

[24]

Falko Matern, Christian Riess, and Marc Stamminger. 2019. Exploiting visual artifacts to expose deepfakes and face manipulations. In 2019 IEEE Winter Applications of Computer Vision Workshops (WACVW). 83--92.

[25]

Scott McCloskey and Michael Albright. 2018. Detecting gan-generated imagery using color cues. arXiv preprint arXiv:1812.08247 (2018).

[26]

Ryota Natsume, Tatsuya Yatagawa, and Shigeo Morishima. 2018a. Fsnet: An identity-aware generative model for image-based face swapping. In Asian Conference on Computer Vision (ACCV). 117--132.

[27]

Ryota Natsume, Tatsuya Yatagawa, and Shigeo Morishima. 2018b. RSGAN: face swapping and editing using face and hair representation in latent spaces. In Special Interest Group on Computer Graphics and Interactive Techniques Conference. 69:1--69:2.

Digital Library

[28]

Huy H Nguyen, Fuming Fang, Junichi Yamagishi, and Isao Echizen. 2019 a. Multi-task learning for detecting and segmenting manipulated facial images and videos. In IEEE 10th International Conference on Biometrics Theory, Applications and Systems (BTAS). 1--8.

[29]

Huy H Nguyen, Junichi Yamagishi, and Isao Echizen. 2019 b. Use of a capsule network to detect fake images and videos. arXiv preprint arXiv:1910.12467 (2019).

[30]

Ivan Petrov, Daiheng Gao, Nikolay Chervoniy, Kunlin Liu, Sugasa Marangonda, Chris Umé, Jian Jiang, Luis RP, Sheng Zhang, Pingyu Wu, et al. 2020. Deepfacelab: A simple, flexible and extensible face swapping framework. arXiv preprint arXiv:2005.05535 (2020).

[31]

Yuyang Qian, Guojun Yin, Lu Sheng, Zixuan Chen, and Jing Shao. 2020. Thinking in frequency: Face forgery detection by mining frequency-aware clues. In Proceedings of the European conference on computer vision (ECCV). 86--103.

Digital Library

[32]

Nicolas Rahmouni, Vincent Nozick, Junichi Yamagishi, and Isao Echizen. 2017. Distinguishing computer graphics from natural images using convolution neural networks. In 2017 IEEE Workshop on Information Forensics and Security (WIFS). 1--6.

[33]

Andreas Rossler, Davide Cozzolino, Luisa Verdoliva, Christian Riess, Justus Thies, and Matthias Nießner. 2019. Faceforensics+: Learning to detect manipulated facial images. In Proceedings of the IEEE/CVF International Conference on Computer Vision (ICCV). 1--11.

[34]

Ronald Salloum, Yuzhuo Ren, and C-C Jay Kuo. 2018. Image splicing localization using a multi-task fully convolutional network (MFCN). Journal of Visual Communication and Image Representation, Vol. 51 (2018), 201--209.

[35]

Kritaphat Songsri-in and Stefanos Zafeiriou. 2019. Complement face forensic detection and localization with faciallandmarks. arXiv preprint arXiv:1910.05455 (2019).

[36]

Supasorn Suwajanakorn, Steven M. Seitz, and Ira Kemelmacher-Shlizerman. 2015. What Makes Tom Hanks Look Like Tom Hanks. In Proceedings of the IEEE/CVF International Conference on Computer Vision (ICCV). 3952--3960.

Digital Library

[37]

Justus Thies, Michael Zollhöfer, and Matthias Nießner. 2019. Deferred neural rendering: Image synthesis using neural textures. ACM Transactions on Graphics (TOG), Vol. 38, 4 (2019), 1--12.

Digital Library

[38]

Justus Thies, Michael Zollhofer, Marc Stamminger, Christian Theobalt, and Matthias Nießner. 2016. Face2face: Real-time face capture and reenactment of rgb videos. In Proceedings of the IEEE conference on computer vision and pattern recognition (CVPR). 2387--2395.

Digital Library

[39]

Justus Thies, Michael Zollhöfer, Christian Theobalt, Marc Stamminger, and Matthias Nießner. 2018. Headon: Real-time reenactment of human portrait videos. ACM Transactions on Graphics (TOG), Vol. 37, 4 (2018), 1--13.

Digital Library

[40]

Laurens Van der Maaten and Geoffrey Hinton. 2008. Visualizing data using t-SNE. Journal of machine learning research, Vol. 9, 11 (2008).

[41]

Hao Wang, Doyen Sahoo, Chenghao Liu, Ee-peng Lim, and Steven CH Hoi. 2019. Learning cross-modal embeddings with adversarial networks for cooking recipes and food images. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR). 11572--11581.

[42]

Run Wang, Felix Juefei-Xu, Lei Ma, Xiaofei Xie, Yihao Huang, Jian Wang, and Yang Liu. 2020 a. FakeSpotter: A Simple yet Robust Baseline for Spotting AI-Synthesized Fake Faces. In Proceedings of the Twenty-Ninth International Joint Conference on Artificial Intelligence (IJCAI). 3444--3451.

[43]

Sheng-Yu Wang, Oliver Wang, Richard Zhang, Andrew Owens, and Alexei A Efros. 2020 b. CNN-generated images are surprisingly easy to spot... for now. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR). 8695--8704.

[44]

Yandong Wen, Kaipeng Zhang, Zhifeng Li, and Yu Qiao. 2016. A discriminative feature learning approach for deep face recognition. In Proceedings of the European conference on computer vision (ECCV). 499--515.

[45]

Olivia Wiles, A Koepke, and Andrew Zisserman. 2018. X2face: A network for controlling face generation using images, audio, and pose codes. In Proceedings of the European conference on computer vision (ECCV). 670--686.

Digital Library

[46]

Wayne Wu, Yunxuan Zhang, Cheng Li, Chen Qian, and Chen Change Loy. 2018. Reenactgan: Learning to reenact faces via boundary transfer. In Proceedings of the European conference on computer vision (ECCV). 603--619.

Digital Library

[47]

Xinsheng Xuan, Bo Peng, Wei Wang, and Jing Dong. 2019. On the generalization of GAN image forensics. In Chinese conference on biometric recognition. 134--141.

[48]

Xin Yang, Yuezun Li, and Siwei Lyu. 2019. Exposing deep fakes using inconsistent head poses. In IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP). 8261--8265.

[49]

Sergey Zagoruyko and Nikos Komodakis. 2017. Paying More Attention to Attention: Improving the Performance of Convolutional Neural Networks via Attention Transfer. In Proceedings of the 5th International Conference on Learning Representations (ICLR).

[50]

Egor Zakharov, Aliaksandra Shysheya, Egor Burkov, and Victor Lempitsky. 2019. Few-shot adversarial learning of realistic neural talking head models. In Proceedings of the IEEE/CVF International Conference on Computer Vision (ICCV). 9459--9468.

[51]

Yun-xuan Zhang, Si-wei Zhang, Yue He, Cheng Li, Chen Change Loy, and Zi-wei Liu. 2019. One-shot Face Reenactment. In 30th British Machine Vision Conference (BMVC). 10.

[52]

Peng Zhou, Xintong Han, Vlad I Morariu, and Larry S Davis. 2017. Two-stream neural networks for tampered face detection. In IEEE Conference on Computer Vision and Pattern Recognition Workshops (CVPRW). 1831--1839.

Cited By

Caibo FChunxiao LYuye WQidang Z(2024)Face forgery detection with image patch comparison and residual map estimationJournal of Image and Graphics10.11834/jig.23014929:2(457-467)Online publication date: 2024
https://doi.org/10.11834/jig.230149
Li CZheng ZBin YWang GYang YLi XShen H(2024)Pixel Bleach Network for Detecting Face Forgery Under CompressionIEEE Transactions on Multimedia10.1109/TMM.2023.330124226(2585-2597)Online publication date: 2024
https://doi.org/10.1109/TMM.2023.3301242
Nie HLu SWu JZhu J(2024)Deep Model Intellectual Property Protection With Compression-Resistant Model WatermarkingIEEE Transactions on Artificial Intelligence10.1109/TAI.2024.33511165:7(3362-3373)Online publication date: Jul-2024
https://doi.org/10.1109/TAI.2024.3351116
Show More Cited By

Index Terms

Metric Learning for Anti-Compression Facial Forgery Detection
1. Computing methodologies
  1. Artificial intelligence
    1. Computer vision
      1. Computer vision problems

Recommendations

Forgery detection using feature-clustering in recompressed JPEG images

JPEG images are widely used in a large range of applications. The properties of JPEG compression can be used for detection of forgery in digital images. The forgery in JPEG images requires the image to be resaved thereby, re-compression of image. ...
A survey on deep learning-based image forgery detection
Highlights
- Reviewing all types of image forgeries and their standard datasets including copy-move, splicing, and inpainting.
- Reviewing all traditional forgery detection methods including blockbased, keypoint-based, and hybrid.
- Investigating ...
Abstract
Image is known as one of the communication tools between humans. With the development and availability of digital devices such as cameras and cell phones, taking images has become easy anywhere. Images are used in many medical, forensic medicine, ...
Shift recompression-based feature mining for detecting content-aware scaled forgery in JPEG images
MDMKDD '12: Proceedings of the Twelfth International Workshop on Multimedia Data Mining

Content-aware image resizing, also known as image retargeting, seam carving, content-aware scaling, is originally proposed to automatically remove the paths of least importance, known as seams, to reduce image size or insert seams to extend it, in order ...

Comments

Information & Contributors

Information

Published In

cover image ACM Conferences

MM '21: Proceedings of the 29th ACM International Conference on Multimedia

October 2021

5796 pages

ISBN:9781450386517

DOI:10.1145/3474085

General Chairs:
Heng Tao Shen
University of Electronic Science&Technology of China, China
,
Yueting Zhuang
Zhejiang University, China
,
John R. Smith
IBM, USA
,
Program Chairs:
Yang Yang
University of Electronic Science and Technology of China, China
,
Pablo Cesar
CWI&TU Delft, The Netherlands
,
Florian Metze
FACEBOOK, Inc., USA
,
Balakrishnan Prabhakaran
University of Texas at Dallas, USA

Copyright © 2021 ACM.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

Sponsors

SIGMM: ACM Special Interest Group on Multimedia

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 17 October 2021

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Research-article

Funding Sources

Key Research Development Program of Hubei Province
National Natural Science Foundation of China
National Key Research Development Program of China

Conference

MM '21

Sponsor:

SIGMM

MM '21: ACM Multimedia Conference

October 20 - 24, 2021

Virtual Event, China

Acceptance Rates

Overall Acceptance Rate 995 of 4,171 submissions, 24%

Upcoming Conference

MM '24

Sponsor:
sigmm

The 32nd ACM International Conference on Multimedia

October 28 - November 1, 2024

Melbourne , VIC , Australia

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

9
Total Citations
View Citations
265
Total Downloads

Downloads (Last 12 months)40
Downloads (Last 6 weeks)1

Reflects downloads up to 11 Sep 2024

Other Metrics

View Author Metrics

Citations

Cited By

Caibo FChunxiao LYuye WQidang Z(2024)Face forgery detection with image patch comparison and residual map estimationJournal of Image and Graphics10.11834/jig.23014929:2(457-467)Online publication date: 2024
https://doi.org/10.11834/jig.230149
Li CZheng ZBin YWang GYang YLi XShen H(2024)Pixel Bleach Network for Detecting Face Forgery Under CompressionIEEE Transactions on Multimedia10.1109/TMM.2023.330124226(2585-2597)Online publication date: 2024
https://doi.org/10.1109/TMM.2023.3301242
Nie HLu SWu JZhu J(2024)Deep Model Intellectual Property Protection With Compression-Resistant Model WatermarkingIEEE Transactions on Artificial Intelligence10.1109/TAI.2024.33511165:7(3362-3373)Online publication date: Jul-2024
https://doi.org/10.1109/TAI.2024.3351116
Huang JDu CZhu XMa SNepal SXu C(2023)Anti-Compression Contrastive Facial Forgery DetectionIEEE Transactions on Multimedia10.1109/TMM.2023.334710326(6166-6177)Online publication date: 26-Dec-2023
https://dl.acm.org/doi/10.1109/TMM.2023.3347103
Abdulhamid MHashim A(2023)Enhanced Preprocessing Stage For Feature Extraction of Deepfake Detection Based on Deep Learning Methods2023 7th International Symposium on Innovative Approaches in Smart Technologies (ISAS)10.1109/ISAS60782.2023.10391672(1-6)Online publication date: 23-Nov-2023
https://doi.org/10.1109/ISAS60782.2023.10391672
Huang BWang ZYang JAi JZou QWang QYe D(2023)Implicit Identity Driven Deepfake Face Swapping Detection2023 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)10.1109/CVPR52729.2023.00436(4490-4499)Online publication date: Jun-2023
https://doi.org/10.1109/CVPR52729.2023.00436
Bai YZou QChen XLi LDing ZChen L(2023)Extreme Low-Resolution Action Recognition with Confident Spatial-Temporal Attention TransferInternational Journal of Computer Vision10.1007/s11263-023-01771-4131:6(1550-1565)Online publication date: 8-Mar-2023
https://dl.acm.org/doi/10.1007/s11263-023-01771-4
Xue MWang XSun SZhang YWang JLiu W(2023)Compression-resistant backdoor attack against deep neural networksApplied Intelligence10.1007/s10489-023-04575-853:17(20402-20417)Online publication date: 12-Apr-2023
https://dl.acm.org/doi/10.1007/s10489-023-04575-8
Cao JMa CYao TChen SDing SYang X(2022)End-to-End Reconstruction-Classification Learning for Face Forgery Detection2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)10.1109/CVPR52688.2022.00408(4103-4112)Online publication date: Jun-2022
https://doi.org/10.1109/CVPR52688.2022.00408

View Options

Get Access

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Media

Figures

Other

Tables

View Table of Contents