Pixel Bleach Network for Detecting Face Forgery Under Compression
Authors: 
Congrui Li, Ziqiang Zheng, Yi Bin, Guoqing Wang, Yang Yang, Xuesheng Li, Heng Tao ShenAuthors Info & Claims
Congrui Li, Ziqiang Zheng, Yi Bin, Guoqing Wang, Yang Yang, Xuesheng Li, Heng Tao ShenAuthors Info & ClaimsAbstract
The existing face forgery algorithms have achieved remarkable progress in how to generate reasonable facial images and can even successfully deceive human beings. Considering public security, face forgery detection is of vital importance, making it essential to design face forgery detection algorithms to detect forgery images over the Internet. Despite the great success achieved by the existing Deepfake detection algorithms, they usually failed to achieve satisfactory Deepfake detection performance when deployed to handle the forgery videos in practice. One significant reason is compression. The videos over the Internet are inevitably compressed considering the transmission efficiency. The video compression results in significant Deepfake detection performance degradation for the existing Deepfake detection algorithms. To address this issue, in this article, we propose a generic, simple yet effective “bleaching” pre-processing module based on the generative model and the high-level feature representations to produce a bleached image, which shares a similar appearance with the compressed images. The bleached images with recovered information can be identified accurately by the optimized Deepfake detection models without retraining. The proposed method has utilized a redesigned feature representation, which serves as a navigator to effectively and sufficiently alter the feature distribution in the high-dimensional space to remedy the difference between real facial images and forgery counterparts. Thus, the proposed method can successfully avoid misclassification. Comprehensive and extensive experiments are carried out on four low-quality Faceforensics++ datasets, demonstrating the effectiveness of our method in recovering the information loss caused by the compression artifacts across various backbones and compression.
References
[1]
J. Thies, M. Zollhofer, M. Stamminger, C. Theobalt, and M. Nießner, “Face2face: Real-time face capture and reenactment of RGB videos,” in Proc. IEEE/CVF Conf. Comput. Vis. Pattern Recognit., 2016, pp. 2387–2395.
[2]
R. Natsume, T. Yatagawa, and S. Morishima, “RSGAN: Face swapping and editing using face and hair representation in latent spaces,” ACM siggraph posters, pp. 1–2, 2018.
[3]
A. Pumarola, A. Agudo, A. M. Martinez, A. Sanfeliu, and F. Moreno-Noguer, “Ganimation: Anatomically-aware facial animation from a single image,” in Proc. Eur. Conf. Comput. Vis., 2018, pp. 818–833.
[4]
W. Wu, Y. Zhang, C. Li, C. Qian, and C. C. Loy, “ReenactGAN: Learning to reenact faces via boundary transfer,” in Proc. Eur. Conf. Comput. Vis., 2018, pp. 603–619.
[5]
Y. Nirkin, Y. Keller, and T. Hassner, “FSGAN: Subject agnostic face swapping and reenactment,” in Proc. IEEE/CVF Conf. Comput. Vis. Pattern Recognit., 2019, pp. 7184–7193.
[6]
Z. Zheng, Y. Hu, Y. Bin, X. Xu, Y. Yang, and H. T. Shen, “Composition-aware image steganography through adversarial self-generated supervision,” IEEE Trans. Neural Netw. Learn. Syst., 2022.
[7]
C. Vondrick, H. Pirsiavash, and A. Torralba, “Generating videos with scene dynamics,” in Proc. Adv. Neural Inf. Process. Syst., 2016, pp. 613–621.
[8]
I. Goodfellow et al., “Generative adversarial nets,” in Proc. Adv. Neural Inf. Process. Syst., 2014, pp. 2672–2680.
[9]
X. Yang, Y. Li, and S. Lyu, “Exposing deep fakes using inconsistent head poses,” in Proc. IEEE Int. Conf. Acoust. Speech Signal Process., 2019, pp. 8261–8265.
[10]
U. A. Ciftci, I. Demir, and L. Yin, “FakeCatcher: Detection of synthetic portrait videos using biological signals,” IEEE Trans. Pattern Anal. Mach. Intell..
[11]
Y. Qian, G. Yin, L. Sheng, Z. Chen, and J. Shao, “Thinking in frequency: Face forgery detection by mining frequency-aware clues,” in Proc. Eur. Conf. Comput. Vis., 2020, pp. 86–103.
[12]
L. Li et al., “Face X-ray for more general face forgery detection,” in Proc. IEEE/CVF Conf. Comput. Vis. Pattern Recognit., 2020, pp. 5001–5010.
[13]
H. Dang, F. Liu, J. Stehouwer, X. Liu, and A. K. Jain, “On the detection of digital face manipulation,” in Proc. IEEE/CVF Conf. Comput. Vis. Pattern Recognit., 2020, pp. 5781–5790.
[14]
D. Afchar, V. Nozick, J. Yamagishi, and I. Echizen, “MesoNet: A compact facial video forgery detection network,” in Proc. IEEE Int. Workshop Inf. Forensics Secur., 2018, pp. 1–7.
[15]
Z. Liu, X. Qi, and P. H. Torr, “Global texture enhancement for fake face detection in the wild,” in Proc. IEEE/CVF Conf. Comput. Vis. Pattern Recognit., 2020, pp. 8060–8069.
[16]
J. Li, H. Xie, J. Li, Z. Wang, and Y. Zhang, “Frequency-aware discriminative feature learning supervised by single-center loss for face forgery detection,” in Proc. IEEE/CVF Conf. Comput. Vis. Pattern Recognit., 2021, pp. 6458–6467.
[17]
H. Liu et al., “Spatial-phase shallow learning: Rethinking face forgery detection in frequency domain,” in Proc. IEEE/CVF Conf. Comput. Vis. Pattern Recognit., 2021, pp. 772–781.
[18]
J. Frank et al., “Leveraging frequency analysis for deep fake image recognition,” in Proc. Int. Conf. Mach. Learn., 2020, pp. 3247–3258.
[19]
F. Chollet, “Xception: Deep learning with depthwise separable convolutions,” in Proc. IEEE/CVF Conf. Comput. Vis. Pattern Recognit., 2017, pp. 1251–1258.
[20]
A. Rossler et al., “Faceforensics++: Learning to detect manipulated facial images,” in Proc. IEEE/CVF Conf. Comput. Vis. Pattern Recognit., 2019, pp. 1–11.
[21]
A. Radford, L. Metz, and S. Chintala, “Unsupervised representation learning with deep convolutional generative adversarial networks,” in Proc. Int. Conf. Learn. Representations, 2016.
[22]
J. Song, J. Zhang, L. Gao, Z. Zhao, and H. T. Shen, “AgeGAN: Face aging and rejuvenation with dual conditional GANs,” IEEE Trans. Multimedia, vol. 24, pp. 791–804, 2021.
[23]
Z. Zheng, Z. Yu, H. Zheng, Y. Yang, and H. T. Shen, “One-shot image-to-image translation via part-global learning with a multi-adversarial framework,” IEEE Trans. Multimedia, vol. 24, pp. 480–491, 2022.
[24]
K. Tero, A. Timo, L. Samuli, and L. Jaakko, “Progressive growing of GANs for improved quality, stability, and variation,” in Proc. Int. Conf. Learn. Representations, 2018.
[25]
Y. Choi et al., “StarGAN: Unified generative adversarial networks for multi-domain image-to-image translation,” in Proc. IEEE/CVF Conf. Comput. Vis. Pattern Recognit., 2018, pp. 8789–8797.
[26]
T. Karras, S. Laine, and T. Aila, “A style-based generator architecture for generative adversarial networks,” in Proc. IEEE/CVF Conf. Comput. Vis. Pattern Recognit., 2019, pp. 4401–4410.
[27]
K. Dale et al., “Video face replacement,” in Proc. SIGGRAPH Asia, 2011, pp. 1–10.
[28]
F. Liu, R. Zhu, D. Zeng, Q. Zhao, and X. Liu, “Disentangling features in 3D face shapes for joint face reconstruction and recognition,” in Proc. IEEE/CVF Conf. Comput. Vis. Pattern Recognit., 2018, pp. 5216–5225.
[29]
J. Thies et al., “Real-time expression transfer for facial reenactment,” ACM Trans. Graph., vol. 34, no. 6, pp. 183–1, 2015.
[30]
R. Wang et al., “Fakespotter: A simple yet robust baseline for spotting AI-synthesized fake faces,” in Proc. 29th Int. Conf. Int. Joint Conf. Artif. Intell., 2021.
[31]
Z. Zheng et al., “Asynchronous generative adversarial network for asymmetric unpaired image-to-image translation,” IEEE Trans. Multimedia, vol. 25, pp. 2474–2487, 2022.
[32]
H. Zhao et al., “Multi-attentional deepfake detection,” in Proc. IEEE/CVF Conf. Comput. Vis. Pattern Recognit., 2021, pp. 2185–2194.
[33]
X. Li et al., “Sharp multiple instance learning for deepfake video detection,” in Proc. ACM Int. Conf. Multimedia, 2020, pp. 1864–1872.
[34]
S. A. Khan and H. Dai, “Video transformer for deepfake detection with incremental learning,” in Proc. ACM Int. Conf. Multimedia, 2021, pp. 1821–1828.
[35]
F. Ding et al., “Anti-forensics for face swapping videos via adversarial training,” IEEE Trans. Multimedia, vol. 24, pp. 3429–3441, 2022.
[36]
Y. Nirkin, L. Wolf, Y. Keller, and T. Hassner, “Deepfake detection based on discrepancies between faces and their context,” IEEE Trans. Pattern Anal. Mach. Intell., vol. 44, no. 10, pp. 6111–6121, Oct. 2022.
[37]
L. Zhang, T. Qiao, M. Xu, N. Zheng, and S. Xie, “Unsupervised learning-based framework for deepfake video detection,” IEEE Trans. Multimedia, 2022.
[38]
J. Wang et al., “M2TR: Multi-modal multi-scale transformers for deepfake detection,” in Proc. Int. Conf. Multimedia Retrieval, 2022, pp. 615–623.
[39]
B. Zi, M. Chang, J. Chen, X. Ma, and Y.-G. Jiang, “Wilddeepfake: A challenging real-world dataset for deepfake detection,” in Proc. ACM Int. Conf. Multimedia, 2020, pp. 2382–2390.
[40]
Y. Li, X. Yang, P. Sun, H. Qi, and S. Lyu, “Celeb-DF: A large-scale challenging dataset for deepfake forensics,” in Proc. IEEE/CVF Conf. Comput. Vis. Pattern Recognit., 2020, pp. 3207–3216.
[41]
P. Yu, J. Fei, Z. Xia, Z. Zhou, and J. Weng, “Improving generalization by commonality learning in face forgery detection,” IEEE Trans. Inf. Forensics Secur., vol. 17, pp. 547–558, 2022.
[42]
J. Hu, X. Liao, W. Wang, and Z. Qin, “Detecting compressed deepfake videos in social networks using frame-temporality two-stream convolutional network,” IEEE Trans. Circuits Syst. Video Technol., vol. 32, no. 99, pp. 1089–11021, Mar. 2022.
[43]
L. Sun, H. Zhang, X. Mao, S. Guo, and Y. Hu, “Super-resolution reconstruction detection method for deepfake hard compressed videos,” J. Electron. Inf. Technol., vol. 43, no. 200531, 2021, Art. no.
[44]
S. Woo et al., “Add: Frequency attention and multi-view based knowledge distillation to detect low-quality compressed deepfake images,” in Proc. AAAI Conf. Artif. Intell., 2022, pp. 122–130.
[45]
J. Zhang, J. Ni, and H. Xie, “Deepfake videos detection using self-supervised decoupling network,” in Proc. IEEE Int. Conf. Multimedia Expo, 2021, pp. 1–6.
[46]
M. Li, B. Liu, Y. Hu, L. Zhang, and S. Wang, “Deepfake detection using robust spatial and temporal features from facial landmarks,” in Proc. IEEE Int. Workshop Biometrics Forensics, 2021, pp. 1–6.
[47]
Y. Huang, F. Juefei-Xu, Q. Guo, Y. Liu, and G. Pu, “FakeLocator: Robust localization of GAN-based face manipulations,” IEEE Trans. Inf. Forensics Secur., vol. 17, pp. 2657–2672, 2022.
[48]
S. Cao, Q. Zou, X. Mao, D. Ye, and Z. Wang, “Metric learning for anti-compression facial forgery detection,” in Proc. ACM Int. Conf. Multimedia, 2021, pp. 1929–1937.
[49]
A. Haliassos, K. Vougioukas, S. Petridis, and M. Pantic, “Lips don't lie: A generalisable and robust approach to face forgery detection,” in Proc. IEEE/CVF Conf. Comput. Vis. Pattern Recognit., 2021, pp. 5039–5049.
[50]
K. Roth, Y. Kilcher, and T. Hofmann, “The odds are odd: A statistical test for detecting adversarial examples,” in Proc. Int. Conf. Mach. Learn., 2019, pp. 5498–5507.
[51]
F. Schroff, D. Kalenichenko, and J. Philbin, “FaceNet: A unified embedding for face recognition and clustering,” in Proc. IEEE/CVF Conf. Comput. Vis. Pattern Recognit., 2015, pp. 815–823.
[52]
Y. Wen, K. Zhang, Z. Li, and Y. Qiao, “A discriminative feature learning approach for deep face recognition,” in Proc. 14th Eur. Conf. Comput. Vis., 2016, pp. 499–515.
[53]
J. Thies, M. Zollhöfer, and M. Nießner, “Deferred neural rendering: Image synthesis using neural textures,” ACM Trans. Graph., vol. 38, no. 4, pp. 1–12, 2019.
[54]
J. Zhang et al., “FaceSwapNet: Landmark guided many-to-many face reenactment,” 2019, arXiv:1905.11805.
[55]
J. Deng et al., “RetinaFace: Single-stage dense face localisation in the wild,” 2019, arXiv:1905.00641.
[56]
D. P. Kingma and J. Ba, “ADAM: A method for stochastic optimization,” in Proc. Int. Conf. Learn. Representations, 2014.
[57]
A. Krizhevsky, I. Sutskever, and G. E. Hinton, “ImageNet classification with deep convolutional neural networks,” in Proc. Adv. Neural Inf. Process. Syst., 2012, pp. 1097–1105.
[58]
C. Szegedy et al., “Going deeper with convolutions,” in Proc. IEEE/CVF Conf. Comput. Vis. Pattern Recognit., 2015, pp. 1–9.
[59]
K. He, X. Zhang, S. Ren, and J. Sun, “Deep residual learning for image recognition,” in Proc. IEEE/CVF Conf. Comput. Vis. Pattern Recognit., 2016, pp. 770–778.
[60]
M. Tan and Q. Le, “EfficientNet: Rethinking model scaling for convolutional neural networks,” in Proc. Int. Conf. Mach. Learn., 2019, pp. 6105–6114.
[61]
N. Dogonadze, J. Obernosterer, and J. Hou, “Deep face forgery detection,” 2020, arXiv:2004.11804.
[62]
B. Biggio et al., “Evasion attacks against machine learning at test time,” in Proc. Joint Eur. Conf. Mach. Learn. Knowl. Discov. Databases, 2013, pp. 387–402.
[63]
C. Szegedy et al., “Intriguing properties of neural networks,” in Proc. Int. Conf. Learn. Representations, 2014.
[64]
I. J. Goodfellow, J. Shlens, and C. Szegedy, “Explaining and harnessing adversarial examples,” in Proc. Int. Conf. Learn. Representations, 2014.
[65]
P. Tabacof, J. Tavares, and E. Valle, “Adversarial images for variational autoencoders,” 2016, arXiv:1612.00155.
[66]
Z. Guo, G. Yang, J. Chen, and X. Sun, “Fake face detection via adaptive manipulation traces extraction network,” Comput. Vis. Image Understanding, vol. 204, 2021, Art. no.
[67]
Y. Sun et al., “Face forgery detection based on facial region displacement trajectory series,” in Proc. IEEE/CVF Winter Conf. Appl. Comput. Vis., 2023, pp. 633–642.
Recommendations
A bibliography of pixel-based blind image forgery detection techniques
With the advent of powerful image editing tools, manipulating images and changing their content is becoming a trivial task. Now, you can add, change or delete significant information from an image, without leaving any visible signs of such tampering. ...
An Anti-JPEG Compression Digital Watermarking Technology with an Ability in Detecting Forgery Region for Color Images
CDCIEM '12: Proceedings of the 2012 International Conference on Computer Distributed Control and Intelligent Enviromental MonitoringA watermarking algorithm for color images was proposed, which is resistant with the JPEG lossy compression and can locate tampered regions. Through a certain mathematical operation with an Arnold scrambled key image and low DCT coefficients of the ...
Detecting double JPEG compression with the same quantization matrix
Detection of double joint photographic experts group (JPEG) compression is of great significance in the field of digital forensics. Some successful approaches have been presented for detecting double JPEG compression when the primary compression and the ...
Comments
Information & Contributors
Information
Published In
1520-9210 © 2023 IEEE. Personal use is permitted, but republication/redistribution requires IEEE permission. See https://www.ieee.org/publications/rights/index.html for more information.
Publisher
IEEE Press
Publication History
Published: 02 August 2023
Qualifiers
- Research-article
Contributors
Other Metrics
Bibliometrics & Citations
Bibliometrics
Article Metrics
- 0Total Citations
- 0Total Downloads
- Downloads (Last 12 months)0
- Downloads (Last 6 weeks)0
Reflects downloads up to 04 Oct 2024
Other Metrics
Citations
View Options
View options
Get Access
Login options
Check if you have access through your login credentials or your institution to get full access on this article.
Sign in