Individualized Deepfake Detection Exploiting Traces Due to Double Neural-Network Operations

Abstract

In today’s digital landscape, journalists urgently require tools to verify the authenticity of facial images and videos depicting specific public figures before incorporating them into news stories. Existing deepfake detectors are not optimized for this detection task when an image is associated with a specific and identifiable individual. This study focuses on the deepfake detection of facial images of individual public figures. We propose to condition the proposed detector on the identity of the identified individual given the advantages revealed by our theory-driven simulations. While most detectors in the literature rely on perceptible or imperceptible artifacts present in deepfake facial images, we demonstrate that the detection performance can be improved by exploiting the idempotency property of neural networks. In our approach, the training process involves double neural-network operations where we pass an authentic image through a deepfake simulating network twice. Experimental results show that the proposed method improves the area under the curve (AUC) from 0.92 to 0.94 and reduces its standard deviation by 17%. For evaluating the detection performance of individual public figures, a facial image dataset with individuals’ names is required, a criterion not met by the current deepfake datasets. To address this, we curated a dataset comprising 32k images featuring 45 public figures, which we intend to release to the public after the paper is published.

1 Introduction

Figure 1: The inference pipeline of the proposed individualized deepfake detector leveraging the near-idempotence property and identity conditioning. The identity conditioning is achieved by combining the identity-aware processing trace and the input identity vector. To leverage the idempotence property, the test image is passed through a reconstruction operator $R$. If the test image exhibits a marginal change in the observed amount of processing traces, the test image is considered “deepfake”; if a significant change is observed, the image is considered “authentic”.

A deepfake refers to a seemingly authentic image or video generated by a deep neural network. When it comes to human faces, a manipulation method may comprise reenactment, replacement, editing, and synthesis [41]. While deepfakes can facilitate numerous appealing and advantageous applications, the act of replacing the face in a staged image or video with the face of a public figure can pose a serious threat to the society. Given the continuous influx of deepfake videos on public platforms, journalists need to pay special attention to those that relate to significant public interest, such as those featuring celebrities or politicians [41, 26]. The deepfake generation methods evolved with autoencoder-based approaches [1], GANs [6], and diffusion models [47]. The latest diffusion-based models such as [47, 49] can surpass GAN-based models in producing photorealistic images. Nevertheless, even in the present day, autoencoder-based models remain threatening in terms of malicious use. This is due to the availability of several free, downloadable, and user-friendly applications built on autoencoder, such as FaceSwap [5], Faceswap-GAN [6], DeepFaceLab [4], and df [2]. In this work, we focus on Faceswap-GAN.

Most deepfake detectors were built to detect the whole population of deepfake videos, i.e., deepfake videos of whatever identities are targeted. However, victims of deepfakes are most often public figures and their deepfake videos are more detrimental due to their widespread public exposure. In this work, we propose a deepfake image detection system customized for individual subjects. Our theory-driven simulations suggest that identity conditioning on deepfake detection tends to exhibit advantages in more challenging detection tasks. As our experimental results will show, the existing tools for deepfake face detection that encompass the whole population may work suboptimally for a specific public figure. The proposed detector for specific individuals is especially useful for journalism. For example, before reporting news based on an image of a public figure of unknown authenticity, a journalist can apply the proposed detection tool to determine its authenticity.

Our approach to deepfake detection draws inspiration from a series of studies leveraging the near-idempotence property of an operation. This method has been particularly effective in various image forensics tasks, including double JPEG compression detection, unknown video codec identification, and source camera identification  [32, 53, 9, 23, 40]. In these studies, researchers leverage the near-idempotence of a respective operation, such as certain type of JPEG compression, video compression, or color demosaicing algorithm. The strict idempotence property asserts that an idempotent operation, $f(\cdot)$, results in no change to $f(x)$ when it is applied iteratively, i.e., $f(f(x))=f(x)$. Using slightly different terminology, if $f(f(x))$ approximately equals $f(x)$, the operation is nearly idempotent. In many detection problems of multimedia forensics, the nearly idempotent nature of a forgery method allows an analyst to apply the forgery operation multiple times and observe the changes to determine whether the input was forged for the first time, i.e., input forged for more than once will exhibit minimal changes.

In this work, we demonstrate that near-idempotence is also applicable to the neural network-based Faceswap-GAN [6]. To explore this, we emulate the potential deepfake operation that an attacker might employ, utilizing publicly available data of a public figure and making assumptions about the neural network architecture. Fig. 1 illustrates the inference pipeline of the proposed detector. We feed a test image into the emulated deepfake generator. The expected change in the image due to this operation is dependent on whether the image has undergone a similar operation before. If the image is a deepfake, the near-idempotence property ensures that the change will be minimal. From the standpoint of the deepfake feature extractor, a deepfake image will exhibit processing traces both before and after the operation, leading to subtle observed changes. Conversely, an authentic image without the deepfake operation lacks any processing traces of the neural network, resulting in a significant observable change. The contributions of this paper are threefold.

• We propose to use the near-idempotence property of neural networks for deepfake face detection, introducing a distinct direction of improvement compared to the state of the art. The idempotence-driven approach can potentially complement existing methods.

• We demonstrate that identity conditioning can significantly improve the deepfake detection performance over the state-of-the-art end-to-end CNN classifiers.

• Our detector can focus on specific individuals. Individualized detectors are better suited for journalism.

2 Related Work

2.1 Generation of Deepfake Faces

Early methods of face-swapping such as Bitouk et al. [10] were limited to using two images of two particular persons with similar poses. The images were first aligned with the help of landmark detection, then cropped, and postprocessed including color correction. Subsequent researchers [19] improved those with a 3-D facial model from the source video. The next advancement emerged after the proposal of a deep-learning-based face-swapping architecture [1] built upon one shared encoder and two individual decoders. Faceswap-GAN [6] is the GAN improvement over [1] where the performance of shared encoder and individual decoders further improve as a result of the GAN’s internal interplay mechanism between the generator and discriminator. However, the architectures of [1, 6] can swap faces between only two identities involved in training. Researchers have proposed identity agnostic architectures decoupling the identity extraction from the attribute extraction [8, 45, 42, 43, 37].

2.2 Protection Against Deepfake

Researchers have been exploring different methods to detect deepfakes. In the first category, the artifacts of synthetic videos are exploited for deepfake detection such as the absence of eye blinking [38], inconsistency in head pose. [54], disparities in color components [36], and inconsistency between inner face and outer face [25]. In the second category, researchers used either an end-to-end convolutional neural network (CNN) structure  [51] or a combined CNN with a recurrent neural network (RNN) [29]. In the third category, researchers exploit processing traces left by the neural networks for deepfake detection. The researchers exploited the features like spatial domain local convolutional features [28], spectral distortion caused by up-convolutions [26], and upsampling artifacts in the frequency domain [27].

Identity-driven Deepfake Detection. Instead of detecting deepfake videos for the whole population, recent work also exploited characteristics of a specific person for deepfake detection. Agarwal et al. [7] targeted deepfake videos of a specific individual by capturing speaking patterns. Cozzolino et al. [21] proposed to learn the temporal features of how a specific person moves and talks. Dong et al. [25] calculated $\ell^{2}$ distance between the computed identity vector from the inner face and the expected identity vector drawn from a reference set of identity vectors. In this work, we extract the deepfake traces conditioned on the identity.

2.3 Idempotency as a Multimedia Forensics Tool

In multimedia forensics, one way to detect counterfeiting is to exploit the near-idempotence property, i.e., the minor changes caused by the repetitive application of adversarial operations. It shares the same sprit of the law of diminishing returns, a widely used concept in economics [13, 50]. The detection of double JPEG compression, source camera identification, and video codec identification are three exemplary applications of the near-idempotence property. The ratio of stable image blocks has been used by researchers to detect the number of prior JPEG compressions [35, 15]. Huang et al. [32] found that the number of dissimilar JPEG coefficients between two subsequent JPEG compression decreases monotonically. Bestagini et al. [9] detected unknown video encoding by recompressing a video with each of the candidates. For source camera identification, the researchers have leveraged the near-idempotence property of an auto-white balancing method [23] and that of color demosaicing strategy [40]. In economics, the law of diminishing returns states that additional inputs to a fixed amount of identical inputs increase productivity at a decreasing rate [13]. If the additional inputs are considered repetitive operations, then the law of diminishing returns may be considered as near-idempotence. In this study, we show that the near-idempotence property of neural networks assists in deepfake image detection.

2.4 Unsupervised Pretraining

Unsupervised pertaining has been proposed for feature extraction for many tasks of computer vision. Chen et al. [18] found that larger networks, for example, larger ResNet, pretrained in an unsupervised manner followed by supervised training with only $10\%$ of labeled data can outperform fully supervised networks for general computer vision tasks. Newell and Deng [44] showed that pretrained networks are more advantageous in low data regimes compared to ubiquitous data. Their results suggest that pretrained networks should be tested on diverse downstream tasks. Bulat et al. [14] proposed task-agnostic self-supervised pretraining on in-the-wild facial data for representation learning. Zheng et al. [58] proposed weakly supervised facial representation learning using vast facial images available on the web with linguistic descriptions. In this work, we utilize the facial features from Bulat et al. [14] to additionally learn the deepfake traces.

3 Threat Model

In this work, we consider an attacker who is smart enough to find and use open-source face-swapping software such as [6, 5, 1] on the facial images from the publicly available videos of a public figure. More specifically, we consider Faceswap-GAN [6] as a potential method that the attacker can use. The attacker is free to use any public or private videos of a second person to depict a story and they want to convince the public of the involvement of a targeted public figure. For example, the attacker can record prearranged videos at a professional studio and later replace the actor’s face with that of the public figure. The attacker can harvest videos of the public figure from multiple sources, including social media, news channels, movies, and YouTube. Different sources of videos offer varied image quality, compression levels, and processing histories. For example, public interview videos of a public figure available on YouTube are expected to be less edited than video clips from movies. In our proposed detection method, we assume that we, as forensic analysts, have access to the various sources of public figure videos, but we do not know exactly from what source the attacker took videos for deepfake generation. For example, the attacker can use videos from social media, where we will only use public interview recordings of that public figure to train the neural network based detector.

4 Proposed Detector via Near-Idempotence and Identity Conditioning

In the challenge of identifying deepfake faces for public figures, we confront an image of unknown authenticity, claimed to be a specific public figure. Our approach to addressing this problem makes use of the extensive collection of authentic images or videos of the said public figure from YouTube. The training process of our proposed deepfake detector is depicted in Fig. 2 and the inference pipeline is shown in Fig. 1.

Figure 2: The training pipeline of the proposed deepfake detector leveraging the near-idempotence property of the deepfake generator. A side-by-side comparison with conventional deepfake detectors is also shown. In the proposed method, an authentic image is passed through a deepfake simulating network or reconstruction operator, twice. Due to the near-idempotence property, the features for the first and the second outputs will be nearly identical. The features are obtained from an identity-aware feature extractor that is trained separately. We freeze the feature extractor network and train a Siamese network and an identity decoder to increase the Euclidean distance between the first pair (consisting of the authentic image and the first output image) and to decrease the Euclidean distance between the second pair (consisting of the first and the second output images).

Our proposed detector has four distinct components. First, the reconstruction operator is a neural network operation that stimulates the deepfake generation operation for a public figure. We found this operation nearly idempotent. Second, the feature extractor is finetuned with a teacher network and is able to capture the identity information while extracting the features. Third, the identity decoder takes as input the explicit identity, i.e., the index of the public figure, and learns as a constant identity vector that arguments the feature space. It contains the necessary person-specific information of that public figure, and when combined with the identity-aware feature can effectively compute the deepfake features conditioned on identity. Fourth, the Siamese network serves as the ultimate binary classification block in the proposed architecture. It learns to extract the features linked to the idempotency of the deepfake operation. It produces a larger distance before and after reconstruction for a test authentic image and a smaller distance for a test deepfake image.

4.1 Reconstruction Operator and Idempotence-Driven Detection

We employ a dedicated reconstruction operator $R$ for each public figure as shown in Fig. 1 and  Fig. 2. When the original image is authentic, the first operation generates a deepfake image, and the second operation produces a doubly processed deepfake. We verified experimentally that the reconstruction operator $R$ serves as a reliable approximation of a specific type of deepfake generation tool, such as FaceSwap-GAN [6], and that the deepfake generation process is nearly idempotent. In this context, the distance between a deepfake image and its corresponding doubly processed deepfake tends to be close to zero. This characteristic is leveraged in the training and inference system.

The next consideration is how to obtain the identity-specific reconstruction operator. For each public figure within our scope, we accumulate numerous images of that public figure and train a neural network based on an autoencoder utilizing the encoder and decoder architecture from FaceSwap-GAN [6]. This network learns the facial characteristics of the public figure, and when given a facial image of that public figure, it can reproduce approximately the same image as the output. Since the objective of this network is to replicate the input facial image of an identity, we refer to the resulting operator as the reconstruction operator or emulated deepfake generator. Some examples of reconstructed images are shown in Fig. 3.

(a)

(b)

Figure 3: (a) Facial regions from raw images (first row) and reconstructed images (second row). The reconstructed images are singly processed. (b) Facial regions from deepfake images (first row) and reconstructed images (second row). The reconstructed images are doubly processed. The reconstruction models trained with images from the same person result in good visual quality for both raw and deepfake images.

The reconstruction operator $R$ exhibits near-zero changes to a deepfake image due to the near-idempotence. Consequently, the feature level Euclidean distance between the two is expected to be small. On the other hand, an authentic image and its corresponding processed image will be substantially different as the operation leaves discernible traces in the processed image. Considering the capability of our deepfake feature extractor (see Sec. 4.2) to detect these traces, the features will exhibit significant dissimilarity, resulting in a higher distance compared to the deepfake scenario.

Based on the above considerations, the initial problem of detecting whether an image is authentic or deepfake is now reframed as evaluating the change of the image in the feature space through the reconstruction operation. When this change, quantified as the Euclidean distance, approaches zero, the image is classified as a deepfake; otherwise, it is considered authentic. Denoting the input image by f, the reframed problem is to evaluate whether f and $R(\text{f})$ are the same or not, where $R$ is our reconstruction operator. Treating f and $R(\text{f})$ as two inputs, we note that the Siamese network [12] is a powerful approach for discerning similarity or dissimilarity between two inputs. Our use of the Siamese network will be discussed in Sec. 4.4.

4.2 Identity-Aware Feature Extractor

Motivation. Conventional deepfake feature extraction network $B(\cdot)$ extracts the deepfake features $B({\rm f})$ for a test image ${\rm f}$ ignoring the person identity ${\rm I}$ [11, 22, 57] or considers the identity features irrelevant to forgery detection [30, 52]. Our work found that the identity-aware feature, $B^{{}^{\prime}}({\rm f})$, which extracts identity information in addition to the deepfake features, is more effective for deepfake detection. This may be explained that a distinct extracted feature may not be equally distinguishable for every identity for the classification. If a feature extractor does not allow the passing of the identity information, the later network can not learn the statistics of the features individually for each identity. This will be limited to learning the average pattern. Such average distributions of the features will lead to the error probability of the Bayesian classifier as follows:

$$P_{\text{e}}^{\text{com}}\!=\!\mathbb{P}(H_{0})\mathbb{P}(C\!=\!1|H_{0})\!+\!\mathbb{P}(H_{1})\mathbb{P}(C\!=\!0|H_{1}).$$ (1)

where $\mathbb{P}(\cdot)$ is the probability measure, $H_{0}$ and $H_{1}$ are two hypotheses, $C$ is the predicted class. On the other hand, if the feature extractor allows passing the identity, the later network can distinguish the features for each identity separately. Knowing the distributions of the features for each identity separately will lead to the error probability:

$$P_{\text{e}}^{\text{ind}}\!=\!\frac{1}{N}\sum_{\text{I}\in\mathbb{I}}\, \mathbb{P}(H_{0}|\text{I})\mathbb{P}(C\!=\!1|H_{0},\text{I})$$ (2)

$$+\, \mathbb{P}(H_{1}|\text{I})\mathbb{P}(C\!=\!0|H_{1},\text{I}),$$

where $\mathbb{I}$ is the set of all identities. In Sec. 3 of the supplementary document, we showed that the latter identity-conditioning approach is more powerful in reducing classification error. We conducted a performance comparison between two methods through theory-driven simulations, demonstrating that $P_{\text{e}}^{\text{ind}}$ tends to be lower (better) than $P_{\text{e}}^{\text{com}}$. Furthermore, we observed that the gain of $P_{\text{e}}^{\text{ind}}$ over $P_{\text{e}}^{\text{ind}}$ is more significant when the deepfake traces for individuals are more unique and the detection problem is intrinsically more difficult.

Training. To make the feature extraction network identity-aware, we use a neural network such that the earlier layers extract identity-aware features along with other features, and the later layers extract deepfake traces. We use a learned facial representation, trained by Bulat et al. [14] as the starting point of training $B^{{}^{\prime}}({\cdot})$. Their trained network has an architecture of ResNet. For extracting deepfake features, we tune the portion of the network after the “conv4” block.

We reused the model and initial weights from Bulat et al. [14] for the following three reasons. First, having an existing network that lets personal identity pass through makes our task easier to additionally learn the deepfake traces. In comparison, training a network simultaneously for personal identity and deepfake detection would require joint training of two downstream tasks, which is harder. Second, a deeper network trained with unlabelled data is less biased to any specific portion of the dataset [18]. Bulat et al. [14] pretrained the ResNet architecture with ${\sim}10$ million facial images. Consequently, the initial layers of the network are anticipated to learn a robust representation of features, including the identity. The network is also tested over multiple downstream tasks and therefore, it is a good candidate for extracting facial features [44]. Third, according to Newell and Deng [44], there is an advantage in unsupervised pretraining with unlabeled data when the labeled finetuning dataset is small, which aligns with our labeled training dataset comprising 295 videos from 59 celebrities.

Figure 4: Backbone network training for identity-aware deepfake feature extraction. An authentic and deepfake image pair is passed through the teacher and student networks. Teacher network passes down the deepfake trace knowledge to the student network through loss functions $L_{1}$ and $L_{2}$. The loss function $L_{3}$ increases the feature distance between the authentic and the deepfake image.

The training for the backbone network $B^{{}^{\prime}}(\cdot)$ is depicted in Fig. 4. The input is an image pair, consisting of an authentic image and its corresponding deepfake, generated using a deepfake generation tool. The input is passed through a student network $B_{\rm s}$ and a teacher network $B_{\rm t}$ in parallel. The student network is composed of the pretrained facial representation learning backbone [14] and a concatenated task adaptation head for learning specifically the deepfake traces. The layers after the “conv4” block of the pretrained backbone and the task adaptation head are the tunable portions of the student network. We then utilize the EfficientNetAutoAttB4ST [11] as the teacher network to distill the knowledge for learning the deepfake traces. To adapt the deepfake traces based on personal identity, we add a loss function $L_{3}$ that contrasts the learned traces of a deepfake and its corresponding authentic image in addition to the knowledge distillation losses $L_{1}$ and $L_{2}$. Given the authentic facial image of identity $\rm I$, $\rm f_{auth}$, and its corresponding deepfake image $\rm f_{df}$, the loss terms are defined as follows:

$$L_{1} =D^{2}\Big{(}B_{\rm t}({\rm f_{auth}}),B_{\rm s}(\rm f_{auth})\Big{)},$$ (3a)

$$L_{2} =D^{2}\Big{(}B_{\rm t}({\rm f_{df}}),B_{\rm s}(\rm f_{df})\Big{)},$$ (3b)

$$L_{3} =\Big{[}\max\Big{(}0,m\!-\!D\big{(}B_{\rm s}({\rm f_{auth}}),B_{\rm s}({\rm f_{df}})\big{)}\Big{)}\Big{]}^{2},$$ (3c)

where $D(\cdot)$ is the Euclidean distance and $m$ is the margin of the hinge loss. The three loss terms are combined as $\alpha(L_{1}+L_{2})+\beta L_{3}$, with hyperparameters $\alpha$ and $\beta$. $L_{1}$ and $L_{2}$ contribute to the knowledge distillation for learning the deepfake traces and $L_{3}$ contributes to learning the deepfake traces according to identity.

4.3 Identity Decoder for Feature Conditioning

Our identity decoder is a single-layer fully-connected neural network that maps the one-hot-encoded index of a public figure to the feature space generated by our feature extractor. We combine the output of the identity decoder with the output of the identity-aware feature extractor that contains the joint information of the deepfake feature and identity. The extra marginal information provided by the identity decoder can have the effect of conditioning the identity-aware feature, in a similar spirit as in the Bayes rule.

4.4 Contrastive Learning

The Siamese network contains two identical subnetworks that process the two inputs parallelly. The subnetworks learn a manifold for each of the inputs adopting contrastive loss that allows a powerful discrimination between the two inputs. In our work, we designed each of the subnetworks as a single-layer neural network that takes as input the features of the corresponding image and outputs a vector of length $50$. We experimentally verified that this length is enough for discriminating the two cases. Let us call the two subnetworks of the Siamese network $S_{n_{1}}$ and $S_{n_{2}}$, where the first one processes the features of f and the second one processes the features of $R(\text{f})$. We used contrastive loss [31] to train the Siamese network as follows:

$$\begin{split}L\big{(}\text{f},\!R(\text{f}),\!Y\big{)}\!=\!(1\!-\!Y)D^{2}_{S_{n}}{+}Y\!\left[\max\left(0,m\!-\!D_{S_{n}}\right)\right]^{2},\end{split}$$ (4)

where $D_{S_{n}}$ is the Euclidean distance between the processed manifolds, i.e., $D_{S_{n}}=\left\|S_{n_{1}}\left(X_{1}\right)-S_{n_{2}}\left(X_{2}\right)\right\|_{2}$, $X_{1}$ is the identity-conditioned features of f, $X_{2}$ is the identity-conditioned features of $R(\text{f})$, $m>0$ is a margin, and $Y\in\{0,1\}$ is the known binary label of f, i.e., is $1$ if f authentic, and $0$ otherwise. We learned the weights of the identity decoder and the two subnetworks of the Siamese network using this loss function. Additionally, in contrast to the standard Siamese network, we decoupled the weights of the two subnetworks, $S_{n_{1}}$ and $S_{n_{2}}$, similar to CLIP [46], resulting in performance enhancement.

5 Experimental Results

5.1 Dataset Curation

Deepfake detection methods generally perform well in the in-dataset evaluation, but the performance drops significantly in cross-dataset evaluation [17]. To provide reliable measurements for the performance of the deepfake detection method, we report only the cross-dataset evaluation results. As our deepfake detector conditions on the identity, we need two independent datasets containing facial images of the same set of identities. Most of the public deepfake detection datasets such as DFDC [24], DFD [3], Deeper Forensics [33] do not explicitly mention identity information associated with the videos. It is hard to find the same persons in another dataset, which would be necessary to perform the cross-dataset evaluation of individualized deepfake detection. To the best of our knowledge, our work is the first work that evaluates the deepfake detection performance on another dataset for each person separately. We curate a dataset from Celeb-DF [39] and CACD [16]. Our curated dataset contains $32k$ facial images of $45$ public figures sourced from YouTube videos for the train set and from the cross-age facial image dataset of the same public figures for the test dataset. We plan to release the source code and dataset for public use after publication.

For the training dataset, we use real videos from the Celeb-DF dataset [39], which is a popular deepfake detection dataset of 59 public figures. We sample frames from the videos at $5$ frames per second (fps), and detect faces from the videos using the MTCNN [56] face detection network. For each individual $i$, we have facial images $f^{\text{cdf}}_{i,j,k}$ from the $j$th authentic Celeb-DF video, where $j\in\{1,...,10\}$, $k\in\{1,...,N_{f}\}$, and $N_{f}$ is the number of the frames extracted from the video.

Examining multiple candidate datasets, we narrowed down to the CACD [16] dataset for cross-dataset evaluation. CACD [16] contains cross-age facial images of 2,000 public figures with an overlap of 45 public figures with the Celeb-DF dataset. From the CACD dataset, we have authentic images $f^{\text{cacd}}_{i,j,k}$ for the $i$th identity and $j$th available age group of that identity, where $j\in\{1,...,10\}$, $k\in\{1,...,N_{i}\}$, and $N_{i}$ is the number of the images available for that age group. To generate deepfake faces for the test set, for each person $i$, we choose another identity $m$ from the database of 2,000 persons and then train a Faceswap-GAN [6] model using the facial images $f^{\text{cacd}}_{i,j,k}$ and $f^{\text{cacd}}_{m,j,k}$.

5.2 Experimental Setup

As our proposed deepfake detector is designed to operate only when the identity is provided, we conduct our evaluation on our curated dataset comprising two distinct subsets of facial images with an overlap of $45$ public figures. We use the cropped facial images from the video frames of Celeb-DF [39] as the training set and the images from CACD [16] as the test set.

Our proposed method had two stages of training. In the first stage, we trained the identity-aware feature extractor. For this training, we resized the facial images to 224-by-224 and used random cropping and random horizontal flipping for image augmentation. As shown in Fig. 4, we used a pair of images for the backbone training. We enforced identical cropping within the same pair consisting of an authentic image and its deepfake. We used SGD optimizer and a learning rate of $10^{-3}$. The contrastive loss margin was $50$ and the values of $\alpha$ and $\beta$ were varied manually within $[\alpha,\beta]\in(0,1)^{2}$. We set $[\alpha,\beta]$ to $[1,0]$ during the initial 1,500 epochs of training and subsequently modified it to $[0,1]$ for the next 1,500 epochs. In the second stage, we trained the Siamese network and the identity decoder. For this training, we used Adam optimizer, and the contrastive loss margin was $2$ and the learning rate was determined by the grid search within the range of $(10^{-6},10^{-5})$.

For face reconstructor training, we separated the facial images from the last five videos $f^{\text{cdf}}_{i,j,k},j\in\{6,\dots,10\}$ of the Celeb-DF dataset. For the final classification network training, we randomly selected facial images from one video $f^{\text{cdf}}_{i,j,k},j\in\{1,\dots,5\}$ as the validation set and facial images from other four videos as the training set. We repeated this process four times to ensure the results would be statistically stable. As for the test set, we used all of the real and face-swapped images that we generated from CACD. In each training session, the neural network with the smallest validation loss was chosen as the final network for the test set.

5.3 Performance Gain

In this subsection, we compare the performance of our proposed method with two state-of-the-art methods. The first baseline considered is the Xception [20] network trained on the FaceForensics++ dataset [48] with deepfake videos generated by four methods including Faceswap [5]. The second one is the EfficientNetAutoAttB4ST [11] network trained on the DFDC dataset [24], a dataset consisting of deepfake videos generated by various popular face-swapping methods, such as Facewap-GAN [6], StyleGAN [34], Faceswap [5], and NTH [55]. The performance of the two baseline approaches on our test dataset is presented in Tab. 1.

Table 1: Comparison of deepfake detection performance of the proposed method leveraging near-idempotence and identity conditioning with the state of the art.

Method	AUC	AUC	AUC trimmed
	Mean (SD)	Median (IQR)	Mean (10%)
Xception [20]	0.792 (0.11)	0.799 (0.14)	0.799
Xception [20] (tuned)	0.887 (0.07)	0.896 (0.09)	0.894
EfficientNet [11]	0.728 (0.13)	0.733 (0.16)	0.732
EfficientNet [11] (tuned)	0.920 (0.06)	0.926 (0.07)	0.927
Proposed	0.940 (0.05)	0.958 (0.05)	0.947

To ensure a fair comparison with our proposed method, we conducted fine-tuning on these two baseline methods using our training dataset. This involved keeping the features frozen and training a classification layer on top of the features until the performance was saturated on the validation dataset. After finetuning, EfficientNetAutoAttB4ST [11] had an AUC mean of $0.92$ across identities with a sample standard deviation of $0.06$.

To evaluate the idea of utilizing idempotency and identity conditioning, we applied the double neural network operation and obtained the features from our trained identity-aware feature extractor. We concatenated those with the features of EfficientNetAutoAttB4ST [11]. Tab. 1 reveals that the proposed method can achieve an AUC mean of $0.94$ across identities, an increase of $0.02$ from Bonettini et al. [11]. The AUC median across identities was $0.958$ with a gain of $0.032$ from the baseline [11]. The 10%-trimmed mean was $0.947$ with a gain of $0.02$. The AUC standard deviation was reduced by $0.01$ or $17\%$ and the AUC interquartile range was reduced by $0.02$ or $29\%$ compared to the baseline [11]. This result demonstrates that idempotency and identity conditioning can improve performance in validity and variation. The detection results on the test dataset for six of the $45$ public figures are shown in Fig. 5.

Figure 5: ROC curves for deepfake detection using the proposed method. Each plot contains results from a public figure and each curve represents a trial of training the network. AUC values are large with small standard deviations, indicating good performance.

The averaged AUC value among all public figures is $0.940$ and the sample standard deviation is $0.05$. We also performed $t$-tests and the proposed method is significantly better than those of the off-the-shelf detectors in terms of AUC. The larger variance of the AUC values of the baseline methods implies that the deepfake detector may perform convincingly for one identity, but it has a greater risk of exhibiting unacceptable performance for others. This makes the baseline methods less attractive for journalists.

5.4 Ablation Studies

Tab. 2 displays the results of ablation studies. In the first ablation study, we applied our idempotent strategy (with identity decoder) using the EfficientNetAutoAttB4ST features. In the second study, we concatenated the features from the identity-aware feature extractor with the features of EfficientNetAutoAttB4ST as we did in our proposed method and used a feedforward network to classify the images. The first ablation achieved the AUC mean of $0.926$ and the AUC median was $0.928$. The sample standard deviation and interquartile range were $0.05$ and $0.06$. The second ablation achieved the AUC mean of $0.893$ and the AUC median was $0.920$. The sample standard deviation and interquartile range were $0.10$ and $0.13$. The achieved AUC values are much lower compared to the proposed method. This confirms that the identity conditioning and idempotence strategy have synergy (positive interaction).

Table 2: Ablation studies for the proposed method.

Method	AUC	AUC	AUC trimmed
	Mean (SD)	Median (IQR)	Mean (10%)
Proposed	0.940 (0.05)	0.958 (0.05)	0.947
Idempotence	0.926 (0.05)	0.928 (0.06)	0.932
Identity-aware features	0.893 (0.10)	0.920 (0.13)	0.904

6 Discussion

In the current work, the reconstruction model is the deepfake generation tool, i.e., Faceswap-GAN, one of the most popular and effective off-the-shelf tools for deepfake generation. When the reconstruction model does not match the tool used for deepfake generation, the processing traces caused by deepfake and reconstruction may be different, but both traces are generated by neural networks. For example, the reconstruction model is a Faceswap-GAN model whereas the deepfake video is generated by a Faceswap model. A more sophisticated classifier may be needed to exploit the processing traces left by different network structures. Under the proposed double operations framework, a more general Siamese neural network for processing-trace manifold learning may work, but we leave this exploration to future work.

Compared to end-to-end CNN-based classifiers, our proposed method targets deepfake detection for individuals, with main applications on public figures. Although our method needs training the reconstruction models, the training can be done in advance for each public figure. For example, a journalist can train the reconstruction models for various candidates before they need to verify videos for reporting tasks. Journalists may also share or collaboratively train detectors within their professional networks. To let the detection system support a new individual, the journalist will need to train a reconstruction operator for that individual and then finetune the Siamese network.

7 Conclusion and Future Work

In this work, we have proposed to use the method of double neural network operations and individual conditioning for the deepfake detection. The proposed detector can achieve better detection performance than end-to-end CNN-based detectors on our curated dataset of public figures with identity labels. We have found that utilizing identity information can make the deepfake detector more reliable. In future work, we plan to extend the double-operations detection to scenarios with mismatched neural network architectures.

Supplementary Material

1 Tool for Deepfake Generation

High-quality face-swapped videos may be generated by tools based on convolutional autoencoder models. An autoencoder consists of two neural networks, an encoder and a decoder. The encoder $f_{\mathrm{enc}}$ will map the original input $x\in\mathbb{R}^{d_{1}}$ to a lower-dimensional representation $f_{\mathrm{enc}}(x)\in\mathbb{R}^{d_{2}}$, where $d_{1}$ and $d_{2}$ are dimensions of the input and the embedding space, respectively, and $d_{1}>d_{2}$. The decoder $f_{\mathrm{dec}}$ will reconstruct input $x$ from the lower dimensional representation, i.e., $x^{\prime}=f_{\mathrm{dec}}(f_{\mathrm{enc}}(x))$. Denote loss function $L:\mathbb{R}^{d_{1}}\times\mathbb{R}^{d_{1}}\rightarrow\mathbb{R}$. With training data $\{x_{i}\}_{i=1}^{N}$, an autoencoder can be trained by minimizing a loss, $\frac{1}{N}\sum_{i=1}^{N}L(x_{i},x_{i}^{\prime})$. The autoencoder-based deepfake generation tool consists of a shared encoder and two decoders, as shown in Fig. 6. Faceswap-GAN fsgS is one of the most popular and effective publicly available tools based on autoencoder and can generate high-quality deepfake videos dolhansky2020deepfakeS. In this work, we train Faceswap-GAN models to reconstruct videos and generate deepfake videos. The output videos of Faceswap-GAN will contain processing traces left by the neural network.

2 Deepfake Generation and Reconstruction Model $R$

For each target video, we generated a deepfake video by feeding a Faceswap-GAN model with the target video and a video of a public figure with a known identity. The video of the public figure was collected from YouTube and has the same gender as in the template video. For deepfake generation, we swap the face of the public figure onto the face of the person in the target video since public figures are usually the victims of deepfakes.

For deepfake detection using double operations, a journalist does not know which videos were used to generate the potentially fake video and knows only the identity of the video in question. Therefore, the reconstruction model was trained using videos of the known public figure from scenes other than the videos used for deepfake generation. The reconstructing unit $R$ is chosen to share the same neural network structure as the deepfake generating unit, i.e., a Faceswap-GAN. For a specific public figure, we trained a Faceswap-GAN using five videos, with $40,\!000$ iterations to ensure a good reconstruction quality.

Figure 6: A schematic for autoencoder-based deepfake generator. The two encoders have shared weights and are used to extract features of the faces from both persons A and B. The decoders for A and B are trained to reconstruct the faces of persons A and B, respectively. A dashed arrow represents the deepfake generation step, i.e., feature A is sent to decoder B and the reconstructed image is the face of person B with facial expressions from person A.

3 Advantage of Identity-Conditioned Feature Extraction

Let us consider a set of images $S$ containing authentic and deepfake images. Each images is associated with an identity $k\in\{1,\dots,K\}$. $S$ may be decomposed into disjoint sets as follows:

$$S=\bigcup_{k=1}^{K}S^{(k)}=S_{\text{auth}}\cup S_{\text{df }}=S_{0}\cup S_{1},$$ (5)

where $S^{(k)}$ is the set of all images belonging to individual $k$, $S_{\text{auth}}$ and $S_{\text{df}}$ are the sets of all authentic and deepfake images, respectively, and $S_{0}$ and $S_{1}$ are the acceptance region and rejection region partitioned by a decision rule van2004detection.

Let us define $g:S\to\mathbb{R}$ as a powerful manifold-learning feature extractor for deepfake traces extraction so that the extracted 1-D feature $x=g(\text{f})$ for real images $\text{f}\in S_{\text{auth}}$ and fake images $\text{f}\in S_{\text{df}}$ exhibit different distributions. To facilitate our theoretical analysis and simulation, we consider the following hypotheses concerning an observation $x$ for individual $k$:

$$H_{0}:\ x=g\big{(}\text{f}\big{)} \sim\mathcal{N}\big{(}\mu_{0}^{(k)},\sigma^{2}\big{)},\quad\text{f}\in S^{(k)}\cap S_{\text{auth}},$$ (6a)

$$H_{1}:\ x=g\big{(}\text{f}\big{)} \sim\mathcal{N}\big{(}\mu_{1}^{(k)},\sigma^{2}\big{)},\quad\text{f}\in S^{(k)}\cap S_{\text{df}},$$ (6b)

where $\mu_{0}^{(k)}$ and $\mu_{1}^{(k)}$ have Gaussian priors, namely,

$$\mu_{0}^{(k)} \sim\mathcal{N}\left(u_{0},\sigma_{\mu}^{2}\right),$$ (7a)

$$\mu_{1}^{(k)} \sim\mathcal{N}\left(u_{1},\sigma_{\mu}^{2}\right),$$ (7b)

where we set $0=u_{0}<u_{1}\in\mathbb{R}$ without loss of generality, and $\sigma_{\mu}^{2}$ is the variance of the priors. Fig. 7(a) illustrates the probability density functions (PDFs) of $x=g(\text{f})$ under $H_{0}$ and $H_{1}$ for five individuals. When identity information is unknown, the PDFs under each hypothesis merges into one as shown in Fig. 7(b).

(a)

(b)

(c)

Figure 7: Theory-driven simulation results: (a) probability density functions of extracted deepfake feature for $K=5$ identities. Different identities’ feature can have different distributions, as reflected by different $\mu_{0}^{(k)}$ with prior $u_{0}$ and different $\mu_{1}^{(k)}$ with prior $u_{1}$ for each identity $k$; (b) combined probability density function of extracted deepfake feature. If the identity information is not considered, then the individual distributions will mix into a single distribution; and (c) deepfake detection performance with and without the knowledge of the identity. Detection performance is better when the identity information is known. A larger gain can be achieved for the case of more unique individualized deepfake traces (larger $\sigma_{\mu}$) and more difficult detection problems (smaller $|u_{1}-u_{0}|$).

The Bayes risk van2004detection for an arbitrary rejection region $S_{1}$ is defined as

$$r(S_{1})=C_{10}\!\operatorname{\mathbb{P}}(S_{1}|H_{0})\!\operatorname{\mathbb{P}}(H_{0})\!+\!C_{01}\!\operatorname{\mathbb{P}}(S_{0}|H_{1})\!\operatorname{\mathbb{P}}(H_{1}),$$ (8)

where $\mathbb{P}(\cdot)$ is the probability measure, $C_{ij}$ is the cost incurred by choosing $H_{i}$ when $H_{j}$ is true, and $\operatorname{\mathbb{P}}(H_{i})$ is the prior. To focus on the effect of identity conditioning, we assume that the dataset $S$ is balanced, i.e., $\operatorname{\mathbb{P}}(H_{0})=\operatorname{\mathbb{P}}(H_{1})=0.5$ and the incurred costs are the same, i.e., $C_{01}=C_{10}=1$. With these assumptions, the Bayes risk is reduced to the overall error probability $P_{\text{e}}$.

We define $S_{i}^{(k)}=S_{i}\cap S^{(k)}$ to further segment the acceptance region $S_{0}$ and the rejection region $S_{1}$ by individuals:

$$P_{\text{e}} =\frac{1}{2}\big{[}\operatorname{\mathbb{P}}(S_{1}|H_{0})+\operatorname{\mathbb{P}}(S_{0}|H_{1})\big{]}$$ (9a)

$$=\frac{1}{2}\Big{[}\operatorname{\mathbb{P}}\big{(}\cup_{k=1}^{K}S_{1}^{(k)}|H_{0}\big{)}+\operatorname{\mathbb{P}}\big{(}\cup_{k=1}^{K}S_{0}^{(k)}|H_{1}\big{)}\Big{]}$$ (9b)

$$=\frac{1}{2}\left[\sum_{k=1}^{K}\operatorname{\mathbb{P}}(S_{1}^{(k)}|H_{0})+\sum_{k=1}^{K}\operatorname{\mathbb{P}}(S_{0}^{(k)}|H_{1})\right]$$ (9c)

$$=\frac{1}{2}\bigg{\{}1+\sum_{k=1}^{K}\left[\operatorname{\mathbb{P}}(S_{1}^{(k)}|H_{0})-\operatorname{\mathbb{P}}(S_{1}^{(k)}|H_{1})\right]\bigg{\}}.$$ (9d)

Standard hypothesis testing technique van2004detection allows us to derive from (9d) the optimal decision rule that minimizes the Bayes risk or error probability. One can proceed with the derivation and the decision rule turns out to be separable for each individual $k$ and in the form of the likelihood ratio test, namely,

$$S_{1}^{(k)}=\left\{x>\frac{\mu_{0}^{(k)}+\mu_{1}^{(k)}}{2}=T^{(k)}\right\},$$ (10)

where $T^{(k)}$ is the optimal decision threshold.

Using the optimal decision rule, one can calculate the minimal error probability following  (9c):

$$P_{\text{e}}^{\text{ind}}=\frac{1}{2}\sum_{k=1}^{K}\Big{[} \operatorname{\mathbb{P}}(S_{1}^{(k)}|H_{0})+\operatorname{\mathbb{P}}(S_{0}^{(k)}|H_{1})\Big{]}$$ (11a)

$$=\frac{1}{2}\sum_{k=1}^{K}\Big{[} \operatorname{\mathbb{P}}(S_{1}^{(k)}|S^{(k)},H_{0})\operatorname{\mathbb{P}}(S^{(k)}|H_{0})$$

$$+ \operatorname{\mathbb{P}}(S_{0}^{(k)}|S^{(k)},H_{1})\operatorname{\mathbb{P}}(S^{(k)}|H_{1})\Big{]}$$ (11b)

$$=\frac{1}{2K}\!\sum_{k=1}^{K}\!\Big{[}\! \operatorname{\mathbb{P}}(S_{1}^{(k)}\!|S^{(k)}\!,\!H_{0})\!+\!\operatorname{\mathbb{P}}(S_{0}^{(k)}\!|S^{(k)}\!,\!H_{1})\!\Big{]}$$ (11c)

$$=\frac{1}{2K}\sum_{k=1}^{K}\Big{[} 1\!-\!\Phi\Big{(}\tfrac{T^{(k)}-\mu_{0}^{(k)}}{\sigma}\Big{)}\!+\!\Phi\Big{(}\tfrac{T^{(k)}-\mu_{1}^{(k)}}{\sigma}\Big{)}\Big{]}$$ (11d)

$$=\frac{1}{K}\sum_{k=1}^{K}\,\, \Phi\left(-d_{k}\right).\quad\blacksquare$$ (11e)

Here, (11c) is due to the assumption that the identities are uniformly distributed over the dataset, i.e., $\operatorname{\mathbb{P}}(S^{(k)}|H_{0})=\mathbb{P}(S^{(k)}|H_{1})=1/K$, $\Phi$ is the cumulative density function (CDF) of standard Gaussian, and $d_{k}=\big{(}\mu_{1}^{(k)}-\mu_{0}^{(k)}\big{)}\big{/}2\sigma$.

In contrast, when there is no information about the identity, the hypothesis testing problem is reduced to the basic form as shown in Fig. 7(b). One can prove the following identity-agnostic optimal decision rule:

$$S_{1}^{(k)}=\left\{x>\frac{u_{0}+u_{1}}{2}=T\right\},\ \forall k.$$ (12)

The minimal error probability $P_{\text{e}}^{\text{com}}$ with all identities mixed is then given by:

$$P_{\text{e}}^{\text{com}} =\frac{1}{2}\sum_{k=1}^{K}\Big{[}\operatorname{\mathbb{P}}(S_{1}^{(k)}|H_{0})+\operatorname{\mathbb{P}}(S_{0}^{(k)}|H_{1})\Big{]}$$ (13a)

$$=\frac{1}{2K}\sum_{k=1}^{K}\Big{[}1-\Phi\big{(}\tfrac{T-\mu_{0}^{(k)}}{\sigma}\big{)}+\Phi\big{(}\tfrac{T-\mu_{1}^{(k)}}{\sigma}\big{)}\Big{]}.$$ (13b)

Plugging in $T$ and using the second-order Taylor expansion on $\Phi(\cdot)$ around $d_{k}$, we obtain,

$$P_{\text{e}}^{\text{com}}\approx P_{\text{e}}^{\text{ind}}+\frac{1}{2K}\sum_{k=1}^{K}\left[-\Phi^{\prime\prime}\left(d_{k}\right)\right]\alpha_{k}^{2}.\quad\blacksquare$$ (14)

Here, $\alpha_{k}=\big{[}(u_{0}-\mu_{0}^{(k)})+(u_{1}-\mu_{1}^{(k)})\big{]}\big{/}2\sigma$, $\Phi^{\prime\prime}(\cdot)$ is the second-order derivative of $\Phi$, and $-\Phi^{\prime\prime}(d_{k})>0$. This reveals that $P_{\text{e}}^{\text{com}}$ is larger (worse) than $P_{\text{e}}^{\text{ind}}$, highlighting the significance of identity conditioning for detection.

Fig. 7(c) demonstrates the result of $P_{\text{e}}^{\text{ind}}$ and $P_{\text{e}}^{\text{com}}$ generated by a large number of iterations for $u_{1}-u_{0}\in\{0.5\sigma,\,1.0\sigma,\,1.5\sigma\}$. It is observed that the performance is improved when the individual distributions are used by the detector and such effect is amplified with a larger $\sigma_{\mu}$ [i.e., more unique individualized deepfake traces; larger $|\alpha_{k}|$ as in (14)] and with a smaller $|u_{1}-u_{0}|$ [i.e., more intrinsically difficult detection problems; smaller $d_{k}$ in (14) for $\Phi^{\prime\prime}(\cdot)$’s monotonically increasing interval on the positive half of the axis]. We used $K=5$ identities for this simulation and verified via simulation that the performance is not sensitive to the choice of $K$.

4 Fine-Grained Performance Analysis Over Identities

The detection performance for an overall population of unknown composition may not be the most interesting metric from the perspective of a journalist when they target a specific celebrity or politician. Individualized deepfake detection proposed in this work allows more tailored optimization on an individual basis. The performance of the proposed individualized deepfake detector and two baseline methods for every public figure is shown in Fig. 8. The figure reveals that the performance of baseline methods is less consistent across the identity. For some identities, the performance of the baseline methods is significantly worse than their own average performance. This underscores the greater reliability and consistency of the proposed method in deepfake detection of public figures.

Figure 8: The performance of the deepfake detectors, measured in $1-\text{AUC}$ (the smaller, the better), varies with the identities. The red and cyan peaks reveal that the baseline methods without utilizing identity information are less likely to perform well for specific individuals.

ieeenat_fullname supp