Disrupting Diffusion: Token-Level Attention Erasure Attack against Diffusion-based Customization

Abstract.

With the development of diffusion-based customization methods like DreamBooth, individuals now have access to train the models that can generate their personalized images. Despite the convenience, malicious users have misused these techniques to create fake images, thereby triggering a privacy security crisis. In light of this, proactive adversarial attacks are proposed to protect users against customization. The adversarial examples are trained to distort the customization model’s outputs and thus block the misuse. In this paper, we propose DisDiff (Disrupting Diffusion), a novel adversarial attack method to disrupt the diffusion model outputs. We first delve into the intrinsic image-text relationships, well-known as cross-attention, and empirically find that the subject-identifier token plays an important role in guiding image generation. Thus, we propose the Cross-Attention Erasure module to explicitly ”erase” the indicated attention maps and disrupt the text guidance. Besides, we analyze the influence of the sampling process of the diffusion model on Projected Gradient Descent (PGD) attack and introduce a novel Merit Sampling Scheduler to adaptively modulate the perturbation updating amplitude in a step-aware manner. Our DisDiff outperforms the state-of-the-art methods by 12.75% of FDFR scores and 7.25% of ISM scores across two facial benchmarks and two commonly used prompts on average. The codes are available here.

1. Introduction

In the era of pre-trained generative AI, individuals wield the power of creating multi-media contents that appears genuinely authentic with just a single click. Despite the success of GAN- and VAE-based image synthesis methods(Karras et al., 2020; Patashnik et al., 2021; Xu et al., 2018; Tao et al., 2022; Razavi et al., 2019; Liu et al., 2024), more impressive and controllable results are achieved by recently proposed diffusion models, such as DALL-E(Ramesh et al., 2022), Imagen(Saharia et al., 2022) and Stable Diffusion(StabilityAI, 2024). These diffusion model based text-to-image generation methods can generate almost any style of image by a simple corresponding text instruction, and show the unprecedented capacity for producing high-quality images across various applications.

Figure 1. Comparison of DreamBooth, Anti-DreamBooth with DisDiff. The first row shows the unprotected DreamBooth, which learns from the subjects and generates them into different scenes. The second row shows the protected results. Protected by Anti-DreamBooth, the face is still identified. Our DisDiff achieves unidentified or even unrecognized outputs, boosting the protection performances.

With the users’ desire for image synthesis of their subjects, e.g., daily supplies, pets or family members, customized image generation steps onto the stage, with representatives like Textual Inversion(Gal et al., 2023) and DreamBooth(Ruiz et al., 2023). Leveraging the pre-trained text-to-image diffusion models, Textual Inversion learns a pseudo-word in the well-editable word embedding space of text encoder, i.e., $S*$, to represent the newly provided concept. DreamBooth fine-tunes the diffusion models with a small set of subject images and a subject-identifier token (e.g., sks) included prompt, such as ”a photo of sks dog”. Iteratively training large amounts of model parameters, DreamBooth achieves superior performance compared to Textual Inversion. More recently, Custom Diffusion(Kumari et al., 2023) fine-tunes the selected parameters in the attention layers to reduce the memory overhead, achieving efficient customization.

Despite the convenient editing of users’ specified subjects, these methods present a dual-edged dilemma. Malicious users exploit this customization to fabricate synthetic images or videos and spread them on the internet. The generated fake products(Chen et al., 2020; Yang et al., 2023; Kim et al., 2022) engender potential security and privacy vulnerabilities for individuals. In response to defend against this, both passive and proactive protections have been proposed for safeguarding privacy. Passive protections, commonly known as DeepFake detection(Zhao et al., 2021; Zhang et al., 2022; Ge et al., 2022; Iqbal et al., 2023; Wang et al., 2023a; Liao et al., 2023), are employed to determine the authenticity of images. The detecting models are trained to discriminate whether the images are real or fake. However, fake images have been spread before detected and these methods often struggle with novel synthetic techniques(Wang et al., 2021). Regarding this matter, proactive protections are proposed with adversarial attacks to disrupt the fake image synthesis(Yang et al., 2021; Wu et al., 2023). These applications distort the fake image generation with noticeable artifacts, thereby providing cues for detection.

As diffusion-based customization methods emerge, the urgency of protecting users’ privacy against these techniques becomes paramount. As one of the pioneering works, Anti-DreamBooth(Van Le et al., 2023) aims to disrupt the generation of images produced by DreamBooth. Specifically, Anti-DreamBooth tries to alternatively learn a surrogate diffusion model (or utilize a fine-tuned fixed one) and the adversarial perturbations to improve the attack performances. Despite the destruction of the visual quality of generated images by Anti-DreamBooth, the face still exists. This is obvious not only for human observers (visually) but also for detection algorithms (quantitatively). As illustrated in Figure 1, the generated images of both DreamBooth and Anti-DreamBooth are recognized, i.e., there exists an evident face in the image. Besides, the generated face can even be correctly identified, i.e., being classified as the same identity by a face recognition algorithm. These undesired issues of privacy information leakage become the motivation of our work.

In this paper, we propose DisDiff, a novel adversarial attack method against diffusion-based customization. We first delve into the training process of DreamBooth through the lens of the attention mechanism. As training proceeds, the diffusion model gradually establishes the association between ”sks person” and the target individual. This association can be reflected by the cross-attention maps in the UNet(Ronneberger et al., 2015), which serves as the core guidance of text-image generation. In light of this, we propose a novel Cross-Attention Erasure module (CAE), to diminish the cross-attention’s guiding effects. On the other hand, we explore the impact of the sampling procedure of diffusion models on adversarial attacks inspired by (Wang et al., 2023b). We compute Hybrid Quality Scores (HQS) for different timesteps to measure the importance of these steps on adversarial learning. Then, we introduce the Merit Sampling Scheduler (MSS) to adaptively modulate the PGD step sizes during attack, i.e., at the steps where HQS is large, PGD step size is magnified by MSS and vice versa. Both CAE and MSS modules are empirically proven to boost the adversarial attack performances on diffusion-based customizations.

In brief, our contributions are summarised as follows:

1) We propose DisDiff, a proactive adversarial attack method against diffusion-based customization, to protect users’ privacy.

2) We delve into the image-text relationship and justify its importance on generation guidance. On this basis, we propose a novel Cross-Attention Erasure module to ”erase” the subject-identifier token-related attention map.

3) Observing the diffusion sampling process and PGD, we introduce a Merit Sampling Scheduler, which restricts the perturbation updating step length in a step-aware manner.

4) Experiments show that DisDiff outperforms the SOTA method by 12.75% of FDFR scores and 7.25% of ISM scores across two facial benchmarks and commonly used prompts on average.

2. RELATED WORK

2.1. Text-to-image Diffusion

As latent-variable generative models, diffusion models establish a Markov chain of continuous timesteps to gradually introduce Gaussian noise into the training data. Afterward, the models learn to reverse the diffusion process and reconstruct data samples from the induced noises. In response to the growing demand for multimedia generation, considerable research efforts have been devoted to exploring text-to-image diffusion models. Stable Diffusion (SD)(StabilityAI, 2024), rooted in the Latent Diffusion Models(Rombach et al., 2022), significantly amplifies the scope of text-to-image synthesis applications.

With the input of Gaussian noises and user-elaborated prompts, SD generates diversified images through the denoising process. It has been demonstrated that the text prompts serve as an essential controller for image generation, with even minor modifications leading to significantly different outcomes. A body of works focuses on the role of image-text relationships in the generation, which is known as attention maps. For precise SD text-to-image generation, Prompt-to-Prompt(Hertz et al., 2023) explores cross-attention layers during the diffusion backward inference process. Attend-and-Excite(Chefer et al., 2023) modifies the latent codes by maximizing the dominant attention values for each subject token.

2.2. Diffusion based Customization

Despite the diversity of generated images by diffusion models, users often seek to synthesize specific concepts for their personal subjects. This task is named user customization(Brooks et al., 2023; Raj et al., 2023; Li et al., 2024; Wei et al., 2023) and has been widely studied. LoRA(Hu et al., 2022) enables the diffusion models for specific styles or tasks by fine-tuning the extra parameters. This technique allows the model to associate image representations with the prompts describing them effectively. Textual Inversion(Gal et al., 2023) harnesses the diffusion models to reflect a novel concept to a pseudo-word, denoted as $S*$, within the malleable word embedding space of the text encoder. It allows for the seamless integration of unique concepts into the generative process, enhancing the model’s ability to produce tailored and specific imagery.

DreamBooth(Ruiz et al., 2023) fine-tunes the diffusion model with a subject-identifier token consisting prompt (e.g., ”a photo of sks person”) and several images of the indicated subject. Through this process, the model becomes ”familiar” with recognizing features associated with ”sks person,” enabling it to generate various representations in response to different prompts. Despite the convenience of diffusion-based customization, there exist some potential risks. Malicious users may misuse these tools to invade privacy, for instance, generating illicit pornographic images or harmful content. Such misuse highlights the importance of ethical considerations and robust safeguards in the development of text-to-image synthesis technologies.

2.3. Adversarial Attacks on Image Generation

Recognizing the urgency for privacy protection, researchers have introduced adversarial attacks to defend against malicious image manipulations. Inspired by adversarial attacks against classification tasks(Madry et al., 2018; Dong et al., 2018; Zhong and Deng, 2022; An et al., 2024), Yeh et al.(Yeh et al., 2021) and Ruiz et al.(Ruiz et al., 2020) aim at distorting or neutralizing image editing DeepFake techniques. He et al.(He et al., 2022) harness the StyleGAN inversion technique, encoding the protected images into latent space and adding perturbations. CMUA-Watermark(Huang et al., 2022) and TAFIM(Aneja et al., 2022) further train universal watermarks against various image manipulations.

Within the realm of diffusion-based customization for privacy protection, GLAZE(Shan et al., 2023) and AdvDM(Liang et al., 2023) employ invisible perturbations on personal images, protecting users from style theft and painting imitation. Differently, Anti-DreamBooth devises to generate adversarial noises and distort the outputs of customized diffusion methods, especially DreamBooth. SimAC(Wang et al., 2024) further takes a sense of the perception in the frequency domain of images and leverages a greedy algorithm to select timesteps. Different from them, our DisDiff sets out from the image-text relationship and disrupts the internal textual guidance. Besides, we take into consideration the integration of diffusion sampling and adversarial learning and introduce the Merit Sampling Scheduler to adaptively restrict the perturbation updating amplitude.

3. Method

3.1. Preliminaries

Stable Diffusion consists of an autoencoder and a conditional UNet(Ronneberger et al., 2015) denoiser. Firstly, the encoder $\mathcal{E}(\cdot)$ is devised to map a given image $x\in X$ into a spatial latent code $z=\mathcal{E}(x)$. The decoder $\mathcal{D}$ is then trained to map the latent code back to the input image such that $\mathcal{D}(\mathcal{E}(x))\approx x$. Secondly, the conditional denoiser $\epsilon_{\theta}(\cdot)$ is performed to predict the added noises guided by the text prompt $y$. At this time, a pre-trained CLIP(Radford et al., 2021) text encoder $c(\cdot)$ is used to generate text embeddings. Representing the prompt $y$ as a conditioning vector denoted by $c(y)$, the diffusion model $\epsilon_{\theta}$ is trained to minimize the loss function:

(1) $$L_{DM}=\mathbb{E}_{z\sim E(x),y,\epsilon\sim\mathcal{N}(0,\textbf{I}),t}\left\|\epsilon-\epsilon_{\theta}(z_{t},t,c(y))\right\|_{2}^{2}.$$

During inference, latent $z_{T}$ is sampled from $\mathcal{N}(0,\textbf{I})$ and progressively denoised to yield the latent $z_{0}$. Subsequently, $z_{0}$ is fed into the decoder to generate the image $x^{\prime}=\mathcal{D}(z_{0})$.

Figure 2. The overview framework of Disrupting Diffusion. The prompt ”a photo of sks person” is used for training. At every denoising process, we acquire the attention maps of the subject to calculate $L_{DB}$ and $L_{CAE}$. Then, the gradients are aggregated and fed into the Merit Sampling Scheduler, which is used for PGD attack to update $x_{adv}$.

DreamBooth fine-tunes the diffusion model parameters for user customization. The training dataset consists of a subject-specific set $X_{s}$ and a class-specific set $X_{c}$, accompanied by corresponding prompts $y_{s}$ and $y_{c}$, e.g., ”a photo of sks [class noun]” and ”a photo of [class noun]”, respectively. $X_{s}$ comprises various personal images, while $X_{c}$ serves to mitigate model overfitting. Therefore, DreamBooth utilizes a two-part loss to train the diffusion models:

(2) $$\begin{split}L_{DB}=\mathbb{E}_{z\sim E(x_{s}),\epsilon\sim N(0,\textbf{I}),t}\left\|\epsilon-\epsilon_{\theta}(z_{t},t,c(y_{s}))\right\|_{2}^{2}\\ \hskip 11.38109pt+\lambda_{DB}\mathbb{E}_{z\sim E(x_{c}),\epsilon\sim N(0,\textbf{I}),t}\left\|\epsilon-\epsilon_{\theta}(z_{t},t,c(y_{c}))\right\|_{2}^{2},\end{split}$$

where $\lambda_{DB}$ represents a balanced hyper-parameter. In the inference process, users modify the subject-specific prompt to their desire, e.g., ”a photo of sks [class noun] on the mountain”. The model then generates this specific subject on the mountain as described.

Adversarial attacks manipulate model $f$ by introducing imperceptible perturbations to the input data. As to the image generation, the attacker aims to distort or nullify the generation model’s outputs by an optimal perturbation $\delta$. The perturbations are bounded within an $\eta$-ball according to distance metric $\ell_{p}$, where $\eta$ represents the maximum allowed perturbation magnitude. The goal of adversarial attacks is to maximize the loss function, ensuring the outputs are different from the ground truth $y_{true}$. $\delta_{adv}$ is optimized by:

(3) $$\delta_{\text{adv}}=\arg\max L(f(x+\delta),y_{\text{true}}),s.t.\|\delta\|_{p}\leq\eta.$$

Projected Gradient Descent (PGD)(Madry et al., 2018) is a widely used method for adversarial attacks. The perturbations are computed by disrupting the input along the direction of Eq.3 iteratively. Every iteration process of PGD updates the adversarial example $x^{\prime}$ as follows:

(4) $$x^{\prime}_{0}=x,$$

(5) $$x^{\prime}_{k}=\Pi_{x,\eta}(x^{\prime}_{k-1}+\gamma\cdot\text{sgn}(\nabla_{x}L(f(x^{\prime}_{k-1}),y_{\text{true}}))),$$

where $\Pi_{x,\eta}(z)$ confines the pixel values of $z$ within the $\eta$-ball, $\gamma$ represents the step size, and $k$ denotes the number of iterations.

3.2. Cross-Attention Erasure

Motivation. As in the literature(Hertz et al., 2023; Chefer et al., 2023), cross-attention maps represent the influence of tokens on image pixels, which control the text-guided image generation in the diffusion model. Thus, we first delve into the training process of DreamBooth through the lens of attention maps. Specifically, given a textual guidance composed of sequence $W=\{w_{1},w_{2},...,w_{n}\}$, we derive the corresponding attention maps $M=\{m_{1},m_{2},$ $...,m_{n}\}$, where n is the word number. At timestep $t$, $m_{t}$ is computed through the diffusion forward process with latent code $z_{t}$:

(6) $$m_{t}=\text{Softmax}\left(\frac{QK^{\top}}{\sqrt{d}}\right),$$

where query $Q$ is equal to $W_{Q}\cdot c(y)$, key $K$ is equal to $W_{K}\cdot c(y)$. $W_{Q}$ and $W_{K}$ are weight parameters of the query and key projection layers and $d$ is the channel dimension of key and query features.

Figure 3. Visualizations of cross-attention maps from unprotected DreamBooth and DisDiff. For the unprotected ones, the diffusion model captures the subject-identifier token ”sks” (highlighted red areas on the face) and generates customized images. However, DisDiff erases the model’s attention on that token and dramatically distorts the model’s outputs.

Notably, we aggregate these attention maps by averaging across all $16\times 16$ attention layers and heads, as they are proved to convey the most semantic information(Hertz et al., 2023). Besides, it is observed that the original attention maps may not fully reflect whether an object is generated in the image(Chefer et al., 2023). In other words, a single patch with high attention value might originate from partial information passed from the token. To mitigate this, we further apply a Gaussian filter over $m_{t}$, which ensures that the attention value of the maximally-activated patch depends on its neighboring patches, facilitating a smoother attention distribution across the image.

Figure 3 shows the visualizations of attention maps. As the customization prompt is characterized by a subject-identifier token $w_{s}$(”sks”) and a class-identifier token $w_{c}$(”person”), we maintain our focus on their related attention maps. The red highlighted areas reveal high relativity between the token and the generated image. Clearly, for unprotected subjects, the fine-tuned diffusion model successfully associates the subject-identifier token ”sks” with faces. However, the class-identifier token ”person” becomes less prominent. That is, ”sks” actually stands on the subject. We compute the relative energy of the subject-identifier token by:

(7) $$E(m_{s})=\frac{\sum_{A\in m_{s}}A^{2}}{\sum_{m_{i}\in M}\sum_{A\in m_{i}}A^{2}},$$

where $m_{s}$ is the attention map of $w_{s}$ and $A$ stands on the pixel value in the maps. $E(m_{s})$ reflects the energy weight percentage of the subject-identifier token. To protect personal privacy from customization, we thereby try to ”erase” the energy associated with the values in $m_{s}$. Concretely, the proposed Cross-Attention Erasure loss is as follows:

(8) $$L_{CAE}=(1-E(m_{s}))^{2}.$$

Our adversarial attack focuses on maximizing $L_{cae}$, and the optimal perturbation $\delta$ can be learned by training procedure in Section 3.4.

The intuition of our CAE module is that after erasing the cross-attention map of the subject-identifier token (with eliminated highlighted areas), the model would lose the direction of text guidance. As a result, the generated images would exclude the identity information or even the subjects themselves. The protecting results are shown in the last two rows in Figure 3. Compared with highlighted red areas of unprotected DreamBooth(the first two rows), the impact of the subject-identifier token on image generation dramatically decreases. In light of this, the learned perturbations distort image-text correlations of the diffusion model, affecting unidentified or unrecognized inference outputs.

3.3. Merit Sampling Scheduler

Note that in the adversarial learning procedure of DisDiff, the timestep sampling process of diffusion models is included. Thus, to evaluate the importance of steps for adversarial learning, we adopt the Hybrid Quality Score (HQS) metric inspired by (Wang et al., 2023b). Assuming that $G_{t}$ is the computed gradients on images, we first calculate the L1 norm of the gradients by:

(9) $$N_{t}=\sum\limits_{i}|G_{t}^{i}|.$$

$G_{t}$ is then converted by a softmax function to the confident map $p_{t}$. The entropy is computed by:

(10) $$H_{t}=-\sum\limits_{i}p_{t}^{i}\text{log}p_{t}^{i}.$$

Afterward, the sequences $N=\{N_{1},N_{2},...,N_{T}\}$ and $H=\{H_{1},H_{2},\allowbreak{}...,H_{T}\}$ are normalized to [0,1] and the HQS metric is computed as:

(11) $$\text{HQS}(t)=\text{norm}(N_{t})-\text{norm}(H_{t}).$$

Figure 4 shows the HQS evaluation of various timesteps. High HQS stands on high values of gradient norm and low entropy, indicating that the model pays more attention at this timestep and mainly focuses on identity-related areas (such as faces). During early iterations when HQS is high, larger PGD steps may be beneficial for adversarial attacks. Conversely, as the HQS decreases, smaller PGD steps can help fine-tune the generated adversarial samples, as well as avoid excessive deviation from the original input distribution.

Figure 4. Analysis of varying timesteps. As the timestep becomes larger, not only does the HQS metric significantly decrease but also the model pays more attention to identity-irrelevant information, illustrating that the steps in the former are more important for perturbation training.

Besides, we also visualize the cross-attention map of the subject-identifier token for different timesteps. As can be seen, when the timesteps are large, the model pays more attention to the irrelevant information of the identity. As the timestep gets smaller, the attention gradually moves to the identity’s faces. Observations on HQS and attention maps both empirically justify that the timesteps sampling is essential to adversarial attacks.

Therefore, we resort to adopting a time-dependent function as the Merit Sampling Scheduler (MSS) to adjust the perturbation updating steps in PGD adaptively. We empirically design the time-aware decreasing function as:

(12) $$h(t)=\frac{1}{2}(cos(\frac{\pi t}{T})+1),$$

where $T$ is the max timestep to sample so that the amplitude of $h(t)$ is limited in [0,1]. In this regard, our proposed MSS dynamically adjusts the step size of the perturbation updating. The scheduler is then used to be an enhancement for PGD attacks. Given the overall loss $L$, PGD process in Eq.5 is modified as:

(13) $$x^{\prime}_{k}=\Pi_{x,\eta}(x^{\prime}_{k-1}+\gamma\cdot h(t)\cdot\text{sgn}(\nabla_{x}L(f(x^{\prime}_{k-1}),y_{\text{true}}))).$$

By incorporating MSS, we associate adversarial learning with the diffusion model sampling process. The modified PGD updates the perturbations precisely, contributing to better attack performances.

Algorithm 1 Disrupting Diffusion Step

1:Input: Stable Diffusion model $SD$, protected image $X_{adv}$, clean images batch $X_{cl}$, prompt $W=\{w_{1},w_{2},...,w_{n}\}$, scheduler $h(t)$, surogate model steps $t_{1}$, PGD steps $t_{2}$

2:procedure DBSTEP(SD, X,W)

3:     Random sample timestep $t$

4:     $X^{\prime},\_\leftarrow SD(X,W,t)$

5:     $L_{DB}\leftarrow MSE(X_{s}^{\prime},X_{s})$

6:     $SD\leftarrow SD-\alpha_{DB}\nabla L_{DB}$

7:     return $SD$

8:end procedure

9:For $t$ in {1, 2, …, $t_{1}$} :

10: $SD\leftarrow DBSTEP(SD,X_{cl},W)$

11:For $t$ in {1, 2, …, $t_{2}$} :

12: $X_{adv}^{\prime},M_{t}\leftarrow SD(X_{adv},t,W)$

13: $M_{t}\leftarrow\text{Softmax}(M_{t})$

14: $L_{DisDiff}\leftarrow MSE(X^{\prime}_{adv},X_{adv})+\lambda\cdot L_{CAE}(M_{t})$

15: $X_{adv}\leftarrow X_{adv}+\gamma\cdot h(t)\cdot\text{sgn}(\nabla_{X_{adv}}L_{DisDiff})$

16:For $t$ in {1, 2, …, $t_{1}$} :

17: $SD\leftarrow DBSTEP(SD,X_{adv},W)$

3.4. Training Procedure

To sum up, the proposed DisDiff combines the DreamBooth loss with the Cross-Attention Erasure loss as the overall loss:

(14) $$L_{DisDiff}=L_{DB}+\lambda L_{CAE},$$

where $\lambda$ is a trade-off hyper-parameter. To conduct the adversarial attack, we utilize modified PGD (Eq.13) and maximize the loss function to train the perturbations:

(15) $$\delta=\underset{\delta}{\text{argmax}}\,L_{DisDiff}.$$

We conduct an alternative training strategy similar to (Van Le et al., 2023): at each epoch, we first adopt a clean image set $X_{c}$ to train the surrogate model for $t_{1}$ steps. The adversarial example $x^{\prime}$ is then updated by PGD attack for $t_{2}$ steps and the model is once again trained by the adversarial examples.

Table 1. The comparison of attack performances on VGGFace2 and CelebA-HQ datasets.

Dataset	Method	“a photo of sks person”				“a dslr portrait of sks person”
		FDFR↑	ISM↓	FID↑	BRISQUE↑	FDFR↑	ISM↓	FID↑	BRISQUE↑
VGGFace2	w/o Protect(Ruiz et al., 2023)	0.06	0.56	236.37	20.37	0.21	0.44	279.05	7.61
	AdvDM(Liang et al., 2023)	0.10	0.38	359.65	17.39	0.11	0.29	397.64	27.97
	Anti-DB(Van Le et al., 2023)	0.62	0.32	462.12	37.10	0.72	0.27	448.98	38.89
	DisDiff	0.77	0.27	476.28	42.46	0.95	0.06	473.38	41.26
CelebA-HQ	w/o Protect(Ruiz et al., 2023)	0.07	0.63	154.63	13.15	0.30	0.46	221.89	8.75
	AdvDM(Liang et al., 2023)	0.06	0.59	197.59	24.55	0.30	0.42	235.47	16.88
	Anti-DB(Van Le et al., 2023)	0.56	0.44	386.06	41.58	0.48	0.33	384.80	35.86
	DisDiff	0.62	0.40	412.19	44.98	0.55	0.34	400.38	38.35

The training process of DisDiff is illustrated in Algorithm 1. For convenience, we omit the class-specific datasets and prompts of the DreamBooth procedure.

4. EXPERIMENTS

4.1. Experimental Setup

Evaluation Benchmarks.

Figure 5. Comparison under other inference prompts. Four rows show different image edit prompts: distance, expression, action, and location, respectively.

We quantitatively evaluate our approach on two benchmarks: CelebA-HQ(Liu et al., 2015) and VGGFace2(Cao et al., 2018). For each dataset, we choose 50 identities as the protecting subjects. 8 different images are prepared for each subject and divided equally into two subsets: the clean image set and the target protection set. All images undergo center-cropping and resizing procedures, resulting in a uniform resolution of 512 × 512.

DreamBooth and Adversarial Attack Procedures. We address three versions of pre-trained SD from HuggingFace(Face, 2024): v1.4, v1.5 and v2.1. For DreamBooth, the text-encoder and UNet model are trained with a batch size of 2 and the learning rate of $5\times 10^{-7}$ for 1,000 training steps. The prompt is ”a photo of sks person” for the subject images and ”a photo of person” for class-specific datasets. The PGD attack is configured with $\gamma$ = 0.005 and a default noise budget $\eta$ = 0.05 for VGGFace2 and 0.07 for CelebA-HQ respectively. We train the perturbations by 50 iterations, each containing 3 steps of surrogate model training and 6 steps of perturbation training. The hyper-parameter $\lambda$ in $L_{DisDiff}$ is set to 0.1 for both datasets.

Evaluation Metrics. We utilize two widely used image quality metrics, FID(Heusel et al., 2017) and BRISQUE(Mittal et al., 2012), to test the disrupting results. Besides, images generated from these models may lack detectable faces, which we quantify as the Face Detection Failure Rate (FDFR), measured by the RetinaFace detector(Deng et al., 2020). We also extract face recognition embeddings using the ArcFace recognizer(Deng et al., 2019) and compute the cosine distance to the average face embedding of the entire user’s clean image set, referred to as Identity Score Matching (ISM).

Figure 6. Ablation study of (a) the proposed module CAE and MSS, (b) different schedulers.

4.2. Comparison with SOTAs

We conduct comparisons with two SOTA anti-customization baselines: AdvDM(Liang et al., 2023) and Anti-DreamBooth(Van Le et al., 2023). For a fair comparison, we reproduce their source codes and train adversarial examples with equivalent noise budgets. These protected examples are then used to fine-tune the diffusion models (DreamBooth), with prompt ”a photo of sks person.” Then, the trained diffusion models are tested by prompts ”a photo of sks person” and ”a dslr portrait of sks person”. As can be seen in Table 1, our method demonstrates notable performances across four metrics and two prompts. Specifically, higher FDFR rates signify a decreasing number of detected faces, while lower ISM rates indicate less similarity with the original subject. Increased FID and BRISQUE rates emphasize the lower image quality, resulting in significant distortions of the outputs.

Figure 7. Visualizations of identity images, attention maps, and generated images before and after protecting.

To further verify the efficacy, we also test our method under various prompts with distance, expression, action, and location descriptions, respectively. As shown in Figure 5, DreamBooth edits the images as the prompts describe. AdvDM pushes the outputs to the target image so that the results show a similar texture to that image. Although Anti-DreamBooth distorts the output images, the identity is still recognizable. Our proposed DisDiff leads to more unrecognized faces and unidentified subjects by directly disrupting the cross-attention module, which greatly boosts the performances.

4.3. Ablation Study

Impact of Cross-Attention Erasure. To testify to the impacts of our proposed Cross-Attention Erasure, we visualize the attention maps and the corresponding generated images. As shown in the first row of Figure 7, highlighted red areas occur on the identity face for the token ”sks”, which indicates that the diffusion model pays much more attention to the learned identity with ”sks”. This is intuitive since DreamBooth progressively makes the token ”sks” and the special person connected closely. This situation still exists when the identity is protected by AdvDM and Anti-DreamBooth, which is undesired for privacy protection. Compared with them, DisDiff successfully erases the impact of ”sks”. That is, the diffusion model ignores the token ”sks” and its guidance on generation, thus has no idea about whom to generate. The images protected with our method thus exhibit greater blurriness and indistinctiveness.

Module and Scheduler Selection. We conduct the ablation study of the proposed Cross-Attention Erasure (CAE) and Merit Sampling Scheduler (MSS) modules. As shown in Figure 6(a), we set the metrics of baseline method Anti-DreamBooth as 1 and calculate the relative magnitude of DisDiff. Clearly, when utilizing CAE or MSS, the lower ISM and higher FDFR illustrate that the customization outputs become unidentified and unrecognized, indicating DisDiff’s effectiveness. Also, we apply two widely used decreasing functions to serve as the scheduler in MSS: the logarithm decreasing function and diffusion models sampling sequence $\bar{\alpha_{t}}$. We set the scheduler in Eq.12 as the baseline and compare the attack performances in Figure 6(b). Finally, we select the cosine function in Eq.12 as the default, which achieves the lowest ISM rate.

Table 2. Attack performances with different noise budget $\eta$ and * indicates the default setting.

$\eta$	LPIPS	FDFR↑	ISM↓	FID↑	BRISQUE↑
0	-	0.06	0.56	236.37	20.37
0.01	$\approx$ 0	0.08	0.53	271.92	33.93
0.03	0.01	0.59	0.37	426.68	40.20
0.05$*$	0.03	0.77	0.27	476.28	42.46
0.07	0.07	0.82	0.26	485.02	43.63
0.09	0.11	0.85	0.16	494.51	40.83

Table 3. Attack performances on different generator versions on VGGFace2. ”Self” means the perturbation training and DreamBooth process share the same surrogate model, while ”SD v2.1” means the perturbation training with SD v2.1 and DreamBooth with other versions.

Surrogate	Method	SD v1.4				SD v1.5
		FDFR↑	ISM↓	FID↑	BRISQUE↑	FDFR↑	ISM↓	FID↑	BRISQUE↑
w/o Protect		0.09	0.56	231.13	19.79	0.09	0.55	217.22	18.33
self	Anti-DB	0.88	0.07	511.59	42.77	0.93	0.07	517.03	46.09
	DisDiff	0.96	0.04	522.57	46.26	0.99	0.05	512.70	55.90
v2.1	Anti-DB	0.89	0.10	495.50	38.25	0.90	0.12	506.64	44.02
	DisDiff	0.99	0.02	505.66	44.60	0.96	0.08	510.93	50.83

Table 4. Attack performances on VGGFace2 when the DreamBooth training prompt and subject-identifier token are different from the perturbation training stage. $S*$ is ”t@t” for the first row and ”sks” for the second row.

DreamBooth prompt	Method	”a photo of $S_{*}$ person”				”a dslr portrait of $S_{*}$ person”
		FDFR↑	ISM↓	FID↑	BRISQUE↑	FDFR↑	ISM↓	FID↑	BRISQUE↑
”a dslr portrait of sks person”	Anti-DB	0.01	0.16	279.37	19.74	0.57	0.30	459.65	39.25
	DisDiff	0.05	0.20	270.47	21.95	0.85	0.21	476.27	40.14
”sks” → ”t@t”	Anti-DB	0.60	0.34	454.53	40.61	0.41	0.31	396.92	34.50
	DisDiff	0.73	0.28	464.09	42.24	0.71	0.25	414.26	38.87

Noise Budget. The noise budget $\eta$ represents the permissible magnitude of perturbations, and a large noise budget is easily perceptible by humans. LPIPS is used to measure the image quality before and after attack. As shown in Table 2, DisDiff shows high FID and BRISQUE scores when the budget is as small as 0.03. Besides, LPIPS is still low when we set $\eta=0.05$(default), meaning that the perturbation is still invisible. Moreover, the attack performances become stronger as the budget increases, which is also reasonable since the larger the perturbation is, the greater the attack performances can be achieved.

4.4. Robust Analysis

Attacks on different generators. Considering the potential malicious uses of DreamBooth across different diffusion model generators, we conduct experiments to assess the impact. As shown in Table 3, we first test under the self-surrogate setting (indicated ”self”), where the perturbation training and DreamBooth models are the same version. Experiments show that DisDiff achieves better performances than Anti-DreamBooth and generalizes well on SD v1.4 and v1.5. Then, we test the generated perturbations robustness under different generators (indicated ”v2.1”). We employ the adversarial examples generated with SD v2.1 to train DreamBooth under v1.4 and v1.5. The FDFR approaches 1.0 and ISM approaches 0, indicating minimal faces generated. These results illustrate that the adversarial examples perform well against different generators.

Attacks while prompt or subject-identifier mismatching. Assuming that the adversarial learning prompts differ from the ones used to train DreamBooth, we conduct experiments to test the robustness, as illustrated in Table 4. In the first row, we employ ”a photo of sks person” for adversarial learning and ”a photo of a dslr portrait of sks person” for DreamBooth, conducting inference for both. DisDiff and Anti-DreamBooth perform poorly when confronted with the prompt ”a photo of sks person” due to prompt mismatching. However, when presented with the prompt ”a dslr portrait of sks person”, DisDiff demonstrates superior performances. This is partially due to that the training prompt becomes more complicated and the models are overfitting.

We also evaluate the results by changing the DreamBooth subject-identifier token to ”t@t”, as in the second row. Our DisDiff remains well-generalized to ”t@t”. The reason is that we apply the softmax function in CAE. The summary of all the softmax attention maps for every pixel is 1. After erasing the attention map of ”sks”, the values of other tokens become larger. As a result, the attention map of substitute token ”t@t” is erased even though it is not involved in perturbation training. To sum up, DisDiff is robust to different DreamBooth prompts and subject-identifier tokens.

Table 5. Attack performances on other diffusion customization methods on VGGFace2.

	Method	FDFR↑	ISM↓	FID↑	BRISQUE↑
LoRA	w/o Protect	0.11	0.43	248.05	17.31
	Anti-DB	0.68	0.36	403.38	44.38
	DisDiff	0.73	0.21	418.70	44.70
TI	w/o Protect	0.04	0.42	222.70	7.59
	Anti-DB	0.09	0.27	289.66	39.37
	DisDiff	0.14	0.24	308.77	41.23

Attacks on different customization methods. To verify our protection generality across various generation methods, we reproduce wildly used customization methods: Low-rank Adaptation (LoRA)(Hu et al., 2022) and Textual Inversion (TI)(Gal et al., 2023). We train customization with LoRA and TI and conduct adversarial attacks with Anti-DreamBooth and DisDiff. As can be seen in Table 5, compared to unprotected DreamBooth, both Anti-DreamBooth and DisDiff successfully attack these two customization methods. DisDiff shows better evaluation scores, indicating the robustness of protection ability across different customization methods.

5. Conclusion

In this paper, we propose a novel adversarial attack method, Disrupting Diffusion (DisDiff). We track the cross-attention maps and propose a Cross-Attention Erasure module to disrupt these maps. We also introduce a time-aware Merit Sampling Scheduler to adaptively adjust the PGD steps. DisDiff successfully disrupts customization’s outputs as unrecognizable and unidentifiable.