General Phrase Debiaser: Debiasing Masked Language Models at a Multi-Token Level

Abstract

The social biases and unwelcome stereotypes revealed by pretrained language models are becoming obstacles to their application. Compared to numerous debiasing methods targeting word level, there has been relatively less attention on biases present at phrase level, limiting the performance of debiasing in discipline domains. In this paper, we propose an automatic multi-token debiasing pipeline called General Phrase Debiaser, which is capable of mitigating phrase-level biases in masked language models. Specifically, our method consists of a phrase filter stage that generates stereotypical phrases from Wikipedia pages as well as a model debias stage that can debias models at the multi-token level to tackle bias challenges on phrases. The latter searches for prompts that trigger model’s bias, and then uses them for debiasing. State-of-the-art results on standard datasets and metrics show that our approach can significantly reduce gender biases on both career and multiple disciplines, across models with varying parameter sizes.

Index Terms—  Social Bias, Stereotype, Pretrained Language Model, Masked Language Model, NLP

1 Introduction

Recently, masked language models (MLMs) [1, 2, 3, 4, 5, 6] are employed in both traditional tasks like text classification [7, 8, 9] and diverse multimodal tasks [10, 11] when combined with models like image generators [12, 13]. We aim to develop MLMs with minimal human biases, even when the pretraining data unavoidably contains these biases. However, correcting implicit biases in pretrained MLMs can be very challenging, especially considering the high cost of retraining models from scratch.

Existing studies [14, 15, 16, 17, 18, 19] have introduced intuitive approaches that use additional corpus to retrieve contextualized embeddings or locate the biases and fine-tune accordingly. But they are rely on external human-written corpus. Auto-Debias[20] hires the prompt[21] template ”[attribute word] [T]…[T] [MASK]” to guide MLMs to automatically search for prompts that makes the model show its bias, and then fine-tune MLMs with them. Nevertheless, real-world language environments are not so ideal, meaning both attribute words and stereotypes should be treated as multi-token. While these method only correct biases at the word level, lead to struggling at the phrase level.

Motivated by this, we propose an automatic multi-token debias pipeline called General Phrase Debiaser to address the limitations of automatic debiasing mentioned above. The major contributions of our work are:

• Unlike existing methods, we debias MLMs at the phrase granularity. In order to reduce the cost of manually constructing the phrase list, we get the stereotypical phrases filtered from hyperlinks of Wikipedia pages in Phrase Filter Stage.

• With the multi-token debias head we proposed, “discriminative” prompts can be searched in Model Debias Stage. These cloze-style prompts have the highest disagreement in generating stereotypical phrases (e.g., mathematical theory/dance art) with respect to demographic words (e.g., man/woman). Then we fine-tune the model using searched prompts.

• Different from the Auto-Debias’ fine-tuning stage, our approach derives loss from stereotypical phrases, rather than from the entire vocabulary belonging to the model itself. This allows our method to adjust the model parameters more specifically without affecting any other gender-independent word or knowledge.

• We conduct experiments on three well-known open-source MLMs: BERT[1], ALBERT[2], and DistilBERT[5], and achieves state-of-the-art performance (0.12, 0.16, and 52) on SEAT test.

Our code and debiased model files are available at https://
github.com/BingkangShi/general-phrase-debiaser.

Fig. 1: The proposed General phrase Debiaser pipeline has two stages. In phrase filter stage, we filter out stereotypical phrases $S_{weighted}$ and $S_{unweighted}$ from WikiPedia pages with MLM and stereotypical seeds. In model debias stage, we search biased prompts at multi-token granularity, and fine-tuning MLM with them.

Fig. 2: Computation of multi-token JSD loss.

2 General Phrase Debiaser

2.1 Phrase Filter Stage

To minimize the cost of manually constructing stereotypes in many specific domains, we use the MLM which needs to be debiased to filter hyperlinks of WikiPedia pages. Hyperlinked phrases $S_{raw}$ that semantically similar to stereotypical seeds can be filtered out. And stereotype seeds comprising $N_{topic}$ topics need to be manually specified, with each topic having $n_{i}$ hyponyms. We choose career, math, art, and science as topics to construct stereotypes, so $N_{topic}$ is 4 in this paper. It should be noted that the filtered phrases under the math, art, and science topics are generated by our Phrase Filter Stage, while the phrases under the career topic were provided by previous work [15].

Let $\mathcal{M}$ be a MLM and $\mathcal{M}_{CLS}$ be the process of computing the classification embedding (CLS) that represents a sentence. The embedding of a phrase can be computed as follows:

$$\hat{E}\left(x\right)=\frac{1}{\left|T\right|}\sum_{t\in T}{\mathcal{M}_{CLS}\left(t\left(x\right)\right)}\\$$ (1)

where $t$ represents a sentence template, and $T$ is a set that includes all templates. We refer to the 14 blank-filling templates used in the SEAT test [22], such as ”this is a __.” or ”__ is here.”.

Then the cosine similarity between phrase $ph_{raw}\in S_{raw}$ and $w_{i,j}$ can be computed through:

$$d\left(i,j,ph_{raw}\right)=cos(\hat{E}\left(ph_{raw}\right),\ \hat{E}\left(w_{i,j}\right))\\$$ (2)

where $w_{i,j}$ is the $j$-th hyponym of the $i$-th topic, $j\in n_{i}$ and $i\in N_{topic}$.

We define the quantity of $ph_{raw}$ in phrase set ${\hat{s}}_{i,j}$ as $topK_{p}$, according to $d\left(i,j,ph_{raw}\right)$ sorted by ascending order. So we can collect $S_{weighted}$ with:

$$S_{weighted}=(S_{0},S_{1},\cdots,{\ S}_{N_{topic}-1}\ )\\$$ (3)

where the phrase set of $i$-th topic is $S_{i}=({\hat{s}}_{0},{\hat{s}}_{1},\cdots,\ {\hat{s}}_{n_{i}-1}\ )$. After removing duplicate phrases, $S_{weighted}$ is transformed into $S_{unweighted}$.

2.2 Finding Biased Prompts

Previous attempts of Auto-Debias [20] used cloze-style prompts to detect biases in attribute words within stereotypes. Let $\mathcal{V}$ be vocabulary of a MLM, and a prompt $x_{prompt}\in\mathcal{V}$ is a sequence of words with one masked token [MASK] and one attribute token. A MLM can be probed by a cloze-style prompt, such as $x_{prompt}$=”[attribute] majors in [MASK].”. The ”[attribute]” is assigned to be filled in a set $\mathcal{C}=\{(c_{1,1},c_{1,2},\cdots,c_{1,m}),(c_{2,1},c_{2,2},\cdots,c_{2,m}),\cdots\}$ composed of $m$-tuples, derived from the gender word list in [15]. And the position of ”[MASK]” serves the purpose of being predicted by $\mathcal{M}$ for a stereotypical word. So we can obtain stereotypical word probability as:

$$P\left([MASK]=v|\mathcal{M},x_{prompt}\left(c\right)\right)$$ (4)

$$=softmax\left(\mathcal{M}_{[MASK]}\left(v|x_{prompt}\left(c\right)\right)\right)$$

where $v\in\mathcal{V}$. $x_{prompt}\left(c\right)=c\oplus x\oplus\left[{MASK}\right]$ is a string composed of $x_{prompt}$ and $c\in\mathcal{C}$. For example, $x_{prompt}\left(c\right)$ = ”$c$ majors in [MASK].”.

While the above method is effective only when stereotypes are single-token. Thus we introduce a probability calculation method for stereotypes at multi-token granularity (as shown in Fig.2):

$$P_{c_{k,m}}^{(n)}=P(\left[\mathbf{t}\mathbf{a}\mathbf{r}\mathbf{g}\mathbf{e}\mathbf{t}\right]={ph}_{i}|\mathcal{M},x_{prompt}^{{}^{\prime}}(c_{k,m}^{({n})}))$$ (5)

$$=softmax({\sum\limits_{l\in n}{logit_{l}}})$$

while the logit corresponding to the [MASK] token position should be:

$$logit_{l}=\mathcal{M}_{{[MASK]}_{l}}({ph}_{i}|x_{prompt}^{{}^{\prime}}(c_{k,m}^{({n})}))\\$$ (6)

where $\left[\mathbf{t}\mathbf{a}\mathbf{r}\mathbf{g}\mathbf{e}\mathbf{t}\right]$ is multiple [MASK] token sequence of length $n$, and $n$ is the maximum length of stereotypical phrase ${ph}_{i}\in S_{unweighted}$. $c_{k,m}^{({n})}$ containing $n$ [MASK] tokens is the $m$-th phrase of the $k$-th tuple in set $\mathcal{C}$. Here $x_{prompt}\left(c\right)$ should evolve into:

$$x_{prompt}^{{}^{\prime}}\left(c_{k,m}^{({n})}\right)=c_{k,m}^{({n})}\oplus x\oplus\left[{\mathbf{t}\mathbf{a}\mathbf{r}\mathbf{g}\mathbf{e}\mathbf{t}}\right]$$ (7)

$$=c_{k,m}^{({n})}\oplus x\oplus\left[{MASK}\right]\left[{MASK}\right]\cdots\left[{MASK}\right]$$

By repeatedly applying Eq. (5), we can obtain the distributions $P^{(1)}$, $P^{(2)}$, … , $P^{(n)}$ for stereotypical phrases with different token length. In step 2 and step 3 in Fig.1, we use Jensen-Shannon Divergence (JSD), which is a symmetric and smooth Kullback–Leibler divergence (KLD), to measure the difference between multiple distributions $P_{c_{k,1}}^{(n)}+P_{c_{k,2}}^{(n)}+\cdots+P_{c_{k,m}}^{(n)}$ as follow:

$${JSD\ loss}_{k}^{({n})}={\sum\limits_{n}{JSD({P_{c_{k,1}}^{(n)},P_{c_{k,2}}^{(n)},\cdots,P_{c_{k,m}}^{(n)}})}}$$ (8)

$$=\frac{1}{m}{\sum\limits_{i}{KLD\left(P_{c_{k,i}}^{(n)}\middle|\middle|\frac{P_{c_{k,1}}^{(n)}+P_{c_{k,2}}^{(n)}+\cdots+P_{c_{k,m}}^{(n)}}{m}\right)}}$$

In this paper, JSD measures the difference between the two-gender distributions, so $m=2$. The KLD between two distributions $p_{i}$ and $p_{j}$ can be computed as: $KLD\left(p_{i}\middle|\middle|p_{j}\right)={\sum_{v\in\mathcal{V}}{p_{i}(v)}}{\mathit{\log}(\frac{p_{i}(v)}{p_{j}(v)})}$.

The loss of an input $x_{prompt}^{{}^{\prime}}$ can be seen as the sum of the overall probability distribution differences of $\mathcal{C}$:

$$loss\left(x_{prompt}^{{}^{\prime}}\right)={\sum\limits_{k}{JSD\ loss}_{k}^{({multi})}}$$ (9)

$$={\sum\limits_{k}{\sum\limits_{n}{JSD\ loss}_{k}^{(n)}}}$$

Step 2 in Fig.1 shows we employ Beam Search[23] to find biased prompts that maximize the $loss(x_{prompt}^{{}^{\prime}})$. Searched biased prompts will be collected for fine-tuning MLM in the step 3.

2.3 Fine-tuning MLM with Prompts

Model	SEAT-6	SEAT-6b	SEAT-7	SEAT-7b	SEAT-8	SEAT-8b	avg.
BERT	0.48	0.11	0.25	0.25	0.40	0.61	0.35
+Context-Debias[15]	1.13	-	0.34	-	0.12	-	0.53
+FairFil[19]	0.18	0.08	0.12	0.08	0.20	0.24	0.15
+Auto-Debias[20]	0.08	0.02	0.36	0.40	0.12	0.20	0.20
+General Phrase Debiaser	0.00	0.13	0.19	0.10	0.02	0.27	0.12
ALBERT	0.51	0.02	0.58	1.02	0.99	1.20	0.72
+Context-Debias[15]	0.18	-	0.05	-	0.77	-	0.33
+General Phrase Debiaser	0.04	0.30	0.01	0.02	0.33	0.29	0.16
DistilBERT	1.26	0.25	0.31	1.22	0.74	0.98	0.79
+Context-Debias[15]	1.34	-	1.01	-	0.97	-	1.11
+General Phrase Debiaser	0.60	0.32	0.21	0.99	0.23	0.79	0.52

Table 1: Gender debiasing results of SEAT on BERT, ALBERT and DistilBERT. SEAT-6 and SEAT-6b are tests about career, SEAT-7, SEAT-7b, SEAT-8 and SEAT-8b are about discipline. Scores closer to 0 are better. ”-” means the value is not reported in the original paper.

Model	CoLA	SST-2	MRPC	STS-B	QQP	MNLI	QNLI	RTE	WNLI
BERT	0.59	0.93	0.89/0.85	0.89/0.88	0.91/0.88	0.85/0.85	0.92	0.65	0.56
+General Phrase Debiaser	0.56	0.93	0.89/0.84	0.89/0.89	0.90/0.88	0.85/0.85	0.92	0.65	0.56
ALBERT	0.55	0.92	0.92/0.89	0.91/0.91	0.91/0.88	0.85/0.85	0.92	0.73	0.39
+General Phrase Debiaser	0.54	0.93	0.90/0.86	0.91/0.91	0.91/0.88	0.85/0.85	0.92	0.73	0.42
DistilBERT	0.47	0.91	0.89/0.84	0.86/0.86	0.90/0.87	0.82/0.82	0.88	0.58	0.56
+General Phrase Debiaser	0.46	0.91	0.89/0.84	0.86/0.86	0.90/0.87	0.82/0.82	0.89	0.62	0.56

Table 2: GLUE test results of the original and the gender-debiased MLMs with our method.

Given that existing work [15] demonstrated the presence of biases in all parameters of the models, we choose to fine-tune the entire $\mathcal{M}$ to mitigate biases in the model with searched biased prompts in step 2. This corresponds to step 3 as illustrated in Fig.1.

In contrast to the prompt search phase, during the debiasing fine-tuning, we aim to minimize $loss(x_{prompt}^{{}^{\prime}})$ to reduce the distribution discrepancy of $\mathcal{M}$ on $S_{weighted}$ induced by $x_{prompt}^{{}^{\prime}}$. This distribution discrepancy is specific to $S_{weighted}$, indicating that our method propagates gradients through each phrase in the entire stereotype, rather than debiasing the entire vocabulary $\mathcal{V}$ of $\mathcal{M}$ as done in Auto-Debias. As a result, our debiasing approach pays more attention to stereotypical phrases in $S_{weighted}$ and has less impact on unrelated words.

3 Results and Evaluation

3.1 Evaluation Data And Details

Debias Data & Language Capability Data: We evaluate the proposed General Phrase Debiaser on two dataset: (1) the Sentence Embedding Association Test (SEAT) [22] which provides a commonly used metric for assessing biases in PLM embeddings, and (2)the General Language Understanding Evaluation (GLUE) benchmark [24] which measures common language modeling capability. We evaluate our method on 3 MLMs with differnt sized parameters: BERT [1], ALBERT [2], and distilBERT [5], and compare the proposed method with 3 other algorithms: Context-Debias[15], FairFil[19] as well as Auto-Debias[20].

Implementation Details: Hyperparameters play a critical role in final performance [25, 26, 27]. For completeness sake, we then introduce the hyperparameters we used in our study. In step 2 of Fig.1, we use $S_{unweighted}$ which has 624 phrases. The maximum biased prompt length $PL$ is 5 and beam search width $K$ is 100. We use the 5,000 highest frequency words in Wikipedia as the search space $\mathcal{V}$, to avoid noise in the vocabulary $\mathcal{V}$ and speed up the prompt search process. In step 3, we use $S_{weighted}$ with more than 500 phrases instead of $S_{unweighted}$ because $S_{weighted}$ takes into account the varying weights of different stereotypical phrases, resulting in better debiasing effects (as shown in Table 1). And we choose $\mathcal{C}^{*}$ (an extension of $\mathcal{C}$, derived from the gender word list in [15]) as attribute phrases to construct more fine-tuning data. All models are trained with AdamW [28] optimizer and early stopping strategy. Our experiments run on a single NVIDIA 3090Ti.

3.2 Evaluation Result And Analysis

We run General Phrase Debiaser in both career field and discipline field at the same time. The effect size score of the SEAT[16] benchmark we report in Table 1 measures the association between two sets of target concepts and two sets of attributes. It is obtained by calculating the normalized distance between a set of attribute sentence vectors and a set of concept sentence vectors output by the model, and the closer this distance is to 0, the less biased the model is. The result demonstrates that our method is capable of reducing model biases, lowering original average scores of BERT, ALBERT, and DistilBERT in the six SEAT tests from 0.35, 0.72, and 0.79, respectively, to 0.12, 0.16, and 0.52. Furthermore, compared to other approaches in the entire benchmark, including those relying on manual datasets or generating data automatically, General Phrase Debiaser shows the state-of-the-art debiasing performance. We find the global superiority of our model is derived from three aspects through analysis:

Simultaneous Debiasing across Multiple Domains. Our method can effectively eliminate the gender bias in career, math, art, and science simultaneously, without requiring multiple debiasing process on the same model.

Knowledge Debiasing in Phrase Granularity. Our method operates at the phrase granularity rather than the word granularity, making it easier to probe and mitigate biases in disciplines. For example, in Table 2, General Phrase Debiaser achieves the best average score in the four tests (SEAT-7 to SEAT-8) concerning math, art and science.

Keep Language Capability after Debias.we test gender-debiased versions of BERT, ALBERT, and DistilBERT on the General Language Understanding Evaluation (GLUE) benchmark [24]. The test results are presented in Table 2. Gender-debiased versions of BERT, ALBERT, and DistilBERT show a little decrease in scores compared to the original models on the GLUE test, demonstrating our General MLM Debiaser alleviates the bias concerns while also maintaining language modeling capability.

4 Conclusion

Our proposed method can debias MLMs at phrase granularity while also maintaining language modeling capability, and gets state of the art in SEAT test. Although decoder-only LLMs are gaining popularity, we still consider bias mitigation in encoder-only models crucial. Moreover, the concepts presented here can also be applied to cross-modal models involving encoder-only models.