Linguistic Bias in ChatGPT: Language Models Reinforce Dialect Discrimination

Eve Fleisig Genevieve Smith^†^†footnotemark: Madeline Bossi^†^†footnotemark: Ishita Rustagi^†^†footnotemark: Xavier Yin^†^†footnotemark: Dan Klein
University of California, Berkeley
{efleisig, genevieve.smith, madeline_bossi, ishita.rustagi, nzxyin, klein}@berkeley.edu Starred authors all contributed jointly to the process of designing and implementing the project.

Abstract

We present a large-scale study of linguistic bias exhibited by ChatGPT covering ten dialects of English (Standard American English, Standard British English, and eight widely spoken non-“standard” varieties from around the world). We prompted GPT-3.5 Turbo and GPT-4 with text by native speakers of each variety and analyzed the responses via detailed linguistic feature annotation and native speaker evaluation. We find that the models default to “standard” varieties of English; based on evaluation by native speakers, we also find that model responses to non-“standard” varieties consistently exhibit a range of issues: lack of comprehension (10% worse compared to “standard” varieties), stereotyping (16% worse), demeaning content (22% worse), and condescending responses (12% worse). We also find that if these models are asked to imitate the writing style of prompts in non-“standard” varieties, they produce text that exhibits lower comprehension of the input and is especially prone to stereotyping. GPT-4 improves on GPT-3.5 in terms of comprehension, warmth, and friendliness, but it also results in a marked increase in stereotyping (+17%). The results suggest that GPT-3.5 Turbo and GPT-4 exhibit linguistic discrimination in ways that can exacerbate harms for speakers of non-“standard” varieties.

Eve Fleisig^†^†thanks: Starred authors all contributed jointly to the process of designing and implementing the project. Genevieve Smith^†^†footnotemark: Madeline Bossi^†^†footnotemark: Ishita Rustagi^†^†footnotemark: Xavier Yin^†^†footnotemark: Dan Klein University of California, Berkeley {efleisig, genevieve.smith, madeline_bossi, ishita.rustagi, nzxyin, klein}@berkeley.edu

1 Introduction

Despite their growing usage, popular tools such as ChatGPT powered by language models can exhibit harms towards marginalized groups, such as increased stereotyping and poorer performance. A growing area of research has examined harms faced by speakers on the basis of dialect bias–difficulties faced by speakers of dialects, or language varieties, that have fewer speakers or are stigmatized as nonstandard. Given the vast numbers of people who speak varieties of English other than Standard American English (SAE), the variety typically produced by ChatGPT, our research sought to examine how ChatGPT performs for speakers of minoritized (or non-“standard”) varieties of English.

Our work addresses two central questions. First, how does the behavior of ChatGPT differ in response to different varieties of English? Second, how and to what extent (if at all) do ChatGPT responses exhibit harms toward speakers of minoritized varieties of English, such as by stereotyping speakers of minoritized varieties? Because standard varieties of English, particularly SAE, dominate available training data and are prioritized in research and industry contexts, we hypothesized that “standard” varieties of English would be treated as the default and receive innocuous responses. By contrast, we hypothesized that models would produce potentially harmful responses when responding to minoritized varieties.

We prompted both GPT-3.5 Turbo and GPT-4 with text in ten varieties of English: two standard varieties, SAE and Standard British English (SBE); and eight minoritized varieties: African American English (AAE), Indian English, Irish English, Jamaican English, Kenyan English, Nigerian English, Scottish English, and Singaporean English.¹¹1“Standard” language is an “abstracted, idealized, homogeneous spoken language…imposed from above” and modeled on “the written language” Lippi-Green (1994). “Standard” language is not actively spoken by any real community; moreover, all language varieties have more and less “standard” versions. We use “standard varieties” to refer to Standard American English and Standard British English because they have by far the most global prestige and influence. We use “minoritized varieties” for the other varieties tested (African-American, Indian, Irish, Jamaican, Kenyan, Nigerian, Scottish, and Singaporean English). First, to understand whether ChatGPT imitates features of input varieties, we annotated the responses to each variety for a set of paradigmatic linguistic features of that variety (Section 4). Then, to understand whether speakers of minoritized varieties experience performance differences or potential harms when using language models, we surveyed native speakers of each variety for multiple qualities of the generated outputs (Section 5).

In our first study, we find that distinctive linguistic features are reduced in responses to all minoritized language varieties, while responses to SAE and SBE retain the most features by a considerable margin. For minoritized varieties, feature retention appears to correlate with speaker population size. In our second study, we find that model responses to minoritized varieties are generally perceived as more stereotyping, demeaning, unnatural, and condescending; and less able to comprehend the input. We also find that when GPT-3.5 is prompted to imitate the input dialect, its responses exacerbate stereotyping content and lack of comprehension. GPT-4 responses imitating the input improve on GPT-3.5 in terms of warmth, comprehension, and friendliness, but further exacerbate stereotyping.

Given ChatGPT’s presumed excellent performance on English, understanding performance discrepancies for English language varieties globally is critical. These discrepancies can limit language models’ ease of use for minoritized populations, which may exacerbate existing global inequities. Meanwhile, advancement of limiting stereotypes and other harms could discourage speakers of minoritized varieties from using language models and reinforce discriminatory perspectives.

2 Related Work

Languages typically exhibit wide variation associated with speakers from different regions, social groups, or identities Labov (2006); Eckert and Rickford (2009). Speakers of language varieties that do not enjoy status as a “standard” dialect face discrimination across settings including housing, employment, education, and criminal justice Adger et al. (2014); Baugh (2005); Drożdżowicz and Peled (2024); Rickford and King (2016). Dialect discrimination often serves as a proxy for other forms of discrimination, such as racism, classism, and xenophobia Baker-Bell (2020); Wiley and Lukes (1996).

Issues of linguistic discrimination in natural language processing (NLP) have raised increasing concern, since English–and particularly its “standard” varieties–are the status quo Bender (2019); Joshi et al. (2020). Similarly, even within English, prioritization of standard varieties could result in differential performance and opportunity allocation, as well as linguistic profiling Nee et al. (2022).

Previous work has explored some dialect biases in language models. This research has largely focused on AAE, for which studies have found evidence of bias in hate speech detection Sap et al. (2019), language identification Blodgett et al. (2018), speech recognition Koenecke et al. (2020); Martin and Tang (2020); Martin and Wright (2023); Wassink et al. (2022); Zellou and Holliday (2024), and text generation Deas et al. (2023). Hofmann et al. (2024) also find that language models exhibit harmful stereotypes about AAE speakers in hypothetical decisions, such as employment and criminal conviction. On synthetic data for several varieties, Ziems et al. (2023) find disparities on common NLP tasks such as semantic parsing. On other varieties of English, Yong et al. (2023) find mixed results for generation of code-mixed Southeast Asian dialects and Ryan et al. (2024) find disparities on a dialog intent prediction task for Indian and Nigerian English speakers.

Our research aimed to address several gaps in the existing literature. To address that most research has focused on AAE or synthetic data, we studied responses to native speaker-authored text in a large-scale study of ten widely spoken varieties of English globally. In addition, we aimed to understand how harms affect native speakers in the increasingly common setting of casual interaction with a language model such as ChatGPT by having native speakers evaluate open-ended GPT-3.5 Turbo and GPT-4 responses to text in the varieties they speak. To complement previous work based on automatic evaluation metrics with a richer understanding of native speaker perspectives, we recruited native speakers to rate the responses along multiple axes and provide free-text feedback on their experiences.

3 Approach

We selected ten varieties of English (AAE, Indian English, Irish English, Jamaican English, Kenyan English, Nigerian English, SBE, Scottish English, Singaporean English, and SAE) based on factors including first and second language speaker population counts, availability of linguistic literature on the varieties, geographic spread, and socio-historical context. We aimed to include varieties with larger speaker populations, which represent significant potential and current user groups for tools like ChatGPT. It was also essential to select varieties with enough linguistic description to determine distinctive features for each variety. Finally, we ensured that the varieties chosen have a sufficient geographic spread and reflect different socio-historical contexts by which English came to be spoken in a particular area.

English language data was collected from a variety of sources. Nigerian, Jamaican, Indian, Irish, and Kenyan English data was drawn from the International Corpus of English (ICE) Greenbaum and Nelson (1996); Hundt and Gut (2012). For each of these varieties, we chose to only analyze social letters in order to mimic the informal tone and style of text that users would use in dialogue with language models. SAE and SBE were sourced from Reddit posts on US and UK cities’ subreddits, respectively Zhang (2023).²²2This dataset was chosen over others with UK data because it permitted filtering out Northern Irish and Scottish locations. AAE was sourced from Blodgett et al. (2018). Scottish English data was drawn from the correspondence and letters subset of the SCOTS corpus Anderson et al. (2007). Singaporean English data was sourced from the text messages in the CoSEM corpus Gonzales et al. (2024).

3.1 Overview of studies

We conducted two studies to understand language model behavior in response to minoritized varieties. Before assessing potential harms, we first aimed to descriptively characterize model behavior in response to minoritized varieties. For this first study, we prompted GPT-3.5 Turbo to respond to inputs in the minoritized varieties. We annotated the inputs and responses for linguistic features of each variety to understand whether model responses retain features of the input variety (Section 4). We also sought to understand whether certain types of features from an input variety are retained more than others and what factors might influence feature retention.

For our second study, we investigated potential harms that could arise from model responses to minoritized varieties (both by default, and specifically when it attempts to produce a minoritized variety). We first collected additional responses, prompting GPT-3.5 Turbo and GPT-4 to imitate the input varieties when responding to each variety. Responses under each scenario (GPT-3.5 without imitation, GPT-3.5 with imitation, and GPT-4 with imitation) were annotated by native speakers of each variety for a range of potential harms, such as stereotyping content and comprehension of the input. We analyzed the responses to understand how language models may perpetuate harms against native speakers of minoritized varieties, and whether the nature or extent of these harms changes when models explicitly try to imitate the variety or when more powerful models can imitate the features of the variety more convincingly (Section 5).

4 Study 1: Assessing linguistic features of default responses

For our first study, we conducted evaluations to test the following hypotheses: (1) that ChatGPT responses will have a reduction in the features of different varieties of English for all varieties tested except SAE; and (2) that ChatGPT responses will have increased American orthography. Ten of the most prominent features of each variety were selected for our analysis. These features were selected based on existing linguistic descriptions of each variety, focusing on the morphosyntactic features (word- and sentence-level features that can be observed in written data) that the existing documentation deems particularly distinctive (further details and full annotation guides in Appendix B). These features range from distinctive lexical items (e.g. flat meaning ‘apartment’ for SBE) to distinctive sentence structures (e.g. lack of subject-verb inversion in yes-no questions in Indian English).

We also identified the orthography of the output: American, British, or either (no distinctive features found). The focus on American and British orthography stems from the socio-historical context of English colonization in the British Isles, Africa, Americas, and Asia, and the United States’ expanding sphere of influence and colonization efforts in the Pacific, which has led to most English language communities adopting the orthography of Britain or the US (e.g., analyse/analyze, favour/favor). As with the linguistic features discussed above, distinctive orthographic features were determined based on existing linguistic description.

For each variety, we sampled approximately 50 messages³³3We removed content that did not qualify as informal writing, such as newspaper letters to the editor; this resulted in a minimum of 44 messages per variety used to prompt the model. to prompt GPT-3.5 Turbo via the OpenAI API. The inputs provided to the model focus on benign topics related to daily life (e.g. updates about how the author is doing, travel recommendations for particular areas, etc.). The system prompt (Appendix C) encouraged the model to respond directly to the letter. Two reviewers from our research team independently assessed each (input, output) pair for the ten selected distinctive features of the variety, in addition to the orthography. We averaged results for the two reviewers per variety to conduct our evaluations.

4.1 Results

Variety of English	# Features: Inputs	# Features: Outputs	% Retention $\uparrow$
SAE	295	230	78%
SBE	291	210	72%
Indian	73	12	16%
Nigerian	44	5.5	13%
Kenyan	90	9	10%
Irish	26	1	4%
AAE	63	2	3%
Scottish	37	1	3%
Singaporean	40	1	3%
Jamaican	51	1	2%

Table 1: Overview of language varieties and features represented in inputs and GPT-3.5 outputs.

Model outputs retain features of SAE and SBE far more than those of other varieties, though some features of other varieties are still retained.

Appendix A, Table 4 lists the distinctive features retained across input-output pairs for each variety. SAE had the least reduction in linguistic features, with a 77.9% feature retention rate, followed by SBE at 72.2%. Outputs in response to the remaining eight varieties had far lower retention of linguistic features (Table 1). Five varieties experienced only 2-3% feature retention in the generated outputs. Indian, Nigerian, and Kenyan English experienced significant but less extreme reductions of linguistic features in outputs (10-16% retention).

Refer to caption — Figure 1: Estimated maximum speaker population (in millions) vs. retention rate for minoritized varieties.

Feature retention rate correlates with estimated maximum speaker population.

Curiously, the model neither retains features from all minoritized varieties equally nor produces exclusively SAE features. This could be due to the amount of available training data for each variety, which likely depends on the number of speakers. Due to the lack of reliable estimates for the amount of available training data or number of speakers of each variety, we estimate maximum speaker population based on the population of each country where the variety is spoken.⁴⁴4For AAE, we instead estimate speaker population based on the African-American population of the United States. We recognize as a limitation that the relationship between African-American English and the African-American community is ambiguous and contested (AAE speakers may not all be African-American, and vice versa) King (2020). The speaker estimates we use are intended as estimated upper bounds to understand how much data in these dialects is potentially available, and are not meant to unequivocally identify the dialect with the entire community for which it is named. Although members of these populations may not necessarily be speakers of these varieties, and speakers from other regions may also speak these varieties, they serve as approximate estimates for the maximum speaker population. Indeed, the retention rate for minoritized varieties correlates with estimated maximum speaker population for the variety (Figure 1). This suggests that the training data available to language models may influence the extent to which they retain features of different varieties.

In regards to orthography, the percent of outputs in American orthography increased for every language variety, while the percent of outputs in British orthography decreased (Figure 2).⁵⁵5Except AAE, for which no British orthography was observed in inputs or outputs. For all varieties except SAE and AAE, use of American orthography increased by 13-43% in the outputs and use of British orthography decreased by 13-63%. Even for SBE, British orthography decreased significantly in the outputs (-39.71% for British orthography; +29.18% for American orthography).

We also explored common linguistic features of each language variety and whether they are maintained in the outputs. Appendix A, Table 3 gives the three most common distinctive features found in the inputs for each variety. Compared to the inputs, the generated outputs exhibit significant reduction in features for all English varieties except SBE and SAE. For instance, 19 Kenyan English inputs in the data display article omission (e.g. All I wish you is Ø happy stay in Kenyan English rather than All I wish you is a happy stay in SAE), while only one generated output displays this linguistic feature. Thus, GPT-3.5 outputs tend to reproduce “standard” language, while features distinctive to minoritized varieties are omitted.

In contrast, responses to SBE and SAE exhibit some increases in variety-specific linguistic features. There are 44 instances of adverbs modifying verbs in the SBE inputs (e.g. Personally, I find it slightly unethical), and 46 instances in the corresponding outputs. Similarly, 45 SAE inputs include present tense verbs that end in -s with 3rd person subjects (e.g. that helps), while 48 generated outputs include this feature. However, these two features are grammatical in both SBE and SAE; that is, both varieties allow the type of adverb use exemplified above and use 3rd person singular -s marking on verbs. In this way, even these rare cases of feature increase in GPT-3.5 outputs often simply replicate Standard American English features.

The distinctive features that are retained tend to be lexical features, or features that are grammatical in SAE.

To consider which distinctive features were retained across input-output pairs, we calculated retention rates for each feature: if a feature was present in an input and its corresponding output, this example was counted as a retention for that feature. All varieties–except SBE and SAE–have very limited feature retention. Of these varieties, Kenyan and Indian English had the highest retention (3 out of 10 features retained each), while Jamaican English had no features retained. Most other varieties fall in between, with one retained feature each.

The most commonly retained type of feature–seen in all but one English variety in Table 4–is lexical, including borrowed and distinctive words. The retention of lexical features in GPT-3.5 outputs is unsurprising because these features are generally more common and more visible than grammatical features, which relate to more subtle linguistic patterns such as word order or morphological marking (e.g., past tense -ed). In fact, these examples of lexical retention often involve ChatGPT parroting back a word from the input, though sometimes changing the spelling to be more in line with Standard American or British orthography (e.g. GPT-3.5’s our leisure activities, including beer, music, and nyama choma in response to Particularly with beer, music, and nyawa-choma; Kenyan English).

Many more features are retained in the SAE and SBE datasets (9 out of 10 features retained each). However, the vast majority of these retained features are either lexical or grammatical in both SBE and SAE, since these two English varieties have much in common. For SBE, one lexical feature is retained, while eight grammatical features are retained. All eight of these grammatical features are also found in SAE. For SAE, all nine retained features are grammatical. All retained features except for “singular collectives” (i.e. The government is discussing… in SAE vs. The government are discussing… in SBE) are also grammatical in SBE. This pattern highlights that even when GPT-3.5 retains a high number of distinctive features in the language that it produces, this language still closely aligns with Standard American English.

Finally, we examined which distinctive features were introduced by GPT-3.5 (Appendix A, Table 5). If a feature was present in an output but was not in the corresponding input, this example was counted as an introduction for that feature. Only SAE and SBE have feature introductions; no features of any other English variety are introduced by GPT-3.5 Introduced features are uniformly less frequent than retained features: every introduction frequency in Table 5 is lower than the corresponding retention frequency in Table 4. It is also notable that nearly all introduced features are grammatical in both SAE and SBE. The two exceptions are distinctive British lexical items, which are not found in SAE, and singular collective nouns in SAE but not SBE. Both of these features only have a single introduction each. Once again, this pattern highlights that even when GPT-3.5 uses distinctive features in the language that it produces, this language still closely aligns with Standard American English.

5 Study 2: Native speaker evaluation of output disparities

Our second study explored to what extent ChatGPT outputs might perpetuate harms in response to speakers of minoritized language varieties. Our analyses aimed to answer three questions:

•

By default, what harms do native speakers of minoritized varieties face when interacting with language models, relative to speakers of standard varieties?
•

How do these harms change if the language model is specifically prompted to imitate the input variety?
•

Does using a newer model that is better at imitating minoritized varieties (GPT-4 instead of GPT-3.5) improve or worsen these harms?

For this study, after completing the annotation process, we selected all of the input letters for each language variety for which the input or output contained at least one feature. Then, these selected input letters were fed into GPT-3.5 and GPT-4 with a new system prompt that instructs the model to attempt to match the style and tone of the letter in its response. Native speakers of each variety were then recruited to evaluate the responses via Prolific (see Appendix D for details on recruitment, filtering, consent, and compensation). Each annotator completed a survey consisting of twelve input-output pairs in random order, with six of the outputs coming from GPT-3.5 (prompted simply to respond); three from GPT-3.5 (prompted to imitate the input); and three from GPT-4 (prompted to imitate the input). Outputs were distributed such that each output was annotated by at most two annotators. For each variety, at least 11 annotators were recruited (mean=15.7). This resulted in 910 total responses (mean of 91 per variety).

For each (input, output) pair, annotators assessed the output on 5-point Likert scales for nine qualities: stereotyping, demeaning content, condescension, formality, comprehension, naturalness, warmth, friendliness, and respect. We included a short reflection at the end asking questions for speakers to reflect on their evaluation responses. We also asked annotators to optionally provide demographic and background information to ensure we incorporated diversity amongst our native speaker participants, account for commonly found linguistic variation along demographic dimensions, and ensure participant familiarity with the English variety being examined. See Appendix D for details on survey format.

5.1 Results

We first compared responses to minoritized varieties against responses to the “standard” varieties, SAE and SBE, when GPT-3.5 is prompted simply to write a response to the input. Both SAE and SBE exhibit very similar patterns in the results, with ratings within 0.25 points of each other for all criteria except demeaning content and formality.

GPT-3.5 responses to minoritized varieties are perceived as worse than responses to standard varieties on most axes.

On average, responses to minoritized varieties were rated as 22% more demeaning and 16% more stereotyping. Responses to Indian and Jamaican English were seen as most demeaning and responses to Irish and AAE seen as least demeaning. Responses were seen as more stereotyping for all minoritized varieties except AAE, with responses to Nigerian, Indian, and Irish English seen as particularly stereotyping. The fact that AAE is an exception here is unexpected, given the well-documented evidence of discriminatory outputs in response to AAE (e.g., Hofmann et al., 2024; see also Section 2). This could be a result of deliberate efforts to improve performance for AAE on these models, though it is unclear if any such mitigations have been implemented.

Responses to minoritized varieties were also rated on average as 10% worse at comprehending the input and 12% more condescending. Responses for every minoritized variety were seen as more condescending than responses to SAE and SBE, with responses to Jamaican and Singaporean English perceived as particularly condescending. Responses to minoritized varieties were also typically perceived as less natural (10% gap on average). Though several varieties are rated as similarly natural to SAE and SBE (or slightly higher, in the case of Nigerian English), several are rated as significantly less natural (Scottish, Indian, Singaporean, and AAE).

The level of formality differed across varieties, with responses to Indian English rated as most formal and responses to Jamaican English seen as least formal. Warmth and formality tend to be inversely correlated, as expected. Most varieties are rated as similarly warm and friendly. As expected, warmth and friendliness ratings are correlated: for example, Indian English responses are rated as lowest for both criteria and Irish English responses are rated as highest for both criteria. Counterintuitively, responses to non-SAE varieties are generally perceived as more respectful (+12% on average). This could be due to responses in a standard variety being perceived as more respectful by the participants (see also Section 5.2).

The differences in stereotyping, demeaning content, comprehension, naturalness, respect, and condescension between standard and minoritized varieties are all significant at $p=0.05$ . No significant differences were found in warmth, friendliness, or formality.⁶⁶6We performed a two-tailed t-test with Benjamini-Hochberg correction for multiple tests.

Responses imitating the input dialect exacerbate stereotyping content and lack of comprehension.

Comparing the responses in which GPT-3.5 is prompted simply to reply to the input, versus to reply in the style of the input (Figure 4), we find that comprehension decreases across all varieties (-6% for all varieties; -6% for minoritized varieties specifically) and stereotyping increases across all varieties (+9% for all varieties; +11% for minoritized varieties). Formality decreases across all varieties (-14% for all varieties; -15% for minoritized varieties). No significant changes were found along other axes. The increase in stereotyping content and lack of comprehension suggests that imitating the input dialect can exacerbate potential harms. These effects do appear to be relatively uniform: speakers of "standard" varieties and speakers of minoritized varieties reported similar changes. The decrease in formality could be helpful, if it ameliorates undue formality, or could exacerbate harms if perceived as overly familiar.

Imitation by GPT-4 improves on some axes but worsens stereotyping.

Comparing the responses in which the model is asked to imitate the style of the input (Figure 4), we see that imitative responses from GPT-4 are rated as better than imitative responses from GPT-3.5 in terms of comprehension (+11% for all varieties; +11% for minoritized varieties), warmth (+8% for all varieties; +9% for minoritized varieties), and friendliness (+7% for all varieties; +8% for minoritized varieties). These results suggest that GPT-3.5 improves on GPT-4 along multiple dimensions. In particular, comprehension is rated as higher for imitative GPT-4 outputs than even GPT-3.5 without imitation, which could improve quality of service for speakers of minoritized varieties.

However, responses show a marked increase in stereotyping (+17% for all varieties; +12% for minoritized varieties). This result suggests that, although GPT-4 might be better able to imitate features of the input variety, this ability comes at the cost of increased stereotyping.

Formality decreases across all varieties (-19% for all varieties; -17% for minoritized varieties), which could improve or worsen quality of service depending on speaker perspectives. Differences along other axes were not significant. We also see that stereotyping and demeaning content increase more for standard varieties (+39%, +25%) than for minoritized varieties (+12%, +0%): when GPT-3.5 imitates the input style, stereotyping/demeaning content is more severe for minoritized varieties, whereas stereotyping/demeaning content appears to a similar extent across varieties under GPT-4. However, the disparity between the level of stereotyping/demeaning content for minoritized vs. standard varieties shrinks under GPT-4 not because these qualities improve for minoritized varieties, but because they worsen for standard varieties. Stereotyping and demeaning content remain a problem for minoritized varieties in responses from GPT-4. Theresults when the models imitate the input variety suggest that, although GPT-4 improves on GPT-3.5 for several axes, prompting models to produce non-standard varieties does not resolve speakers’ concerns about model responses and in fact introduces new concerns regarding increased stereotyping.

5.2 Qualitative Native Speaker Feedback

When soliciting native speaker feedback, we also asked annotators to provide free-text responses regarding their experience annotating the data. These responses indicated a wide range of attitudes regarding model responses to minoritized varieties.

Several annotators expressed surprise that the models performed as well as they did. One Jamaican English speaker was “kind of impressed that ChatGPT could understand that much from Jamaican [patois].” A Nigerian English speaker reported being “glad chatgpt is almost thinking and responding like people like me,” while another “had a really good time” and “was really surprised that was coming from chatGPT.” Feedback from SAE and SBE speakers was generally positive, though some noted that the responses felt “excessively friendly,” “very formal,” or “somewhat stilted.”

However, others reported that the responses felt unnatural in a variety of ways: a Nigerian English speaker felt “like I was being stereotyped a bit,” a Kenyan English speaker felt that “some responses were not as friendly,” and a Singaporean English speaker felt that some “felt too formal […] a little robotic.” An African American English speaker explained that while AAE speakers are “familiar with the concept of code-switching […] a chatbot can’t make those tweaks,” causing responses to seem “just a little…off.”

Other annotators expressed more frustration and discomfort at the model responses, particularly those imitating the input. One AAE speaker described being “somewhat disturbed” by the idea of chatbots reproducing AAE. A Singapore English speaker wrote that the outputs “do not feel like they’re written by the typical Singaporean” and another felt that “the super exaggerated Singlish in one of the responses was slightly cringeworthy.” These responses highlight the range of reactions that native speakers feel regarding model responses to minoritized varieties, as well as some of the failure modes of model responses: unnatural tones, undue formality, excessive stereotyping, and the potential for appropriation or disparagement of varieties for which discrimination is already common.

6 Discussion & Conclusion

Our research illustrates the differences in GPT-3.5 and GPT-4 responses for different varieties of English. Study 1 finds that GPT-3.5 retains features of SAE and SBE more than features of minoritized varieties of English, and often introduces American orthography. The distinctive features that were retained tended to be lexical items or features that are grammatical in SAE. For minoritized varieties, the feature retention rate correlates with estimated maximum speaker population, potentially reflecting available training data.

Study 2 illustrates how model responses can fail to adequately serve speakers of minoritized varieties through increased stereotyping, demeaning content, condescension, and lack of comprehension. GPT-3.5 responses that specifically attempt to imitate the input further exacerbate stereotyping content and lack of comprehension. Exceptions to this trend highlight places where model quality has improved: GPT-4 responses imitating the input tend to improve on imitative GPT-3.5 responses in terms of comprehension, warmth, and friendliness. However, the GPT-4 responses exhibit even higher levels of stereotyping, suggesting that reducing stereotyping content in response to minoritized varieties is of particular concern.

Current language model responses reinforce the dominance of SAE and, to a lesser extent, SBE. Discrepancies in output quality mean some language communities may not benefit from these tools as much as speakers of “standard” language varieties. Meanwhile, harmful responses can perpetuate discriminatory ideologies about minoritized language communities. As use of language models increases globally, these tools risk reinforcing power dynamics that prioritize speakers of “standard” language varieties over others.

7 Limitations

In beginning the study, we initially sought to access Twitter data. However, we were not able to access data given changes in the leadership of Twitter (now X), which prevented access to the Twitter API for researchers. We therefore pivoted to find informal, written data for the various language varieties from different sources. While we captured informal, written language data for all varieties, some of the data was in the form of letters from the International Corpus of English, while other language varieties were sourced from social media (SAE, SBE, Scottish) and text messages (Singaporean). This meant there were some differences in the level of informality for language data.

In addition, survey responses were collected through Prolific, which is only available in some countries (most OECD countries, Croatia, and South Africa). We used Prolific because it facilitated survey logistics and there were users of each of the ten target English varieties on the platform. However, these users were consolidated primarily in Europe and North America. While some of the ten English varieties come from this part of the world, many other varieties originate elsewhere (e.g., Nigeria, Singapore, etc.). Speakers of English varieties whose countries were not available on Prolific were necessarily based elsewhere. Given that location is a parameter of linguistic variation, it is possible that speakers on Prolific have linguistic differences from those elsewhere, though we are not currently aware of any ways in which this fact impacted our findings.

Our study only examined varieties of English, but dialect discrimination is present in other languages as well. Understanding model behavior in response to different varieties of other languages is an important direction for future work.

Acknowledgments

We are very grateful to Gabriella Licata for the time, effort, and invaluable expertise that she put into supporting this work. Many thanks as well to the Berkeley NLP group for providing feedback on the paper.

References

Adger et al. (2014) Carolyn Temple Adger, Walt Wolfram, and Donna Christian. 2014. Dialects in Schools and Communities. Routledge.
Alo and Mesthrie (2004) M.A. Alo and Rajend Mesthrie. 2004. Nigerian English: Morphology and syntax. In A Handbook of Varieties of English, pages 323–339. De Gruyter Mouton.
Anderson et al. (2007) Jean Anderson, Dave Beavan, and Christian Kay. 2007. Scots: Scottish corpus of texts and speech. In Creating and Digitizing Language Corpora: Volume 1: Synchronic Databases, pages 17–34. Springer.
Baker-Bell (2020) April Baker-Bell. 2020. Linguistic justice. NCTE-Routledge Research Series. Routledge, London, England.
Baugh (2005) John Baugh. 2005. Linguistic profiling. In Black linguistics, pages 167–180. Routledge.
Beare (2019) Kenneth Beare. 2019. American English to British English vocabulary.
Bender (2019) Emily M. Bender. 2019. The #BenderRule: On naming the languages we study and why it matters.
Blodgett et al. (2018) Su Lin Blodgett, Johnny Wei, and Brendan O’Connor. 2018. Twitter Universal Dependency parsing for African-American and mainstream American English. In Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 1415–1425, Melbourne, Australia. Association for Computational Linguistics.
Britain (2007) David Britain. 2007. Grammatical variation in England. In David Britain, editor, Language in the British Isles, pages 75–104. Cambridge University Press.
Buregeya (2013) Alfred Buregeya. 2013. Kenyan English. In The Mouton World Atlas of Variation in English, pages 466–474. De Gruyter Mouton.
Corrigan (2013) Karen P. Corrigan. 2013. The Atlantic Archipelago of the British Isles. In The Oxford Handbook of World Englishes, pages 335–370. Oxford University Press.
Deas et al. (2023) Nicholas Deas, Jessica Grieser, Shana Kleiner, Desmond Patton, Elsbeth Turcan, and Kathleen McKeown. 2023. Evaluation of African American language bias in natural language generation. In Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing, pages 6805–6824, Singapore. Association for Computational Linguistics.
Dictionaries of the Scots Language (2022) Dictionaries of the Scots Language. 2022. Dictionaries of the Scots Language.
Drożdżowicz and Peled (2024) Anna Drożdżowicz and Yael Peled. 2024. The complexities of linguistic discrimination. Philosophical Psychology, page 1–24.
Eckert and Rickford (2009) Penelope Eckert and John R Rickford, editors. 2009. Style and Sociolinguistic Variation. Cambridge University Press, Cambridge, England.
Gargesh and Sailaja (2013) Ravinder Gargesh and Pingali Sailaja. 2013. South Asia. In The Oxford Handbook of World Englishes, pages 425–447. Oxford University Press.
Gonzales et al. (2024) Wilkinson D W Gonzales, Jakob Leimgruber, Mie Hiramoto, and Junjie Lim. 2024. The corpus of singapore english messages (cosem).
Green (2002) Lisa J. Green. 2002. African American English: A linguistic introduction. Cambridge University Press.
Greenbaum and Nelson (1996) Sidney Greenbaum and Gerald Nelson. 1996. The international corpus of english (ice) project. World Englishes, 15(1):3–15.
Gut (2013) Ulrike Gut. 2013. English in West Africa. In The Oxford Handbook of World Englishes, pages 491–507. Oxford University Press.
Hofmann et al. (2024) Valentin Hofmann, Pratyusha Ria Kalluri, Dan Jurafsky, and Sharese King. 2024. Dialect prejudice predicts ai decisions about people’s character, employability, and criminality. Preprint, arXiv:2403.00742.
Huber and Dako (2008) M. Huber and K. Dako. 2008. Ghanaian English: Morphology and syntax. In Rajend Mesthrie, editor, Varieties of English, volume 3. Mouton de Gruyter, Berlin.
Hundt and Gut (2012) Marianne Hundt and Ulrike Gut. 2012. Varieties of English Around the World. John Benjamins Publishing Company, Amsterdam.
Joshi et al. (2020) Pratik Joshi, Sebastin Santy, Amar Budhiraja, Kalika Bali, and Monojit Choudhury. 2020. The state and fate of linguistic diversity and inclusion in the NLP world. In Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics, pages 6282–6293, Online. Association for Computational Linguistics.
Kallen (2013) Jeffrey L. Kallen. 2013. Irish English Volume 2: The Republic of Ireland. De Gruyter Mouton.
Kerswill (2007) Paul Kerswill. 2007. Standard and non-standard English. In David Britain, editor, Language in the British Isles, pages 34–51. Cambridge University Press.
King (2020) Sharese King. 2020. From african american vernacular english to african american language: Rethinking the study of race and language in african americans’ speech. Annual Review of Linguistics, 6(1):285–300.
Koenecke et al. (2020) Allison Koenecke, Andrew Nam, Emily Lake, Joe Nudell, Minnie Quartey, Zion Mengesha, Connor Toups, John R. Rickford, Dan Jurafsky, and Sharad Goel. 2020. Racial disparities in automated speech recognition. Proceedings of the National Academy of Sciences, 117(14):7684–7689.
Labov (2006) William Labov. 2006. The social stratification of English in New York city. Cambridge University Press.
Leimgruber (2013) Jakob R. E. Leimgruber. 2013. Singapore English: Structure, variation, and usage. Cambridge University Press.
Lim (2013) Lisa Lim. 2013. Southeast Asia. In The Oxford Handbook of World Englishes, pages 448–471. Oxford University Press.
Lippi-Green (1994) Rosina L. Lippi-Green. 1994. Accent, standard language ideology, and discriminatory pretext in the courts. Language in Society, 23:166.
Martin and Tang (2020) Joshua L Martin and Kevin Tang. 2020. Understanding racial disparities in automatic speech recognition: The case of habitual" be". In Interspeech, pages 626–630.
Martin and Wright (2023) Joshua L Martin and Kelly Elizabeth Wright. 2023. Bias in automatic speech recognition: The case of african american language. Applied Linguistics, 44(4):613–630.
Millar (2007) Robert McColl Millar. 2007. Northern and Insular Scots. Edinburgh University Press.
Nee et al. (2022) Julia Nee, Genevieve Macfarlane Smith, Alicia Sheares, and Ishita Rustagi. 2022. Linguistic justice as a framework for designing, developing, and managing natural language processing tools. Big Data & Society, 9(1):205395172210909.
Oxford English Dictionary (2023) Oxford English Dictionary. 2023. Introduction to Indian English.
Rickford and King (2016) John R Rickford and Sharese King. 2016. Language and linguistics on trial: Hearing rachel jeantel (and other vernacular speakers) in the courtroom and beyond. Language, pages 948–988.
Ryan et al. (2024) Michael J Ryan, William Held, and Diyi Yang. 2024. Unintended impacts of llm alignment on global representation. arXiv preprint arXiv:2402.15018.
Sand (2013) Andrea Sand. 2013. Jamaican English. In The Mouton World Atlas of Variation in English, pages 210–221. De Gruyter Mouton.
Sap et al. (2019) Maarten Sap, Dallas Card, Saadia Gabriel, Yejin Choi, and Noah A Smith. 2019. The risk of racial bias in hate speech detection. In Proceedings of the 57th annual meeting of the association for computational linguistics, pages 1668–1678.
Schmied (2013) Josef Schmied. 2013. East African English. In The Oxford Handbook of World Englishes, pages 472–490. Oxford University Press.
Sharma (2005) Devyani Sharma. 2005. Language transfer and discourse universals in Indian English article use. In Studies in Second Language Acquisition, pages 535–566. Cambridge University Press.
Turner (2023) Camille Turner. 8 American English grammar rules to sound like you’re from the States [online]. 2023.
Wassink et al. (2022) Alicia Beckford Wassink, Cady Gansen, and Isabel Bartholomew. 2022. Uneven success: automatic speech recognition and ethnicity-related dialects. Speech Communication, 140:50–70.
Wiley and Lukes (1996) Terrence G. Wiley and Marguerite Lukes. 1996. English-only and standard english ideologies in the u.s. TESOL Quarterly, 30(3):511.
Yong et al. (2023) Zheng Xin Yong, Ruochen Zhang, Jessica Forde, Skyler Wang, Arjun Subramonian, Holy Lovenia, Samuel Cahyawijaya, Genta Winata, Lintang Sutawika, Jan Christian Blaise Cruz, Yin Lin Tan, Long Phan, Long Phan, Rowena Garcia, Thamar Solorio, and Alham Aji. 2023. Prompting multilingual large language models to generate code-mixed texts: The case of south East Asian languages. In Proceedings of the 6th Workshop on Computational Approaches to Linguistic Code-Switching, pages 43–63, Singapore. Association for Computational Linguistics.
Zellou and Holliday (2024) Georgia Zellou and Nicole Holliday. 2024. Linguistic analysis of human-computer interaction. Frontiers in Computer Science, 6:1384252.
Zhang (2023) Jiuyu Zhang. 2023. Reddit us uk subreddits dataset.
Ziems et al. (2023) Caleb Ziems, William Held, Jingfeng Yang, Jwala Dhamala, Rahul Gupta, and Diyi Yang. 2023. Multi-VALUE: A framework for cross-dialectal English NLP. In Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 744–768, Toronto, Canada. Association for Computational Linguistics.

Appendix A Additional Details on Feature Annotation

Table 2 provides details on changes in orthography between the input and output. Table 3 details the top distinctive features for each variety. Table 4 details the top features retained per variety. Table 5 lists the distinctive features that were introduced in GPT-3.5 outputs for the varieties of English in our study and the percentage of input-output pairs that contain an introduction for that feature.

	Orthography
	American		British		Either
Variety	Input	Change	Input	Change	Input	Change
SAE	47%	+5%	0%	-2%	50%	-3%
SBE	4%	+29%	45%	-40%	51%	+11%
AAE	2%	+9%	0%	0%	98%	-9%
Indian	0%	+36%	59%	-50%	41%	+14%
Irish	1%	+26%	58%	-56%	40%	+30%
Jamaican	1%	+22%	13%	-13%	86%	-9%
Kenyan	4%	+34%	34%	-30%	62%	-4%
Nigerian	2%	+43%	22%	-21%	76%	-22%
Scottish	0%	+26%	67%	-63%	33%	+37%
Singaporean	1%	+13%	28%	-22%	71%	+9%

Table 2: Orthographic changes in inputs and GPT-3.5 outputs.

English Variety	Feature Name	Example	Input Count	Output Count
Nigerian	Article omission	do __ traditional wedding	11.5	0.5
	Borrowed words	out in oyibo land	6	2
	Extended progressive	I’ve been having testimonies	3.5	0
Kenyan	Article omission	prosper for __ better life	31.5	2.5
	Borrowed words	removing maize from the shamba	19.5	5
	Extended progressive	I’m hoping that the date was changed	19	1
African American	Distinctive words	just been put on blast	16	2
	Copula omission	You __ cool	15.5	0
	Invariant present	Emanda don’t consider	10.5	0
Jamaican	-ed optionality	my friend who use to live	7.5	0
	Article omission	to __ new area	4.5	0
	Invariant present	He don’t know	4.5	0
Indian	Article omission	I was __ research fellow	20.5	0.5
	Borrowed words	he misses chappattis	17	7
	Distinctive words	I have fixed Monday 5th February	15	2
Irish	Object inversion	Nadine hadn’t it done at that time	3.5	0
	Do be	I do be living in Cork	2.5	0
	Borrowed words	hear all the craic	1.5	0
Singaporean	Copula omission	Your parcel __ stuck at customs	9	0
	-ed optionality	disciplined and focus_ girl	8.5	0
	Invariant present	Tomorrow never come	5	0
Scottish	Borrowed words	tonight, Hogmanay	21.5	1
	-na	I didna see	5	0
	Cleft constructions	It was one of the few games I enjoyed	3	0
Standard British	Adverbs	I’m currently doing	43.5	45.5
	Comparison	better career options	37	29.5
	Distinctive words	helped me find flats	32.5	16.5
Standard American	Copula required	Fallon is the next town	48	46
	3rd singular -s	if that helps out	45	47.5
	Relative clauses	forest which will pretty much	41.5	37.5

Table 3: Most common distinctive features per English language variety

English Variety	Feature Name	Example	Retention Frequency
Nigerian	Borrowed words	you believe Alayi dialete will	4%
Kenyan	Borrowed words	regarding the harambee	10%
	Article omission	in __ T.T. Cool atmosphere	4%
	Extended progressive	you are trusting in God	2%
African American	Distinctive words	being called "bae"	4%
Jamaican	—
Indian	Borrowed words	sad news of Shri Panchawagh’s passing	14%
	Distinctive words	purchased a flat	4%
	Off	shed off all your teaching responsibilities	4%
Irish	Borrowed words	Enjoy the craic!	2%
Singaporean	Borrowed words	to best support ahma	2%
Scottish	Borrowed words	tonight on Hogmanay	2%
Standard American	Copula required	Mount Rushmore is on my list	92%
	3rd singular -s	it sounds quite affordable	88%
	Adverbs	I’ll definitely keep them in mind	67%
	Relative clauses	infrastructure that Alaska has	63%
	Comparison	the craziest or coolest	41%
	There existentials	there are still some great places	21%
	Singular collectives	the … commission seems like	8%
	Single negation	may not charge a fee	7%
	Distinctive words	with its bars and music venues	6%
Standard British	3rd singular -s	it sounds like	96%
	Adverbs	I’ll definitely keep that	81%
	Relative Clauses	struggles that contribute to	71%
	Comparison	more lively	46%
	Distinctive words	open to flatsharing	31%
	Past distinctions	a customer’s bag went missing	16%
	There existentials	there are accommodations	11%
	Single negation	I’m not a big clubber	8%
	Reflexives	as a cyclist myself	4%

Table 4: Features retained across inputs and outputs. Distinctive features that were never retained are omitted.

English Variety	Feature Name	Example	Introduction Frequency
Standard American	Adverbs	the Senate carefully considers	16%
	Relative clauses	stereotypes that hinder progress	12%
	There existentials	there are plenty of activities	9%
	Comparison	the broader context	8%
	Reflexives	if I find myself in Brookings	7%
	3rd singular -s	the Game Loop sounds like	7%
	Singular collectives	the city of Tempe has taken	2%
Standard British	Comparison	it’ll make it easier	13%
	Relative clauses	anything else __ I should keep in mind	12%
	There existentials	there are helpful people	12%
	Adverbs	I will definitely google it	10%
	Reflexives	musicians like yourselves	4%
	Single negation	I don’t mind paying a bit	4%
	3rd singular -s	it seems like	3%
	Distinctive words	I’ll check the timetables	2%
	Past distinctions	I saw the second hand	2%

Table 5: Features introduced in outputs that are not present in inputs. Distinctive features not in the table were never introduced (i.e. have an introduction frequency of 0%). Only introduction of SAE and SBE features was found in the data.

Appendix B Annotation guides for linguistic features

B.1 Standard American English

Distinctive words: Annotate as 1 any words that are unique to SAE (Beare 2019).

•

bathroom
•

fries
•

yard
•

vacation
•

etc.

Reflexives: Annotate as 1 any instance of the reflexive pronouns myself, yourself, himself, herself, ourselves, yourselves, themselves (Kerswill 2007: 43).

3rd singular -s: Annotate as 1 any instance of 3rd person singular -s on a present tense verb (Britain 2007: 86).

•

He swims.
•

She eats.

Singular collectives: Annotate as 1 any instance of a collective noun that triggers singular verbal agreement (Turner 2023).

•

The staff is taking the day off.

Copula required: Annotate as 1 any instance of the auxiliary verb be.

•

The dog is barking.

Single negation: Annotate as 1 any instance of single negation where double negation would be possible in other English varieties (Kerswill 2007: 43).

•

I don’t want any.

Adverbs: Annotate as 1 any instance of an -ly adverb modifying a verb (Kerswill 2007: 43).

•

Come quickly!

Relative clauses: Annotate as 1 any instance of a relative clause introduced by that, which, or a null relativizer.

•

the book (that) you gave me

There existentials: Annotate as 1 any instance of the plural verbs are or were in a there existential with a plural subject (Britain 2007: 91).

•

There were papers scattered everywhere.

Comparison: Annotate as 1 any instance of a comparative or superlative with only one instance of comparative or superlative morphological marking (Britain 2007: 103).

•

It’s easier than it used to be.

B.2 Standard British English

Distinctive words: Annotate as 1 any words that are unique to Standard British English (Beare 2019).

•

loo ‘bathroom’
•

biscuit ‘cookie’
•

crisps ‘chips’
•

rubbish ‘trash’
•

holiday ‘vacation’
•

etc.

Reflexives: Annotate as 1 any instance of the reflexive pronouns myself, yourself, himself, herself, ourselves, yourselves, themselves (Kerswill 2007: 43).

3rd singular -s: Annotate as 1 any instance of 3rd person singular -s on a present tense verb (Britain 2007: 86).

•

He swims.
•

She eats.

Do: Annotate as 1 any instance of do or did being used as a main verb, but not done (Kerswill 2007: 43).

•

I did my homework.

Past distinctions: Annotate as 1 any instance of a past verb like saw, did (as a main verb), ate, etc. where the simple past form of the verb is different from its past participle form (i.e. seen, done, eaten; Kerswill 2007: 43).

•

I saw the film.

Single negation: Annotate as 1 any instance of single negation where double negation would be possible in other English varieties (Kerswill 2007: 43).

•

I don’t want any.

Adverbs: Annotate as 1 any instance of an -ly adverb modifying a verb (Kerswill 2007: 43).

•

Come quickly!

Relative clauses: Annotate as 1 any instance of a relative clause introduced by that, which, or a null relativizer.

•

the book (that) you gave me

There existentials: Annotate as 1 any instance of the plural verbs are or were in a there existential with a plural subject (Britain 2007: 91).

•

There were papers scattered everywhere.

Comparison: Annotate as 1 any instance of a comparative or superlative with only one instance of comparative or superlative morphological marking (Britain 2007: 103).

•

It’s easier than it used to be.

B.3 African American English

Distinctive words: Annotate as 1 any words that are unique to AAE (Green 2002: 21-31).

•

ashy ‘the whitish coloration of black skin due to exposure to the cold and wind’
•

kitchen ‘the hair at the nape of the neck which is inclined to be very kinky’
•

saditty ‘uppity acting Black people who put on airs’
•

etc.

Habitual be: Annotate as 1 any instance of the verb be used as an invariant auxiliary verb to indicate the recurrence of an event (Green 2002: 25).

•

They be waking up too early.

Remote past been: Annotate as 1 any instance of the invariant auxiliary verb been used to situate an event or the start of an event in the remote past (Green 2002: 25, 56).

•

They been left.

Invariant present: Annotate as 1 any instance of a present tense verb with a 3rd person singular subject that lacks -s morphological marking (Green 2002: 38).

•

He eat_.

Copula omission: Annotate as 1 any instance of omission of the verb be in contexts where it’s required in SAE (Green 2002: 38-41).

•

She __ tall.

Ain’t: Annotate as 1 any instance of the word ain’t (Green 2002: 39-41).

•

He ain’t been eating.

Done: Annotate as 1 any instance of the invariant auxiliary verb done used to indicate that an event has ended (Green 2002: 60).

•

I told him you done changed.

Double negation: Annotate as 1 any instance of multiple negators like don’t, no, and nothing used in a single negative sentence (Green 2002: 77, 79).

•

I don’t never have no problems.

No ’s: Annotate as 1 any instance of possession indicated by putting the possessor and the noun next to each other, with no need for ’s (Green 2002: 102).

•

Sometime Rolanda_ bed don’t be made up.

It/they existentials: Annotate as 1 instances of the words it and they used in constructions to indicate that something exists (Green 2002: 80).

•

It’s some coffee in the kitchen.

B.4 Indian English

Borrowed words: Annotate as 1 any words that have been borrowed into Indian English from other languages spoken in India (Oxford English Dictionary 2023).

•

bhajan ‘a devotional song’
•

dupatta ‘a doubled or two-layered length of cloth worn by women as a scarf, veil, or shoulder wrap’
•

sadhana ‘dedicated practice or learning to achieve an (esp. spiritual) goal’
•

etc.

Distinctive English words: Annotate as 1 any instance of English words that are used in a distinctive way in Indian English (Oxford English Dictionary 2023).

•

kitty party ‘a social lunch at which those attending contribute money to a central pool and draw lots, the winner receiving the money and hosting the next lunch’
•

lunch home ‘a small restaurant or other eatery’
•

shuttler ‘a badminton player’
•

etc.

Extended progressive: Annotate as 1 any instance of progressive aspect (i.e. be + verb-ing) used in innovative contexts when compared to SAE, especially with stative verbs like have, know, understand, and love (Gargesh and Sailaja 2013: 435).

•

Mohan is having two houses.

Off: Annotate as 1 the particle off combining with a range of verbs to change the meaning slightly (Oxford English Dictionary 2023).

•

Let’s finish it off. ‘Let’s finish it and be done with it.’

Transitivity swap: Annotate as 1 any instance of verbs that are transitive in SAE acting intransitively in Indian English or verbs that are intransitive in SAE acting transitively in Indian English (Oxford English Dictionary 2023).

•

We enjoyed __ very much.

Terms of address: Annotate as 1 any terms of address that appear after the person’s name rather than before (Oxford English Dictionary 2023).

•

Mangesh uncle

No inversion: Annotate as 1 any instance where subjects and verbs don’t invert in questions (Gargesh and Sailaja 2013: 435).

•

What you would like to read?

Embedded inversion: Annotate as 1 any instance where subjects and verbs in embedded questions invert (Gargesh and Sailaja 2013: 435).

•

We asked when would you begin.

Invariant isn’t it: Annotate as 1 the expression isn’t it used invariably as a tag or echo question (Gargesh and Sailaja 2013: 435).

•

You are going tomorrow, isn’t it?

Article omission: Annotate as 1 any instance of an article like a or the being omitted in contexts where it would be required in SAE (Sharma 2005: 545-546).

•

What about getting __ girl to marry from India?

B.5 Kenyan English

Borrowed words: Annotate as 1 any words that have been borrowed into Kenyan English from Swahili and other languages spoken in Kenya (Schmied 2013: 479-481).

•

ugali ‘maize-based dish’
•

matatu ‘collective taxi, minibus’
•

pole (sana) ‘sorry, politeness expression’
•

etc.

Article omission: Annotate as 1 any instance of an article like a or the being omitted in contexts where it would be required in SAE (Buregeya 2013: 468).

•

He noted that __ Electoral Commission of Kenya expects the Government to come out and explain itself.

Invariant isn’t it: Annotate as 1 the expression isn’t it used invariably as a tag or echo question (Buregeya 2013: 468).

•

We are all God’s children, isn’t it?

Myself: Annotate as 1 any instance of myself used as a subject in coordinations with and (Buregeya 2013: 467).

•

My brother and myself live far away from our family home.

Object pronoun drop: Annotate as 1 any instance of an object pronoun (i.e. words like it, him, us) being omitted where it would be required in SAE (Buregeya 2013: 467).

•

I really appreciate __.

Non-count plural marking: Annotate as 1 any instance of a mass noun (i.e. a noun that can’t combine directly with numbers) getting plural marking with -s (Buregeya 2013: 467).

•

We sell equipments.
•

etc.

•

Are you understanding me?

Than what: Annotate as 1 any instance of what following than in a comparative clause (Buregeya 2013: 468).

•

It’s harder than what you think.

No inversion: Annotate as 1 any instance where subjects and verbs don’t invert in questions (Buregeya 2013: 469).

•

We’ll meet him where?

Pronoun + subject doubling: Annotate as 1 any instance of subjects being doubled using pronouns that appear at the beginning of the sentence (Buregeya 2013: 469).

•

Us, we love money.

B.6 Nigerian English

Borrowed words: Annotate as 1 any words that have been borrowed into Nigerian English from other languages spoken in Nigeria (Gut 2013).

•

oga ‘master’
•

dodo ‘fried plantain’
•

burukutu ‘a type of alcoholic drink’
•

etc.

•

I am smelling something burning.

Doubly marked past: Annotate as 1 any instance of the past tense in negatives and interrogatives being doubly marked with the past tense form of do and the past tense verb form (Alo and Mesthrie 2004: 325).

•

He did not went.

Invariant isn’t it: Annotate as 1 the expression isn’t it used invariably as a tag or echo question (Alo and Mesthrie 2004: 327).

•

You like that, isn’t it?

Article omission: Annotate as 1 any instance of an article like a or the being omitted in contexts where it would be required in SAE (Alo and Mesthrie 2004: 331).

•

have __ bath
•

give __ chance
•

etc.

Non-count plural marking: Annotate as 1 any instance of a mass noun (i.e. a noun that can’t combine directly with numbers) getting plural marking with -s (Alo and Mesthrie 2004 via Gut 2013).

•

furnitures
•

equipments
•

aircrafts
•

etc.

Resumptive pronouns: Annotate as 1 any relative clause that contains a resumptive pronoun (i.e. a pronoun within the relative clause that refers back to the noun at the beginning of the relative clause; Huber and Dako 2008: 372 via Gut 2013).

•

the book that I read it

To variation: Annotate as 1 any instance of infinitive to being absent with verbs where it would appear in SAE or being added to verbs where it wouldn’t appear in SAE (Alo and Mesthrie 2004: 329).

•

enable him __ do it

Unmarked comparatives: Annotate as 1 any instance of a comparative appearing without comparative morphology like -er (Alo and Mesthrie 2004: 330).

•

He has __ money than his brother.

Reduplication: Annotate as 1 any instance of adjectives or adverbs undergoing reduplication (i.e. doubling of a word or a part of a word) for word formation or emphasis (Alo and Mesthrie 2004: 336).

•

small-small things ‘insignificant things’

B.7 Jamaican English

No -ed: Annotate as 1 any instance of a past tense verb form that would have -ed in SAE but appears with no -ed in Jamaican English (Sand 2013: 214).

•

When I first started this, they terrify_ the hell out of me.

Non-count plural marking: Annotate as 1 any instance of a mass noun (i.e. a noun that can’t combine directly with numbers) getting plural marking with -s (Sand 2013: 212).

•

toxic wastes
•

etc.

Article omission: Annotate as 1 any instance of an article like a or the being omitted in contexts where it would be required in SAE (Sand 2013: 212).

•

__ Computer is a thing that every day you learn.

The + proper name: Annotate as 1 any instance of a definite article like the used with proper names or names of institutions or groups of people (Sand 2013: 212).

•

In 1987 the Victoria Park was transformed.

•

At least we’re agreeing with the DEH.

Copula omission: Annotate as 1 any instance of omission of the verb be in contexts where it’s required in SAE (Sand 2013: 215).

•

Mary __ in the garden.

Auxiliary omission: Annotate as 1 any instance of auxiliary verbs (e.g. form of be or have) omitted where they would be required in SAE (Sand 2013: 215).

•

What __ you been up to?

Double negation: Annotate as 1 any instance of multiple negators like don’t, no, and nothing used in a single negative sentence (Sand 2013: 214).

•

Me and him don’t have nothing.

Invariant present: Annotate as 1 any instance of a present tense verb with a 3rd person singular subject that lacks -s morphological marking (Sand 2013: 214).

•

I’m a person who love_ music.

No inversion: Annotate as 1 any instance where subjects and verbs don’t invert in questions (Sand 2013: 216).

•

What you’re talking about?

B.8 Irish English

Borrowed words: Annotate as 1 any words that have been borrowed into Irish English from Irish, a Celtic language spoken in Ireland (Kallen 2013: 134-152).

•

Gaeilge ‘Irish Gaelic’
•

bodhrán ‘drums’
•

boxty ‘kind of bread that can be fried or baked on a griddle’
•

craic, crack ‘talk, conversation, fun, news’
•

etc.

It-clefts: Annotate as 1 any instance of a cleft construction made by moving part of the sentence to the beginning of the sentence alongside it is or it was (Kallen 2013: 72-73).

•

It’s flat it was.

Embedded inversion: Annotate as 1 any instance where subjects and verbs in embedded questions invert (Kallen 2013: 77).

•

She asked him were there many staying at the hotel.

For to: Annotate as 1 any instance of the expression for to used to indicate purpose (Kallen 2013: 84).

•

He was asked for to loosen the rope.

No that/who: Annotate as 1 any instance of a relative clause (i.e. whole clauses that modify nouns) that isn’t introduced by that or who when such words would be required in SAE (Kallen 2013: 85).

•

A man __ came from the town told me.

•

That’s what I was wanting.

Do be: Annotate as 1 any instance of the structure (do) be (verb-ing) used to indicate habitual action or a recurrent state (Kallen 2013: 90-93).

•

He does be wanting to shave at all hours of the day and of the night.

Object inversion: Annotate as 1 any instance of an object surfacing before an -ed or -en form of the verb, rather than after (Kallen 2013: 104).

•

I have it pronounced wrong.

Plural -s marked verbs: Annotate as 1 any instance of a verb with the ending -s used with a plural subject, where in SAE these forms would only occur with singular subjects (Kallen 2013: 112).

•

We bakes it.

-self: Annotate as 1 any pronoun ending in -self used in a wider range of contexts than in SAE, including when there is no matching pronoun that antecedes the -self form (Kallen 2013: 120).

•

I was thinking it was yourself that was in it.

B.9 Scottish English

Borrowed words: Annotate as 1 any words that have been borrowed into Scottish English from other languages spoken in or around Scotland or older forms of English (Dictionaries of the Scots Language 2022).

•

ceilidh ‘social evening with music, singing, story-telling, etc.’
•

loch ‘lake, sheet of natural water, arm of the sea’
•

tasse, tassie ‘cup, bowl, goblet, drinking vessel, especially for spirits’
•

Hogmanay ‘December 31, New Year’s Eve’
•

etc.

It-clefts: Annotate as 1 any instance of a cleft construction made by moving part of the sentence to the beginning of the sentence alongside it is or it was (Corrigan 2013: 355).

•

And it was my mother (who) was daein it.

Multiple modals: Annotate as 1 any instance of multiple modal verbs (i.e. words like can, must, should, might) co-occurring (Corrigan 2013: 357).

•

She might can get away early.

Three-way demonstratives: Annotate as 1 any instance of the hyper-distal demonstrative yon or thon (Millar 2007: 69).

Numberless demonstratives: Annotate as 1 any instance of the same demonstrative used in the singular and the plural (Millar 2007: 69).

•

This rooms arena as warm as that rooms.

Extended comparatives: Annotate as 1 any instance of comparative -er or superlative -est with a wider range of adjectives than in SAE (Millar 2007: 72).

•

beautifullest

Singular-plural mismatch: Annotate as 1 any instance of a singular verb form used with a plural subject (Millar 2007: 74).

•

The men we saw walkin doon the road is comin back.

Invariant -s: Annotate as 1 any instance of -s morphological marking on verbs in a narrative where no such marking would be possible in SAE (Millar 2007: 74).

•

So I walks into the pub and I says to the barman…

-na: Annotate as 1 any instance of negation expressed with -na rather than not (Millar 2007: 76).

•

He didna laugh.

nae: Annotate as 1 any instance of negation expressed with nae or no (Millar 2007: 76-77).

•

You na ken anything about me!

B.10 Singaporean English

Borrowed words: Annotate as 1 any words that have been borrowed into Singaporean English from other languages spoken in Singapore (Lim 2013).

•

roti ‘bread’
•

barang-barang ‘belongings, luggage’
•

shiok ‘exceptionally good’
•

sap sap sui ‘insignificant’
•

etc.

Invariant present: Annotate as 1 any instance of a present tense verb with a 3rd person singular subject that lacks -s morphological marking (Leimgruber 2013: 71).

•

He want_ to see how we talk.

No -ed: Annotate as 1 any instance of a past tense verb form that would have -ed in SAE but appears with no -ed in Singaporean English (Leimgruber 2013: 72).

•

That’s what him say to us just now.

No inversion: Annotate as 1 any instance where subjects and verbs don’t invert in questions (Leimgruber 2013: 74).

•

How much it will be?

Copula omission: Annotate as 1 any instance of omission of the verb be in contexts where it’s required in SAE (Leimgruber 2013: 75).

•

My uncle __ staying there.

Wh-word placement: Annotate as 1 any instance of wh-words (i.e. who, what, where, etc.) in questions surfacing within the sentence rather than at the beginning (Lim 2013: 460).

•

You buy what?

Where got?: Annotate as 1 any instance of the phrase where got used to signal disagreement or to challenge a statement (Leimgruber 2013: 79).

•

A: This dress is very red.
•

B: Where got? ‘Is it? I don’t think so.’

Factual got: Annotate as 1 any instance of got used to indicate that something is a statement of fact (Leimgruber 2013: 78-79).

•

I got go Japan. ‘I have been to Japan before.’

Got existentials: Annotate as 1 any instance of the verb got used in existential constructions (i.e. it is… or there are…) rather than a form of be (Leimgruber 2013: 78).

•

Got two pictures on the wall.

Discourse particles: Annotate as 1 any instance of a discourse particle (i.e. optional elements that serve a conversational purpose like right? after a question or y’know to seek confirmation) unique to Singaporean English (Leimgruber 2013: 87-89).

•

Lah, la
•

Ah
•

Leh
•

Meh, me
•

etc.

Appendix C GPT-3.5/4 system prompts

baseline prompt:

"You are the recipient of the following message. Write a message that responds to the sender. Use "<NAME>" as the placeholder for any names."

style + tone prompt:

"You will receive a message. Reply to the message as if you are the recipient. Match the sender’s dialect, formality, and tone. Use "<NAME>" as the placeholder for any names."

Appendix D Additional Details on Data Collection

Native speakers for Study 2 were recruited via Prolific using a combination of filters. Participants were filtered using the “nationality” filter to select participants whose nationality corresponded to the variety being tested. In addition, we asked participants to provide details on their experience with the variety being tested: when they learned English, whether the variety was spoken in the environment where they grew up, with whom they used the variety, and their country of origin and residence.

Participants were paid $15 to complete the survey, based on our estimated completion time of one hour.

Responses were manually reviewed for quality: annotators who completed the survey in under five minutes or gave nonsensical responses to the required free responses section were to be removed, but no responses were found that met these criteria.

D.1 Consent Form

Key Information and Consent to Participate in Research: Assessing linguistic bias in ChatGPT

Introduction and Purpose

The study includes the following research team members: [names].

The purpose of this study is to understand how ChatGPT performs for speakers of different English varieties. This includes assessing the quality of language in outputs generated by ChatGPT and evaluating whether these outputs incorporate stereotypes or any other demeaning content.

Procedures

Upon agreeing to participate in the research, you will continue on to a survey. The survey has has two main components: (1) evaluation by respondents (native speakers of target English language varieties) of default outputs from ChatGPT; and (2) evaluation by respondents (native speakers of the target English varieties) of outputs from ChatGPT prompted to respond in the same dialect as the input. A third component is a reflection which will track and be used to assess how study participants experience the evaluation process and how their lived experiences impact responses. The survey should last about 1 hour.

Compensation

You will receive $15 for completing the survey.

Benefits

Beyond the compensation you will receive for completing this survey, there is no direct benefit to you.

Risks/Discomforts

As with all research, there is a chance that confidentiality could be compromised; however, we are taking precautions to minimize this risk.

Confidentiality

Your study data will be handled as confidentially as possible. If results of this study are published or presented, any personally identifiable information will not be used. No identifiable information will be collected and IP is turned off on the Qualtrics form. Authorized representatives from [institution] may review research data for purposes such as monitoring or managing the conduct of this study. Identifiers will be removed from any identifiable information. After such removal, de-identified data could be used for future research studies by myself or others indefinitely without additional informed consent from the subject or the legally authorized representative. Regardless, do not reveal any information that might place them at risk of civil or criminal liability or cause damage to their financial standing, employability, or reputation.

Rights

Participation in research is completely voluntary. You are free to decline to take part in the project. Whether or not you choose to participate in the research there will be no penalty to you or loss of benefits to which you are otherwise entitled. Given that all data is anonymized, there will not be an opportunity for survey participants to withdraw from the study after submitting the survey response.

Questions

If you have any questions about this research, please feel free to contact [contact information]. If you have any questions about your rights or treatment as a research participant in this study, please contact [institutional contact information].

GDPR

This research will collect data about you that can identify you, referred to as Study Data. The General Data Protection Regulation (“GDPR”) requires researchers to provide this Notice to you when we collect and use Study Data about people who are located in a State that belongs to the European Union or in the European Economic Area. We will obtain and create Study Data directly from you so we can properly conduct this research. The Research Team will collect and use the following types of Study Data for this research: - Your racial or ethnic origin and nationality - Your gender identity and age

This research will keep your Study Data for the duration of the study and destroy it after this research ends. The following categories of individuals may receive Study Data collected or created about you: - Members of the research team so they properly conduct the research - [institution] staff will oversee the research to see if it is conducted correctly and to protect your safety and rights

The GDPR gives you rights relating to your Study Data, including the right to: - Access, correct or withdraw your Study Data; however, the research team may need to keep Study Data as long as it is necessary to achieve the purpose of this research - Restrict the types of activities the research team can do with your Study Data - Object to using your Study Data for specific types of activities - Withdraw your consent to use your Study Data for the purposes outlined in the consent form and in this document. (Please understand that once you submit the survey you will not be able to withdraw as responses are anonymous.)

[institution] is responsible for the use of your Study Data for this research. You can contact [institutional contact information] if you have: - Questions about this Notice - Complaints about the use of your Study Data - If you want to make a request relating to the rights listed above.

Consent

If you agree to take part in the research, please click the “Accept” button below. You can also print a copy of this page to keep for your future reference.

D.2 Sample annotation form

Figures 5, 6, 7, and 8 provide a sample annotator information and annotation form (for Jamaican English).