The Dynamics of Social Conventions in LLM populations:
Spontaneous Emergence, Collective Biases and Tipping Points

Our approach builds on Wittgenstein’s general model of linguistic conventions, where repeated interactions lead to collective agreement between two players [30]. Theoretical extensions of this approach have argued that purely local interactions taking place on social networks can lead to population-wide, or ‘global’, coordinated behavior [1, 2, 31, 6]. Theoretical predictions for our study are based on the naming game model of convention formation, where agents, aiming to coordinate in pairwise interactions, accumulate a memory of past plays, which they then use to "guess" the words their subsequent partners will use [7, 8]. Extensive numerical and analytical studies have shown how the model captures the rapid growth of universally shared social conventions in different settings [6]. Derived laboratory experiments involving human participants in naming games have provided the first empirical evidence for the spontaneous emergence of shared linguistic conventions [9].

The naming game framework has also been applied to study norm change and critical mass theory, which posits that committed minorities can overturn stable social conventions once their size reaches a tipping point, or ‘critical mass’. Theoretical models suggest critical masses between 10% and 40% of the population [21, 32]. Empirical evidence from controlled social coordination experiments, which closely follow the scheme described above, supports a 25% threshold [22]. However, real-world observations reveal a wider range, with some studies proposing 30-40% for gender conventions in corporate leadership [24, 33], and others indicating that minorities as small as 0.3% can trigger significant linguistic and social changes [34, 35, 36, 28].

A simulation ‘trial’ consists of a population of $N$ interacting agents. At each time-step, two agents are randomly selected for interaction. Both agents select a convention, or ‘name’, from a pool of a finite size $W$, and attempt to blindly coordinate with one another. If they manage to coordinate, they are rewarded with an increase in their game score, otherwise they are penalized. Agents are not informed that their co-player is sampled from a population and are not incentivized to reach a ‘global’ consensus but only to coordinate in a pairwise manner with their partner on each round. Importantly, agents are able to remember details about the past $M$ interactions they participated in, including their co-player’s convention choice, their own convention, whether the interaction was successful or not, and their own accumulated score over these $M$ interactions. The agents’ memory is initially empty, so that at their first interaction they produce a random convention chosen from the pool of available names. After each interaction, agents see the conventions they and their co-player have chosen, and their cumulative score is updated accordingly. Finally, in the experiments on norm change and critical mass theory, we introduce a small number of adversarial agents (i.e., a ‘committed minority’) into each population, who attempt to overturn the established convention by consistently promoting a novel alternative [21, 22].

These dynamics reflect common types of online interactions where community members engage directly with a large, often anonymous population – using chat interfaces or messaging technologies – leading to the adoption of linguistic and behavioral conventions that enable effective coordination with other participants’ expectations [9, 37, 38, 22], and here are implemented with four different LLM models: Llama 2 70b, Llama 3 70B, Llama 3.1 70B, and Claude 3.5 Sonnet (see Methods).

Interactions within the game take place in the form of a series of text-based moves. In each interaction, the LLM agent is given a text prompt comprised of a system prompt and a user input prompt. The system prompt contains all information about the game. The user input requests the agent to predict a player’s next action based on the history of choices in the $M$ most recent interactions. This positions the agent as an external observer of the game, tasked with forecasting the upcoming round. In practice, these decisions dictate the state of play. Agents receive no information about the players’ identities or personalities, such as whether they are rational actors. Consequently, we can interpret the agent’s recommendations as their de-facto participation in the game. The system prompt (see Materials and Methods) is designed such that the agent’s output follows a consistent format, from which we can extract its decision. Following previous works on LLMs’ cognitive abilities [39], we ask the agent to ‘think step by step’ and to explicitly consider the history of play. The prompt thus encourages agents to make a decision based on their previous experience, but provides no instruction as to how it should be used in the decision making process. Agents are asked to select a name from the name pool, which is presented to them as a list of $W$ unique letters sampled from the English alphabet. Ordering bias is removed by randomizing the list of presented letters for each player at every interaction. A successful interaction garners equal rewards for the participating agents, whereas a failure to coordinate results in a penalty. In the absence of human guidance, LLMs are notoriously bad at arithmetic. To avoid decision errors based on a misjudgment of the game state, we explicitly provide the agent with both the payoff they obtained at each round and their cumulative score within memory range. Lastly, to ensure that the responses generated by the LLM are correctly guided by the prompt and not merely the result of random hallucinations [40], we have implemented a meta-prompting strategy to assess the LLM’s understanding of the given instructions. This practice, previously used in evaluating LLMs within game-theoretical frameworks [41], consists of posing a series of text comprehension queries to the LLM and evaluating the precision of its responses. The LLMs subjected to our testing demonstrated good comprehension capabilities, as detailed in SI Section 6.4.

Fig. 1 shows that group-wide linguistic conventions spontaneously emerge across all models. Initial interactions have a low probability of being successful, but stochastic fluctuations break the initial symmetry between the conventions, and eventually one becomes dominant. The inset of Fig. 1 shows that the theoretical model (see SI Section 6.2 for a description) captures the dynamics generated by the LLM populations. The curves in Figure 1 concern a population size of $N=24$ agents, but convergence is also observed for larger systems ($N=200$, see Fig. SI5) and larger name pools ($W=26$, see Fig. SI1). A population round is defined as $N$ microscopic interactions, a common approach in multi-agent simulations [42].

Having established that social conventions emerge, a natural question arises: what are these conventions? The single Latin alphabet letters available in the name pool are all equally valid as global conventions, and so we would expect them to all to have the same probability to become the accepted social convention, as supported by the theoretical model [8] (see also SI Section 6.2). However, the experimental results present a different picture (Fig. 2A). The probability that a particular name becomes the social convention is neither uniform nor deterministic. Some names appear to have a significantly higher likelihood of becoming the adopted convention than others. This pattern holds across models, although the preferred names vary between models.

Two hypotheses could explain the observed behavior. The selection process may be non-uniform due to (i) intrinsic model (i.e., individual single-agent) biases or (ii) prompt features, specifically the order in which names in the name pool are presented to the agents, as noted in a different context [43]. The latter hypothesis can be discarded since, as mentioned above, the names are presented to the agents in a list in randomized order for each agent and at every interaction.

Having ruled out the order of name presentation as a factor, we can focus on the role of individual (i.e. single-agent) biases in shaping collective behaviour. The hypothesis that individual bias can be responsible for a collective bias is supported by the theoretical model. When the model is run with only two names, a bias towards a particular name quickly results in unilateral convergence on that name at the population level (see Fig. SI3). To test this intuition in our experiment, we examine the selection preferences of individual agents during their first round, when they have no prior memory. We find that individual biases are indeed possible. For example, when agents can choose any letter from the complete English alphabet, the population systematically converges on the letter ‘A’ because individual agents overwhelmingly prefer to select it over all other letters, even without prior memory (see Fig. SI1). However, a similar test on the frequency of name selection by agents with no prior memory for the case of Fig. 1, where the name pool contains ten elements but not the letter ’A’, yields mixed results. Under these conditions, individual Llama 2 70b and Claude 3.5 Sonnet agents are unbiased across conventions in this name pool ($\chi^{2}$-test, $P$ = 0.100, 0.410), whereas individual Llama 3/3.1 agents exhibit a significant statistical skew in their name selections (see Fig. SI2). In all cases, the final consensus distribution shows that specific names are favoured as a consensus option, even if they appeared to be less likely to be selected in the initial step (Fig. 2A). Thus, both social conventions and collective biases in the selection process emerge also in absence of individual biases.

The findings suggest that collective bias may stem from the convention formation process itself, as agents are exposed to diverse memory states with different name combinations and success-failure sequences. To test this hypothesis, we focus on the case of a name pool size $W=2$, since tracking potential confounders of bias becomes impractical as the space of possible names increases. Fig. 2B shows shows that across all models, although agents are initially unbiased, local communication and coordination lead to a collective bias toward a specific convention, which we term the ‘strong convention’ (as opposed to its ‘weak’ counterpart).

The top row of Table 1 shows a case where there is no individual bias towards a particular name in the first interaction ($P$=0.116 > 0.05, indicating that the evidence is not strong enough to reject the hypothesis that the agent is unbiased). In the second interaction, agents have some memory influencing their decision, but the observed outcome probability remains symmetric ($P$=0.110). We observe that if an agent succeeds in the first interaction, it will almost surely continue to use the successful name in the next interaction ($99.4\%$ of the time in the data in Table  1, with similar results in real simulations and for other models). However, if an agent fails, it will almost surely switch names ($97.3\%$ of the time). In all tested cases with $W=2$, and across all models, an asymmetric selection bias emerges by the agent’s third interaction, distinguishing between the ‘weak’ and a ‘strong’ conventions. Across all observed memory states (noting that some configurations are extremely rare or do not occur due to the agent’s decision-making), agents are more likely to produce the strong name in 5 out of 8 memory configurations. These configurations account for the vast majority of observed states ($99.2\%$ of the possible states resulting from interaction 2 in Table 1, with similar patterns observed in other models). In subsequent interactions, agents are more likely to encounter the strong name associated with success, reinforcing its use and ultimately leading to consensus on that name as the social convention.

In summary, our results suggest that a collective bias emerges from repeated interactions among agents who, when tested in isolation (i.e., in interaction 1), appear to be unbiased in their decision making. This collective preference breaks the initial symmetry among the different alternatives that could become the social convention, favouring some over others (or, in the case of $W=2$, one over the other). It is important to emphasise that this dynamically emerging bias is not required for the spontaneous emergence of a convention. The collective and individual biases of these agents drive the consensus towards particular conventions. For reference, the theoretical model produces conventions without any individual bias, but accommodates it at the individual level to explain the dominance of specific conventions over competing alternatives [8, 6] (see Fig. SI3). In LLMs, on the contrary, we observe that bias emerges when agents develop diverse memory states, which form through a collective process of agent-to-agent communication.

Social conventions are steady states of the system: once a convention spontaneously emerges, the population adheres to it indefinitely (see Fig. SI6). A natural question concerns the stability of such steady states: how resistant is a convention to deliberate efforts to overturn it? To address this question, we investigate whether a committed minority can ‘flip’ an equilibrium consensus on a convention. We consider the scenario in which a population has long converged on a convention and every agent has solely observed that convention in the past $M$ interactions (which were, therefore, all successful). We then introduce a ‘committed minority’ of agents producing an alternative convention [21, 22]. These committed agents follow a fixed strategy and use the alternative convention at all times. We test populations using the same two-name ($W=2$) conditions as in our convergence experiments. We simulate a consensus on each name per combination and introduce its complementary name as an adversary.

Figure 3 shows that when the committed minority reaches the critical threshold, the whole population adopts their convention. Below this threshold, the population settles into a mixed state, as committed agents always use the minority convention. Interestingly, the critical mass of the committed minority needed to trigger a new convention depends on the convention itself. The stronger name (i.e., the name more likely to become the social convention had we started with no prior memory, as seen in the previous section) requires a larger committed minority to be overturned. Conversely, a smaller number of adversarial agents can overturn a consensus on the weaker name.

Interestingly, the relative strength of the two conventions can vary so widely depending on the LLM that committed groups as small as 2% (Llama 3 70B) or as large as 67% (Llama 2 70b) - effectively no longer a minority - were observed (see Fig. 3. In Llama 3.1 70B populations, the bias is so strongly weighted against the weaker convention that the population spontaneously switches to the alternative, stronger convention without requiring any committed agents at all.

The system prompt comprises of three components: i) a fixed prompt that outlines the game’s rules, including the payoff structure and the player’s objective, ii) a dynamic memory prompt that provides contextual information about the state of play within the player’s memory range, and iii) an instructional prompt that provides information for how the agent should format its response. The user prompt asks the agent to select a name to use in the current interaction. We use zero-shot prompting to directly extract the agent’s name decision in response to the state of play. We do not provide instructions as to how agents should decide their next move, nor do we present them with example strategies. We ask the agent to behave in a self-interested manner, and the only part of the prompt in which we suggest to the agent that it should consider partaking in coordination is when we state that the agent’s objective is to ‘maximise their own accumulated point tally, conditional on the behaviour of their co-player’. We apply fixed payoffs for successful and failed interactions, set at +100 and -50 points respectively.

For our experiments, we use homogeneous populations of the following LLM agents: Llama 3 70B ¹¹1 All Llama family models are open-sourced LLMs, released under a commercial use license (https://ai.meta.com/llama/license/).²²2 We use versions of the Llama 3 family models hosted by Hugging Face, which we access through the Inference API (https://huggingface.co/inference-api/serverless). We quantise Llama 2 70b into a 4-bit version using Hugging Face’s Transformers library, and run the model locally (https://huggingface.co/docs/transformers)., Llama-3.1 70B, Llama 2 70b (in 4-bit quantisation format), and Claude Sonnet 3.5 (see Table  2 for specific versions). In these autoregressive LLMs, each newly generated word is produced based on previously inputted and generated words, and so the sequence of generation matters. More precisely, the probability distribution for predicting the next word is conditional on the product of all previous word probability distribution. To mimic LLMs deployed in real-world application, we demand all agents in our experiments to behave non-deterministically by fixing them with a non-zero constant temperature. This means that for each agent the next generated word is randomly selected from the conditional probability distribution. We use K-sampling to restrict the probability distribution of the next word to the next $K$ most likely words, thus increasing the likelihood of high probability words and decreasing the likelihood of low probability words which are outside of the name pool (see Table  SI3 for all parameter values).

We quantify the individual bias of agents by measuring the number of times each convention was produced in the first round of the game, when their memory inventory is empty, over $T$ trials. Experiments with $W=2$ are effectively a Bernoulli trial, and so we measure whether the agent is biased by performing a two-tailed exact Binomial test with the observed proportions. We calculate the p-value, $P$, using the null probability $p=0.5$, and reject the hypothesis that the model is biased if $P$<0.05. For the case of $W=10$, we perform a $\chi^{2}$-test, and also test the null hypothesis that the model is neutral in its convention selection. Thus, we use the expected value $0.1T$ in our calculations, and again reject the null hypothesis that the model is unbiased if $P$<0.05.

To determine the critical size of the committed minority, we identify the point at which the majority consensus is overturned. A consensus flip occurs when 95% of the past $3N$ interactions succeed after the introduction of the committed minority. For Llama 3, we tested the smallest minority needed to overturn a weak convention majority, then repeated the experiment with a strong convention majority to measure the critical mass within the same time frame. For other models, the critical mass threshold is defined as the minimum proportion of committed agents that is required to flip the consensus within 30 population rounds. These criteria account for potential fluctuations in non-deterministic agent decisions.

The Naming Game model simulates a population of $N$ agents engaging in pairwise negotiation interactions, demonstrating the emergence of global consensus on conventions through local coordination mechanisms. In the canonical formulation [8], agents must reach consensus on the name for an object using only local interactions, similar to our experimental framework. Agents possess internal lexicons with unlimited word capacity (although this is not a necessary initial condition of the model), initially empty. The interaction protocol involves random selection of agent pairs, where the designated speaker transmits a randomly chosen word from their lexicon (or invents a new one if it is empty) to the hearer. If the hearer recognises the word in their own lexicon, both agents retain only the communicated word, while in case of failure, the hearer incorporates the novel word into their lexicon. The non-equilibrium dynamics of this system exhibit three distinct temporal phases: (i) an innovation phase characterised by word creation, (ii) a propagation phase involving lexicon reorganization, and (iii) a convergence phase culminating in global consensus. In our experimental framework, we set an initial condition whereby agents can only invent new words from a finite word pool of size $W<N$. This condition means that the initial innovation phase is extremely short, as seen in the inset of Fig. 1. This model provides insights into the dynamics of language evolution and convention formation in both human and artificial communication systems.

Fig. SI3 shows the production probability trajectories of a simulation of the theoretical model with a lexicon of two words ($W=2$).