Peter Andras, John Lazarus, Gilbert Roberts and Steven J Lynden: Uncertainty and Cooperation

©Copyright JASSS

Peter Andras, John Lazarus, Gilbert Roberts and Steven J Lynden (2006)

Uncertainty and Cooperation: Analytical Results and a Simulated Agent Society

Journal of Artificial Societies and Social Simulation vol. 9, no. 1
<https://www.jasss.org/9/1/7.html>

For information about citing this article, click here

Received: 05-Jun-2005 Accepted: 03-Dec-2005 Published: 31-Jan-2006

Abstract

: Uncertainty is an important factor that influences social evolution in natural and artificial environments. Here we distinguish between three aspects of uncertainty. Environmental uncertainty is the variance of resources in the environment, perceived uncertainty is the variance of the resource distribution as perceived by the organism and effective uncertainty is the variance of resources effectively enjoyed by individuals. We show analytically that perceived uncertainty is larger than environmental uncertainty and that effective uncertainty is smaller than perceived uncertainty, when cooperation is present. We use an agent society simulation in a two dimensional world for the generation of simulation data as one realisation of the analytical results. Together with our earlier theoretical work, results here show that cooperation can buffer the detrimental effects of uncertainty on the organism. The proposed conceptualisation of uncertainty can help in understanding its effects on social evolution and in designing artificial social environments.
Keywords:: Agent-Based Modelling, Cooperation, Social Interaction Simulation, Uncertainty

Introduction

1.1

Our aim here is to examine the nature of uncertainty in natural environments and its influence on cooperation in societies of organisms. We distinguish between three types of uncertainty that we term environmental, perceived and effective uncertainty (what we here call environmental and perceived uncertainty might, in a psychological or biological context, alternatively be termed objective and subjective uncertainty respectively). We define environmental uncertainty as the variance of outcomes of events influencing the fitness of the organism. These events can take a variety of forms, such as interactions with resources, competitors or mates, but for simplicity we focus on resource distribution in this paper. We define perceived uncertainty as the variance of the resource distribution as perceived by the organism and effective uncertainty as the variance of the resources effectively enjoyed by the individuals living in the environment. We show here how these aspects of environmental uncertainty are calculated and analyse the relationship between them. We then consider how cooperation between individuals (Fehr and Fischbacher 2003, Fehr and Rockenbach 2003) influences the effective uncertainty of resources gained.

1.2

Elsewhere we have shown theoretically that environmental uncertainty and perceived uncertainty (Andras et al. 2003, Andras and Lazarus 2005) increase cooperation. This theoretical work is supported by findings in human behaviour. Common pool resource groups, such as fisheries, tend to endure where there is greater environmental uncertainty (Ostrom 1990), and sharing in human laboratory experiments increases when gains are more uncertain (Kameda et al. 2002). We are not aware of comparable studies in non-human species. Here we show that cooperation reduces effective uncertainty and thus responds adaptively to increases in environmental and perceived uncertainty by buffering the organism against what otherwise would be a corresponding increase in the variance of resources gained. The advantage to the organism in reducing effective uncertainty is that it reduces the probability of damagingly low resource intake rates. Although it correspondingly reduces the probability of gaining very high intake rates this cost is less than the benefit of avoiding low intake rates since fitness is commonly a diminishing returns function of resource level (Andras and Lazarus 2005).

Figure 1. The determination of the minimum acceptable amount of resources: the horizontal axis shows the amount of resources; the vertical axis shows the expected benefits gained from a given amount of resources and also the corresponding expected cost of the exploration; the curved continuous line is the graph of the expected benefits (a diminishing returns graph); the continuous straight line is the graph of expected costs; the minimum acceptable amount of resources R_m is the resource value for which the cost and benefit curves intersect.

1.3

Following an analytical treatment of uncertainty and cooperation we use an agent society simulation approach (Andras et al. 2003) to examine how our analytical findings are expressed in a particular social scenario. In our simulated world the agents play prisoners' dilemma (Axelrod and Hamilton 1981) type games in order to generate new resources, using their existing resources. We chose the prisoner's dilemma scenario for interactions between agents because this is a commonly used scenario in theoretical work on cooperation (e.g., Axelrod and Hamilton 1981; Milinski et al. 2002; Nowak and Sigmund 1998; Riolo et al. 2001; Roberts and Sherratt 1998) and allows the comparison of our results with other work on similar topics. Environmental uncertainty is represented as the variance of newly generated resources.

1.4

The rest of the paper is structured as follows. In Section 2 we discuss the relationship between environmental and perceived uncertainty and in Section 3 the relationship between perceived and effective uncertainty when individuals cooperate. In Section 4 the agent society simulation is described and conclusions are presented in Section 5.

Environmental and perceived uncertainty

2.1

Without restricting the level of generality, we consider uncertainty in terms of variance of resource distributions. Let us denote by D(R) for R ≥ 0 the probability density function of a distribution of resources in an individual's environment, and let σ² be the variance of this distribution. The environmental uncertainty is σ², the variance of the distribution of resources in the environment. We assume that the variance of the resource distribution is finite. This assumption is valid for a wide range of natural resource distributions (at least within their usual observable range). However, we note that it is possible for a resource distribution to have infinite variance (e.g. stock market) with significant effect on the observable range of the distribution.

2.2

Individuals cannot perceive the full distribution of the resources. The reason for this is that they might be selective (e.g., ignoring too small amounts of resources), or are limited in their ability to exploit all the resources in the environment. We capture this situation by choosing a sufficiently general scenario, that of the selective rejection of low amounts of resources caused by the harshness of the environment (e.g., cold, arid or dangerous environment). In a harsher environment it is more costly for individuals to explore resources, raising the minimum amount of acceptable resource (i.e., the amount at which the benefit gained equals the exploration costs). For example, in the case of rodent foraging individuals shift to more profitable foods under conditions of high predation risk (Hay and Fuller 1981; Bowers 1988).

2.3

Resources are considered acceptable by individuals only if R>R_m, where R_m is the minimum amount of acceptable resource, which is the amount of resources at which the expected cost of exploring the resource is equal to the benefits gained from exploring the resource (see Figure 1). This means that the part of the resource distribution corresponding to resource values R ≤ R_m is ignored by individuals living in this environment, subjectively equating this part of the distribution with a Dirac δ distribution centred in 0 multiplied by the probability

(1)

We assume that the harshness of the environment is not excessive, in the sense that more than half of the full resource distribution is in the acceptable range of individuals, i.e.

(2)

2.4

The perceived uncertainty is the variance of the perceived resource distribution. The perceived resource distribution has the following probability density function:

(3)

2.5

Calculating the variance of the perceived resource distribution we get:

(4)

Where μ_s is the mean of the perceived resource distribution. (We note that replacing the ignored part of the environmental resource distribution with the above specified Dirac δ distribution, means that in terms of mean and variance calculations we do not ignore the segment of the perceived distribution between 0 and R_m, instead for this portion we consider the Dirac δ component of the perceived resource distribution.)

2.6

To compare the perceived and environmental uncertainty we calculate the difference between the variances of perceived and environmental resource distributions. After calculations we get that:

(5)

where μ is the mean of the environmental resource distribution.

2.7

If we have that

(6)

it can be proven that

(7)

We also have that

(8)

2.8

Equations (7) and (8) imply that

(9)

If R∈[0,R_m]. So, combining equations (5) and (9) we deduce that

(10)

showing that the perceived uncertainty is higher than the environmental uncertainty.

2.9

Our analysis shows that we need to differentiate between environmental and perceived uncertainties. The environmental uncertainty is the variance of the whole resource distribution within the environment, while the perceived uncertainty is the variance of the modified resource distribution perceived by the individuals living in the environment. We have shown here that perceived uncertainty is larger than environmental uncertainty if our assumptions are satisfied (i.e. finite environmental uncertainty and more than half of the full resource distribution within the acceptable range). (We note that in the context of everyday life it may happen that perceived uncertainty is lower than environmental uncertainty, e.g. in case of high speed driving, or stock market crashes. The reason for this is that in these cases the environmental uncertainty is infinite, the perceived uncertainty is finite, and the infinite nature of the environmental uncertainty is expressed by relatively frequently observable extreme ranges of relevant distributions.)

Perceived uncertainty, effective uncertainty and cooperation

3.1

Cooperation between individuals essentially means sharing of the uncertainty associated with their individual lives. In the context of the simplified scenario described in the previous section, individuals search for resources and by cooperation they share their finds. By sharing they reduce the variance of the distribution of the resources available for them.

3.2

Putting it more formally, let us suppose that c is the proportion of those individuals that cooperate, and 1-c is the proportion of individuals that cheat. This means that the proportion of those who benefit from cooperation is c², of those who cheat is c(1-c), of those who are cheated is c(1-c), and of those who do not enter in sharing interaction with others is (1-c)². When both agents try to cheat, they do not share their resources and their perceived resource distributions do not change as an effect of the cheat/cheat interaction.

3.3

Those individuals who cheat gain extra resources without sharing and those who are cheated lose resources without gaining from sharing. The simplest way to model these effects on their perceived resource distribution is to consider that the resource distribution for cheaters is shifted to the right, and for those who are cheated is shifted to the left. This means that we get the following formulas for the mean and variance of the effective resource distribution:

(11)

where the subscript 'coop' refers to those agents who benefit from cooperation, 'cheat' refers to agents that cheat, 'suck' refers to agents that are cheated, 'no' refers to agents which do not participate in sharing interactions, and r is the amount by which the resource distribution is shifted for the cheating (to the right, i.e. +r) and cheated agents (to the left, i.e. -r);

(12)

where

(13)

and σ²_coop is the variance of the resource distribution for those who benefit from cooperation, σ²_cheat is the variance of the resource distribution for those who cheat, σ²_suck is the variance of the resource distribution for those who are cheated, and σ²_no is the variance of the resource distribution for those who do not participate in sharing interaction with others.

3.4

In accordance with the above suppositions the effect on the variance of the cheater and cheated resource distributions is equivalent to shifting the cut-off point R_m to the right for those who are cheated, and to the left for the cheaters. Consequently, following the implication of equation (10), the resource variance for cheaters will be lower than the perceived variance defined in the previous section, and the variance for those who are cheated will be larger than the perceived variance. To make things simple let us suppose that

(14)

and

(15)

3.5

In case of individuals who do not enter in sharing interactions their resource variance is exactly the perceived variance. For individuals who participate in cooperation the resource variance is

(16)

3.6

Considering equations (13) — (16) we calculate the effective resource variance for the population of individuals using equation (17).

3.7

Equation (17) shows that the effective uncertainty measured for the whole population is smaller than the perceived uncertainty if there is some level of cooperation within the society of individuals.

(17)

3.8

Equations (17) and (13) also show that if there is no cooperation between individuals (i.e. c = 0) the effective uncertainty is equal to the perceived uncertainty (σ²_e = σ²_s). Cooperation reduces effective uncertainty, buffering individuals against high environmental and perceived uncertainty.

3.9

We have shown in this section that a third important aspect of uncertainty is effective uncertainty, which is measured as the variance of the distribution of resources gained by the whole population of individuals constituting a social group or society. Our analysis shows that the effective uncertainty is lower than the perceived uncertainty if there is some level of cooperation within the population.

Multi-agent simulation

4.1

We built an agent-based simulation (using the programming language Delphi 4 for development) to examine how our analytical results would be expressed in a particular social environment. The code can be downloaded here. Further work of this kind can show how a variety of social and environmental factors influence uncertainty, and how uncertainty impacts on the social world. Our agents own resources (R) that they spend on living costs and use to generate new resources for the future. The agents live in a two-dimensional world with discrete time. Each agent has a position (x,y) and changes location by random movements, i.e. (x_new, y_new) = (x,y) + (u_x,u_y), where u_x,u_y are small random numbers. The agents occupy point-like positions within their world. This implies that agents may occupy positions very close to each other; however it is unlikely that their positions would be identical. The agents move in their world together with their resources. The agents live at most for T_max discrete time units, and they start their life with a randomly set amount T₀ of time steps already spent (i.e. the actual life length of the agent is T_max — T₀).

4.2

The agents have an inclination toward cooperation or cheating, expressed as p the probability of cooperation. If p < 0.5 they are more likely to cheat than to cooperate. They select their behaviour for each interaction in a probabilistic manner biased by their inclination. This is done by choosing a random number q from a uniform distribution over [0,1]; if q < p they cooperate, otherwise they cheat. In each time unit, each agent randomly chooses an interaction partner from its neighbourhood and the partners decide whether to cooperate or cheat. The neighbourhood of an agent consists of a given size set of closest other agents (in our simulation we considered the 10 closest other agents).

4.3

The agents play a Prisoner's Dilemma type game with their chosen partner. The agents enter the game with all of their available resources. The amount of resources that an agent owns after playing the game is given by the actual payoff for the agent from the game. The actual payoffs for the agents are determined by sampling a random variable X_R that has normal distribution N(μ_X,σ_X). The mean value μ_X is determined by the amount of available resources according to the pay-off table shown in Table 1. The values in the pay-off table are such that the table satisfies the conditions of Prisoner's Dilemma games (Axelrod and Hamilton 1981). The variance σ_X represents the environmental uncertainty. In other words, suppose there are two agents A₁, A₂ having resource amounts R^t₁, R^t₂ at moment t. The agents play the game entering with all their available resources. According to the pay-off matrix of the game (see Table 1) the values μ_X1, μ_X2 are determined using the resource amount values R^t₁, R^t₂. For both agents a new resource amount value is picked by taking sample values R^t+1₁, R^t+1₂ from their calculated resource distributions N(μ_X1,σ_X) and N(μ_X2,σ_X). These values will be the resource amounts available for the agents at moment t+1.


Table 1: The pay-off matrix for the cooperation / cheating game. Entries indicate the payoffs to the row player followed by the column player. R₁ and R₂ are the amount of resources of the row and column player respectively, and Δ=[f(R₁+R₂)-f(R₁)-f(R₂)]₊ (i.e., it takes only the positive values of the expression in brackets and it is zero if the value of the expression is negative). The function f is a diminishing return function, and R₁ and R₂ are typically in the range where 2f(x) ≤ f(2x), and 0 < a < 1.

μ_X	Cooperate	Cheat
Cooperate	f(R₁)+Δ/2, f(R₂)+Δ/2	a·f(R₁), f(R₂)+Δ
Cheat	f(R₁)+Δ, a·f(R₂)	f(R₁), f(R₂)

4.4

Environmental harshness is modelled by a cut-off value R_m. Resource amounts below the cut-off value are equated to zero. Varying the value of R_m allows us to investigate how environmental, perceived and effective uncertainty relate to each other in the context of an agent society with agents that may cooperate. We did not explicitly model the perception of the environment by our agents. It is sufficient for our purposes to model their subjective perceptions by choosing the cut-off value R_m.

4.5

The agents produce offspring at the end of their lifetime. The offspring inherit their parent's behavioural inclination with some small random change (i.e., p_offspring = p_parent+u, u∈[-w,w], and w is a small number, e.g., w=0.025). The number of offspring (n) depends on the amount of resources of the agent according to the equation

(18)

where μ is the mean and σ_R is the variance of resources in the current population of agents, and b₁, b₂, n₀ are parameters. The offspring share equally the resources of their parent. The offspring start their life from their parent's last location with minor random changes, implying that the offspring of each agent will be closely packed at the beginning of their life. The cluster of offspring diffuses with time, as the offspring make their random movements. The generation of offspring guarantees the evolutionary change in the population of our agents. Successful agents produce many offspring, while unsuccessful agents produce few or no offspring. The success of the agents (i.e. the amount of resources that they accumulate) depends on their inclination to cooperation / cheating and on the uncertainty of their environment.

Figure 2. The relationship between the environmental and perceived variances for different resource amount cut-off points

4.6

We simulated the evolution of 20 agent populations at four different cut-off levels representing environments with different harshness but with the same environmental uncertainty (i.e. we ran 20 simulations for each level of environmental harshness). Each population started with around 1500 individuals and the simulation ran for 1000 time units, the agents' mean lifetime being 50 time units (T_max = 60, T₀∈[0,20] with uniform distribution). The inclination toward cooperation by the agents was set randomly according to a uniform distribution over [0,1]. We calculated for each simulation, for each time round the environmental, perceived, and effective variances of the generated resources (i.e. environmental, perceived and effective uncertainties). The perceived variance was calculated by considering the amounts of resources that could be generated without cooperation and applying the cut-off at the appropriate set level of R_m. The effective variance of resources was calculated considering the effects of the cooperation games played by the agents and also the cut-off at the level of R_m, i.e. considering the actual payoffs received by the agents after playing their games.

4.7

Based on the earlier theoretical analysis we expect that the perceived variance should be above the environmental variance, and that the effective variance should be below the perceived variance. We are also interested in the nature of the relationship between these aspects of environmental uncertainty.

4.8

Figure 2 presents the relationship between the environmental and perceived variances of the resources for four levels of cut-off resource amounts (R_m = 0, 0.1, 0.2, 0.3). These graphs show that indeed as we expected the perceived uncertainty is larger than the environmental uncertainty if the cut-off level is above zero. The graphs also show that as the environmental harshness increases (i.e. increasing cut-off level) the difference between the perceived and environmental uncertainty increases and this difference grows more rapidly as the environmental uncertainty grows.

4.9

Figure 3 shows the relationship between the perceived and effective variances of the resources. The graphs show that the effective uncertainty stays in a narrow range for the whole range of perceived uncertainty values. The graphs also show that the range of effective variances is well below the values of perceived variances. The fact that the range of effective variances is very narrow indicates that agent societies have the appropriate level of cooperation that allows the stable growth (possibly zero growth rate — in case of close to horizontal population size growth curves) of the society (see Figure 4). Population size decreases as the value of the resource cut-off point increases (see Figure 4) since as this value increases a greater range of resource values are ignored and therefore more opportunities for resource gain are lost.

4.10

The results confirm our expectations and show that indeed it is important to consider the three identified aspects of uncertainty in order to understand the influence of uncertainty on the evolution of social structures, and in particular on the evolution of cooperation. The results also point to the need for a more detailed investigation of the relationship between the level of cooperation and the difference between the perceived and effective uncertainty.

Figure 3. The relationship between the perceived and effective variances for different resource amount cut-off points

Figure 4. The evolution of the average population count (the size of the population) calculated for the four simulation cases with different cut-off points

Conclusions

5.1: We have analysed three aspects of uncertainty in the context of cooperation in communities of selfish individuals and have shown that perceived uncertainty is greater than environmental uncertainty, and that effective uncertainty is lower than perceived uncertainty if there is some level of cooperation within the environment.
5.2: We analysed a series of simulations of a multi-agent world, in which populations of agents evolve. The simulation data demonstrated how the three types of uncertainty are related in a particular social context.
5.3: Our analysis highlights the importance of uncertainty for the evolution of social interactions, and clarifies the key aspects of uncertainty in this context. Our work shows that cooperation reduces effective uncertainty, thus protecting individual organisms from the dangers of high environmental or perceived uncertainty. We believe that experimental biological data analysed in sufficient detail will confirm our predictions about the three aspects of uncertainty based on analytical and simulation results.
5.4: Our work also has implications in the context of designing artificial agent worlds. In this respect our results point to the importance of considering the effects of uncertainty for the development of such agent worlds. In particular, our work highlights the importance of appropriate tuning of perceived uncertainty and of the effects of cooperation on the effective uncertainty, which might contribute significantly to the achievement of the desired mixture of cooperative and non-cooperative behaviour within the artificial agent world.

References

ANDRAS P and Lazarus J (2005) Cooperation, risk and the evolution of teamwork. In: Teamwork: Multi-Professional Perspectives (ed Gold N), pp.56-77. Palgrave Macmillan, New York.

ANDRAS P, Roberts G, and Lazarus J (2003) Environmental risk, cooperation and communication complexity. In Adaptive Agents and Multi-Agent Systems, (eds Alonso E, Kudenko D, and Kazakov D) pp. 49-65. Springer-Verlag, Berlin.

AXELROD R and Hamilton W D (1981) The evolution of cooperation. Science, 211, pp.1390-1396.

BOWERS M A (1988) Seed removal experiments on desert rodents: the microhabitat by moonlight effect. Journal of Mammalogy, 69, pp.201-204.

FEHR E and Fischbacher U (2003) The nature of human altruism. Nature, 425, pp.785-791.

FEHR E and Rockenbach B (2003) Detrimental effects of sanctions on human altruism. Nature, 422, pp.137-140.

HAY M E and Fuller P J (1981) Seed escape from heteromyid rodents: the importance of microhabitat and seed preference. Ecology, 62, pp.1395-1399.

KAMEDA, T Takezawa, M, Tindale, R and Smith, C (2002) Social sharing and risk reduction: Exploring a computational algorithm for the psychology of windfall gains. Evolution and Human Behavior, 23, pp.11-33.

MILINSKI M, Semmann D, and Krambeck H J (2002) Reputation helps solve the 'tragedy of the commons'. Nature, 415, pp.424-426.

NOWAK M A and Sigmund K (1998) Evolution of indirect reciprocity by image scoring. Nature, 393, pp.573-577.

OSTROM, E (1990) Governing the Commons: The Evolution of Institutions for Collective Action. Cambridge: Cambridge University Press.

RIOLO R, Cohen M D, and Axelrod R (2001) Evolution of cooperation without reciprocity. Nature, 414, pp.441-443.

ROBERTS G and Sherratt T N (1998) Development of cooperative relationships through increasing investment. Nature, 394, pp.175-179.

Button Return to Contents of this issue