Decision-making under uncertainty: biases and Bayesians

It is not certain that everything is uncertain.

Blaise Pascal

Abstract

Animals (including humans) often face circumstances in which the best choice of action is not certain. Environmental cues may be ambiguous, and choices may be risky. This paper reviews the theoretical side of decision-making under uncertainty, particularly with regard to unknown risk (ambiguity). We use simple models to show that, irrespective of pay-offs, whether it is optimal to bias probability estimates depends upon how those estimates have been generated. In particular, if estimates have been calculated in a Bayesian framework with a sensible prior, it is best to use unbiased estimates. We review the extent of evidence for and against viewing animals (including humans) as Bayesian decision-makers. We pay particular attention to the Ellsberg Paradox, a classic result from experimental economics, in which human subjects appear to deviate from optimal decision-making by demonstrating an apparent aversion to ambiguity in a choice between two options with equal expected rewards. The paradox initially seems to be an example where decision-making estimates are biased relative to the Bayesian optimum. We discuss the extent to which the Bayesian paradigm might be applied to the evolution of decision-makers and how the Ellsberg Paradox may, with a deeper understanding, be resolved.

Appendices

Appendix 1: optimal bias for a frequentist estimate

If p _a is estimated after having witnessed k successes from n previous trials, the unbiased frequentist estimate of the probability of success is k/n. If the estimated probability of success, p _est, is used as if it is the true probability of success, then the expected pay-off for true p _a will be given by

$$ \sum\limits_{{k:p_{\text{est}} > p_{r} }} {P(k|n)(p_{a} b - (1 - p_{a} )c)} $$

where

$$ P(k|n) = {\frac{n!}{k!(n - k)!}}\,p_{a}^{k} (1 - p_{a} )^{n - k} $$

is the probability of k successes from n trials.

We introduce a bias, s, as the fraction of the distance from k/n upward (downward) towards 1 (0) for positive (negative) values of bias. By regarding p _a as having been chosen from a uniform distribution over the interval (0,1), we can then calculate the minimal optimal bias for a range of n and b/c, as shown in Fig. 2. By minimal optimal bias, we mean the bias closest to zero for which the expected return is maximised; the reason for the qualification is that there is typically a range of values of bias for which the expected return is maximised.

Fig. 2

Optimal bias as a function of number of trials and benefit/cost

The ridges in the graph are a result of the discrete nature of the number of successes and the number of trials. As the number of trials increases and the resulting estimate improves, the optimal bias reduces towards zero, together with the step changes becoming smaller.

Appendix 2: use of the beta distribution

A beta distribution is a probability distribution on the interval [0,1], the pdf of which is defined by the two shape parameters (hyperparameters), α > 0 and β > 0, according to:

$$ {\text{Beta}}(\alpha ,\beta ) = f(x;\alpha ,\beta ) = {\frac{{x^{\alpha - 1} (1 - x)^{\beta - 1} }}{B(\alpha ,\beta )}} $$

where B (α, β) is the beta function, in this case serving as a normalisation constant.

For α = β = 1, the beta distribution is no longer a function of x, so provides the uniform distribution.

Due to the Bernoulli nature of the trials, the posterior distribution following an update is also a beta distribution, as are all subsequent updates (Carlin & Louis, 1996). To see that this is so, let $f(k|p) = p^{k} (1 - p)^{n - k}$ denote the likelihood of witnessing k successes from n trials when the probability of success is p. Then, if the prior $\pi (p)$ is a beta distribution, we have:

$$ \begin{aligned} P(p|k) & \propto f(k|p)\pi (p) \\ & \propto p^{k + \alpha - 1} (1 - p)^{n - k + \beta - 1} \\ & \propto {\text{Beta}}(\alpha^{\prime } ,\beta^{\prime } ) \\ \end{aligned} $$

where $\alpha^{\prime } = \alpha + k$ and $\beta^{\prime } = \beta + n - k$.

The mean of the posterior distribution, $\alpha^{\prime } /(\alpha^{\prime } + \beta^{\prime } )$, then provides the expected value of p _a as $E(p_{a} ) = {\frac{k + \alpha }{n + \alpha + \beta }}$.

Appendix 3: comparing the mean of the posterior with the critical probability

The expected probability of success at any stage is given by

$$ P({\text{success}}) = \int\limits_{0}^{1} {f(p)p{\text{d}}p = E(p_{a} )} $$

Using this expected value, it is best not to apply any bias before comparing with p _r .

Let us assume that, having updated the uniform prior with the trial data and obtained a posterior beta distribution with mean p _m , the individual finds that p _m  = p _r . Will it matter whether the individual chooses to act or not?

If the individual chooses to act, then integrating the expected pay-off across the distribution of possible p values, where E (p) = p _m  = p _r , the expected value of acting is:

$$ \int\limits_{0}^{1} {f(p)(pb - (1 - p)c){\text{d}}p = E(p)b - (1 - E(p)c) = p_{r} b - (1 - p_{r} )c = 0} $$

which is the same expected value as not acting. Therefore, if the posterior estimate was greater than the critical probability (i.e. p _m  > p _r ), it would be best to act and vice versa with p _m  < p _r . The mean of the posterior is therefore a sufficient statistic to compare with p _r when deciding whether to act.

Thus, it is best not to apply any bias before comparing with p _r .