Extreme Value Theory: A primer

Harald E. Rieder
Lamont-Doherty Earth Observatory
9/8/2014

Based on :
Coles (2001) An Introduction to Statistical Modelling of Extreme Values, Springer
Davison (2005): Extreme Values, Encyclopedia of Biostatistics, Wiley.
Katz (2013): Chapter 2 - Statistical Methods for Nonstationary Extremes, in Extremes
in a Changing Climate, Detection, Analysis and Uncertainty, Springer
Cooley (2013): Chapter 4 - Return Periods and Return Levels under Climate Change, in
Extremes in a Changing Climate, Detection, Analysis and Uncertainty, Springer

1
Introduction

IPCC, AR5, WGI, Chapter 01

2
Introduction

IPCC, AR5, WGI, Chapter 01

3
 Introduction
Normal Distribution (or Gaussian, or ‘bell curve’)
is a continuous probability distribution given by

𝑥−𝜇 2
1 −
𝐹 𝑥, 𝜇, 𝜎 = 𝑒 2𝜎2
𝜎 2𝜋

where the parameter μ is the mean of the distribution (and also its median
and mode) and the parameter σ is the standard deviation.

Source: introcs.cs.princeton.edu

4
 Introduction
Statistical extreme value theory is a field of statistics dealing with extreme
values, i.e., large deviations from the median of probability distributions. The
theory assesses the type of probability distribution generated by processes.

Extreme value distributions are the limiting distributions for the minimum
or the maximum of large collections of independent random variables from
the same arbitrary distribution. By definition extreme value theory focuses on
limiting distributions (which are distinct from the normal distribution).

Two approaches exist for practical extreme value applications. The first
method relies on deriving block maxima (minima) series, the second method
relies on extracting peak values above (below) a certain threshold from a
continuous record.

A third approach the so-called r-largest order statistics represents a

compromise between the block maxima and peak over threshold approach.

5
 Data Sets
Let’s look on some examples with real world data:
(1) maximum daily 8-hour surface ozone from CastNet Site PSU106
(2) daily maximum temperature from NYC Central Park Belvedere Tower

6
 Data Sets
For simplicity we focus on summer time (JJA) data only
and we consider extreme values as:
(1) mda8 O3 > 75 ppb (NAAQS)
1988-2000 vs 2001-2013: shift in mean -7.8 ppb; change in variance -3.6 ppb

(2) Tmax ≥ 25 degree C (summer day)

1876-1944 vs 1945-2013: shift in mean +0.86 C; change in variance -0.025 C

We want to visualize how these changes in mean or variance or both affect

the distributions and in particular the probabilistic frequency of extremes.

7
Influence of shift in mean and/or change in variance

8
Influence of shift in mean and/or change in variance

9
Comparison of observed distributions with least-square fitted normal distributions
Compare observed distributions with Normal distributions
Quantile Observed Gaussian
0.10 38 ppb 36 ppb
0.75 70 ppb 71 ppb
0.9 82 ppb 81 ppb
0.95 92 ppb 88 ppb
0.99 106 ppb 100 ppb

Quantile Observed Gaussian

0.10 22.2 C 22.3 C
0.75 30.0 C 30.1 C
0.9 32.2 C 32.5 C
0.95 33.9 C 34.0 C
0.99 36.1 C 36.7 C

10
Testing for normality

The Shapiro-Wilk test is a standard test to check whether a sample X1,...,Xk

stems from a normally distributed population.

( 𝑘
𝑖=1 𝑎𝑖 𝑥(𝑖) )
2
The test statistic (𝑊) is 𝑊 = 𝑘 2
𝑖=1(𝑥𝑖 −𝑥 )

where 𝑥𝑖 is the ith-smallest number in the sample, 𝑥 = (𝑥𝑖 + ⋯ + 𝑥𝑘 )/𝑘 is

𝑚𝑇 𝑉 −1
the sample mean, and 𝑎𝑖 are constants 𝑎1 , … , 𝑎𝑘 = , where
(𝑚𝑇 𝑉 −1 𝑉 −1 𝑚)1/2
𝑚 = 𝑚1 , … , 𝑚𝑘 𝑇 and 𝑚1 , … , 𝑚𝑘 are the iid expected values of the order
statistics from the normal distribution and V represents the covariance matrix
of these order statistics.

The null hypothesis, the population is normally distributed, is rejected when

the p-value of the test is below a defined significance value (e.g., 0.05)

11
 Statistical Extreme Value Theory
Extreme value theory (EVT) is concerned with the occurrence and sizes of
rare events, be they larger or smaller than usual.

Here we want to review briefly the most common EVT approaches and
models and look into some applications.

There has been rapid development over the last

decades in both theory and applications. A
comprehensive introduction to statistics of
extremes is provided by Coles (2001).

12
 Statistical Extreme Value Theory
Further Reading

13
 Statistical Extreme Value Theory
Frequently discussion of extremes concerns high extremes, maxima. Also we
will focus our initial discussion on maxima. Though it shall be noted that
dealing with minima follows the same approaches and in applications all
needed to be done is reverse the sings of the observations and apply
procedures for maxima as

min 𝑥𝑖 = −max⁡(−𝑥𝑖 )

14
 Generalized Extreme Value Distribution
Block Maxima

The Extremal Types Theorem (ETT) (e.g. Leadbetter et al., 1983) addresses
the following question: Given a set of independent identically distributed
random variables X1, ...,Xk, what are the possible limiting distributions of

𝑀𝑘 = 𝑎𝑘 max 𝑋1 , … , 𝑋𝑘 − 𝑏𝑘 ⁡𝑎𝑠⁡𝑘 → ∞⁡?

𝑘
𝑥 − 𝑏𝑘
𝐹 𝐺(𝑥)
𝑎𝑘 𝑘→∞
The answer is that if a nondegenerate limiting cumulative distribution (cdf)
exists for some sequences of constants ak and bk, it must fall into one of the
three classes

15
 Generalized Extreme Value Distribution

The three types of distributions represent the Gumbel, Frechet and Weibull
distributions. The ETT guarantees that if a limit exists for maxima, it must
have one of these specified forms.

16
 Generalized Extreme Value Distribution
In a more modern approach these distributions are combined into the
generalized extreme value distribution (GEV) with cdf
𝑦−𝜇 −1/𝜉
𝐻 𝑦 = exp − 1 + 𝜉 , −∞ < 𝜇, 𝜉 < ∞, 𝜎 > 0,
𝜎

define for values of 𝑦 for which 1+ 𝜉(𝑦⁡- 𝜇⁡)/𝜎 > 0.

where 𝜇 is the location parameter, 𝜉 is the shape parameter, and

𝜎 > 0 is the scale parameter.

The shape parameter ξ governs the distribution type:

type I with ξ = 0 (Gumbel,light tailed)
type II with ξ > 0 (Frechet, heavy tailed)
type III with ξ < 0 (Weibull, bounded)

17
 Generalized Extreme Value Distribution
GEV type I with ξ = 0 (Gumbel, light tailed)
Domain of attraction for many common distributions (e.g., normal,
exponential, gamma), not frequently found to fit ‘real world data’

GEV type II with ξ > 0 (Frechet, heavy tailed)

Fits found for precipitation, stream flow, economic damage

GEV type III with ξ < 0 (Weibull, bounded)

Fits found for temperature, wind speed, pollutants, sea level

18
 Block Maxima - Application
It is important to note that the location parameter 𝜇 is not the mean but
does represent the ‘center’ of the distribution, and the scale parameter 𝜎⁡is
not the standard deviation but does govern the size of the deviations about
𝜇.

A typical application would be to fit a GEV to the annual maximum of a

variable. Note that the block size for maxima is freely variable though
applications must be consistent with the maxima of a given block.

INSERT GRAPH of ANNUAL MAX Tmax Central Park

19
 Block Maxima - Application
Fit GEV distribution to annual MAX of summertime (JJA) temperature
𝑦−𝜇 −1/𝜉
𝐻 𝑦 = exp − 1 + 𝜉 , −∞ < 𝜇, 𝜉 < ∞, 𝜎 > 0,
𝜎

𝜇 = 35.28 (± 0.16)
𝜎⁡= 1.74 (± 0.12)
𝜉⁡= -0.19 (± 0.06)

GEV distribution type III with ξ < 0 (Weibull, bounded)

20
Block Maxima - Application

FIT
OBS

21
 Return Levels
The fitted distribution than can be used to estimate the 𝒎-year return level,
which represents the high quantile for which the probability that the annual
maximum exceeds this quantile is 1/𝑚.
Under the assumption of stationarity the return level is the same for all years,
giving rise to the notion of the return period. The return period of a
particular event is the inverse of the probability that the event will be
exceeded in any given year, i.e. the 𝑚-year return level is associated with a
return period of 𝑚 years.

22
 Return Levels

𝑦−𝜇 −1/𝜉
GEV 𝑦 = exp − 1 + 𝜉
𝜎

Computing the return level 𝑟𝑚 such that

GEV 𝑟𝑚 = 1 − 𝑚

𝑟𝑚 = GEV −1 (1 − 𝑚)
𝜎 −𝜉
Hence, 𝑟𝑚 = 𝜇 + − ln 1 − 𝑚 −1
𝜉

23
 Return Levels
𝜎 −𝜉
𝑟𝑚 = 𝜇 + − ln 1 − 𝑚 −1
𝜉

Estimating the return level 𝑟𝑚

𝜎 −𝜉
𝑟𝑚 = 𝜇 + − ln 1 − 𝑚 −1
𝜉

Where GEV parameter estimates 𝜇 , 𝜎, 𝜉 , are derived via

-Maximum likelihood estimation (MLE) (inbuilt in gev packages in R, MatLab)

-Methods of moments (PWM, GPWM)
-Exhaustive tail-index approaches

24
 Return Levels
In the stationary case there is a one to one relationship between the 𝒎-
year return level and the 𝒎-year return period (reciprocal of exceedance in
any given year).

A 10-year event might be more interpretable by lay audiences as a 0.1

probability of occurrence in any given year. Though this leads to two possible
interpretations of the ‘𝑚-year event’.

(1) The expected number of events in 𝑚 years is 1.

(2) The expected waiting time till the next exceedance is 𝑚 years.

Under the assumption of stationarity both of this interpretations are correct.

We will discuss return level interpretations under non-stationarity a bit later

in this class.

25
 Extending beyond block maxima
One argument against the application of a block-maximum approach is that
use of maxima alone is wasteful of data: most of the information in the
sample is ignored.

To overcome this limitation several other approaches have been developed

based on the point process characterization.

Here we want to review 2 common applications:

(1) the r-largest order model for extremes
(2) peak over threshold modeling

26
 r-largest order statistics
r-largest order extremes
The r-largest observations among Y1,...,Yk, will contain more information
about the extremes than the maximum alone.

Let 𝑦𝑡 (1) ≥ ⁡ … 𝑦𝑡 (𝑟) be the r-largest observation in a block, or equivalently

time period, 𝑡 ∈ 1,2, … 𝑇 and define the location 𝜇, shape 𝜎 and scale
𝜉parameters as in the GEV then the likelihood 𝐿𝑡 for block 𝑡 based on the r-
largest order statistics model is

This expression is related to the GEV distribution whose probability density

function is obtained on setting 𝑟⁡=1.

27
 r-largest order statistics application
Block maximum 3-largest observations per summer
𝜇 = 35.28 (± 0.16) 𝜇 = 34.69 (± 0.11)
𝜎⁡= 1.74 (± 0.12) 𝜎⁡= 1.67 (± 0.08)
𝜉⁡= -0.19 (± 0.06) 𝜉⁡= -0.28 (± 0.05)

10yr return level: 38.5 10yr return level: 37.9

25yr return level: 39.4 25yr return level: 38.8
50yr return level: 40.0 50yr return level: 39.4
100yr return level: 40.6 100yr return level: 39.8

28
 Peak over threshold
The peak over threshold approach is based on the idea of modelling data
over a high enough threshold.

The cdf of the amount by which an observation exceeds a high threshold 𝑢,

given that it has done so, is
1
𝑦−𝑢 −
𝜉
𝑝𝑟 𝑋 > 𝑢 + 𝑦⁡|⁡𝑋 > 𝑢 = 𝐺 𝑦 = 1 − 1 + 𝜉 ,⁡
𝜎
𝑦−𝑢
𝜎 > 0, 𝑦 > 𝑢, 1+𝜉 > 0;⁡
𝜎

which is called the Generalized Pareto distribution.

The shape parameter 𝜉⁡has the same meaning as in the GEV type with
type I with ξ = 0 (light tailed, exponential type)
type II with ξ > 0 (heavy tailed, Pareto type)
type III with ξ < 0 (bounded, beta type)

29
 Peak over threshold
It shall be noted that in the GPD setting the scale parameter 𝝈⁡is dependent
on the threshold.

Meaning if the distribution of the excess Yi has an exact GP distribution

(rather than only approximate) then increasing the threshold from 𝑢 to 𝑢∗
would result in another GP distribution with the same shape parameter ξ, but
an adjusted scale parameter given by

𝜎 𝑢∗ = 𝜎 𝑢 + 𝜉 𝑢∗ − 𝑢 , 𝑢∗ > 𝑢

Note that the scale parameter would increase if ξ > 0 and decrease if ξ < 0.
Consistent with the exponential distribution, there would be no change in the
scale parameter if ξ = 0.

30
 Peak over threshold
Selection of a threshold involves a delicate trade-off between bias and
variance.

Too high a threshold will reduce the number of exceedances and thus
increase the estimation variance and the reliability of the parameter
estimates, whereas too low a threshold will induce a bias because the GPD
will fit the exceedances poorly.

31
 Peak over threshold - Application

Parameter Model 1 Model 2

𝜇 70 77
𝜎 22.77 (±2.48) 17.36 (±2.26)
𝜉 -0.37 (± 0.06) -0.29 (± 0.08)
AIC 791.871 616.086

32
 Peak over threshold - Application

Parameter Model 1 Model 2

𝜇 70 77 AIC = 2𝑘 − 2ln 𝐿
𝜎 22.77 (±2.48) 17.36 (±2.26) 𝑘 … number of parameters
𝜉 -0.37 (± 0.06) -0.29 (± 0.08) 𝐿⁡ … maximized value of likelihood function
AIC 791.871 616.086

33
 Dependence and Declustering
Dependence among observation

The EVT concepts introduced build on the assumption of independent

identically distributed variables. Though we know that in practice most
extreme values arise from a series of dependent observations.

Fortunately for sets of observations without long-range dependence the

same limiting results apply for maxima of dependent time series as for
independent time series (see Leadbetter et al., 1983 and Leadbetter and
Rotzen 1988, for details).

In practice if a series of observations X1,...,Xk, with no long-range dependence

of extremes has short-range dependence, which leads to extremes occurring
in clusters with mean size 1/𝜃, where 0 ≤ 𝜃 ≤ 1; ⁡𝜃 is called the extremal
index of the process.

34
Dependence and Declustering

35
Dependence and Declustering

36
 Dependence and Declustering
∗ ∗
So if we consider a series of independent variables⁡𝑋 1 , … , 𝑋 𝑘 with the same
marginal distribution as 𝑋𝑗 , then 𝑀𝑘 = 𝑎𝑘 max 𝑋1 , … , 𝑋𝑘 − 𝑏𝑘 has a
nondegenerate limiting distribution 𝐻(𝑦)⁡ if and only if
∗ ∗ ∗
𝑀 𝑘 = 𝑎𝑘 max 𝑋 1 , … , 𝑋 𝑘 − 𝑏𝑘 has a nondegenerate distribution 𝐻 ∗ (𝑦),
𝜃
and 𝐻 𝑦 = 𝐻 ∗ (𝑦) .

Thus the limiting distribution is unaffected by short term dependence as

although the location and scale parameters differ, 𝐻(𝑦) and 𝐻 ∗ (𝑦) have the
same shape parameter .

Thus in practice the solution to clustering is to (i) identify clusters, and (ii) fit
the point process model to cluster maxima.

RECAP GEV: As the GEV is automatically fitted to block maxima, parameters

are automatically adjusted for any temporal clustering.

37
 Stationarity vs. Non-stationarity
Stationarity vs. non-stationarity
In statistics, a stationary process is a stochastic process whose joint
probability distribution does not change when shifted in time.

The approaches and examples discussed so far have all assumed stationarity
in the underlying time series. Non-stationarity can be introduced in EVT
models by expressing one or multiple parameters as a function of a covariate
(e.g. time) .

38
 Non-Stationary Block Maxima
Non-stationary Block maxima
As candidate model for the non-stationary GEV we can assume a model
where linear trends in the location and log-transformed scale parameter [to
constrain 𝜎 𝑡 > 0 ] are considered while no trend is considered in the shape
parameter.

𝜇 𝑡 = 𝜇0 + 𝜇1 𝑡, ln 𝜎 𝑡 = 𝜎0 + 𝜎1 𝑡, 𝜉 𝑡 = 𝜉

The parameter 𝜇1 can be interpreted as slope of a linear trend in the center

of the distribution, and the transformed parameter exp ( 𝜎1 ) as the
appropriate rate of change in the scale (or size of the distribution).

39
 Non-Stationary Block Maxima
Such trend can be readily interpreted in terms of the corresponding time
varying quantile (or ‘effective’ return level) which would reduce to a
conventional return level (with return period 1/𝑚) if it would not vary with
time.

If the location and/or scale parameter have linear time trends, then the
effective return level would also change linearly.

So we can fit three different forms of GEV distributions to our data.

(1) A stationary GEV in which none of the parameters depends on time
(2) A nonstationary GEV in which either 𝜇 or ln(𝜎) depend on time
(3) A nonstationary GEV in which both 𝜇 and ln(𝜎) depend on time

Model comparison and model selection involves the minimized negative log
likelihood for the candidate models via AIC.

40
 Non-Stationary Block Maxima - Application
Lets look on an example for our Central Park Tmax data.

We compare two models:

(1) A stationary GEV in which none of the parameters depends on time
(2) A nonstationary GEV in which 𝜇 depends on time

+0.08 C /decade

41
 Non-Stationary Block Maxima - Application
Lets look on an example for our Central Park Tmax data.

We compare two models:

(1) A stationary GEV in which none of the parameters depends on time
(2) A nonstationary GEV in which 𝜇 depends on time

Parameter Model 1 Model 2

+0.08 C /decade
𝜇 35.28 (±0.17) 35.31 (±0.16)
+ μ(t)
𝜎 1.75 (±0.12) 1.75 (±0.12)
𝜉 -0.19 (± 0.06) -0.21 (± 0.05)
AIC 565.73 564.63
5-year RL 37.5 37.5
100-year RL 40.6 40.4

42
Non-Stationary Block Maxima - Application

43
 Non-Stationary r-largest order extremes
Nonstationarity in r-largest order extremes

Introducing time dependency in parameter estimates works for r-largest

order models as for block maxima. Thus as for block maxima we can consider
fits of three different forms of GEV distributions to our data.

(1) A stationary GEV in which none of the parameters depends on time

(2) A nonstationary GEV in which either 𝜇 or ln(𝜎) depend on time
(3) A nonstationary GEV in which both 𝜇 and ln(𝜎) depend on time

Let’s consider this in an example for the 3 warmest summer days from the
Central Park record.

44
 Non-Stationary r-largest order extremes
As for the block maxima approach we want to compare two models:
(1) A stationary r-largest order model where none of the GEV parameters
depends on time
𝜇 𝑡 = 𝜇, ln 𝜎 𝑡 = 𝜎, 𝜉 𝑡 = 𝜉

(2) A nonstationary r-largest order model where 𝜇 depends on time

𝜇 𝑡 = 𝜇0 + 𝜇1 𝑡, ln 𝜎 𝑡 = 𝜎⁡⁡⁡𝜉 𝑡 = 𝜉

+0.08 C /decade

45
 Non-Stationary r-largest order extremes - Application

Parameter Model 1 Model 2

𝜇 38.84 (±0.09) 38.86 (±0.16) + μ(t)
𝜎 1.69 (±0.06) 1.69 (±0.06)
𝜉 -0.21 (± 0.03) -0.22 (± 0.03)
AIC 1653.7 1644.1
5-year RL 37.0 (37.5) 37.0 (37.5) +0.08 C /decade
100-year RL 39.8 (40.6) 39.7 (40.4)

46
Non-Stationary r-largest order extremes - Application

47
 Non-Stationary peak over threshold
Non-stationarity in Peak over Threshold models
Frequently we are interested in extremes defined as exceedances of a certain
threshold and we know that the POT model is the suitable EVT model for
such type of analysis. Non-stationarity can be addressed in POT models
though a bit caution is needed as:

(1) Normally the threshold we are interested in is a fixed quantity,

nevertheless a time-varying threshold can be introduced in POT models.
(2) The scale parameter is dependent on u thus care must be given in the
interpretation of σ when using a time varying threshold.
(3) If we are only interested in the change in return levels due to changes in
the evolution of the data (e.g., a trend) we can assess changes by fitting
the GPD to different time periods an compare the RL estimates.

48
 Non-Stationary peak over threshold
Let’s look on mda8 O3 return periods for two different time periods
(1) 1988-2000
(2) 2001-2013

49
 Return level estimates under non-stationarity
Communicating risk in a non-stationary setting – notes on return level
estimates under non-stationarity

Let 𝐹 𝑦 be the distribution function of 𝑀(𝑦). In any particular year there is

still a one to one relationship between a probability of exceedance and a high
quantile.

So for a given level of interest 𝑟, it is straightforward to express the yearly risk

in terms of probability.

𝑝 𝑦 = 𝑃 𝑀𝑦 > 𝑟 = 1 − 𝐹𝑦 𝑟 .

Once 𝐹𝑦 is estimated it is straightforward to provide yearly estimates of the

probability of an exceedance 𝑝 𝑦 .

50
 Return level estimates under non-stationarity
Risk calculations however proceed in the opposite direction. Normally one
starts with a return period in the stationary case and finds the corresponding
level.

Inverting the previous procedure arriving at a probability of exceedance 𝑝,

one starts with the probability of exceedance 𝑝 and solves

𝐹𝑦 𝑟𝑝 (𝑦) = 1 − 𝑝 .

The exceedance level 𝑟𝑝 (𝑦) changes with every year thus communicates
clearly the changing nature of risk.

51
 Return level estimates under non-stationarity
If we define the return level 𝑟𝑚 ⁡as the expected waiting time until an
exceedance occurs in 𝑚 years, then 𝑟𝑚 ⁡is the solution to equation

∞ 𝑖

𝑚 =1+ 𝐹(𝑦)(𝑟𝑚 )
𝑖=1 𝑦=1

It shall be noted that as it cannot be written as geometric series, solving for

𝑟𝑚 is not straightforward. For the case 𝐹𝑦 (𝑟) is monotonically decreasing as
𝑦 → ∞ (extremes are getting more extreme) it is possible to bound the right
hand side of the equation above (see Cooley 2013 for more details).

52
 Return level estimates under non-stationarity
The other interpretation of an 𝑚-year return period under the stationary case
was that the expected number of exceedances in 𝑚 years is one.

To extend this for the non-stationary case we aim to find the level 𝑟𝑚 for
which the expected number of exceedances in 𝑚 years is one.

So if we let 𝑁 be the number of exceedances that occur in the 𝑚 years

beginning with year 𝑦 = 1 and ending at year 𝑦 = 𝑚. Then as the probability
of an exceedance is no longer constant from year to year and we define the
𝑚-year return level 𝑟𝑚 ⁡to be the solution to the equation
𝑚

1= 1 − 𝐹𝑦 𝑟𝑚 .
𝑦=1

53
 Suitability of EVT models for climate extremes

Block Maximum Models:

Warmest daily Tmax (TXx), Warmest daily Tmin (TNx)
Coldest daily Tmax (TXn), Coldest daily Tmin (TNn)
Wettest day (RX1day)
Wettest consecutive 5 days (RX5day)

R-largest order models:

Cold days (TX10p), Cold Nights (TN10p)
Warm days (TX90p), Warm Nights (TN90p)

Peak-Over-Threshold Models:
Frost days (FD), Tropical Nights (TR)
Cold days (TX10p), Cold Nights (TN10p)
Warm days (TX90p), Warm Nights (TN90p)

55
Kodra & Ganguly, Nature Scientific Reports, 2014
 Suitability of EVT models for climate extremes
For those who are using R

Packages for extreme value analysis

evd
ismev
POT
extRemes

57