IEOR E4602: Quantitative Risk Management Spring 2016

c 2016 by Martin Haugh

Multivariate Distributions
We will study multivariate distributions in these notes, focusing1 in particular on multivariate normal,
normal-mixture, spherical and elliptical distributions. In addition to studying their properties, we will also discuss
techniques for simulating and, very briefly, estimating these distributions. Familiarity with these important
classes of multivariate distributions is important for many aspects of risk management. We will defer the study
of copulas until later in the course.

1 Preliminary Definitions
Let X = (X1 , . . . Xn ) be an n-dimensional vector of random variables. We have the following definitions and
statements.

Definition 1 (Joint CDF) For all x = (x1 , . . . , xn )> ∈ Rn , the joint cumulative distribution function (CDF)
of X satisfies
FX (x) = FX (x1 , . . . , xn ) = P (X1 ≤ x1 , . . . , Xn ≤ xn ).

Definition 2 (Marginal CDF) For a fixed i, the marginal CDF of Xi satisfies

FXi (xi ) = FX (∞, . . . , ∞, xi , ∞, . . . ∞).

It is straightforward to generalize the previous definition to joint marginal distributions. For example, the joint
marginal distribution of Xi and Xj satisfies Fij (xi , xj ) = FX (∞, . . . , ∞, xi , ∞, . . . , ∞, xj , ∞, . . . ∞). If the
joint CDF is absolutely continuous, then it has an associated probability density function (PDF) so that
Z x1 Z xn
FX (x1 , . . . , xn ) = ··· f (u1 , . . . , un ) du1 . . . dun .
−∞ −∞

Similar statements also apply to the marginal CDF’s. A collection of random variables is independent if the joint
CDF (or PDF if it exists) can be factored into the product of the marginal CDFs (or PDFs). If
X1 = (X1 , . . . , Xk )> and X2 = (Xk+1 , . . . , Xn )> is a partition of X then the conditional CDF satisfies

FX2 |X1 (x2 |x1 ) = P (X2 ≤ x2 |X1 = x1 ).

If X has a PDF, f (·), then it satisfies

Z xk+1 Z xn
f (x1 , . . . , xk , uk+1 , . . . , un )
FX2 |X1 (x2 |x1 ) = ··· duk+1 . . . dun
−∞ −∞ fX1 (x1 )

where fX1 (·) is the joint marginal PDF of X1 . Assuming it exists, the mean vector of X is given by
>
E[X] := (E[X1 ], . . . , E[Xn ])

whereas, again assuming it exists, the covariance matrix of X satisfies

Cov(X) := Σ := E (X − E[X]) (X − E[X])>

 

1 We will not study copulas in these notes as well defer this topic until later in the course.
Multivariate Distributions 2

so that the (i, j)th element of Σ is simply the covariance of Xi and Xj . Note that the covariance matrix is
symmetric so that Σ> = Σ, its diagonal elements satisfy Σi,i ≥ 0, and it is positive semi-definite so that
x> Σx ≥ 0 for all x ∈ Rn . The correlation matrix, ρ(X) has as its (i, j)th element ρij := Corr(Xi , Xj ). It is
also symmetric, positive semi-definite and has 1’s along the diagonal. For any matrix A ∈ Rk×n and vector
a ∈ Rk we have

E [AX + a] = AE [X] + a (1)

>
Cov(AX + a) = A Cov(X) A . (2)

Finally, the characteristic function of X is given by

h > i
φX (s) := E eis X for s ∈ Rn (3)

and, if it exists, the moment-generating function (MGF) is given by (3) with s replaced by −i s.

2 The Multivariate Normal Distribution

If the n-dimensional vector X is multivariate normal with mean vector µ and covariance matrix Σ then we write

X ∼ MNn (µ, Σ).

The standard multivariate normal has µ = 0 and Σ = In , the n × n identity matrix. The PDF of X is given by
1 1 >
Σ−1 (x−µ)
f (x) = e− 2 (x−µ) (4)
(2π)n/2 |Σ|1/2

where | · | denotes the determinant, and its characteristic function satisfies

h > i > 1 >
φX (s) = E eis X = eis µ− 2 s Σs . (5)

Recall again our partition of X into X1 = (X1 , . . . , Xk )> and X2 = (Xk+1 , . . . , Xn )> . If we extend this
notation naturally so that
   
µ1 Σ11 Σ12
µ = and Σ = .
µ2 Σ21 Σ22

then we obtain the following results regarding the marginal and conditional distributions of X.

Marginal Distribution
The marginal distribution of a multivariate normal random vector is itself multivariate normal. In particular,
Xi ∼ MN(µi , Σii ), for i = 1, 2.

Conditional Distribution
Assuming Σ is positive definite, the conditional distribution of a multivariate normal distribution is also a
multivariate normal distribution. In particular,

X2 | X1 = x1 ∼ MN(µ2.1 , Σ2.1 )
Multivariate Distributions 3

where µ2.1 = µ2 + Σ21 Σ−1 −1

11 (x1 − µ1 ) and Σ2.1 = Σ22 − Σ21 Σ11 Σ12 .

Linear Combinations
Linear combinations of multivariate normal random vectors remain normally distributed with mean vector and
covariance matrix given by (1) and (2), respectively.

Estimation of Multivariate Normal Distributions

The simplest and most common method of estimating a multivariate normal distribution is to take the sample
mean vector and sample covariance matrix as our estimators of µ and Σ, respectively. It is easy to justify this
choice since they are the maximum likelihood estimators. It is also common to take n/(n − 1) times the sample
covariance matrix as an estimator of Σ as this estimator is known to be unbiased.

Testing Normality and Multivariate Normality

There are many tests that can be employed for testing normality of random variables and vectors. These include
standard univariate tests and tests based on QQplots, as well omnibus moment tests based on whether the
skewness and kurtosis of the data are consistent with a multivariate normal distribution. Section 3.1.4 of MFE
should be consulted for details on these tests.

2.1 Generating Multivariate Normally Distributed Random Vectors

Suppose we wish to generate X = (X1 , . . . , Xn ) where X ∼ MNn (0, Σ). Note that it is then easy to handle the
case where E[X] 6= 0. Let Z = (Z1 , . . . , Zn )> where the Zi ’s are IID N(0, 1) for i = 1, . . . , n. If C is an
(n × m) matrix then it follows that
C> Z ∼ MN(0, C> C).
Our problem therefore reduces to finding C such that C> C = Σ. We can use the Cholesky decomposition of Σ
to find such a matrix, C.

The Cholesky Decomposition of a Symmetric Positive-Definite Matrix

A well known fact from linear algebra is that any symmetric positive-definite matrix, M, may be written as

M = U> DU

where U is an upper triangular matrix and D is a diagonal matrix with positive diagonal elements. Since Σ is
symmetric positive-definite, we can therefore write
√ √ √ √
Σ = U> DU = (U> D)( DU) = ( DU)> ( DU).
√
The matrix C = DU therefore satisfies C> C = Σ. It is called the Cholesky Decomposition of Σ.

The Cholesky Decomposition in Matlab and R

It is easy to compute the Cholesky decomposition of a symmetric positive-definite matrix in Matlab and R using
the chol command and so it is also easy to simulate multivariate normal random vectors. As before, let Σ be an
(n × n) variance-covariance matrix and let C be its Cholesky decomposition. If X ∼ MN(0, Σ) then we can
generate random samples of X in Matlab as follows:

Sample Matlab Code

Multivariate Distributions 4

>> Sigma = [1.0 0.5 0.5;

0.5 2.0 0.3;
0.5 0.3 1.5];
>> C = chol(Sigma);
>> Z = randn(3,1000000);
>> X = C’*Z;
>> cov(X’)

ans =
0.9972 0.4969 0.4988
0.4969 1.9999 0.2998
0.4988 0.2998 1.4971

We must be very careful2 in Matlab3 and R to pre-multiply Z by C> and not C. We have the following
algorithm for generating multivariate random vectors, X.
Generating Correlated Normal Random Variables

generate Z ∼ MN(0, I)
/∗ Now compute the Cholesky Decomposition ∗/
compute C such that C> C = Σ
set X = C> Z

3 Normal-Mixture Models
Normal-mixture models are a class of models generated by introducing randomness into the covariance matrix
and / or the mean vector. Following the development of MFE we have the following definition of a normal
variance mixture:

Definition 3 The random vector X has a normal variance mixture if

√
X ∼ µ + W AZ

where
(i) Z ∼ MNk (0, Ik )
(ii) W ≥ 0 is a scalar random variable independent of Z and
(iii) A ∈ Rn×k and µ ∈ Rn are a matrix and vector of constants, respectively.

Note that if we condition on W , then X is multivariate normally distributed. This observation also leads to an
obvious simulation algorithm for generating samples of X: first simulate a value of W and then simulate X
conditional on the generated value of W . We are typically interested in the case where rank(A) = n ≤ k and
Σ = AA> is a full-rank positive definite matrix. In this case we obtain a non-singular normal variance mixture.
Assuming W is integrable4 , we immediately see that

E[X] = µ and Cov(X) = E[W ] Σ.

2 Wemust also be careful that Σ is indeed a genuine variance-covariance matrix.
3 Unfortunately,some languages take C> to be the Cholesky Decomposition rather C. You must therefore always be aware
of exactly what convention your programming language / package is using.
4 That is, W is integrable if E[W ] < ∞.
Multivariate Distributions 5

We refer to µ and Σ as the location vector and dispersion matrix of the distribution. It is also clear that the
correlation matrices of X and AZ are the same as long as W is integrable. This means that if A = Ik then the
components of X are uncorrelated, though they are not in general independent. The following result5
emphasizes this point.

Lemma 1 Let X = (X1 , X2 ) have a normal mixture distribution with A = I2 , µ = 0 and E[W ] < ∞ so that
Cov(X1 , X2 ) = 0. Then X1 and X2 are independent if and only if W is a constant with probability 1. (If W is
constant then X1 and X2 are IID N(0, W ).)
Proof: If W is a constant then it immediately follows from the independence of Z1 and Z2 that X1 and X2 are
also independent. Suppose now that X1 and X2 are independent. Note that
E[|X1 | |X2 |] = E[W |Z1 | |Z2 |] = E[W ] E[|Z1 | |Z2 |]
 √ 2
≥ E[ W ] E[|Z1 | |Z2 |] = E[|X1 |] E[|X2 |]

with equality only if W is a constant. But the independence of X1 and X2 implies that we must have equality
and so W is indeed constant almost surely.

Example 1 (The Multivariate Two-Point Normal Mixture Model)

Perhaps the simplest example of the normal-variance mixture is obtained when W is a discrete random variable.
If W is binary and takes on two values, w1 and w2 with probabilities p and 1 − p, respectively, then we obtain
the two-point normal mixture model. We can create a two regime model by setting w2 large relative to w1 and
choosing p large. Then W = w1 can correspond to an ordinary regime whereas W = w2 corresponds to a stress
regime.

Example 2 (The Multivariate t Distribution)

The multivariate t distribution with ν degrees-of-freedom (dof) is obtained when we take W to have an inverse
gamma distribution or equivalently, if ν/W ∼ χ2ν and this is the more familiar description of the t distribution.
We write X ∼ tn (ν, µ, Σ) and note that Cov(X) = νΣ/(ν − 2) but this is only defined when ν > 2. As we
can easily simulate chi-squared random variables, it is clearly also easy to simulate multivariate t random
vectors. The multivariate t distribution plays an important role in risk management as it often provides a very
good fit to asset return distributions.

We can easily calculate the characteristic function of a normal variance mixture. Using (5), we obtain
h > i h h > ii
φX (s) = E eis X = E E eis X |W
h > 1 > i
= E eis µ− 2 W s Σs
 
is> µ c 1 >
= e W s Σs (6)
2
 
c (·) is the Laplace transform of W . As a result, we sometimes use the notation X ∼ Mn µ, Σ, W
where W c for
normal variance mixtures. We have the following proposition6 showing that affine transformations of normal
variance mixtures remain normal variance mixtures.
 
Proposition 1 If X ∼ Mn µ, Σ, W c and Y = BX + b for B ∈ Rk×n and b ∈ Rk then
 
Y ∼ Mk Bµ + b, BΣB> , W c .

The proof is straightforward using (6). This result is useful in the following setting: suppose a collection of risk
factors has a normal variance mixture distribution. Then the usual linear approximation to the loss distribution
will also have a (1-dimensional) normal variance mixture distribution.
5 This result is Lemma 3.5 in MFE.
6 This is Proposition 3.9 in MFE.
Multivariate Distributions 6

Normal Mean-Variance Mixtures

We could also define normal mixture distributions where the mean vector, µ, is also a function of the scalar
random variable, W , so that µ = m(W ), say. We would still obtain that X is multivariate normal, conditional
on W . An important class of normal mean-variance mixtures is given by the so-called generalized hyperbolic
distributions. They, as well as the normal variance mixtures, are closed under addition, are easy to simulate and
can be calibrated using standard statistical techniques. We will not study these normal mean-variance mixtures
in this course but MFE should be consulted if further details are required.

4 Spherical and Elliptical Distributions

We now provide a very brief introduction to spherical and elliptical distributions. Spherical distributions
generalize uncorrelated multivariate normal and t distributions. In addition to having uncorrelated7 components,
they have identical and symmetric marginal distributions. The elliptical distributions can be obtained as affine
transformations of the spherical distributions. They include, for example, general multivariate normal and t
distributions as well as all spherical distributions. Elliptical distributions are an important class of distributions:
they inherit much of the normal distribution’s tractability yet they are sufficiently rich to include empirically
plausible distributions such as, for example, many heavy-tailed distributions.
Our introduction to these distributions will be very brief for several reasons. First, there is a large body of
literature associated with spherical and elliptical distributions and we simply don’t have time to study this
literature in any great detail. Second, we are already familiar with the (heavy-tailed) multivariate t distribution
which often provides an excellent fit to financial return data. Hence, the need to study other multivariate
empirically plausible distributions is not quite so pressing. And finally, it is often the case that our ultimate goal
is to study the loss8 distribution. This is a univariate distribution, however, and it is often more convenient to
take a reduced form approach and to directly estimate this distribution rather than estimating the multivariate
distribution of the underlying risk factors. We may see an example of this approach later in the course when we
use extreme value theory (EVT) to estimate the VaR of a portfolio.

4.1 Spherical Distributions

We first define spherical distributions. In order to do so, recall that a linear transformation U ∈ Rn×n is
orthogonal if UU> = U> U = In .
Definition 4 A random vector X = (X1 , . . . , Xn ) has a spherical distribution if

UX ∼ X (7)

for every orthogonal linear transformation, U ∈ Rn×n .

In particular, (7) implies that the distribution of X is invariant under rotations. A better understanding of
spherical distributions may be obtained from the following9 theorem.

Theorem 2 The following are equivalent:

1. X is spherical.
2. There exists a function ψ(·) such that for all s ∈ Rn ,

φX (s) = ψ(s> s) = ψ(s21 + · · · + s2n ). (8)

7 Theonly spherical distribution that has independent components is the standard multivariate normal distribution.
8 Assuming of course that we have already decided it is a good idea to estimate the loss distribution. This will not be the
case when we have too little historical data, or the market is too “crowded” or otherwise different to how it behaved historically.
9 This is Theorem 3.19 in MFE.
Multivariate Distributions 7

3. For all a ∈ Rn
a> X ∼ ||a|| X1
where ||a||2 = a> a = a21 + · · · + a2n .

Proof: The proof is straightforward but see Section 3.3 of MFE for details.

Part (2) of Theorem 2 shows that the characteristic function of a spherical distribution is completely determined
by a function of a scalar variable. This function, ψ(·), is known as the generator of the distribution and it is
common to write X ∼ Sn (ψ).

Example 3 (Multivariate Normal)

We know the characteristic function of the standard multivariate normal, i.e. X ∼ MNn (0, In ), satisfies
1 >
φX (s) = e− 2 s s

and so it follows from (8) that X is spherical with generator ψ(s) = exp(− 21 s).

Example 4 (Normal Variance Mixtures)

 
Suppose X has a standardized, uncorrelated normal variance mixture so that X ∼ Mn 0, In , W
c . Then (6)

and part (2) of Theorem 2 imply that X is spherical with ψ(s) = W

c (s/2).

It is worth noting that there are also spherical distributions that are not normal variance mixture distributions.
Another important and insightful result regarding spherical distributions is given in the following theorem. A
proof may be found in Section 3.4 of MFE.

Theorem 3 The random vector X = (X1 , . . . , Xn ) has a spherical distribution if and only if it has the
representation
X ∼ RS
where S is uniformly distributed on the unit sphere S n−1 := {s ∈ Rn : s> s = 1} and R ≥ 0 is a random
variable independent of S.

4.2 Elliptical Distributions

Definition 5 The random vector X = (X1 , . . . Xn ) has an elliptical distribution if

X ∼ µ + AY

where Y ∼ Sk (ψ) and A ∈ Rn×k and µ ∈ Rn are a matrix and vector of constants, respectively.

We therefore see that elliptical distributions are obtained via multivariate affine transformations of spherical
distributions. It is easy to calculate the characteristic function of an elliptical distribution. We obtain
h > i
φX (s) = E eis (µ + A Y)
>
h > >
i
= eis µ E ei(A s) Y
>
= eis µ ψ s> Σs


Multivariate Distributions 8

where as before Σ = AA> . It is common to write X ∼ En (µ, Σ, ψ) and we refer to µ and Σ as the location
vector and dispersion matrix, respectively. It is worth mentioning, however, that Σ and ψ are only uniquely
determined up to a positive constant.
As mentioned earlier, the elliptical distributions form a rich class of distributions, including both heavy- and
light-tailed distributions. Their importance is due to this richness as well as to their general tractability. For
example, elliptical distributions are closed under linear operations. Moreover, the marginal and conditional
distributions of elliptical distributions are elliptical distributions. They may be estimated using maximum
likelihood methods such as the EM algorithm or other iterative techniques. Additional information and
references may be found in MFE but we note that software applications such as R or Matlab will often fit these
distributions for you.