Chap 6

Chapter 6
Univariate time series modelling and forecasting
‘Introductory Econometrics for Finance’ c Chris Brooks 2013 1

Univariate Time Series Models
Where we attempt to predict returns using only information contained

in their past values.
Some Notation and Concepts
A Strictly Stationary Process

A strictly stationary process is one where
P{yt1 ≤ b1 , . . . , ytn ≤ bn } = P{yt1 +m ≤ b1 , . . . , ytn +m ≤ bn }
A Weakly Stationary Process

Univariate Time Series Models (Cont’d)
If a series satisfies the next three equations, it is said to be weakly or
covariance stationary
(1) E (yt ) = µ t = 1, 2, . . . , ∞
(2) E (yt − µ)(yt − µ) = σ 2 < ∞
(3) E (yt1 − µ)(yt2 − µ) = γt2 −t1 ∀ t1 , t2
So if the process is covariance stationary, all the variances are the

same and all the covariances depend on the difference between t1 and
t2 . The moments
E (yt − E (yt ))(yt−s − E (yt−s )) = γs , s = 0, 1, 2, . . .
are known as the covariance function.

The covariances, γs , are known as autocovariances.

Univariate Time Series Models (Cont’d)
However, the value of the autocovariances depend on the units of

measurement of yt .
It is thus more convenient to use the autocorrelations which are the
autocovariances normalised by dividing by the variance:
γs
τs = , s = 0, 1, 2, . . .
γ0
If we plot τs against s=0,1,2,... then we obtain the autocorrelation

function or correlogram.

A White Noise Process
A white noise process is one with (virtually) no discernible structure.

A definition of a white noise process is
E (yt ) = µ
var(yt ) = σ 2
2
σ if t = r
γt−r =
0 otherwise
Thus the autocorrelation function will be zero apart from a single

peak of 1 at s=0. τ̂s ∼ approx. N(0, 1/T ) where T = sample size
We can use this to do significance tests for the autocorrelation

coefficients by constructing a confidence interval.

A White Noise Process (Cont’d)
For example, a 95 % confidence interval would be given by
1
±1.96 × √
T
If the sample autocorrelation coefficient, τ̂s , falls outside this region

for any value of s, then we reject the null hypothesis that the true
value of the coefficient at lag s is zero.

Joint Hypothesis Tests
We can also test the joint hypothesis that all m of the τk correlation
coefficients are simultaneously equal to zero using the Q-statistic
developed by Box and Pierce:
m
X
Q=T τ̂k2
k=1
where T=sample size, m=maximum lag length

The Q-statistic is asymptotically distributed as a χ2m .
However, the Box Pierce test has poor small sample properties, so a
variant has been developed, called the Ljung-Box statistic:
m
∗
X τ̂k2
Q = T (T + 2) ∼ χ2m
T −k
k=1
This statistic is very useful as a portmanteau (general) test of linear

dependence in time series.
An ACF Example
Question:
Suppose that a researcher had estimated the first 5 autocorrelation
coefficients using a series of length 100 observations, and found them
to be (from 1 to 5): 0.207, -0.013, 0.086, 0.005, -0.022.
Test each of the individual coefficient for significance, and use both
the Box-Pierce and Ljung-Box tests to establish whether they are
jointly significant.
Solution:
A coefficient would be significant if it lies outside (-0.196,+0.196) at
the 5% level, so only the first autocorrelation coefficient is significant.
Q=5.09 and Q*=5.26
Compared with a tabulated χ2 (5)=11.1 at the 5% level, so the 5
coefficients are jointly insignificant.

Moving Average Processes
Let ut (t = 1, 2, 3, . . . ) be a sequence of independently and
identically distributed (iid) random variables with E(ut ) = 0 and
var(ut ) = σ 2 , then
yt = µ + ut + θ1 ut−1 + θ2 ut−2 + · · · + θq ut−q
is a qth order moving average model MA(q).

Its properties are
E(yt ) = µ
var(yt ) = γ0 = 1 + θ12 + θ22 + · · · + θq2 σ 2

Covariances
(
(θs + θs+1 θ1 + θs+2 θ2 + · · · + θq θq−s ) σ 2 for s = 1, . . . , q
γs =
0 for s > q

Example of an MA Problem
1 Consider the following MA(2) process:
yt = ut + θ1 ut−1 + θ2 ut−2
where ut is a zero mean white noise process with variance σ 2 .

i. Calculate the mean and variance of Xt
ii. Derive the autocorrelation function for this process (i.e. express the
autocorrelations, τ1 , τ2 , ...as functions of the parameters θ1 and θ2 ).
iii. If θ1 = −0.5 and θ2 = 0.25, sketch the acf of Xt .

Solution
i. If E (ut ) = 0, then E(ut−i ) = 0 ∀ i So
E(yt ) = E(ut + θ1 ut−1 + θ2 ut−2 )

= E(ut ) + θ1 E(ut−1 ) + θ2 E(ut−2 ) = 0
var(yt ) = E[yt − E(yt )][yt − E(yt )]
But E(yt ) = 0, so
var(yt ) = E[(yt )(yt )]
var(yt ) = E[(ut + θ1 ut−1 + θ2 ut−2 )(ut + θ1 ut−1 + θ2 ut−2 )]

2
var(yt ) = E ut2 + θ12 ut−1 + θ22 ut−2
2

+ cross-products
But E[cross-products] = 0 since cov(ut , ut−s ) = 0 for s 6= 0.

Solution (Cont’d)
var(yt ) = γ0 = E ut2 + θ12 ut−1

2
+ θ22 ut−2
2

So
= σ 2 + θ12 σ 2 + θ22 σ 2
= 1 + θ12 + θ22 σ 2


Solution (Cont’d)
ii. The acf of yt
γ1 = E[yt − E(yt )][yt−1 − E(yt−1 )]
γ1 = E[yt ][yt−1 ]
γ1 = E[(ut + θ1 ut−1 + θ2 ut−2 )(ut−1 + θ1 ut−2 + θ2 ut−3 )]

2 2

γ1 = E θ1 ut−1 + θ1 θ2 ut−2
γ 1 = θ 1 σ 2 + θ1 θ2 σ 2
γ1 = (θ1 + θ1 θ2 )σ 2

Solution (Cont’d)
γ2 = E[yt − E(yt )][yt−2 − E(yt−2 )]

γ2 = E[yt ][yt−2 ]
2

γ2 = E θ2 ut−2
γ2 = θ 2 σ 2
γ3 = E[yt − E(yt )][yt−3 − E(yt−3 )]
γ3 = E[yt ][yt−3 ]
γ3 = 0
Solution (Cont’d)
So γs = 0 for s > 2.
We have the autocovariances, now calculate the autocorrelations:
γ0
τ0 = =1
γ0
γ1 (θ1 + θ1 θ2 )σ 2 (θ1 + θ1 θ2 )
τ1 = = 2 2
=
1 + θ12 + θ22

γ0 1 + θ1 + θ2 σ 2
γ2 (θ2 )σ 2 θ2
τ2 = = 2 2
=
1 + θ12 + θ22

γ0 1 + θ1 + θ2 σ 2
γ3
τ3 = =0
γ0
γs
τs = =0 ∀ s>2
γ0

Solution (Cont’d)
iii. For θ1 = −0.5 and θ2 = 0.25, substituting these into the formulae
above gives τ1 = −0.476, τ2 = 0.190.

ACF Plot
Thus the acf plot will appear as follows:
1.2
0.8
0.6
0.4
acf
0.2
0
0 1 2 3 4 5
–0.2
–0.4
–0.6
lag, s

Autoregressive Processes
An autoregressive model of order p, an AR(p) can be expressed as
yt = µ + φ1 yt−1 + φ2 yt−2 + · · · + φp yt−p + ut
Or using the lag operator notation:
Lyt = yt−1 Li yt = yt−i
p
X
yt = µ + φi yt−i + ut
i=1
or
p
X
yt = µ + φi Li yt + ut
i=1
or φ(L)yt = µ + ut where φ(L) = (1 − φ1 L − φ2 L2 − · · · − φp Lp ).

The Stationarity Condition for an AR Model
The condition for stationarity of a general AR( p) model is that the

roots of 1 − φ1 z − φ2 z 2 − · · · − φp z p = 0 all lie outside the unit
circle.
A stationary AR(p) model is required for it to have an MA(∞)

representation.
Example 1: Is yt = yt−1 + ut stationary?

The characteristic root is 1, so it is a unit root process (so
non-stationary)
Example 2: Is yt = 3yt−1 − 2.75yt−2 + 0.75yt−3 + ut stationary?

The characteristic roots are 1, 2/3, and 2. Since only one of these lies
outside the unit circle, the process is non-stationary.

The Moments of an Autoregressive Process
The moments of an autoregressive process are as follows. The mean
is given by
φ0
E (yt ) =
1 − φ1 − φ2 − · · · − φp
The autocovariances and autocorrelation functions can be obtained

by solving what are known as the Yule-Walker equations:
τ1 = φ1 + τ1 φ2 + · · · + τp−1 φp
τ2 = τ1 φ1 + φ2 + · · · + τp−2 φp
.. .. ..
. . .
τp = τp−1 φ1 + τp−2 φ2 + · · · + φp
If the AR model is stationary, the autocorrelation function will decay

exponentially to zero.
Sample AR Problem
Consider the following simple AR(1) model
yt = µ + φ1 yt−1 + ut
i. Calculate the (unconditional) mean of yt .

For the remainder of the question, set µ = 0 for simplicity.
ii. Calculate the (unconditional) variance of yt .
iii. Derive the autocorrelation function for yt .

Solution
i. Unconditional mean:
E(yt ) = E(µ + φ1 yt−1 )

E(yt ) = µ + φ1 E(yt−1 )
But also
E(yt ) = µ + φ1 (µ + φ1 E(yt−2 ))
= µ + φ1 µ + φ21 E(yt−2 )
= µ + φ1 µ + φ21 (µ + φ1 E(yt−3 ))

Solution (Cont’d)
So
E(yt ) = µ + φ1 (µ + φ1 E(yt−2 ))
= µ + φ1 µ + φ21 E(yt−2 )
= µ + φ1 µ + φ21 (µ + φ1 E(yt−3 ))
E(yt ) = µ + φ1 µ + φ21 µ + φ31 E(yt−3 )
An infinite number of such substitutions would give
E(yt ) = µ 1 + φ1 + φ21 + · · · ) + φ∞
1 y0
So long as the model is stationary, i.e. |φ1 | < 1, then φ∞

1 = 0.
So
µ
E(yt ) = µ 1 + φ1 + φ21 + · · · =

1 − φ1
Solution (Cont’d)
ii. Calculating the variance of yt : yt = φ1 yt−1 + ut

From Wold’s decomposition theorem:
yt (1 − φ1 L) = ut
yt = (1 − φ1 L)−1 ut
1 + φ1 L + φ21 L2 + · · · ut

yt =
So long as, |φ1 | < 1, this will converge.
var(yt ) = E[yt − E(yt )][yt − E(yt )]

Solution (Cont’d)
but E(yt ) = 0, since µ is set to zero.
var(yt ) = E[(yt )(yt )]

= E ut + φ1 ut−1 + φ21 ut−2 + · · · ut + φ1 ut−1

+φ21 ut−2 + · · ·

= E ut2 + φ21 ut−1

2
+ φ41 ut−2
2

+ · · · + cross-products
= E ut2 + φ21 ut−1
2
+ φ41 ut−2
2

+ ···
= σu2 + φ21 σu2 + φ41 σu2 + · · ·
= σu2 1 + φ21 + φ41 + · · ·

σu2
=
(1 − σu2 )

Solution (Cont’d)
iii. Turning now to calculating the acf, first calculate the autocovariances:
γ1 = cov (yt , yt−1 ) = E[yt − E (yt )][yt−1 − E (yt−1 )]
Since a0 has been set to zero, E(yt ) = 0 and E(yt−1 ) = 0, so
γ1 = E[yt yt−1 ]
under the result above that E(yt ) = E(yt−1 ) = 0. Thus
γ1 = E ut + φ1 ut−1 + φ21 ut−2 + · · · ut−1 + φ1 ut−2

+ φ21 ut−3 + · · ·

2
+ φ31 ut−2
2

γ1 = E φ1 ut−1 + · · · + cross − products
γ1 = φ1 σ 2 + φ31 σ 2 + φ51 σ 2 + · · ·
φ1 σ 2
γ1 =
1 − φ21


Solution (Cont’d)
For the second autocorrelation coefficient,
γ2 = cov(yt , yt−2 ) = E[yt − E(yt )][yt−2 − E(yt−2 )]
Using the same rules as applied above for the lag 1 covariance
γ2 = E[yt yt−2 ]
γ2 = E ut + φ1 ut−1 + φ21 ut−2 + · · · ut−2 + φ1 ut−3

+ φ21 ut−4 + · · ·

γ2 = E φ21 ut−2
2
+ φ41 ut−3
2

+ · · · +cross-products
γ2 = φ21 σ 2 + φ41 σ 2 + · · ·
γ2 = φ21 σ 2 1 + φ21 + φ41 + · · ·

φ21 σ 2
γ2 =
1 − φ21


Solution (Cont’d)
If these steps were repeated for γ3 , the following expression would be

obtained
φ31 σ 2
γ3 =
1 − φ21

and for any lag s, the autocovariance would be given by
φs1 σ 2
γs =
1 − φ21


Solution (Cont’d)
The acf can now be obtained by dividing the covariances by the
variance:
γ0
τ0 = =1
γ0
!
φ1 σ 2
1 − φ21

γ1
τ1 = = ! = φ1
γ0 σ2
1 − φ21

!
φ21 σ 2
1 − φ21

γ2
τ2 = = ! = φ21
γ0 σ 2
1 − φ21

τ3 = φ31
τs = φs1
The Partial Autocorrelation Function (denoted τkk )
Measures the correlation between an observation k periods ago and
the current observation, after controlling for observations at
intermediate lags (i.e. all lags <k).
So τkk measures the correlation between yt and yt−k after removing

the effects of yt−k+1 , yt−k+2 , . . . , yt−1
At lag 1, the acf = pacf always
At lag 2,
τ22 = τ2 − τ12 1 − τ12

For lags 3+, the formulae are more complex.
The pacf is useful for telling the difference between an AR process

and an ARMA process.

The Partial Autocorrelation Function (denoted τkk )
(Cont’d)
In the case of an AR(p), there are direct connections between yt and

yt−s only for s ≤ p.
So for an AR(p), the theoretical pacf will be zero after lag p.
In the case of an MA(q), this can be written as an AR(∞), so there

are direct connections between yt and all its previous values.
For an MA(q), the theoretical pacf will be geometrically declining.

ARMA Processes
By combining the AR(p) and MA(q) models, we can obtain an
ARMA(p,q) model:
φ(L)yt = µ + θ(L)ut
where
φ(L) = 1 − φ1 L − φ2 L2 − · · · − φp Lp and
θ(L) = 1 + θ1 L + θ2 L2 + · · · + θq Lq
or
yt = µ + φ1 yt−1 + φ2 yt−2 + · · · + φp yt−p + θ1 ut−1
+ θ2 ut−2 + · · · + θq ut−q + ut
with
E(ut ) = 0; E ut2 = σ 2 ; E (ut us ) = 0, t 6= s


Summary of the Behaviour of the acf for AR and
MA Processes
An autoregressive process has
a geometrically decaying acf
number of spikes of pacf = AR order
A moving average process has
Number of spikes of acf = MA order
a geometrically decaying pacf

Some sample acf and pacf plots for standard
processes
The acf and pacf are not produced analytically from the relevant
formulae for a model of that type, but rather are estimated using
100,000 simulated observations with disturbances drawn from a
normal distribution.
Figure: Sample autocorrelation and partial autocorrelation functions for an

MA(1) model: yt = −0.5ut−1 + ut
0.05
0
1 2 3 4 5 6 7 8 9 10
–0.05
–0.1
–0.15
acf and pacf
–0.2
–0.25
–0.3
acf
–0.35 pacf
–0.4
–0.45
lag, s
ACF and PACF for an MA(2) Model:
yt = 0.5ut−1 − 0.25ut−2 + ut
0.4
0.3 acf
pacf
0.2
0.1
acf and pacf
0
1 2 3 4 5 6 7 8 9 10
–0.1
–0.2
–0.3
–0.4
lag, s

ACF and PACF for a slowly decaying AR(1) Model:
yt = 0.9yt−1 + ut
0.9
acf
0.8
pacf
0.7
0.6
acf and pacf
0.5
0.4
0.3
0.2
0.1
0
1 2 3 4 5 6 7 8 9 10
–0.1
lag, s

ACF and PACF for a more rapidly decaying AR(1)
Model: yt = 0.5yt−1 + ut
0.6
0.5
acf
0.4 pacf
acf and pacf
0.3
0.2
0.1
0
1 2 3 4 5 6 7 8 9 10
–0.1
lag, s

ACF and PACF for a more rapidly decaying AR(1)
Model with Negative Coefficient: yt = −0.5yt−1 + ut
0.3
0.2
0.1
0
1 2 3 4 5 6 7 8 9 10
acf and pacf
–0.1
–0.2
–0.3
–0.4 acf
pacf
–0.5
–0.6
lag, s

ACF and PACF for a Non-stationary Model (i.e. a
unit coefficient):yt = yt−1 + ut
0.9
0.8
0.7
0.6
acf and pacf
0.5
0.4
0.3
acf
0.2 pacf
0.1
0
1 2 3 4 5 6 7 8 9 10
lag, s

ACF and PACF for an ARMA(1,1):
yt = 0.5yt−1 + 0.5ut−1 + ut
0.8
0.6
acf
pacf
0.4
acf and pacf
0.2
0
1 2 3 4 5 6 7 8 9 10
–0.2
–0.4
lag, s

Chap 6

Uploaded by

Copyright:

Available Formats

Chap 6

Uploaded by

Document Information

Original Title

Copyright

Available Formats

Share this document

Share or Embed Document

Sharing Options

Did you find this document useful?

Is this content inappropriate?

Copyright:

Available Formats

Chap 6

Uploaded by

Copyright:

Available Formats

Chapter 6

Univariate time series modelling and forecasting

‘Introductory Econometrics for Finance’ c Chris Brooks 2013 1

Where we attempt to predict returns using only information contained

Some Notation and Concepts

A Strictly Stationary Process

P{yt1 ≤ b1 , . . . , ytn ≤ bn } = P{yt1 +m ≤ b1 , . . . , ytn +m ≤ bn }

A Weakly Stationary Process

‘Introductory Econometrics for Finance’ c Chris Brooks 2013 2

So if the process is covariance stationary, all the variances are the

E (yt − E (yt ))(yt−s − E (yt−s )) = γs , s = 0, 1, 2, . . .

are known as the covariance function.

‘Introductory Econometrics for Finance’ c Chris Brooks 2013 3

However, the value of the autocovariances depend on the units of

If we plot τs against s=0,1,2,... then we obtain the autocorrelation

‘Introductory Econometrics for Finance’ c Chris Brooks 2013 4

A white noise process is one with (virtually) no discernible structure.

Thus the autocorrelation function will be zero apart from a single

We can use this to do significance tests for the autocorrelation

‘Introductory Econometrics for Finance’ c Chris Brooks 2013 5

For example, a 95 % confidence interval would be given by

If the sample autocorrelation coefficient, τ̂s , falls outside this region

‘Introductory Econometrics for Finance’ c Chris Brooks 2013 6

where T=sample size, m=maximum lag length

This statistic is very useful as a portmanteau (general) test of linear

‘Introductory Econometrics for Finance’ c Chris Brooks 2013 8

yt = µ + ut + θ1 ut−1 + θ2 ut−2 + · · · + θq ut−q

is a qth order moving average model MA(q).

‘Introductory Econometrics for Finance’ c Chris Brooks 2013 9

1 Consider the following MA(2) process:

where ut is a zero mean white noise process with variance σ 2 .

‘Introductory Econometrics for Finance’ c Chris Brooks 2013 10

i. If E (ut ) = 0, then E(ut−i ) = 0 ∀ i So

E(yt ) = E(ut + θ1 ut−1 + θ2 ut−2 )

var(yt ) = E[(yt )(yt )]

var(yt ) = E[(ut + θ1 ut−1 + θ2 ut−2 )(ut + θ1 ut−1 + θ2 ut−2 )]

But E[cross-products] = 0 since cov(ut , ut−s ) = 0 for s 6= 0.

‘Introductory Econometrics for Finance’ c Chris Brooks 2013 11

var(yt ) = γ0 = E ut2 + θ12 ut−1

‘Introductory Econometrics for Finance’ c Chris Brooks 2013 12

ii. The acf of yt

γ1 = E[yt − E(yt )][yt−1 − E(yt−1 )]

γ1 = E[(ut + θ1 ut−1 + θ2 ut−2 )(ut−1 + θ1 ut−2 + θ2 ut−3 )]

‘Introductory Econometrics for Finance’ c Chris Brooks 2013 13

γ2 = E[yt − E(yt )][yt−2 − E(yt−2 )]

γ3 = E[yt − E(yt )][yt−3 − E(yt−3 )]

γ3 = E[(ut + θ1 ut−1 + θ2 ut−2 )(ut−3 + θ1 ut−4 + θ2 ut−5 )]

‘Introductory Econometrics for Finance’ c Chris Brooks 2013 15

‘Introductory Econometrics for Finance’ c Chris Brooks 2013 16

‘Introductory Econometrics for Finance’ c Chris Brooks 2013 17

yt = µ + φ1 yt−1 + φ2 yt−2 + · · · + φp yt−p + ut

Or using the lag operator notation:

Lyt = yt−1 Li yt = yt−i

or φ(L)yt = µ + ut where φ(L) = (1 − φ1 L − φ2 L2 − · · · − φp Lp ).

The condition for stationarity of a general AR( p) model is that the

A stationary AR(p) model is required for it to have an MA(∞)

Example 1: Is yt = yt−1 + ut stationary?

Example 2: Is yt = 3yt−1 − 2.75yt−2 + 0.75yt−3 + ut stationary?

‘Introductory Econometrics for Finance’ c Chris Brooks 2013 19