International
North-Holland
Journal
A FORECASTING
Robert KUNST
Instituie
for Aduanced
of Forecasting
2 (1986)
COMPARISON
447
447-456
OF SOME VAR TECHNIQUES
and Klaus NEUSSER *
Studies,
A-1060
Vienna,
Austria
Abstract:
Higher dimensional multivariate time series models suffer from the problem of overparametrisation which impairs their forecasting performance. Starting from such unrestricted vector autoregressive models the paper discusses two ways to cope’ with this
difficulty. The first approach reduces the number of free parameters by applying a subset
modelling strategy. The second approach takes a Bayesian point of view by formulating
‘priors’ which are then combined with sample information,
but leaving the original
specification unaltered. Using Austrian quarterly macroeconomic time series a comparative study is undertaken by running alternative forecasting exercises. Both methods
improve out-of-sample forecasting performance substantially at the cost of some bias in
ex-post simulations. Comparing the ex-ante predictions of the two approaches, the
former does better at short horizons whereas the latter gains as the forecast horizon
lengthens.
Keywords:
Multivariate time-series methods, Forecasting evaluation, Empirical study, Subset modeling, Bayesian analysis.
1. Introduction
Building up big structural models incrementally equation by equation has been criticized on
theoretical as well as on empirical grounds [see Sims (1980) Lucas and Sargent (1981)]. Of special
concern to these authors are the often arbitrary exclusion restrictions used to identify the models. As
far as the forecasting performance of these models is concerned, simple univariate ARIMA models
seem to be superior or at least equivalent [see McNees (1981)]. An alternative proposed by Sims
(1980) is to build vector autoregressive models with no a priori exclusion restrictions. This methodology however quickly exhausts the available degrees of freedom, since each variable has to appear in
each equation with the same lag specification. The scarcity of observations relative to the number of
parameters to be estimated brings up the problem of overparametrisation,
and the test statistics as
well as the forecasts of these models tend to deteriorate rapidly [see Fair (1979)].
This paper examines two ways to overcome this difficulty. The first one is to start out with the
unrestricted vector autoregressive model and to use the concept of Wiener-Granger causality [see
Granger (1969)] to reduce the dimensionality of the model. This procedure is similar in spirit to the
‘general-to-specific’ approach of Hendry et al. (1984) in that it tests which variables contribute
significantly to the precision of forecasting and excludes all other variables. Since the pre-test
estimator has been shown to be statistically inferior to Stein rules [see Judge and Brock (1983)], it is
tempting to contrast this approach in its forecasting performance with Bayesian point of view. This is
* The authors
0169-2070/86/$3.50
would
like to thank
0 1986, Elsevier
Manfred
Science
Deistler
Publishers
and two referees
for their
B.V. (North-Holland)
comments
and criticism
448
R. Kunst,
K. Neuwer
/ Forecasting
comparison
of VA R techniques
done by taking the unrestricted vector autoregressive system and combining the sample information
with prior information
on the parameters and their standard errors. This method has been
successfully applied by Litterman (1980), and Doan, Litterman and Sims (1984) to macroeconomic
data of the United States.
These alternative modeling techniques are evaluated by actually building models of the Austrian
economy and by using them to generate different forecasting scenarii. First, an ex-ante forecast from
1981 to 1984 is computed, so that a comparison with actual outcomes is possible. Second, an ex-post
forecast is performed to see how well the models are able to track reality.
The evaluation of alternative forecasting techniques has been the subject of extensive research
which are well summarized by Fildes (1985). Especially, it may be interesting to compare the results
obtained in this paper with the ones of Kling and Bessler (1985), who also tried to evaluate
alternative multivariate forecasting procedures - including among a dozen different techniques
Litterman’s procedure.
The plan of the paper is as follows. Section 2 presents the data and their transformations. The
next two sections present the construction of the restricted and the Bayesian vector autoregressive
model which are then compared by their forecasting performance in section 5. The last section finally
summarizes the findings and outlines future research.
2. The data
The paper proposes to analyze a small quarterly forecast model of the Austrian economy
consisting of 8 variables over the period 1964.1-1984.4:
real gross domestic product (GDP),
employment (LIZ), monetary base (MB), the deflator of gross domestic product (PGDP), the
average bond yield in the secondary market (R), the terms-of-trade (TT) calculated with respect to
goods exports and imports, average earnings of employees (WAGE)
and real goods exports (XG).
Although the selection of variables is somewhat arbitrary, they represent nonetheless key economic
indicators for any economy. ’
Before starting the actual model building process, the logarithm of each variable, with the
exception of the interest rate, is taken. This transformation is supposed to be variance stabilizing, but
it does not take out the trend. For the restricted vector autoregressive model (RVAR), which is built
in the philosophy of Box/Jenkins, all variables were therefore differenced once *. For the other
techniques no additional data transformations
are carried out in order to avoid the problems
associated with overdifferencing, which may be relevant for the interest rate and the terms of trade,
and to allow for more flexibility. Since testing is not an issue in these approaches, and since OLS will
give consistent estimates, the relative forecasting performance should not depend upon the handling
of the trend. The seasonal adjustment was done by adding seasonal dummies to each of the eight
equations in each model. Although this method is simple in postulating repeating fixed patterns, it
allows to treat the seasonality alike in each approach.
3. A restricted vector autoregressive model
Building an ‘unrestricted’ VAR model, as proposed by Sims (1980), is likely to lead very many
separately insignificant parameters, so that the search for restrictions which reduce the dimensional’ The data are available upon request
refered to Amdt (1982).
’ Nelson and Plosser (1982) presented
than a deterministic
one.
from
the authors.
evidence
that
The reader
a stochastic
especially
trend
model
interested
is more
in the Austrian
appropriate
economic
for economic
situation
is
time series
R. Kunst,
K. Neurser
/ Forecasting
compurison
of VA R techniques
449
ity of the model seems to be a rewarding task. This is especially motivated by a possible disturbing
influence of insignificant parameters on forecasting. 3
There is no unique way for setting up such restrictions. In many cases, they are based on
information extracted from theory concerning the structure and interaction of the time series. This
approach leads to some kind of approximation
of a usual structural econometric model. An
alternative is the specification of restrictions on empirical grounds, which is based on the following
procedure:
- An unrestricted time series model is estimated. F- and t-statistics are noted. Those
influence of one variable and a specific lag on an other variable, respectively.
- Insignificant variables and insignificant lags of variables are eliminated. The significance
set to loosely to 15% to reduce the possibility of neglecting significant regressors which
increased due to multicollinearity.
- The resulting model, that is, an autoregressive model with zero restrictions, is estimated
for forecasting.
test the
level was
might be
and used
To cope with possible effects of multicollinearity and to retain a reasonable amount of degrees of
freedom, the total number of regressors is restricted to 20. Thus, stepwise selection and elimination
of variables was necessary. This procedure, which is almost identical with the one used in Kunst
(1985) where it was applied successfully to a 1Zdimensional
system, started with univariate
autoregressive modelling of each of the eight time series. The maximal autoregressive lag was chosen
by the AIC criterion, and only in cases where AIC suggested an excessive number of autoregressive
lags BIC was preferred [see Akaike (1974)].
Now, by the method of OLS regression the first four lags of one or two of the other variables were
added to the p own lags identified by AIC or alternatively by BIC. Additionally, a constant and
three seasonal dummies were inserted as regressors to correct for nonstationary seasonal behavior of
the data. The resulting equation is
x, = &,Xtpj + 2PjyIej+ Xp, + c + e,.,
Applying the procedure outlined above reduces the number of parameters from 320 to only 84. The
set of regressors in the restricted model (RVAR) depends on the search strategy. A comparison of all
possible subset models, as suggested by Haggan and Oyetunji (1984) would be rather tedious and
time consuming. Anyway, the RVAR should not be ‘far away’ from the optimal model.
The significance of explaining variables offers a rough causality test in the spirit of Granger (1969)
and Sims (1972). With Granger and Sims, a variable is said to cause another one if and only if its lags
reduce the forecasting variance additionally to the proper lags of the variable to be forecasted.
Accurate significance levels, however, are not valid due to the iterative elimination process.
Influences which are quicker than one quarter of a year are not captured by the lags model. However,
according to Pierce and Haugh (1977), they are reflected in the residual correlations. The direction
cannot be determined from the data. This phenomenon is known as ‘instantaneous causality’ and
the, partially substantial, correlations are given in table 1. Within the framework of a forecasting
model, instantaneous causality could be interpreted as an indicator for flaws in the model, that is,
information which could be used for improving forecasts but is not. On the other hand, if the model
is viewed as correct, it is impossible to improve point forecasts by using these correlations which
however do affect stochastic forecasting.
3 This
fact is well documented
in the paper
by Fair
(1979)
450
R. Kunst,
Table 1
Cross-correlations
of residuals
from
K. Neusser
RVAR
/ Forecasting
comparison
of VA R techniques
model.
GDP
LE
MB
PGDP
R
TT
WA GE
XG
1.00
0.23
- 0.04
- 0.00
- 0.07
0.06
0.04
0.42
0.23
1.00
-0.18
- 0.01
- 0.07
- 0.02
-0.15
0.07
- 0.04
-0.18
1.00
0.04
-0.25
0.16
0.04
0.09
-0.00
- 0.01
0.04
1.00
-0.12
0.15
0.33
- 0.04
- 0.07
- 0.07
-0.25
- 0.12
1.00
- 0.39
- 0.07
- 0.03
0.06
- 0.02
0.16
0.15
PO.39
1.00
- 0.11
~ 0.28
0.04
- 0.15
0.04
0.33
~ 0.07
-0.11
1.00
0.26
0.42
0.07
0.09
- 0.04
- 0.03
- 0.28
0.26
1.00
GDP
LE
MB
PGDP
R
TT
WAGE
XG
4. Bayesian vector autoregression
The Bayesian approach starts with the presumption that the given data set does not contain
information
in every dimension. This means that by fitting an overparameterized system some
coefficients turn out to be non-zero just by pure chance. Since the influence of the corresponding
variables is just accidental and does not correspond to a stable relationship inherent in the data, the
out-of-sample forecasting performance of such models deteriorates quickly. The role of the Bayesian
prior can therefore be described as prohibiting coefficients to be nonzero ‘too easily’. Only if the
data really provide information will the barrier raised by the prior be broken through.
The actual model building procedure’ starts by postulating a VAR with 6 lags for each of the 8
variables, which in contrast to the RVAR model are not differenced. The next step consists in the
specification of a prior distribution for the coefficients. In this paper the so called ‘Minnesota prior’
is used [see Doan, Litterman and Sims (1984)]. It specifies a random walk process with drift for each
of the variables involved and does not allow for influences of own lags beyond the first one and of
other variables. This specification does not represent a genuine Bayesian prior, since it does not
characterize the beliefs of an investigator, who usually postulates some relationships among those
variables. The Minnesota prior could however be regarded as the intersection of the a priori beliefs of
many economists. In this sense it represents an improvement over the so called ‘diffuse prior’, which
is often used to represent the notion of ‘knowing little’. Furthermore, this specification may be
justified on the grounds that most macroeconomic time series seem to follow a stochastic trend
model [see Nelson and Plosser (1982)].
Since the prior distribution is specified as a multivariate normal, it is necessary to set, besides the
mean, the standard deviations of the coefficient of variable j with lag 1 in equation i. 4 Instead of
specifying each standard deviation separately, they are set as a function of 3 ‘meta-parameters’ r, w
and d:
s(i, j, 1) =Tf(i, j)g(l)‘j/‘j,
f(i,j)=l.O
=w
g(l)=d’-‘,
if
i=j,
if
i#j,
where
OIw11,
Osdsl,
and where the scaling factors si and sj adjust for the relative size of the variables.
4 The
covariances
are set equal
to zero.
R. Kunst,
K. Nezmer
/ Forecasting
comparison
of VAR techniques
451
The parameter r stands for the overall tightness of the prior; a smaller value indicating that more
weight is given to the prior. The function g(l) sets the form of the lag pattern, which in this case has
a geometrically declining shape with decay parameter d; the higher the lag the smaller the
corresponding standard deviation. The symmetric function f(i, j) controls the interaction among
the different variables; a higher w allows for more interaction by setting a higher a priori standard
deviation for cross effects. In this investigation two specifications of the parameters 7, d, and w are
considered:
‘loose prior’ (BVAR-loose) : 7 = 0.2
‘tight prior’ (BVAR-tight)
: 7 = 0.1
d = 1 .O w = 1 .O
d = 0.5
w = 0.5
In the first case the lag pattern does not decay and own lags and lags of other variables are treated
alike. in the second case the overall tightness is high, the lag pattern decays rapidly, and cross
influences are given less weight. The specification also includes four seasonal dummies for each
equation. Since no constant term then enters the equations, the coefficients of these variables will
determine the drift in the time series. These coefficients are not restricted a priori and are fully
determined by the data.
5. Forecasts
One of the main purposes of an econometric model is forecasting. With structural models it is
possible to evaluate economic theories or to gain insight into economic processes. With time series
models, this goal is more or less abandoned. Thus, the acid test for an econometric time series model
is its forecasting performance. The best model will be the one which forecasts best. The comparative
investigation in this paper relies on two different forecasting scenarii and several criteria for the
goodness of fit over the forecasting intervals. Furthermore, the two approaches are contrasted to an
‘unrestricted’ VAR model with 6 lags for each variable 5 in each equation (UVAR-6) to demonstrate
how overparametrisation affects forecasts performance.
The first scenario consists of an ‘ex-ante’ forecast over the period 1981.1-1984.4. For this exercise
each model is estimated over the period 1964.1-1980.4 and used to generate forecasts of each
variable from 1981.1 up to 1984.4. Then the information of the next quarter, 1981.1, is incorporated
into the model by updating the parameters through Kalman filtering and a new set of forecasts is
generated, now going from 1981.2 to 1984.4. This procedure is repeated until 1984.4 is reached when
all the available information is used. ’ In this way 16 one period, 15 two period, 14 three period, . . . ,
and 1 sixteen period forecast are generated, which can be checked against the actual realizations. The
statistics used to evaluate the different forecasts are the root mean square error (RMSE), the
U-statistic of Theil (1966), 7 and the mean absolute percentage error (MAPE). The first two criteria
are based on quadratic loss functions. Regrettably, the true forecaster’s loss function is unknown. It
even may be skew, with different costs for optimistic and pessimistic mistakes [see Jager (1985)].
However, as the modelling procedures rely on quadratic criteria, quadratic loss is a useful technical
assumption.
’ The variables
were not differenced.
6 During this updating
the parameter
vector has been kept fixed. This is especially
relevant for the RVAR model, since in this
case the search procedure
should have been repeated in each step.
’ Theil’s
(i-statistic
compares
the improvement
of a particular
forecast
against
a ‘no-change’
forecast.
Values over one
indicate that a ‘no-change’
forecast would have been better.
452
R. Kunst,
K. Neusser
/ Forecasting
of VA R techniques
comparison
Whereas ex-ante forecasting is methodologically closer to the actual situation of the forecaster
who iteratively adapts his model to the data, ex-post forecasting is compatible with asymptotic
theory which postulates a true model that is completely identifiable in the long run. The second
exercise therefore computes ex-post forecasts over 1981.1-1984.4, where the whole sample period is
used for estimation. The statistics in this case are the mean forecast error (the bias), the standard
deviation, and the value of the T-statistic against the null hypothesis that the mean forecast error is
zero.
The general conclusion to be drawn from the models’ ex-ante performance is documented in
tables 2a-c which show a marked superiority of RVAR and BVAR-tight over UVAR-6 and
BVAR-loose which is not shown here to save space. For short forecast horizons up to 3 quarters
ahead, RVAR almost dominates. For longer horizons, RVAR loses ground relative to BVAR-tight
which predicts four of the series better at step 8, but only one at step 1. RVAR and BVAR-loose are
the only specifications with all U-statistics for one period ahead smaller than one, however,
BVAR-loose quickly deteriorates as the forecast horizon becomes longer. Steps 4 and 8, which reflect
the treatment of seasonality, put up severe problems for all models, but BVAR-tight seems to be
most robust in this direction. The seasonal patterns show the worst influences in the GDP, LE and
Table 2a
&-ante
forecasts
from
the UVAR-6
model
for the period
1981.1-1984.4.
a
Variable
Forecast
step
GDP
0.0270
0.31
0.39
0.0398
0.38
0.58
0.0525
0.56
0.79
0.0704
3.63
1.08
0.0979
1.08
1.60
0.1142
1.05
1.72
0.1241
1.19
1.99
0.1148
3.18
1.74
LE
0.0106
0.56
0.12
0.0204
0.78
0.22
0.0305
1.51
0.33
0.0381
3.86
0.41
0.0551
2.48
0.57
0.0685
2.40
0.74
0.0802
3.34
0.87
0.0900
5.00
1 .Ol
MB
0.0287
1.13
0.46
0.0448
1.13
0.84
0.0540
0.99
0.94
0.0616
0.89
1.06
0.0640
0.74
1.17
0.0807
0.78
1.35
0.0817
0.67
1.31
0.0943
0.68
1.69
PGDP
0.0246
0.93
6.48
0.0226
0.66
5.18
0.0273
0.63
7.01
0.0220
0.42
4.66
0.0368
0.54
8.84
0.0255
0.33
5.36
0.0483
0.56
10.58
0.0581
0.60
11.69
R
0.633
1.55
5.90
1.246
1.66
10.95
1.792
1.74
16.40
2.061
1.63
31.26
2.540
1.68
27.84
3.009
1.72
34.06
3.420
1.74
38.70
3.445
1.60
39.01
TT
0.483
1.08
61.23
0.0611
1.40
82.67
0.0756
1.46
110.9
0.0985
3.33
159.7
0.1039
2.24
155.1
0.1213
2.89
223.6
0.1447
2.69
255.5
0.1575
4.30
299.4
WAGE
0.0287
0.33
0.61
0.0357
0.98
0.76
0.0430
0.46
0.92
0.0445
0.85
1.06
0.0595
0.55
1.31
0.0655
0.82
1.49
0.0837
0.67
1.88
0.1037
1.00
2.41
XG
0.0992
1.60
2.01
0.1160
2.22
2.11
0.2094
2.97
4.01
0.2466
4.41
4.62
0.2763
3.41
5.48
0.2868
3.32
5.85
0.3093
3.11
6.52
0.2676
2.84
5.61
1
a Rows
one to three
2
show
3
the RMSE,
4
Theil’s
U-statistic,
5
6
and the MAPE,
respectively.
I
8
R. Kunst,
Table 2b
&-ante
forecasts
from
the RVAR
model
K. Neusser
/ Forecasting
for the period
comparison
1981.1-1984.4.
of VAR techniques
453
a
Variable
Forecast
1
2
GDP
0.0142
0.16
0.22
0.0176
0.17
0.26
0.0192
0.20
0.30
0.225
1.16
0.37
0.0311
0.34
0.52
0.0389
0.36
0.65
0.0464
0.45
0.78
0.0478
1.32
0.80
LE
0.0033
0.18
0.03
0.0059
0.23
0.06
0.0086
0.43
0.10
0.0110
1.11
0.13
0.0144
0.65
0.17
0.0178
0.62
0.20
0.0216
0.90
0.26
0.0242
1.34
0.29
MB
0.0198
0.78
0.34
0.0295
0.74
0.51
0.0335
0.61
0.56
0.0362
0.52
0.56
0.0371
0.43
0.63
0.0400
0.38
0.74
0.0390
0.32
0.74
0.0324
0.23
0.63
PGDP
0.0120
0.45
2.86
0.0166
0.49
4.37
0.0171
0.39
4.58
0.0210
0.40
5.20
0.0225
0.33
5.71
0.0236
0.31
6.12
0.0204
0.24
5.06
0.0218
0.22
5.27
R
0.3066
0.75
2.75
0.5467
0.73
4.74
0.7991
0.77
76.33
0.9691
0.77
9.49
1.231
0.81
11.82
1.501
0.86
14.39
1.741
0.89
17.71
1.916
0.89
20.88
TT
0.0311
0.69
38.01
0.0398
0.91
51.74
0.0480
0.93
61.33
0.0519
1.76
71.51
0.0590
1.27
86.97
0.0639
1.52
95.52
0.0709
1.32
108.9
0.0708
1.94
118.1
WA GE
0.0121
0.14
0.26
0.0111
0.30
0.22
0.0137
0.15
0.31
0.0152
0.29
0.35
0.0240
0.22
0.52
0.0233
0.29
0.53
0.0314
0.25
0.72
0.0343
0.33
0.77
XG
0.0389
0.63
0.76
0.0450
0.86
0.82
0.0600
0.85
1.12
0.0691
1.23
1.37
0.0775
0.96
1.72
0.0889
1.03
2.02
0.0949
0.96
2.20
0.1136
1.21
2.52
a Rows
one to three
step
show
the RMSE,
4
3
Theil’s
U-statistic,
5
6
and the MAPE,
I
8
respectively
XG series, an indicator for possible seasonal trends rather than seasonal constants to be present in
the data. The performance of UVAR-6 can be improved by setting all lags of order 5 and 6 to zero.
This agrees with the supposed deteriorating influence of insignificant parameters. It is worth to
contrast this results with the one obtained by Kling and Bessler (1985) who arrive at exactly the
opposite conclusion, that is, an improvement in forecasting with an increasing number of free
parameters.
The forecasts of the terms of trade TT are unsatisfactory with all models. LE and XG show
tendencies to get out of control in the long run, especially with UVAR-6 and BVAR-loose. However,
it should be mentioned that 1981-84 represents a difficult time range for forecasting with widely
unexpected changes in employment levels and foreign trade.
It is possible to improve upon these forecasts by scanning over different values of the ‘metaparameters’ 7, d and w. Using the log-determinant of the matrix composed by cross products of 8
quarters-ahead ex-ante forecast errors during the period 1981.1-1984.4 as a criterion function,
optimal values of these parameters can be found [Doan, Litterman and Sims (1984, p. 12)]. Applying
this method to the data at hand, values of d close to one and w close to zero - leaving T unchanged
at 0.1 - have been obtained. This means the ‘optimal data based prior’ would be a univariate
454
R. Kunst,
Table 2c
E.x-ante forecasts
from
K. Neusser
the BVAR-tight
model
/ Forecasting
for the period
comparison
1981.1-1984.4.
of VAR
techniques
a
Variable
Forecast
1
2
3
GDP
0.0147
0.17
0.23
0.0229
0.22
0.36
0.0240
0.26
0.37
0.0221
1.14
0.36
0.0304
0.33
0.47
0.0390
0.36
0.65
0.0372
0.36
0.57
0.0312
0.86
0.55
IE
0.0061
0.33
0.06
0.0075
0.29
0.09
0.0122
0.60
0.14
0.0154
1.56
0.17
0.0202
0.91
0.24
0.0243
0.85
0.30
0.0294
1.22
0.36
0.0325
1.81
0.40
MB
0.0261
1.03
0.45
0.0378
0.95
0.62
0.0385
0.71
0.67
0.0300
0.43
0.47
0.0405
0.47
0.69
0.0502
0.48
0.84
0.0509
0.42
0.93
0.0380
0.27
0.60
PGDP
0.0191
0.72
5.06
0.0241
0.71
6.26
0.0253
0.58
6.00
0.0277
0.52
7.54
0.0354
0.52
8.57
0.0397
0.52
9.34
0.0372
0.43
8.73
0.0368
0.38
9.44
R
0.3569
0.87
3.09
0.6686
0.89
6.09
0.9274
0.90
8.28
1.123
0.89
10.40
1.376
0.91
13.18
1.597
0.91
15.80
1.775
0.91
18.59
1.931
0.90
21.63
TT
0.0409
0.91
58.22
0.0437
1.00
54.70
0.0471
0.91
67.70
0.0319
1.08
51.12
0.0450
0.97
73.31
0.0461
1.10
73.09
0.0554
1.03
95.03
0.0452
1.23
79.97
WA GE
0.0264
0.30
0.57
0.0359
0.98
0.85
0.0293
0.31
0.62
0.0195
0.37
0.45
0.0337
0.31
0.76
0.0466
0.59
1.02
0.0413
0.33
0.89
0.0266
0.26
0.53
XG
0.0419
0.68
0.83
0.0372
0.71
0.74
0.0460
0.865
0.82
0.0443
0.79
0.88
0.0560
0.69
1.09
0.0610
0.71
1.34
0.0533
0.54
1.16
0.0688
0.733
1.50
a Rows
one to three
step
show
the RMSE,
Theil’s
4
U-statistic,
5
6
and the MAPE,
7
8
respectively.
autoregressive model for each of the variables with no cross effects between variables. * This
unsatisfactory result is the consequence of a symmetric f(i, j) matrix which treats each of the eight
variables in the system alike and could be remedied by putting a weak economic structure on the
prior standard deviations. This is done by dividing the set of variables into core variables (in this case
GDP, LE, and WAGE) which are thought to be important in explaining all the variables of the
system and into the rest (PGDP, MB, R, TT, XG) which are thought to be of lesser importance. This
method results in a considerable improvement in the forecasting performance, especially in the
variables which have proven to be hard to forecast (R, TT, and XG).
Table 3 gives the results from ex-post forecasting and shows severe biases, especially with the
‘good’ models BVAR-tight and RVAR. It might be concluded that UVAR forecasts provide no
information
relative to no-change but do not show any systematic tendency towards over- or
underestimation. As mentioned before, ex-post forecasting methodologically
favors non-Bayesian
VAR, so the RVAR biases for all but one of the series are even more surprising. This can, however,
be explained by regarding the RVAR estimates as pre-test estimates, whose bias is a well-known fact.’
* This
result
corresponds
to the conclusion
of Kling
and Bessler
(1985, b. 15) that univariate
AR
models
are difficult
to beat.
R. Kunst,
K. Neuwr
/ Forecasting
comparison
455
of VA R technrques
6. Summary and conclusions
The results in the last sections have demonstrated that the RVAR and BVAR-tight are superior in
most respects to the other model specifications. This evidence suggests that the problems associated
with an overparametrised model can be avoided by either reducing the number of parameters
through exclusion restrictions or by placing prior restrictions on the parameters in a Bayesian way.
Using either of these techniques the forecasting performance can be considerably improved. It
therefore pays to trade a small bias against more precision in forecasting. RVAR dominates
BVAR-tight for shorter forecasting horizons but not over longer ones.
It is worth emphasizing that both approaches can be improved. First of all, there is the problem of
seasonality. The results showed that fixed seasonal factors are not the best way to handle it. The
relative forecasting performance is, however, not affected by this deficiency, since each technique has
used the same method. In the case of RVAR, it seems worthwhile to apply several test procedures to
Table 3
&-post
forecasts
Variables
of the different
models
for the period
1981.1
to 1984.4.
a
Models
Bayesian vector
autoregression
tight
Unrestricted
vector autoregression
loose
(6 lags)
Restricted
vector
autoreg.
GDP
0.01233
0.01029
4.790
0.00220
0.00601
1.463
- 0.01089
0.00669
0.651
0.03224
0.01715
7.520
LE
0.1258
0.01057
4.760
0.00555
0.00615
3.607
0.00210
0.00458
1.836
0.01522
0.01206
5.048
MB
- 0.01202
0.02451
1.962
- 0.00619
0.01910
1.296
- 0.00203
0.01586
0.511
0.00244
0.02278
0.427
PGDP
- 0.01584
0.01098
5.769
- 0.00276
0.00833
1.324
0.00058
0.00803
0.290
- 0.00901
0.01050
3.435
R
0.03204
0.89012
0.144
-0.13098
0.40098
1.307
-0.11350
0.29172
1.556
0.78934
1.191
2.650
TT
0.00096
0.03138
0.126
- 0.00126
0.02199
0.230
0.00062
0.01816
0.136
- 0.02980
0.03308
3.603
WA GE
0.00389
0.01885
0.826
o.oOQ49
0.00704
0.279
0.00122
0.00696
0.705
0.02356
0.02330
4.044
XG
0.00543
0.02954
0.736
- 0.00489
Q.02215
0.883
- 0.00172
0.01978
0.347
0.07242
0.05062
5.723
a The first entry for each variable
denotes the average forecast
reports the value of the T-statistics
against the null hypothesis
error, the second entry its standard
that the mean of the forecast error
deviation,
is zero.
and the third
456
R. Kunst,
K. Neusser
/ Forecasting
comparison
of VA R techniques
refine the specification. This may well lead to a further reduction in the number of parameters,
thereby approaching more and more a structural model. Furthermore, alternative subset modeling
strategies should be tried, since the final model may depend on the search procedure. A possible
disadvantage with this methodology is that as new information
becomes available one has, in
principle, to start over again with the search procedure since in the meantime the causality structure
may have changed. As has been mentioned earlier, the Bayesian approach can be improved by
putting more structure based on economic theory into the prior thereby abandoning the symmetric
treatment of all variables. This would make the approach more Bayesian in spirit since the prior can
now reflect better the a priori beliefs of the investigator. On the other hand the greater flexibility
makes it more difficult to find the optimal forecasting model.
References
Akaike,
H., 1974, A new look at statistical
model identification,
IEEE Transactions
on Automatic
Control,
AC-19, 716-723.
Amdt,
SW., 1982, The political
economy
of Austria
(American
Enterprise
Institute,
Washington,
DC).
Doan,
T., R. Litterman
and C. Sims, 1984, Forecasting
and conditional
projection
using realistic
prior
distributions,
Econometric
Reviews
3, l-100.
Fair, R., 1979, An analysis of the accuracy
of four macroeconometric
models, Journal
of Political
Economy
87, 701-718.
Fildes, R., 1985, Quantitative
forecasting
- the state of the art: Econometric
models, Journal
of the Operational
Research
Society 36, no. 7, 549-580.
Granger,
C.W.J.,
1969, Investing
causal relations
by econometric
models and cross-spectral
methods,
Econometrica
37,
424-438.
Haggan,
V. and O.B. Oyetunji,
1984, On the selection
of subset autoregressive
time series models,
Journal
of Time Series
Analysis
5, no. 2, 103-113.
Hendry,
D.F., A.R. Pagan and J.D. Sargan,
1984, Dynamic
specification,
in: Z. Griliches
and M.D.
Intriligator,
eds.,
Handbook
of econometrics,
Vol. II (North-Holland,
Amsterdam)
1023-1100.
Jlger, A., 1985, A note on the informational
efficiency
of Austrian
economic
forecasts,
Empirica
12, no. 2, 247-260.
Judge, G.G. and M.E. Brock, 1983, Biased estimation,
in: Z. Griliches
and M.D. Intriligator,
eds., Handbook
of econometrics,
Vol. I (North-Holland,
Amsterdam)
599-649.
Kling, J.L. and D.A. Bessler, 1985, A comparison
of multivariate
procedures
for economic
time series, International
Journal of
Forecasting
1, 5-24.
Kunst, R.M., 1985, Ein Zeitreihenmodell
fur die iistereichische
Wirtschaft
(A time series model for the Austrian
economy),
Research memorandum
no. 211 (IHS, Vienna).
Litterman,
R.B., 1980, A Bayesian
procedure
for forecasting
with vector autoregressions,
Working
paper (Massachusetts
Institute
of Technology,
Cambridge,
MA).
Lucas, R. and T. Sargent,
1981, After Keynesian
macroeconomics,
in: R.E. Lucas, Jr. and T.J. Sargent,
eds., Rational
expectations
and econometric
practice,
Vol. 1 (University
of Minnesota
Press, Minneapolis,
MN) 295-319.
McNees,
SK., 1981, The methodology
of macroeconometric
model comparisons,
in: J. Kmenta
and J.B. Ramsey,
eds.,
Large-scale
macroeconometric
models (North-Holland,
Amsterdam)
397-422.
Nelson, C.R. and C.I. Plosser, 1982, Trends and random walks in macroeconomic
time series, Journal of Monetary
Economics
10,139-162.
Pierce and Haugh, 1977, Causality
in temporal
systems, Journal
of Econometrics
5, 265-293.
Sims, C.A., 1980, Macroeconomics
and reality, Econometrica
48, l-48.
Theil, H., 1966, Applied
economic
forecasting
(North-Holland,
Amsterdam).