ML+

(https://www.machinelearningplus.com/)
Let's DataScience

Vector Autoregression (VAR) – Comprehensive

Guide with Examples in Python
Search here..

by Selva Prabhakaran (https://www.machinelearningplus.com/author/selva86/) |

Recommended

(/#facebook) (/#twitter) (/#whatsapp) (/#linkedin) (/#reddit) Time Series Analysis in Python – A

Comprehensive Guide with Examples
(/#google_bookmarks)
(https://www.machinelearningplus.com/time-
series/time-series-analysis-python/)
(https://www.addtoany.com/share#url=https%3A%2F%2Fwww.machinelearningplus.com%2Ftime-series%2Fvector-autoregression-
examples- Vector Autoregression (VAR) – Comprehensive
Guide with Examples in Python
python%2F&title=Vector%20Autoregression%20(VAR)%20%E2%80%93%20Comprehensive%20Guide%20with%20Examples%20in%20Python)
(https://www.machinelearningplus.com/time-
series/vector-autoregression-examples-python/)
Vector Autoregression (VAR) is a forecasting algorithm that can be used when two or
ARIMA Model – Complete Guide to Time Series
more time series influence each other. That is, the relationship between the time Forecasting in Python
series involved is bi-directional. In this post, we will see the concepts, intuition behind (https://www.machinelearningplus.com/time-
series/arima-model-time-series-forecasting-python/)
VAR models and see a comprehensive and correct method to train and forecast VAR
Augmented Dickey Fuller Test (ADF Test) – Must
models in python using statsmodels . Read Guide
(https://www.machinelearningplus.com/time-
series/augmented-dickey-fuller-test/)

KPSS Test for Stationarity

(https://www.machinelearningplus.com/time-
series/kpss-test-for-stationarity/)

(https://www.ezoic.com/what-is-ezoic/)

Recent Posts report this ad

What is Tokenization in Natural Language

Vector Autoregression (VAR) – Comprehensive Guide with Examples in Python. Photo by Processing (NLP)?
(https://www.machinelearningplus.com/nlp/what-is-
Kyran Low.
tokenization-in-natural-language-processing/)

Matplotlib Plotting Tutorial – Complete overview

of Matplotlib library
(https://www.machinelearningplus.com/plots/matplotlib-
plotting-tutorial/)

How to implement Linear Regression in

TensorFlow
(https://www.machinelearningplus.com/deep-
learning/linear-regression-tensorflow/)
Content Brier Score – How to measure accuracy of
probablistic predictions
. Introduction (https://www.machinelearningplus.com/statistics/brier-
score/)
. Intuition behind VAR Model Formula
. Building a VAR Send
modelMeinPost
Python
Updates!
Enter your email Subscribe
. Import the datasets
 . Visualize the Time Series Modin – How to speedup pandas by changing
one line of code
. Testing Causation using Granger’s Causality Test
(https://www.machinelearningplus.com/python/modin-
. Cointegration Test speedup-pandas/)

. Split the Series into Training and Testing Data Dask – How to handle large dataframes in
. Check for Stationarity and Make the Time Series Stationary python using parallel computing
(https://www.machinelearningplus.com/python/dask-
. How to Select the Order (P) of VAR model tutorial/)
. Train the VAR Model of Selected Order(p) Text Summarization Approaches for NLP –
. Check for Serial Correlation of Residuals (Errors) using Durbin Watson Statistic Practical Guide with Generative Examples
(https://www.machinelearningplus.com/nlp/text-
. How to Forecast VAR model using statsmodels summarization-approaches-nlp-example/)
. Train the VAR Model of Selected Order(p)
Bias Variance Tradeoff – Clearly Explained
. Invert the transformation to get the real forecast (https://www.machinelearningplus.com/machine-
learning/bias-variance-tradeoff/)
. Plot of Forecast vs Actuals
Gradient Boosting – A Concise Introduction from
. Evaluate the Forecasts
Scratch
. Conclusion (https://www.machinelearningplus.com/machine-
learning/gradient-boosting/)

Complete Guide to Natural Language Processing

1. Introduction (NLP) – with Practical Examples

(https://www.machinelearningplus.com/nlp/natural-
language-processing-guide/)
First, what is Vector Autoregression (VAR) and when to use it?
Portfolio Optimization with Python using Efficient
Frontier with Practical Examples
Vector Autoregression (VAR) is a multivariate forecasting algorithm that is used when (https://www.machinelearningplus.com/machine-
learning/portfolio-optimization-python-example/)
two or more time series (https://www.machinelearningplus.com/time-series/time-series-
What does Python Global Interpreter Lock –
analysis-python/) influence each other. (GIL) do?
(https://www.machinelearningplus.com/python/python-
global-interpreter-lock-gil/)
That means, the basic requirements in order to use VAR are:

. You need at least two time series (variables) Tags

. The time series should influence each other.
Chatbot
(https://www.machinelearningplus.com/tag/chatbot/)
Alright. So why is it called ‘Autoregressive’?
Classification
(https://www.machinelearningplus.com/tag/cla
Confidence Interval
(https://www.machinelearningplus.com/tag/confidence-
interval/) dask
(https://www.machinelearningplus.com/tag/dask/)
data.table
(https://www.machinelearningplus.com/tag/data-

table/) Data Manipulation

(https://www.machinelearningplus.com/tag/d
manipulation/) Debugging

(https://www.machinelearningplus.com/tag/debugging/)

Evaluation Metrics
It is considered as an Autoregressive model because, each variable (Time Series) is
(https://www.machinelearningplus.com/tag/evaluation-
modeled as a function of the past values, that is the predictors are nothing but the lags
metrics/) Exercises
(time delayed value) of the series.
(https://www.machinelearningplus.com/tag/exerc
FastText
Ok, so how is VAR different from other Autoregressive models like AR, ARMA or ARIMA? (https://www.machinelearningplus.com/tag/fasttext/)
Gensim
(https://www.machinelearningplus.com/tag/gensim/)
The primary difference is those models are uni-directional, where, the predictors
HuggingFace
influence the Y and not vice-versa. Whereas, Vector Auto Regression (VAR) is bi-
(https://www.machinelearningplus.com/tag/huggingface/)
directional. That is, the variables influence each other.
Julia
(https://www.machinelearningplus.com/tag/ju
Julia Packages
(https://www.machinelearningplus.com/tag/julia-
packages/) LDA
(https://www.machinelearningplus.com/tag/lda/)
 Lemmatization
Cheap Forex VPS (https://www.machinelearningplus.com/tag/lemmatization
Linear Regression
(https://www.machinelearningplus.com/tag/linear-
regression/) Logistic
OPEN
(https://www.machinelearningplus.com/tag/logistic/) Loop
(https://www.machinelearningplus.com/tag/loop/)

Machine Learning
(https://www.machinelearningplus.
learning/) Matplotlib
(https://www.machinelearningplus.com/
NLP
(https://www.machinelearningplus.com
Ad NLTK
(https://www.machinelearningplus.com/tag/nltk/)
Numpy
We will go more in detail in the next section.
(https://www.machinelearningplus.com/tag/numpy/)
P-Value (https://www.machinelearningplus.com/tag/p-
In this article you will gain a clear understanding of: value/) plots
(https://www.machinelearningplus.com/tag/plots/)
Practice Exercise
Intuition behind VAR Model formula
(https://www.machinelearningplus.com/tag/practice-exercise/)
How to check the bi-directional relationship using Granger Causality
Procedure to building a VAR model in Python Python
How to determine the right order of VAR model
Interpreting the results of VAR model
(https://www.machinelearnin
How to generate forecasts to original scale of time series R
(https://www.machinelearningplus.com/tag/
2. Intuition behind VAR Model Formula Regex (https://www.machinelearningplus.com/tag/regex/)

Regression

If you remember in Autoregression models (https://www.machinelearningplus.com/time- (https://www.machinelearningplus.com/tag/regression/)

Residual Analysis
series/arima-model-time-series-forecasting-python/), the time series is modeled as a
(https://www.machinelearningplus.com/tag/residual-analysis/)
linear combination of it’s own lags. That is, the past values of the series are used to
Significance Tests
forecast the current and future. (https://www.machinelearningplus.com/tag/significa
tests/) Soft Cosine Similarity
A typical AR(p) model equation looks something like this: (https://www.machinelearningplus.com/tag/soft-cosine-

spaCy
similarity/)

(https://www.machinelearningplus.com/tag/s
Stationarity
(https://www.machinelearningplus.com/tag/stationarity/)
Statistics
(https://www.machinelearningplus.com/wp-
(https://www.machinelearningplus.com/tag/sta
content/uploads/2019/07/Equation_ARp_Model-min.png)
Tensorflow
(https://www.machinelearningplus.com/tag/tensorflow/)
where α is the intercept, a constant and β1, β2 till βp are the coefficients of the lags of TextBlob
Y till order p. (https://www.machinelearningplus.com/tag/textblob/)
TextSummarization

(https://www.machinelearningplus.com/tag/textsummarization/)
Order ‘p’ means, up to p-lags of Y is used and they are the predictors in the equation.
Text Summarization
The ε_{t} is the error, which is considered as white noise. (https://www.machinelearningplus.com/tag/text-
summarization/) Time Series
(https://www.machinelearningplus.com/tag/tim
series/) Topic Modeling
(https://www.machinelearningplus.com/tag/topic-
modeling/) T Test
(https://www.machinelearningplus.com/tag/t-test/)
Visualization
(https://www.machinelearningplus.com/tag/visualization/
Alright. So, how does a VAR model’s formula look like?

In the VAR model, each variable is modeled as a linear combination of past values of
itself and the past values of other variables in the system. Since you have multiple time
series that influence each other, it is modeled as a system of equations with one
equation per variable (time series).

That is, if you have 5 time series that influence each other, we will have a system of 5
equations.

Well, how is the equation exactly framed?

Let’s suppose, you have two variables (Time series) Y1 and Y2, and you need to forecast
the values of these variables at time (t).

To calculate Y1(t), VAR will use the past values of both Y1 as well as Y2. Likewise, to
compute Y2(t), the past values of both Y1 and Y2 be used.

For example, the system of equations for a VAR(1) model with two time series (variables
`Y1` and `Y2`) is as follows:

(https://www.machinelearningplus.com/wp-
content/uploads/2019/07/Equation_VAR1_Model-min.png)

Where, Y{1,t-1} and Y{2,t-1} are the first lag of time series Y1 and Y2 respectively.

The above equation is referred to as a VAR(1) model, because, each equation is of

order 1, that is, it contains up to one lag of each of the predictors (Y1 and Y2).
Since the Y terms in the equations are interrelated, the Y’s are considered as
endogenous variables, rather than as exogenous predictors.

Likewise, the second order VAR(2) model for two variables would include up to two
lags for each variable (Y1 and Y2).

(https://www.machinelearningplus.com/wp-
content/uploads/2019/07/Equation_VAR2_Model-min.png)

Can you imagine what a second order VAR(2) model with three variables (Y1, Y2 and Y3)
would look like?

(https://www.machinelearningplus.com/wp-
content/uploads/2019/07/Equation_VAR2_Model_with_three_Ys-min.png)

As you increase the number of time series (variables) in the model the system of
equations become larger.

3. Building a VAR model in Python

The procedure to build a VAR model involves the following steps:

. Analyze the time series characteristics

. Test for causation amongst the time series
. Test for stationarity
. Transform the series to make it stationary, if needed
. Find optimal order (p)
. Prepare training and test datasets
. Train the model
. Roll back the transformations, if any.
. Evaluate the model using test set
. Forecast to future
 import pandas as pd
import numpy as np
import matplotlib.pyplot as plt
%matplotlib inline

# Import Statsmodels
from statsmodels.tsa.api import VAR
from statsmodels.tsa.stattools import adfuller
from statsmodels.tools.eval_measures import rmse, aic

4. Import the datasets

For this article let’s use the time series used in Yash P Mehra’s 1994 article: “Wage
Growth and the Inflation Process: An Empirical Approach”.

This dataset has the following 8 quarterly time series:

1. rgnp : Real GNP.

2. pgnp : Potential real GNP.
3. ulc : Unit labor cost.
4. gdfco : Fixed weight deflator for personal consumption expenditure excluding food and en
5. gdf : Fixed weight GNP deflator.
6. gdfim : Fixed weight import deflator.
7. gdfcf : Fixed weight deflator for food in personal consumption expenditure.
8. gdfce : Fixed weight deflator for energy in personal consumption expenditure.

Let’s import the data.

 filepath = 'https://raw.githubusercontent.com/selva86/datasets/master/Raotbl6.csv (http
df = pd.read_csv(filepath, parse_dates=['date'], index_col='date')
print(df.shape) # (123, 8)
df.tail()

(https://www.machinelearningplus.com/wp-
content/uploads/2019/07/Multi_dimensional_time_series_VAR-min.png)

5. Visualize the Time Series

# Plot
fig, axes = plt.subplots(nrows=4, ncols=2, dpi=120, figsize=(10,6))
for i, ax in enumerate(axes.flatten()):
data = df[df.columns[i]]
ax.plot(data, color='red', linewidth=1)
# Decorations
ax.set_title(df.columns[i])
ax.xaxis.set_ticks_position('none')
ax.yaxis.set_ticks_position('none')
ax.spines["top"].set_alpha(0)
ax.tick_params(labelsize=6)

plt.tight_layout();

(https://www.machinelearningplus.com/wp-content/uploads/2019/07/actuals_VAR.png)

Actual Multi Dimensional Time Series for VAR model

Each of the series have a fairly similar trend patterns over the years except for gdfce
and gdfim , where a different pattern is noticed starting in 1980.
Alright, next step in the analysis is to check for causality amongst these series. The
Granger’s Causality test and the Cointegration test can help us with that.

6. Testing Causation using Granger’s Causality Test

The basis behind Vector AutoRegression is that each of the time series in the system
influences each other. That is, you can predict the series with past values of itself along
with other series in the system.

Using Granger’s Causality Test, it’s possible to test this relationship before even building
the model.

So what does Granger’s Causality really test?

Granger’s causality tests the null hypothesis that the coefficients of past values in the
regression equation is zero.

In simpler terms, the past values of time series (X) do not cause the other series (Y). So,
if the p-value obtained from the test is lesser than the significance level of 0.05, then,
you can safely reject the null hypothesis.

The below code implements the Granger’s Causality test for all possible combinations
of the time series in a given dataframe and stores the p-values of each combination in
the output matrix.
 from statsmodels.tsa.stattools import grangercausalitytests
maxlag=12
test = 'ssr_chi2test'
def grangers_causation_matrix(data, variables, test='ssr_chi2test', verbose=False):
"""Check Granger Causality of all possible combinations of the Time series.
The rows are the response variable, columns are predictors. The values in the table
are the P-Values. P-Values lesser than the significance level (0.05), implies
the Null Hypothesis that the coefficients of the corresponding past values is
zero, that is, the X does not cause Y can be rejected.

data : pandas dataframe containing the time series variables

variables : list containing names of the time series variables.
"""
df = pd.DataFrame(np.zeros((len(variables), len(variables))), columns=variables, index=
for c in df.columns:
for r in df.index:
test_result = grangercausalitytests(data[[r, c]], maxlag=maxlag, verbose=False)
p_values = [round(test_result[i+1][0][test][1],4) for i in range(maxlag)]
if verbose: print(f'Y = {r}, X = {c}, P Values = {p_values}')
min_p_value = np.min(p_values)
df.loc[r, c] = min_p_value
df.columns = [var + '_x' for var in variables]
df.index = [var + '_y' for var in variables]
return df

grangers_causation_matrix(df, variables = df.columns)

(https://www.machinelearningplus.com/wp-content/uploads/2019/07/Grangers-Causality-
Test-Results-Matrix-min.png)

So how to read the above output?

The row are the Response (Y) and the columns are the predictor series (X).

For example, if you take the value 0.0003 in (row 1, column 2), it refers to the p-value of
pgnp_x causing rgnp_y . Whereas, the 0.000 in (row 2, column 1) refers to the p-value of
rgnp_y causing pgnp_x .

So, how to interpret the p-values?

If a given p-value is < significance level (0.05), then, the corresponding X series
(column) causes the Y (row).
For example, P-Value of 0.0003 at (row 1, column 2) represents the p-value of the
Grangers Causality test for pgnp_x causing rgnp_y , which is less that the significance
level of 0.05.

So, you can reject the null hypothesis and conclude pgnp_x causes rgnp_y .

Looking at the P-Values in the above table, you can pretty much observe that all the
variables (time series) in the system are interchangeably causing each other.

This makes this system of multi time series a good candidate for using VAR models to
forecast.

Next, let’s do the Cointegration test.

7. Cointegration Test
Cointegration test helps to establish the presence of a statistically significant connection
between two or more time series.

But, what does Cointegration mean?

To understand that, you first need to know what is ‘order of integration’ (d).

Order of integration(d) is nothing but the number of differencing required to make a

non-stationary time series stationary.

Now, when you have two or more time series, and there exists a linear combination of
them that has an order of integration (d) less than that of the individual series, then the
collection of series is said to be cointegrated.

Ok?
When two or more time series are cointegrated, it means they have a long run,
statistically significant relationship.

This is the basic premise on which Vector Autoregression(VAR) models is based on. So,
it’s fairly common to implement the cointegration test before starting to build VAR
models.

Alright, So how to do this test?

Soren Johanssen in his paper (1991) (https://www.jstor.org/stable/2938278?

seq=1#page_scan_tab_contents) devised a procedure to implement the cointegration
test.

It is fairly straightforward to implement in python’s statsmodels , as you can see below.

from statsmodels.tsa.vector_ar.vecm import coint_johansen

def cointegration_test(df, alpha=0.05):

"""Perform Johanson's Cointegration Test and Report Summary"""
out = coint_johansen(df,-1,5)
d = {'0.90':0, '0.95':1, '0.99':2}
traces = out.lr1
cvts = out.cvt[:, d[str(1-alpha)]]
def adjust(val, length= 6): return str(val).ljust(length)

# Summary
print('Name :: Test Stat > C(95%) => Signif \n', '--'*20)
for col, trace, cvt in zip(df.columns, traces, cvts):
print(adjust(col), ':: ', adjust(round(trace,2), 9), ">", adjust(cvt, 8), ' => ' ,

cointegration_test(df)
Results:

Name :: Test Stat > C(95%) => Signif

----------------------------------------
rgnp :: 248.0 > 143.6691 => True
pgnp :: 183.12 > 111.7797 => True
ulc :: 130.01 > 83.9383 => True
gdfco :: 85.28 > 60.0627 => True
gdf :: 55.05 > 40.1749 => True
gdfim :: 31.59 > 24.2761 => True
gdfcf :: 14.06 > 12.3212 => True
gdfce :: 0.45 > 4.1296 => False

8. Split the Series into Training and Testing Data

Splitting the dataset into training and test data.

The VAR model will be fitted on df_train and then used to forecast the next 4
observations. These forecasts will be compared against the actuals present in test data.

To do the comparisons, we will use multiple forecast accuracy metrics, as seen later in
this article.

nobs = 4
df_train, df_test = df[0:-nobs], df[-nobs:]

# Check size
print(df_train.shape) # (119, 8)
print(df_test.shape) # (4, 8)

9. Check for Stationarity and Make the Time Series

Stationary
Since the VAR model requires the time series you want to forecast to be stationary, it is
customary to check all the time series in the system for stationarity.

Just to refresh, a stationary time series is one whose characteristics like mean and
variance does not change over time.

So, how to test for stationarity?

There is a suite of tests called unit-root tests. The popular ones are:

. Augmented Dickey-Fuller Test (ADF Test)

(https://www.machinelearningplus.com/time-series/augmented-dickey-fuller-test/)
. KPSS test (https://www.machinelearningplus.com/time-series/kpss-test-for-
stationarity/)
. Philip-Perron test

Let’s use the ADF test for our purpose.

By the way, if a series is found to be non-stationary, you make it stationary by
differencing the series once and repeat the test again until it becomes stationary.

Since, differencing reduces the length of the series by 1 and since all the time series has
to be of the same length, you need to difference all the series in the system if you
choose to difference at all.

Got it?

Let’s implement the ADF Test.

Open a Free trial

account.
Never trade alone again. Try the
trading room today for Free. Trade
with the best mentors.
FS Trading Room

Sign Up

First, we implement a nice function ( adfuller_test() ) that writes out the results of the ADF
test for any given time series and implement this function on each series one-by-one.
 def adfuller_test(series, signif=0.05, name='', verbose=False):
"""Perform ADFuller to test for Stationarity of given series and print report"""
r = adfuller(series, autolag='AIC')
output = {'test_statistic':round(r[0], 4), 'pvalue':round(r[1], 4), 'n_lags':round(r[2]
p_value = output['pvalue']
def adjust(val, length= 6): return str(val).ljust(length)

# Print Summary
print(f' Augmented Dickey-Fuller Test on "{name}"', "\n ", '-'*47)
print(f' Null Hypothesis: Data has unit root. Non-Stationary.')
print(f' Significance Level = {signif}')
print(f' Test Statistic = {output["test_statistic"]}')
print(f' No. Lags Chosen = {output["n_lags"]}')

for key,val in r[4].items():

print(f' Critical value {adjust(key)} = {round(val, 3)}')

if p_value <= signif:

print(f" => P-Value = {p_value}. Rejecting Null Hypothesis.")
print(f" => Series is Stationary.")
else:
print(f" => P-Value = {p_value}. Weak evidence to reject the Null Hypothesis.")
print(f" => Series is Non-Stationary.")

Call the adfuller_test() on each series.

# ADF Test on each column

for name, column in df_train.iteritems():
adfuller_test(column, name=column.name)
print('\n')

Results:
 Augmented Dickey-Fuller Test on "rgnp"
-----------------------------------------------
Null Hypothesis: Data has unit root. Non-Stationary.
Significance Level = 0.05
Test Statistic = 0.5428
No. Lags Chosen = 2
Critical value 1% = -3.488
Critical value 5% = -2.887
Critical value 10% = -2.58
=> P-Value = 0.9861. Weak evidence to reject the Null Hypothesis.
=> Series is Non-Stationary.

Augmented Dickey-Fuller Test on "pgnp"

-----------------------------------------------
Null Hypothesis: Data has unit root. Non-Stationary.
Significance Level = 0.05
Test Statistic = 1.1556
No. Lags Chosen = 1
Critical value 1% = -3.488
Critical value 5% = -2.887
Critical value 10% = -2.58
=> P-Value = 0.9957. Weak evidence to reject the Null Hypothesis.
=> Series is Non-Stationary.

Augmented Dickey-Fuller Test on "ulc"

-----------------------------------------------
Null Hypothesis: Data has unit root. Non-Stationary.
Significance Level = 0.05
Test Statistic = 1.2474
No. Lags Chosen = 2
Critical value 1% = -3.488
Critical value 5% = -2.887
Critical value 10% = -2.58
=> P-Value = 0.9963. Weak evidence to reject the Null Hypothesis.
=> Series is Non-Stationary.

Augmented Dickey-Fuller Test on "gdfco"

-----------------------------------------------
Null Hypothesis: Data has unit root. Non-Stationary.
Significance Level = 0.05
Test Statistic = 1.1954
No. Lags Chosen = 3
Critical value 1% = -3.489
Critical value 5% = -2.887
Critical value 10% = -2.58
=> P-Value = 0.996. Weak evidence to reject the Null Hypothesis.
=> Series is Non-Stationary.

Augmented Dickey-Fuller Test on "gdf"

-----------------------------------------------
Null Hypothesis: Data has unit root. Non-Stationary.
Significance Level = 0.05
Test Statistic = 1.676
No. Lags Chosen = 7
 Critical value 1% = -3.491
Critical value 5% = -2.888
Critical value 10% = -2.581
=> P-Value = 0.9981. Weak evidence to reject the Null Hypothesis.
=> Series is Non-Stationary.

Augmented Dickey-Fuller Test on "gdfim"

-----------------------------------------------
Null Hypothesis: Data has unit root. Non-Stationary.
Significance Level = 0.05
Test Statistic = -0.0799
No. Lags Chosen = 1
Critical value 1% = -3.488
Critical value 5% = -2.887
Critical value 10% = -2.58
=> P-Value = 0.9514. Weak evidence to reject the Null Hypothesis.
=> Series is Non-Stationary.

Augmented Dickey-Fuller Test on "gdfcf"

-----------------------------------------------
Null Hypothesis: Data has unit root. Non-Stationary.
Significance Level = 0.05
Test Statistic = 1.4395
No. Lags Chosen = 8
Critical value 1% = -3.491
Critical value 5% = -2.888
Critical value 10% = -2.581
=> P-Value = 0.9973. Weak evidence to reject the Null Hypothesis.
=> Series is Non-Stationary.

Augmented Dickey-Fuller Test on "gdfce"

-----------------------------------------------
Null Hypothesis: Data has unit root. Non-Stationary.
Significance Level = 0.05
Test Statistic = -0.3402
No. Lags Chosen = 8
Critical value 1% = -3.491
Critical value 5% = -2.888
Critical value 10% = -2.581
=> P-Value = 0.9196. Weak evidence to reject the Null Hypothesis.
=> Series is Non-Stationary.

The ADF test confirms none of the time series is stationary. Let’s difference all of them
once and check again.

# 1st difference
df_differenced = df_train.diff().dropna()

Re-run ADF test on each differenced series.

 Open a Free trial
account.
Never trade alone again. Try the
trading room today for Free. Trade
with the best mentors.
FS Trading Room

Sign Up

# ADF Test on each column of 1st Differences Dataframe

for name, column in df_differenced.iteritems():
adfuller_test(column, name=column.name)
print('\n')
 Augmented Dickey-Fuller Test on "rgnp"
-----------------------------------------------
Null Hypothesis: Data has unit root. Non-Stationary.
Significance Level = 0.05
Test Statistic = -5.3448
No. Lags Chosen = 1
Critical value 1% = -3.488
Critical value 5% = -2.887
Critical value 10% = -2.58
=> P-Value = 0.0. Rejecting Null Hypothesis.
=> Series is Stationary.

Augmented Dickey-Fuller Test on "pgnp"

-----------------------------------------------
Null Hypothesis: Data has unit root. Non-Stationary.
Significance Level = 0.05
Test Statistic = -1.8282
No. Lags Chosen = 0
Critical value 1% = -3.488
Critical value 5% = -2.887
Critical value 10% = -2.58
=> P-Value = 0.3666. Weak evidence to reject the Null Hypothesis.
=> Series is Non-Stationary.

Augmented Dickey-Fuller Test on "ulc"

-----------------------------------------------
Null Hypothesis: Data has unit root. Non-Stationary.
Significance Level = 0.05
Test Statistic = -3.4658
No. Lags Chosen = 1
Critical value 1% = -3.488
Critical value 5% = -2.887
Critical value 10% = -2.58
=> P-Value = 0.0089. Rejecting Null Hypothesis.
=> Series is Stationary.

Augmented Dickey-Fuller Test on "gdfco"

-----------------------------------------------
Null Hypothesis: Data has unit root. Non-Stationary.
Significance Level = 0.05
Test Statistic = -1.4385
No. Lags Chosen = 2
Critical value 1% = -3.489
Critical value 5% = -2.887
Critical value 10% = -2.58
=> P-Value = 0.5637. Weak evidence to reject the Null Hypothesis.
=> Series is Non-Stationary.

Augmented Dickey-Fuller Test on "gdf"

-----------------------------------------------
Null Hypothesis: Data has unit root. Non-Stationary.
Significance Level = 0.05
Test Statistic = -1.1289
No. Lags Chosen = 2
 Critical value 1% = -3.489
Critical value 5% = -2.887
Critical value 10% = -2.58
=> P-Value = 0.7034. Weak evidence to reject the Null Hypothesis.
=> Series is Non-Stationary.

Augmented Dickey-Fuller Test on "gdfim"

-----------------------------------------------
Null Hypothesis: Data has unit root. Non-Stationary.
Significance Level = 0.05
Test Statistic = -4.1256
No. Lags Chosen = 0
Critical value 1% = -3.488
Critical value 5% = -2.887
Critical value 10% = -2.58
=> P-Value = 0.0009. Rejecting Null Hypothesis.
=> Series is Stationary.

Augmented Dickey-Fuller Test on "gdfcf"

-----------------------------------------------
Null Hypothesis: Data has unit root. Non-Stationary.
Significance Level = 0.05
Test Statistic = -2.0545
No. Lags Chosen = 7
Critical value 1% = -3.491
Critical value 5% = -2.888
Critical value 10% = -2.581
=> P-Value = 0.2632. Weak evidence to reject the Null Hypothesis.
=> Series is Non-Stationary.

Augmented Dickey-Fuller Test on "gdfce"

-----------------------------------------------
Null Hypothesis: Data has unit root. Non-Stationary.
Significance Level = 0.05
Test Statistic = -3.1543
No. Lags Chosen = 7
Critical value 1% = -3.491
Critical value 5% = -2.888
Critical value 10% = -2.581
=> P-Value = 0.0228. Rejecting Null Hypothesis.
=> Series is Stationary.

After the first difference, Real Wages (Manufacturing) is still not stationary. It’s critical
value is between 5% and 10% significance level.

All of the series in the VAR model should have the same number of observations.

So, we are left with one of two choices.

That is, either proceed with 1st differenced series or difference all the series one more
time.
 # Second Differencing
df_differenced = df_differenced.diff().dropna()

Re-run ADF test again on each second differenced series.

# ADF Test on each column of 2nd Differences Dataframe

for name, column in df_differenced.iteritems():
adfuller_test(column, name=column.name)
print('\n')

Results:
 Augmented Dickey-Fuller Test on "rgnp"
-----------------------------------------------
Null Hypothesis: Data has unit root. Non-Stationary.
Significance Level = 0.05
Test Statistic = -9.0123
No. Lags Chosen = 2
Critical value 1% = -3.489
Critical value 5% = -2.887
Critical value 10% = -2.58
=> P-Value = 0.0. Rejecting Null Hypothesis.
=> Series is Stationary.

Augmented Dickey-Fuller Test on "pgnp"

-----------------------------------------------
Null Hypothesis: Data has unit root. Non-Stationary.
Significance Level = 0.05
Test Statistic = -10.9813
No. Lags Chosen = 0
Critical value 1% = -3.488
Critical value 5% = -2.887
Critical value 10% = -2.58
=> P-Value = 0.0. Rejecting Null Hypothesis.
=> Series is Stationary.

Augmented Dickey-Fuller Test on "ulc"

-----------------------------------------------
Null Hypothesis: Data has unit root. Non-Stationary.
Significance Level = 0.05
Test Statistic = -8.769
No. Lags Chosen = 2
Critical value 1% = -3.489
Critical value 5% = -2.887
Critical value 10% = -2.58
=> P-Value = 0.0. Rejecting Null Hypothesis.
=> Series is Stationary.

Augmented Dickey-Fuller Test on "gdfco"

-----------------------------------------------
Null Hypothesis: Data has unit root. Non-Stationary.
Significance Level = 0.05
Test Statistic = -7.9102
No. Lags Chosen = 3
Critical value 1% = -3.49
Critical value 5% = -2.887
Critical value 10% = -2.581
=> P-Value = 0.0. Rejecting Null Hypothesis.
=> Series is Stationary.

Augmented Dickey-Fuller Test on "gdf"

-----------------------------------------------
Null Hypothesis: Data has unit root. Non-Stationary.
Significance Level = 0.05
Test Statistic = -10.0351
No. Lags Chosen = 1
 Critical value 1% = -3.489
Critical value 5% = -2.887
Critical value 10% = -2.58
=> P-Value = 0.0. Rejecting Null Hypothesis.
=> Series is Stationary.

Augmented Dickey-Fuller Test on "gdfim"

-----------------------------------------------
Null Hypothesis: Data has unit root. Non-Stationary.
Significance Level = 0.05
Test Statistic = -9.4059
No. Lags Chosen = 1
Critical value 1% = -3.489
Critical value 5% = -2.887
Critical value 10% = -2.58
=> P-Value = 0.0. Rejecting Null Hypothesis.
=> Series is Stationary.

Augmented Dickey-Fuller Test on "gdfcf"

-----------------------------------------------
Null Hypothesis: Data has unit root. Non-Stationary.
Significance Level = 0.05
Test Statistic = -6.922
No. Lags Chosen = 5
Critical value 1% = -3.491
Critical value 5% = -2.888
Critical value 10% = -2.581
=> P-Value = 0.0. Rejecting Null Hypothesis.
=> Series is Stationary.

Augmented Dickey-Fuller Test on "gdfce"

-----------------------------------------------
Null Hypothesis: Data has unit root. Non-Stationary.
Significance Level = 0.05
Test Statistic = -5.1732
No. Lags Chosen = 8
Critical value 1% = -3.492
Critical value 5% = -2.889
Critical value 10% = -2.581
=> P-Value = 0.0. Rejecting Null Hypothesis.
=> Series is Stationary.

All the series are now stationary.

Join the best trading

room.
Never trade alone again. Try the
trading room today for Free. Trade
with the best mentors.
FS Trading Room

Sign Up
Let’s prepare the training and test datasets.

10. How to Select the Order (P) of VAR model

To select the right order of the VAR model, we iteratively fit increasing orders of VAR
model and pick the order that gives a model with least AIC.

Though the usual practice is to look at the AIC, you can also check other best fit
comparison estimates of BIC , FPE and HQIC .

model = VAR(df_differenced)
for i in [1,2,3,4,5,6,7,8,9]:
result = model.fit(i)
print('Lag Order =', i)
print('AIC : ', result.aic)
print('BIC : ', result.bic)
print('FPE : ', result.fpe)
print('HQIC: ', result.hqic, '\n')

Results:
 Lag Order = 1
AIC : -1.3679402315450664
BIC : 0.3411847146588838
FPE : 0.2552682517347198
HQIC: -0.6741331335699554

Lag Order = 2
AIC : -1.621237394447824
BIC : 1.6249432095295848
FPE : 0.2011349437137139
HQIC: -0.3036288826795923

Lag Order = 3
AIC : -1.7658008387012791
BIC : 3.0345473163767833
FPE : 0.18125103746164364
HQIC: 0.18239143783963296

Lag Order = 4
AIC : -2.000735164470318
BIC : 4.3712151376540875
FPE : 0.15556966521481097
HQIC: 0.5849359332771069

Lag Order = 5
AIC : -1.9619535608363954
BIC : 5.9993645622420955
FPE : 0.18692794389114886
HQIC: 1.268206331178333

Lag Order = 6
AIC : -2.3303386524829053
BIC : 7.2384526890885805
FPE : 0.16380374017443664
HQIC: 1.5514371669548073

Lag Order = 7
AIC : -2.592331352347129
BIC : 8.602387254937796
FPE : 0.1823868583715414
HQIC: 1.9483069621146551

Lag Order = 8
AIC : -3.317261976458205
BIC : 9.52219581032303
FPE : 0.15573163248209088
HQIC: 1.8896071386220985

Lag Order = 9
AIC : -4.804763125958631
BIC : 9.698613139231597
FPE : 0.08421466682671915
HQIC: 1.0758291640834052

In the above output, the AIC drops to lowest at lag 4, then increases at lag 5 and then
continuously drops further.
Let’s go with the lag 4 model.

An alternate method to choose the order(p) of the VAR models is to use the
model.select_order(maxlags) method.

The selected order(p) is the order that gives the lowest ‘AIC’, ‘BIC’, ‘FPE’ and ‘HQIC’
scores.

x = model.select_order(maxlags=12)
x.summary()
(https://www.machinelearningplus.com/wp-
content/uploads/2019/07/VAR_Order_Selection_Table-min.png)

According to FPE and HQIC, the optimal lag is observed at a lag order of 3.

I, however, don’t have an explanation for why the observed AIC and BIC values differ
when using result.aic versus as seen using model.select_order() .

Since the explicitly computed AIC is the lowest at lag 4, I choose the selected order as
4.

11. Train the VAR Model of Selected Order(p)

 model_fitted = model.fit(4)
model_fitted.summary()

Results:
 Summary of Regression Results
==================================
Model: VAR
Method: OLS
Date: Sat, 18, May, 2019
Time: 11:35:15
--------------------------------------------------------------------
No. of Equations: 8.00000 BIC: 4.37122
Nobs: 113.000 HQIC: 0.584936
Log likelihood: -905.679 FPE: 0.155570
AIC: -2.00074 Det(Omega_mle): 0.0200322
--------------------------------------------------------------------
Results for equation rgnp
===========================================================================
coefficient std. error t-stat prob
---------------------------------------------------------------------------
const 2.430021 2.677505 0.908 0.364
L1.rgnp -0.750066 0.159023 -4.717 0.000
L1.pgnp -0.095621 4.938865 -0.019 0.985
L1.ulc -6.213996 4.637452 -1.340 0.180
L1.gdfco -7.414768 10.184884 -0.728 0.467
L1.gdf -24.864063 20.071245 -1.239 0.215
L1.gdfim 1.082913 4.309034 0.251 0.802
L1.gdfcf 16.327252 5.892522 2.771 0.006
L1.gdfce 0.910522 2.476361 0.368 0.713
L2.rgnp -0.568178 0.163971 -3.465 0.001
L2.pgnp -1.156201 4.931931 -0.234 0.815
L2.ulc -11.157111 5.381825 -2.073 0.038
L2.gdfco 3.012518 12.928317 0.233 0.816
L2.gdf -18.143523 24.090598 -0.753 0.451
L2.gdfim -4.438115 4.410654 -1.006 0.314
L2.gdfcf 13.468228 7.279772 1.850 0.064
L2.gdfce 5.130419 2.805310 1.829 0.067
L3.rgnp -0.514985 0.152724 -3.372 0.001
L3.pgnp -11.483607 5.392037 -2.130 0.033
L3.ulc -14.195308 5.188718 -2.736 0.006
L3.gdfco -10.154967 13.105508 -0.775 0.438
L3.gdf -15.438858 21.610822 -0.714 0.475
L3.gdfim -6.405290 4.292790 -1.492 0.136
L3.gdfcf 9.217402 7.081652 1.302 0.193
L3.gdfce 5.279941 2.833925 1.863 0.062
L4.rgnp -0.166878 0.138786 -1.202 0.229
L4.pgnp 5.329900 5.795837 0.920 0.358
L4.ulc -4.834548 5.259608 -0.919 0.358
L4.gdfco 10.841602 10.526530 1.030 0.303
L4.gdf -17.651510 18.746673 -0.942 0.346
L4.gdfim -1.971233 4.029415 -0.489 0.625
L4.gdfcf 0.617824 5.842684 0.106 0.916
L4.gdfce -2.977187 2.594251 -1.148 0.251
===========================================================================

Results for equation pgnp

===========================================================================
coefficient std. error t-stat prob
---------------------------------------------------------------------------
const 0.094556 0.063491 1.489 0.136
L1.rgnp -0.004231 0.003771 -1.122 0.262
L1.pgnp 0.082204 0.117114 0.702 0.483
 L1.ulc -0.097769 0.109966 -0.889 0.374

(... TRUNCATED because of long output....)

(... TRUNCATED because of long output....)
(... TRUNCATED because of long output....)

Correlation matrix of residuals

rgnp pgnp ulc gdfco gdf gdfim gdfcf gdfce
rgnp 1.000000 0.248342 -0.668492 -0.160133 -0.047777 0.084925 0.009962 0.205557
pgnp 0.248342 1.000000 -0.148392 -0.167766 -0.134896 0.007830 -0.169435 0.032134
ulc -0.668492 -0.148392 1.000000 0.268127 0.327761 0.171497 0.135410 -0.026037
gdfco -0.160133 -0.167766 0.268127 1.000000 0.303563 0.232997 -0.035042 0.184834
gdf -0.047777 -0.134896 0.327761 0.303563 1.000000 0.196670 0.446012 0.309277
gdfim 0.084925 0.007830 0.171497 0.232997 0.196670 1.000000 -0.089852 0.707809
gdfcf 0.009962 -0.169435 0.135410 -0.035042 0.446012 -0.089852 1.000000 -0.197099
gdfce 0.205557 0.032134 -0.026037 0.184834 0.309277 0.707809 -0.197099 1.000000

12. Check for Serial Correlation of Residuals (Errors)

using Durbin Watson Statistic
Serial correlation of residuals is used to check if there is any leftover pattern in the
residuals (errors).

What does this mean to us?

If there is any correlation left in the residuals, then, there is some pattern in the time
series that is still left to be explained by the model. In that case, the typical course of
action is to either increase the order of the model or induce more predictors into the
system or look for a different algorithm to model the time series.

So, checking for serial correlation is to ensure that the model is sufficiently able to
explain the variances and patterns in the time series.

Alright, coming back to topic.

A common way of checking for serial correlation of errors can be measured using the
Durbin Watson’s Statistic.

(https://www.machinelearningplus.com/wp-
content/uploads/2019/07/Durbin_Watson_Statistic_Formula-min.png)
The value of this statistic can vary between 0 and 4. The closer it is to the value 2, then
there is no significant serial correlation. The closer to 0, there is a positive serial
correlation, and the closer it is to 4 implies negative serial correlation.

from statsmodels.stats.stattools import durbin_watson

out = durbin_watson(model_fitted.resid)

for col, val in zip(df.columns, out):

print(adjust(col), ':', round(val, 2))

Results:

rgnp : 2.09
pgnp : 2.02
ulc : 2.17
gdfco : 2.05
gdf : 2.25
gdfim : 1.99
gdfcf : 2.2
gdfce : 2.17

The serial correlation seems quite alright. Let’s proceed with the forecast.

13. How to Forecast VAR model using statsmodels

In order to forecast, the VAR model expects up to the lag order number of
observations from the past data.

This is because, the terms in the VAR model are essentially the lags of the various time
series in the dataset, so you need to provide it as many of the previous values as
indicated by the lag order used by the model.

# Get the lag order

lag_order = model_fitted.k_ar
print(lag_order) #> 4

# Input data for forecasting

forecast_input = df_differenced.values[-lag_order:]
forecast_input
 4

array([[ 13.5, 0.1, 1.4, 0.1, 0.1, -0.1, 0.4, -2. ],

[-23.6, 0.2, -2. , -0.5, -0.1, -0.2, -0.3, -1.2],
[ -3.3, 0.1, 3.1, 0.5, 0.3, 0.4, 0.9, 2.2],
[ -3.9, 0.2, -2.1, -0.4, 0.2, -1.5, 0.9, -0.3]])

Let’s forecast.

# Forecast
fc = model_fitted.forecast(y=forecast_input, steps=nobs)
df_forecast = pd.DataFrame(fc, index=df.index[-nobs:], columns=df.columns + '_2d')
df_forecast

(https://www.machinelearningplus.com/wp-
content/uploads/2019/07/VAR_Forecasts_raw.png)

The forecasts are generated but it is on the scale of the training data used by the
model. So, to bring it back up to its original scale, you need to de-difference it as many
times you had differenced the original input data.

In this case it is two times.

14. Invert the transformation to get the real forecast

def invert_transformation(df_train, df_forecast, second_diff=False):
"""Revert back the differencing to get the forecast to original scale."""
df_fc = df_forecast.copy()
columns = df_train.columns
for col in columns:
# Roll back 2nd Diff
if second_diff:
df_fc[str(col)+'_1d'] = (df_train[col].iloc[-1]-df_train[col].iloc[-2]) + df_fc
# Roll back 1st Diff
df_fc[str(col)+'_forecast'] = df_train[col].iloc[-1] + df_fc[str(col)+'_1d'].cumsum
return df_fc
 df_results = invert_transformation(train, df_forecast, second_diff=True)
df_results.loc[:, ['rgnp_forecast', 'pgnp_forecast', 'ulc_forecast', 'gdfco_forecast',
'gdf_forecast', 'gdfim_forecast', 'gdfcf_forecast', 'gdfce_forecast']]

(https://www.machinelearningplus.com/wp-content/uploads/2019/07/VAR-Forecasts-
min.png)

Join the best trading

room.
Never trade alone again. Try the
trading room today for Free. Trade
with the best mentors.
FS Trading Room

Sign Up

The forecasts are back to the original scale. Let’s plot the forecasts against the actuals
from test data.

15. Plot of Forecast vs Actuals

fig, axes = plt.subplots(nrows=int(len(df.columns)/2), ncols=2, dpi=150, figsize=(10,10))
for i, (col,ax) in enumerate(zip(df.columns, axes.flatten())):
df_results[col+'_forecast'].plot(legend=True, ax=ax).autoscale(axis='x',tight=True)
df_test[col][-nobs:].plot(legend=True, ax=ax);
ax.set_title(col + ": Forecast vs Actuals")
ax.xaxis.set_ticks_position('none')
ax.yaxis.set_ticks_position('none')
ax.spines["top"].set_alpha(0)
ax.tick_params(labelsize=6)

plt.tight_layout();
(https://www.machinelearningplus.com/wp-
content/uploads/2019/07/forecast_vs_actuals_VAR.png)

Forecast vs Actuals comparison of VAR model

16. Evaluate the Forecasts

To evaluate the forecasts, let’s compute a comprehensive set of metrics, namely, the
MAPE, ME, MAE, MPE, RMSE, corr and minmax.
from statsmodels.tsa.stattools import acf
def forecast_accuracy(forecast, actual):
mape = np.mean(np.abs(forecast - actual)/np.abs(actual)) # MAPE
me = np.mean(forecast - actual) # ME
mae = np.mean(np.abs(forecast - actual)) # MAE
mpe = np.mean((forecast - actual)/actual) # MPE
rmse = np.mean((forecast - actual)**2)**.5 # RMSE
corr = np.corrcoef(forecast, actual)[0,1] # corr
mins = np.amin(np.hstack([forecast[:,None],
actual[:,None]]), axis=1)
maxs = np.amax(np.hstack([forecast[:,None],
actual[:,None]]), axis=1)
minmax = 1 - np.mean(mins/maxs) # minmax
return({'mape':mape, 'me':me, 'mae': mae,
'mpe': mpe, 'rmse':rmse, 'corr':corr, 'minmax':minmax})

print('Forecast Accuracy of: rgnp')

accuracy_prod = forecast_accuracy(df_results['rgnp_forecast'].values, df_test['rgnp'])
for k, v in accuracy_prod.items():
print(adjust(k), ': ', round(v,4))

print('\nForecast Accuracy of: pgnp')

accuracy_prod = forecast_accuracy(df_results['pgnp_forecast'].values, df_test['pgnp'])
for k, v in accuracy_prod.items():
print(adjust(k), ': ', round(v,4))

print('\nForecast Accuracy of: ulc')

accuracy_prod = forecast_accuracy(df_results['ulc_forecast'].values, df_test['ulc'])
for k, v in accuracy_prod.items():
print(adjust(k), ': ', round(v,4))

print('\nForecast Accuracy of: gdfco')

accuracy_prod = forecast_accuracy(df_results['gdfco_forecast'].values, df_test['gdfco'])
for k, v in accuracy_prod.items():
print(adjust(k), ': ', round(v,4))

print('\nForecast Accuracy of: gdf')

accuracy_prod = forecast_accuracy(df_results['gdf_forecast'].values, df_test['gdf'])
for k, v in accuracy_prod.items():
print(adjust(k), ': ', round(v,4))

print('\nForecast Accuracy of: gdfim')

accuracy_prod = forecast_accuracy(df_results['gdfim_forecast'].values, df_test['gdfim'])
for k, v in accuracy_prod.items():
print(adjust(k), ': ', round(v,4))

print('\nForecast Accuracy of: gdfcf')

accuracy_prod = forecast_accuracy(df_results['gdfcf_forecast'].values, df_test['gdfcf'])
for k, v in accuracy_prod.items():
print(adjust(k), ': ', round(v,4))

print('\nForecast Accuracy of: gdfce')

accuracy_prod = forecast_accuracy(df_results['gdfce_forecast'].values, df_test['gdfce'])
for k, v in accuracy_prod.items():
print(adjust(k), ': ', round(v,4))
Forecast Accuracy of: rgnp
mape : 0.0192
me : 79.1031
mae : 79.1031
mpe : 0.0192
rmse : 82.0245
corr : 0.9849
minmax : 0.0188

Forecast Accuracy of: pgnp

mape : 0.0005
me : 2.0432
mae : 2.0432
mpe : 0.0005
rmse : 2.146
corr : 1.0
minmax : 0.0005

Forecast Accuracy of: ulc

mape : 0.0081
me : -1.4947
mae : 1.4947
mpe : -0.0081
rmse : 1.6856
corr : 0.963
minmax : 0.0081

Forecast Accuracy of: gdfco

mape : 0.0033
me : 0.0007
mae : 0.4384
mpe : 0.0
rmse : 0.5169
corr : 0.9407
minmax : 0.0032

Forecast Accuracy of: gdf

mape : 0.0023
me : 0.2554
mae : 0.29
mpe : 0.002
rmse : 0.3392
corr : 0.9905
minmax : 0.0022

Forecast Accuracy of: gdfim

mape : 0.0097
me : -0.4166
mae : 1.06
mpe : -0.0038
rmse : 1.0826
corr : 0.807
minmax : 0.0096

Forecast Accuracy of: gdfcf

mape : 0.0036
me : -0.0271
mae : 0.4604
 mpe : -0.0002
rmse : 0.5286
corr : 0.9713
minmax : 0.0036

Forecast Accuracy of: gdfce

mape : 0.0177
me : 0.2577
mae : 1.72
mpe : 0.0031
rmse : 2.034
corr : 0.764
minmax : 0.0175

17. Conclusion
In this article we covered VAR from scratch beginning from the intuition behind it,
interpreting the formula, causality tests, finding the optimal order of the VAR model,
preparing the data for forecasting, build the model, checking for serial autocorrelation,
inverting the transform to get the actual forecasts, plotting the results and computing
the accuracy metrics.

Hope you enjoyed reading this as much as I did writing it. I will see you in the next
one.

ALSO ON MACHINELEARNINGPLUS.COM

A Comprehensive KPSS Test

Guide with … How to Train Text … Top 50 matplotlib … Stationari

2 years ago • 25 comments 8 months ago • 1 comment 2 years ago • 25 comments 2 years ago • 2
Time series is a sequence of Text Classification is the A compilation of the Top 50 KPSS test is a
observations recorded at process of categorizing texts. matplotlib plots most useful to check for st
regular time intervals. … SpaCy makes text … in data analysis and … series around

What do you think?

109 Responses

Upvote Love

50 Comments machinelearningplus.com 🔒 Disqus' Privacy Policy 

1 Login

 Recommend 4 t Tweet f Share Sort by Newest

Join the discussion…

LOG IN WITH
OR SIGN UP WITH DISQUS ?

Name

WL Pretorius • 3 months ago

Hi, nice article. How would I forecast 1 step ahead in time? I mean not comparing forecast of test
set, i.e. forecasting one new value in the future?
△ ▽ • Reply • Share ›

WL Pretorius > WL Pretorius • 3 months ago

1 quarter ahead i.e. 1989-10-01?
△ ▽ • Reply • Share ›
 Reply Share ›

ezer elbaz • 3 months ago • edited

Thanks a lot for this helpful and detailed article :)
one questions\help please if possible
your help please with the code - if i only need to do 1 differencing, how should i change the code for
the:
def invert_transformation?
△ ▽ • Reply • Share ›

Ary • 6 months ago

Thanks for the post.
I have a question, How do you check for Serial Correlation of Residuals (Errors) using Lagrange
Multiplier Tests?
1△ ▽ • Reply • Share ›

Paul Matthew • 6 months ago • edited

Great overview, thanks. Now that there is a model built, how do we forecast the next period? This
one only goes up until the end of the current dataset.
1△ ▽ • Reply • Share ›

Sam • 7 months ago

Thanks for sharing. It-s a wonderful job. I just wonder how to save the figures.
△ ▽ • Reply • Share ›

W_Jing • 9 months ago

Thanks for sharing. Just wondering if you know how to get confidence interval of all the predicted
values.
△ ▽ • Reply • Share ›

vishal sharma • 9 months ago

no explanation or use of cointegration test
△ ▽ • Reply • Share ›

des c • 9 months ago

nice. I tried to run diff onced only, to see the error rate comparison with running diff twice, so i try
to run df_results = invert_transformation(df_train, df_forecast, second_diff=False), i get
KeyError: 'rgnp_1d' because the column isn't there, did i miss out anything?
△ ▽ • Reply • Share ›

Syed Rizvi • 10 months ago

Very helpful and detailed article ! Just wanted to ask - Why you are not doing
normalization/standardization here ? If there is need for data normalization when do we need to do
that ?
△ ▽ • Reply • Share ›

Selva Prabhakaran Mod > Syed Rizvi • 10 months ago

Don't see harm in dong normalization, however by design it is not necessary. The weights
get adjusted accordingly, so, if the scale of one of the series is too high, you still get fairly
similar predictions
△ ▽ • Reply • Share ›

Michael Spacey • a year ago • edited

this is an incredibly detailed article. are you on Twitter? would love to follow. thanks!
△ ▽ • Reply • Share ›

Selva Prabhakaran Mod > Michael Spacey • a year ago

I run the @R_Programming handle.. Thanks for asking

△ ▽ • Reply • Share ›

Anatolijus • a year ago

Hi, thank you for the very interesting article.I see from the article that Granger’s Causality test has
been made before checking of stationarity of time series. Can we rely on the results of the such test?
△ ▽ • Reply • Share ›

Selva Prabhakaran Mod > Anatolijus • a year ago

The stationarity must be checked first. I overlooked this at the time of writing I'm afraid.
Thanks for noticing this!
△ ▽ • Reply • Share ›

Chris • a year ago

If one wanted to test for seasonality how would one go about doing that?
△ ▽ • Reply • Share ›

Selva Prabhakaran Mod > Chris • a year ago

Approved
△ ▽ • Reply • Share ›

Ary • a year ago • edited

Hello, thank you for your help. Why not do you check seasonality?
△ ▽ • Reply • Share ›

Selva Prabhakaran Mod > Ary • a year ago

It may be contributed by the participating series, if so, seasonality will be taken care of.
△ ▽ • Reply • Share ›

Moe • a year ago

Thanks for sharing this. this is very helpful to understand VAR. however, would you please explain
what would be the change for multi-variate time series situation. I need to solve multivariate time
series here. I have more than 8 columns/attributes for each of my time series.
thanks
△ ▽ • Reply • Share ›

Selva Prabhakaran Mod > Moe • a year ago

This example illustrates multivariate situation Moe. Not sure if I understood your question
correctly
△ ▽ • Reply • Share ›

Suhas S Katte • a year ago

Hello , Thank you for this post . I have been trying to implement VAR time series on my own
dataset which consists of two attributes . The ADF test results show that the series is stationary.
The train dataset consists of 24000 values.The forecast values gives a constant value after a 100
predictions out of 9380 predictions being done, what could possibly be going wrong ?
△ ▽ • Reply • Share ›

Selva Prabhakaran Mod > Suhas S Katte • a year ago

May be too much noise making the series harder to forecast.. can say for sure without
looking at it
△ ▽ • Reply • Share ›

Nick • a year ago • edited

Thanks for the excellent post.

I have a question regarding step 10. You say

According to FPE and HQIC, the optimal lag is observed at a lag order of 3

However, the output of

x = model.select_order(maxlags=12)
x.summary()

shows that the optimal lag order is 12 - this is where all 4 metrics (aic, bic, fpe, hqic) have their
minima.

Why is there a discrepancy?

1△ ▽ • Reply • Share ›

Selva Prabhakaran Mod > Nick • a year ago

I dont know the answer to the 'why'. I'm sorry. But what you could do is pick one per your
intuition and see if the forecasts are good enough.
△ ▽ • Reply • Share ›

Amr Abdullah • a year ago

Hello, Thank you for this article. When I try to run step 14. I get the error "Train is not defined"

Shouldn't it be

df_results = invert_transformation(train_df, df_forecast, second_diff=True)

not
df_results = invert_transformation(train, df_forecast, second_diff=True)

Thank you again

△ ▽ • Reply • Share ›
 Reply Share ›

Selva Prabhakaran Mod > Amr Abdullah • a year ago

It's a typo. Thanks for noticing Amr

△ ▽ • Reply • Share ›

Kevin Kwong • a year ago

Hi. When I tried model.fit(0), error occur as below:
IndexError: index 0 is out of bounds for axis 0 with size 0
Do you know how to solve this?

Also, what is the "FALSE" in Cointegration Test mean? Is it mean that the data is not correlated to
others?

Also, I have tried to test the model with other dataset. The result is quite poor. Is there any way to
improve the accuracy?
△ ▽ • Reply • Share ›

Selva Prabhakaran Mod > Kevin Kwong • a year ago

'FALSE' could indicate the variables may not be inter-dependent to each other. Why would
you want to fit a 0 model btw?
△ ▽ • Reply • Share ›

Nisar • a year ago

Hi,
As i observed in the grangers_causality_matrix
1) The p_value : 0.0620 for gdfim_x and rgnp_y which is greater than significance. what does it
mean? does it mean that gdfim_x is not causing rgnp_y if i am not wrong.

2) That matrix is not symmetric could you please explain which one is causing which one.
△ ▽ • Reply • Share ›

Selva Prabhakaran Mod > Nisar • a year ago

1) Probably perhaps
2) I've named the predictor with suffic '_x' and dependent variable with suffix '_y'
△ ▽ • Reply • Share ›

Sirojiddin Nuriev • 2 years ago • edited

Hi,
I have time-series data 100000x20 (10 minutes). I am using LinearRegression(sklearn) model. My
architecture looks like VAR model. but I don't need to do my data stationary. And I can use that
model real-time as one-step-ahead forecasting. I should not retrain or refit.
Q1: is it VAR model?
Q2: what is the difference between my model and VAR?
Q3: Can I use VAR model for this data as real-time one-step ahead forecasting?
Please explain detail, thank you!
△ ▽ • Reply • Share ›

Selva Prabhakaran Mod > Sirojiddin Nuriev • a year ago

1) It is not a VAR model

2) In VAR model, all the participating series will influence each other both directions. The
architecture of linear regression is not like that.
3. You may be able to perhaps, if it satisfies the above condition and the performance is
fairly good.
△ ▽ • Reply • Share ›

William Constantine • 2 years ago • edited

When I run the Granger causality code, as you defined it and using the same data, I get all zeros in
the p-value matrix. Here is the run with verbose=True:

Y = rgnp, X = rgnp, P Values = [0.0, 1.0, 0.0024, 1.0, 1.0, 1.0, 1.0, 1.0, 0.0, 1.0, 1.0, 1.0]
Y = pgnp, X = rgnp, P Values = [0.0, 1.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0]
Y = ulc, X = rgnp, P Values = [0.0, 1.0, 1.0, 1.0, 0.0, 1.0, 0.0, 0.0, 0.0, 1.0, 0.0, 1.0]
Y = gdfco, X = rgnp, P Values = [0.0, 1.0, 0.0, 1.0, 0.0, 1.0, 0.0, 1.0, 0.0, 1.0, 0.0, 1.0]

In the definition of grangers_causation_matrix you have

min_p_value = np.min(p_values)
df.loc[r, c] = min_p_value

which selects the minimum p-value for all lags tested and is the reason for my causality matrix to
be filled with all zeros. My questions are:
Q1: The results you display for the grangers_causation_matrix are very different from what I am
i l i hi
see more

△ ▽ • Reply • Share ›

Selva Prabhakaran Mod > William Constantine • a year ago

Thanks for sharing William. Apologize for the very late reply.
△ ▽ • Reply • Share ›

Gizem • 2 years ago

Hi,

As i started to study for VAR, I realized this example has seasonality.So how do you handle with
that ? I could not find good explanation about seasonality in VAR so can you please explain. Thank
you!
△ ▽ • Reply • Share ›

Selva Prabhakaran Mod > Gizem • 2 years ago • edited

Could you explain more about how you say there is seasonality?
VAR models the information contained only among the chosen group. Any seasonality
modeled has to come from within this group
△ ▽ • Reply • Share ›

suruchi • 2 years ago

I did not understand the invert function as why are you taking cumulative sum for the differenced
series?
△ ▽ • Reply • Share ›

Selva Prabhakaran Mod > suruchi • a year ago

the idea is to undo the differencing operations. Try reverse engineering it without the
solution I've stated
△ ▽ • Reply • Share ›

Wilson Mupfururirwa • 2 years ago

hi, l am trying to do vector autoregression model for a larger dataset than the one here. Mine has
10419 columns and 58 rows. l followed the procedure here and when running my model to show
summary, it shows me this error: LinAlgError: 17-th leading minor of the array is not positive
definite

l dont know how to solve this, please help me..

△ ▽ • Reply • Share ›

Selva Prabhakaran Mod > Wilson Mupfururirwa • 2 years ago

Hi Wilson,

We need a reproducible example to be able to pinpoint whats happening. You might want to
try and follow this thread: https://stackoverflow.com/q... to resolve this.
△ ▽ • Reply • Share ›

Alex Doffmam • 2 years ago

I'm fairly sure your method for testing Granger causality requires some diagnostics on whether the
OLS in that function is well-specified.
△ ▽ • Reply • Share ›

Selva Prabhakaran Mod > Alex Doffmam • 2 years ago

Can you please elaborate on this?

△ ▽ • Reply • Share ›

Alex Doffmam > Selva Prabhakaran • 2 years ago • edited

If you look in the code base for the function grangercausalitytests, they apply
statsmodels' OLS to compare the two series (with or without the x you test if
Granger-cause y). So, as is common with OLS, one needs to check for iid errors etc.

Especially in time series, there's often autocorrelation.

Btw, there's also a built in test_causality function inside the VAR class. But I just
started working with statsmodels, so not sure yet what the difference is (besides
what data it uses)
△ ▽ • Reply • Share ›

Salahadin Seid • 2 years ago

Thanks This article helps me to understand VAR
 Thanks. This article helps me to understand VAR.
△ ▽ • Reply • Share ›

Selva Prabhakaran Mod > Salahadin Seid • 2 years ago

Welcome :)
△ ▽ • Reply • Share ›

Adam Hurta • 2 years ago

VAR model explained thoroughly and in a easy-to-understand fashion! Thank you!
△ ▽ • Reply • Share ›

Selva Prabhakaran Mod > Adam Hurta • 2 years ago

welcome :)
△ ▽ • Reply • Share ›

moar • 2 years ago

A very complete and thorough explanation on the topics. Thx!!
△ ▽ • Reply • Share ›

Selva Prabhakaran Mod > moar • 2 years ago

Welcome :)
△ ▽ • Reply • Share ›

(https://www.ezoic.com/what-is-
ezoic/)
report this ad

Search … Search Subscribe to Blog

Enter your email address to receive notifications
of new posts by email.

Email Address

Subscribe

Copyright ML+ (https://www.machinelearningplus.com/). All rights reserved.

Home (https://www.machinelearningplus.com) Contact Us (https://www.machinelearningplus.com/contact-us/)
Privacy Policy (https://www.machinelearningplus.com/privacy-policy/) About Selva (https://www.machinelearningplus.com/about/)
Terms and Conditions (https://www.machinelearningplus.com/terms-of-use/)