Topic 3c - Multivariate Regression Model

Dr. Olalekan Aworinde

Department of Economics
Babcock University
Ilishan-Remo

These presentation notes are based on Theory of Econometrics Second Edition by

A. Koutsoyiannis
The Multiple Linear Regession Model

The …rst stage of any econometric research is economic theory. For

example, we will use the theory of demand. The theory states that
"the higher the price the lower the quantity demanded and the higher
the level of income the higher the quantity demanded.
The next stage is the speci…cation of the model. In specifying the
model we have to know the number of equations of the model and
their precise mathematical form. In this class, we will concentrate on
single equation for now.
The next thing is the a-priori expectation of the model and this deals
with the signs and magnitude of the parameter estimates.
The dependent variable is called the regressand and the
independent/explanatory variable is called the regressors.
Model Speci…cation

Y = f (X1 ; X2 )

where Y = quantity demanded and X1 = Price of the commodity, X2

is the income of the consumer.

Yi = b0 + b1 X1 + b2 X2

The above speci…cation shows that price of the commodity and

income of the consumer is the only factor in‡uencing the level of
quantity demanded.
Our aim is to estimate the numerical values of b̂0 , b̂1 and b̂2 .
On theoretical a-priori b̂1 is expected to be negative and b̂2 is
expected to be positive if it is a normal good.
The Normal Equations of OLS

The true relationship is Yi = b0 + b1 X1 + b2 X2

The estimated relationship Yi = b̂0 + b̂1 X1i + b̂2 X2i + ei
The estimated regression line Ŷi = b̂0 + b̂1 X1i + b̂2 X2i
ei = Yi Ŷi
Substituting Ŷi we have ei = Yi b̂0 b̂1 X1i b̂2 X2i

Squaring both sides and taking the sum we have;

n n
Σ ei2 = Σ (Yi b̂0 b̂1 X1i b̂2 X2i )2
i =1 i =1

The sum of squared residual deviations is to be minimized with

respect to b̂0 , b̂1 and b̂2 so as to have the normal equations.
The Normal Equations of OLS (Cont’d)

The necessary condition is that the …rst order derivatives be equal to

zero that is;

∂Σe 2 ∂Σe 2 ∂Σe 2

= 0, = 0 and =0
∂b̂0 ∂b̂1 ∂b̂2
Using Chain rule; if y = f (w ) and w = f (x )
dy dy dw
) = .
dx dw dx
Where w = Σ(Yi b̂0 b̂1 X1i b̂2 X2i )
∂Σe 2 ∂w
Σe 2 = w 2, ) = 2w , = 1
∂w ∂b̂0
∂Σe 2
= 2Σ(Yi b̂0 b̂1 X1i b̂2 X2i ). ( 1) = 0
∂b̂0
ΣYi = nb̂0 + b̂1 ΣX1i + b̂2 ΣX2i
The Normal Equations of OLS (Cont’d)

∂Σe 2 ∂w
= 2w , = X1i
∂w ∂b̂1
∂Σe 2
= 2Σ(Yi b̂0 b̂1 X1i b̂2 X2i ). ( X1i ) = 0
∂b̂1
ΣX1i Yi = b̂0 ΣX1i + b̂1 ΣX1i2 + b̂2 ΣX1i X2i
∂Σe 2 ∂w
= 2w , = X2i
∂w ∂b̂2
∂Σe 2
= 2Σ(Yi b̂0 b̂1 X1i b̂2 X2i ). ( X2i ) = 0
∂b̂2
ΣX2i Yi = b̂0 ΣX2i + b̂1 ΣX1i X2i + b̂2 ΣX2i2
The normal equation gives;
ΣYi = nb̂0 + b̂1 ΣX1i + b̂2 ΣX2i
ΣX1i Yi = b̂0 ΣX1i + b̂1 ΣX1i2 + b̂2 ΣX1i X2i
The Normal Equations of OLS (Cont’d)

ΣX2i Yi = b̂0 ΣX2i + b̂1 ΣX1i X2i + b̂2 ΣX2i2

Expressing in matrix of the form
AX = B
2 32 3 2 3
n ΣX1 ΣX2 b̂0 ΣY
4ΣX1 ΣX12 ΣX1 X2 5 4b̂1 5 = 4ΣX1 Y 5
ΣX2 ΣX1 X2 ΣX22 b̂2 ΣX2 Y

Using the deviation formula

b̂0 = b̂0 = Ȳ b̂1 X̄1 b̂2 X̄2
(Σx1 y )(Σx22 ) (Σx2 y )(Σx1 x2 )
b̂1 =
(Σx12 )((Σx22 ) (Σx1 x2 )2
(Σx2 y )(Σx12 ) (Σx1 y )(Σx1 x2 )
b̂2 =
(Σx12 )((Σx22 ) (Σx1 x2 )2
The Normal Equations of OLS (Cont’d)

Using the Crammer’s rule for the absolute values we have

det b̂0 det b̂1 det b̂2
b̂0 = , b̂1 = and b̂2 =
jdet Aj jdet Aj jdet Aj

Σe 2
RY2 .X 1 .X 2 = 1 or
Σy 2
b̂1 (Σx1 y ) + b̂2 (Σx2 y )
RY2 .X 1 .X 2 =
Σy 2
The standard error is given as
s
Σe 2 1 X̄12 Σx22 + X̄2 Σx12 2X̄1 X̄2 Σx1 x2
s (b̂0 ) = . +
n k n (Σx12 )((Σx22 ) (Σx1 x2 )2
s
Σe 2 Σx22
s (b̂1 ) = .
n k (Σx12 )((Σx22 ) (Σx1 x2 )2
The Normal Equations of OLS (Cont’d)

s
Σe 2 Σx12
s (b̂2 ) = .
n k (Σx1 )((Σx22 ) (Σx1 x2 )2
2
Example

Example: Given that table below where Y = quantity demanded, X1

= price of the commodity and X2 = income of the consumer.
Estimate the regression line and how reliable are the tests of
signi…cant.
Y X1 X2
100 5 1000
75 7 600
80 6 1200
70 6 500
50 8 300
65 7 400
90 5 1300
100 4 1100
110 3 1300
60 9 300
Solution

Using the deviation formular we arrived at the following values;

b̂0 = b̂0 = Ȳ b̂1 X̄1 b̂2 X̄2

b̂0 = 80 ( 7.19)(6) (0.0143)800 = 111.69
(Σx1 y )(Σx22 ) (Σx2 y )(Σx1 x2 )
b̂1 =
(Σx12 )((Σx22 ) (Σx1 x2 )2
( 300)(1580000) (65000)( 5900) 90500
b̂1 = 2
= = 7.188
(30)(1580000) ( 5900) 12590
(Σx2 y )(Σx12 ) (Σx1 y )(Σx1 x2 )
b̂2 =
(Σx12 )((Σx22 ) (Σx1 x2 )2
(65000)(30) ( 300)( 5900) 180
b̂2 = 2
= = 0.0143
(30)(1580000) ( 5900) 12590
Solution (Cont’d)
Using the Cramers Rule for the absolute values we have the following;
2 32 3 2 3
n ΣX1 ΣX2 b̂0 ΣY
4ΣX1 ΣX12 ΣX1 X2 5 4b̂1 5 = 4ΣX1 Y 5
ΣX2 ΣX1 X2 ΣX22 b̂2 ΣX2 Y
2 32 3 2 3
10 60 8000 b̂0 800
4 60 390 42100 5 4b̂1 5 = 4 4500 5
8000 42100 7980000 b̂2 705000
For easy calculation we can divide the matrix by say 1000 and we now
have; 2 32 3 2 3
0.01 0.06 8 b̂0 0.8
40.06 0.39 42.1 5 4b̂1 5 = 4 4.5 5
8 42.1 7980 b̂2 705
0.39 42.1 0.06 42.1 0.06 0.39
det A = 0.01 0.06 +8
42.1 7980 8 7980 8 42.1
Solution (Cont’d)

det A = 0.01 j3112.2 1772.41j 0.06 j478.8 336.8j + 8 j2.526 3.12j

det A = 0.01(1339.79) 0.06(142) + 8( 0.594)
det A = 13.3979 8.52 4.752 = 0.1259
det b̂0
b̂0 =
2det A 3
0.8 0.06 8
det b̂0 = 4 4.5 0.39 42.1 5
705 42.1 7980
0.39 42.1 4.5 42.1 4.5 0.39
det b̂0 = 0.8 0.06 +8
42.1 7980 705 7980 705 42.1
det b̂0 = 0.8(3112.2 1772.41) 0.06(35910 29680.5)+
0.8(189.45 274.95)
det b̂0 = 0.8(1339.79) 0.06(6229.5) + 8( 85.5)
Solution (Cont’d)

det b̂0 = 1071.832 373.77 684 = 14.062

det b̂0 14.062
b̂0 = = = 111.6918
det A 0.1259

det b̂1
b̂1 =
2det A 3
0.01 0.8 8
det b̂1 = 40.06 4.5 42.1 5
8 705 7980
4.5 42.1 0.06 42.1 0.06 4.5
det b̂1 = 0.01 0.8 +8
705 7980 8 7980 8 705
det b̂1 = 0.01(35910 29680.5) 0.8(478.8 336.8) + 8(42.3 36)
det b̂1 = 0.01(6229.5) 0.8(142) + 8(6.3)
det b̂1 = 62.295 113.6 + 50.4 = 0.905
Solution (Cont’d)

det b̂1 0.905

b̂1 = = = 7.188
det A 0.1259

det b̂2
b̂2 =
2det A 3
0.01 0.06 0.8
det b̂2 = 40.06 0.39 4.5 5
8 42.1 705
0.39 4.5 0.06 4.5 0.06 0.39
det b̂2 = 0.01 0.06 + 0.8
42.1 705 8 705 8 42.1
det b̂2 = 0.01(274.95 189.45) 0.06(42.3 36) + 0.8(2.526 3.12)
det b̂2 = 0.01(85.5) 0.06(6.3) + 0.8( 0.594)
det b̂2 = 0.855 0.378 0.4752 = 0.0018
det b̂2 0.0018
b̂2 = = = 0.0143
det A 0.1259
Solution (Cont’d)

b̂1 (Σx1 y ) + b̂2 (Σx2 y )

RY2 .X 1 .X 2 =
Σy 2
( 7.19)( 300) + (0.0143)(65000)
RY2 .X 1 .X 2 = = 0.894
3450
Σe 2
RY2 .X 1 .X 2 = 1
Σy 2
) Σe 2 = Σy 2 (1 R 2 )
Σe 2 = 3450(1 0.894)
Σe 2
= 3450(0.106) = 365.7
s
Σe 2 1 X̄12 Σx22 + X̄2 Σx12 2X̄1 X̄2 Σx1 x2
s (b̂0 ) = . +
n k n (Σx12 )((Σx22 ) (Σx1 x2 )2
r h i
365.7 1 36 (1580000 )+640000 (30 ) 2 (6 800 )( 5900 )
s (b̂0 ) = 10 3 . 10 + (30 )(1580000 ) ( 5900 ) 2 = 23.5
Solution (Cont’d)

s
Σe 2 Σx22
s (b̂1 ) = .
n k (Σx12 )((Σx22 ) (Σx1 x2 )2
s
365.7 1580000
s (b̂1 ) = .
10 3 (30)(1580000) ( 5900)2
s (b̂1 ) = 2.55
s
Σe 2 Σx12
s (b̂2 ) = .
n k (Σx12 )((Σx22 ) (Σx1 x2 )2
s
365.7 30
s (b̂2 ) = .
10 3 (30)(1580000) ( 5900)2
s (b̂2 ) = 0.01
Solution (Cont’d)

n
Σ ( et et 1)
2
278.0227
t =2
DW = n = = 1.055
263.306
Σ et2
t =1

The estimated relationship is of the form,

Ŷ = 111.7 7.19X1 + 0.014X2
s (b̂i ) = (23.5) (2.55) (0.01)
t-stat = 4.75 2.8 1.28
R2 = 0.894
DW = 1.055