Location via proxy:   [ UP ]  
[Report a bug]   [Manage cookies]                
Scribd
Upload
61 views

Topic3 Multiple Regression

The document discusses the multiple regression model. It introduces the model with two independent variables and shows how it can be extended to multiple independent variables. It then covers OLS estimation of the multiple regression model, interpreting the coefficients, assessing goodness of fit using R-squared, and properties of the OLS estimator including assumptions of the Gauss-Markov theorem. It also addresses estimating the variance of OLS estimators, issues around model specification, and potential for omitted variable bias.

Uploaded by

Zulutan
Copyright
© © All Rights Reserved
Available Formats
Download as PDF, TXT or read online on Scribd
61 views

Topic3 Multiple Regression

The document discusses the multiple regression model. It introduces the model with two independent variables and shows how it can be extended to multiple independent variables. It then covers OLS estimation of the multiple regression model, interpreting the coefficients, assessing goodness of fit using R-squared, and properties of the OLS estimator including assumptions of the Gauss-Markov theorem. It also addresses estimating the variance of OLS estimators, issues around model specification, and potential for omitted variable bias.

Uploaded by

Zulutan
Copyright
© © All Rights Reserved
Available Formats
Download as PDF, TXT or read online on Scribd
You are on page 1/ 12

EC3090 Econometrics

Topic 3: The Multiple Regression Model

Reading:
Wooldridge, Chapter 3
Gujarati and Porter, Chapter 7

Topic 3: The Multiple Regression Model


1. The model with two independent variables
Say we have information on more variables that theory tells us may
influence Y:
Yi 0 1 X1i 2 X 2i ui
0 : measures the average value of Y when X1 and X2 are zero
1 and 2 are the partial regression coefficients/slope coefficients which
measure the ceteris paribus effect of X1 and X2 on Y, respectively
Key assumption:
E ui | X 1i , X 2i 0

For k independent variables:


Yi 0 1 X1i 2 X 2i ........ k X ki ui
E ui | X 1i , X 2i ,..........., X ki 0

Covui , X 1i Covui , X 2i ........ Covui , X ki 0

Topic 3: The Multiple Regression Model


2. OLS estimation of the multiple regression model
Simultaneously choose the values of the unknown parameters of the
population model that minimise the sum of the squared residuals:

u Y
n

2
i

i 1

i 1

1 X 1i 2 X 2i ....... k X ki

The first order conditions are given by the k+1 equations:

Y
n

i 1

0 1 X 1i 2 X 2i ....... k X ki 0

1i

X
i 1

..
..
n

X
i 1

ki

Y
i

1 X 1i 2 X 2i ....... k X ki 0

1 X 1i 2 X 2i ....... k X ki 0

Note: these equations can also be obtained using MM estimation

Topic 3: The Multiple Regression Model


2. OLS estimation of the multiple regression model
Consider the case where k=2. OLS requires that
we minimise:
2

u Y
n

i 1

2
i

i 1

1 X 1i 2 X 2i

The first order conditions are given by the 3 equations:

Y
n

i 1

X
i 1

1i

X
i 1

1 X 1i 2 X 2i 0

2i

1 X 1i 2 X 2i 0

Y
i

1 X 1i 2 X 2i 0

Solve simultaneously to find OLS parameter estimator

Topic 2: The Simple Regression Model


2. OLS estimation of the multiple regression model
Algebraic Properties:
n

1. u i 0
i 1
n

2.

3.

Y Y

i 1

u 0

ki i

Y u
i 1

i i

4. X 1 , X 2 ,....., X k , Y is always on the regression line

Topic 3: The Multiple Regression Model


3.

Interpreting the coefficients of the Multiple Regression Model


OLS slope coefficients depend on the relationship between each of
the individual variables and Y and on the relationship between the
Xs (illustrate)
In the two-variable example re-write the OLS estimator for 1 as:
n

1 r1iYi
i 1

r
i 1

2
1i

where r1 are the OLS residuals from a simple regression of X1 on X2.


Thus, 1 gives the pure effect of X1 on Y, i.e., netting out the effect
of X2.
Predicted Values: Yi 0 1 X1i 2 X 2i ........ k X ki
Residuals: ui Yi Yi
If ui 0 model under-predicts Y
If ui 0 model over-predicts Y

Topic 3: The Multiple Regression Model


3.

Interpreting the coefficients of the Multiple Regression Model


Relationship between simple and multiple regression estimates.
~
~
1 1 21
where the coefficients are OLS estimates from:
~ ~ ~
Yi 0 1 X 1i
Yi 0 1 X1i 2 X 2i
~ ~
~
X 2i 0 1 X 1i

The inclusion of additional regressors will affect the slope estimates.


~

But 1 1 where:
1. 2 0
2. ~ 0
1

Topic 3: The Multiple Regression Model


4.

Goodness-of-Fit in the Multiple Regression Model


How well does regression line fit the observations?
As in simple regression model define:
n
SST=Total Sum of Squares Yi Y 2
i 1
n
2
SSE=Explained Sum of Squares i1Yi Y
n
SSR=Residual Sum of Squares ui2

Y Y
n

R
2

i 1
n

Y Y
i 1

i 1

SSE
SSR
1
SST
SST

Recall: SST = SSE + SSR SSE SST and SSE > 0


0 SSE/SST 1
R2 never decreases as more independent variables are added use
adjusted R2 :
Includes punishment for
SSR n k 1
2
R 1
adding more variables to the
SST n 1
model

Topic 3: The Multiple Regression Model


5. Properties of OLS Estimator of Multiple Regression Model
Gauss-Markov Theorem
Under certain assumptions known as the Gauss-Markov assumptions the
OLS estimator will be the Best Linear Unbiased Estimator
Linear: estimator is a linear function of the data
Unbiased: E 0 0


E
k

Best: estimator is most efficient estimator, i.e., estimator has the


minimum variance of all linear unbiased estimators

Topic 3: The Multiple Regression Model


5. Properties of OLS Estimator of Multiple Regression Model
Assumptions required to prove unbiasedness:
A1: Regression model is linear in parameters
A2: X are non-stochastic or fixed in repeated sampling
A3: Zero conditional mean
A4: Sample is random
A5: Variability in the Xs and there is no perfect collinearity in the Xs
Assumptions required to prove efficiency:
A6: Homoscedasticity and no autocorrelation

V ui | X 1i , X 2i ,...., X ki 2
Covui , u j 0

Topic 3: The Multiple Regression Model


6. Estimating the variance of the OLS estimators
Need to know dispersion (variance) of sampling distribution of OLS
estimator in order to show that it is efficient (also required for inference)
In multiple regression model:

V k

SSTk 1 Rk2

Depends on:
a) 2: the error variance (reduces accuracy of estimates)
b) SSTk: variation in X (increases accuracy of estimates)
c) R2k: the coefficient of determination from a regression of Xk on all
other independent variables (degree of multicollinearity reduces accuracy
of estimates)

What about the variance of the error terms 2?


n
1

ui2

n k 1 i 1
2

Topic 3: The Multiple Regression Model


7. Model specification
Inclusion of irrelevant variables:
OLS estimator unbiased but with higher variance if Xs correlated
Exclusion of relevant variables:
Omitted variable bias if variables correlated with variables included in
the estimated model
Yi 0 1 X1i 2 X 2i
~ ~ ~
Y
Estimated Model:
i 0 1 X 1i
~ ~

1 1 21
OLS estimator:

True Model:

~
~
E

Biased:
1
1
2 1
Omitted Variable Bias: Bias ~1 2~1

You might also like