Lecture Two (Copy)

Econometrics 5/30/2023
2.1 Concept of Regression Function

 It is a statistical tool for understanding the relationship
between different variables.
CHAPTER TWO  Usually we want to know the causal effect of one variable on
another. For instance we might ask the question how much
extra income do people receive if they have had one more year
of education all other things equal. When I represents income
and E education this is equivalent to asking What is ∂I/∂E?
Simple Linear Regression  To answer this question the econometrician collects data on
income and education, and uses it to run a regression
model equation.
1 By: Mmaru S.(MA) 30/05/2023 2 By: Mmaru S.(MA) 30/05/2023
Regression Vs Correlation? Regression Vs Correlation?

 How can we analyze the association between two or more  In our example, “how your GPA is affected by the time
variables? you spend in library”
 E.g. The association b/n the time (t) you spend at library &  Regression is estimation or prediction of the average
your academic performance (GPA)
value of a dependent variable on the basis of the fixed
 Both regression & correlation analyses are useful to assess the
association between two or more variables…..e.g. what
values of other variables.
implies corr (y,x) = -0.8?  Regression analysis is concerned with the study of the
 How ever, regression and correlation are different concept. dependence of one variable (the dependent variable) on
 Regression is useful to assess the causal relationship (which one or more other variables (the explanatory variable(s))
variable causes what) (Gujrati, 2004).
 Correlation measures the strength of linear association
b/n variables.
 The value for a correlation coefficient is always between -
By: Mmaru S.(MA) 30/05/2023 By: Mmaru S.(MA) 30/05/2023
3 4
1 and 1.
Examples of Correlation in Real Life

 Identify the following examples that exist real-life scenarios 1. Estimate a relationship among economic variables, such
of negative, positive, and no correlation between variables? as y = f(x).
 Time Spent Running vs. Body Fat  Establish if there is a statistically significant relationship
 Time Spent Watching TV vs. Exam Scores between two variables.
 Height vs. Weight E.G Income and spending, wage and gender, student height and
 Temperature vs. Ice Cream Sales exam scores.
 Coffee Consumption vs. Intelligence 2. Forecast or predict the value of one variable, y, based on
the value of another variable, x.
 Shoe Size vs. Movies Watched
What we know about the relationship to forecast unobserved
values.
E.G what will are sales be over the next quarter?
2.2 Simple linear regression (SLR) Linear Vs Non-linear regression

 Generally, in regression, we predict scores on one variable  Regression models can be either linear or nonlinear.
(Y) from the scores on other variables (Xs).  In a linear model, the relationships between variables are
straight-line,
e.g.
 In simple linear regression, the predictions of one

dependent variable (Y) when plotted as a function of one
 When there is only one predictor variable, the prediction independent (X) variable form a straight line.
method is called simple regression.  See the figure below
 In simple linear regression, the predictions of Y when
7plotted as S.(MA)
By: Mmaru a function of X form a straight line. 30/05/2023 8 By: Mmaru S.(MA) 30/05/2023
 Example: SLR (taken from Wooldridge 5th edition)  In a non-linear model, the relationships between variables
are curved lines.
Nature of error term

Cont’d…  In econometrics, we deal with the stochastic relationship. Why?
 Generally, we use Simple Linear Regression; when we are That is, why should we add the Error Term?
considering a linear relationship between one dependent & one  There are some justifications for the inclusion of the error term
independent variable in the model. These are:
 We say it simple b/c we use only 2 variables) i. Effect of the omitted variables from the model
ii. Measurement Error
 We use the term linear b/c the relationship is linear
iii. Wrong Mathematical Specification of the Model
 But, do you expect this to happen in reality? iv. Errors in Aggregation
 Can you give me an example of real life phenomena represented v. The randomness of human behavior
by this model?  In order to take all these sources of error into account, we
introduce the stochastic/random disturbance term into our
 E.g. Assume a model relating your wage to education & other
econometric models and hence the complete simple econometric
unobserved variables (u)
model becomes: 𝑌 = 𝛼 + 𝛽𝑥 + 𝜇
11 What
 about
By: Mmaru experience, natural ability,….?
S.(MA) 30/05/2023
Variation
12 By: Mmaruin Y= Explained variation+ Unexplained variation
S.(MA) 30/05/2023
Concept of PRF and SRF Concept of PRF and SRF

Recall to assumptions  The specific functional form of is assumed to be linear.
I. X is fixed in repeated sampling but Y’s
That is, EY( / X  X )  X (E( ))
i i i i
vary(Conditional distribution of Yi given values of
 However we don’t have data on all Y’s (population) and we
Xi). have sample information on Y for a given values of X’s.
II. ∈ 𝜇 = 0
 Unlike the population case, we have one value of y for each
Now for each conditional distribution of Y, we can
compute its mean or average value known as the valve of x. Now, we can estimate the PRF based on sample
conditional mean or conditional expectation of Y information but the estimation is not accurate due to
given X takes the specific values of xi. It is denoted
by: E(Yi / X  Xi )  f ( X i ) . Such function is known as the sampling fluctuations. Terefore, the regression function we
population regression function (PRF). It measures obtained from the sample is known as sample regression
how the mean response of Y varies with X.
13 By: Mmaru S.(MA) 30/05/2023 14
function (SRE).
By: Mmaru S.(MA)
30/05/2023
Concept of PRF and SRF 2.3. Assumption of Classical Linear

Regression Model
Population regression function (PRF) is the line of best
fit of the whole population and the sample regression 
function(SRF) is based on a sample.
The main objective of regression analysis is the

estimation of the PR: Yi     Xi  i based on the
^ ^
SRF: Y     X  e
15 i i i 16 By: Mmaru S.(MA) 30/05/2023
30/05/2023 By: Mmaru S.(MA)
Cont’d… Cont’d…


 

Cont’d…

 Fig 2.4(a): Homoscedasticity
 Fig 2.4(b): Heteroscedasticity
 

Cont’d… 2.4 Estimation Techniques

 Assumption 8: Variability in X values  We have seen that a relationship between variables can be
 The X values in a given sample must not all be the
linear/nonlinear.
same.  E.g.
 Technically, Var( X ) must be a finite positive number.  No matter what the r/s actually is, we have to know how to estimate
This means that X assumes d/t values in a given the parameters (the β).
sample; but it assumes fixed values in hypothetical  The most widely known estimation techniques are;
repeated samples.  Method of Ordinary Least Squares (OLS)
 Assumption 9: The regression model is  Method of Moments (MM)
correctly specified  Method of Maximum Likelihood Estimation (ML)
 This means that the mathematical form of the
model is correctly specified & all important
explanatory variables are included in it.
 There is no specification bias or error in the
model used in empirical analysis.
2.4.1 Method of Ordinary Least Squares (OLS

estimation) Cont’d
 The method of OLS is attributed to Carl Friedrich The ordinary least squares (OLS) method of Estimation:
Gauss, a German mathematician. In the regression model Yi = a + bXi + ei , the values of the
 Under certain assumptions, the method has some very parameters a and b are not known.
attractive statistical properties. These properties have When they are estimated from a sample of size n, we obtain the
made OLS one of the most powerful & popular sample regression line given by:
methods of regression. To understand this method, we
  
Yi  a  bX i where i=1,...,n
have first to explain the least squares principle.  
 Estimation of 𝛼 & 𝛽 by OLS involves finding values for  where a and b are estimated by a and b , respectively, and ˆY is the
estimated value of Y.
the estimates ȧ and Ḃ and which will minimize the sum
 The deviations between the observed and estimated values of Y are
of square of the residuals. called the residuals êi , that is:
 Recall that the least square method involves minimizing
eî = Yi − Yî , i = 1, 2, . . ., n
the sum of squared residuals.
Cont’d Cont’d
The magnitude of the residuals is the vertical distance between the The sum of squares of the errors (SSE) is:
actual observed points and the estimating line (see the figure below).
By partial differentiation of the SSE with respect toˆ and ˆ

and equating the results to zero we get:
 The estimating line will have a ‘good fit’ if it minimizes the

error between the estimated points on the line and the actual Re-arranging the two equations, we get the so-called normal
observed points that were used to draw it equations
2.4.2 Statistical Properties of Least Square

Cont’d Estimators: The Gauss-Markov Theorem
The ideal or optimum properties that the OLS estimates
possess may be summarized by well known theorem
This is the simple linear regression (or two-variable) model in known as the Gauss-Markov Theorem.
deviations form. Statement of the theorem: “Given the assumptions of
The OLS estimator from the above equation is given by through solving the classical linear regression model, the OLS
the two simultaneous equations, where the lower cases are in estimators, in the class of linear and unbiased
deviation form: estimators, have the minimum variance, i.e. the OLS
: estimators are BLUE.
𝑛 ∑ 𝑋𝑖 𝑌𝑖 − ∑ 𝑋𝑖 ∑ 𝑌𝑖 ∑(𝑋𝑖 − 𝑋)(𝑌𝑖 − 𝑌) ∑ 𝑥𝑖 𝑦𝑖 ∑ 𝑥𝑖 𝑌𝑖 ∑ 𝑋𝑖 𝑦𝑖
𝛽2 = = = = = According to this theorem, under the basic assumptions
𝑛 ∑ 𝑋𝑖2 − (∑ 𝑋𝑖 )2 (∑(𝑋𝑖 − 𝑋))2 ∑ 𝑥𝑖2 ∑ 𝑥𝑖2 ∑ 𝑥𝑖2
of the classical linear regression model, the least squares
∑ 𝑋𝑖2 ∑ 𝑌𝑖 − ∑ 𝑋𝑖 ∑ 𝑋𝑖 𝑌𝑖 estimators are linear, unbiased and have minimum
𝛽1 = = 𝑌 − 𝛽2 𝑋
𝑛 ∑ 𝑋𝑖2 − (∑ 𝑋𝑖 )2 variance (i.e. are best of all linear unbiased estimators).
Cont’d… Cont’d
Sometimes the theorem referred as the BLUE Recall Assumptions of the CLRM:
theorem i.e. Best, Linear, Unbiased Estimator. An 1. The true model is: Yi = a + bXi + ei
estimator is called BLUE if: 2. The error terms have zero mean: E(e i ) = 0
A. Linear: a linear function of the random variable,
3. Homoskedasticity (error terms have constant
such as, the dependent variable Y. variance):
B. Unbiased: its average or expected value is equal
to the true population parameter.
4. No error autocorrelation (the error terms ei are
C. Minimum variance: it has a minimum variance in
statistically independent of each other)
the class of linear and unbiased estimators. An
unbiased estimator with the least variance is
known as an efficient estimator. 5. Xi are deterministic (non-stochastic): Xi and e j are
independent for all i, j
Cont’d Cont’d
The Gauss-Markov Theorem: Proof of Gauss-Markov Theorem:
Under assumptions (1) – (5) of the CLRM, the OLS estimators Here we will prove that ̂ is the BLUE of  . The proof for ̂ can
be done similarly.
ˆ and ˆ are Best Linear Unbiased Estimators (BLUE).
The theorem tells us that of all estimators of and which are
linear and which are unbiased, the estimators
resulting from OLS have the minimum variance, that
is, ˆ and ˆ are the best (most efficient) linear unbiased
estimators (BLUE) of  and  .
Note: If some of the assumptions stated above do not hold, then
OLS estimators are no more BLUE!!!  Thus, we can see that ̂ is a linear estimator as it can be written
as a weighted average of the individual observations on Y
Cont’d Cont’d
c) To show that ̂ has the smallest variance out of all linear
unbiased estimators of  .
Now we have: (since x is non-stochastic

(assumption 5),
And E (ei ) = 0 (assumption
2)).
Thus:
B hat is an unbiased
37 By: Mmaru S.(MA) estimator 30/05/2023
of B 38 By: Mmaru S.(MA) 30/05/2023
Cont’d Cont’d
We have seen above (in proof (a)) that the OLS estimator of can be
expressed as:
Note that (**) follows from assumptions (3) and (4)

Cont’d Cont’d
Taking expectations we have:
Cont’d Statistical inference in simple linear regression model

Thus, we have shown that: Estimation of standard error
 To make statistical inferences about the true (population) regression
coefficient  , we make use of the estimator ̂ and its variance
 Since ( (which is a sum of squares of real numbers) is always greater Var(̂ ) . We have already seen that:
than or equal to zero, we have:
This implies that the variance of ̂ is the smallest as compared to

the variance of any other linear unbiased estimator of  .  Since this variance depends on the unknown parameter , we
have to estimate . As shown above, an unbiased estimator of
given by:
Maximum likelihood (ML) method of estimation also gives the
same results as the OLS
2.4.2 Statistical Test of Significance of the OLS

Cont’d
Estimators
 After estimation of the parameters there are important
issues to be considered by the researcher.
 We have to know that to what extent our estimates are
reliable enough and acceptable for further purpose.
 That means, we have to evaluate the degree of
representativeness of the estimate to the true
The estimated variance of the random term: population parameter.
Where, k is the number of parameters to be  Simply a model must be tested for its significance

2
: ei estimated in the model. before it can be used for any other purpose.
u 
2
 Xi
2
var(ˆ1 )   u
2 1  The available test criteria are divided in to three
nk var(ˆ0 )   u
2
groups: Theoretical a priori criteria, statistical criteria
By: Mmaru S.(MA)
2
n x i  xi 2 and econometric criteria
45 30/05/2023 46 By: Mmaru S.(MA) 30/05/2023
Cont’d… Cont’d..
1. The square of correlation coefficient (𝒓𝟐 ) which
 Priori criteria set by economic theories are in line
is used for judging the explanatory power of the linear
with the consistency of coefficients of econometric
regression of Y on X or on X’s. The square of
model to the economic theory.
correlation coefficient in simple regression is known
 Statistical criteria, also known as first order tests,
as the coefficient of determination and is given by
are set by statistical theory and refer to evaluate the
(𝑅 ). The coefficient of determination measures the
statistical reliability of the model.
goodness of fit of the line of regression on the
 Econometric criteria refer to whether the
observed sample values of Y and X.
assumptions of an econometric model employed in
2. The standard error test of the parameter
estimating the parameters are fulfilled or not.
estimates applied for judging the statistical reliability
There are two most commonly used tests in
of the estimates. This test measures the degree of
econometrics. These are:
confidence that we may attribute to the estimates.
I. Test of model adequacy

The coefficient of determination is the measure of the amount
Cont’d
or proportion of the total variation of the dependent variable  Is the estimated equation a useful one? To answer this, an
that is determined or explained by the model or the presence of objective measure of some sort is desirable.
the explanatory variable in the model.  The total variation in the dependent variable Y is given by:
The total variation of the dependent variable is split in two
additive components; a part explained by the model and a part
represented by the random term. The total variation of the
dependent variable is measured from its arithmetic mean.  Our goal is to partition this variation into two: one that
Total Variation in 𝒀𝒊 = ∑(𝒀𝒊 − 𝒀)𝟐 accounts for variation due to the regression equation (explained
Total explained variation = ∑(𝒀𝒊 − 𝒀)𝟐 portion) and another that is associated with the unexplained portion
Total unexplained variation= ∑ 𝜺𝒊 𝟐 of the model.
NB. The closer the observation to the line, the better the
goodness of fit, i.e. the better is the explanation of the variations
of Y by the changes in the explanatory variables.
 Total variation = explained variation due to regression +

unexplained variation due to error
 Sum of squares total = sum of squares due to regression + sum of
squares due to error
51 By: Mmaru S.(MA) 30/05/2023 By: Mmaru
52  SST= S.(MA)
SSR+SSE 30/05/2023
Cont’d Cont’d
Show this is zero
Cont’d Cont’d
From remarks (1) and (2) it follows that  The error (residual or unexplained) sum of squares (ESS) is
a measure of the dispersion of the observed values of Y about
the regression line. This is computed as:
Thus, the cross product term in equation (*) vanishes, and we are left  If a regression equation does a good job of describing the relationship
with:
between two variables, the explained sum of squares should
constitute a large proportion of the total sum of squares.
 Thus, it would be of interest to determine the magnitude of
this proportion by computing the ratio of the explained
 In other words, the total sum of squares (TSS) is decomposed into
sum of squares to the total sum of squares.
regression (explained) sum of squares (RSS) and error (residual or  This proportion is called the sample coefficient of
unexplained) sum of squares (ESS). determination.
Cont’d Cont’d
The higher the coefficient of determination is the
better the fit. Conversely, the smaller the
Note coefficient of determination is the poorer the fit.
1) The proportion of total variation in the dependent variable That is why the coefficient of determination is
(Y) that is explained by changes in the independent variable used to compare two or more models.
(X) or by the regression line is equal to: R2 x100%. One minus the coefficient of determination is
2) The proportion of total variation in the dependent variable called the coefficient of non-determination, and it
(Y) that is due to factors other than X (for example, due to gives the proportion of the variation in the
excluded variables, chance, etc) is equal to: (1– R2 ) x dependent variable that remained undetermined
100%. or unexplained by the model.
Cont’d ii) Testing the significance of a given regression coefficient

Tests for the coefficient of determination ( R 2): Since the sample values of the intercept and the coefficient
The largest value that R 2 can assume is 1 (in which case all are estimates of the true population parameters, we have to
observations fall on the regression line), and the smallest test them for their statistical reliability.
it can assume is zero. The significance of a model can be seen in terms of the
A low value of R 2 is an indication that: amount of variation in the dependent variable that it
explains and the significance of the regression coefficients.
 X is a poor explanatory variable in the sense that variation in
There are different tests that are available to test the
X leavesY unaffected, or
statistical reliability of the parameter estimates. The
 while X is a relevant variable, its influence on Y is weak as following are the common ones;
compared to some other variables that are omitted from the
A) The standard error test
regression equation, or
B) The standard normal test
 the regression equation is misspecified (for example, an
exponential relationship might be more appropriate. C) The students t-test
All of these testing procedures reach on the same conclusion.
ii) Testing the significance of a given regression coefficient A. The Standard Error Test
• The significance level is a measure of how strong the This test first establishes the two hypotheses that are going to be
sample evidence must be before determining the results tested which are commonly known as the null and alternative
are statistically significant. hypotheses. The null hypothesis addresses that the sample is
• Significance level (a) should be chosen in advance. coming from the population whose parameter is not significantly
• A significance level of 0.05 (which is α = 5%) is widely different from zero while the alternative hypothesis addresses that
used. the sample is coming from the population whose parameter is
• Our decision will always be rejecting the null significantly different from zero. The two hypotheses are given as
hypothesis if the P-value is less than the level of α we follows:
chose N.B Standard deviation is the positive square root of the
• Rejection of our null hypothesis suggests that our variance. I.e. The standard error of a given coefficient is the
research hypothesis is true positive square root of the variance of the coefficient.
• Moreover, to carry out a t-test, we need to know the
61 standard errors of individual estimators
By: Mmaru S.(MA) 30/05/2023 62 By: Mmaru S.(MA) 30/05/2023
Consider the simple linear regression model Next, compare the standard errors of the estimates with the
numerical values of the estimates and make decision.
If there no relationship between X and Y then the null hypothesis A) If the standard error of the estimate is less than half of the
is expressed as: numerical value of the estimate, we can conclude that the estimate

1 
The alternative hypothesis is that there is a significant relationship is statistically significant.That is, if se (  i )  (  i ) ,
2
between X and Y, that is, reject the null hypothesis and we can conclude that the estimate is
In order to reject or not reject the null hypothesis, we calculate statistically significant.
the standard error test statistic given by: B) If the standard error of the estimate is greater than half of the
  U2 numerical value of the estimate, the parameter estimate is not
se (  1 ) 

1 
statistically reliable.That is, if se (  i )  (  i ) ,
𝑆𝐸 𝛽 = 𝑉𝑎𝑟(𝛽 )  x i2 2
conclude to accept the null hypothesis and conclude that the
𝑆𝐸 𝛽 = 𝑉𝑎𝑟(𝛽 )
  U2  X i2
63 By: Mmaru S.(MA) se (  0 )  30/05/2023 estimate
64 is not
By: Mmaru S.(MA)statistically significant 30/05/2023
n  x i2
Cont’d… Error in statistical decision-making

Note that, βj measures the partial effect of Xj on (the I. A Type I error means rejecting the null hypothesis when it’s
actually true. It means concluding that results are statistically significant
expected value of) y,
when, in reality, they came about purely by chance or because of
As in simple linear regression for inference on βj we use unrelated factors. The risk of committing this error is the significance
the t-distribution. level (alpha or α) you choose. That’s a value that you set at the
We will need the standard error (se) of the least squares beginning of your study to assess the statistical probability of obtaining
estimator. your results (p value).
The estimator for a coefficient becomes more precise II. Type II error means not rejecting the null hypothesis when it’s
(se decreases) when: actually false. This is not quite the same as “accepting” the null
hypothesis, because hypothesis testing can only tell you whether to
the residual standard deviation σε decreases,
reject the null hypothesis. Instead, a Type II error means failing to
variation of the values for the x’s increases, conclude there was an effect when there actually was. In reality, your
study may not have had enough statistical power to detect an effect of a
By: Mmaru S.(MA)
65 30/05/2023
certainBy:size.
66 Mmaru S.(MA) 30/05/2023
One tail test or two tail test

The level of significance
If the inequality sign in the alternative hypothesis is ≠ , then it
implies a two tail test and divide the chosen level of significance It is defined as the fixed probability of wrong elimination of
by two; decide the critical rejoin or critical value of t called tc. null hypothesis when in fact, it is true.
But if the inequality sign is either > or < then it indicates one tail The level of significance is stated to be the probability of
test and there is no need to divide the chosen level of significance type I error and is preset by the researcher with the
by two to obtain the critical value of to from the t-table. outcomes of error.
BASIS OF COMPARISON ONE-TAILED TEST TWO-TAILED TEST
Meaning A statistical hypothesis test in A significance test in which
The level of significance is the measurement of the
which alternative hypothesis has alternative hypothesis has two statistical significance. It defines whether the null
only one end, is known as one ends, is called two-tailed test.
tailed test. hypothesis is assumed to be accepted or rejected. It is
Hypothesis Directional Non-directional expected to identify if the result is statistically significant
Region of rejection Either left or right Both left and right for the null hypothesis to be false or rejected.
Determines If there is a relationship between If there is a relationship between It is denoted by the Greek symbol α (alpha).
variables in single direction. variables in either direction.
A parameter which is significant at 1% is also significant at
Result Greater or less than certain value. Greater or less than certain range
of values. 5 and 10%.
Sign in alternative hypothesis > or < ≠
Numerical example B) The Standard Normal Test

This test is based on the normal distribution. The test is
1. Supple that from a sample of size n = 30, we estimate
applicable if:
the following supply function.
The standard deviation of the population is known irrespective
𝑄 = 120 + 0.6𝑝 + 𝑒
of the sample size
𝑆𝐸: 1.7 (0.025)
The standard deviation of the population is unknown provided
 Test the significance of the slope parameter at 5% level that the sample size is sufficiently large (n>30).
of significance using the standard error test. It is true that most of the times the null and alternative
Solution: Estimated regression function is given by: hypotheses are mutually exclusive.
 𝑄 = 𝛽 + 𝛽 𝑃 ; 𝑄 = 120 + 0.6𝑃 If these conditions cannot be fulfilled, we apply the student’s
t-test.
 𝑆𝐸 𝛽 = 0.025; ⁄ 𝛽 = 0.3; 0.025 < 0.3
Accepting the null hypothesis means that rejecting the
 This implies that 𝑆𝐸 𝛽 < ⁄ 𝛽 . The implication is alternative hypothesis and rejecting the null hypothesis means
69
𝛽 By:isMmaru
statistically
S.(MA)
significant at 5% level of significance.
30/05/2023 70
accepting the alternative hypothesis.
By: Mmaru S.(MA) 30/05/2023
Cont’d… Cont’d….
One-Sample Z -Test The standard normal test or Z-test is outline as follows;
 A one-sample z test is used to check if there is a 1.Test the null hypothesis against the alternative hypothesis
difference between the sample mean and the population 2. Determine the level of significant. It is the probability of
mean when the population standard deviation is known. committing type I error, i.e. the probability of rejecting the null
The formula for the z test statistic is given as follows: hypothesis while it is true. It is common in applied econometrics
𝑍= to use 5% level of significance.
3. Determine the tabulated value of Z from the table.
 𝑋 is the sample mean, 4. Make decision.
 μis the population mean, If 𝑍 < 𝑍 , accept the null hypothesis and conclude that
 σ is the population standard deviation and the estimate is not statistically significant, while, If
 n is the sample size. 𝑍 > 𝑍 , reject the null hypothesis and conclude that the.
71 By: Mmaru S.(MA) 30/05/2023 72 By: Mmaru S.(MA) estimate is statistically significant. 30/05/2023
C) The Student t-Test Cont’d

Consider the simple linear regression model And compare this figure with the value from the student’s t
distribution with (n-2) degrees of freedom for a given
significance level .
If there no relationship between X and Y then the null hypothesis is  Decision rule: If
expressed as:
then we reject the null hypothesis, and conclude that there is
a significant relationship between X and Y.
The alternative hypothesis is that there is a significant relationship
 With t-test
between X and Y, that is,
 H0: b1 = 0
 H1: b1 ≠ 0 or Ha: b1 > 0 or Ha: b1 < 0
In order to reject or not reject the null hypothesis, we calculate the  We have two options:
test statistic given by: 1) Use Critical Region (CR) or p-value from SPSS/STATA
to reject H0
2) Use the t-value on the output and compare with the t-
73 By: Mmaru S.(MA) 30/05/2023 74 critical at n-k-1 degrees of freedom By: Mmaru S.(MA) 30/05/2023
How to interpret the t-statistic result? Example: t-test for individual  ’s

 1st we have to choose the appropriate level of  Hypothesis test, e.g. H0: j = 0.5 versus Ha: j ≠ 0.5,
significance α = either
Test Statistic: estimate  value under H0 j
 1%, which is 0.01 t 
SE SE (  j )
 5%, which is 0.05 or ˆ j  0.5 Value under H0
 10%, which is 0.1
t
se( ˆ j )
 Then compare the p-value from the output against the (k +1) β - parameters in the model
chosen level of α under H0 , test statistic follows a t-distribution with df =
 reject H0 if the p-value is less than the chosen level of α dfSSR = n – (k + 1).
 For instance, in our Stata output suggests that the H0 is Under H0, test statistic t follows a t-distribution with
reject even at 1% level.
75 By: Mmaru S.(MA) 30/05/2023 76 df=n – (k +1); By: Mmaru S.(MA) 30/05/2023
Example Cont’d…
1. Suppose that from a sample size n =20 we estimate the Step 2: Since the alternative hypothesis (𝐻 ) is
following consumption function. stated by inequality sign (≠), it is a two tail test,
𝐶 = 100 + 0.70 + 𝑒 hence we divide ⁄ = . ⁄ = 0.025 to obtain
75.5 (0.21)
the critical value of ‘t’ at ⁄ = 0.025 & 18 degree
Solution:
of freedom (df) i.e. (n-2=20-2).
 The values in the brackets are standard errors. We want to
test the null hypothesis: 𝐻 : 𝛽 = 0 against the • From the t-table 𝑡 at 0.025 level of significance
alternative: 𝐻 : 𝛽 ≠ 0 using the t-test at 5% level of & 18 df is 2.10.
significance. Step 3: Since t*=3.3 and 𝑡 = 2.1, 𝑡 ∗ > 𝑡 . It
Step one: The t-value for the test statistic is: implies that 𝛽 is statistically significant.
𝛽−0 𝛽 0.70
𝑡∗ = = = = 3.3
77 By: Mmaru S.(MA)
𝑆𝐸(𝛽 ) 𝑆𝐸(𝛽 ) 0.21 30/05/2023 78 By: Mmaru S.(MA) 30/05/2023
III. Confidence Interval Estimation of the regression Cont’d

Coefficients
 Confidence interval provides a range of values which are likely to  The confidence intervals are constructed in such a way that the
contain the true regression parameter. probability of the interval to contain the true parameter is
 Rejecting the null hypothesis does not mean that the parameter
estimates are correct estimates of the true population parameters.
Symbolically,
 It means that the estimate comes from the sample drawn from the
population whose population parameter is significantly different
from zero. In order to define the range within which the true
parameter lies, we must construct a confidence interval for the
parameter. Like we constructed confidence interval estimates for a
given population mean, using the sample mean, we can construct
100 1 − 𝛼 % confidence intervals for the sample regression
coefficients. To do so we need to have the standard errors of the
sample regression coefficients.
Cont’d… To construct a confidence interval:

 The limit within which the true 𝛽 lies at (1 − 𝛼)% • Confidence interval (C.I.) of level C gives information about
degree of confidence is: 𝛽 − 𝑆𝐸 𝛽 𝑡 , 𝛽 + 𝑆𝐸 𝛽 𝑡 : accuracy of estimate of β (population parameter)
 Where 𝑡 is the critical value of t at ⁄ confidence • Using the fact that the difference between the sample
interval and n -2 degree of freedom. estimate and the population estimate for the β has a t-
 The test procedure is outlined as follows. distribution (with n-k-1 degrees of freedom), we can
𝐻 :𝛽 = 0
𝐻 :𝛽 ≠ 0
construct a CI as;
• Generally:   
ˆ j  t / 2 , n  ( k 1)  se ˆ j
 Decision rule: If the hypothesized value of 𝛽 in the null – Lower limit: use ‘-’
hypothesis is within the confidence interval, accept 𝐻 – Upper limit: use ‘+’
and reject 𝐻 . The implication is that 𝛽 is statistically • C gives the degree of trust
insignificant; while if the hypothesized value of 𝛽 in the – When C = 95%, the chance that the C.I. contains the
null hypothesis is outside the limit, reject 𝐻 and accept unknown parameter β is 0.95
𝐻 . This indicates 𝛽 is statistically significant.
Numerical example Cont’d…

1. Suppose we have estimated the following Solution:
regression line from a sample of 20 observations. Step 1: The limit within which the true 𝛽 lies at 95%
𝑌 = 128.5 + 2.88𝑋 + 𝑒 confidence interval is 𝛽 ± 𝑆𝐸 𝛽 𝑡
38.2 (0.85) 𝛽 = 2.88, 𝑆𝐸 𝛽
• The values in the bracket are standard errors. = 0.85, & 𝑡 𝑎𝑡 0.025 𝑙𝑒𝑣𝑒𝑙 𝑜𝑓 𝑠𝑖𝑔𝑛𝑖𝑓𝑖𝑐𝑎𝑛𝑐𝑒 &
18 𝑑𝑒𝑔𝑟𝑒𝑒 𝑜𝑓 𝑓𝑟𝑒𝑒𝑑𝑜𝑚 𝑖𝑠 2.10
A. Construct 95% confidence interval for the slope
of parameter 𝛽 ± 𝑆𝐸 𝛽 𝑡 = 2.88 ± 2.10 0.85 = 2.88 ± 1.79
B. Test the significance of the slope parameter  The confidence interval is: (1.09, 4.67)
using constructed confidence interval. Step 2: The value of 𝛽 in the null hypothesis is zero w/c
implies it is outside the confidence interval. Hence 𝛽 is
statistically significant.
2.5 Prediction Cont’d

The simplest economic relationship is represented through a two Y = 0.6X + 120 functional relationship is deterministic
variable model (also called the simple linear regression model) or exact, that is, given income we can determine
which is given by: the exact expenditure of a household.
Y = a + bX But in reality this rarely happens: different households with
Example: Suppose the relationship between expenditure (Y) and the same income are not expected to spend equal amounts
income (X) of households is expressed as: due to habit persistence, geographical and time variation,
Y = 0.6X + 120 etc.Thus, we should express the regression model as:
Here, on the basis of income, we can predict expenditure. For  Y = a + bX + e
instance, if the income of a certain household is 1500 Birr, then
where e is the random error term (also called
the estimated expenditure will be: expenditure = 0.6(1500) +
disturbance term).
120 = 1020 Birr
2.6 Partial Correlation Coefficients & their Cont’d..

Interpretation  One possibility is to calculate the total amount of deviation
 How do we measure relationships?  But, in this case, the positive and negative deviations would
 In general; if there are a relationship between two variables, cancel out……..Then what????
then as one variable deviates from its mean, the other variable  Also, by simply adding the deviations, we would gain little
should deviate from its mean in the same or the directly insight into the relationship between the variables.
opposite way.  So, we can squared the deviations to eliminate the problem of
 When there is a very similar pattern of deviations for positive and negative deviations cancelling out each other.
both variables, we say the relationship is positive, if not,  Since we have two variables, rather than squaring each
negative deviation, we can for the second variable. multiply the
 So, how do we calculate the exact relationship between the deviation for one variable by the corresponding deviation.
87
pattern of differences or similarity of the two variables
By: Mmaru S.(MA) 30/05/2023 88
Cont’d… Cont’d..
 If both deviations are positive or negative then this will
give us a positive value (indicative of the deviations  On the other hand, a negative covariance indicates that as
being in the same direction), one variable deviates from the mean (e.g. increases), the
 But if one deviation is positive and one negative then other deviates from the mean in the opposite direction
the resulting product will be negative. (e.g. decreases).
Calculating the covariance is a good way to assess  There is, however, one problem with covariance as
whether two variables are related to each other. a measure of the relationship
 A positive covariance indicates that as one variable  That is, it depends upon the scales of measurement used.
deviates from the mean, the other variable deviates in  So, covariance is not a standardized measure.
the same direction.
89 90
 Thus, we have to express the covariance in a standard unit of
measurement.  By standardizing the covariance we end up with
 The standardized covariance is known as a correlation coefficient
and is defined as:
a value that has to lie between −1 and +1
 It is called Pearson’s Correlation Coefficient):
n
 1  rxy  1
Sxy x x y  y
i i n  So, if you find a correlation coefficient less than
r  i1
 xisx x  yisy y −1 or more than +1
SxxSyy n n
xi x  yi  y
2 2 i1
you can be sure that something has gone
i1 i1
hideously wrong!.
i.e.  1  rxy  1
91 92
How to interpret Correlation Coefficient???? How to interpret . . . . .

 Generally; correlation coefficient; So, what does the coefficient of +1 indicate?
measures the strength of the linear association  It indicates that the two variables are perfectly positively
between two quantitative variables correlated,
so as one variable increases, the other increases by a
Not affected by linear transformations of x or y
proportionate amount.
But affected by non-linear transformations, e.g.
 Conversely, a coefficient of −1 indicates a perfect
log(.) . . . . negative relationship: if one variable increases, the
Both variables can be interchanged (doesn’t other decreases by a proportionate amount.
indicate any causal relationship)  A coefficient of zero indicates no linear relationship at
all and so if one variable changes, the other stays the
93 94 same.
Hypothesis testing in correlation

Graphically
 Although we can directly interpret the size of a correlation coefficient, we
 ryx = rxy
have seen in previous chapter that scientists like to test hypotheses
 r -1≤ r ≤ 1 using probabilities.
 Correlation close to 1 in absolute value  strong linear
 In the case of a correlation coefficient we can test the hypothesis
association (positive or negative from sign)
that the correlation is different from zero (i.e. different from ‘no
 Values close to 0  little or no association
relationship’).
 If we find that our observed coefficient was very unlikely to
r = -1 y r=0 y
y happen, that indicates
r=1
 there was no effect in the population H0: r=0
x x x Ha: r≠0
 then we can gain confidence that the relationship that we have
observed is statistically meaningful.
96
Example n
 x  x  y  y
S xy i i n
 Suppose that the Ethiopian ministry of tourism expects that coffee sales r  i 1
   xisx x   yisy y 
S xx S yy n n
 x  x    y  y
2 2 i 1
rise when more adult visitors come by (see the table below) i 1
i
i 1
i
Returns of coffee (x) Number of adult visitors (y) x y

xi  x  xi  x  yi  y  xi  x 2  yi  y 
2
yi  y
35 29
35 29 14 -25 -338 182 625
21 35 21 35 21 35 735 441 1225
19 48 19 48 19 48 912 361 2304
23 85 23 85 1955 529 7225
23 85 7 75 7 76 529 49 5712
7 75 24 52 24 52 1248 576 2704
24 52 n
 x  x  yi i  y
 How to measure relationship between coffee sales (x) and number of r i 1
 0.579
n n
visitors (y)?  x  x    y  y 
2 2
97 98 i i
i 1 i 1
Interpretation Interpretation
 r = -0.5791
 Negative correlation between return of coffee and number
of adult visitors
 This means; the association between the two variables is
negative
 Strange!
 Why do you think this has happened?
 Have we controlled other things (other variables)?
 E.g. what if temperature is controlled (since the weather
condition may affect the visitors decision to come to
Ethiopia)
Cont’d… Generally
If we have more data measured (information on
 There is a type of correlation that can be done that allows
temperature), will the relationship between coffee sales
you to look at the relationship between two variables
and visitors held same?
when the effects of a third variable are held constant.
returns of coffee number of adult temperature
 For example, if I want to analyze your exam
(c) visitors (v) (t)
performance may be negatively related to exam
35 29 12
anxiety, but positively related to revision time,
21 35 14
 and revision time itself was negatively related to exam
19 48 17
23 85 26 anxiety.
7 75 24  This scenario is complex,
24 52 18
101 102
How to deal with this issue: Partial Correlation Definition

 In our previous example; given that we know that  Correlation in general, is a measure of linear dependence
revision time is related to both exam anxiety and (association) between 2 variables,
exam performance,  it does not necessarily imply any cause-and-effect
 then if we want a pure measure of the relationship relationship
between exam anxiety and exam performance we  Partial correlation: “is the correlation between 2
need to take account of the influence of revision variables where the effects of other variables are held constant
time. (controlled)”
 Therefore, we could conduct a partial correlation  In our example: removes the effect of temperature
between exam anxiety and exam performance while  Let’s see the output of that example after and before
controlling for the effect of temperature
‘controlling’ for the effect of revision time.
103 104
Case 1: when we do not control the effect of temperature Illustrative Example: 1. Given a data on hours studied and
grade on exam are correlated.
A. Find the estimated parameter's?
B. Draw estimated regression line?
C. Interpreted the coefficient
D. Suppose the number of hours studied is
3, what is the predicted grade on exam?
E. Coefficient of determination and
interpreter the result?
Case 2: when we control the effect of temperature F. Correlation coefficient
G. Conduct student t-test with 5 % level of
significance.
H. Construct 95% confidence interval for
the slope of parameter
I. Test the significance of the slope
parameter using constructed confidence
106 interval. 30/05/2023
By: Mmaru S.(MA)
Illustrative Example
2. The following results have been obtained from a
simple of 11 observations on the values of sales (Y) of a
firm and the corresponding prices (X).
𝑋 = 519.18, 𝑋 𝑌 = 1,296,836
𝑌 = 217.82,
𝑋 = 3134,543 𝑌 = 539,512
I. Estimate the regression line of sale on price and

interpret the results
II. What is the part of the variation in sales which is not
explained by the regression line?
107 III. Estimate the price elasticity of sales.30/05/2023
By: Mmaru S.(MA)

Lecture Two (Copy)

Uploaded by

Copyright:

Available Formats

Lecture Two (Copy)

Uploaded by

Document Information

Original Title

Copyright

Available Formats

Share this document

Share or Embed Document

Sharing Options

Did you find this document useful?

Is this content inappropriate?

Copyright:

Available Formats

Lecture Two (Copy)

Uploaded by

Copyright:

Available Formats

Econometrics 5/30/2023

2.1 Concept of Regression Function

1 By: Mmaru S.(MA) 30/05/2023 2 By: Mmaru S.(MA) 30/05/2023

Regression Vs Correlation? Regression Vs Correlation?

Examples of Correlation in Real Life

5 By: Mmaru S.(MA) 30/05/2023 6 By: Mmaru S.(MA) 30/05/2023

2.2 Simple linear regression (SLR) Linear Vs Non-linear regression

 In simple linear regression, the predictions of one

9 By: Mmaru S.(MA) 30/05/2023 10 By: Mmaru S.(MA) 30/05/2023

Nature of error term

Concept of PRF and SRF Concept of PRF and SRF

Concept of PRF and SRF 2.3. Assumption of Classical Linear

The main objective of regression analysis is the

17 By: Mmaru S.(MA) 30/05/2023 18 By: Mmaru S.(MA) 30/05/2023

19 By: Mmaru S.(MA) 30/05/2023 20 By: Mmaru S.(MA) 30/05/2023

 Fig 2.4(a): Homoscedasticity

 Fig 2.4(b): Heteroscedasticity

21 By: Mmaru S.(MA) 30/05/2023 22 By: Mmaru S.(MA) 30/05/2023

23 By: Mmaru S.(MA) 30/05/2023 24 By: Mmaru S.(MA) 30/05/2023

Cont’d… 2.4 Estimation Techniques

2.4.1 Method of Ordinary Least Squares (OLS

By partial differentiation of the SSE with respect toˆ and ˆ

 The estimating line will have a ‘good fit’ if it minimizes the

2.4.2 Statistical Properties of Least Square

Now we have: (since x is non-stochastic

Note that (**) follows from assumptions (3) and (4)

39 By: Mmaru S.(MA) 30/05/2023 40 By: Mmaru S.(MA) 30/05/2023

41 By: Mmaru S.(MA) 30/05/2023 42 By: Mmaru S.(MA) 30/05/2023

Cont’d Statistical inference in simple linear regression model

This implies that the variance of ̂ is the smallest as compared to

2.4.2 Statistical Test of Significance of the OLS

I. Test of model adequacy

 Total variation = explained variation due to regression +

Show this is zero

53 By: Mmaru S.(MA) 30/05/2023 54 By: Mmaru S.(MA) 30/05/2023

Cont’d ii) Testing the significance of a given regression coefficient

Cont’d… Error in statistical decision-making

One tail test or two tail test

Numerical example B) The Standard Normal Test

C) The Student t-Test Cont’d

How to interpret the t-statistic result? Example: t-test for individual  ’s

III. Confidence Interval Estimation of the regression Cont’d

Cont’d… To construct a confidence interval:

Numerical example Cont’d…

2.5 Prediction Cont’d

2.6 Partial Correlation Coefficients & their Cont’d..

How to interpret Correlation Coefficient???? How to interpret . . . . .

Hypothesis testing in correlation

Returns of coffee (x) Number of adult visitors (y) x y

How to deal with this issue: Partial Correlation Definition

I. Estimate the regression line of sale on price and

You might also like