Unit Three: The Simple Regression Model

3.1.Introduction of Simple regression

Economic theories are mainly concerned with the relationships among various economic
variables. These relationships, when phrased in mathematical terms, can predict the effect of one
variable on another. The functional relationships of these variables define the dependence of one
variable upon the other variable (s) in the specific form. Regression analysis refers to estimating
functions showing the relationship between two or more variables and corresponding tests. It
includes estimating a simple linear function between two variables.

3.2.Ordinary Least Square Method (OLS) and Classical Assumptions

There are two major ways of estimating regression functions. These are the ordinary least square
method and maximum likelihood (MLH) method. Both the methods are basically similar to their
application in estimations that you may be aware of in statistics courses. The ordinary least
square method is the easiest and the most commonly used method as opposed to the maximum
likelihood (MLH) method which is limited by its assumptions. For instance, the MLH method is
valid only for large sample as opposed to the OLS method which can be applied to smaller
samples. Owing to this merit, our discussion mainly focuses on the ordinary least square (OLS).
The (Ordinary) least square (OLS) method of estimating parameters or regression function is
about finding or estimating values of the parameters (  and  ) of the simple linear regression
function given below for which the errors or residuals are minimized. Thus, it is about
minimizing the residuals or the errors.
Yi    X i  U i 2.5
The above identity represents population regression function (to be estimated from total
enumeration of data from the entire population). But, most of the time it is difficult to generate
population data owing to several reasons; and most of the time we use sample data and we
estimate sample regression function. Thus, we use the following sample regression function for
the derivation of the parameters and related analysis.
Yˆi  ˆ  ˆX i 2.6

1
 i) Classical Assumptions
For the validity of a regression function and its attributes the data we use or the terms related to
our regression function should fulfill the following conditions known as classical assumptions.
1. The error terms „Ui‟ are randomly distributed or the disturbance terms are not correlated.
This means that there is no systematic variation or relation among the value of the error
terms (Ui and Uj); Where i = 1, 2, 3, …….., j = 1, 2, 3, ……. and i  j .
Cov (Ui , Uj) = 0 for i  j . Note that the same argument holds for residual terms when
we use sample data or sample regression function. Thus, Cov (ei, ej) = 0 for i  j .
Otherwise, the error terms do not serve an adjustment purpose rather it causes an
autocorrelation problem.
2. The disturbance terms „Ui‟ have zero mean. This implies the sum of the individual
disturbance terms is zero. The deviations of the values of some of the disturbance terms are
negative; some are zero and some are positive and the sum or the average is zero. This is
given by the following identities.
U i
E(Ui) =  0 . Multiplying both sides by (sample size „n‟) we obtain the following.
n
 E(Ui) =  U i  0 . The same argument is true for sample regression function and so
for residual terms given as follows  ei 0
If this condition is not met, then the position of the regression function (or curve) will not
be the same as where it is supposed to be. This results in an upward (if the mean of the
error term or residual term is positive) or down ward (if the mean of the error term or
residual term is negative) shift in the regression function. For instance, suppose we have
the following regression function.
i   0  1  i  Ui
i   E (Yi / X i )   0  1  i   U i 
i   E (Yi / X i )   0  1 X i if E (U i )  0
Otherwise the estimated models will be biased and cause the regression function to shift. For
instance, if E (U i )  0 (or positive) it is going to shift the estimation upward from the true
representative model. Similar argument is true for residual term of sample regression
function. This is demonstrated by the following figure.

2
 Figure 1: Regression Function/curve if the mean of error term is not zero

3. The disturbance terms have constant variance in each period. This is given as follows:
Var U i   E ((U i  E (U i )) 2   =  u . This assumption is known as the
2 2

assumption of homoscedasticity. If this condition is not fulfilled or if the variance of the

error terms varies as sample size changes or as the value of explanatory variables changes,
then this leads to Heteroscedasticity problem.
4. Explanatory variables „Xi‟ and disturbance terms „Ui‟ are uncorrelated or independent. All
the co-variances of the successive values of the error term are equal to zero. This condition
is given by Cov U i , X i   0 . It is followed from this that the following identity holds
true;  ei X i  0 . The value in which the error term assumed in one period does not
depend on the value in which it assumed in any other period. If this condition is not met by
our data or variables, our regression function and conclusions to be drawn from it will be
invalid. This assumption is known as the assumption of non-autocorrelation or non-serial
correlation.
5. The explanatory variable Xi is fixed in repeated samples. Each value of Xi does not vary for
instance owing to change in sample size. This means the explanatory variables are non-
random and hence distributional free variable.
6. Linearity of the model in parameters. The simple linear regression requires linearity in
parameters; but not necessarily linearity in variables. The same technique can be applied to
estimate regression functions of the following forms: Y = f (X ); Y = f (X 2); Y = f (X 3);
Y = f (X – kX ); and so on. What is important is transforming the data as required.

3
 7. Normality assumption-The disturbance term Ui is assumed to have a normal distribution
with zero mean and a constant variance. This assumption is given as follows:
Ui ˜ 
N 0,  u
2
. This assumption is a combination of zero mean of error term
assumption and homoscedasticity assumption. This assumption or combination of
assumptions is used in testing hypotheses about significance of parameters. It is also useful
in both estimating parameters and testing their significance in maximum likelihood
method.
8. Explanatory variables should not be perfectly, linearly and/or highly correlated. Using
explanatory variables which are highly or perfectly correlated in a regression function
causes a biased function or model. It also results in multicollinearity problem.
9. The relationship between variables (or the model) is correctly specified. For instance all the
necessary variables are included in the model. The variables are in the form that best
describes the functional relationship. For instance, “Y = f (X 2)” may better reflect the
relationship between Y and X than “Y = f (X )”.
10. The explanatory variables do not have identical value. This assumption is very important
for improving the precision of estimators.
ii) OLS method of Estimation
Estimating a linear regression function using the Ordinary Least Square (OLS) method is simply
about calculating the parameters of the regression function for which the sum of square of the
error terms is minimized. The procedure is given as follows. Suppose we want to estimate the
following equation
Yi   0  1 X i  U i
Since most of the time we use sample (or it is difficult to get population data) the corresponding
sample regression function is given as follows.

Yˆi  ˆ0  ˆ1 X i

From this identity, we solve for the residual term ' ei ' , square both sides and then take sum of
both sides. These three steps are given (respectively as follows.

ei  Yi  Yˆi  Yi  ˆ0  ˆ1 X i 2.7

2

 ei   Yi  ˆ0  ˆ1 X i 
2
2.8

Where,  ei  RSS= Residual Sum of Squares.

2

The method of OLS involves finding the estimates of the intercept and the slope for which the
sum squares given by the Equation is minimized. To minimize the residual sum of squares we
take the first order partial derivatives of Equation 2.8 and equate them to zero.

4
That is, the partial derivative with respect to ˆ 0 :

  ei
 
2
 2  Yi  ˆ0  ˆX i (1)  0 2.9
ˆ
0

 Y i  ˆ0  ˆ1 X i  0  2.10

  Yi  nˆ0  ˆ1  X i =0 2.11

 Y i  nˆ0  ˆ1  X i 2.12

Where n is the sample size.

Partial derivative With respect to ˆ1

  ei
 
2
 2  Yi  ˆ0  ˆ1 X i ( X i )  0 2.13
ˆ 1

 Y Xi i  ˆ0 X i  ˆ1 X i2  0  2.14

  X i Yi  ˆ0  X i  ˆ1  X i
2
2.15

^ ^
Note that the equation (Yi   0  1 X i ) 2 is a composite function and we should apply a chain
rule in finding the partial derivatives with respect to the parameter estimates.
Equations 2.12 and 2.15 are together called the system of normal equations. Solving the system
of normal equations simultaneously we obtain:

n XY  ( X )( Y )
ˆ1 
Or
n X 2  ( X ) 2

_ _
  XY  n Y X and we have ˆ 0  Y  ˆ1 X
1  _
from above
 X i2  n X 2
Now we have the formula to estimate the simple linear regression function. Let us illustrate with
example.
Example 2.4: Given the following sample data of three pairs of „Y‟ (dependent variable) and „X‟
(independent variable), find a simple linear regression function; Y = f(X).

5
 Yi Xi
10 30
20 50
30 60

a) find a simple linear regression function; Y = f(X)

b) Interpret your result.
c) Predict the value of Y when X is 45.
Solution
a. To fit the regression equation we do the following computations.
Yi Xi Yi Xi Xi2
10 30 300 900
20 50 1000 2500
30 60 1800 3600
Sum 60 140 3100 7000
Mean Y = 20 140
X =
3

n XY  ( X )( Y )
3(3100)  (140)(60)
ˆ1    0.64
3(7000)  (140) 2
n X 2  ( X ) 2

ˆ0  Y  ˆ1 X  20  0.64(140 / 3)  10

Thus the fitted regression function is given by: Yˆi   10  0.64 X i

b) Interpretation, the value of the intercept term,-10, implies that the value of the dependent
variable „Y‟ is – 10 when the value of the explanatory variable is zero. The value of the
slope coefficient ( ˆ  0.64 ) is a measure of the marginal change in the dependent
variable „Y‟ when the value of the explanatory variable increases by one. For instance, in
this model, the value of „Y‟ increases on average by 0.64 units when „X‟ increases by
one.

c) Yˆi   10  0.64 X i =-10+(0.64)(45)=18.8

That means when X assumes a value of 45, the value of Y on average is expected to be 18.8.
The regression coefficients can also be obtained by simple formulae by taking the deviations
between the original values and their means. Now, if
xi  X i  X , and yi  Yi  Y

6
 x y
i i

Then, the coefficients can be represented by alternative formula given below ˆ1 
x i
2

, and ˆ0  Y  ˆ1 X

Example 2.5: Find the regression equation for the data under Example 2.4, using the shortcut
formula. To solve this problem we proceed as follows.
Yi Xi y x xy x2 y2
10 30 -10 -16.67 166.67 277.78 100
20 50 0 3.33 0.00 11.11 0
30 60 10 13.33 133.33 177.78 100
Sum 60 140 0 0 300.00 466.67 200
Mean 20 46.66667

Then

x i yi
300 , and ˆ0  Y  ˆ1 X =20-(0.64) (46.67) = -10 with results
ˆ1    0.64
466.67
x i
2

similar to previous case.

Mean and Variance of Parameter Estimates
We have seen that how the numerical values of the parameter estimates can be obtained using
OLS estimating techniques. Now let us see their distributional nature, i.e. the mean and variance
of the parameter estimates. There can be several samples of the same size that can be drawn from
the same population. For each sample, the parameter estimates have their own specific numerical
values. That means the values of the estimates are different when we go from one sample to
another. Therefore, the parameter estimate have different values for estimating a given true
population parameter. That means the parameter estimates are random in their nature and should
have distinct distribution with the corresponding parameters. Remember we have discussed in
the previous sections that both the error term and the dependent variable are assumed to be
normally distributed. Thus, the parameter estimates also have a normal distribution with their
associative mean and variance. Formula for mean and variance of the respective parameter
estimates and the error term are given below (procedure to drive is given in Annex A)
 
1. The mean of 1  E (1 )  1
     U2
2. The variance of 1  Var ( 1 )  E ((1  E ( )) 2 
 xi2

7
   
3. The mean of  0  E (  0 )   0
    U2  X i2
4. The variance of  0  E (( 0  E ( 0 )) 2 
n  xi2

 ei2
5. The estimated value of the variance of the error term  U 
2

n2

Hypothesis Testing of OLS Estimates

After estimation of the parameters there are important issues to be considered by the researcher.
We have to know that to what extent our estimates are reliable enough and acceptable for further
purpose. That means, we have to evaluate the degree of representativeness of the estimate to the
true population parameter. Simply a model must be tested for its significance before it can be
used for any other purpose. In this subsection we will evaluate the reliability of model estimated
using the procedure we explained above.
The available test criteria are divided in to three groups: Theoretical a priori criteria, statistical
criteria and econometric criteria. Priori criteria set by economic theories are in line with the
consistency of coefficients of econometric model to the economic theory. Statistical criteria, also
known as first order tests, are set by statistical theory and refer to evaluate the statistical
reliability of the model. Econometric criteria refer to whether the assumptions of an econometric
model employed in estimating the parameters are fulfilled or not. There are two most commonly
used tests in econometrics. These are:
1. The square of correlation coefficient ( r 2 ) which is used for judging the explanatory power of
the linear regression of Y on X or on X‟s. The square of correlation coefficient in simple
regression is known as the coefficient of determination and is given by R 2 . The coefficient of
determination measures the goodness of fit of the line of regression on the observed sample
values of Y and X.
2. The standard error test of the parameter estimates applied for judging the statistical reliability
of the estimates. This test measures the degree of confidence that we may attribute to the
estimates.
i) The Coefficient of determination (R2)
The coefficient of determination is the measure of the amount or proportion of the total variation
of the dependent variable that is determined or explained by the model or the presence of the
explanatory variable in the model. The total variation of the dependent variable is split in two
additive components; a part explained by the model and a part represented by the random term.
The total variation of the dependent variable is measured from its arithmetic mean.
_
Total var iationin Yi  (Yi  Y ) 2
 _
Total exp lained var iation  (Yi  Y ) 2
Total un exp lained var iation   ei2

8
The total variation of the dependent variable is given in the following form; TSS=ESS + RSS,
which means total sum of square of the dependent variable is split into explained sum of square
and residual sum of square.

ei  yi  y i

yi  yi  ei
 2 
yi2  yi  ei2  2 yi ei
2 
 y i2   y i   ei2  2  y i ei

But  y i ei  0
2
Therefore,  y   y i   ei2
2
i

The coefficient of determination is given by the formula

 (Yˆ  Y )2  yˆ
2
i i

R2   
Explained Variation in Y 2.16
Total Variation in Y
 (Y i  Y )2 y i
2

 2 
Since  y i   1  x i y i the coefficient of determination can also be given as

 1  xi y i
R2 
 y i2

Or
e e t t 1 ˆ
P 2.17
e
2
t 1

The higher the coefficient of determination is the better the fit. Conversely, the smaller the
coefficient of determination is the poorer the fit. That is why the coefficient of determination is
used to compare two or more models. One minus the coefficient of determination is called the
coefficient of non-determination, and it gives the proportion of the variation in the dependent
variable that remained undetermined or unexplained by the model.
ii) Testing the significance of a given regression coefficient
Since the sample values of the intercept and the coefficient are estimates of the true population
parameters, we have to test them for their statistical reliability.
The significance of a model can be seen in terms of the amount of variation in the dependent
variable that it explains and the significance of the regression coefficients.
There are different tests that are available to test the statistical reliability of the parameter
estimates. The following are the common ones;

9
 A) The standard error test
B) The standard normal test
C) The students t-test
Now, let us discuss them one by one.
A) The Standard Error Test
This test first establishes the two hypotheses that are going to be tested which are commonly
known as the null and alternative hypotheses. The null hypothesis addresses that the sample is
coming from the population whose parameter is not significantly different from zero while the
alternative hypothesis addresses that the sample is coming from the population whose parameter
is significantly different from zero. The two hypotheses are given as follows:
H0: βi=0
H1: βi≠0
The standard error test is outlined as follows:
1. Compute the standard deviations of the parameter estimates using the above formula for
variances of parameter estimates. This is because standard deviation is the positive square root of
the variance.
  U2
se( 1 ) 
 xi2
  U2  X i2
se(  0 ) 
n  xi2
2. Compare the standard errors of the estimates with the numerical values of the estimates and
make decision.
A) If the standard error of the estimate is less than half of the numerical value of the estimate, we
 1 
can conclude that the estimate is statistically significant. That is, if se(  i )  (  i ) , reject the
2
null hypothesis and we can conclude that the estimate is statistically significant.
B) If the standard error of the estimate is greater than half of the numerical value of the estimate,
 1 
the parameter estimate is not statistically reliable. That is, if se(  i )  (  i ) , conclude to accept
2
the null hypothesis and conclude that the estimate is not statistically significant.
B) The Standard Normal Test
This test is based on the normal distribution. The test is applicable if:
 The standard deviation of the population is known irrespective of the sample size
 The standard deviation of the population is unknown provided that the sample size is
sufficiently large (n>30).
The standard normal test or Z-test is outline as follows;
10
 1. Test the null hypothesis H 0 :  i  0 against the alternative hypothesis H1 : i  0

2. Determine the level of significant (  ) in which the test is carried out. It is the probability
of committing type I error, i.e. the probability of rejecting the null hypothesis while it is
true. It is common in applied econometrics to use 5% level of significance.
3. Determine the theoretical or tabulated value of Z from the table. That is, find the value of
Z 2 from the standard normal table. Z 0.025  1.96 from the table.
4. Make decision. The decision of statistical hypothesis testing consists of two decisions;
either accepting the null hypothesis or rejecting it.
If Z cal  Z tab , accept the null hypothesis while if Z cal  Z tab , reject the null hypothesis. It

is true that most of the times the null and alternative hypotheses are mutually exclusive.
Accepting the null hypothesis means that rejecting the alternative hypothesis and
rejecting the null hypothesis means accepting the alternative hypothesis.
 
Example: If the regression has a value of 1 =29.48 and the standard error of 1 is 36. Test the
hypothesis that the value of 1  25 at 5% level of significance using standard normal test.
Solution: We have to follow the procedures of the test.
H 0 : 1  25
H1 : 1  25
After setting up the hypotheses to be tested, the next step is to determine the level of significance
in which the test is carried out. In the above example the significance level is given as 5%.
The third step is to find the theoretical value of Z at specified level of significance. From the
standard normal table we can get that Z 0.025  1.96 .
The fourth step in hypothesis testing is computing the observed or calculated value of the
standard normal distribution using the following formula.
ˆ1  1 29.48  25
Z cal    0.12 . Since the calculated value of the test statistic is less than the
se( ˆ1 ) 36
tabulated value, the decision is to accept the null hypothesis and conclude that the value of the
parameter is 25.
C) The Student t-Test
In conditions where Z-test is not applied (in small samples), t-test can be used to test the
statistical reliability of the parameter estimates. The test depends on the degrees of freedom that
the sample has. The test procedures of t-test are similar with that of the z-test. The procedures are
outlined as follows;
1. Set up the hypothesis. The hypotheses for testing a given regression coefficient is given by:

11
 H 0 : i  0
H1 :  i  0
2. Determine the level of significance for carrying out the test. We usually use a 5% level
significance in applied econometric research.
3. Determine the tabulated value of t from the table with n-k degrees of freedom, where k is the
number of parameters estimated.
4. Determine the calculated value of t. The test statistic (using the t- test) is given by:
ˆi
tcal 
se( ˆi )
The test rule or decision is given as follows:
Reject H0 if | t cal | t / 2,nk
iii) Confidence Interval Estimation of the regression Coefficients
We have discussed the important tests that that can be conducted to check model and parameters
validity. But one thing that must be clear is that rejecting the null hypothesis does not mean that
the parameter estimates are correct estimates of the true population parameters. It means that the
estimate comes from the sample drawn from the population whose population parameter is
significantly different from zero. In order to define the range within which the true parameter
lies, we must construct a confidence interval for the parameter. Like we constructed confidence
interval estimates for a given population mean, using the sample mean (in Introduction to
Statistics), we can construct 100(1- ) % confidence intervals for the sample regression
coefficients. To do so we need to have the standard errors of the sample regression coefficients.
The standard error of a given coefficient is the positive square root of the variance of the
coefficient. Thus, we have discussed that the formulae for finding the variances of the regression
coefficients are given as.

X
2
i

Variance of the intercept is given by var( ˆ0 )   u

2
2.18
n xi
2

1
Variance of the slope ( ˆ1 ) is given by var( ˆ1 )   u
2
2.19
x
2
i

e
2
i

Where, u2  (2.20) is the estimate of the variance of the random term and k is
nk
the number of parameters to be estimated in the model. The standard errors are the positive
square root of the variances and the 100 (1- ) % confidence interval for the slope is given by:

12
    
1  t (n  k )(se( 1 ))  1  1  t (n  k )(se( 1 ))
2 2

1  ˆ1  t / 2,n k ( se( ˆ1 )) 2.21

And for the intercept:
 0  ˆ0  t / 2,n k ( se( ˆ 0 )) 2.22
Example 2.6: The following table gives the quantity supplied (Y in tons) and its price (X pound
per ton) for a commodity over a period of twelve years.
Table 1: Data on supply and price for given commodity
Y 69 76 52 56 57 77 58 55 67 53 72 64
X 9 12 6 10 9 10 7 8 12 6 11 8

13
Table 2: Data for computation of different parameters
Time Y X XY X2 Y2 x Y Xy x2 y2 Yˆ ei ei2
1 69 9 621 81 4761 0 6 0 0 36 63.00 6.00 36.00
2 76 12 912 144 5776 3 13 39 9 169 72.75 3.25 10.56
3 52 6 312 36 2704 -3 -11 33 9 121 53.25 -1.25 1.56
4 56 10 560 100 3136 1 -7 -7 1 49 66.25 -10.25 105.06
5 57 9 513 81 3249 0 -6 0 0 36 63.00 -6.00 36.00
6 77 10 770 100 5929 1 14 14 1 196 66.25 10.75 115.56
7 58 7 406 49 3364 -2 -5 10 4 25 56.50 1.50 2.25
8 55 8 440 64 3025 -1 -8 8 1 64 59.75 -4.75 22.56
9 67 12 804 144 4489 3 4 12 9 16 72.75 -5.75 33.06
10 53 6 318 36 2809 -3 -10 30 9 100 53.25 -0.25 0.06
11 72 11 792 121 5184 2 9 18 4 81 69.50 2.50 6.25
12 64 8 512 64 4096 -1 1 -1 1 1 59.75 4.25 18.06
Sum 756 108 6960 1020 48522 0 0 156 48 894 756.00 0.00 387.00

Use Tables (Table 1 and Table 2) to answer the following questions

1. Estimate the Coefficient of determination (R2)
2. Run significance test of regression coefficients using the following test methods
A) The standard error test
B) The students t-test
3. Fit the linear regression equation and determine the 95% confidence interval for the slope.
Solution
1. Estimate the Coefficient of determination (R2)
Refer to Example 2.6 above to determine how much percent of the variations in the quantity
supplied is explained by the price of the commodity and what percent remained unexplained.
Use data in Table 2 to estimate R2 using the formula given below.

e
2
i
387
R2  1  1  1  0.43  0.57
894
y i
2

This result shows that 57% of the variation in the quantity supplied of the commodity under
consideration is explained by the variation in the price of the commodity; and the rest 37%
remain unexplained by the price of the commodity. In other word, there may be other important
explanatory variables left out that could contribute to the variation in the quantity supplied of the
commodity, under consideration.

14
 2. Run significance test of regression coefficients using the following test methods
Fitted regression line for the data given is:

Yi  33.75  3.25 X i , where the numbers in parenthesis are standard errors of the respective
(8.3) (0.9)
coefficients.
A. Standard Error test
In testing the statistical significance of the estimates using standard error test, the following
information needed for decision.
Since there are two parameter estimates in the model, we have to test them separately.

Testing for 1
  
We have the following information about 1 i.e 1 =3.25 and se( 1 )  0.9
The following are the null and alternative hypotheses to be tested.
H 0 : 1  0
H 1 : 1  0
 
Since the standard error of 1 is less than half of the value of 1 , we have to reject the null

hypothesis and conclude that the parameter estimate 1 is statistically significant.

Testing for  0

Again we have the following information about  0
 
 0  33.75 and se(  0 )  8.3
The hypotheses to be tested are given as follows;
H0 : 0  0
H1 :  0  0
 
Since the standard error of  0 is less than half of the numerical value of  0 , we have to reject the

null hypothesis and conclude that  0 is statistically significant.
B. The students t-test
In the illustrative example, we can apply t-test to see whether price of the commodity is
significant in determining the quantity supplied of the commodity under consideration? Use
=0.05.
The hypothesis to be tested is:

15
 H 0 : 1  0
H 1 : 1  0

The parameters are known. ˆ1  3.25, se(ˆ1 )  0.8979

Then we can estimate tcal as follows
ˆi 3.25
t cal    3.62
se( ˆi ) 0.8979
Further tabulated value for t is 2.228. When we compare these two values, the calculated t is
greater than the tabulated value. Hence, we reject the null hypothesis. Rejecting the null
hypothesis means, concluding that the price of the commodity is significant in determining the
quantity supplied for the commodity.
In this part we have seen how to conduct the statistical reliability test using t-statistic. Now let us
see additional information about this test. When the degrees of freedom is large, we can conduct
t-test without consulting the t-table in finding the theoretical value of t. This rule is known as “2t-
rule”. The rule is stated as follows;
The t-table shows that the values of t changes very slowly if the degrees of freedom (n-k) are
greater than 8. For example the value of t0.025 changes from 2.30 (when n-k=8) to 1.96(when n-
k=∞). The change from 2.30 to 1.96 is obviously very slow. Consequently, we can ignore the
degrees of freedom (when they are greater than 8) and say that the theoretical value of tcal is 2.0.
Thus, a two tail test of a null hypothesis at 5% level of significance can be reduced to the
following rules.
1. If t cal is greater than 2 or less than -2, we reject the null hypothesis
2. If tcal is less than 2 or greater than -2, accept the null hypothesis.
3. Fit the linear regression equation and determine the 95% confidence interval for the slope.


Fitted regression model is indicated Yi  33.75  3.25 X i ,where the numbers in parenthesis are
(8.3) (0.9)
standard errors of the respective coefficients. To estimate confidence interval we need standard
error which is determined as follows

e
2
i
387 387
u2     38.7
nk 12  2 10
1 1
var( ˆ1 )   u  38.7( )  0.80625
2

48
x 2

The standard error of the slope is se( ˆ1 )  var( ˆ1 )  0.80625  0.8979

16
The tabulated value of t for degrees of freedom 12-2=10 and /2=0.025 is 2.228.
Hence the 95% confidence interval for the slope is given by:
ˆ1  3.25  (2.228)(0.8979)  3.25  2  3.25  2, 3.25  2  1.25, 5.25 . The result tells us that
at the error probability 0.05, the true value of the slope coefficient lies between 1.25 and 5.25

Properties of OLS Estimators

The ideal or optimum properties that the OLS estimates possess may be summarized by well
known theorem known as the Gauss-Markov Theorem.
Statement of the theorem: “Given the assumptions of the classical linear regression model, the
OLS estimators, in the class of linear and unbiased estimators, have the minimum variance, i.e.
the OLS estimators are BLUE.
According to this theorem, under the basic assumptions of the classical linear regression model,
the least squares estimators are linear, unbiased and have minimum variance (i.e. are best of all
linear unbiased estimators). Sometimes the theorem referred as the BLUE theorem i.e. Best,
Linear, Unbiased Estimator. An estimator is called BLUE if:
a. Linear: a linear function of the random variable, such as, the dependent variable Y.
b. Unbiased: its average or expected value is equal to the true population parameter.
c. Minimum variance: It has a minimum variance in the class of linear and unbiased
estimators. An unbiased estimator with the least variance is known as an efficient
estimator.
According to the Gauss-Markov theorem, the OLS estimators possess all the BLUE properties.
The detailed proof of these properties are presented in Annex B

Extensions of Regression Models

As pointed out earlier non linearity may be expected in many Economic Relationships. In other
words the relationship between Y and X can be non-linear rather than linear. Thus, once the
independent variables have been identified the next step is to choose the functional form of the
relationship between the dependent and the independent variables. Specification of the functional
form is important, because a correct explanatory variable may well appear to be insignificant or
to have an unexpected sign if an inappropriate functional form is used. Thus the choice of a
functional form for an equation is a vital part of the specification of that equation. The choice of
a functional form almost always should be based on an examination of the underlying economic
theory. The logical form of the relationship between the dependent variable and the independent
variable in question should be compared with the properties of various functional forms, and the
one that comes closest to that underlying theory should be chosen for the equation.
Some Commonly Used Functional Forms
a) The Linear Form: It is based on the assumption that the slope of the relationship between the
independent variable and the dependent variable is constant.

17
 Y
 i i=1,2,...K
X

Y
X => Y
 K
X

In this case elasticity is not constant.

Y / Y Y X X
NY ,X     i
X / X X Y Y
If the hypothesized relationship between Y and X is such that the slope of the relationship can be
expected to be constant and the elasticity can therefore be expected to be variable, then the linear
functional form should be used.
Note: Economic theory frequently predicts only the sign of a relationship and not its functional
form. Under such circumstances, the linear form can be used until strong evidence that it is
inappropriate is found. Thus, unless theory, common sense, or experience justifies using some
other functional form, one should use the linear model.
b) Log-linear, double Log or constant elasticity model
The most common functional form that is non-linear in the variable (but still linear in the
coefficients) is the log-linear form. A log-linear form is often used, because the elasticities and
not the slopes are constant i.e.,  =  Constant.

Output

Input

18
Thus, given the assumption of a constant elasticity, the proper form is the exponential (log-
linear) form.

Given: Yi   0 X i i eU i
The log-linear functional form for the above equation can be obtained by a logarithmic
transformation of the equation.

ln Yi  ln  0   i ln X i  U i
The model can be estimated by OLS if the basic assumptions are fulfilled.
demand gd(log f)

ln Yi  ln  0  1 ln X i
 1
Yi   0 X i

price log f price

The model is also called a constant elasticity model because the coefficient of elasticity between
Y and X (1) remains constant.
Y X d ln Y
   1
X Y d ln X
This functional form is used in the estimation of demand and production functions.
Note: We should make sure that there are no negative or zero observations in the data set before
we decide to use the log-linear model. Thus log-linear models should be run only if all the
variables take on positive values.
c) Semi-log Form
The semi-log functional form is a variant of the log-linear equation in which some but not all of
the variables (dependent and independent) are expressed in terms of their logs. Such models
expressed as:
( i ) Yi   0   1 ln X 1i  U i ( lin-log model ) and ( ii ) ln Yi   0   1 X 1i  U i ( log-lin
model ) are called semi-log models. The semi-log functional form, in the case of taking the log
of one of the independent variables, can be used to depict a situation in which the impact of X on
Y is expected to „tail off‟ as X gets bigger as long as 1 is greater than zero.

19
 y

1<0
Y=0+1Xi

1>0

Example: The Engel‟s curve tends to flatten out, because as incomes get higher, a smaller
percentage of income goes to consumption and a greater percentage goes to saving.
 Consumption thus increases at a decreasing rate.
 Growth models are examples of semi-log forms
d) Polynomial Form
Polynomial functional forms express Y as a function of independent variables some of which are
raised to powers others than one. For example in a second degree polynomial (quadratic)
equation, at least one independent variable is squared.
Y   0  1 X 1i   2 X 1i   3 X 2i  U i
2

Such models produce slopes that change as the independent variables change. Thus the slopes of
Y with respect to the Xs are
Y Y
  1  2  2 X 1 , and  3
X 1 X 2
In most cost functions, the slope of the cost curve changes as output changes.

20
 Y Y

A) B)

X
Xi Impact of age on earnings
a typical cost curve

Simple transformation of the polynomial could enable us to use the OLS method to estimate the
parameters of the model
X1  X 3
2
Setting
 Y   0   1 X 1i   2 X 3   3 X 2i  U i
e) Reciprocal Transformation (Inverse Functional Forms)
The inverse functional form expresses Y as a function of the reciprocal (or inverse) of one or
more of the independent variables (in this case X1):
1
Yi   0  1 ( )   2 X 2i  U i
X 1i
Or
1
Yi   0  1 ( )   2 X 2i  U i
X 1i

The reciprocal form should be used when the impact of a particular independent variable is
expected to approach zero as that independent variable increases and eventually approaches
infinity. Thus as X1 gets larger, its impact on Y decreases.

1 0  0
Y  0 
X 1i 1  0

0

1 0  0
Y  0 
X 1i 1  0

21
An asymptote or limit value is set that the dependent variable will take if the value of the X-
variable increases indefinitely i.e. 0 provides the value in the above case. The function
approaches the asymptote from the top or bottom depending on the sign of 1.
Example: Phillips curve, a non-linear relationship between the rate of unemployment and the
percentage wage change.
1
Wt   0  1 ( ) Ut
Ut

22