Location via proxy:   [ UP ]  
[Report a bug]   [Manage cookies]                
Scribd Logo
Upload

Welcome to Scribd!

  • 205 views

    Econometrics: Two Variable Regression: The Problem of Estimation

    Uploaded by

    sajidalirafique
    This document provides an overview of the method of ordinary least squares (OLS) for estimating parameters in a linear regression model. It discusses how OLS minimizes the sum of squared residuals to produce optimal linear estimators with desirable statistical properties. It also outlines the assumptions required for OLS, such as exogeneity of regressors and homoscedasticity. The document defines key metrics like the variance of OLS estimators, the coefficient of determination, and sample correlation coefficient. It establishes that OLS produces the best linear unbiased estimators under the classical linear regression model.

    Copyright:

    Attribution Non-Commercial (BY-NC)
    Download as PDF, TXT or read online from Scribd

    Econometrics: Two Variable Regression: The Problem of Estimation

    Uploaded by

    sajidalirafique
    205 views28 pages
    This document provides an overview of the method of ordinary least squares (OLS) for estimating parameters in a linear regression model. It discusses how OLS minimizes the sum of squared residuals to produce optimal linear estimators with desirable statistical properties. It also outlines the assumptions required for OLS, such as exogeneity of regressors and homoscedasticity. The document defines key metrics like the variance of OLS estimators, the coefficient of determination, and sample correlation coefficient. It establishes that OLS produces the best linear unbiased estimators under the classical linear regression model.

    Original Title

    Topic 4

    Copyright

    © Attribution Non-Commercial (BY-NC)

    Available Formats

    PDF, TXT or read online from Scribd

    Did you find this document useful?

    Is this content inappropriate?

    This document provides an overview of the method of ordinary least squares (OLS) for estimating parameters in a linear regression model. It discusses how OLS minimizes the sum of squared residuals to produce optimal linear estimators with desirable statistical properties. It also outlines the assumptions required for OLS, such as exogeneity of regressors and homoscedasticity. The document defines key metrics like the variance of OLS estimators, the coefficient of determination, and sample correlation coefficient. It establishes that OLS produces the best linear unbiased estimators under the classical linear regression model.

    Copyright:

    Attribution Non-Commercial (BY-NC)
    Download as PDF, TXT or read online from Scribd
    Download as pdf or txt
    205 views28 pages

    Econometrics: Two Variable Regression: The Problem of Estimation

    Uploaded by

    sajidalirafique
    This document provides an overview of the method of ordinary least squares (OLS) for estimating parameters in a linear regression model. It discusses how OLS minimizes the sum of squared residuals to produce optimal linear estimators with desirable statistical properties. It also outlines the assumptions required for OLS, such as exogeneity of regressors and homoscedasticity. The document defines key metrics like the variance of OLS estimators, the coefficient of determination, and sample correlation coefficient. It establishes that OLS produces the best linear unbiased estimators under the classical linear regression model.

    Copyright:

    Attribution Non-Commercial (BY-NC)
    Download as PDF, TXT or read online from Scribd
    Download as pdf or txt
    of 28

    ECONOMETRICS

    Two Variable Regression: The


    Problem of Estimation

    Muhammad Arsalan Hashmi


    College of Management Sciences
    PAF KIET
    The Method of Ordinary Least
    Squares
     The method of OLS was developed Carl F Gauss.
     The estimators obtained from using this method has
    very desirable statistical properties.
     The OLS method depends on the least square
    principle, which is as follows:
    Suppose we have the following SRF to estimate the
    PRF
    The Method of OLS (Continued)
     Diagrammatically, the estimates of ui are as
    follows:
    The Method of OLS (Continued)
     An intuitively appealing criterion is to
    minimise the sum of residuals, i.e.
     However, this criterion has several
    limitations. For e.g. all the residuals get equal
    weight despite their size.
     To avoid these problems, we minimise the
    sum of squared residuals, , where
    The Method of OLS (Continued)

     To compute least squares estimators β^1 and


    β^2 , we begin by partially differentiating the
    preceding expression wrt β^1 and β^2

     These simultaneous equations are called


    normal equations.
    The Method of OLS (Continued)
     Solving the normal equations simultaneously,
    gives us the estimators of β^1 and β^2 for the
    simple regression model.
    Assumptions Underlying the
    Method of OLS

     If our objective is to perform statistical inference in


    addition to estimation, we must make some
    assumptions about how Xi (the explanatory
    variable) and ui (the disturbance term) are
    generated.
     The classical linear regression model is based on
    ten assumptions:
    1. Linear regression model: The regression model is
    linear in the parameters
    Yi = β1 + β2Xi + ui
    Assumptions Underlying the
    Method of OLS (Continued)
    2. X values are fixed in repeated sampling: That is,
    X is assumed to be non-stochastic.
    3. Zero mean of the disturbance term, ui: That is,
    the conditional mean of ui is zero.
    E(ui|Xi) = 0
    4. Homoscedasticity or equal variance of ui: That
    is, the conditional variance of ui are identical.
    Var(ui|Xi) = E[ui - E(ui|Xi)]2
    = E(u2i|Xi)
    = σ2
    Assumptions Underlying the
    Method of OLS (Continued)

     Diagrammatic depiction of homoscedasticity


    Assumptions Underlying the
    Method of OLS (Continued)
    5. No autocorrelation between the
    disturbances:
    Given two X values, Xi and Xj (where i≠j), the
    correlation between ui and uj (where i≠j) is
    zero.
    Assumptions Underlying the
    Method of OLS (Continued)
    6. Zero covariance between ui and Xi:

    7. The number of observations n must be greater than the


    number of parameters to be estimated.
    8. Variability in X values: The X values in a given sample must not
    all be the same.
    9. The regression model is correctly specified
    10. There is no perfect multicollinearity: That is, there is no
    perfect linear relationships among the explanatory variables.
    Variances & Standard Errors of
    Least Square Estimates
     As the least square estimates of parameters
    change from sample to sample, we need
    some measure of reliability (or precision) of
    the estimators β^1 and β^2.
     The precision of an estimate is measured by
    its standard error.
     The standard error is merely the standard
    deviation of the sampling distribution of the
    estimator.
    Variances & Standard Errors of Least
    Square Estimates (Continued)

     For a bivariate regression model, it can be


    shown that the variances and standard errors
    of β^1 and β^2 are as follows:

    (See Appendix 3A for details)


    Variances & Standard Errors of Least
    Square Estimates (Continued)

     Moreover, the unbiased estimator of σ2 (the conditional


    variance of ui), is

     An unbiased estimator of the standard error (se) is

    NB: The (n-2) term denotes the degrees of freedom (df).


    Variances & Standard Errors of Least
    Square Estimates (Continued)

    Prominent features of variances (and standard


    errors) of estimators β^1 and β^2.
     Given σ2, the larger the variation in X values,
    the smaller the variance of β^2. Also given
    the larger the variance σ2, the larger the
    variance of β^2
     The variance of β^1 is directly proportional to
    σ2 and and inversely proportional to n and
    Variances & Standard Errors of Least
    Square Estimates (Continued)

     The dependence of β^1 and β^2 on each other


    is measured by their covariance
    Gauss-Markov Theorem
     If the assumptions of CLRM are satisfied, the OLS
    estimators are BLUE (Best Linear Unbiased
    Estimator). For e.g. if we have a regression as
    follows:

    and the assumptions of CLRM are valid, then the


    estimator β^2 is BLUE.
    NB: See Appendix 3A for proof of the Gauss-Markov
    theorem.
    Gauss-Markov Theorem
    (Continued)
    Note
     We have been considering the finite (small)
    sample properties of estimators.
     It is important to distinguish between finite-
    sample properties and asymptotic properties
    of estimators.
    Coefficient of Determination: r2
     The coefficient of determination r2 is a
    summary measure that tells how well the
    sample regression line fits the data.
     We compute r2 as follows:

     Summing and squaring on both sides gives


    Coefficient of Determination: r2
    (Continued)
     The former expression can be written as

    TSS = ESS + RSS

    That is, the total variation in the observed Y


    values about their mean can be decomposed
    into two parts, one attributable to the
    regression line and the other to random
    forces.
    Coefficient of Determination: r2
    (Continued)
    Coefficient of Determination: r2
    (Continued)
     As before,
    TSS = ESS + RSS
    Divide both sides with TSS
    1= ESS + RSS
    TSS TSS

    Where r2 = ESS =
    TSS
    Coefficient of Determination: r2
    (Continued)
     Alternatively, r2 = 1 – RSS =
    TSS

     Essentially, r2 measures the proportion or


    percentage of the total variation in Y explained by
    the regression model.
     Two properties of r2
    a. It is a non-negative quantity
    b. r2 lies between 0 and 1, i.e. 0≤ r2 ≤1
    Sample Coefficient of Correlation
     A measure of linear association between two
    variables is the coefficient of correlation.
     Do not confuse the coefficient of correlation
    with the coefficient of determination.
     One may compute the sample correlation
    coefficient, r, in the following ways

     OR
    Properties of the Sample Coefficient
    of Correlation
     It can be positive or negative.
     r lies between -1 and +1, i.e. -1≤ r ≤1
     It is symmetrical
     If two variables X and Y are independent, r=0.
    However, r=0 does not necessarily imply that the
    variables are independent.
     It is a measure of linear association only.
     Although a measure of linear association between
    two variables, it does not imply a cause-and-effect
    relationship.
    So far
     We have studied the estimation aspect of the
    simple regression model in detail.
     The other major component of classical
    statistical inference is hypothesis testing.
     Before delving into hypothesis testing, we
    need to consider the probability distributions
    of β^1, β^2 and other related issues.
    Reading
     Gujarati, D (2003), “Basic Econometrics” 4th
    Edition, McGraw Hill. Chapter 3.

    You might also like