Location via proxy:   [ UP ]  
[Report a bug]   [Manage cookies]                
Scribd Logo
Upload

Welcome to Scribd!

  • 200 views

    Multicollinearity Exercise

    Uploaded by

    Debaraj Sarkar
    This document summarizes output from a regression analysis examining factors that influence law school GPA. It finds evidence of multicollinearity between the independent variables based on variance inflation factors above 10 and condition indexes over 30. Ridge regression is used to address this by adding a constant to the diagonal of the correlation matrix. The smallest ridge constant value that reduces all VIFs below 10 is 0.005. Variable X4 has a coefficient with the "wrong" sign, which is corrected by removing the first principal component or using a ridge constant value of at least 0.001.

    Copyright:

    Attribution Non-Commercial (BY-NC)
    Download as DOC, PDF, TXT or read online from Scribd

    Multicollinearity Exercise

    Uploaded by

    Debaraj Sarkar
    200 views6 pages
    This document summarizes output from a regression analysis examining factors that influence law school GPA. It finds evidence of multicollinearity between the independent variables based on variance inflation factors above 10 and condition indexes over 30. Ridge regression is used to address this by adding a constant to the diagonal of the correlation matrix. The smallest ridge constant value that reduces all VIFs below 10 is 0.005. Variable X4 has a coefficient with the "wrong" sign, which is corrected by removing the first principal component or using a ridge constant value of at least 0.001.

    Original Description:

    Multicollinearity Exercise

    Copyright

    © Attribution Non-Commercial (BY-NC)

    Available Formats

    DOC, PDF, TXT or read online from Scribd

    Did you find this document useful?

    Is this content inappropriate?

    This document summarizes output from a regression analysis examining factors that influence law school GPA. It finds evidence of multicollinearity between the independent variables based on variance inflation factors above 10 and condition indexes over 30. Ridge regression is used to address this by adding a constant to the diagonal of the correlation matrix. The smallest ridge constant value that reduces all VIFs below 10 is 0.005. Variable X4 has a coefficient with the "wrong" sign, which is corrected by removing the first principal component or using a ridge constant value of at least 0.001.

    Copyright:

    Attribution Non-Commercial (BY-NC)
    Download as DOC, PDF, TXT or read online from Scribd
    Download as doc, pdf, or txt
    200 views6 pages

    Multicollinearity Exercise

    Uploaded by

    Debaraj Sarkar
    This document summarizes output from a regression analysis examining factors that influence law school GPA. It finds evidence of multicollinearity between the independent variables based on variance inflation factors above 10 and condition indexes over 30. Ridge regression is used to address this by adding a constant to the diagonal of the correlation matrix. The smallest ridge constant value that reduces all VIFs below 10 is 0.005. Variable X4 has a coefficient with the "wrong" sign, which is corrected by removing the first principal component or using a ridge constant value of at least 0.001.

    Copyright:

    Attribution Non-Commercial (BY-NC)
    Download as DOC, PDF, TXT or read online from Scribd
    Download as doc, pdf, or txt
    of 6

    Multicollinearity Exercise

    Use the attached SAS output to answer the questions. [OPTIONAL: Copy the SAS program
    below into the SAS editor window and run it.] Please dont print out all the output shown below
    or the from the SAS job if you decide to run it.
    1. Use at least three different methods to diagnose whether multicollinearity is a problem for
    this set of data.
    2. Identify which variables are key participants in the most serious near linear dependency in
    the data. (How do you know this?)
    3. Which variable has the wrong sign for its coefficient in this regression? Explain why its
    sign is wrong.
    4. What is the smallest value of the ridge constant (k) that fixes the sign of the coefficient you
    named in #3?
    5. What is the smallest value of the ridge constant (k) that reduces all VIFs so that they are
    below the guideline of 10?
    6. What is the smallest value of k that seems (in your opinion) to stabilize the coefficients?
    7. If one principle component is removed, give the estimated coefficients for X1, X2, X3, X4.
    Does this fix the one with the wrong sign?
    *******************************************************************
    ************
    LAW SCHOOL ADMISSION DATA
    ******************
    ****************
    PARTLY FROM PAGE 599 OF SMITH ***************
    *******************************************************************;
    **** DATA FOR 20 STUDENTS ******
    Y IS THE LAW SCHOOL GPA
    X1 IS THE UNDERGRADUATE SCHOOL GPA
    X2 IS THE LMAT PERCENTILE
    X3 IS A RATING OF THE UNDERGRADUATE SCHOOL QUALITY
    X4 IS THE GRE SCORE;
    DATA LAW;
    INPUT Y X1 X2 X3 X4 NO $; CARDS;
    3.42 3.28 .96 6
    1330
    1
    3.60 3.18 .97 7
    1370
    2
    3.28 2.89 .93 5
    1140
    3
    3.75 3.72 .99 8
    1520
    4
    3.36 3.18 .95 6
    1270
    5
    3.96 3.50 .98 8
    1450
    6
    3.31 3.04 .94 5
    1200
    7
    3.33 3.87 .95 5
    1340
    8
    3.60 3.54 .96 7
    1350
    9
    4.00 3.27 .99 10
    1480
    a
    3.28 3.30 .95 5
    1280
    b
    3.44 3.29 .91 7
    1080
    c
    3.25 3.17 .93 5
    1170
    d
    3.75 3.62 .97 8
    1410
    e
    3.30 3.34 .96 5
    1330
    f
    3.20 3.08 .90 4
    1010
    g
    3.50 3.37 .96 6
    1340
    h
    3.28 3.16 .94 5
    1220
    i
    3.17 3.20 .95 4
    1270
    j
    3.31 3.10 .94 5
    1210
    k
    ;
    TITLE 'LAW SCHOOL ADMISSIONS DATA';
    PROC CORR; VAR Y X1 X2 X3 X4;
    PROC REG; MODEL Y=X1 X2 X3 X4

    / COLLIN VIF;

    PROC REG RIDGE = 0 TO .01 BY .001 OUTEST=B;


    MODEL Y=X1 X2 X3 X4 ;
    PROC PRINT;
    PROC PLOT; PLOT (X1 X2 X3 X4) * _RIDGE_ / VREF=0 VPOS=25 HPOS=45;
    PROC REG DATA = LAW
    MODEL Y=X1 X2 X3 X4;
    PROC PRINT;

    RIDGE= 0 TO .01 BY .001 OUTEST=C OUTVIF;

    PROC REG DATA = LAW


    MODEL Y=X1 X2 X3 X4;

    PCOMIT=1 2 3 OUTEST=C;

    PROC PRINT;
    run;

    LAW SCHO OL ADMISSIONS DATA


    Correlation Analysis
    Pearson Correlation Coefficients/Prob > |R|under Ho: Rho=0 /N = 20
    Y

    X1

    X2

    X3

    X4

    1.00000
    0.0

    0.47331
    0.76094
    0.95925
    0.76574
    0.0350
    0.0001
    0.0001
    0.0001

    X1

    0.47331
    0.0350

    1.00000
    0.52911
    0.42078
    0.65377
    0.0
    0.0164
    0.0647
    0.0018

    X2

    0.76094
    0.0001

    0.52911
    0.0164

    1.00000
    0.69724
    0.98781
    0.0
    0.0006
    0.0001

    X3

    0.95925
    0.0001

    0.42078
    0.0647

    0.69724
    0.0006

    1.00000
    0.69983
    0.0
    0.0006

    X4

    0.76574
    0.0001

    0.65377
    0.0018

    0.98781
    0.0001

    0.69983
    1.00000
    0.0006
    0.0

    Model:MO DEL1
    Dependent Variable:Y
    Analysisof Variance
    Sum of
    Mean
    DF
    Squares
    Square

    Source
    Model
    Error
    C Total

    4
    15
    19

    1.07143
    0.07106
    1.14249

    Root MSE
    0.06883
    Dep Mean
    3.45450
    C.V.
    1.99243

    0.26786
    0.00474
    R-square
    Adj R-sq

    F Value

    Prob>F

    56.542

    0.0001

    0.9378
    0.9212

    Parameter Estimates
    Parameter
    Standard T for H0:
    Variable DF
    Estimate
    Error Parameter=0
    INTERCEP 1
    -2.378637 24.38266385
    X1
    1
    0.125719 0.64288464
    X2
    1
    6.058256 32.14872369
    X3
    1
    0.129773 0.01417076
    X4
    1
    -0.000878 0.00646192

    Prob > |T|

    -0.098
    0.9236
    0.196
    0.8476
    0.188
    0.8531
    9.158
    0.0001
    -0.136
    0.8937

    Variance
    Variable DF
    Inflation
    INTERCEP
    X1
    1
    X2
    1
    X3
    1
    X4
    1

    1 0.00000000
    96.67364263
    2280.9435770
    1.99014566
    2880.9838356

    CollinearityDiagnostics
    Condition Var Prop Var Prop Var Prop Var Prop Var Prop

    Nu mber Eigenvalue
    1
    4.95347
    2
    0.04096
    3
    0.00348
    4
    0.00209
    5 1.47569E-7

    OBS
    1
    2
    3
    4
    5
    6
    7
    8
    9
    10
    11
    12

    OBS

    _TYPE_
    PARMS
    RIDGE
    RIDGE
    RIDGE
    RIDGE
    RIDGE
    RIDGE
    RIDGE
    RIDGE
    RIDGE
    RIDGE
    RIDGE

    INTERCEP

    1 -2.37864
    2 -2.37864
    3
    0.89844
    4
    1.17880
    5
    1.28047
    6
    1.33140
    7
    1.36094
    8
    1.37947
    9
    1.39160
    10
    1.39968
    11
    1.40504
    12
    1.40850

    X2

    X3

    X4

    1.00000 0.0000 0.0000 0.0000 0.0012 0.0000


    10.99716 0.0000 0.0001 0.0000 0.5624 0.0000
    37.70759 0.0000 0.0022 0.0000 0.3141 0.0004
    48.66811 0.0000 0.0150 0.0000 0.1117 0.0005
    5794 1.0000 0.9827 1.0000 0.0106 0.9991

    _MODEL_
    MO DEL1
    MO DEL1
    MO DEL1
    MO DEL1
    MO DEL1
    MO DEL1
    MO DEL1
    MO DEL1
    MO DEL1
    MO DEL1
    MO DEL1
    MO DEL1

    Index INTERCEP X1

    X1

    0.12572
    0.12572
    0.03989
    0.03249
    0.02977
    0.02838
    0.02755
    0.02700
    0.02663
    0.02637
    0.02617
    0.02603

    _DEPVAR_
    Y
    Y
    Y
    Y
    Y
    Y
    Y
    Y
    Y
    Y
    Y
    Y

    _RIDGE_

    .
    0.000
    0.001
    0.002
    0.003
    0.004
    0.005
    0.006
    0.007
    0.008
    0.009
    0.010

    X2
    6.05826
    6.05826
    1.73602
    1.36524
    1.23008
    1.16182
    1.12178
    1.09627
    1.07920
    1.06748
    1.05935
    1.05373

    X3

    .
    .
    .
    .
    .
    .
    .
    .
    .
    .
    .
    .

    _PCOMIT_

    0.068829
    0.068829
    0.068871
    0.068880
    0.068886
    0.068892
    0.068899
    0.068907
    0.068915
    0.068925
    0.068935
    0.068947

    X4

    0.12977
    0.12977
    0.12932
    0.12907
    0.12883
    0.12859
    0.12836
    0.12813
    0.12790
    0.12767
    0.12744
    0.12721

    _RMSE_

    -.00087848
    -.00087848
    -.00000776
    0.00006863
    0.00009765
    0.00011320
    0.00012307
    0.00013001
    0.00013524
    0.00013938
    0.00014279
    0.00014568

    Plotof X1*_RIDGE_. Legend: A = 1 obs, B = 2 obs, etc.


    X1

    0.15

    0.10

    0.05
    A

    A A

    A A A A A A A

    0.00

    0.000 0.002 0.004 0.006 0.008 0.010

    -1
    -1
    -1
    -1
    -1
    -1
    -1
    -1
    -1
    -1
    -1
    -1

    Ridge regression controlvalue


    Plotof X2*_RIDGE_. Legend: A = 1 obs, B = 2 obs, etc.
    X2

    6 A

    2
    A

    A A

    A A A A A A A

    0.000 0.002 0.004 0.006 0.008 0.010


    Ridge regression controlvalue
    Plotof X3*_RIDGE_. Legend: A = 1 obs, B = 2 obs, etc.
    X3

    0.15

    A A

    A A A A A A A A A

    0.10

    0.000 0.002 0.004 0.006 0.008 0.010


    Ridge regression controlvalue

    Plotof X4*_RIDGE_. Legend: A = 1 obs, B = 2 obs, etc.


    X4

    0.00025

    A A A A A A A A

    A
    0 A

    -0.00025

    -0.0005

    -0.00075

    -0.001

    0.000 0.002 0.004 0.006 0.008 0.010


    Ridge regression controlvalue

    OBS
    1
    2
    3
    4
    5
    6
    7
    8
    9
    10
    11
    12
    13
    14
    15
    16
    17
    18
    19
    20
    21
    22
    23
    OBS
    1
    2
    3
    4
    5
    6
    7
    8
    9
    10
    11
    12
    13
    14
    15
    16
    17
    18
    19
    20
    21
    22
    23

    O
    B
    S
    1
    2
    3
    4

    _MODEL_ _TYPE_
    MO DEL1
    PARMS
    MO DEL1
    RIDGEVIF
    MO DEL1
    RIDGE
    MO DEL1
    RIDGEVIF
    MO DEL1
    RIDGE
    MO DEL1
    RIDGEVIF
    MO DEL1
    RIDGE
    MO DEL1
    RIDGEVIF
    MO DEL1
    RIDGE
    MO DEL1
    RIDGEVIF
    MO DEL1
    RIDGE
    MO DEL1
    RIDGEVIF
    MO DEL1
    RIDGE
    MO DEL1
    RIDGEVIF
    MO DEL1
    RIDGE
    MO DEL1
    RIDGEVIF
    MO DEL1
    RIDGE
    MO DEL1
    RIDGEVIF
    MO DEL1
    RIDGE
    MO DEL1
    RIDGEVIF
    MO DEL1
    RIDGE
    MO DEL1
    RIDGEVIF
    MO DEL1
    RIDGE

    _DEPVAR_ _RIDGE_ _PCOMIT_


    Y
    .
    .
    0.068829
    Y
    0.000
    .
    .
    Y
    0.000
    .
    0.068829
    Y
    0.001
    .
    .
    Y
    0.001
    .
    0.068871
    Y
    0.002
    .
    .
    Y
    0.002
    .
    0.068880
    Y
    0.003
    .
    .
    Y
    0.003
    .
    0.068886
    Y
    0.004
    .
    .
    Y
    0.004
    .
    0.068892
    Y
    0.005
    .
    .
    Y
    0.005
    .
    0.068899
    Y
    0.006
    .
    .
    Y
    0.006
    .
    0.068907
    Y
    0.007
    .
    .
    Y
    0.007
    .
    0.068915
    Y
    0.008
    .
    .
    Y
    0.008
    .
    0.068925
    Y
    0.009
    .
    .
    Y
    0.009
    .
    0.068935
    Y
    0.010
    .
    .
    Y
    0.010
    .
    0.068947

    _RMSE_

    INTERCEP
    X1
    X2
    X3
    X4
    Y
    -2.37864
    0.1257
    6.06 0.12977
    -0.00 -1
    .
    96.6736 2280.94 1.99015
    2880.98 -1
    -2.37864
    0.1257
    6.06 0.12977
    -0.00 -1
    .
    3.9053
    59.05 1.95543
    74.08 -1
    0.89844
    0.0399
    1.74 0.12932
    -0.00 -1
    .
    2.1861
    17.99 1.94548
    22.21 -1
    1.17880
    0.0325
    1.37 0.12907
    0.00 -1
    .
    1.8012
    8.89 1.93597
    10.72 -1
    1.28047
    0.0298
    1.23 0.12883
    0.00 -1
    .
    1.6537
    5.48 1.92660
    6.41 -1
    1.33140
    0.0284
    1.16 0.12859
    0.00 -1
    .
    1.5803
    3.84 1.91732
    4.34 -1
    1.36094
    0.0275
    1.12 0.12836
    0.00 -1
    .
    1.5373
    2.93 1.90811
    3.19 -1
    1.37947
    0.0270
    1.10 0.12813
    0.00 -1
    .
    1.5090
    2.37 1.89898
    2.48 -1
    1.39160
    0.0266
    1.08 0.12790
    0.00 -1
    .
    1.4887
    2.00 1.88992
    2.02 -1
    1.39968
    0.0264
    1.07 0.12767
    0.00 -1
    .
    1.4731
    1.74 1.88093
    1.70 -1
    1.40504
    0.0262
    1.06 0.12744
    0.00 -1
    .
    1.4606
    1.55 1.87201
    1.47 -1
    1.40850
    0.0260
    1.05 0.12721
    0.00 -1
    _ _
    _
    D _P
    M
    _ ERC _
    I
    O
    T P IO R
    N
    D
    Y VD M M
    T
    E
    P AG I S
    C
    L
    E RET E
    P
    X
    X
    X
    X
    _
    _ ___ _
    T
    1
    2
    3
    4
    Y
    MO DEL1 PARMS Y ..0.06883 -2.37864 0.12572 6.05826 0.12977 -.00087848 -1
    MO DEL1 IPC Y .1 0.06670 1.52761 0.02349 0.90751 0.12951 0.00015692 -1
    MO DEL1 IPC Y .2 0.10775 -0.66338 -0.13747 3.65788 0.06579 0.00053840 -1
    MO DEL1 IPC Y .3 0.13095 -0.76037 0.20866 2.78208 0.03571 0.00051382 -1

    You might also like