Lecture 4:

Simple Linear
Regression -
Part III

Prof. Amany
E. Aly
Lecture 4: Simple Linear Regression - Part III
Comparing
Models: The
Analysis of
Variance

The F-Test for

Regression
Prof. Amany E. Aly
Interpreting
p−values Professor of Statistics, Department of Mathematics, Faculty of Science,
The Helwan University, Ain Helwan, Cairo, Egypt.
Coefficient of
Determina-
tion,
R2 4 / 3 /2024
Confidence
Intervals and
Tests
The Intercept
Slope

Prof. Amany E. Aly ( Professor of Statistics, Lecture

Department
4: Simple
of Mathematics,
Linear Regression
Faculty- of
Part
Science,
III Helwan University,
4 / 3Ain
/2024
Helwan, Cairo,
1 / 58 Eg
 Presentation Outline

Lecture 4:
Simple Linear
Regression -
1 Comparing Models: The Analysis of
Part III

Prof. Amany
Variance
E. Aly

Comparing 2 The F-Test for Regression

Models: The
Analysis of
Variance

The F-Test for

3 Interpreting p−values
Regression

Interpreting
p−values
4 The Coefficient of Determination, R2
The
Coefficient of
Determina-
5 Confidence Intervals and Tests
tion,
R2 The Intercept
Confidence
Intervals and
Tests Slope
The Intercept
Slope

Prof. Amany E. Aly ( Professor of Statistics, Lecture

Department
4: Simple
of Mathematics,
Linear Regression
Faculty- of
Part
Science,
III Helwan University,
4 / 3Ain
/2024
Helwan, Cairo,
3 / 58 Eg
 Lecture 4:
Simple Linear
Regression -
Part III

Prof. Amany
E. Aly

Comparing
Models: The
The analysis of variance provides a
Analysis of
Variance convenient method of comparing the fit
The F-Test for
Regression
of two or more mean functions for the
Interpreting
p−values

The
same set of data.
Coefficient of
Determina-
tion,
R2

Confidence
Intervals and
Tests
The Intercept
Slope

Prof. Amany E. Aly ( Professor of Statistics, Lecture

Department
4: Simple
of Mathematics,
Linear Regression
Faculty- of
Part
Science,
III Helwan University,
4 / 3Ain
/2024
Helwan, Cairo,
4 / 58 Eg
 I: Best line parallel to the x−axis.

Lecture 4:
Simple Linear
Regression -
Part III
Finding best line parallel to the horizontal
Prof. Amany
E. Aly or x−axis.
Comparing
Models: The
Analysis of
Variance
An elementary alternative to the simple
The F-Test for
Regression

Interpreting
regression model suggests fitting the mean
p−values

The
function
Coefficient of
Determina-
tion,
R2 E(Y |X = x) = β0 (2.13)
Confidence
Intervals and
Tests
The Intercept
Slope

Prof. Amany E. Aly ( Professor of Statistics, Lecture

Department
4: Simple
of Mathematics,
Linear Regression
Faculty- of
Part
Science,
III Helwan University,
4 / 3Ain
/2024
Helwan, Cairo,
5 / 58 Eg
 Lecture 4:
Simple Linear
Regression -
Part III

Prof. Amany
The ols estimate of the mean function is
E. Aly

Comparing
b |X) = βb0,
E(Y
Models: The
Analysis of
Variance

The F-Test for

Regression
where βb0 is the value of β0 that minimizes
Interpreting
p−values
n
X
The
Coefficient of
Determina-
tion,
(yi − β0)2
R2
i=1
Confidence
Intervals and
Tests
The Intercept
Slope

Prof. Amany E. Aly ( Professor of Statistics, Lecture

Department
4: Simple
of Mathematics,
Linear Regression
Faculty- of
Part
Science,
III Helwan University,
4 / 3Ain
/2024
Helwan, Cairo,
6 / 58 Eg
 Lecture 4:
Simple Linear
Regression -
In this case
Part III n
X
(yi − β0)2,
Prof. Amany
E. Aly RSS(β0) =
Comparing
Models: The i=1
Analysis of
Variance so
The F-Test for n
Regression D RSS(β0) X
Interpreting =2 (yi − β0)(−1) = 0
p−values

The
D β0 i=1
Coefficient of
Determina-
tion,
R2
i.e. n
Confidence
X
Intervals and
Tests
yi = n β0 ⇒ βb0 = y
The Intercept
Slope i=1
Prof. Amany E. Aly ( Professor of Statistics, Lecture
Department
4: Simple
of Mathematics,
Linear Regression
Faculty- of
Part
Science,
III Helwan University,
4 / 3Ain
/2024
Helwan, Cairo,
7 / 58 Eg
 Lecture 4:
Simple Linear
Regression -
The residual sum of squares is
Part III

Prof. Amany Xn
(yi − β0)2
E. Aly

Comparing
RSS(β0) =
Models: The
Analysis of i=1
Variance
Xn
The F-Test for
Regression

Interpreting
= (yi − y)2
p−values
i=1
The
Coefficient of
Determina- = SY Y (2.14)
tion,
R2

Confidence
Intervals and
RSS has n − 1 degree of freedom, df ,
Tests
The Intercept
Slope
= n− (no of parameters to be estimated.)
Prof. Amany E. Aly ( Professor of Statistics, Lecture
Department
4: Simple
of Mathematics,
Linear Regression
Faculty- of
Part
Science,
III Helwan University,
4 / 3Ain
/2024
Helwan, Cairo,
8 / 58 Eg
 II: Best line of arbitrary slope.

Lecture 4:
Simple Linear
Regression - Finding best line of arbitrary slope.
Part III

E(Y |X = x) = β0 + β1 x
Prof. Amany
E. Aly (2.15)
Comparing
Models: The
Analysis of
Variance

The F-Test for

n
X
[yi − (β0 + β1 x)]2
Regression

Interpreting
RSS(β0, β1) =
p−values

The
i=1
Coefficient of
Determina- SXY
tion,
R2 βb1 =
Confidence SXX
Intervals and
Tests
The Intercept
βb0 = y − βb1 x
Slope

Prof. Amany E. Aly ( Professor of Statistics, Lecture

Department
4: Simple
of Mathematics,
Linear Regression
Faculty- of
Part
Science,
III Helwan University,
4 / 3Ain
/2024
Helwan, Cairo,
9 / 58 Eg
 Lecture 4:
Simple Linear
Regression -
Part III

Prof. Amany
RSS has n − 2 degree of freedom, df ,
E. Aly
= n− (no of parameters to be estimated.)
Comparing
Models: The
Analysis of
Variance (SXY )2
The F-Test for RSS(β0, β1) = SY Y − (2.16)
Regression
SXX
Interpreting
p−values

= SY Y − βb12 SXX
The
Coefficient of
Determina-
tion,
R2

Confidence
Intervals and
Tests
The Intercept
Slope

Prof. Amany E. Aly ( Professor of Statistics, Lecture

Department
4: Simple
of Mathematics,
Linear Regression
Faculty- of
Part
Science,
III Helwan University,
4 / 3 Ain
/2024
Helwan,10
Cairo,
/ 58 Eg
 Lecture 4:
Simple Linear
Regression -
Part III

Prof. Amany
E. Aly

Comparing
Models: The
Analysis of
Variance

The F-Test for

Regression

Interpreting
p−values

The
Coefficient of
Determina-
tion,
R2

Confidence
Intervals and
Tests
The Intercept
Slope

Prof. Amany E. Aly ( Professor of Statistics, Lecture

Department
4: Simple
of Mathematics,
Linear Regression
Faculty- of
Part
Science,
III Helwan University,
4 / 3 Ain
/2024
Helwan,11
Cairo,
/ 58 Eg
 Lecture 4:
Simple Linear
Regression -
Part III

Prof. Amany
The estimates of β0 under the two mean
E. Aly

Comparing
functions are different, just as the meaning
Models: The
Analysis of
Variance
of β0 in the two mean functions is
The F-Test for
Regression
different.
Interpreting
p−values For (2.13), β0 is the average of the yis,
The
Coefficient of
Determina-
tion,
For (2.15), β0 is the expected value of
R2

Confidence
Y when X = 0.
Intervals and
Tests
The Intercept
Slope

Prof. Amany E. Aly ( Professor of Statistics, Lecture

Department
4: Simple
of Mathematics,
Linear Regression
Faculty- of
Part
Science,
III Helwan University,
4 / 3 Ain
/2024
Helwan,12
Cairo,
/ 58 Eg
 Lecture 4:
Simple Linear
Regression -
Part III

Prof. Amany
E. Aly The difference between the sum of squares
Comparing
Models: The
at (2.14) and that at (2.16)
Analysis of
Variance
is the reduction in
The F-Test for
Regression

Interpreting
residual sum of squares due to enlarging
p−values

The
the mean function from (2.13) to the
Coefficient of
Determina-
tion, simple regression mean function (2.15).
R2

Confidence
Intervals and
Tests
The Intercept
Slope

Prof. Amany E. Aly ( Professor of Statistics, Lecture

Department
4: Simple
of Mathematics,
Linear Regression
Faculty- of
Part
Science,
III Helwan University,
4 / 3 Ain
/2024
Helwan,13
Cairo,
/ 58 Eg
 Lecture 4:
Simple Linear
Regression -
Part III The sum of squares due to regression,
Prof. Amany
E. Aly SSreg , defined by
Comparing
Models: The
Analysis of
Variance
SSreg = SY Y − RSS
(SXY )2
 
The F-Test for
Regression
= SY Y − SY Y −
Interpreting
p−values SXX
(SXY )2
The
Coefficient of
Determina-
tion, =
R2

Confidence
SXX
Intervals and
Tests
The Intercept
Slope

Prof. Amany E. Aly ( Professor of Statistics, Lecture

Department
4: Simple
of Mathematics,
Linear Regression
Faculty- of
Part
Science,
III Helwan University,
4 / 3 Ain
/2024
Helwan,14
Cairo,
/ 58 Eg
 Lecture 4:
Simple Linear
Regression -
Part III

Prof. Amany
E. Aly
The df for SSreg is
Comparing
Models: The
Analysis of
(n − 1) − (n − 2) = 1
Variance

The F-Test for

Regression for simple regression.
Interpreting
p−values

The
Coefficient of
Determina-
tion,
R2

Confidence
Intervals and
Tests
The Intercept
Slope

Prof. Amany E. Aly ( Professor of Statistics, Lecture

Department
4: Simple
of Mathematics,
Linear Regression
Faculty- of
Part
Science,
III Helwan University,
4 / 3 Ain
/2024
Helwan,15
Cairo,
/ 58 Eg
 Lecture 4:
Simple Linear
Regression -
Part III
The ANOVA is always computed relative
Prof. Amany
E. Aly
to a specific larger mean function, here
Comparing
Models: The
Analysis of
given by (2.15), and a smaller mean
Variance

The F-Test for function (2.13).

Regression

Interpreting
p−values
The smaller one is obtained from the
The
Coefficient of
larger by setting some parameters to zero,
Determina-
tion,
R2
or occasionally setting them to some other
Confidence
Intervals and
Tests
known value.
The Intercept
Slope

Prof. Amany E. Aly ( Professor of Statistics, Lecture

Department
4: Simple
of Mathematics,
Linear Regression
Faculty- of
Part
Science,
III Helwan University,
4 / 3 Ain
/2024
Helwan,16
Cairo,
/ 58 Eg
 ANOVA table

Lecture 4:
Simple Linear
Regression -
Part III

Prof. Amany
E. Aly

Comparing
Models: The
Analysis of
Variance

The F-Test for

Regression

Interpreting
p−values

The
Coefficient of
Determina-
tion,
R2

Confidence
Intervals and
Tests
The Intercept
Slope

Prof. Amany E. Aly ( Professor of Statistics, Lecture

Department
4: Simple
of Mathematics,
Linear Regression
Faculty- of
Part
Science,
III Helwan University,
4 / 3 Ain
/2024
Helwan,17
Cairo,
/ 58 Eg
 ANOVA table for Forbes’s data

Lecture 4:
Simple Linear
Regression -
Part III

Prof. Amany
E. Aly

Comparing
Models: The
Analysis of
Variance

The F-Test for

Regression

Interpreting
p−values

The
Coefficient of
Determina-
tion,
R2

Confidence
Intervals and
Tests
The Intercept
Slope

Prof. Amany E. Aly ( Professor of Statistics, Lecture

Department
4: Simple
of Mathematics,
Linear Regression
Faculty- of
Part
Science,
III Helwan University,
4 / 3 Ain
/2024
Helwan,18
Cairo,
/ 58 Eg
 Presentation Outline

Lecture 4:
Simple Linear
Regression -
1 Comparing Models: The Analysis of
Part III

Prof. Amany
Variance
E. Aly

Comparing 2 The F-Test for Regression

Models: The
Analysis of
Variance

The F-Test for

3 Interpreting p−values
Regression

Interpreting
p−values
4 The Coefficient of Determination, R2
The
Coefficient of
Determina-
5 Confidence Intervals and Tests
tion,
R2 The Intercept
Confidence
Intervals and
Tests Slope
The Intercept
Slope

Prof. Amany E. Aly ( Professor of Statistics, Lecture

Department
4: Simple
of Mathematics,
Linear Regression
Faculty- of
Part
Science,
III Helwan University,
4 / 3 Ain
/2024
Helwan,20
Cairo,
/ 58 Eg
 Lecture 4:
Simple Linear
Regression -
Part III
If the sum of squares for regression SSreg
Prof. Amany
E. Aly
is large, then the simple regression mean
Comparing
Models: The
function
Analysis of
Variance

The F-Test for E(Y |X = x) = β0 + β1x

Regression

Interpreting
p−values

The
should be a significant improvement over
Coefficient of
Determina-
tion,
the mean function given by
R2

Confidence
Intervals and
Tests
E(y|X = x) = β0.
The Intercept
Slope

Prof. Amany E. Aly ( Professor of Statistics, Lecture

Department
4: Simple
of Mathematics,
Linear Regression
Faculty- of
Part
Science,
III Helwan University,
4 / 3 Ain
/2024
Helwan,21
Cairo,
/ 58 Eg
 Lecture 4:
Simple Linear
Regression -
Part III
We need to be able to judge how large is
Prof. Amany
E. Aly ”large”. This is done by comparing the
Comparing
Models: The
regression mean square, SSreg divided by
its df , to the residual mean square σ 2.
Analysis of
Variance

The F-Test for

Regression

Interpreting
We call this ratio F .
p−values

The
Coefficient of
Determina-
tion, (SY Y − RSS)/1 SSreg /1
R2 F = =
Confidence
Intervals and σb2 σb2
Tests
The Intercept
Slope

Prof. Amany E. Aly ( Professor of Statistics, Lecture

Department
4: Simple
of Mathematics,
Linear Regression
Faculty- of
Part
Science,
III Helwan University,
4 / 3 Ain
/2024
Helwan,22
Cairo,
/ 58 Eg
 Lecture 4:
Simple Linear
Regression -
Part III
Formally, we can consider testing the null
Prof. Amany
E. Aly
hypothesis (N H) against the alternative
Comparing hypothesis (AH)
Models: The
Analysis of
Variance

The F-Test for

Regression

Interpreting
p−values
N H : E(Y |X = x) = β0
The
Coefficient of
Determina-
tion,
R2 AH : E(Y |X = x) = β0 + β1 x
Confidence
Intervals and
Tests
The Intercept
Slope

Prof. Amany E. Aly ( Professor of Statistics, Lecture

Department
4: Simple
of Mathematics,
Linear Regression
Faculty- of
Part
Science,
III Helwan University,
4 / 3 Ain
/2024
Helwan,23
Cairo,
/ 58 Eg
 Lecture 4:
Simple Linear
Regression -
Part III If the errors are N (0, σ 2) or the sample
Prof. Amany
E. Aly
size is large enough, then under N H, F
Comparing
Models: The
Analysis of
will follow an F −distribution with df
Variance

The F-Test for

associated with the numerator and
Regression

Interpreting denominator of F , 1 and n − 2 for simple

p−values

The
Coefficient of
regression. This is written
Determina-
tion,
R2

Confidence
F v F (1, n − 2).
Intervals and
Tests
The Intercept
Slope

Prof. Amany E. Aly ( Professor of Statistics, Lecture

Department
4: Simple
of Mathematics,
Linear Regression
Faculty- of
Part
Science,
III Helwan University,
4 / 3 Ain
/2024
Helwan,24
Cairo,
/ 58 Eg
 Lecture 4:
Simple Linear
Regression -
Part III

Prof. Amany
E. Aly

Comparing
Models: The
For Forbes’ data, we compute
Analysis of
Variance

The F-Test for 425.639

Regression
F = = 2963
Interpreting
p−values 0.144
The
Coefficient of
Determina-
tion,
R2

Confidence
Intervals and
Tests
The Intercept
Slope

Prof. Amany E. Aly ( Professor of Statistics, Lecture

Department
4: Simple
of Mathematics,
Linear Regression
Faculty- of
Part
Science,
III Helwan University,
4 / 3 Ain
/2024
Helwan,25
Cairo,
/ 58 Eg
 Lecture 4:
Simple Linear
Regression -
Part III

Prof. Amany
E. Aly

Comparing
Models: The
We obtain a significance level or p−value
Analysis of
Variance for this test by comparing F to the
The F-Test for
Regression percentage points of the
Interpreting
p−values

The
F (1, n − 2)-distribution.
Coefficient of
Determina-
tion,
R2

Confidence
Intervals and
Tests
The Intercept
Slope

Prof. Amany E. Aly ( Professor of Statistics, Lecture

Department
4: Simple
of Mathematics,
Linear Regression
Faculty- of
Part
Science,
III Helwan University,
4 / 3 Ain
/2024
Helwan,26
Cairo,
/ 58 Eg
 Lecture 4:
Simple Linear
Regression -
Part III

Prof. Amany
E. Aly The p−value is shown as approximately
Comparing
Models: The zero, meaning that, if the N H were true,
Analysis of
Variance

The F-Test for

the change of F exceeding its observed
Regression

Interpreting
value is essentially zero.
p−values

The This is very strong evidence against N H

Coefficient of
Determina-
tion,
R2
and in favor of AH.
Confidence
Intervals and
Tests
The Intercept
Slope

Prof. Amany E. Aly ( Professor of Statistics, Lecture

Department
4: Simple
of Mathematics,
Linear Regression
Faculty- of
Part
Science,
III Helwan University,
4 / 3 Ain
/2024
Helwan,27
Cairo,
/ 58 Eg
 Presentation Outline

Lecture 4:
Simple Linear
Regression -
1 Comparing Models: The Analysis of
Part III

Prof. Amany
Variance
E. Aly

Comparing 2 The F-Test for Regression

Models: The
Analysis of
Variance

The F-Test for

3 Interpreting p−values
Regression

Interpreting
p−values
4 The Coefficient of Determination, R2
The
Coefficient of
Determina-
5 Confidence Intervals and Tests
tion,
R2 The Intercept
Confidence
Intervals and
Tests Slope
The Intercept
Slope

Prof. Amany E. Aly ( Professor of Statistics, Lecture

Department
4: Simple
of Mathematics,
Linear Regression
Faculty- of
Part
Science,
III Helwan University,
4 / 3 Ain
/2024
Helwan,29
Cairo,
/ 58 Eg
 Lecture 4:
Simple Linear
Regression -
Part III

Prof. Amany
E. Aly A small p−value provides evidence against
Comparing
Models: The
the N H.
Analysis of
Variance

The F-Test for

Regression

Interpreting
In some research areas, it has become
p−values

The
traditional to adopt a fixed significance
Coefficient of
Determina-
tion, level when examining p−values.
R2

Confidence
Intervals and
Tests
The Intercept
Slope

Prof. Amany E. Aly ( Professor of Statistics, Lecture

Department
4: Simple
of Mathematics,
Linear Regression
Faculty- of
Part
Science,
III Helwan University,
4 / 3 Ain
/2024
Helwan,30
Cairo,
/ 58 Eg
 Lecture 4:
Simple Linear
Regression -
Part III

Prof. Amany
E. Aly For example, if a fixed significance level of
Comparing
Models: The α is adopted, then we would say that an
Analysis of
Variance

The F-Test for

NH is rejected at level α if the p−value is
Regression

Interpreting
less than α.
p−values

The
Coefficient of
Determina-
tion,
R2
The most common choice for α is 0.05.
Confidence
Intervals and
Tests
The Intercept
Slope

Prof. Amany E. Aly ( Professor of Statistics, Lecture

Department
4: Simple
of Mathematics,
Linear Regression
Faculty- of
Part
Science,
III Helwan University,
4 / 3 Ain
/2024
Helwan,31
Cairo,
/ 58 Eg
 Lecture 4:
Simple Linear
Regression -
Part III

Prof. Amany
E. Aly

Comparing
Models: The
Which would mean that, were the N H to
Analysis of
Variance be true, we would incorrectly find evidence
The F-Test for
Regression
against it about 5% of the time, or about
Interpreting
p−values

The
1 test in 20.
Coefficient of
Determina-
tion,
R2

Confidence
Intervals and
Tests
The Intercept
Slope

Prof. Amany E. Aly ( Professor of Statistics, Lecture

Department
4: Simple
of Mathematics,
Linear Regression
Faculty- of
Part
Science,
III Helwan University,
4 / 3 Ain
/2024
Helwan,32
Cairo,
/ 58 Eg
 Presentation Outline

Lecture 4:
Simple Linear
Regression -
1 Comparing Models: The Analysis of
Part III

Prof. Amany
Variance
E. Aly

Comparing 2 The F-Test for Regression

Models: The
Analysis of
Variance

The F-Test for

3 Interpreting p−values
Regression

Interpreting
p−values
4 The Coefficient of Determination, R2
The
Coefficient of
Determina-
5 Confidence Intervals and Tests
tion,
R2 The Intercept
Confidence
Intervals and
Tests Slope
The Intercept
Slope

Prof. Amany E. Aly ( Professor of Statistics, Lecture

Department
4: Simple
of Mathematics,
Linear Regression
Faculty- of
Part
Science,
III Helwan University,
4 / 3 Ain
/2024
Helwan,34
Cairo,
/ 58 Eg
 Lecture 4:
Simple Linear
Regression -
Part III

Prof. Amany
E. Aly
The coefficient of determination, R2, is
Comparing
defined by:
Models: The
Analysis of
Variance SSreg
The F-Test for R2 =
Regression

Interpreting
SY Y
p−values RSS
The = 1−
Coefficient of
Determina-
tion,
SY Y
R2

Confidence
Intervals and
Tests
The Intercept
Slope

Prof. Amany E. Aly ( Professor of Statistics, Lecture

Department
4: Simple
of Mathematics,
Linear Regression
Faculty- of
Part
Science,
III Helwan University,
4 / 3 Ain
/2024
Helwan,35
Cairo,
/ 58 Eg
 Lecture 4:
Simple Linear
Regression -
Part III

Prof. Amany
For Forbes data,
E. Aly

SSreg 425.63910
Comparing
Models: The R2 = = = 0.995
Analysis of
Variance SY Y 427.79402
The F-Test for
Regression

Interpreting
p−values
Thus about 99.5% of the variability in the
The
Coefficient of
Determina-
tion,
observed values or 100 × log(P ressure)
R2

Confidence
is explained by boiling point.
Intervals and
Tests
The Intercept
Slope

Prof. Amany E. Aly ( Professor of Statistics, Lecture

Department
4: Simple
of Mathematics,
Linear Regression
Faculty- of
Part
Science,
III Helwan University,
4 / 3 Ain
/2024
Helwan,36
Cairo,
/ 58 Eg
 Lecture 4:
Simple Linear
Regression -
Part III

Prof. Amany
E. Aly
We can write
Comparing SSreg
2 (SXY )2 2
Models: The
Analysis of R = = = rxy .
Variance

The F-Test for

SY Y SXX × SY Y
Regression

Interpreting
p−values
Thus R2 is the same as the square of the
The
Coefficient of
sample correlation between the predictor
Determina-
tion,
R2
and the response.
Confidence
Intervals and
Tests
The Intercept
Slope

Prof. Amany E. Aly ( Professor of Statistics, Lecture

Department
4: Simple
of Mathematics,
Linear Regression
Faculty- of
Part
Science,
III Helwan University,
4 / 3 Ain
/2024
Helwan,37
Cairo,
/ 58 Eg
 Presentation Outline

Lecture 4:
Simple Linear
Regression -
1 Comparing Models: The Analysis of
Part III

Prof. Amany
Variance
E. Aly

Comparing 2 The F-Test for Regression

Models: The
Analysis of
Variance

The F-Test for

3 Interpreting p−values
Regression

Interpreting
p−values
4 The Coefficient of Determination, R2
The
Coefficient of
Determina-
5 Confidence Intervals and Tests
tion,
R2 The Intercept
Confidence
Intervals and
Tests Slope
The Intercept
Slope

Prof. Amany E. Aly ( Professor of Statistics, Lecture

Department
4: Simple
of Mathematics,
Linear Regression
Faculty- of
Part
Science,
III Helwan University,
4 / 3 Ain
/2024
Helwan,39
Cairo,
/ 58 Eg
 Lecture 4:
Simple Linear
Regression -
Part III

Prof. Amany
E. Aly
The intercept is used to illustrate the
Comparing general form of confidence intervals for
Models: The
Analysis of
Variance normally distributed estimates.
The F-Test for
Regression
The standard error of the intercept is
Interpreting
p−values r
The
Coefficient of 1 x2
Determina- se(β0) = σ
b b +
tion,
R2 n SXX
Confidence
Intervals and
Tests
The Intercept
Slope

Prof. Amany E. Aly ( Professor of Statistics, Lecture

Department
4: Simple
of Mathematics,
Linear Regression
Faculty- of
Part
Science,
III Helwan University,
4 / 3 Ain
/2024
Helwan,41
Cairo,
/ 58 Eg
 Lecture 4:
Simple Linear
Regression -
Part III Hence a (1 − α) × 100% confidence
Prof. Amany
E. Aly
interval for the intercept is the set of
Comparing
Models: The
Analysis of
points β0 in the interval
Variance

The F-Test for

Regression βb0 − t(α/2, n − 2) se(βb0)
Interpreting
p−values

The
6 β0
Coefficient of
Determina-
tion,
6 βb0 + t(α/2, n − 2) se(βb0)
R2

Confidence
Intervals and
Tests
The Intercept
Slope

Prof. Amany E. Aly ( Professor of Statistics, Lecture

Department
4: Simple
of Mathematics,
Linear Regression
Faculty- of
Part
Science,
III Helwan University,
4 / 3 Ain
/2024
Helwan,42
Cairo,
/ 58 Eg
 Lecture 4:
Simple Linear
Regression -
For Forbes’ data,
Part III
r
Prof. Amany 1 (202.95294)2
E. Aly
se(βb0) = 0.37903 + = 3.3
Comparing
Models: The
17 530.78235
Analysis of
Variance For a 90% confidence interval,
The F-Test for
Regression i.e. 1 − α = 0.90 ⇒ α= 0.10 
Interpreting
p−values α 0.1
The t( , n − 2) = t , 17 − 2
Coefficient of
Determina- 2 2
tion,
R2

Confidence
= t(0.05, 15)
Intervals and
Tests
The Intercept
= 1.753.
Slope

Prof. Amany E. Aly ( Professor of Statistics, Lecture

Department
4: Simple
of Mathematics,
Linear Regression
Faculty- of
Part
Science,
III Helwan University,
4 / 3 Ain
/2024
Helwan,43
Cairo,
/ 58 Eg
 Lecture 4:
Simple Linear
Regression -
Part III

Prof. Amany
E. Aly
The interval is
Comparing −42.138 − 1.753(3.340) 6 β0 6 −42.136 + 1.753(3.340)
Models: The
Analysis of
Variance

The F-Test for

Regression
−47.993 6 β0 6 −36.282
Interpreting
p−values

The
Coefficient of
Ninety percent of such intervals will include the
Determina-
tion,
R2
true value.
Confidence
Intervals and
Tests
The Intercept
Slope

Prof. Amany E. Aly ( Professor of Statistics, Lecture

Department
4: Simple
of Mathematics,
Linear Regression
Faculty- of
Part
Science,
III Helwan University,
4 / 3 Ain
/2024
Helwan,44
Cairo,
/ 58 Eg
 Lecture 4:
Simple Linear
Regression -
A hypothesis test of
Part III

Prof. Amany
E. Aly N H : β0 = β0∗, β1 arbitrary
Comparing
Models: The AH : β0 6= β0∗, β1 arbitrary
Analysis of
Variance

The F-Test for

Regression
is obtained by computing the t−statistic
Interpreting
p−values
βb0 − β0∗
The
Coefficient of
t= ∼ t(n − 2)
Determina-
tion, se(β0)
b
R2

Confidence
Intervals and
which means referring this ratio to the
Tests
The Intercept
Slope
t−distribution with n − 2 df .
Prof. Amany E. Aly ( Professor of Statistics, Lecture
Department
4: Simple
of Mathematics,
Linear Regression
Faculty- of
Part
Science,
III Helwan University,
4 / 3 Ain
/2024
Helwan,45
Cairo,
/ 58 Eg
 Lecture 4:
Simple Linear
Regression -
Part III In Forbes’ data, consider testing
Prof. Amany
E. Aly N H : β0 = −35 against
Comparing
Models: The AH : β0 6= −35.
Analysis of
Variance

The F-Test for

The statistic is
Regression

Interpreting −42.138 − (−35)

p−values
t= = 2.137
The
Coefficient of
3.340
Determina-
tion,
R2 which has a p−value near 0.05, providing
Confidence
Intervals and
Tests
some evidence against N H.
The Intercept
Slope

Prof. Amany E. Aly ( Professor of Statistics, Lecture

Department
4: Simple
of Mathematics,
Linear Regression
Faculty- of
Part
Science,
III Helwan University,
4 / 3 Ain
/2024
Helwan,46
Cairo,
/ 58 Eg
 Slope

Lecture 4:
Simple Linear
Regression -
Part III

Prof. Amany
E. Aly The standard error of βb1 is
Comparing
Models: The
σ
Analysis of
se(βb1) = √
b
Variance

The F-Test for

Regression
SXX
Interpreting 0.37903
p−values
= √ = 0.0165
The
Coefficient of
Determina-
530.78235
tion,
R2

Confidence
Intervals and
Tests
The Intercept
Slope

Prof. Amany E. Aly ( Professor of Statistics, Lecture

Department
4: Simple
of Mathematics,
Linear Regression
Faculty- of
Part
Science,
III Helwan University,
4 / 3 Ain
/2024
Helwan,47
Cairo,
/ 58 Eg
 Lecture 4:
Simple Linear
Regression -
Part III

Prof. Amany
A a (1 − α) × 100% confidence interval
E. Aly

Comparing
for the intercept is the set of points β1 in
Models: The
Analysis of
Variance
the interval
The F-Test for
Regression

Interpreting
βb1 − t(α/2, n − 2) se(βb1)
p−values

The
Coefficient of
6 β1
Determina-
tion,
R2
6 βb1 + t(α/2, n − 2) se(βb1)
Confidence
Intervals and
Tests
The Intercept
Slope

Prof. Amany E. Aly ( Professor of Statistics, Lecture

Department
4: Simple
of Mathematics,
Linear Regression
Faculty- of
Part
Science,
III Helwan University,
4 / 3 Ain
/2024
Helwan,48
Cairo,
/ 58 Eg
 Lecture 4:
Simple Linear
Regression -
Part III

Prof. Amany
A 95% confidence interval for the slope
E. Aly
95% ⇒ 1 − α = 0.95 ⇒ α = 0.05
Comparing  
Models: The
Analysis of α 0.05
Variance
t( , n − 2) = t , 17 − 2
The F-Test for
Regression 2 2
Interpreting
p−values = t(0.025, 15)
The
Coefficient of
Determina- = 2.131.
tion,
R2

Confidence
Intervals and
Tests
The Intercept
Slope

Prof. Amany E. Aly ( Professor of Statistics, Lecture

Department
4: Simple
of Mathematics,
Linear Regression
Faculty- of
Part
Science,
III Helwan University,
4 / 3 Ain
/2024
Helwan,49
Cairo,
/ 58 Eg
 Lecture 4:
Simple Linear
Regression -
Part III

Prof. Amany
E. Aly

Comparing
The interval is
Models: The
Analysis of
Variance
0.8955 − 2.131(0.0164) 6 β1 6 0.8955 + 2.131(0.0164)
The F-Test for
Regression

Interpreting
p−values
0.861 6 β1 6 0.930
The
Coefficient of
Determina-
tion,
R2

Confidence
Intervals and
Tests
The Intercept
Slope

Prof. Amany E. Aly ( Professor of Statistics, Lecture

Department
4: Simple
of Mathematics,
Linear Regression
Faculty- of
Part
Science,
III Helwan University,
4 / 3 Ain
/2024
Helwan,50
Cairo,
/ 58 Eg
 Lecture 4:
Simple Linear
Regression -
Part III

Prof. Amany
As an example of a test for slope equal to
E. Aly

Comparing
zero, consider the Ft. Collins snowfall data
Models: The
Analysis of
Variance
presented on page 7.
The F-Test for
Regression
The test of interest is
Interpreting
p−values

The
N H : β1 = 0
Coefficient of
Determina-
tion,
R2 AH : β1 6= 0
Confidence
Intervals and
Tests
The Intercept
Slope

Prof. Amany E. Aly ( Professor of Statistics, Lecture

Department
4: Simple
of Mathematics,
Linear Regression
Faculty- of
Part
Science,
III Helwan University,
4 / 3 Ain
/2024
Helwan,51
Cairo,
/ 58 Eg
 Lecture 4:
Simple Linear
Regression -
For the Ft. Collins data,
Part III
0.20335 − 0
Prof. Amany
E. Aly t= = 1.553.
Comparing
0.1310
Models: The
Analysis of
To get a significance level for this test,
Variance

The F-Test for

Regression
compare t with the t(91) distribution;
Interpreting
p−values
the two-sided p−value is 0.124, suggesting
The
Coefficient of
no evidence against the N H.
Determina-
tion,
R2 That is
Confidence
Intervals and
Tests
Early and Late season snowfalls are
The Intercept
Slope independent.
Prof. Amany E. Aly ( Professor of Statistics, Lecture
Department
4: Simple
of Mathematics,
Linear Regression
Faculty- of
Part
Science,
III Helwan University,
4 / 3 Ain
/2024
Helwan,52
Cairo,
/ 58 Eg
 Lecture 4:
Simple Linear
Regression -
Part III !2
Prof. Amany βb1
E. Aly
t2 =
Comparing
Models: The
se(βb1)
Analysis of
Variance 2
The F-Test for βb1
Regression
=
Interpreting
p−values σb2/SXX
The
Coefficient of 2
Determina-
tion,
βb1 SXX
R2 =
Confidence
Intervals and σb2
Tests
The Intercept
Slope
= F
Prof. Amany E. Aly ( Professor of Statistics, Lecture
Department
4: Simple
of Mathematics,
Linear Regression
Faculty- of
Part
Science,
III Helwan University,
4 / 3 Ain
/2024
Helwan,53
Cairo,
/ 58 Eg
 Lecture 4:
Simple Linear
Regression -
Part III

Prof. Amany
E. Aly

Comparing
Models: The
Analysis of
So the square of a t statistic with d df
Variance

The F-Test for

is equivalent to
Regression

Interpreting an F −statistic with (1, d) df.

p−values

The
Coefficient of
Determina-
tion,
R2

Confidence
Intervals and
Tests
The Intercept
Slope

Prof. Amany E. Aly ( Professor of Statistics, Lecture

Department
4: Simple
of Mathematics,
Linear Regression
Faculty- of
Part
Science,
III Helwan University,
4 / 3 Ain
/2024
Helwan,54
Cairo,
/ 58 Eg
 Lecture 4:
Simple Linear
Regression -
Part III

Prof. Amany
E. Aly
For linear regression models, no conflict
Comparing
occurs and the two tests are equivalent.
Models: The
Analysis of
Variance In nonlinear and logistic regression
The F-Test for
Regression models, the analog of the t test will not
Interpreting
p−values

The
be identical to the analog of the F
Coefficient of
Determina-
tion,
test, and they can give conflicting
R2

Confidence
conclusions.
Intervals and
Tests
The Intercept
Slope

Prof. Amany E. Aly ( Professor of Statistics, Lecture

Department
4: Simple
of Mathematics,
Linear Regression
Faculty- of
Part
Science,
III Helwan University,
4 / 3 Ain
/2024
Helwan,55
Cairo,
/ 58 Eg
 Lecture 4:
Simple Linear
Regression - 0.7
Part III

Prof. Amany 0.6

E. Aly

Comparing 0.5
Models: The
Analysis of
Variance 0.4
The F-Test for
Regression
0.3
Interpreting
p−values
0.2
The
Coefficient of α = 0.05
Determina- 0.1
tion,
R2 F
Confidence 1 2 3 4
Intervals and
Tests
The Intercept
Slope

Prof. Amany E. Aly ( Professor of Statistics, Lecture

Department
4: Simple
of Mathematics,
Linear Regression
Faculty- of
Part
Science,
III Helwan University,
4 / 3 Ain
/2024
Helwan,56
Cairo,
/ 58 Eg
 Lecture 4:
Simple Linear
Regression -
0.4
Part III

Prof. Amany
E. Aly

0.3
Comparing
Models: The
Analysis of
Variance

The F-Test for 0.2

Regression
1-α
Interpreting
p−values

The 0.1
Coefficient of
Determina-
tion,
R2 α \2 α/2

Confidence -3 -2 L -1 1 U 2 3
Intervals and
Tests
The Intercept
Slope

Prof. Amany E. Aly ( Professor of Statistics, Lecture

Department
4: Simple
of Mathematics,
Linear Regression
Faculty- of
Part
Science,
III Helwan University,
4 / 3 Ain
/2024
Helwan,57
Cairo,
/ 58 Eg
 Lecture 4:
Simple Linear
Regression -
Part III

Prof. Amany
E. Aly P (L < β0 < U ) = 1 − α
Comparing
Models: The
Analysis of
Variance
P − value = P (F > F0 )
The F-Test for
Regression

Interpreting 
p−values
 > α, don’t reject H0 ;
The
Coefficient of
Determina-
=
tion,
< α, reject H0 .

R2

Confidence
Intervals and
Tests Where F0 is the observed value of F .
The Intercept
Slope

Prof. Amany E. Aly ( Professor of Statistics, Lecture

Department
4: Simple
of Mathematics,
Linear Regression
Faculty- of
Part
Science,
III Helwan University,
4 / 3 Ain
/2024
Helwan,58
Cairo,
/ 58 Eg