Contemporary Business

Statistics, 3e
by
Williams, Sweeney, and Anderson

Slides by

JOHN
LOUCKS
St. Edward’s
University

© 2009 Cengage South-Western. All Rights Reserved Slide

1
 Chapter 14, Part A
Simple Linear Regression

 Simple Linear Regression Model

 Least Squares Method
 Coefficient of Determination
 Model Assumptions
 Testing for Significance

© 2009 Cengage South-Western. All Rights Reserved Slide

2
 Simple Linear Regression

 Managerial decisions often are based on the

relationship between two or more variables.
 Regression analysis can be used to develop an
equation showing how the variables are related.
 The variable being predicted is called the dependent
variable and is denoted by y.
 The variables being used to predict the value of the
dependent variable are called the independent
variables and are denoted by x.

© 2009 Cengage South-Western. All Rights Reserved Slide

3
 Simple Linear Regression

 Simple linear regression involves one independent

variable and one dependent variable.
 The relationship between the two variables is
approximated by a straight line.
 Regression analysis involving two or more
independent variables is called multiple regression.

© 2009 Cengage South-Western. All Rights Reserved Slide

4
 Simple Linear Regression Model

 The equation that describes how y is related to x and

an error term is called the regression model.
 The simple linear regression model is:

y = b0 + b1x +e

where:
b0 and b1 are called parameters of the model,
e is a random variable called the error term.

© 2009 Cengage South-Western. All Rights Reserved Slide

5
 Simple Linear Regression Equation

 The simple linear regression equation is:

E(y) = 0 + 1x

• Graph of the regression equation is a straight line.

• b0 is the y intercept of the regression line.
• b1 is the slope of the regression line.
• E(y) is the expected value of y for a given x value.

© 2009 Cengage South-Western. All Rights Reserved Slide

6
 Simple Linear Regression Equation

 Positive Linear Relationship

E(y)

Regression line

Intercept Slope b1
b0
is positive

© 2009 Cengage South-Western. All Rights Reserved Slide

7
 Simple Linear Regression Equation

 Negative Linear Relationship

E(y)

Intercept
b0 Regression line

Slope b1
is negative

© 2009 Cengage South-Western. All Rights Reserved Slide

8
 Simple Linear Regression Equation

 No Relationship

E(y)

Intercept Regression line

b0
Slope b1
is 0

© 2009 Cengage South-Western. All Rights Reserved Slide

9
 Estimated Simple Linear Regression Equation

 The estimated simple linear regression equation

ŷ  b0  b1 x

• The graph is called the estimated regression line.

• b0 is the y intercept of the line.
• b1 is the slope of the line.
• ŷ is the estimated value of y for a given x value.

© 2009 Cengage South-Western. All Rights Reserved Slide

10
 Estimation Process

Regression Model Sample Data:

y = b0 + b1x +e x y
Regression Equation x1 y1
E(y) = b0 + b1x . .
Unknown Parameters . .
b0, b1 xn y n

Estimated
b0 and b1 Regression Equation
provide estimates of ŷ  b0  b1 x
b0 and b1 Sample Statistics
b0, b1

© 2009 Cengage South-Western. All Rights Reserved Slide

11
 Least Squares Method

 Least Squares Criterion

min  (y i  y i ) 2

where:
yi = observed value of the dependent variable
for the ith observation
y^i = estimated value of the dependent variable
for the ith observation

© 2009 Cengage South-Western. All Rights Reserved Slide

12
 Least Squares Method

 Slope for the Estimated Regression Equation

b1   ( x  x )( y  y )
i i

 (x  x )
i
2

where:
xi = value of independent variable for ith
observation
yi = value of dependent variable for ith
_ observation
x = mean value for independent variable
_
y = mean value for dependent variable

© 2009 Cengage South-Western. All Rights Reserved Slide

13
 Least Squares Method

 y-Intercept for the Estimated Regression Equation

b0  y  b1 x

© 2009 Cengage South-Western. All Rights Reserved Slide

14
 Simple Linear Regression

 Example: Reed Auto Sales

Reed Auto periodically has a special week-long sale.
As part of the advertising campaign Reed runs one or
more television commercials during the weekend
preceding the sale. Data from a sample of 5 previous
sales are shown on the next slide.

© 2009 Cengage South-Western. All Rights Reserved Slide

15
 Simple Linear Regression

 Example: Reed Auto Sales

Number of Number of
TV Ads (x) Cars Sold (y)
1 14
3 24
2 18
1 17
3 27
Sx = 10 Sy = 100
x2 y  20

© 2009 Cengage South-Western. All Rights Reserved Slide

16
 Estimated Regression Equation

 Slope for the Estimated Regression Equation

b1   ( x i  x )( y i  y ) 20
 5
 (xi  x ) 2
4
 y-Intercept for the Estimated Regression Equation
b0  y  b1 x  20  5(2)  10
 Estimated Regression Equation
yˆ  10  5x

© 2009 Cengage South-Western. All Rights Reserved Slide

17
 Using Excel’s Chart Tools for
Scatter Diagram & Estimated Regression Equation
 Excel Worksheet (showing data)
A B C D
1 Week TV Ads Cars Sold
2 1 1 14
3 2 3 24
4 3 2 18
5 4 1 17
6 5 3 27
7

© 2009 Cengage South-Western. All Rights Reserved Slide

18
 Using Excel’s Chart Tools for
Scatter Diagram & Estimated Regression Equation
 Producing a Scatter Diagram
Step 1 Select cells B1:C6
Step 2 Click the Insert tab on the Excel ribbon
Step 3 In the Charts group, click Scatter
Step 4 When the list of scatter diagram subtypes appears:
Click Scatter with only Markers
Step 5 In the Chart Layouts group, click Layout 1
Step 6 Select the Chart Title and replace it with Reed
Auto Sales Estimated Regression Equation

continue

© 2009 Cengage South-Western. All Rights Reserved Slide

19
 Using Excel’s Chart Tools for
Scatter Diagram & Estimated Regression Equation
 Producing a Scatter Diagram
Step 7 Select the Horizontal Axis Title and replace it
with TV Ads
Step 8 Select the Vertical Axis Title and replace it with
Cars Sold
Step 9 Right click on the legend and click Delete
Step 10 Position the mouse pointer over any Vertical
Axis Major Gridline in the scatter diagram and
right-click to display a list of options and then
choose Delete

© 2009 Cengage South-Western. All Rights Reserved Slide

20
 Using Excel’s Chart Tools for
Scatter Diagram & Estimated Regression Equation
 Adding the Trendline
Step 11 Position the mouse pointer over any data point
in the scatter diagram and right-click to display
a list of options
Step 12 Choose Add Trendline
Step 13 When the Format Trendline dialog box appears:
Select Trendline Options and then
Choose Linear from the Trend/Regression
Type list
Choose Display Equation on chart
Click Close

© 2009 Cengage South-Western. All Rights Reserved Slide

21
 Using Excel’s Chart Tools for
Scatter Diagram & Estimated Regression Equation

Reed Auto Sales Estimated Regression Line

30

25
20
Cars Sold

y = 5x + 10
15

10
5

0
0 1 2 3 4
TV Ads

© 2009 Cengage South-Western. All Rights Reserved Slide

22
 Coefficient of Determination

 Relationship Among SST, SSR, SSE

SST = SSR + SSE

 i
( y  y )2
  i
( ˆ
y  y )2
 i i
( y  ˆ
y )2

where:
SST = total sum of squares
SSR = sum of squares due to regression
SSE = sum of squares due to error

© 2009 Cengage South-Western. All Rights Reserved Slide

23
 Coefficient of Determination

 The coefficient of determination is:

r2 = SSR/SST

where:
SSR = sum of squares due to regression
SST = total sum of squares

© 2009 Cengage South-Western. All Rights Reserved Slide

24
 Coefficient of Determination

r2 = SSR/SST = 100/114 = .8772

The regression relationship is very strong; 87.72%

of the variability in the number of cars sold can be
explained by the linear relationship between the
number of TV ads and the number of cars sold.

© 2009 Cengage South-Western. All Rights Reserved Slide

25
 Using Excel to Compute the
Coefficient of Determination
 Displaying the Coefficient of Determination
Step 1 Position the mouse pointer over any data point
in the scatter diagram and right-click to display
a list of options
Step 2 Choose Add Trendline
Step 3 When the Trendline dialog box appears:
Select Trendline Options and then
Choose Display R-squared value on chart
Click Close

© 2009 Cengage South-Western. All Rights Reserved Slide

26
 Using Excel to Compute the
Coefficient of Determination

Reed Auto Sales Estimated Regression Line

30

25
20
Cars Sold

y = 5x + 10
15 2
R = 0.8772
10
5

0
0 1 2 3 4
TV Ads

© 2009 Cengage South-Western. All Rights Reserved Slide

27
 Sample Correlation Coefficient

rxy  (sign of b1 ) Coefficient of Determination

rxy  (sign of b1 ) r 2

where:
b1 = the slope of the estimated regression
equation ˆ
y b
0b
x
1

© 2009 Cengage South-Western. All Rights Reserved Slide

28
 Sample Correlation Coefficient

rxy  (sign of b1 ) r 2

yˆ
The sign of b1 in the equation
1
0 5
isx“+”.

rxy = + .8772

rxy = +.9366

© 2009 Cengage South-Western. All Rights Reserved Slide

29
 Assumptions About the Error Term e

1. The error  is a random variable with mean of zero.

2. The variance of  , denoted by  2, is the same for

all values of the independent variable.

3. The values of  are independent.

4. The error  is a normally distributed random

variable.

© 2009 Cengage South-Western. All Rights Reserved Slide

30
 Testing for Significance

To test for a significant regression relationship, we

must conduct a hypothesis test to determine whether
the value of b1 is zero.

Two tests are commonly used:

t Test and F Test

Both the t test and F test require an estimate of s 2,

the variance of e in the regression model.

© 2009 Cengage South-Western. All Rights Reserved Slide

31
 Testing for Significance

 An Estimate of s 2
The mean square error (MSE) provides the estimate
of s 2, and the notation s2 is also used.

s 2 = MSE = SSE/(n - 2)

where:

SSE   ( yi  yˆ i ) 2   ( yi  b0  b1 xi ) 2

© 2009 Cengage South-Western. All Rights Reserved Slide

32
 Testing for Significance

 An Estimate of s
• To estimate s we take the square root of s 2.
• The resulting s is called the standard error of
the estimate.

SSE
s  MSE 
n2

© 2009 Cengage South-Western. All Rights Reserved Slide

33
 Testing for Significance: t Test

 Hypotheses

H0 : 1  0
H a : 1  0

 Test Statistic

b1 s
t where sb1 
sb1 2
( xi  x )

© 2009 Cengage South-Western. All Rights Reserved Slide

34
 Testing for Significance: t Test

 Rejection Rule

Reject H0 if p-value < a

or t < -tor t > t

where:
t is based on a t distribution
with n - 2 degrees of freedom

© 2009 Cengage South-Western. All Rights Reserved Slide

35
 Testing for Significance: t Test

1. Determine the hypotheses. H0 : 1  0

Ha : 1  0
2. Specify the level of significance. a = .05

b1
3. Select the test statistic. t 
sb1

4. State the rejection rule. Reject H0 if p-value < .05

or |t| > 3.182 (with
3 degrees of freedom)

© 2009 Cengage South-Western. All Rights Reserved Slide

36
 Testing for Significance: t Test

5. Compute the value of the test statistic.

b1 5
t   4.63
sb1 1.08

6. Determine whether to reject H0.

t = 4.541 provides an area of .01 in the upper
tail. Hence, the p-value is less than .02. (Also,
t = 4.63 > 3.182.) We can reject H0.

© 2009 Cengage South-Western. All Rights Reserved Slide

37
 Confidence Interval for 1

 We can use a 95% confidence interval for 1 to test

the hypotheses just used in the t test.
 H0 is rejected if the hypothesized value of 1 is not
included in the confidence interval for 1.

© 2009 Cengage South-Western. All Rights Reserved Slide

38
 Confidence Interval for 1

 The form of a confidence interval for 1 is: t/2sb1

is the
b1  t /2sb1 margin
b1 is the of error
point where t / 2 is the t value providing an area
estimator
of a/2 in the upper tail of a t distribution
with n - 2 degrees of freedom

© 2009 Cengage South-Western. All Rights Reserved Slide

39
 Confidence Interval for 1

 Rejection Rule
Reject H0 if 0 is not included in
the confidence interval for 1.
 95% Confidence Interval for 1
b
1t/2s
b1
= 5 +/- 3.182(1.08) = 5 +/- 3.44
or 1.56 to 8.44
 Conclusion
0 is not included in the confidence interval.
Reject H0

© 2009 Cengage South-Western. All Rights Reserved Slide

40
 Testing for Significance: F Test

 Hypotheses

H0 : 1  0
Ha : 1  0
 Test Statistic

F = MSR/MSE

© 2009 Cengage South-Western. All Rights Reserved Slide

41
 Testing for Significance: F Test

 Rejection Rule

Reject H0 if
p-value < a
or F > F
where:
F is based on an F distribution with
1 degree of freedom in the numerator and
n - 2 degrees of freedom in the denominator

© 2009 Cengage South-Western. All Rights Reserved Slide

42
 Testing for Significance: F Test

1. Determine the hypotheses. H0 : 1  0

H a : 1  0

2. Specify the level of significance. a = .05

3. Select the test statistic. F = MSR/MSE

4. State the rejection rule. Reject H0 if p-value < .05

or F > 10.13 (with 1 d.f.
in numerator and
3 d.f. in denominator)

© 2009 Cengage South-Western. All Rights Reserved Slide

43
 Testing for Significance: F Test

5. Compute the value of the test statistic.

F = MSR/MSE = 100/4.667 = 21.43

6. Determine whether to reject H0.

F = 17.44 provides an area of .025 in the upper
tail. Thus, the p-value corresponding to F = 21.43
is less than 2(.025) = .05. Hence, we reject H0.
The statistical evidence is sufficient to conclude
that we have a significant relationship between the
number of TV ads aired and the number of cars sold.

© 2009 Cengage South-Western. All Rights Reserved Slide

44
 Some Cautions about the
Interpretation of Significance Tests
 Rejecting H0: b1 = 0 and concluding that the
relationship between x and y is significant does
not enable us to conclude that a cause-and-effect
relationship is present between x and y.
 Just because we are able to reject H0: b1 = 0 and
demonstrate statistical significance does not enable
us to conclude that there is a linear relationship
between x and y.

© 2009 Cengage South-Western. All Rights Reserved Slide

45
 End of Chapter 14, Part A

© 2009 Cengage South-Western. All Rights Reserved Slide

46