SIMPLE LINEAR REGRESSION (cont.

)
TIME-SERIES ANALYSIS
 EVALUATING SIMPLE LINEAR REGRESSION

There is just one independent The relationship between X and Y

variable in the model. can be depicted with a straight line.

Ex 1: The manager of Colonial Furniture has been reviewing weekly

advertising expenditures. During the past six months all advertisements for the
store have been appeared in the local newspaper. The number of ads per week
has varied from one to seven. The store’s sales staff has been tracking the
number of customers who enter the store each week. The number of ads and
the number of customers per week for the past 26 weeks have been stored in
the file Xr11-72.

Number of ads per week is the independent variable (X), and the
number of customers per week is the dependent variable (Y).
The purpose of advertising in the local newspaper is to boost sales, so
we expect a positive relationship between X and Y.

2
a) Graph the paired observations of X and Y.

800
700
600

Customer
500
400
300
200
100
0
0 1 2 3 4 5 6 7 8
Ads

This scatter diagram shows that, at least in the sample, there is some rather
weak positive linear relationship between the numbers of ads and customers.

b) Determine the sample regression line.

We are going to rely on Excel. However, at home, try to reproduce the results
manually using the sum of squares: SSx = 86.654, SSy = 463802 and SSxy =
1850.6.
3
Using Excel:
SUMMARY OUTPUT

Regression Statistics
Multiple R 0.292
R Square 0.085
Adjusted R Square 0.047
Standard Error 132.960
Observations 26.000

Coefficients Standard Error t Stat P-value Lower 95% Upper 95%

Intercept 296.920 64.305 4.617 0.000 164.200 429.639
Ads 21.356 14.283 1.495 0.148 -8.123 50.835

c) Interpret the coefficients. yˆ  ˆ 0  ˆ1 x  296.920  21.356 x

ˆ 0  296.92 This means that when X,= 0, i.e. no advertisement is placed
in the local newspaper, the expected number of customers
per week is 296.92.

ˆ1  21.356 suggesting that on a given week for each additional

advertisement in the local newspaper the number of
customers is expected to increase by 21.356.

4
d) Find and interpret the coefficient of determination.
Regression Statistics
Multiple R 0.292
R Square 0.085
Adjusted R Square 0.047
Standard Error 132.960
Observations 26.000

This suggests that only 8.5% of the total variation in the number of customers
can be explained, or is due to, the variation in the number of advertisements.
The remaining 91.5% is unexplained, i.e. is due to some other factors than ads.
This model has a rather poor overall fit.

• Similar to the mean of a single population, say µx, the unknown

parameters of a population regression model, β0 and β1, can be
estimated with point estimators and with interval estimators, and we
can also use hypothesis testing to verify whether the sample at hand
supports certain statements about these parameters.
However, to do so, first we have to study the sampling distributions of
the β0-hat and β1-hat least squares estimators.
5
• Just as the sampling distribution of the sample mean, X-bar, depends
on the the mean, standard deviation and shape of the X population, the
sampling distributions of the β0-hat and β1-hat least squares estimators
depend on the properties of the {Yj } sub-populations (j=1,…, n).

y j   0  1 x j   j
Given xj, the properties of the {Yj } sub-population are determined by
the εj error/random variable.
As regards the probability distributions of εj ( j =1,…, n), it is assumed that:
i. Each εj is normally distributed, Yj is also normal;
ii. Each εj has zero mean, E(Yj) = β0 + β1 xj
iii.Each εj has the same Var(Yj) = σε2 is also
variance, σε2, constant;
iv. The errors are independent of {Yi} and {Yj}, ij, are also
each other, independent;
v. The error does not depend on The effects of X and ε on Y
the independent variable(s). can be separated from
each other.

6
The first three assumptions can be illustrated as follows:

E(Y)

E(Y)  β0  β1X
Yi : N (β0+β1xi ; σ )

Yj : N (β0+β1xj ; σ )

X
xi xj

• If all 5 assumptions are met, then the β0-hat and β1-hat least squares
estimators are also normally distributed:

ˆ 0 : N (  0 ; ˆ )
0
ˆ1 : N ( 1; ˆ )
1

7
 ˆ 0   0 ˆ1  1
and are standard normal random variables.
 ˆ0
 ˆ
1

However, the standard errors (standard deviations) of the β0-hat and β1-hat
estimators are unknown and they depend on the standard deviation of ε, σε,
which is also unknown. They have to be estimated from the sample,
similarly to β0 and β1.

Standard error of estimate: the sample standard deviation of ε.

SSE
In the case of simple linear regression (k=1) it is s 
n2
Replacing σε with its estimate, sε, the estimated standard errors of
β0-hat and β1-hat are

sˆ  s
 i /n
x 2

and sˆ 
s
0
SS x 1
SS x

8
 ˆ 0   0 ˆ1  1
and are t random variables with n -2 degrees
s ˆ s ˆ of freedom (df).
0 1

The C% confidence interval estimators of β0 and β1 are

βˆ0  tα/ 2 ,n 2 s βˆ βˆ1  tα/ 2 ,n 2 s βˆ

0 1

(Ex 1)
e) Find the standard error of estimate and the standard error of the slope
estimator.

Solving by hand, we have to start with the sum of squares for error, SSE.
A useful computational formula for SSE is
SS xy2 1850.6 2
SSE  SS y  SSE  463802   424280.2
SS x 86.654

9
 Standard Error of Estimate
SSE 424280.2
Hence s    132.96
n2 26  2
Regression Statistics
Multiple R 0.292
R Square 0.085
Adjusted R Square 0.047
s 132.96 Standard Error 132.960
and sˆ    14.283
1
SS x 86.654 Observations 26.000

Coefficients Standard Error t Stat P-value

Intercept 296.920 64.305 4.617 0.000
Ads 21.356 14.283 1.495 0.148

10
f) Determine the 95% confidence interval estimate of β1.
C = 95, so α/2 = 0.025
t0.025, 24  2.064
n = 26, so df = 24

Thus βˆ  t
1 α/ 2 ,n 2 s βˆ  21.356  2.064 14.283  (-8.124 ; 50.836)
1

Coefficients Standard Error t Stat P-value Lower 95% Upper 95%

Intercept 296.920 64.305 4.617 0.000 164.200 429.639
Ads 21.356 14.283 1.495 0.148 -8.123 50.835

and with 95% confidence the slope coefficient is within this interval.
Notice that this interval has a negative lower limit and a positive upper limit,
i.e. the slope coefficient can be negative, positive or zero.
The number of advertisement does not necessarily have a positive
impact on the number of customers, so it might not be worth to spend
on advertisements in the local newspaper.
11
 • The sampling distributions of the β0-hat and β1-hat least squares
estimators can also be used for testing the regression coefficients.
Recall (see the week 9 notes) that in regression analysis the most
important question is whether there is a linear relationship between
X and Y (β1 0), and if there is, whether this relationship is positive
(β1> 0) or negative (β1< 0).
To this end we can conduct two-tail or one-tail t-tests about β1, the
same way as we test µ when σ is unknown.

ˆ1   0,1
The test statistic is t
sˆ
1

where β0,1 denotes the hypothetical value of β1 (often zero).

Granted that H0 is true, t has a Student’s t distribution with n -2 degrees
of freedom (for k=1).

(Ex 1)
g) Is there sufficient evidence at the 5% level to conclude that the number of
advertisements and the number of customers are linearly related?
12
The question suggests that H0 : β1= 0 and HA : β1 0, so β0,1= 0.
Since this a two-tail test, there are two critical values: -tα/2 = -2.064, tα/2= 2.064
(see part f).
Accordingly, reject H0 if the value of the test statistic calculated from the sample
is either smaller than -2.064 or greater than 2.064.

ˆ1 21.356
tobs    1.495 Since 1.495 is in the non-rejection
sˆ 14.283 region we maintain H0.
1

Coefficients Standard Error t Stat P-value Lower 95% Upper 95%

Intercept 296.920 64.305 4.617 0.000 164.200 429.639
Ads 21.356 14.283 1.495 0.148 -8.123 50.835

p-value > α = 0.05

Maintain H0 and conclude at the 5% level of significance that the
numbers of advertisements and customers are not linearly related.
13
Ex 2: (Example 1 of week 9)
Pat Statsdud postulated that the longer one studied, the better one’s grade.
To test this theory, Pat regressed final mark (Y) on study time (X) and
obtained the following sample regression equation:

yˆ  ˆ 0  ˆ1 x  21.590  1.877 x

f) Can we conclude at the 1% significance level that Pat’s theory is correct, I.

there is a positive linear relationship between study time and final mark?
Right-tail test with H0 : β1= 0 and HA : β1> 0.

Coefficients Standard Error t Stat P-value Lower 95% Upper 95%

Intercept 21.590 2.835 7.614 0.000 15.963 27.216
Stdytime 1.877 0.097 19.443 0.000 1.686 2.069

p-value < 0.01

Reject H0 and conclude at the 1% level of significance that Pat’s theory

is supported by the sample evidence.

14
 USING THE SAMPLE REGRESSION EQUATION
• If the fit of the sample regression equation is satisfactory, it can be
used to predict the dependent variable or to estimate its mean value.

For a particular element of a Y For the expected value of a Y

sub-population. sub-population.
E.g.: What is the final mark of Tom who E.g.: What is the mean final mark of all
spent 30 hours on studying? those students who spent 30 hours on
i.e., given x = 30, how large is y? studying?
i.e., given x = 30, how large is E(y)?

The dependent variable can be predicted or estimated in two ways:

with a single value or with an interval.
For a given X value, the point forecast of Y and the point estimator of
the mean of the {Y} sub-population are the same: ˆ ˆ
yˆ   0  1 x
Ex.2 Predict the final mark when study time is 30 hours.

yˆ  ˆ0  ˆ1 x  21.590  1.877  30  77.9

 INTRODUCTION TO TIME-SERIES ANALYSIS
• According to classical time-series analysis an observed time series is
the combination of some pattern and random variations.
The aim is to separate them from each other in order to
a) describe to historical pattern in the data,
and to
b) prepare forecasts by projecting the revealed historical pattern
into the future.

• Traditionally, there are two types of methods for identifying the pattern.

Smoothing: Decomposition:
The random fluctuations are The time series is broken into its
removed from the data by smoothing components and the pattern is the
the time series. combination of the systematic parts.

• The pattern itself is likely to contain some, or all, of the following three
components: trend, seasonal and cyclical.
16
 Trend: The long-term general change in the level of the data with a
duration of longer than a year.

Yt It can be linear (straight line) Yt  a  bt

or non-linear (smooth curve), like e.g.

exponential Yt  ab t

t
quadratic Yt  a  bt  ct 2 etc.

Hourly earnings: Manufacturing: Major seven countries Broad money: (sa): Sweden
1995=100 1995=100
120 140

100 120

100
80
80
60
60
40
40

20 20

0 0
Sep-70 Sep-80 Sep-90 Sep-00 Jan-61 Jan-71 Jan-81 Jan-91 Jan-01

17
 Seasonal variations: Regular wavelike fluctuations of constant
length, repeating themselves within a period of
no longer than a year.
Yt

Seasonal variations are usually

associated with the four seasons of the
year, but they may also refer to any
systematic pattern that occurs during a
month, a week or even a single day.
t

Hungary: Commodity output: Cement Australia: Retail turnover: Department stores

'000 tonnes $m
600 2500

500
2000

400
1500
300
1000
200

500
100

0 0
Dec-82 Dec-85 Dec-88 Dec-91 Dec-94 Dec-97 Dec-00 Jan-83 Jan-86 Jan-89 Jan-92 Jan-95 Jan-98 Jan-01

18
 Cyclical variations: Wavelike movements, quasi regular
fluctuations around the long-term trend,
lasting longer than a year.
Yt Peak

The time gap between the beginning

trough and ending trough is the length
of the cycle, while the vertical distance
between the through and the peak is
the amplitude of the cycle.
Beginning Ending t
trough trough

Aus: Dwelling units approved: Private: New houses Expenditure on GDP: Construction: United States: (sa)
Number bln 96 USD
40000 900

800
35000

700
30000
600
25000
500

20000
400

15000 300
Dec-70 Dec-75 Dec-80 Dec-85 Dec-90 Dec-95 Dec-00 Dec-60 Dec-70 Dec-80 Dec-90 Dec-00

19
The time period between the beginning trough and the peak is called expansion
phase, while the period between the peak and the ending trough is termed
contraction phase.
Cyclical variations are often attributed to business cycles, i.e. to the ups and
downs in the general level of business activity.
Seasonal and cyclical variations might be very similar in their appearance.
However, while seasonal variations are absolutely regular and occur over
calendar periods no longer than a year, cyclical variations might and do change
both in their intensity (amplitude) and duration, and they last longer than a year.
It is far more difficult to study and predict the cyclical component than
the seasonal component.

• The random variations of the data comprise the deviations of the

observed time series from the underlying pattern.
When this irregular component is strong compared to the (quasi-)
regular components, it tends to hide the seasonal and cyclical
variations, and it is difficult to be detached from the pattern.
However, if we manage to capture the trend, the seasonal and cyclical
variations, the remaining changes do not have any discernible pattern,
so they are totally unpredictable.
20
 • The four components of a time series (T: trend, S: seasonal, C: cyclical,
R: random) can be combined in different ways. Accordingly, the time
series model used to describe the observed data (Y) can be

Additive: Multiplicative:
Yt  Tt  S t  Ct  Rt Yt  Tt  S t  Ct  Rt

E.g.: If the trend is linear, these two models look as follows:

Yt  (a  bt )  S t  Ct  Rt Yt  (a  bt )  S t  Ct  Rt

In an additive model the seasonal, In a multiplicative model the

cyclical and random variations are seasonal, cyclical and random
absolute deviations from the trend. variations are relative (percentage)
deviations from the trend.
They do not depend on the The higher the trend, the
level of the trend. more intensive these
variations are.

21
 Austria: Domestic demand: R etail sales: Volume Australia: Retail turnover: Recreational goods
1995=100 $m
160 900

800
140
700
120
600

100 500

400
80
300
60
200

40 100
Dec-76 Dec-80 Dec-84 Dec-88 Dec-92 Dec-96 Dec-00 Jan-83 Jan-86 Jan-89 Jan-92 Jan-95 Jan-98 Jan-01

These time series have an increasing linear trend component, but

the fluctuations around this trend the fluctuations around this trend
have the same intensity; are more and more intensive.

Though in practice the multiplicative model is the more popular, both

models have their own merits and, depending on the nature of the time
series to be analysed, they are equally acceptable.

22