Statistical Inference Point Estimators Estimating The Population Mean Using Confidence Intervals
Statistical Inference Point Estimators Estimating The Population Mean Using Confidence Intervals
Contents
I Statistical inference
I Point estimators
I The estimation of the population mean and variance
I Estimating the population mean using confidence intervals
I Confidence intervals for the mean of a normal population with known
variance
I Confidence intervals for the mean in large samples
I Confidence intervals for the population proportion
I Confidence intervals for the mean of a normal population with
unknown variance
I Estimating the population variance using confidence intervals
I Confidence intervals for the variance of a normal population
Chapter 1. Statistical inference in one population
Learning goals
At the end of this chapter you should know how to:
I Estimate the unknown population parameters from the sample data
I Construct confidence intervals for the unknown population
parameters from the sample data:
I In the case of a normal distribution: confidence intervals for the
population mean and variance
I In large samples: confidence intervals for the population mean and
proportion
I Interpret the confidence interval
I Understand the impact of the sample size, confidence level, etc on
the length of the confidence interval
I Calculate a sample size needed to control a given interval width
Chapter 1. Statistical inference in one population
References
I Newbold, P. Statistics for Business and Economics
I Chapters 7 and 8 (8.1-8.6)
I Ross, S. Introduction to Statistics
I Chapter 8
Statistical inference: key words (i)
X 1 , X2 , . . . , X n F x1 , x2 , . . . , xn
X F ) Sample ) Observed sample
+ + +
Estimator of X (r. v.) Estimate of X (number)
X = E[X ]
Expected value of X
( X
Sample mean
( x
Sample mean
Point estimators: introduction
Bias[X ] = E[X ] X
Population Estimator Minimum Variance
parameter T (X n ) Bias Unbiased? Unbiased Estimator?
Pop. mean X X E[X ] X = 0 Yes Yes, if X normal
Pop. prop. pX pX E[pX ] pX = 0 Yes Yes
Pop. var. X2 X2 E[ X2 ] 2
X 6= 0 No No
Pop. var. X2 sX2 E[sX2 ] 2
X = 0 Yes Yes, if X normal
In general, X X E[X ] X Often Rarely
Point estimators: properties (ii)
Var[X ,1 ]
Relative efficiency(X ,1 , X ,2 ) =
Var[X ,2 ]
Note:
I sometimes the inverse is used as a definition
I in any case, an estimator with smaller variance is more efficient
Point estimators: properties (iii)
Note:
I the mean squared error of an unbiased estimator equals its variance
I an estimator with smaller MSE is better
I the minimum variance unbiased estimator has the smallest
variance/MSE among all estimators
I How do we come up with the definition of the estimator T ?
I In some situations, there exists an optimal estimator called minimum
variance unbiased estimator.
I If thats not the case, there are various alternative methods that
yield reasonable estimators, for example:
I Maximum likelihood estimation
I Method of moments
Point estimation: example
80
x = =8
10
782 10(8)2
sx2 = = 15.78
10 1
1+1+0+1+1+0+0+0+0+0
px =
10
= 0.4
Point estimation: example
2
Example: Let X = n(n+1) (X1 + 2X2 + . . . + nXn ) be an estimator of the
population mean based on a SRS X n . Compare this estimator with the sample
mean, X .
2
We know that X is an unbiased estimator of X , whose variance is nX .
X is also unbiased: And its variance/MSE is:
" # " #
2 2
E[X ] = E (X1 + 2X2 + . . . + nXn ) V[X ] = V (X1 + 2X2 + . . . + nXn )
n(n + 1) n(n + 1)
!2
2 2
= (E[X1 ] + 2E[X2 ] + . . . + nE[Xn ]) 2 2
=indep. (V[X1 ] + 2 V[X2 ] + . . . + n V[Xn ])
n(n + 1) n(n + 1)
2 n(n+1)(2n+1)/6
=id (X + 2X + . . . + nX )
n(n + 1) z }| {
4 2 2 2 2
=id X (1 + 2 + ... + n )
n(n+1)/2 n2 (n + 1)2
2X z }| {
= (1 + 2 + . . . + n) = X 2(2n + 1) 2
n(n + 1) = X
3n(n + 1)
) Bias[X ] = 0
2 2(2n + 1) 2
MSE [X ] = V[X ] + 0 = X
3n(n + 1)
2
X /n 3(n + 1)
Relative efficiency(X , X ) = 2(2n+1) 2
=
2(2n + 1)
3n(n+1) X
Its easy to see that for n 2, this ratio is smaller than 1 so X is a more
efficient estimator for X .
From point estimation to confidence interval estimation
P ( 2 (T1 (X n ), T2 (X n )) = 1
(T1 (x n ), T2 (x n ))
double inequality
z }| {
P(1 /2 quantile<C (X n , )</2 quantile) = 1
X
pX N(0, 1)
X / n
p
I Note: the standard deviation of X , n, (or any other stats) is
X/
called the standard error
Confidence interval for the population mean, normal
population with known variance
'
X N(X , X2 = 1.22 ) X = 1.2
n = 25 x = 19.8
SRS: n = 25
1 = 0.95 ) /2 = 0.025
Sample: x = 19.8 z/2 = z0.025 = 1.96
1.2
CI0.95 (X ) = 19.8 1.96 p
25
= (19.8 0.47)
Area= = (19.33, 20.27)
0.025
Interpretation: We can be 95%
confident that X is in
(19.33, 20.27)
z0.025 = 1.96
Frequency interpretation of the CI, conf. level eect
In this simulated example, 150 samples of the same size n = 50 were generated
2
from X N(X = 5, X = 12 ) and 150 CI1 (X ) were constructed with
= 0.1 and = 0.01.
X in approximately 150(0.9) = 135 ints. X in approximately 150(0.99) = 148.5 ints.
(but not in 150(0.1) = 15) (but not in 150(0.01) = 1.5)
(1 ) = 0.9, n = 50 (1 ) = 0.99, n = 50
150
150
| | | |
| |
| | | | | |
| | | |
|| | || |
| | | |
| | | |
| | | | | |
| | | | | |
| | | |
| | | |
| | | |
| | | |
| | | |
| | | | | |
| | | | | |
| | | |
| || | ||
| || | ||
| | | |
| | | |
100
100
| | | | | |
| || | ||
|| ||
| |
| || | ||
| | | |
| | | |
| | | |
| | | |
Index
Index
| | | |
| | | |
| |
| | | | | |
| | | |
| | | |
| | | |
| || | ||
|| ||
| | | | | |
| | | |
| |
| | | | | |
| | | | | |
50
50
| |
|| | || |
| | | |
| | | |
| |
|| | || |
| |
| || | ||
| | | |
| | | |
| | | |
| | | |
| | | |
| | | |
| | | |
| | | |
|| ||
| | | |
| | | |
| | | |
| |
| | | | | |
| | | |
| | | |
| | | |
0
0
6.0 5.5 5.0 4.5 4.0 6.0 5.5 5.0 4.5 4.0
150
| | || |
|
| | | |
| |
| | | |
|| | || |
|
| |
| | | | |
| | | || |
| | | | | |
| | |
| | || |
| | ||
| | | |
| | ||
| | | || |
| | | | |
| | |
| |
| || | | |
| || | | |
| | ||
| | | | |
100
100
| | | | ||
| || ||
|| |
| || |
| || |
| | | |
| | | ||
| | | |
| | | |
Index
Index
| | | |
| | |
| | ||
| | | |
| | ||
| | || |
| | || |
| || ||
|| |||
| | | ||
| | | |
| |
| | | | |
|
| | | ||
|
50
50
| |
|| | | ||
| | | |
| | | | |
|
|| | | | |
| ||
| || | |
| | | | |
| |
| | | | |
| | | |
| | |
| | |||
| | |
| | || |
|| |
| | | ||
| | | |
| | || |
|
| | | |
||
| | | ||
| |
| | | | |
0
0
6.0 5.5 5.0 4.5 4.0 6.0 5.5 5.0 4.5 4.0
The width of the interval decreases with the increasing sample size (keeping
everything else the same). Why?
'
X N(X , X2 = 1.82 ) z/2 X
2 p 1
n
p
SRS: n =? 2z/2 X n
85.93 = (2(2.575)(1.8))2 n
width
z }| {
z/2 X To satisfy the managers
CI0.99 (X ): 2 p 2(0.5) = 1 requirement, a sample of at least
n
86 observations is needed.
Area=
0.005
z0.005 = 2.575
Confidence interval for the population mean in large
samples
X X
p approx. N(0, 1)
X / n
Confidence interval for the population mean in large
samples
Z
z/2 z }| {
z }| { X X
4. Therefore P(z1 /2 < p < z/2 ) = 1
X / n
Confidence interval for the population mean in large
samples
X X
z/2 < p < z/2
X / n
'
X Bernoulli(pX ) 1 = 0.9 ) /2 = 0.05
z/2 = z0.05 = 1.645
0 s 1
SRS: n = 344 large CI0.9 (pX ) = @0.241 1.645
0.241(1 0.241)
A
344
= (0.241 0.038)
83
Sample: px = 344
= 0.241 = (0.203, 0.279)
z0.05 = 1.645
Confidence interval for the population mean, normal
population with unknown variance
X X
p tn 1
sX / n
Confidence interval for the population mean, normal
population with unknown variance
tn 1
z}|{ 1
P(tn 1;1 /2 < T < tn 1;/2 ) =1 2 2
t (Student) density tn1 ; 12 = tn1 ; 2 tn1 ; 2
n
Recall: if T tn , E[T ] = 0, V[T ] = n 2
T tn 1
tn z }| {
1;/2
z }| { X X
4. Therefore P(tn 1;1 /2 < p < tn 1;/2 ) =1
sX / n
Confidence interval for the population mean, normal
population with known variance
'
sx = 0.96 = 0.98
n=6 x = 19.48
SRS: n = 6 small 1 = 0.9 ) /2 = 0.05
116.9 tn 1;/2 = t5;0.05 = 2.015
Sample: x = 6
= 19.4833
0.98
CI0.9 (X ) = 19.48 2.105 p
2282.41 6(19.4833)2 6
sx2 = = 0.96
6 1 = (19.48 0.81)
= (18.67, 20.29)
Area= Interpretation: We can be 90%
0.05
confident that the population mean
fuel consumption for these cars, X ,
t5 ; 0.05 = 2.015 is between 18.67 and 20.29
Example: finding a confidence interval for X
Example: 8.4 (cont.) in Excel: Go to menu: Data, submenu: Data
Analysis, choose function: Descriptive Statistics.
Column A (data), in yellow (sample mean, half-width tn 1;/2 psxn , lower
end-point (cell D3-D16), upper end-point (cell D3+D16)).
2
t and distributions
I Recall that T tn if T = p Z2 , where Z N(0, 1) and 2
n follows a
n /n
chi-square distribution with df = n, independent of Z .
I On the other hand, 2n is the distribution of the sum of n independent
squared N(0, 1) random variables.
I Note that the rescaled sample quasi variance follows a chi-square
distribution with n 1 degrees of freedom
Pn n 2
(n 1)sX2 X )2 X
i=1 (Xi Xi X
2
= 2
= 2n 1
X X X
i=1
2
t and N(0, 1) densities densities
0.4
0.15
df=20
0.3
N(0,1) df=15
df=10 df=10
0.10
df=5 df=5
0.2
df=3
0.05
0.1
0.00
0.0
4 2 0 2 4 0 10 20 30 40
Confidence interval for the population variance, normal
population
(n 1)sX2 2
2 n 1
X
Confidence interval for the population variance, normal
population
P( 2
n 1;1 /2 < 2
n 1 < 2
n 1;/2 ) =1
1
2 2
Chi-square density
2n1 ; 1 2n1 ;
2 2
2 2
Recall: E[ n] = n, V[ n] = 2n
2
n 1
z }| {
2 (n 1)sX2 2
4. Therefore P( n 1;1 /2 < 2
< n 1;/2 ) =1
X
Confidence interval for the population variance, normal
population
2
5. Solve the double inequality for X:
2 (n 1)sX2 2
n 1;1 /2 < 2 < n 1;/2
X
1 2 1
2 > (n 1)sX2
X
> 2
n 1;1 /2 n 1;/2
(n 1)sX2 2 (n 1)sX2
2 > X > 2
n 1;1 /2 n 1;/2
'
X N(X , X2 ) n = 15
1 = 0.9 ) /2 = 0.05
2 2
SRS: n = 15 n 1;1 /2 = 14;0.95 = 6.57
2 2
n 1;/2 = 14;0.05 = 23.68
Sample: sx = 0.8
2 14(0.64) 14(0.64)
CI0.9 ( X) = ,
23.68 6.57
Area= Area=
= (0.378, 1.364) )
p p
0.05 0.05 CI0.9 ( X) = ( 0.378, 1.364)
= (0.61, 1.17)
214 ; 0.95 214 ; 0.05 p
=6.57 =23.68
To obtain CI ( X ) we apply to the
end-points of CI ( X2 )
Confidence intervals formulae
X X
Normal data p N(0, 1) X 2 x z/2 pX , x + z/2 pX
Known variance X/ n n n
X X
Mean Nonnormal data p approx. N(0, 1) X 2 x z/2 px , x + z/2 px
Large sample X / n n n
r #
Bernoulli data pX pX px (1 px )
q approx. N(0, 1) pX 2 px z/2
Large sample pX (1 pX )/n n
Normal data X X sx sx
p tn 1 X 2 x tn 1,/2 pn , x + tn 1,/2 pn
Unknown variance sX / n
0 1
(n 2
1)sX (n 1)sx2 (n 1)sx2
2 2 @ A
Variance Normal data 2 n 1 X 2 2 , 2
X n 1;/2 n 1;1 /2
0v v 1
2 u u
(n 1)sX (n 1)sx2 (n 1)sx2
2 @u u A
Standard dev. Normal data 2 n 1 X 2 t 2 ,t 2
X n 1;/2 n 1;1 /2
Confidence intervals for the population mean:
when to use what?
.X normal
&X normal
. known
& unknown
.
n small
&
n large
#
z-based (exact)
#
t-based (exact)
#
Methods beyond
#
z-based
Est II (approx. CLT)