ch7

Chapter 7
Point Estimation of Parameters and

Sampling Distributions
Introduction
It is desired to estimate a parameter (like a
population mean , a population proportion
p, or a population standard deviation ) by
using the results of a random sample.
Point Estimation
Let  denote a parameter.
Defn A point estimate of a parameter  is the

observed value of a suitable statistic which
is regarded as a plausible value of . The
selected statistic is called the point
estimator of .
Point Estimation
Typical Point Typical
Point
Parameter Estimator Estimate
X X

p̂ p̂
p = X/n = x/n
 S s
Point Estimation
EXA We want to estimate the proportion of
people in Cleveland who like liver. A random
sample of 500 Clevelanders is taken resulting
in 110 saying they like liver.
Random Sample, Statistic
The random variables X1, X2,…,Xn are a
random sample of size n iff the Xi ‘s are
independent random variables where each Xi
has the same probability distribution (iid).
A statistic is any function of the observations

in a random sample. The probability
distribution of a statistic is called a sampling
distribution.
Central Limit Theorem (CLT)
Assume X1, X2,…,Xn is a random sample of
size n from an unknown population or a non-
normal population with mean  and standard
deviation , then
X is approximately N(,2/n)
for large n.
Rule of Thumb: Large n  n > 30.
Illustration of CLT using R
xbar=matrix(0,1000)
for(i in 1:1000)
{
xbar[i]=mean(rexp(2))
}
mean(xbar)
sd(xbar)
hist(xbar)
qqnorm(xbar)
CLT
EXA The number of flaws per square yard in
a type of carpet material varies with mean 1.3
flaws per square yard and standard deviation
1.5 flaws per square yard. An inspector
samples at random 200 square yards of the
material. Find the probability that the mean
number of flaws is at most 1 per square yard.
Unbiased Estimators
Defn θˆ is an unbiased estimator of  iff
θˆ ) = . Also, the bias ofθˆ
E(
equals θˆ
E( ) - .
X
EXA a. Is p̂ an unbiased estimator of ?
b. Is an unbiased estimator of p?
c. Is S2 an unbiased estimator of
2.
Standard Error
Defn Standard error of θˆ = θˆ = Var( θ̂ ) .
If  ˆ contains unknown parameters that can

θ
be estimated, the estimated standard error
of θˆ is found by plugging in estimates of
these unknown parameters in the standard
error of θˆ (notation: SE( θˆ )).
Standard Error
The standard error should be given along with
the point estimate to indicate the precision
of the estimate.
EXA Estimate a population mean  when  is

unknown.
Goal for Point Estimators
We want a point estimator for θ that has
small bias and small spread.
If you have an unbiased estimator, then the

bias is 0, which is as small as you can get
it.
The sample size affects the spread. If the

spread is too large, increase n.
Minimum Variance Unbiased Estimator
(MVUE)
Among the class of all unbiased estimators of
θ, the one with the smallest variance is called
the minimum variance unbiased estimator
(MVUE).
If X1, X2,…, Xn is a random sample of size n

from a normal distribution with mean μ and
variance σ2, then X is the MVUE for μ.
Mean Squared Error
The mean squared error of θˆ in estimating
θ is
MSE = E[ ( θˆ - θ)2 ].
Note: MSE = Var( θˆ ) + bias2 .
EXA MSE of X in estimating μ is σ2/n.

Mean Squared Error
Recall that we want a point estimator for θ that
has small bias and small spread.
Small spread can be measured by small MSE.
Say we have two unbiased estimators for θ.

Considering their two MSE’s, the estimator
with the smaller MSE is the preferred
estimator.
Methods of Point Estimation
We will consider methods of moments and

maximum likelihood estimation.
Method of Moments
Let X1, X2,…,Xn be a random sample from the
probability distribution f(x), where f(x) can be a
discrete probability mass function or a
continuous probability density function.
The kth population moment (or distribution

moment) is E(Xk), k = 1, 2, ….
For k = 1, we have E(X) = μ which is the

population mean or first population moment.
Method of Moments (m.o.m)
The kth sample moment is
For k = 1, we have X .
In m.o.m. estimation, the kth population

moment is estimated by the kth sample
moment.
Always start out by writing down the mean and
see if the parameter of interest can be written
as a function of the mean.
EXA Assume that X1, X2,…,Xn is a random

sample from a Binomial(1,p) (which is a
Bernoulli(p)) distribution. Find m.o.m.
estimator of p.

sample from a Binomial(m,p) distribution. Find
m.o.m. estimator of p.

sample from an Exponential(λ) distribution.
Find m.o.m. estimator of λ.

sample from a N(,2) distribution. Find
m.o.m. estimator of (,2).
Maximum Likelihood Estimation
(MLE)
Assume that X is a random variable with
probability distribution f(x;θ), where θ is a
single unknown parameter. Let x1, x2, …, xn be
the observed values in a random sample of
size n. Then the likelihood function of the
sample is:
L(θ) = f(x1;θ) ∙ f(x2; θ) ∙…∙ f(xn; θ).

MLE
The likelihood function is a function of only the
unknown parameter θ. The maximum
likelihood estimator (MLE) of θ is the value of θ
that maximizes the likelihood function L(θ).

sample from a Binomial(m,p) distribution. Find
the MLE of p.
MLE
EXA Assume that X1, X2,…,Xn is a random sample
from an Exponential(λ) distribution. Find the MLE of λ.
EXA Assume that X1, X2,…,Xn is a random sample

from a N(,2) distribution. Find the MLE of (,2).
Properties of the MLE
Under very general and not restrictive conditions when
n is large (i.e., things are great in the world),
1. the MLE of θ has little bias for θ,
2. the variance of the MLE of θ is nearly as
small as the variance of any other
estimator of θ,
and
3. the MLE is approximately normal.
Properties of the MLE
For a function h(θ), the MLE of h(θ) is h( θˆ ).

sample from an Exponential(λ) distribution.
Find the MLE of 1/λ.
MLE
Sometimes the procedure to find the MLE needs
to be adapted a little bit.

sample from an Uniform(0,θ) distribution. Find
the MLE of θ.

ch7

Uploaded by

Copyright:

Available Formats

ch7

Uploaded by

Document Information

Copyright

Available Formats

Share this document

Share or Embed Document

Sharing Options

Did you find this document useful?

Is this content inappropriate?

Copyright:

Available Formats

ch7

Uploaded by

Copyright:

Available Formats

Chapter 7

Point Estimation of Parameters and

Defn A point estimate of a parameter  is the

A statistic is any function of the observations

If  ˆ contains unknown parameters that can

EXA Estimate a population mean  when  is

If you have an unbiased estimator, then the

The sample size affects the spread. If the

If X1, X2,…, Xn is a random sample of size n

Note: MSE = Var( θˆ ) + bias2 .

EXA MSE of X in estimating μ is σ2/n.

Small spread can be measured by small MSE.

Say we have two unbiased estimators for θ.

We will consider methods of moments and

The kth population moment (or distribution

For k = 1, we have E(X) = μ which is the

In m.o.m. estimation, the kth population

EXA Assume that X1, X2,…,Xn is a random

EXA Assume that X1, X2,…,Xn is a random

EXA Assume that X1, X2,…,Xn is a random

EXA Assume that X1, X2,…,Xn is a random

L(θ) = f(x1;θ) ∙ f(x2; θ) ∙…∙ f(xn; θ).

EXA Assume that X1, X2,…,Xn is a random

EXA Assume that X1, X2,…,Xn is a random sample

EXA Assume that X1, X2,…,Xn is a random

EXA Assume that X1, X2,…,Xn is a random

You might also like