Point Estimation

Statistics
Role of Statistic Inference in Decision Making

Real World

Data Collection

Estimation of parameters, choice
of distributions
(Statistic Inference)

Calculation of probability (using
prescribed distribution and
estimated parameters)

Information for decision making
generalization, estimation, parameter statistic

prediction
Populations and Samples
• Population – refers to every member of a group or set. It
can be: finite (e.g. value of a dice roll)
infinite (e.g. weight of people)
• Sample – a part of a population collected by a defined

procedure
– A statistic (note: singular!) is a value calculated from
the sample. E.g. sample mean , sample variance s2,
sample correlation coefficient r.
– Results can be useful only if the sample is
representative of the population
Point Estimate and Interval Estimates

Point estimate is a single value estimate of some
parameter of a population.
Any statistic can be a point estimate. Example:

• Sample mean 𝒙 is a point estimate of the population
mean 
• Sample variance (s2) is a point estimate of the
population variance (σ2).
Interval estimate provides a range of values, within

which a parameter is likely to be located. One example
is “confidence interval” (covered in later lectures).
Two methods of point estimation
- Method of moment
- Method of maximum likelihood
Method of moment
Mean and variance are the 1st and 2nd moments.

Sample moments to estimate “true” (population) moments
Moments used to derive the parameters of the population
distribution
Note: No. of parameters = no. of moments to be used.
E.g. for normal distribution, the parameters  and  are
respectively the mean and standard deviation
For exponential distribution, one parameter   only the
mean is used.
Some Distribution Type

Distribution PMF [pX(x)] or PDF [fX(x)] Mean, E[X] Variance, Var[X]
n
p X ( x )    p x (1  p ) n  x
Binomial  x np np(1-p)
x = 0,1,2,...,n
p X ( x) 
t x e t
Poisson x! t t
x = 0,1,2,...
1  1  x   2 
f X ( x)  exp     
Normal  2  2      2
  x  
1  1  ln x    2 
f X ( x)  exp      2)   2
x 2  2      1 2 e ( 2   e  1
Lognormal      
e 2 
0 x
f X ( x)   exp x  1 1
Exponential
x0  2
Estimation of mean
Parameter: Population mean 
Data: A random sample x1, x2, …, xn
Estimator: 1 n
x
n
 x
i 1
i
S
Standard error of estimate: x 
n
Estimation of variance Recall:
Parameter: Population variance 
 2  E[ X 2 ]   2
Data: A random sample x1, x2, …, xn
Estimator:
1 n 1  n 2  2
S 
2
 i
n  1 i 1
( x  x ) 2
   i
 x
n  1  i 1 
  n x 

(unbiased sample variance)
non-examinable
Standard Error
X = random variable
The population mean,  is a constant (although unknown)
However, the sample mean 𝑥̅ is a random variable
Standard error of 𝑥̅ = standard deviation of 𝑥̅
where x1, x2, … are uncorrelated samples
var 𝑥̅ var 𝑥 … var 𝑥
𝑛. var 𝑋 var 𝑋
since var(X) = var(xi)
Hence, 𝜎 ̅
(all xi have same variance)
Implication: taking the average of multiple samples reduces the uncertainty

(kind of makes sense)
non-examinable
𝒏
Bessel’s correction
𝒏 𝟏
When we estimate variance, ideally we can use the population

(i.e. actual) mean , but this is usually unknown.
In practice, we usually use the sample mean 𝒙, which is always
closer to the sample data compared to the population mean.
Consequently, the variance is underestimated (biased). Bessel’s
𝒏
correction makes the estimate unbiased.
𝒏 𝟏
For large n, it does not make much difference

x  n 1 n
S    ( xi  x )
2 2
 n  1  n i 1
=0
See: https://en.wikipedia.org/wiki/Bessel%27s_correction
𝒏
𝒏 𝟏
Common question:
Do we need to apply Bessel’s correction when the sample
variance is given?
Short answer:
No need to apply, use sample variance as given
Bessel correction is using (n – 1) instead of (n) when calculating

the sample variance S2 from data. It is needed to make the
estimation of S2 unbiased, otherwise we would underestimate S2.
It is widely understood that S2 should already include Bessel’s

correction. For example, if you use Matlab to calculate the sample
variance, it will use Bessel’s correction, consistent with the formula
given in the formula list.
𝒏
𝒏 𝟏
Thus, if a question provides the sample variance, or in real life

somebody provides it, you should assume that Bessel’s correction
was already considered, and you should not apply it again.
To quote Wikipedia:
https://en.wikipedia.org/wiki/Bessel%27s_correction
“This correction is so common that the term "sample variance" and

"sample standard deviation" are frequently used to mean the
corrected estimators using n − 1.”
There is one scenario where Bessel’s correction is not applied to

data in the first place. This scenario is not covered during lectures
so there is no need to worry about it. For info, it is when the
population mean  is known, and  is used in the formula, instead
of the sample mean 𝑥̅. However, this kind of scenario is not
common in practice.
Example
Data for fatigue life of aluminum yield a histogram that fits
the shape of lognormal distribution well.
Sample mean is 𝑥̅ = 26.75 million cycles
Sample variance is s2 = 360.00 (million cycles)2.
Estimate the parameter  and  of the lognormal distribution
using method of moments
Log-normal distribution
 1 
  exp    2 
 2 
 2
 2
  2 e  1 
Take ln & rearranging,
1  2 
  ln    2   ln  1  2 
2   
 S 2
 1
Hence ˆ  ln 1  2   0.64 ˆ  ln x   2  3.08
 x  2
Method of Moments using higher moments
(background only, non-examinable)
Very often, we only have data, but do not know which type of
distribution is appropriate
One solution: use a flexible distribution with more parameters,

and fit to higher moments, e.g. first four moments
Four statistical moments

1st moment, mean  X  E X 
2nd moment, variance  X2  E ( X   X ) 2 

3rd moment, skewness  X  E ( X   X ) /  X
3 3

4th moment, kurtosis  X  E ( X   X ) 4 /  X4
Method of Moments using higher moments

(background only, non-examinable)
Shifted Generalized
Lognormal Distribution
1
f X ( x) 
( x  b)2r (1  1 / r )
1/ r
 1  x  b  
r

 exp  r ln 
 r    

Four parameters
b, , , r
Fitted to 4 moments
Includes many
distributions as
special cases, e.g.
Normal distribution
Lognormal distribution
Uniform distribution
Low Y.M. (2013). A new distribution for fitting four moments and its applications to
reliability analysis. Structural Safety, 42, 12-25.

Point Estimation

Uploaded by

Copyright:

Available Formats

Point Estimation

Uploaded by

Document Information

Original Title

Copyright

Available Formats

Share this document

Share or Embed Document

Sharing Options

Did you find this document useful?

Is this content inappropriate?

Copyright:

Available Formats

Point Estimation

Uploaded by

Copyright:

Available Formats

Statistics

Role of Statistic Inference in Decision Making

generalization, estimation, parameter statistic

• Sample – a part of a population collected by a defined

Point Estimate and Interval Estimates

Any statistic can be a point estimate. Example:

Interval estimate provides a range of values, within

Mean and variance are the 1st and 2nd moments.

Some Distribution Type

where x1, x2, … are uncorrelated samples

var 𝑥̅ var 𝑥 … var 𝑥

Implication: taking the average of multiple samples reduces the uncertainty

When we estimate variance, ideally we can use the population

For large n, it does not make much difference

Bessel correction is using (n – 1) instead of (n) when calculating

It is widely understood that S2 should already include Bessel’s

Thus, if a question provides the sample variance, or in real life

“This correction is so common that the term "sample variance" and

There is one scenario where Bessel’s correction is not applied to

One solution: use a flexible distribution with more parameters,

Four statistical moments

Method of Moments using higher moments

You might also like