Order Statistics - Lecture 8

Presentation Outline
Lecture 8: Order
Statistics from
some Continuous
Distributions
Prof. Magdy E.
El-Adll
Maximum- 1 Maximum-Likelihood Estimation

Likelihood
Estimation
Exercises
2 Exercises
Prof. Magdy E. El-Adll ( Department of Mathematics, Faculty

Lecture of
8: Science,
Order Statistics
Helwan from
University,
some Continuous
Ain Helwan,Distributions
Cairo, Egypt.) 15 / 4 / 2024 1 / 22
Lecture 8: Order
Statistics from
some Continuous
Distributions
Prof. Magdy E.
El-Adll
Maximum- Functions of order statistics appear as maximum likelihood estimators (M-

Likelihood
Estimation LEs) of the parameter θ either when S, the range of the parent distribution,
Exercises depends on θ or when the sample is censored.

Lecture of
8: Science,
Order Statistics
Helwan from
University,
some Continuous
Cairo, Egypt.) 15 / 4 / 2024 2 / 22
Lecture 8: Order
Statistics from
some Continuous
Distributions
Prof. Magdy E.
El-Adll
Maximum- Functions of order statistics appear as maximum likelihood estimators (M-

Likelihood
Estimation LEs) of the parameter θ either when S, the range of the parent distribution,
Exercises depends on θ or when the sample is censored.

Lecture of
8: Science,
Order Statistics
Helwan from
University,
some Continuous
Cairo, Egypt.) 15 / 4 / 2024 2 / 22
Lecture 8: Order
Statistics from
some Continuous
Distributions Instead of having a general discussion on the finite sample and asymptotic
Prof. Magdy E. properties of θ, the MLE of θ, we start with several examples to highlight
El-Adll
various aspects of MLEs based on order statistics.
Maximum-
Likelihood
∂ log L(θ)
Estimation The likelihood equation for censored samples is given by , where
Exercises
∂θ
L is the likelihood function given by (??) for Type I censored samples and
(??) for censored samples of Type II. For most of probability distributions, it
does not produce closed-form solutions. In these situations, the MLE must
be determined numerically, which can require extensive computer-intensive
calculations.

Lecture of
8: Science,
Order Statistics
Helwan from
University,
some Continuous
Cairo, Egypt.) 15 / 4 / 2024 3 / 22
Lecture 8: Order
Statistics from
some Continuous
Distributions Instead of having a general discussion on the finite sample and asymptotic
Prof. Magdy E. properties of θ, the MLE of θ, we start with several examples to highlight
El-Adll
various aspects of MLEs based on order statistics.
Maximum-
Likelihood
∂ log L(θ)
Estimation The likelihood equation for censored samples is given by , where
Exercises
∂θ
L is the likelihood function given by (??) for Type I censored samples and
(??) for censored samples of Type II. For most of probability distributions, it
does not produce closed-form solutions. In these situations, the MLE must
be determined numerically, which can require extensive computer-intensive
calculations.

Lecture of
8: Science,
Order Statistics
Helwan from
University,
some Continuous
Cairo, Egypt.) 15 / 4 / 2024 3 / 22
Lecture 8: Order
Statistics from
some Continuous
Distributions
Prof. Magdy E.
In both of these situations, θ̂ is consistent for θ and θ̂, when appropriately
El-Adll normalized, it is asymptotically normal under certain regularity conditions. In
Maximum- Lawless (1982), the MLEs for several popular distributions are discussed in
Likelihood
Estimation detail.
Exercises
Example 1 (The uniform distribution)
Let Y be uniformly distributed over the interval (0, θ). What is the MLE of
θ for complete samples of size n? Discuss their properties.

Lecture of
8: Science,
Order Statistics
Helwan from
University,
some Continuous
Cairo, Egypt.) 15 / 4 / 2024 4 / 22
Lecture 8: Order
Statistics from
some Continuous
Distributions
Prof. Magdy E.
In both of these situations, θ̂ is consistent for θ and θ̂, when appropriately
El-Adll normalized, it is asymptotically normal under certain regularity conditions. In
Maximum- Lawless (1982), the MLEs for several popular distributions are discussed in
Likelihood
Estimation detail.
Exercises
Example 1 (The uniform distribution)
Let Y be uniformly distributed over the interval (0, θ). What is the MLE of
θ for complete samples of size n? Discuss their properties.

Lecture of
8: Science,
Order Statistics
Helwan from
University,
some Continuous
Cairo, Egypt.) 15 / 4 / 2024 4 / 22
Lecture 8: Order
Statistics from
some Continuous
Distributions
Prof. Magdy E.
El-Adll
It is well known that the MLE of θ is θ̂ = Yn:n . Thus, θ is a complete
Maximum-
Likelihood (n + 1)θ̂
Estimation sufficient statistic, and is the MVUE of θ. Since the range of
n
Exercises
the distribution depends on θ, the regularity conditions that are sufficient to
ensure the normality of θ̂ do not hold.

Lecture of
8: Science,
Order Statistics
Helwan from
University,
some Continuous
Cairo, Egypt.) 15 / 4 / 2024 5 / 22
Lecture 8: Order
Statistics from
some Continuous
Distributions
Prof. Magdy E.

El-Adll Yn:n − θ y
P ≤y = P Yn:n ≤ θ 1 +
Maximum- θ/n n
Likelihood
Estimation
n y o n
= F θ 1+
Exercises
 n
 0, y < −n;
 y n
= 1+ , −n ≤ y ≤ 0;
 n
1, y > 0.


Lecture of
8: Science,
Order Statistics
Helwan from
University,
some Continuous
Cairo, Egypt.) 15 / 4 / 2024 6 / 22
Lecture 8: Order
Statistics from
some Continuous
Distributions
Prof. Magdy E.

El-Adll Yn:n − θ y
P ≤y = P Yn:n ≤ θ 1 +
Maximum- θ/n n
Likelihood
Estimation
n y o n
= F θ 1+
Exercises
 n
 0, y < −n;
 y n
= 1+ , −n ≤ y ≤ 0;
 n
1, y > 0.


Lecture of
8: Science,
Order Statistics
Helwan from
University,
some Continuous
Cairo, Egypt.) 15 / 4 / 2024 6 / 22
Lecture 8: Order
Statistics from
some Continuous
Distributions
Prof. Magdy E.
El-Adll
Therefore, we have P (n(θ̂ − θ)/θ ≤ y) → ey , y ≤ 0. This limiting CDF is
that of −Z where Z is the standard exponential distribution. This is one of
Maximum-
Likelihood the three possible limit distributions for the sample maximum, as we will see
Estimation
Exercises
in the next chapter. The normal distribution is not one of them.
For Type II censored sample where only the first r order statistics are observed,
nYr:n
we have θ̂ = . If r = [np], 0 < p < 1, θ̂ is asymptotically normal.
r

Lecture of
8: Science,
Order Statistics
Helwan from
University,
some Continuous
Cairo, Egypt.) 15 / 4 / 2024 7 / 22
MLE based on Type II censored samples
Lecture 8: Order
Statistics from
some Continuous
Distributions
Prof. Magdy E.
El-Adll
Maximum-
Likelihood
Suppose that the first r order statistics, Y1:n , Y2:n , ..., Yr:n , from the expo-
Estimation nential distribution with a PDF f (y; θ) = 1θ e−y/θ are observed. Find the
Exercises
maximum likelihood estimator of θ and discuss its properties.

Lecture of
8: Science,
Order Statistics
Helwan from
University,
some Continuous
Cairo, Egypt.) 15 / 4 / 2024 8 / 22
Likelihood function for Type II censored samples
Lecture 8: Order
Statistics from
some Continuous
Distributions Recall that the likelihood function based on Type II censored samples is given
Prof. Magdy E. by
El-Adll
Maximum-
Likelihood r
!
Estimation n! Y
Exercises L(θ|r, y) = f (yi ; θ) [1 − F (yr ; θ)]n−r ,
(n − r)! i=1
for y1 < y2 < · · · < yr ., 0 < r ≤ n.
For the exponential distribution, the previous likelihood function takes the
form

Lecture of
8: Science,
Order Statistics
Helwan from
University,
some Continuous
Cairo, Egypt.) 15 / 4 / 2024 9 / 22
Lecture 8: Order
Statistics from
some Continuous r !
n! Y 1 n−r
e−yi /θ e−yr /θ
Distributions
L(θ|r, y) = ,
Prof. Magdy E.
El-Adll
(n − r)! i=1
θ
( r
!)
Maximum- n! 1 X
Likelihood
= θ−r exp − yi + (n − r)yr ,
Estimation
(n − r)! θ i=1
Exercises
for y1 < y2 < · · · < yr , 0 < r ≤ n,

where y = (y1 , y2 , ..., yr ) is the observed vector of order statistics. Hence,
the log likelihood function is
r
!
1 X
`(θ) := log(L(θ|r, y)) = Cr,n − r log(θ) − yi + (n − r)yr .
θ i=1
Lecture of
8: Science,
Order Statistics
Helwan from
University,
some Continuous
Cairo, Egypt.) 15 / 4 / 2024 10 / 22
Log likelihood function and MLE
Lecture 8: Order
Statistics from Hence, the log likelihood function is
some Continuous
Distributions !
r
Prof. Magdy E. 1 X
El-Adll `(θ) := log(L(θ|r, y)) = Cr,n − r log(θ) − yi + (n − r)yr .
θ i=1
Maximum-
Likelihood
Estimation
Differentiating the log likelihood with respect to θ and equating the deriva-
Exercises
tive to zero, and using the random variables in the solution instead of their
observed values, we get the MLE of θ as
( r ) ( r )
1 X 1 X
θ̂ = Yi:n + (n − r)Yr:n = (n − i + 1)(Yi:n − Yi−1:n ) .
r i=1 r i=1
(1)
Lecture of
8: Science,
Order Statistics
Helwan from
University,
some Continuous
Cairo, Egypt.) 15 / 4 / 2024 11 / 22
Lecture 8: Order
Statistics from
some Continuous
Distributions
Prof. Magdy E.
El-Adll
h ∂ 2 `(θ) i
It can be shown after some algebraic calculations that < 0,
Maximum- ∂θ2 θ=θ̂
Likelihood
Estimation which means that θ̂ is the MLE of θ.
Exercises rθ̂
In view of (1) and Theorem ??, the random variable follows gamma
θ
distribution with scale parameter of one and shape parameter of r. That is

Lecture of
8: Science,
Order Statistics
Helwan from
University,
some Continuous
Cairo, Egypt.) 15 / 4 / 2024 12 / 22
Lecture 8: Order
Statistics from "#
some Continuous
Distributions rθ̂ h i
E = r, which implies E θ̂ = θ.
Prof. Magdy E.
El-Adll
θ
Maximum-
Likelihood That is, θ̂ is an unbiased estimator for estimating θ. Moreover,
Estimation
!
Exercises
rθ̂ r2
V ar = 2 V ar θ̂ = r.
θ θ
Or equivalently,
θ2
V ar θ̂ = → 0 as r → ∞.
r
Lecture of
8: Science,
Order Statistics
Helwan from
University,
some Continuous
Cairo, Egypt.) 15 / 4 / 2024 13 / 22
Lecture 8: Order
Statistics from
some Continuous
Distributions Therefore, θ̂ is a consistent estimator for estimating θ. Furthermore, an ap-
Prof. Magdy E. plication of the central limit theorem thus yields
El-Adll
√
Maximum- r(θ̂ − θ) d
−→ N (0, 1) as r → ∞.
Likelihood
Estimation
Exercises
θ̂
Note that θ̂ is a complete sufficient statistic for the parameter θ. Therefore,

θ̂ is the MVUE of θ, and since it is a linear function of the order statistics
r
Yi:n ’s, θ̂ is also the BLUE of θ. The Fisher information contained in θ, is 2
θ
which is a linear function of r and is free of n.

Lecture of
8: Science,
Order Statistics
Helwan from
University,
some Continuous
Cairo, Egypt.) 15 / 4 / 2024 14 / 22
Lecture 8: Order
Statistics from
some Continuous
Distributions
In the reliability literature, the statistic rθ̂ = ri=1 Yi:n + (n − r)Yr:n , which
P
Prof. Magdy E.
El-Adll
represents the total time on test when the experiment was stopped at the
Maximum-
Likelihood rth failure, plays an important role.
Estimation
Exercises Example 2 (Type I censored samples for the exponential distribution)

Find the maximum likelihood estimator of θ based on type I censored samples
from one-parameter exponential distribution.

Lecture of
8: Science,
Order Statistics
Helwan from
University,
some Continuous
Cairo, Egypt.) 15 / 4 / 2024 15 / 22
Lecture 8: Order
Statistics from
some Continuous
Distributions
Clearly, likelihood function based on Type I censored sample from the expo-
Prof. Magdy E.
El-Adll nential distribution is given by
Maximum- 
Likelihood n! −r 1 Pr
θ exp − ( i=1 yi + (n − r)t) ,
Estimation


 (n − r)! θ


Exercises
L(θ|r, y) = for r > 0 and y1 < y2 < · · · < yr ,

 nt
 exp − , for r = 0,


θ

Lecture of
8: Science,
Order Statistics
Helwan from
University,
some Continuous
Cairo, Egypt.) 15 / 4 / 2024 16 / 22
Lecture 8: Order
Statistics from
some Continuous
Distributions where t is the censoring time. It is clear that, if r = 0, the likelihood function
Prof. Magdy E.
El-Adll
is monotonically increasing and, hence, the MLE of θ does not exist. But
the probability
nt of such an event (no failures are observed before the time t)
Maximum-
Likelihood is exp − θ , decreases with n. The MLE of θ provided that R > 0 is given
Estimation
Exercises
by PR
Yi:n + (n − R)t
θ̂ = i=1 .
R
Since R is a discrete random variable, the exact distribution of θ̂ is not a
simple problem.

Lecture of
8: Science,
Order Statistics
Helwan from
University,
some Continuous
Cairo, Egypt.) 15 / 4 / 2024 17 / 22
Presentation Outline
Lecture 8: Order
Statistics from
some Continuous
Distributions
Prof. Magdy E.
El-Adll
Maximum- 1 Maximum-Likelihood Estimation

Likelihood
Estimation
Exercises
2 Exercises

Lecture of
8: Science,
Order Statistics
Helwan from
University,
some Continuous
Cairo, Egypt.) 15 / 4 / 2024 18 / 22
Exercises 1.a & 1.b
Lecture 8: Order
Statistics from
some Continuous
Distributions
Prof. Magdy E.
El-Adll
1 Suppose we have a Type II censored sample from Uniform(0, θ) where
Maximum-
Likelihood only the first r order statistics are observed.
Estimation
nYr:n
Exercises 1 Show that θ̂ = is the MLE of θ.
r
2 If r = n is θ̂ the MLE of θ asymptotically normal? Why?

Lecture of
8: Science,
Order Statistics
Helwan from
University,
some Continuous
Cairo, Egypt.) 15 / 4 / 2024 19 / 22
Exercises 2.a & 2.b
Lecture 8: Order
Statistics from
some Continuous
Distributions 1 Assume we are sampling from the Pareto distribution with shape param-
Prof. Magdy E.
El-Adll eter β whose pdf is given by
f (y; β) = βy −(β+1) ,
Maximum-
Likelihood y ≥ 1 and β > 0.
Estimation
Exercises
1 Find the Fisher information measure contained in the right-censored

sample Y1:n , · · · , Yr:n about the parameter β.
2 Find the MLE of β based on the data in (a) and determine its limiting
distribution as r becomes large. What is its limiting variance?

Lecture of
8: Science,
Order Statistics
Helwan from
University,
some Continuous
Cairo, Egypt.) 15 / 4 / 2024 20 / 22
Exercises 3.a & 3.b
Lecture 8: Order
Statistics from
some Continuous
Distributions
1 Suppose we are sampling from the Rayleigh distribution with CDF
Prof. Magdy E.
El-Adll ( )
2
y
Maximum- F (y; β) = 1 − exp − , y ≥ 0 and β > 0.
Likelihood
Estimation
β
Exercises
1 Assuming the data is a Type II censored sample Y1:n , · · · , Yr:n , deter-

mine β̂, the MLE of β.
2 Now suppose we have a Type I censored sample being censored to the
right at time t. What will be β̃, the MLE of β for this situation?

Lecture of
8: Science,
Order Statistics
Helwan from
University,
some Continuous
Cairo, Egypt.) 15 / 4 / 2024 21 / 22
Exercise 4
Lecture 8: Order
Statistics from
some Continuous
Distributions
Prof. Magdy E.
El-Adll
Maximum- 1 Assuming that we are picking a random sample from a continuous pop-
Likelihood
Estimation ulation, find the smallest sample size n for which the probability level
Exercises of the interval Y1:n , Yn:n is at least 95% when it is used as a confidence
interval for the population median.

Lecture of
8: Science,
Order Statistics
Helwan from
University,
some Continuous
Cairo, Egypt.) 15 / 4 / 2024 22 / 22

Order Statistics - Lecture 8

Uploaded by

Copyright:

Available Formats

Order Statistics - Lecture 8

Uploaded by

Document Information

Original Description:

Copyright

Available Formats

Share this document

Share or Embed Document

Sharing Options

Did you find this document useful?

Is this content inappropriate?

Copyright:

Available Formats

Order Statistics - Lecture 8

Uploaded by

Copyright:

Available Formats

Presentation Outline

Maximum- 1 Maximum-Likelihood Estimation

Prof. Magdy E. El-Adll ( Department of Mathematics, Faculty

Maximum- Functions of order statistics appear as maximum likelihood estimators (M-

Prof. Magdy E. El-Adll ( Department of Mathematics, Faculty

Maximum- Functions of order statistics appear as maximum likelihood estimators (M-

Prof. Magdy E. El-Adll ( Department of Mathematics, Faculty

Prof. Magdy E. El-Adll ( Department of Mathematics, Faculty

Prof. Magdy E. El-Adll ( Department of Mathematics, Faculty

Prof. Magdy E. El-Adll ( Department of Mathematics, Faculty

Prof. Magdy E. El-Adll ( Department of Mathematics, Faculty

Prof. Magdy E. El-Adll ( Department of Mathematics, Faculty

Prof. Magdy E. El-Adll ( Department of Mathematics, Faculty

Prof. Magdy E. El-Adll ( Department of Mathematics, Faculty

Prof. Magdy E. El-Adll ( Department of Mathematics, Faculty

Prof. Magdy E. El-Adll ( Department of Mathematics, Faculty

Prof. Magdy E. El-Adll ( Department of Mathematics, Faculty

for y1 < y2 < · · · < yr , 0 < r ≤ n,

Prof. Magdy E. El-Adll ( Department of Mathematics, Faculty

Note that θ̂ is a complete sufficient statistic for the parameter θ. Therefore,

Prof. Magdy E. El-Adll ( Department of Mathematics, Faculty

Exercises Example 2 (Type I censored samples for the exponential distribution)

Prof. Magdy E. El-Adll ( Department of Mathematics, Faculty

Prof. Magdy E. El-Adll ( Department of Mathematics, Faculty

Prof. Magdy E. El-Adll ( Department of Mathematics, Faculty

Maximum- 1 Maximum-Likelihood Estimation

Prof. Magdy E. El-Adll ( Department of Mathematics, Faculty

Prof. Magdy E. El-Adll ( Department of Mathematics, Faculty

1 Find the Fisher information measure contained in the right-censored

Prof. Magdy E. El-Adll ( Department of Mathematics, Faculty

1 Assuming the data is a Type II censored sample Y1:n , · · · , Yr:n , deter-

Prof. Magdy E. El-Adll ( Department of Mathematics, Faculty

Prof. Magdy E. El-Adll ( Department of Mathematics, Faculty

You might also like