Supplemental Material for Chapter 5

S5-1. s
2
is not Always an Unbiased Estimator of
2

An important property of the sample variance is that it is an unbiased estimator of the
population variance, as demonstrated in Section S3-3 of the Supplemental Text Material.
However, this unbiased property depends on the assumption that the sample data has been
drawn from a stable process; that is, a process that is in statistical control. In statistical
quality control work we sometimes make this assumption, but if it is incorrect, it can have
serious consequences on the estimates of the process parameters we obtain.
To illustrate, suppose that in the sequence of individual observations
x x x x x
t t m 1 2 1
, , , , , ,
+
the process is in-control with mean
0
and standard deviation for the first t observations,
but between x
t
and x
t+1
an assignable cause occurs that results in a sustained shift in the
process mean to a new level
+
0
and the mean remains at this new level for the
remaining sample observations
x x
t m +1
, ,
. Under these conditions, Woodall and
Montgomery (2000-01) show that

2 2 2
( )
( ) ( ) .
( 1)
t m t
E s
m m

(S5-1)
In fact, this result holds for any case in which the mean of t of the observations is
0
and
the mean of the remaining observations is

0
+
, since the order of the observations is
not relevant in computing s
2
. Note that s
2
is biased upwards; that is, s
2
tends to
overestimate
2
. Furthermore, the extent of the bias depends on the magnitude of the shift
in the mean (), the time period following which the shift occurs (t), and the number of
available observations (m). For example, if there are m = 25 observations and the process
mean shifts from
0
to
0
(that is, 1) +
between the 20
th
and the 21
st
observation
(t = 20), then s
2
will overestimate
2
by 16.7% on average. If the shift in the mean occurs
earlier, say between the 10
th
and 11
th
observations, then s
2
will overestimate
2
by 25% on
average.
The proof of Equation S5-1 is straightforward. Since we can write
2 2 2
1
1
1
m
i
i
s x mx
m

_

then
2 2 2 2 2
1 1
1 1
( ) ( ) ( )
1 1
m m
i i
i i
E s E x mx E x mE x
m m

_ _



, ,

Now
1
1
1
1
1
1
1
1
1
2
1
2 2
1 1
0
2 2
0
2 2
0
2
0
2 2
m
E x
m
E x E x
m
t m t m t
m
t m t m
i
i
m
i i
i t
m
i
t

F
H
G
I
K
J

+
F
H
G
I
K
J

+ + + +

+ + +
+

( ) ( ) ( )
( ) ( )( ) ( )
( )( )


c h
c h
and
1
1 1
2
0
2
2
m
mE x
m
m
m t
m m

+

F
H
G
I
K
J
F
H
G
I
K
J
+
L
N
M
M
O
Q
P
P
( )

Therefore
( )
2
2
2 2 2 2
0 0 0
2
2 2 2
0 0 0
2
2 2 2
2 2
1
( ) ( )( )
1
1
( )( )
1
1 ( )
( )( ) ( )
1
1 ( )
( )( ) 1
1
m t
E s t m t m m
m m m
m t
t m t m
m m
m t
m t
m m
m t
m t
m m

1 _
_ _
1 + + + + +

, 1 ,
, ]
1
_ _
+ + + + 1

, , 1
]
1
+
1

]

+

2 2
( )
( )
( 1)
t m t
m m

1 _
1
, ]

S5-2. Should We Use

2
d or
*
2
d in Estimating

via the Range Method?

In the textbook, we make use of the range method for estimation of the process standard
deviation, particularly in constructing variables control charts (for example, see the
and x R charts of Chapter 5). We use the estimator
2
/ R d . Sometimes an alternative
estimator,
*
2
/ R d , is encountered. In this section we discuss the nature and potential uses of
these two estimators. Much of this discussion is adapted from Woodall and Montgomery
(2000-01). The original work on using ranges to estimate the standard deviation of a
normal distribution is due to Tippett (1925). See also the paper by Duncan (1955).
Suppose one has m independent samples, each of size n, from one or more populations
assumed to be normally distributed with standard deviation . We denote the sample
2
ranges of the m samples or subgroups as
R R R
m 1 2
, , , .
Note that this type of data arises
frequently in statistical process control applications and gauge repeatability and
reproducibility (R & R) studies (refer to Chapter 7). It is well-known that
E R d Var R d
i i
( ) ( ) ,
2 3
2 2
and for i m 1 2 , , , where d
2
and d
3
are constants that
depend on the sample size n. Values of these constants are tabled in virtually all textbooks
and training materials on statistical process control. See, for example Appendix table VI
for values of d
2
and d
3
for n = 2 to 25.
There are two estimators of the process standard deviation based on the average sample
range,

R
R
m
i
i
m

1
, (S5-2)
that are commonly encountered in practice. The estimator

/
1 2
R d (S5-3)
is widely used after the application of control charts to estimate process variability and to
assess process capability. In Chapter 3 we report the relative efficiency of the range
estimator given in Equation (S5-3) to the sample standard deviation for various sample
sizes. For example, if n = 5, the relative efficiency of the range estimator compared to the
sample standard deviation is 0.955. Consequently, there is little practical difference
between the two estimators. Equation (S5-3) is also frequently used to determine the usual
3-sigma limits on the Shewhart chart x in statistical process control. The estimator

/
*

2 2
R d (S5-4)
is more often used in gauge R & R studies and in variables acceptance sampling. Here d
2
*
represents a constant whose value depends on both m and n. See Chrysler, Ford, GM
(1995), Military Standard 414 (1957), and Duncan (1986).
Patnaik (1950) showed that R / is distributed approximately as a multiple of a -
distribution. In particular, R / is distributed approximately as d
2
*
/ , where
represents the fractional degrees of freedom for the distribution. Patnaik (1950) used the
approximation

,
_

+ +
3 2 2
*
2
128
5
32
1
4
1
1

d d
. (S5-5)
It has been pointed out by Duncan (1986), Wheeler (1995), and Luko (1996), among
others, that

1
is an unbiased estimator of and that

2
2
is an unbiased estimator of
2
. For

2
2
to be an unbiased estimator of
2
, however, David (1951) showed that no approximation
for d
2
*
was required. He showed that
d d V m
n 2 2
2 1 2 * /
( / ) , + (S5-6)
3
where V
n
is the variance of the sample range with sample size n from a normal population
with unit variance. It is important to note that V d
n

3
2
, so Equation (S5-6) can be easily
used to determine values of d
2
*
from the widely available tables of d
2
and d
3
. Thus, a table
of d
2
*
values, such as the ones given by Duncan (1986), Wheeler (1995), and many others,
is not required so long as values of d
2
and d
3
are tabled, as they usually are (once again, see
Appendix Table VI). Also, use of the approximation

,
_

4
1
1
2 2
d d
given by Duncan (1986) and Wheeler (1995) becomes unnecessary.
The table of d
2
*
values given by Duncan (1986) is the most frequently recommended. If a
table is required, the ones by Nelson (1975) and Luko (1996) provide values of d
2
*
that are
slightly more accurate since their values are based on Equation (S5-6).
It has been noted that as m increases, d
2
*
approaches d
2
. This has frequently been argued
using Equation (19-4) and noting that increases as m increases. The fact that d
2
*
approaches d
2
as m increases is more easily seen, however, from Equation (19-5) as pointed
out by Luko (1996).
Sometimes use of Equation (S5-4) is recommended without any explanation. See, for
example, the AIAG measurement systems capability guidelines [Chrysler, Ford, and GM
(1995)]. The choice between

1
and

2
has often not been explained clearly in the
literature. It is frequently stated that the use of Equation (S5-3) requires that
R
be obtained
from a fairly large number of individual ranges. See, for example, Bissell (1994, p. 289).
Grant and Leavenworth (1996, p. 128) state that Strictly speaking, the validity of the exact
value of the d
2
factor assumes that the ranges have been averaged for a fair number of
subgroups, say, 20 or more. When only a few subgroups are available, a better estimate of
is obtained using a factor that writers on statistics have designated as d
2
*
. Nelson
(1975) writes, If fewer than a large number of subgroups are used, Equation (S5-3) gives
an estimate of which does not have the same expected value as the standard deviation
estimator. In fact, Equation (S5-3) produces an unbiased estimator of regardless of the
number of samples m, whereas the pooled standard deviation does not (refer to Section S3-
5 of the Supplemental Text Material). The choice between

1
and

2
depends upon
whether one is interested in obtaining an unbiased estimator of or
2
. As m increases,
both estimators (S5-3) and (S5-4) become equivalent since each is a consistent estimator of
.
It is interesting to note that among all estimators of the form cR c ( ), > 0 the one
minimizing the mean squared error in estimating has
c d d
2 2
2
/ ( )
*
.
4
The derivation of this result is in the proofs at the end of this section. If we let

( )
*

3
2
2
2

d
d
R
then it is shown in the proofs below that
MSE
d
d
(

)
( )
*

3
2
2
2
2
1
F
H
G
I
K
J
Luko (1996) compared the mean squared error of

2
in estimating to that of

1
and
recommended

2
on the basis of uniformly lower MSE values. By definition,

3
leads to further reduction in MSE. It is shown in the proofs at the end of this section
that the percentage reduction in MSE using

3
instead of

2
is
50
2 2
2
d d
d
*
*
F
H
G
I
K
J
Values of the percentage reduction are given in Table S5-1. Notice that when both the
number of subgroups and the subgroup size are small, a moderate reduction in mean
squared error can be obtained by using

3
.
Table S5-1.
Percentage Reduction in Mean Squared Error from using

3
instead of

2
Subgroup
Size, n
Number of Subgroups, m
1 2 3 4 5 7 10 15 20
2 10.1191 5.9077 4.1769 3.2314 2.6352 1.9251 1.3711 0.9267 0.6998
3 5.7269 3.1238 2.1485 1.6374 1.3228 0.9556 0.6747 0.4528 0.3408
4 4.0231 2.1379 1.4560 1.1040 0.8890 0.6399 0.4505 0.3017 0.2268
5 3.1291 1.6403 1.1116 0.8407 0.6759 0.4856 0.3414 0.2284 0.1716
6 2.5846 1.3437 0.9079 0.6856 0.5507 0.3952 0.2776 0.1856 0.1394
7 2.2160 1.1457 0.7726 0.5828 0.4679 0.3355 0.2356 0.1574 0.1182
8 1.9532 1.0058 0.6773 0.5106 0.4097 0.2937 0.2061 0.1377 0.1034
9 1.7536 0.9003 0.6056 0.4563 0.3660 0.2623 0.1840 0.1229 0.0923
10 1.5963 0.8176 0.5495 0.4138 0.3319 0.2377 0.1668 0.1114 0.0836
Proofs
5
Result 1: Let

, (

) [ ( ) ]
*
+ cR MSE c d cd then
2 2
2
2
2
2 1
Proof:
MSE E cR
c R c R
c E R c E R
(

) [( ) ]
[ ]
( ) ( )




+
+
2
2 2 2
2 2 2
2
2
Now E R Var R E R d m d ( ) ( ) [ ( )] / ( )
2 2
3
2 2
2
2
+ + . Thus
MSE c d m c d c d
c d m d cd
c d cd
(

) / ( )
[ ( / ) ]
[ ( ) ]
*

+ +
+ +
+
2
3
2 2 2
2
2 2
2
2
2 2
3
2
2
2
2
2 2
2
2
2
2
2 1
2 1
Result 2: The value of c that minimizes the mean squared error of estimators of the form
cR in estimating is
d
d
2
2
2
( )
* .
Proof:
MSE c d cd
dMSE
dc
c d d
c
d
d
(

) [ ( ) ]
(

)
[ ( ) ]
( )
.
*
*
*

+


2 2
2
2
2
2
2
2
2
2
2
2
2 1
2 2 0
Result 3: The mean square error of

( ) ( )
* *

3
2
2
2
2 2
2
2
1
F
H
G
I
K
J
d
d
R
d
d
is
.
Proof:
6
MSE
d
d
d
d
d
d
d
d
d
d
d
d
(

)
( )
( )
( )
( ) ( )
( )
*
*
*
* *
*

3
2 2
2
2
4 2
2 2
2
2 2
2 2
2
2
2
2
2
2
2
2 2
2
2
2
2 1
2 1
1
+
L
N
M
O
Q
P
+
L
N
M
O
Q
P

F
H
G
I
K
J
(from result 1)
=
Note that MSE n MSE m (

) (

) .
3 3
0 0 as and as
Result 4: Let

( )
.
* *

2
2
3
2
2
2

R
d
d
d
R and Then
MSE MSE
MSE
(

) (

)
(

)

2 3
2
100
L
N
M
O
Q
P

, the
percent reduction in mean square error using the minimum mean square error estimator
instead of
R
d
2
*
[as recommended by Luko (1996)], is
50
2 2
2
d d
d
*
*
F
H
G
I
K
J
Proof:
Luko (1996) shows that
MSE
d d
d
(

)
( )
*
*

2
2
2 2
2
2

, therefore
7
MSE MSE
d d
d
d
d
d d
d
d d
d
d d
d
d d d d
d
d d
d
d d
d
d d
d
d d
d
(

) (

)
( )
( )
( ) ( )
( )
( ) ( )( )
( )
( )
( )
*
* *
*
*
*
*
*
*
* *
*
*
*
*
*
*
*
*

2 3
2
2 2
2
2 2
2
2
2
2 2 2
2
2
2
2
2
2
2
2 2 2
2
2 2 2 2
2
2
2 2 2
2
2 2
2
2 2 2
2
2 2
2
1
2
2
2



F
H
G
I
K
J

L
N
M
O
Q
P

+ L
N
M
O
Q
P

+ F
H
G
I
K
J

2
2 2 2
2
2
2
*
*
*
( )
( )
F
H
G
I
K
J

d d
d
Consequently
MSE MSE
MSE
d d d
d d d
d d
d
(

) (

)
(

)
( ) / ( )
( ) / ( )
* *
* *
*
*

2 3
2
2
2 2
2
2
2
2
2 2 2
2 2
2
100
2
100 50
L
N
M
O
Q
P

F
H
G
I
K
J
.
S5-3. Determining When the Process has Shifted
Control charts monitor a process to determine whether an assignable cause has occurred.
Knowing when the assignable cause has occurred would be very helpful in its
identification and eventual removal. Unfortunately, the time of occurrence of the
assignable cause does not always coincide with the control chart signal. In fact, given what
is known about average run length performance of control charts, it is actually very
unlikely that the assignable cause occurs at the time of the signal. Therefore, when a signal
occurs, the control chart analyst should look earlier in the process history to determine the
assignable cause.
But where should we start? The Cusum control chart provides some guidance simply
search backwards on the Cusum status chart to find the point in time where the Cusum last
crossed zero (refer to Chapter 8). However, the Shewhart x control chart provides no such
simple guidance.
Samuel, Pignatiello, and Calvin (1998) use some theoretical results by Hinkley (1970) on
change-point problems to suggest a procedure to determine the time of a shift in the
process mean following a signal on the Shewhart x control chart. They assume the
standard x control chart with in-control value of the process mean
0

. Suppose that the

chart signal at subgroup average
T
x . Now the in-control subgroups are
1 2
, ,...,
t
x x x
, and
the out-of-control subgroups are
1 2
, ,...,
t t T
x x x
+ +
, where obviously t T . Their procedure
consists of finding the value of t in the range 0 t T < that maximizes
2
, 0
( )( )
t T t
C T t x
8
where
,
1
1
T
T t i
i t
x x
T t
+

is the reverse cumulative average; that is, the average of the T t
most recent subgroup averages. The value of t that maximizes C
t
is the estimator of the last
subgroup that was selected from the in-control process.
You may also find it interesting and useful to read the material on change point procedures
for process monitoring in Chapter 9.
S5-4. More about Monitoring Variability with Individual Observations
As noted in the textbook, when one is monitoring a process using individual (as opposed to
subgrouped) measurements, a moving range control chart does not provide much useful
additional information about shifts in process variance beyond that which is provided by
the individuals control chart (or a Cusum or EWMA of the individual observations).
Sullivan and Woodall (1996) describe a change-point procedure that is much more
effective that the individuals (or Cusum/EWMA) and moving range chart.
Assume that the process is normally distributed. Then divide the n observations into two
partitions of
1 2
and n n observations, with n
1
= 2, 3, , n 2 observations in the first
partition and n n
1
in the second. The log-likelihood of each partition is computed, using
the maximum likelihood estimators for
2
and in each partition. The two log-
likelihood functions are then added. Call the sum of the two log-likelihood functions L
a
.
Let L
0
denote the log-likelihood computed without any partitions. Then find the maximum
value of the likelihood ratio r = 2(L
a
L
0
). The value of n
1
at which this maximum value
occurs is the change point; that is, it is the estimate of the point in time at which a change
in either the process mean or the process variance (or both) has occurred.
Sullivan and Woodall show how to obtain a control chart for the likelihood ratio r. The
control limits must be obtained either by simulation or by approximation. When the
control chart signals, the quantity r can be decomposed into two components; one that is
zero if the means in each partition are equal, and another that is zero if the variances in
each partition are equal. The relative size of these two components suggests whether it is
the process mean or the process variance that has shifted.
S5-5. Detecting Drifts versus Shifts in the Process Mean
In studying the performance of control charts, most of our attention is directed towards
describing what will happen on the chart following a sustained shift in the process
parameter. This is done largely for convenience, and because such performance studies
must start somewhere, and a sustained shift is certainly a likely scenario. However, a
drifting process parameter is also a likely possibility.
Aerne, Champ, and Rigdon (1991) have studies several control charting schemes when the
process mean drifts according to a linear trend. Their study encompasses the Shewhart
control chart, the Shewhart chart with supplementary runs rules, the EWMA control chart,
and the Cusum. They design the charts so that the in-control ARL is 465. Some of the
previous studies of control charts with drifting means did not do this, and different charts
9
have different values of ARL
0
, thereby making it difficult to draw conclusions about chart
performance. See Aerne, Champ, and Rigdon (1991) for references and further details.
They report that, in general, Cusum and EWMA charts perform better in detecting trends
than does the Shewhart control chart. For small to moderate trends, both of these charts are
significantly better than the Shewhart chart with and without runs rules. There is not
much difference in performance between the Cusum and the EWMA.
S5-6. The Mean Square Successive Difference as an Estimator of
2
An alternative to the moving range estimator of the process standard deviation

is the
mean square successive difference as an estimator of
2
. The mean square successive
difference is defined as
2
1 1
1
1
( )
2( 1)
n
i
i
MSSD x x
n

It is easy to show that the MSSD is an unbiased estimator of
2
. Let
1 2
, ,...,
n
x x x
be a
random sample of size n from a population with mean
2
and variance . Without any
loss of generality, we may take the mean to be zero. Then
2
1
2
2 2
1 1
2
2 2
2
2
1
( ) ( )
2( 1)
1
( 2 )
2( 1)
1
[( 1) ( 1) ]
2( 1)
2( 1)
2( 1)
n
i i
i
n
i i i i
i
E MSSD E x x
n
E x x x x
n
n n
n
n
n

1

1

]
1
+
1

]
+

Therefore, the mean square successive difference is an unbiased estimator of the population
variance.
10