Using Statistics in Sampling

Plans Plans that Result in

Statistical Confidence
Jacalyn L. Schroeder
Statistician, Validation/Quality
Engineering Specialist
 Jackie Schroeder
Presenter/Facilitator
B.S. in Chemical Engineering
M.S. in Applied Statistics
22 Years at 3M
2 Years at Dow Chemical
Medical Devices, Drug Combo Products
Mfg Engineer, Product Service Engineer, Supervisor,
Validation Engineer, Quality Engineer
Validation Engineer 8 Years
Quality Engineering Specialist/Statistician
J.L. Schroeder , 3M, June 2015
 AGENDA
Acceptance Sampling Plan Basics and
Application
Types of Sampling Plans and Plan Selection
Traditional Sampling vs. Modern Acceptance
Sampling Techniques
Criteria for evaluating the effectiveness of
sampling plans/or schemes
Protection offered by a statistically based
sampling plan
Interactive Exercises (Throughout)

J.L. Schroeder, June 2015 4

 Regulations
and Sampling Plans
Per 21 CFR 820.250 Sampling plans, when
used, shall be written and based on a valid
statistical rationale.

Assumptions on which sampling plans are based

shall be verified

We should understand how the sample is taken

(random, stratified, periodic) to represent lot
Sample needs to be unbiased and representative of the
population sampled
J.L. Schroeder, June 2015 5
 Acceptance Sampling
Popularized by during WWII to test
bullets
A SQC technique, where decision to
either accept or reject lot is based on
sample of lot
Main purpose is to assure the
production of lots of acceptable quality

J.L. Schroeder, June 2015 6

 What is a Sampling Plan?
Purpose:
Judge whether quality level of an
incoming shipment or production lot is
likely to be acceptable
Make pass/fail or accept/reject decisions
Important Points:
Sampling plans do not estimate the
quality of the lot
Acceptance sampling is "the middle of the
road" approach between no inspection
and 100% inspection
Sampling plans include both a sample size
and acceptance criteria
J.L. Schroeder, June 2015 7
 Common Application
Acceptance Sampling is commonly employed
when one or several of the following hold:

Testing is destructive

The cost of 100% inspection is very high

100% inspection takes too long

Testing is the only way to detect a bad part

J.L. Schroeder, June 2015 8
 Typical Use (Receiving)
You received a shipment of 5,000
parts from a new supplier.
How will you decide if the quality is
good enough to put in inventory?

J.L. Schroeder, June 2015 9

 How Will You Decide?
1. Inspect all 5,000 parts (100%
inspection)
2. Put entire shipment into inventory
without inspection (0% inspection)
3. Sample via a sampling plan
Sample a portion of the shipment and if
enough parts pass the acceptance criteria
release into inventory

J.L. Schroeder, June 2015 10

Shipper and Receiver Agreement
For smooth transfer of product, it is important that
both the shipper and receiver understand and agree
on:
1. What constitutes conforming and non
conforming units
2. Specification to which the discrete units of
product must conform
3. Basis on which product will be accepted
Shipper: Understand probability that the lot will be accepted
at various quality levels
Receiver: Understand protection that exists to prevent
acceptance of nonconforming product
J.L. Schroeder, June 2015 13
 Basic Approach
Take a representative
sample of n units from lot
of unknown quality (p)

Classify each item as

conforming or
nonconforming

Count the number

that are nonconforming

Compare
count to
accept Count > a
Count a number
a

ACCEPT REJECT 14

Record Result Quality Record

 Basic Classification of Sampling Plans
Attributes Variables
We classify items using We measure items
attributes using variables
A bicycle can be red, The seal strength is 75
blue, or green lbs/in
A part can be The part diameter is 2.5
conforming or inches
nonconforming The fill weight is 9.8
The part is visually grams
acceptable so it passes The line speed is 100
(P) the test; otherwise it ft/min
would fail (F) the test 15
 Considerations for Acceptance
Sampling Plan Selection
Sampling by attributes vs. sampling by variables
Incoming vs. outgoing inspection
Customer Needs
Process considerations
Information on available quality
Integrity or past experience with vendor
Production Type: Continuous, Lot-by-Lot, isolated lots
Lot size (small lots will use different distribution for estimates)
Inspection costs
Inspection procedures
Method of correction of nonconformities
None, scrap, rework, 100% rectifying inspection

J.L. Schroeder, June 2015 16

 Types of Sampling
Definition by ISO 3534-3 (2006)
Acceptance Sampling Plan:
Plan which states the sample sizes to be used and
the associated criteria for lot acceptance; sample
without replacement
Acceptance Sampling Schemes:
A combination of acceptance sampling plans with
switching rules for changing from one plan to
another
Not intended to be used as individual sampling
plans
J.L. Schroeder, June 2015 17
 Acceptance Sampling
Devised to serve different purposes:

ISO 2859 ISO 3951

Neubauer Acceptance J.L. Schroeder, June 2015 18

Sampling Facts
 AQL System (1942 1943)
Sampling Scheme
AQL System (Acceptable Quality Level); Producers Quality Level
Specify an upper limit of Quality, AQL, not to be exceeded by the
producer without penalty of excessive rejected lots.
MIL-STD-105 came about from a need for a sampling system which
did not require 100% inspection for use in testing munitions and for
other destructive tests
Intended for stream of lots
Uses switching rules

J.L. Schroeder, June 2015 20

 AQL (Stream of Lots)
AQL Based Sampling Scheme
Definition of AQL:
When a continuous series of lots is considered, the
AQL is the quality level which, for the purposes of the
sampling inspection, is the limit of a satisfactory
process average.
Schemes not intended to use as individual plans.
Quality levels are specified in terms of AQL for the
producer, while consumer protection is afforded by
the switching rules which lead to tighter plans when
quality is poor.
Designed to: Exert pressure on producer to take
corrective action if quality falls below prescribed
levels and to provide rewards in terms of reduced
sample size for quality improvement.
J.L. Schroeder, June 2015 21
Modern Acceptance Sampling
(1990s Present)

(AQL, 1-)

RQL, )

J.L. Schroeder, June 2015 23

 Modern Acceptance Sampling
1. Evaluate Customer Needs and Process
Considerations
2. Sampling by attributes vs. sampling by
variables
3. Determine appropriate statistical
performance property (i.e. percent
defective, defects per quantity, etc)
4. Choose sampling risks (,), acceptable
quality level (AQL) and reject performance
level (RQL)
5. Develop OC Curve
J.L. Schroeder, June 2015 24
 Sampling Risks
Producers Risk
Alpha Risk ()
Risk of rejecting good product
Prob (rejecting lot of AQL quality level)
Consumers Risk
Beta Risk ()
Risk of accepting bad quality product
Prob (accepting lot at RQL (LTPD) quality level)

OC curve for sampling plan quantifies these risks

J.L. Schroeder, June 2015 25
Operating Characteristic (OC) Curve
Each sampling plan has its own OC curve
Graph shows probability of lot acceptance (Pa)
as a function of the lot quality level (p)
The OC curve can help us determine the plan
performance
OC curves can be used to compare sampling
plans (single, double, multiple)

J.L. Schroeder, June 2015 26

 Every Sampling Plan Has Its Own OC
Curve

1
Probability Pass

Single: n=30, a=0

Single: n=300, a=0
0.5 Single: n=3000, a=0

0 Determine the chances

0 5 10 of passing at different
Percent Defective defective rates.

Resource: Dr. Wayne Taylor

Validation Sampling Plan Class 27
- 2008
 OC Curve Calculated Using
Following Distributions (Attribute)
Binomial - Used by most statistical
software programs
Hypergeometric - Used when sampling
from a small population. Used when lot
size is not significantly greater than
sample size
Poisson - Typically used when evaluating
defects per unit rather than fraction
defective J.L. Schroeder, June 2015 28
 Common OC Curve Calculation
Binomial Distribution Formula (approximate
to true distribution the Hypergeometric)
Accurate when:
/ 10

(n,c)
!
= = 1
! !

J.L. Schroeder, June 2015 29

OC Curve by Binomial Formula

J.L. Schroeder, June 2015 30

Modern Acceptance Sampling
Specify a Plan
Four values may be used to characterize the
OC curve (AQL, 1-) (LTPD, )

Sampling Risks
= Producers Risk
= Consumer Risk

Quality Levels
AQL = Acceptable Quality Level
RQL or LTPD = Reject Quality Level or Lot Tolerance
Percent Defective
J.L. Schroeder, June 2015 31
 What is AQL?

What is routinely accepted

The AQL of a sampling plan is AQL

is a
level of quality (percent defective, defects per
hundred units, etc.) routinely accepted by the
sampling plan
Quality level that the CONSUMER feels to be the
MAXIMUM percent defective, that for sampling
purposes, can be considered the process average.
Products or processes at or better than the AQL are
passed at least 95% (1-)% of the time
J.L. Schroeder, June 2015 32
 What is LTPD (AKA RQL)?
What is routinely rejected

.05 or .10
LTPD

The LTPD of a sampling plan is a level of quality (percent

defective, defects per hundred units, etc.) routinely rejected
by the sampling plan
LARGEST percent defective (average) which will be rejected
by the producer in response to the consumers expectations.
Products or processes at or worse than the LTPD fail most of
the time
Protection provided by the plan
J.L. Schroeder, June 2015 34
 Ideal OC Curve
Sampling plan would accept
every lot (100% of the time) with
fraction defective less than or
equal to AQL
Sampling plan would reject
every lot (100% of the time) with
fraction defective greater than
AQL
Sampling does not realize the
ideal OC Curve

AQL Consumer Expectation

The Quality Engineer sets the AQL
J.L. Schroeder, June 2015 35
 Summary
Purpose of a sampling plan is to make
accept/reject decisions
The AQL describes what it accepts and
determines the chances of passing
The LTPD (RQL) describes what it rejects
and provides consumer protection;
determines the confidence statements that
can be made when passing

Resource: Dr. Wayne Taylor

Validation Sampling Plan Class -
2007 38
ATTRIBUTE SAMPLING PLANS
Single Sampling Plan Example

39
 Basic Approach
Take a representative
sample of n units from lot
of unknown quality (p)

Classify each item as

conforming or
nonconforming

Count the number

that are nonconforming

Compare
count to
accept Count > a
Count a number
a

ACCEPT REJECT 40
 Attribute Sampling Plans
Choose between single or multiple
sampling plans at same protection level
or performance level (same LTPD)

Multiple sampling plans were invented

to give questionable populations
another chance in order to better
understand the true proportion
defective
J.L. Schroeder, June 2015 41
 Double Sampling Plan
(For Percent Defective or Proportion Nonconforming)

START

Take a representative
sample of n1 units and
count number of defectives

Compare
Count a1 count Count r1
Take second representative
sample of n2 units and
ACCEPT REJECT count the total number of
defectives in both samples

Continue To Second Stage

Compare
Count a2 count Count > a2

Resource: Dr. Wayne Taylor

ACCEPT REJECT
Validation Sampling Plan Class -
2008 42
 Interactive Exercise
Which LTPD.05 Plan is Best?
Single vs. Double
1

Pa

0.5

0
0 1 2
Percent Defective

Defective/Nonconforming Units
Single: n=300, a=0
Single: n=630, a=2
Resource: Dr. Wayne Taylor
Double: n1=325, a1=0, r1=3, n2=580, a2=3
Validation Sampling Plan Class -
J.L. Schroeder, June 2015 43
2008
 Exercise Attribute Single
Sampling Plan
Supplier submits a lot (N) of 5000 items
and you subject it to acceptance In symbols:
sampling by attributes
N=5000
Test defines what is
conforming/nonconforming n = 300
Sampling plan requires random sample
(n) of 300 units from lot a = 0, r = 1
Sampling plan defines acceptance
Note: This
number (a) and reject number (r) plan had
Accept shipment if 0 items nonconforming AQL=0.02%
(a = 0) LTPD0.05=1%
Reject shipment if 1 or more items are
nonconforming (r=1)
J.L. Schroeder, June 2015 45
EXERCISES USING VALIDATED
SAMPLING PLAN
DETERMINATION SOFTWARE

46
 Modern Acceptance Sampling
(1990s Present)
H& H Servicco Corp, Stan Hilliard,
www.samplingplans.com
2-Point Sampling Plans (Defective Units)
TP105 for Attributes
TP414 for Variables
Wayne Taylor Inc., Dr. Wayne Taylor,
www.variation.com

J.L. Schroeder, June 2015 47

 Exercise Attribute Sampling Plan
Select AQL, RQL, alpha, beta
Enter values into validated
software program
Assume large lot so estimate by
binomial distribution

J.L. Schroeder, June 2015 48

TP105 Sample Planner
for Attributes

In Fraction
Defective

J.L. Schroeder, June 2015

49
 Sampling Plan Analyzer
For Attributes
Select New Sampling Plan

J.L. Schroeder, June 2015 50

 Sampling Plan Analyzer
Match Single/Double Attribute Plan

J.L. Schroeder, June 2015 51

VARIABLES SAMPLING PLANS

Single Sampling Plan Example

52
 Variables Sampling Plan
Use normal distribution to predict the
percentage of units outside of spec

LSL USL

2.3%
Out of Spec

50 51 52 53 54 55

Fill Volume (ml)

Resource: Dr. Wayne Taylor
Validation Sampling Plan Class - 53
2008
 Variables Basic Approach

START

Take a representative sample of n units,

measure each unit and calculate
average (X) and standard deviation (S)

Calculate Ppk

Is Ppk Ppk

Yes No
ACCEPT REJECT
Resource: Dr. Wayne Taylor
Validation Sampling Plan Class - 54
2008
 Exercise Variables Sampling
Plan Sigma Known
Length is parameter that material must pass.
The product standard lists the individual In symbols:
specification limits for length as 45 90 mm. UISL = 90 mm
LISL = 45 mm
A sigma () of 4 mm has been estimated based off 3
months of data. This gives an estimated process #= 4 mm
capability (Cp) of 1.875. Cp = 1.875

Select:
Select AQL, RQL, , AQL, RQL, ,
Calculate Accept Constant (k) and Sample Size (n) or Calculate:
see Tables of Variables Plans (Sommers) or Jacobson Sample size (n)
Nomograph. and Average
Use k and n to calculate average acceptance values; Accept Limits
(%& , %( )
where, = + ; ! = "
J.L. Schroeder, June 2015 56
TP414 Sample Planner
for Variables

n
! J.L. Schroeder, June 2015 57
 Variables Single for Defective Units

Mean In symbols:
UISL = 90 mm
AL=55.2 Au=79.8 LISL = 45 mm
LISL=45
UISL=90
Sigma hat = 4
mm
k Cp = 1.875
k
AQL = 0.01%,
RQL=5%, =.05,
=.1

k =2.5532
n=2

)* = ++. -
). = /0. 1

58
 Sampling Plan Analyzer
Match Variables to Attribute Plan

J.L. Schroeder, June 2015 59

 Protection
Offered
By A
Sampling Plan
 Confidence Statements
Sampling plans are designed to
demonstrate that the product tested
meets a specified performance level
(LTPD)with certain confidence (typically
1 or 95%)

Passing the plan n=300, a=0 allows one to

state: With 95% confidence the defect
level is below 1% defective.

Resource: Dr. Wayne Taylor

Validation Sampling Plan Class -
2008 LTPD 61
 Determine Plan Protection
Make Confidence Statement
Make Confidence Statement if pass sampling
plan
If lot passes sampling plan, one can be 95% confident
that lot percent defective is below the LTPD
Passing the sampling plan n=300, a=0 allows one
to state:
With 95% confidence the percent nonconforming
units are less than 1% defective
With 95% confidence the percent conforming units
are greater than 99% conforming
Resource: Taylor Enterprises
Statistical Procedures For
Medical Device Industry 2013 J.L. Schroeder, June 2015 62
 Statistical Confidence
CGMP regulations regarding
sampling set forth a number of
requirements for validation:
Sampling plan must result in statistical
confidence ( 211.165(c) and (d))

Make Confidence Statement if pass

sampling plan
J.L. Schroeder, June 2015 63
Questions?

J.L. Schroeder, June 2015 64

 Contact Information
Jacalyn L. Schroeder 3M Company
Title: Statistician, Process Validation/Quality Specialist
E-mail: jlschroeder2@mmm.com
Work Phone: 605-696-1355

Dr. Wayne Taylor, Taylor Enterprises

Website: www.variation.com

Stan Hilliard, H & H Servicco Corp

Website: www.samplingplans.com

J.L. Schroeder, June 2015 65

 References
1. Dr. Wayne A. Taylor, Guide to Acceptance
Sampling, version 1, www.variation.com, Taylor
Enterprises,Inc. Note: Version 2 of new book
TBD.
2. Acceptance Sampling in Quality Control, 2nd
edition, Schilling & Neubauer
3. Acceptance Sampling and Control Class,
Neubauer, 2012
4. Standards: ANSI Z1.4, ANSI Z1.9, ISO 2859-1,
ISO 3951-1
5. Schilling/Johnson, Tables for the Construction
of Matched Single, Double, and Multiple
Sampling Plans with application to MIL-STD-
105D, Journal Quality Technology, October,
1980 66