Global Procurement -

Supplier Quality
Introduction to MSA
Lise Robert SQS (Supplier Quality Specialist)
Rev. 01 – Dec 17th 2018
Introduction to Measurement System Analysis (MSA)

 Everyday our lives are being impacted by more and more data. We have become a
data driven society.
 In business and industry, we are using data in more ways than ever before.  
 Today manufacturing companies gather massive amounts of information through
measurement and inspection. When this measurement data is being used to make
decisions regarding the process and the business in general it is vital that the data is
accurate. If there are errors in our measurement system we will be making decisions
based on incorrect data. We could be making incorrect decisions or producing non-
conforming parts. A properly planned and executed Measurement System Analysis
(MSA) can help build a strong foundation for any data based decision making
process.
What is Measurement System Analysis (MSA)

 MSA is defined as an experimental and mathematical method of

determining the amount of variation that exists within a measurement
process. Variation in the measurement process can directly contribute
to our overall process variability. MSA is used to certify the
measurement system for use by evaluating the system’s accuracy,
precision and stability.
What is a Measurement System?

 Before we dive further into MSA, we should review the definition of a

measurement system and some of the common sources of variation.
A measurement system has been described as a system of related
measures that enables the quantification of particular characteristics.
It can also include a collection of gages, fixtures, software and
personnel required to validate a particular unit of measure or make an
assessment of the feature or characteristic being measured. 
What is a Measurement System?

Variation
Think of Measurement as a
Process
What is a Measurement System?

Measurement
The assignment of numbers to material things to represent the relationships
among them with respect to particular properties.

C. Eisenhart (1963)
What is a Measurement System?

 The sources of variation in a measurement process can include the following:

 Process – test method, specification

 Personnel – the operators, their skill level, training, etc.
 Tools / Equipment – gages, fixtures, test equipment used and their associated
calibration systems
 Items to be measured – the part or material samples measured, the sampling
plan, etc.
 Environmental factors – temperature, humidity, etc.
What is a Measurement System?

 All of these possible sources of variation should be considered during

Measurement System Analysis. Evaluation of a measurement system
should include the use of specific quality tools to identify the most
likely source of variation. Most MSA activities examine two primary
sources of variation, the parts and the measurement of those parts.
The sum of these two values represents the total variation in a
measurement system.
Why Perform Measurement System Analysis (MSA)

 An effective MSA process can help assure that the data being
collected is accurate and the system of collecting the data is
appropriate to the process.
 Good reliable data can prevent wasted time, labor and scrap in a
manufacturing process.
Why Perform Measurement System Analysis (MSA)
Example

10

 A major manufacturing company began receiving calls from several of their customers
reporting non-compliant materials received at their facilities sites. The parts were not
properly snapping together to form an even surface or would not lock in place.
 The process was audited and found that the parts were being produced out of spec.
The operator was following the inspection plan and using the assigned gages for the
inspection. The problem was that the gage did not have adequate resolution to detect
the non-conforming parts. 
 An ineffective measurement system can allow bad parts to be accepted and good
parts to be rejected, resulting in dissatisfied customers and excessive scrap. MSA
could have prevented the problem and assured that accurate useful data was being
collected..
How to Perform Measurement System Analysis (MSA)

11

 MSA is a collection of experiments and analysis performed to evaluate a

measurement system’s capability, performance and amount of uncertainty
regarding the values measured. We should review the measurement data
being collected, the methods and tools used to collect and record the data.

 Our goal is to quantify the effectiveness of the measurement system,

analyze the variation in the data and determine its likely source. We need
to evaluate the quality of the data being collected in regards to location and
width variation. Data collected should be evaluated for bias, stability and
linearity.
How to Perform Measurement System Analysis (MSA)

12

 During an MSA activity, the amount of measurement uncertainty must

be evaluated for each type of gage or measurement tool defined
within the process Control Plans.
 Each tool should have the correct level of discrimination and
resolution to obtain useful data. The process, the tools being used
(gages, fixtures, instruments, etc.) and the operators are evaluated
for proper definition, accuracy, precision, repeatability and
reproducibility.
How to Perform Measurement System Analysis (MSA)

13

Data Classifications
 Prior to analyzing the data and or the gages, tools or fixtures, we
must determine the type of data being collected. The data could be
attribute data or variable data.
 Attribute data is classified into specific values where variable or
continuous data can have an infinite number of values.
How to Perform Measurement System Analysis (MSA)

14

The Master Sample

 To perform a study, you should first obtain a sample and establish the
reference value compared to a traceable standard. Some processes
will already have “master samples” established for the high and low
end of the expected measurement specification.
How to Perform Measurement System Analysis (MSA)

15

The Gage R&R Study

 For gages or instruments used to collect variable continuous data, 
Gage Repeatability and Reproducibility (Gage R & R) can be
performed to evaluate the level of uncertainty within a measurement
system. 
How to Perform Measurement System Analysis (MSA)

16

 To perform a Gage R & R, first select the gage to be evaluated.

 Then perform the following steps:
 Obtain at least 10 random samples of parts manufactured during a regular production run
 Choose three operators that regularly perform the particular inspection
 Have each of the operators measure the sample parts and record the data
 Repeat the measurement process three times with each operator using the same parts
 Calculate the average (mean) readings and the range of the trial averages for each of the operators
 Calculate the difference of each operator’s averages, average range and the range of
measurements for each sample part used in the study
 Calculate repeatability to determine the amount of equipment variation
 Calculate reproducibility to determine the amount of variation introduced by the operators
 Calculate the variation in the parts and total variation percentages
How to Perform Measurement System Analysis (MSA)

17

 The resulting Gage R & R percentage is used as a basis for accepting the
gage. Guidelines for making the determination are found below:
 The measurement system is acceptable if the Gage R & R score falls below 10%
 The measurement system may be determined acceptable depending upon the
relative importance of the application or other factors if the Gage R & R falls
between 10% to 20%
 Any measurement system with Gage R & R greater than 30% requires action to
improve
 Any actions identified to improve the measurement system should be
evaluated for effectiveness
How to Perform Measurement System Analysis (MSA)

18

 When interpreting the results of a Gage R & R, perform a comparison

study of the repeatability and reproducibility values.
 If the repeatability value is large in comparison to the reproducibility
value, it would indicate a possible issue with the gage used for the study.
 The gage may need to be replaced or re-calibrated.
 Adversely, if the reproducibility value is large in comparison with the
repeatability value, it would indicate the variation is operator related.
 The operator may need additional training on the proper use of the gage
or a fixture may be required to assist the operator in using the gage.
How to Perform Measurement System Analysis (MSA)

19

 Gage R & R studies shall be conducted under any of the following

circumstances:
 Whenever a new or different measurement system is introduced
 Following any improvement activities
 When a different type of measurement system is introduced
 Following any improvement activities performed on the current
measurement system due to the results of a previous Gage R & R study
 Annually in alignment with set calibration schedule of the gage
How to Perform Measurement System Analysis (MSA)

20

 Attribute Gage R & R

 Attribute measurement systems can be analyzed using a similar method. Measurement
uncertainty of attribute gages shall be calculated using shorter method as below:
 Determine the gage to be studied
 Obtain 10 random samples from a regular production run
 Select 2 different operators who perform the particular inspection activity regularly
 Have the operators perform the inspection two times for each of the sample parts and
record the data
 Next, calculate the kappa value.
 When the kappa value is greater than 0.6, the gage is deemed acceptable
 If not, the gage may need to be replaced or calibrated
How to Perform Measurement System Analysis (MSA)

21

 Attribute Gage R & R

 The attribute gage study should be performed based on the same criteria listed
previously for the Gage R & R study.
 During MSA, the Gage R&R or the attribute gage study should be completed
on each of the gages, instruments or fixtures used in the measurement
system. The results should be documented and stored in a database for future
reference. It may be required for a PPAP submission to the customer.
 Furthermore, if any issues should arise, a new study can be performed on the
gage and the results compared to the previous data to determine if a change
has occurred. A properly performed MSA can have a dramatic influence on the
quality of data being collected and product quality.
Key terms and definitions
22
 Attribute data – Data that can be counted for recording and analysis (sometimes referred to as go/ no go data)

 Variable data – Data that can be measured; data that has a value that can vary from one sample to the next; continuous
variable data can have an infinite number of values
 Bias – Difference between the average or mean observed value and the target value

 Stability – A change in the measurement bias over a period of time

 A stable process would be considered in “statistical control”

 Linearity – A change in bias value within the range of normal process operation

 Resolution – Smallest unit of measure of a selected tool gage or instrument; the sensitivity of the measurement system to
process variation for a particular characteristic being measured
Key terms and definitions
23

 Accuracy – The closeness of the data to the target or exact value or to an accepted
reference value

 Precision – How close a set of measurements are to each other

 Repeatability – A measure of the effectiveness of the tool being used; the variation of
measurements obtained by a single operator using the same tool to measure the same
characteristic

 Reproducibility – A measure of the operator variation; the variation in a set of data collected
by different operators using the same tool to measure the same part characteristic
Key terms and definitions
24

 Accuracy – The closeness of the data to the target or exact value or to an accepted
reference value

 Precision – How close a set of measurements are to each other

 Repeatability – A measure of the effectiveness of the tool being used; the variation of
measurements obtained by a single operator using the same tool to measure the same
characteristic

 Reproducibility – A measure of the operator variation; the variation in a set of data collected
by different operators using the same tool to measure the same part characteristic
 25

Measurement Systems
Analysis
Measurement Systems Analysis
26

 Basic Concepts of Measurement Systems

 A Process
 Statistics and the Analysis of Measurement Systems
 Conducting a Measurement Systems Analysis
 ISO - TC 69 is the Statistics Group
 Ensures high ‘Data Quality’ (Think of Bias)
Course Focus & Flow
27

 Measurement as a Process
 Mechanical Aspects (vs Destructive)
 Piece part
 Continuous (fabric)
 Features of a Measurement System
 Methods of Analysis
 Gauge R&R Studies
 Special Gauging Situations
 Go/No-Go
 Destructive Tests
Place Timeline Here
28
The Target & Goal
Continuous Improvement 29

Production

Pre-Launch

Prototype

LSL USL
Key Words
30

 Discrimination
Ability to tell things apart
 Bias [per AIAG] (Accuracy)
 Repeatability [per AIAG] (Precision)
 Reproducibility
 Linearity
 Stability
Terminology
31

 Error ≠ Mistake
 Error ≠ Uncertainty
 Percentage Error ≠ Percentage Uncertainty
 Accuracy ≠ Precision
Measurement Uncertainty
32

 Different conventions are used to report measurement uncertainty.

 What does ±5 mean in m = 75 ±5?
 Estimated Standard Deviation: 
 Estimated Standard Error: m = /√N
 Expanded Uncertainty of ± 2 or 3
Sometimes ± 1 (Why?)
 95% or 99% Confidence Interval
 Standard Uncertainty: u
 Combined Standard Uncertainty: u c
Measurement Uncertainty
33

 Typical Reports
 Physici
Measurement as a Process
34

Basic Concepts
 Components of the Measurement System
 Requirements of a Measurement System
 Factors Affecting a Measurement System
 Characteristics of a Measurement System
Features (Qualities) of a Measurement Number
 Units (Scale)
 Accuracy
 Precision (Consistency or Repeatability)
 Resolution (Reproducibility)
Measurement Related Systems
35

Typical Experiences with

Measurement Systems
Basic Concepts
36

 Every Process Produces a “Product”

 Every Product Possesses Qualities (Features)
 Every Quality Feature Can Be Measured
 Total Variation
= Product Variation + Measurement Variation
 Some Variation Inherent in System Design
 Some Variation is Due to a Faulty Performance of the System(s)
The Measurement Process
37

What is the ‘Product’ of the Measurement Process?

What are the Features or Qualities of this Product?
How Can We Measure Those Features?
Measurement Systems Components
38

 Material to be Inspected
Piece
Continuous
 Characteristic to be Measured
 Collecting and Preparing Specimens
 Type and Scale of Measurement
 Instrument or Test Set
 Inspector or Technician
AIAG calls these ‘Appraiser’
 Conditions of Use
Where Does It Start?
39

During the Design (APQP) Stage:

The engineer responsible for determining inspections and tests, and for
specifying appropriate equipment should be well versed in measurement
systems. The Calibration folks should be part of the process as a part of a
cross-functional team.
Variability chosen instrument must be small when compared with:
Process Variability
Specification Limits
Typical Progression
Determine ‘Critical’ 40
Product Engineer
Characteristic

Determine Required
Product Engineer
Resolution
How will the data be
used? Consideration of the Entire
Measurement System for Cross-Functional
the Characteristic
(Variables)

Determine What
Equipment is Already Metrology
Available
Measurement Systems Variables
Fixture 41
Material Inspector Methods Eyesight
Test Method
Air Pressure
Sample Training
Collection
Air Movement
Workmanship
Sample Parallax
Practice Fatigue
Preparation
Samples
Reproducibility
Ergonomic
s Standards

Measurement

Discrimination Vibration
Bias Temperature
Repeatability Lighting
Calibration
Linearity Humidity

Instrument Environment

These are some of the variables in a measurement

system. What others can you think of?
Determining What To Measure
42

External
Requirements  Voice of the Customer
You Must Convert to Technical Features
Convert To
 Technical Features
 Failure Modes Analysis
Internal  Control Plan
Requirements
Voice of the Customer
43

 External and Internal

Customers
 Stated vs Real and
Perceived Needs
 Cultural Needs
 Unintended Uses
 Functional Needs vs.
Technical Features
Customer may specify causes
rather than output
Convert to Technical Features
44

 Agreed upon Measure(s)

 Related to Functional Needs
 Understandable
 Uniform Interpretation Y
Functional Need
 Broad Application
 Economical
 Compatible Z
Technical Feature
 Basis for Decisions
 Failure Modes Analysis
45

 Design FMEA
 Process FMEA
 Identify Key Features
 Identify Control Needs

Critical Features are Defined Here!

 Automotive FMEA
Process Failure Mode And Effects Analysis Low - High

Process: Outside Suppliers Affected: Engineer: 1 - 10

Primary Process Responsibility: Model Year/Vehicle(s): Part Number:
Other Div. Or People Involved: Scheduled Production Released: PFMEA Date: Rev.

Approvals: Quality Assurance Manager Quality Assurance Engineer

Operations Manager Senior Advisor

Detection

Detection
Part Name Recommended

Occured

Occured
Severity

Severity
Operation Potential Failure Potential Effects Of Actions And Actions Responsible

RPN

RPN
Number Process Function Mode Failure Potential Cause Of Failure Current Controls Status Taken Activity

SIR Take TPPE Wrong Material Fragmented Container Insufficient Supplier Control Material Certification 1 9 2 18

Container Material Held In Unpredictable Deployment Improper Handling Required With Each
1 Storage Area Misidentified Material Shipment
Release Verification
Out Of Spec Fragmented Container Supplier Process Control Periodic Audit Of 3 10 3 90
Material Unpredictable Deployment Supplier Material
Contaminated Fragmented Container Open Boxes Visual Inspection 1 9 7 63
Material Unpredictable Deployment
Material Fragmented Container Engineering Change Release Verification 1 10 7 70
Composition Unpredictable Deployment Supplier Change Green "OK" Tag
Change Customer Notification
2 Move To Unreleased Fragmentation Untrained LTO Check For Green "OK" 5 10 1 50

Approved Untrained Personnel Tag At Press

Storage Trace Card
Check List
Training

Leading to MSA. Critical features are determined by the FMEA (RPN

indicators) and put into the Control Plan.
Control Plan / Flow Diagram
47

 Inspection Points
 Inspection Frequency
 Instrument
 Measurement Scale
 Sample Preparation
 Inspection/Test Method
 Inspector (who?)
 Method of Analysis
 GM Process Flow Chart
Process Flow Diagram Approved By:

Part Number: Date: 4/5/93 QA Manager

Part Description: Rev. : C Operations Manager
Prepared By: Senior Advisor
QA Engineer
Fabrication

Inspect
Move

Store

Step Operation Description Item # Key Product Characteristic Item # Key Control Characteristic

1 Move "OK" Vinyl Material 1.0 Material Specs 1.0 Material Certification Tag
From Storage Area and
Load Into Press.

2 Auto Injection Mold Cover 2.0 Tearstrip In Cover 2.1 Tool Setup
In Tool # 2.2 Machine Setup

3.0 Hole Diameter In Cover 2.1 Tool Setup

2.2 Machine Setup

4.0 Flange Thickness In Cover 2.1 Tool Setup

2.2 Machine Setup

5.0 Pressure Control Protrusions 2.1 Tool Setup

Height 2.2 Machine Setup

3 Visually Inspect Cover 6.0 Pressure Control Protrusions 2.1 Tool Setup
Filled Out 2.2 Machine Setup
 Standard Control Plan Example
49
Control Plan Number Key Contact / Phone Date (Orig.) Date (Rev.)

Part No./ Latest Change No. Core Team Customer Engineering Approval/Date

Part Name/Description Supplier/Plant Apoproval/Date Customer Quality Approval/Date

Supplier/Plant Supplier Code Other Approval/date (If Req'd) Other Approval/date (If Req'd)

Characteristics Methods
Machine, Product/
Part/ Process Name/ Device, Special Process Evaluation
Process Operation Jig, Tools Char. Spec/ Measurement Frequ- Control Reaction
Number Description for Mfg. No. Product Process Class Tolerance Technique Size ency Method Plan

This form is on course disk

Ford’s Dimensional Control Plan (DCP)
50
Measurement as a System
51

 Choosing the Right Instrument

 Instrument Calibration Needs
 Standards or Masters Needed
 Accuracy and Precision
 Measurement Practices
 Where
 How Many Places
 Reported Figures
 Significant Figures Rule
 2 Action Figures
 Rule of 10
 Individuals, Averages, High-Lows
Measurement Error
52

Measured Value (y)

=
True Value (x) + Measurement Error

Deming says there is no

such thing as a ‘True’ Value. Consistent (linear)?
Sources of Measurement Error
53

 Sensitivity (Threshold)
Chemical Indicators
 Discrimination
 Precision (Repeatability)
 Accuracy (Bias)
 Damage
 Differences in use by Inspector (Reproducibility)
Training Issues
 Differences Among Instruments and Fixtures
 Differences Among Methods of Use
 Differences Due to Environment
Types of Measurement Scales
54

 Variables
 Can be measured on a continuous scale
 Defined, standard Units of Measurement
 Attributes
 No scale
 Derived ‘Unit of Measurement’
 Can be observed or counted
 Either present or not
 Needs large sample size because of low information content
How We Get Data
55

 Inspection Includes Sensory (e.g..: look,

touch, smell…etc)

 Measurement Magnitude of Quality

 Test
Operational Definitions
56

 Is the container Round?

 Is your software Accurate?

 Is the computer screen Clean?

 Is the truck On Time?

Different Method = Different Results
57

Method 1
Method 2
Out of In Spec
Spec
Measurement System Variability
58

 Small with respect to Process Variation

 Small with respect to Specified Requirements
 Must be in Statistical Control
Measurement IS a Process!
Free of Assignable Causes of variation
Studying the Measurement System
59

 Environmental Factors
 Human Factors
 System Features
 Measurement Studies
Environmental Factors
60

 Temperature
 Humidity Where is the study performed?
1. Lab?
 Vibration 2. Where used?
 Lighting 3. Both?

 Corrosion
 Wear
 Contaminants
Oil & Grease
Aerosols
Human Factors
61

 Training
 Skills
 Fatigue
 Boredom
 Eyesight
 Comfort
 Complexity of Part
 Speed of Inspection (parts per hour)
 Misunderstood Instructions
Human Measurement Errors Unaware of
problem

 Sources of Errors 62

Inadvertent Errors Good Bad

 Attentiveness
 Random OK!
 Good Mistake-Proofing Target
Accept beta
Training Technique Errors
Issue Reject
 Consistent alpha OK!
Wilful Errors (Bad mood)
 Error Types (Can be machine or human)
Type I - Alpha Errors [ risk] Process in
control, but
Type II - Beta Errors [ risk] needs
adjustment,
False alarm
Measurement System Features
63

 Discrimination
Ability to tell things apart
 Bias [per AIAG] (Accuracy)
 Repeatability [per AIAG] (Precision)
 Reproducibility
 Linearity
 Stability
Discrimination
64

 Readable Increments of Scale

 If Unit of Measure is too course: Process variation will be lost in
Rounding Off
 The “Rule of Ten”: Ten possible values between limits is ideal
Five Possible Values: Marginally useful
Four or Less: Inadequate Discrimination
Discrimination
65
Range Charts & Discrimination
66

Indicates Poor
Precision
Bias and Repeatability
67
Precise Imprecise

Accurate

Bias
Inaccurate

You can correct for Bias

You can NOT correct for Imprecision
Bias
68

 Difference between average of

measurements and an Agreed
Bias
Standard Upon standard value
Value
 Known as Accuracy
 Cannot be evaluated without a
Standard
Measurement Scale  Adds a Consistent “Bias Factor” to
ALL measurements
 Affects all measurements in the
same way
Causes of Bias
69

 Error in Master
 Worn components
 Instrument improperly calibrated
 Instrument damaged
 Instrument improperly used
 Instrument read incorrectly
 Part set incorrectly (wrong datum)
Bias
70

 Bias - The difference between the observed Average of

measurements and the master Average of the same parts using
precision instruments. (MSA Manual Glossary)

 The auditor may want evidence that the concept of bias is understood.
Remember that bias is basically an offset from ‘zero’. Bias is linked to
Stability in the sense that an instrument may be ‘zeroed’ during
calibration verification. Knowing this we deduce that the bias changes
with instrument use. This is in part the concept of Drift.
Bias
71

 I choose a caliper (resolution 0.01) for the measurement. I measure a

set of parts and derive the average.
 I take the same parts and measure them with a micrometer
(resolution 0.001). I then derive the average.
 I compare the two averages. The difference is the Bias.
Repeatability
72

 Variation among repeated Repeatability

measurements
 Known as Precision
 Standard NOT required
 May add or subtract from a given Measurement Scale

measurement 5.15 = 99%

 Affects each measurement randomly Margin of Error
Doesn’t address Bias
Repeatability Issues
73

 Measurement Steps
 Sample preparation
 Setting up the instrument
 Locating on the part
 How much of the measurement process should we repeat?
Using Shewhart Charts I
74

Repeatability
Using Shewhart Charts II
75
Evaluating Bias & Repeatability
76

 Same appraiser, Same part, Same instrument

 Multiple readings (n≥10 with 20 to 40 better)
 Analysis AIAG
 Average minus Standard Value = Bias
 5.15* Standard Deviation = Repeatability
 or +/- 2.575  [99% repeatability]
 or +/- 2  [95% repeatability]
 Histogram
 Probability
Repeatability Issues
77

 Making a measurement may involve numerous steps

 Sample preparation
 Setting up the instrument
 Locating the part, etc.
 How much of the measurement process should we repeat? How far
do we go?
Bias & Repeatability Histogram

Never include assignable cause errors

Linearity
79

 The difference in the Bias or Repeatability across the expected

operating range of the instrument.
Plot Biases vs. Ref. Values
80

Linearity = |Slope| * Process Variation = 0.1317*6.00 = 0.79

% Linearity = 100 * |Slope| = 13.17%
Causes of Poor Linearity
81

 Instrument not properly calibrated at both Upper and Lower extremes

 Error in the minimum or maximum Master
 Worn Instrument
 Instrument design characteristics
Reproducibility
82

 Variation in the averages

among different appraisers
repeatedly measuring the
same part characteristic
 Concept can also apply to
variation among different
instruments

Includes repeatability which must be accounted for.

Reproducibility Example
83
Calculating Reproducibility (I)
84

 Find the range of the appraiser averages (R0)

 Convert to Standard Deviation using d2*
(m=# of appraisers; g=# of ranges used = 1)
 Multiply by 5.15
 Subtract the portion of this due to repeatability
Calculating Reproducibility
85

People variance

Times done
Trials
Stability
86

 Variation in measurements
of a single characteristic
 On the same master
 Over an extended period
of time
 Evaluate using Shewhart
charts
Evaluate Stability with Run Charts
87
Stability
88

Both gages are stable, but.....

Importance of Stability
89

 Statistical stability, combined with subject-matter knowledge, allows

predictions of process performance
 Action based on analysis of Unstable systems may increase Variation
due to ‘Tampering’
 A statistically unstable measurement system cannot provide reliable
data on the process
 90

Methods of Analysis
Analysis Tools
91

 Calculations of Average and Standard Deviation

 Correlation Charts
 Multi-Vari Charts
 Box-and-Whisker Plots
 Run charts
 Shewhart charts
Average and Standard Deviation
92
Correlation Charts
93

 Describe Relationships
 Substitute measurement for desired measurement
 Actual measurement to reference value
 Inexpensive gaging method versus Expensive gaging method
 Appraiser A with appraiser B
Substitute Measurements
94

 Cannot directly measure quality

 Correlate substitute measure
 Measure substitute
 Convert to desired quality
Comparing Two Methods
95
Magnetic
 Two methods Line of Perfect Agreement

 Measure parts using both

 Correlate the two
 Compare to “Line of No Bias”
 Investigate differences

Line of Correlation

Stripping
Measurements vs. Reference Data
96
Measurements vs. Reference Correlation
97

Disparity
Comparing Two Appraisers
98
Run Charts Examine Stability
99
Multiple Run Charts

More than 3 appraisers confuses things...

Multi-Vari Charts
High Reading
101

 Displays 3 points
 Length of bar; bar-to-bar; Bar cluster to cluster
 Plot High and Low readings as Length of bar
Average Reading
 Each appraiser on a separate bar
 Each piece in a separate bar cluster

Low Reading
Multi-Vari Type I
102

 Bar lengths are long

 Appraiser differences small
in comparison
 Piece-to-piece hard to detect
 Problem is repeatability
Multi-Vari Type II
103

 Appraiser differences are

biggest source of variation
 Bar length is small in
comparison
 Piece-to-piece hard to detect
 Problem is reproducibility
Multi-Vari Type III
104

 Piece-to-piece variation
is the biggest source of
variation
 Bar length (repeatability)
is small in comparison
 Appraiser differences
(bar-to-bar) is small in
comparison
 Ideal Pattern
Multi-Vari Chart Example
105
Normalized Data
Multi-Vari Chart, Joined
106

Look for similar pattern

Using Shewhart Charts
107

 Subgroup = Repeated measurements,, same piece

 Different Subgroups = Different pieces and/or appraisers
 Range chart shows precision (repeatability)
 Average chart “In Control” shows reproducibility
If subgroups are different appraisers
 Average chart shows discriminating power
If subgroups are different pieces

(“In Control” is BAD!)

Shewhart Charts
108

This is not a good way to plot this data

Too many lines
Shewhart Chart of Instrument
109
 110

Gage R&R Studies

Gauge R&R Studies
111

 Developed by Jack Gantt

 Originally plotted on probability paper
 Revived as purely numerical calculations
 Worksheets developed by AIAG
 Renewed awareness of Measurement Systems as ‘Part of the
Process’

Consider Numerical vs. Graphical Data Evaluations

Terms Used in R&R (I) Minimum of 5.
2 to 10 To
accommodate 112
worksheet factors
 n = Number of Parts [2 to 10]
Parts represent total range of process variation
Need not be “good” parts. Do NOT use consecutive pieces.
Screen for size
 a = Number of Appraisers
3
Each appraiser measures each part r times 1 2 4 5
Study must be by those actually using
 R - Number of trials 1 Outside Low/High
1 Inside Low/High
 Also called “m” in AIAG manual Target
 g = r*a [Used to find d2* in table 2, p. 29 AIAG manual]
Terms Used in R&R (II)
113

 R-barA = Average range for appraiser A, etc.

 R-double bar = Average of R-barA, R-barB
 Rp = Range of part averages Process Variation
 XDIFF = Difference between High & Low appraiser averages
Also a range, but “R” is not used to avoid confusion
 EV = 5.15 = Equipment variation (repeatability)
 EV = 5.15 = Equipment variation (reproducibility)
 PV = Part variation
 TV = Total variation
R&R Calculations
114

Left over
Repeatability

Remember -
Nonconsecutive
Pieces

Left over
Repeatability
Product Process
Measurement Variation
System Variation
Accumulation of Variances
115
Evaluating R&R
116

 %R&R=100*[R&R/TV] (Process Control)

 %R&R=100*[R&R/Tolerance] (Inspection)

 Under 10%: Measurement System Acceptable

 10% to 30%: Possibly acceptable, depending upon use, cost, etc.
 Over 30%: Needs serious improvement
Analysis of Variance I
117

 Mean squares and Sums of squares

 Ratio of variances versus expected F-ratio
 Advantages
Any experimental layout
Estimate interaction effects
 Disadvantages
Must use computer
Non-intuitive interpretation
Analysis of Variance II
118

 The n*r measurements must be done in random sequence [a good

idea anyway]
 Assumes that EV [repeatability] is normal and that EV is not
proportional to measurement [normally a fairly good assumption]
 Details beyond scope of this course
Special Gauging Situations
119

 Go/No-Go
 Destructive Testing
If Gauges were Perfect
120
But Repeatability Means We Never Know The Precise Value

121
So - Actual Part Acceptance Will Look Like This:
122
The Effect of Bias on Part Acceptance
123
Go/No-Go gauges
124

 Treat variables like attributes

 Provide less information on the process, but...
 Are fast and inexpensive
 Cannot use for Process Control
 Can be used for Sorting purposes
“Short” Go/No-Go Study
125

 Collect 20 parts covering the entire process range

 Use two inspectors
 Gage each part twice
 Accept gauge if there is agreement on each of the 20 parts
* May reject a good measuring system
Destructive Tests
126

 Cannot make true duplicate tests

 Use interpenetrating samples
 Compare 3 averages
 Adjust using √n
Destructive Tests: Interpreting Samples
127

AIAG does not address

 128

Summary
Measurement Variation
129

 Observed variation is a combination of the production process PLUS

the measurement process
 The contribution of the measurement system is often overlooked
Types of Measurement Variation
130

 Bias (Inaccuracy)
 Repeatability (Imprecision)
 Discrimination
 Linearity
 Stability
Measurement Systems
131

 Material
 Characteristic
 Sampling and Preparation
 Operational Definition of Measurement
 Instrument
 Appraiser
 Environment and Ergonomics
Measurement Systems Evaluation Tools
132

 Histograms
 Probability paper
 Run Charts
 Scatter diagrams
 Multi-Vari Charts
 Gantt “R&R” analysis
 Analysis of Variance (ANOVA)
 Shewhart “Control” Charts
Shewhart Charts
133

 Range chart shows repeatability

 X-bar limits show discriminating power
 X-double bar shows bias
(if a known standard exists)
 Average chart shows stability
(sub-groups overtime)
 Average chart shows reproducibility
(sub-groups over technicians/instruments)
Conclusion
134

 Rule of Ten
 Operating Characteristic Curve
 Special Problems
Go/No-Go Gages
Attribute Inspection
Destructive Testing