Control Charts

Dr.-Ing. Michael Koch

Institute for Sanitary Engineering, Water Quality and Solid Waste Management Universitt Stuttgart Dep. Hydrochemistry Bandtle 2 70569 Stuttgart GERMANY Tel.: +49 711 685 65444 / Fax: +49 711 685 67809 E-mail: Michael.Koch@iswa.uni-stuttgart.de

on behalf of

Assuring the Quality of Test and Calibration Results - ISO/IEC 17025 5.9

The laboratory shall have quality control procedures for monitoring the validity of tests and calibrations undertaken. The resulting data shall be recorded in such a way that trends are detectable and, where practicable, statistical techniques shall be applied to the reviewing of the results.

Koch, M.: Accreditation and Quality Management - I

on behalf of

Assuring the Quality of Test and Calibration Results - ISO/IEC 17025 5.9

This monitoring shall be planned and reviewed and may include, but not be limited to, the following:
regular use of certified reference materials and/or internal quality control using secondary reference materials; participation in interlaboratory comparison or proficiency-testing programmes; replicate tests or calibrations using the same or different methods; retesting or recalibration of retained items; correlation of results for different characteristics of an item.
3
Koch, M.: Accreditation and Quality Management - I on behalf of

Control Charts Powerful, easy-to-use technique for the control of routine analyses ISO/IEC 17025 demands use wherever practicable

Koch, M.: Accreditation and Quality Management - I

on behalf of

History of Control Charts

introduced by Shewhart in 1931 originally for industrial manufacturing processes for suddenly occurring changes and for slow but constant worsening of the quality Immediate interventions reduce the risk of production of rejects and complaints from the clients
on behalf of

Koch, M.: Accreditation and Quality Management - I

Quality control charts

developed by Shewhart in 1930 for industrial production control

Take samples during the process Measure a quality indicator Mark the measurement in a chart with warning and action limits
concentration upper action limit upper warning limit

target value

lower warning limit lower action limit

group-No.

10

11 12

13

14

15

16

17

18

Koch, M.: Accreditation and Quality Management - I

on behalf of

Principle of control charts

Graphical display of quality based on
A target value and Quality limits

With the following different control limits:

Warning limits: exceeding once is tolerated Action limits: Exceeding requires immediate action

Koch, M.: Accreditation and Quality Management - I

on behalf of

Guidelines and literature

ISO 8258 Control Charts Funk, Dammann and Donnevert: Quality Assurance in Analytical Chemistry. Wiley NORDTEST: Internal Quality Control Handbook for Chemical Laboratories, TR 569, www.nordicinnovation.net/nordtest.cfm ISO/DIS 13530: Guide to analytical quality control for water analysis
on behalf of

Koch, M.: Accreditation and Quality Management - I

Quality control charts in analytical chemistry

concentration +3s +2s 95,5 % 99,7 % warning range target value action range

-2s -3s

serial No or date

Koch, M.: Accreditation and Quality Management - I

on behalf of

Action limits There is probability of only 0.3 % that a (correct) measurement is outside the action limits (3 out of 1000 measurements) Therefore the process should be stopped immediately and searched for errors

10

Koch, M.: Accreditation and Quality Management - I

on behalf of

Warning limits 4.5% of the (correct) values are outside the warning limits. This is not very unlikely Therefore this is only for warning, no immediate action required

11

Koch, M.: Accreditation and Quality Management - I

on behalf of

Calculation of standard deviation measurements marked in the control chart are between-batch standard deviation should also be between-batch estimation from a pre-period of about 20 working days repeatibility STD too narrow limits interlaboratory STD too wide limits
12
Koch, M.: Accreditation and Quality Management - I on behalf of

Requirements for the pre-period Conditions as similar as possible to the control period
not more care if necessary with change of operator intermediate conditions like in routine
repeatibility too narrow limits reproducibility (PT) too wide limits

13

Koch, M.: Accreditation and Quality Management - I

on behalf of

Limits

Fitness for purpose

Action and warning limits have to be compatible with the fitness-for-purpose demands no blind use

14

Koch, M.: Accreditation and Quality Management - I

on behalf of

What are Out-of-control-situations? The control chart shows, that the current situation is very unlikely to occur for normal distributed data

15

Koch, M.: Accreditation and Quality Management - I

on behalf of

Out-of-control-situation 1
concentration upper action limit upper warning limit

target value

lower warning limit lower action limit

group-No.

10

11

12

13

14

15

16

17

18

16

Koch, M.: Accreditation and Quality Management - I

on behalf of

Out-of-control-situation 2
concentration upper action limit upper warning limit

target value

lower warning limit lower action limit

group-No.

10

11

12

13

14

15

16

17

18

17

Koch, M.: Accreditation and Quality Management - I

on behalf of

Out-of-control-situation 3
concentration upper action limit upper warning limit

target value

lower warning limit lower action limit

group-No.

10

11

12

13

14

15

16

17

18

18

Koch, M.: Accreditation and Quality Management - I

on behalf of

Out-of-control-situation 4
concentration upper action limit upper warning limit

target value

lower warning limit lower action limit

group-No.

10

11

12

13

14

15

16

17

18

19

Koch, M.: Accreditation and Quality Management - I

on behalf of

Out-of-control-situation 5
concentration upper action limit upper warning limit

target value

lower warning limit lower action limit

group-No.

10

11

12

13

14

15

16

17

18

20

Koch, M.: Accreditation and Quality Management - I

on behalf of

10

Out-of-control-situations in different publications ISO 8258

1. 2. 3. 4. 5. 6. 7. 8. One point beyond action limits Nine points in a row on the same side of the central line Six points in a row steadily increasing or decreasing Fourteen points in a row alternating up and down Two out of three points in a row beyond warning limits Four out of five points in a row beyond 1s limits on the same side Fifteen points in a row within 1s limits Eight points in a row beyond 1s limits on both sides of the central line

21

Koch, M.: Accreditation and Quality Management - I

on behalf of

Out-of-control-situations in different publications Funk, Dammann, Donnevert 1. One point beyond action limits 2. Two out of three points in a row beyond warning limits 3. Seven points in a row on the same side of the central line 4. Seven points in a row steadily increasing or decreasing 5. 10 out of 11 points in a row on the same side of the central line
22
Koch, M.: Accreditation and Quality Management - I on behalf of

11

Out-of-control-situations in different publications NORDTEST TR569

1. out of control
a. One point beyond action limits b. Two out of three points in a row beyond warning limits

2. in control, but out of statistical control

a. Seven points in a row steadily increasing or decreasing b. 10 out of 11 points in a row on the same side of the central line
23
Koch, M.: Accreditation and Quality Management - I on behalf of

Out-of-control-situations in different publications ISO/DIS 13530 1. One point beyond action limits 2. Two consecutive values beyond warning limits (hopefully to be changed to 2 out of 3) 3. 7 points in a row steadily increasing or decreasing 4. 10 out of 11 points in a row on the same side of the central line (for X-charts only) 5. 7 consecutive control values lie above the mean range (for range-charts only)
24
Koch, M.: Accreditation and Quality Management - I on behalf of

12

Which one to choose? It is up to you to decide There is no prescription, no bible Control charts are just a tool Select the one that fits best for your needs

25

Koch, M.: Accreditation and Quality Management - I

on behalf of

What do Out-of-control Situations Mean? How do I have to React?

Out-of-control situations do not mean:

throw away everything! start again!

they rather mean:

Attention! An improbable situation has happened in the process! Stop the process! Look what has happened!
on behalf of

26

Koch, M.: Accreditation and Quality Management - I

13

What do I have to do? Do not report any results to the client! Recall already reported results! Do not continue to measure! look for the mistakes! The type of out-of-control situation can give valuable hints!

27

Koch, M.: Accreditation and Quality Management - I

on behalf of

Konzentration obere Eingriffsgrenze obere Warngrenze

Reaction to Situation 1

Sollwert

untere Warngrenze untere Eingriffsgrenze

Possibly a singular mistake happened during the analyses of the control sample. Analyse it again. If the value is confirmed, the analytical process must be inspected for a suddenly occurring change. When the mistake is found, continue with measurements!
Gruppen-Nr. 1 2 3 4 5 6 7 8 9 10 11

12

13

14

15

16

17

18

28

Koch, M.: Accreditation and Quality Management - I

on behalf of

14

Konzentration obere Eingriffsgrenze

Reaction to Situation 2

obere Warngrenze

Sollwert

untere Warngrenze

A mistake has occurred which either decreased the precision of the analyses or which led to a shift of the values in one direction (only if the deviation is in the same direction) Possible causes: change of operator, change in the procedure, in the environmental conditions, in the status of the analytical devices etc. Look for the mistake! When the mistake is found, continue with measurements!
untere Eingriffsgrenze Gruppen-Nr. 1 2 3 4 5 6 7 8 9 10 11 12 13

14

15

16

17

18

29

Koch, M.: Accreditation and Quality Management - I

on behalf of

Reaction to Situation 3

concentration upper action limit upper warning limit

target value

lower warning limit

The mean has shifted Possible causes: change of the lot of chemical, solvent etc., new adjustment or calibration of an instrument, change of operator, change in the procedure, in the environmental conditions, in the status of the analytical devices etc. Look for the cause! When the cause is found, continue with measurements! Attention! The new mean could eventually be less biased!
lower action limit group-No.

10

11

12

13

14

15

16

17

18

30

Koch, M.: Accreditation and Quality Management - I

on behalf of

15

Konzentration obere Eingriffsgrenze

Reaction to Situation 4/5

obere Warngrenze

Sollwert

untere Warngrenze untere Eingriffsgrenze

The mean shows a trend Possible causes: Chemicals used are changing, a part of the instrument is changing, the environmental conditions are changing continuously Look for the cause! When the cause is found, continue with measurements!
Gruppen-Nr. 1 2 3 4 5 6 7 8 9 10

11 12 13 14 15 16 17

18

31

Koch, M.: Accreditation and Quality Management - I

on behalf of

Advantages of graphical display

much faster more illustrative clearer

32

Koch, M.: Accreditation and Quality Management - I

on behalf of

16

Control chart types

Mean- / X - Control chart Recovery rate - control chart Differences - control chart Blank value - control chart Range - control chart

33

Koch, M.: Accreditation and Quality Management - I

on behalf of

Different control charts X-chart original Shewhart-chart with single values from analytics mainly to validate precision trueness with reference materials also possible for calibration parameters (slope, intercept)

34

Koch, M.: Accreditation and Quality Management - I

on behalf of

17

Different control charts blank value chart

analysis of a sample, which can be assumed to not contain the analyte special form of the Shewhart chart information about
the reagents the state of the analytical system contamination from environment

enter direct measurements, not calculated values

35
Koch, M.: Accreditation and Quality Management - I on behalf of

Different control charts recovery rate chart - I reflects influence of the sample matrix Principle:
analyse actual sample spike this sample with a known amount of analyte analyse again

Recovery rate:
xspiked x unspiked RR = xexpected
36
Koch, M.: Accreditation and Quality Management - I

100%
on behalf of

18

Different control charts recovery rate chart - II detects only proportional systematic errors constant systematic errors remain undetected spiked analyte might be bound differently to the sample matrix better recovery rate for the spike Target value: 100%
37
Koch, M.: Accreditation and Quality Management - I on behalf of

Different control charts range chart absolute difference between the highest and lowest value of multiple analyses precision check control chart has only upper limits
concentration upper action limit upper warning limit

target value

sample-#

10

11

12

13

14

15

16

17

18

38

Koch, M.: Accreditation and Quality Management - I

on behalf of

19

Different control charts difference chart - I uses difference with its sign
analyse actual sample at the beginning of a series analyse same sample at the end of the series

calculate difference (2nd value 1st value) mark in control chart with the sign
39
Koch, M.: Accreditation and Quality Management - I on behalf of

Different control charts difference chart - II target value: 0

otherwise: drift in the analyses during the series

appropriate for precision and drift check

40

Koch, M.: Accreditation and Quality Management - I

on behalf of

20

Different Control Charts Cusum Chart - I highly sophisticated control chart cusum = cumulative sum = sum of all errors from one target value target value is subtracted from every control analyses and difference added to the sum of all previous differences

41

Koch, M.: Accreditation and Quality Management - I

on behalf of

Different Control Charts - Cusum Chart - II

T = 80 Nr. 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 s = 2.5 x x-T Cusum 82 +2 +2 79 -1 +1 80 0 +1 78 -2 -1 82 +2 +1 79 -1 0 80 0 0 79 -1 -1 78 -2 -3 80 0 -3 76 -4 -7 77 -3 -10 76 -4 -14 76 -4 -18 75 -5 -23
90

85

80

75

70 0 30 20 10 0 0 -10 -20 -30 2 4 6 8 10 12 14 16 2 4 6 8 10 12 14 16

42

Koch, M.: Accreditation and Quality Management - I

on behalf of

21

Different Control Charts - Cusum Chart - III V-mask as indicator for out-of-control situation
30 20 10 0 0 -10 -20 -30 2 4 6 8 10 12 14 16 -10 -20 -30 30

in control

20 10 0 0 2 4 6 8

out of control

10

12

14

16

choose d and so that

very few false alarms occur when the process is under control but an important change in the process mean is quickly detected
on behalf of

43

Koch, M.: Accreditation and Quality Management - I

Different Control Charts Cusum Chart - IV Advantages

it indicates at what point the process went out of control the average run length is shorter
number of points that have to be plotted before a change in the process mean is detected

the size of a change in the process mean can be estimated from the average slope
on behalf of

44

Koch, M.: Accreditation and Quality Management - I

22

Control samples
are useful for the control of the quality of the measurements over longer time period Requirements:
representative for matrix and concentration choose concentration so that the important range is covered (limits!) sufficient amount for longer time period stability for several months (if possible) no influence of the container no changes due to subsampling
45
Koch, M.: Accreditation and Quality Management - I on behalf of

Control samples Standard solutions to verify the calibration control sample must be completely independent from calibration solutions influence of sample matrix can not be detected limited control for precision very limited control for trueness
46
Koch, M.: Accreditation and Quality Management - I on behalf of

23

Control samples Blank samples

samples which probably do not contain the analyte to detect errors due to
changes in reagents new batches of reagents carryover errors drift of apparatus parameters

blank value at the start and at the end allow identification of some systematic trends
on behalf of

47

Koch, M.: Accreditation and Quality Management - I

Control samples Real samples multiple analyses for range and differences charts if necessary separate charts for different matrices rapid precision control no trueness check

48

Koch, M.: Accreditation and Quality Management - I

on behalf of

24

Control samples Real samples spiked with analyte for recovery rate control chart detection of matrix influence if necessary separate charts for different matrices substance for spiking must be representative for the analyte in the sample (binding form!) limited check for trueness
49
Koch, M.: Accreditation and Quality Management - I on behalf of

Control samples Synthetic samples synthetically mixed samples in very rare cases representative for real samples if this is possible precision and trueness check

50

Koch, M.: Accreditation and Quality Management - I

on behalf of

25

Control samples Reference materials

CRM are ideal control samples, but
often too expensive or not available

In-house reference materials are a good alternative

can be checked regularly against a CRM if the value is well known good possibility for trueness check

sample material from interlaboratory tests

51
Koch, M.: Accreditation and Quality Management - I on behalf of

Control Samples and the Analytical Process

Sampling
Control sample (Exception, large effort)

Control sample Control chart Control sample

Sample preparation

Measurement

52

Koch, M.: Accreditation and Quality Management - I

on behalf of

26

Which One? There are a lot of possibilities Which one is appropriate? How many are necessary? There is no general rule!

The laboratory manager has to decide! But there can be assistance

53
Koch, M.: Accreditation and Quality Management - I on behalf of

Choice of Control Charts - I

the more frequent a specific analysis is done the more sense a control chart makes. if the analyses are always done with the same sample matrix, the sample preparation should be included. If the sample matrix varies, the control chart can be limited to the measurement only.

54

Koch, M.: Accreditation and Quality Management - I

on behalf of

27

Choice of Control Charts - II

Some standards or decrees include obligatory measurement of control samples or multiple measurements. Then it is only a minimal additional effort to document these measurements in control charts. In some cases the daily calibration gives values (slope and/or intercept) that can be integrated into a control chart with little effort

55

Koch, M.: Accreditation and Quality Management - I

on behalf of

Special questions - Should the pre-period be renewed from time to time?

only if the target value changes or if it is necessary to adapt the quality targets because of
worsening of the analytical precision or because the present limits are not fit for the purpose any more

56

Koch, M.: Accreditation and Quality Management - I

on behalf of

28

Special questions - How to convert the control period to a new pre-period? If this is required
check the mean for a significant change t-test check the variance for a significant change F-test

57

Koch, M.: Accreditation and Quality Management - I

on behalf of

Special questions Rounding of measurement results Should results be rounded as usual prior to entering in the control chart? not rounded rounded values falsify all statistical calculations

58

Koch, M.: Accreditation and Quality Management - I

on behalf of

29

Benefits of using control charts a very powerful tool for internal quality control changes in the quality of analyses can be detected very rapidly good possibility to demonstrate ones quality and proficiency to clients and auditors
Koch, M.: Accreditation and Quality Management - I on behalf of

59

Rarely Executed Analytical Methods

Control charts are not useful here Alternatives:
Control of recovery rates in the sample matrix (define limits for acceptable recovery!) Repeated measurements for control of precision (define acceptance limits!) Check blank value Check calibration with standard material different from the calibration material Use of (certified) reference materials
on behalf of

60

Koch, M.: Accreditation and Quality Management - I

30

Quality control charts for presentations

stop here
Attention of auditorium

time

61

Koch, M.: Accreditation and Quality Management - I

on behalf of

31