Science

Received 11.18.05 | Revisions Received 1.26.06 | Accepted 2.2.06

Validating New Reagents:

Roadmaps Through the Wilderness
Roberta A. Martindale, BSc (MLS), MT(ASCP),1 George S. Cembrowski MD, PhD,1,2 Lucille J. Journault,1,2 Jennifer L. Crawford,
BSc (MLS),1,2 Chi Tran,1,2 Tammy L. Hofer, BSc (MLS), MBA,1,2 Bev J. Rintoul, BSc (MLS)1,2 Jean S. Der,1,2 Cathy W. Revers,
BSc (MLS),1,2 Cheryl A. Vesso,1,2 Carol E. Shalapay,1,2 Connie I. Prosser, PhD, FCACB,1,2 Donald F. LeGatt, PhD, FCACB1,2
( 1Department of Laboratory Medicine and Pathology, University of Alberta Hospitals, 2Capital Health Authority, Edmonton, Canada)
DOI: 10.1309/BRC6Y37NM3BU97WX

Abstract simply ignoring the QC shift. We offer a variation. Depending on the test type, specific
One of the most frequent quality control issues systematic approach to shifted quality control information is gathered about the magnitude of
faced by laboratory professionals is how to and/or patient data following a reagent lot the shifts in either the QC and/or the patient
respond appropriately to a shift in quality change. We divide laboratory tests into 3 data. The control mean is reset following an
control (QC) following a reagent lot change. types, (1) tests for which the analysis of QC isolated quality control shift. Evaluation of the
Possible actions include adjusting the control specimens is sufficient, (2) tests which shift in patient data is initiated by the
range, checking for shifts in patient data, or demonstrate between reagent lot shifts laboratory director when the shift exceeds a
infrequently, and (3) tests with between lot multiple of the allowable error.

It is well known that reagent lot changes can cause significant whenever a complete change of reagents occurs, “the laboratory
shifts in patient results. Mueller-Hinton agar lot changes can affect . . . demonstrate that changing reagent lot numbers does not
drug susceptibility results.1 In hematology, new lots of Simplate II affect the range used to report patient test results, and control
have caused incorrect bleeding times,2 and new lots of hematology values are not adversely affected by reagent lot number
reagent have affected automated complete blood counts.3 In the changes.”12 Soon after the enactment of CLIA ‘88, the require-
realm of clinical chemistry, reagent lot changes have affected im- ment for lot validation was promulgated in the accreditation
munoassays more than general chemistry tests. Lot-to-lot variation checklists of various professional organizations including that of
has been frequently implicated in shifts of patient and quality con- the College of American Pathologists (CAP).
trol prostate specific antigen (PSA) values,4,5 probably because of The rigor in fulfilling lot validation depends on the knowl-
the extensive application of PSA testing for screening and moni- edge, confidence, and prior practices of the laboratory directors
toring of prostate cancer. Variation in reagent lots has resulted in and the supervisory technologists. In preparing for our first CAP
significant variation in the testing of patients’ hepatitis C accreditation inspection last winter, our senior biochemistry staff
antibody,6 patients’ human chorionic gonadotropin (hCG),7 and and medical biochemistry supervisors spent well over 30 hours
theophylline quality control.8 Even point-of-care cholesterol synthesizing 3 different, test-dependent lot validations. Being
tests9,10 are affected by reagent lot variation. heavily influenced by our quality system, we present these vali-
In some analytical systems, reagent lot changes seem to dations as flow charts. For one group of tests, we determined
primarily affect quality control measurements. The influence of that the analysis of quality control materials was sufficient for
between lot variation on control product analysis is often as- the workup of new reagent lots. For the other tests, the investi-
cribed to “matrix effects”; differences in constituent concentra- gation depended on the history of QC and patient shifts with
tion and constituents in the control product and the plasma or new reagent lots. With a stable method, quality control analysis
serum specimen that the control product is imitating. This was usually adequate; with a method that frequently
phenomenon has been frequently observed in dry reagent sys- demonstrated patient shifts with new reagent lots, we required
tems like that of the Vitros (Ortho-Clinical Diagnostics, the analysis of retained patient specimens as well as quality con-
Rochester NY). Hill and colleagues have shown that slide gen- trol specimens. This general approach is summarized in Table 1.
eration changes in the Vitros can cause significant excursion in Multiple quality control specimens must be run in order to
quality control measurements but little change in re-analyzed detect the presence of a shift. To quantify a quality control shift,
patient specimens.11 we suggest that 4 measurements be made at each control level.
Prior to the 1992 enactment of CLIA ‘88,12 reagent lot val- The shift is calculated by subtracting the usual control mean
idation practices were highly variable. General guidelines8 for from the average of the controls. We suggest that the magnitude
measuring and responding to lot-related shifts in control and of the QC shift be assessed by comparing the control shift to the
patient data were not widely applied and very few laboratories method’s stable standard deviation. If the magnitude of the QC
validated new reagent lots by re-analyzing retained patient speci- shift exceeds 1 SD, we require recalculation and resetting of the
mens. Section 493.1255 of the CLIA ‘88 Standard specifies that QC mean.

labmedicine.com June 2006 䊏 Volume 37 Number 6 䊏 LABMEDICINE 347

Science

To quantify the shift in patient data, we suggest that 10

different previously analyzed patient specimens be selected and
re-analyzed with the new reagents. If possible, the specimens
should be chosen to represent the usually measured concentra-
tion range. The shift is calculated by subtracting the average of
the initial results from the average of the new results. The
reanalysis of 10 specimens will result in greater than 60% of sig-
nificant shifts being discovered and corrected. (Personal commu-
nication, November 17, 2005, A. Srinivasan, LifeScan, Milpitas,
CA). The magnitude of the patient shift is assessed by compar-
ing the shift to a multiple of the allowable error (AE) of the
method. This allowable error may be the CLIA proficiency test
(PT) limit. As CLIA proficiency test limits are not available for
all analytes and are sometimes regarded as too broad, alternate
allowable errors based on physiologic variation are available.13
Table 2 shows the allowable errors that we use in our medical
biochemistry laboratory. We suggest that if the patient shift ex-
ceeds 0.5 x AE, then the method is judged as unacceptable and a
new reagent lot be obtained. Some laboratory professionals may
elect to use more stringent criteria for maximum allowable pa-
tient shift (eg, a maximum bias of 0.33 x AE). Some clinical lab-
oratories may require even tighter limits (eg, clinical trial
laboratories). These laboratories might attempt to use allowable
error limits derived from state of the art analytical performance.

Test-Specific Algorithms for Reagent

Validation

I. Tests where only QC is evaluated (Unstable analyte/ Figure 1_Tests where only QC is evaluated: Unstable analyte/Unstable
Unstable reagents/Insufficient material for retesting/Extremely reagents/Insufficient material for retesting/Extremely tedious or
tedious or time-consuming) time-consuming.
There are at least 4 types of tests where we assess only the
shifts in quality control data, even when between lot variation in
reagents may cause shifts in patient data. Some tests measure
very unstable analytes such as ACTH, insulin, vitamin A, and the discovery of QC shifts exceeding 1 standard deviation (SD).
vitamin E. For these analytes, it would be misleading to reana- The 4 replicate quality control measurements are averaged with
lyze previously analyzed specimens with alternate set of reagents. their average compared to the usual QC average. If the reagents
Some tests use highly unstable reagents (eg, bile acids and free are deemed unacceptable, reagents are remade if prepared in-
fatty acids). These reagents are so unstable that they need to be house. Otherwise, manufacturer controls may be analyzed before
reconstituted daily. In order to validate such reagents with pa- the manufacturer is notified.
tient specimens, we would need to measure the retained speci-
mens daily. For other tests, there may be little or no specimen II. QC-Based Reagent Lot Validation for Methods with
remaining (eg, tissue iron, tissue copper, etc). Finally, some tests Usually Clinically Unimportant Lot-to-Lot Variation
are so laborious or time-consuming that we rarely would repeat Some chemistry methods rarely demonstrate between lot
retained specimens with new reagent lots (eg, fecal fats). variation for either control or patient results. We propose the
Figure 1 shows the algorithm for validating new reagent following criterion for defining a method to be independent of
lots with only quality control analysis. This validation can occur lot variation: a method is stable if the QC bias is less than 1 SD
during patient testing. Briefly, quality control specimens are run for 3 successive reagent lot changes. The history of lot stability
in duplicate until there are 4 control observations at each level. must be documented. Examples of stable methods include elec-
Trouble-shooting and/or recalibration follow out of range QC or trolytes, calcium, phosphorous, total protein, albumin, urea, and

Table 1_General Approach to the Investigation of New Reagent Lots

Test Method QC Analysis Retained Patient Specimen Analysis

Laborious/Infrequently performed/Unstable analyte 4 control replicates, each level None

Stable, generally no QC or patient shift with new reagent lots 4 control replicates, each level 10 retained patient specimens if control shift discovered
Sometimes demonstrates clinically important lot to lot variation in 4 control replicates, each level 10 retained patient specimens
patient specimens with new reagent lot

348 LABMEDICINE 䊏 Volume 37 Number 6 䊏 June 2006 labmedicine.com

 Science

Table 2_Allowable Error (AE) Criteria

Analyte CLIA ‘88 Alternate allowable Analyte CLIA ‘88 Alternate allowable
error limit based on error limit based on
physiologic variation physiologic variation

Alpha-1 Antitrypsin ±3 SD ±10% Human chorionic gonadotropin ±3 SD or

Acetaminophen ±15% Pos/Neg
Acetone ±15% IgA ±3 SD ±15%
Alpha-fetoprotein ±3 SD ±15% IgD ±3 SD
Alanine aminotransferase ±20% ±30% IgE ±3 SD
Albumin ±10% ±4% IgG ±25% ±10%
Aldosterone ±40% IgM ±3 SD ±20%
Alkaline phosphatase ±30% ±12% Imipramine ±20%
Amikacin ±20% Insulin ±35%
Amitriptyline ±20% Ionized calcium ±2%
Ammonia ±30% Iron ±20% ±30%
Amylase ±30% ±16% Isopropanol ±15%
Androsteredione ±25% Lactate ±30%
Angiotensin converting enzyme ±20% Lactate dehydrogenase ±20% ±15%
Apolipoprotein A1 ±10% Lead, blood ±0.19 µmol/L
Apolipoprotein B ±15% or ±10%
Aspartate aminotransferase ±20% ±15% Luteinizing hormone ±20%
Beta 2 microglobulin ±10% Lipase ±30%
Beta hydroxybutyrate ±15% Lipoprotein (a) ±30%
Bilirubin, direct ±45% Lithium ±0.3 mmol/L ±20%
Bilirubin, total ±6.84 µmol/L ±35% or ±20%
or 20% Magnesium ±25% ±5%
CA 125 ±40% Methanol ±15%
CA 15-3 ±20% Methotrexate ±20%
CA 549 ±20% Myoglobin, Urine ±40%
CA 19-9 ±50% N telopeptide ±30%
Calcium ±0.2495 ±2.5% N-acetylprocainamide ±20%
mmol/L Nortriptyline ±20%
Calcium, urine ±30% Oxalate - urine ±50%
Carbamazepine ±25% ±20% Phenobarbital ±20% ±20%
Carcinoembryonic antigen ±25% Phenytoin ±25% ±20%
Carotene ±45% Phosphate ±10%
Ceruloplasmin ±10% Potassium ±0.5 mmol/L ±6%
Chloride ±5% ±1.5% Prealbumin ±15%
Cholesterol, HDL ±30% Primidone ±25% ±20%
Cholesterol, LDL ±15% Procainamide /N-acetyl ±25% ±20%
Cholesterol, total ±10% ±10% procainamide
Complement C3 ±3 SD ±10% Prolactin ±22%
Complement C4 ±3 SD ±20% Protein, total ±10% ±5%
Copper ±10% Parathyroid hormone ±22%
Cortisol ±25% ±30% Pyruvate ±20%
C-peptide ±15% Quinidine ±25% ±20%
Creatine kinase ±30% ±30% Rheumatoid factor ±15%
Creatine kinase, MB ±3 SD ±30% Salicylate ±20%
Creatinine ±26.52 µmol/L ±10% Selenium ±15%
or ±15% Sex hormone binding globulin ±25%
Creatinine, urine ±30% Sodium ±4 mmol/L ±1%
C-reactive protein ±70% T3 uptake ±3 SD ±7%
Cysteine ±10% Thyroid binding globulin ±10%
Desipramine ±20% Testosterone ±16%
Dehydroepiandrosterone sulfate ±12% Theophylline ±25% ±20%
Digoxin ±0.26 nmol/L ±15% Thyroxine (T4) ±12.9 nmol/L ±10%
or ±20% or ±20%
Estradiol ±25% Thyroxine, free (FT4) ±3 SD ±10%
Ethosuximide ±20% ±20% Tobramycin ±20%
Ethanol +25% +15% Transferrin ±5%
Ethylene glycol ±15% Tricyclic antidepressants ±20%
Ferritin ±20% Triglycerides ±25% ±30%
Triiodothyronine (T3) ±3 SD ±15%
Folate, rbc ±30% Troponin ±25%
Free testosterone ±50% Thyroid stimulating hormone ±3 SD ±25%
Follicle stimulating hormone ±15% Urea nitrogen ±0.71 mmol/L ±17%
Gentamicin ±25% ±20% or ±9%
GGT ±25% Uric acid ±17% ±15%
Glucose ±0.33 mmol/L ±10% Valproate ±25% ±20%
or ±10% Vancomycin ±20%
Haptoglobin ±30% Vitamin B12 ±16%
HbA1C ±4% Vitamin C ±30%
Homocysteine ±20% Zinc ±15%

labmedicine.com June 2006 䊏 Volume 37 Number 6 䊏 LABMEDICINE 349

Science

creatinine measured by the Beckman LX-20 and TSH measured with alternate reagent lots. Some of our core chemistry labo-
by the Bayer ADVIA Centaur. ratory assays demonstrate this variation: troponin, hCG,
For these reagent lot-independent methods, we recommend and folate on the Bayer ADVIA Centaur and the enzyme
that only the quality control results be evaluated with the new and turbidimetric tests on the Beckman LX-20. Each sec-
reagent before a new reagent lot is placed into service. For these tion in medical biochemistry maintains a list of tests that
tests, it is acceptable to analyze quality control material just after usually demonstrate large variation with alternate lots of
reagent lot change and then in the usual schedule. If a persistent reagents. For these methods, before being placed into serv-
QC shift is detected in the 48 hours following reagent lot ice, new reagent lots must evaluated for significant shifts in
change we then recommend that the magnitude of the patient patient and quality control results.
bias also be evaluated. Figure 3 shows the algorithm used to validate such un-
Figure 2 shows the algorithm used to validate such stable stable tests. After the reagent lot change, quality control
tests. Because of the infrequent occurrence of significant lot de- specimens are run in duplicate to more rapidly obtain 4
pendent variation, we suggest that this validation occur during replicate observations at each level. If the QC has not
patient testing. Briefly, quality control specimens are run in the shifted, 10 retained patient specimens are analyzed with the
usual manner after reagent lot change. Trouble-shooting and/or new reagent lot and the size of the patient bias assessed. If
recalibration follows the violation of any quality control rule. In the QC has shifted, trouble-shooting and/or recalibration
the case of unsuccessful trouble-shooting, 10 retained patient are attempted. If these procedures do not restore the origi-
specimens or previously analyzed retained PT materials (in the nal QC means (indicating a QC shift), the retained patient
absence of retained patient samples) are reanalyzed. If the patient specimens are analyzed with the new reagent lot. If the pa-
or PT bias is too large, the clinical biochemist or pathologist is tient bias is too large, the clinical biochemist or pathologist
consulted. Otherwise, the QC mean is reset if the QC has shifted is consulted. Otherwise, the QC mean is reset if the QC
by more than 1 SD. has shifted by more than 1 SD.

III. Patient-Based Reagent Lot Validation for Methods

With Significant Lot to Lot Variation
Some chemistry methods can demonstrate large
variations in quality control and/or patient specimen testing

Figure 2_QC-based reagent lot validation for methods with usually Figure 3_Patient-based reagent lot validation for methods with signif-
clinically unimportant lot to lot variation. icant lot to lot variation.

350 LABMEDICINE 䊏 Volume 37 Number 6 䊏 June 2006 labmedicine.com

 Science

Calculations and Record-Keeping

We designed a Microsoft Access program for entering, ana-
lyzing, and retrieving the required quality control and patient
replicate values. Figure 4 shows an output screen from this pro-
gram that summarizes the reagent information as well as the req-
uisite QC replicate data. Figure 5 shows an example of an input
screen for the entry of CK patient data (current and new lot).
The “drop-down” menu of analyte names is linked with the al-
lowable errors shown in Table 2 and permits easy evaluation of
the magnitude of the patient shift.

Discussion
We have been using this new lot qualification system for
the last 16 months. For the LX-20 systems, we evaluated 103
new reagent lots. Two lots of acetaminophen were judged unac-
ceptable; the manufacturer has just narrowed its acceptance cri-
teria for acetaminophen variation. With respect to the other
LX-20 analytes, all were acceptable; the following tests required
quality control adjustments: alkaline phosphatase (2 lots), ala-
nine transaminase (1), amylase (2), C-reactive protein (1), CSF
protein (4), gamma glutamyl transferase (1), lipase (1), microal- Figure 4_Output screen showing evaluation of the control data.
bumin (2), salicylates (1), theophylline (2), and triglycerides (1).
Until recently, only troponin and vancomycin were tested with
the lot qualification system; troponin required quality control
adjustments with 2 lots. We are adding more Centaur
immunoassay tests to our reagent qualification system.
Despite these efforts, we were unable to detect successive
increases in gamma glutamyl transferase patient results. Another
city laboratory discovered our higher results with a patient com-
parison study. We are now scrutinizing our gamma glutamyl
transferase lot validations.
The system has provided us with defensible and logical cri-
teria for remaking new reagents and even returning “defective”
reagent lots. We have found that our QC adjustments are made
with increased confidence. As our chemistry laboratory is large
and run by 4 laboratory scientists, the flowcharts presented here
have standardized the evaluation of new reagents. The Access
data base program has greatly simplified our record keeping of
validation of new reagent lots. LM
Acknowledgement: We wish to acknowledge the initial Figure 5_Input screen showing results of reagent lot comparisons using
patient specimens.
guidance that was provided by Ms. Elsa Quam of the University
of Wisconsin Hospital Clinical Laboratories, Madison, WI.

1. Fuchs PC, Barry AL, Brown SD, et al. Intrepretive criteria and quality control 8. Cembrowski GS, Carey NR. Laboratory Quality Management. Chicago: ASCP Press,
parameters for testing of susceptibilities of Haemophilus influenzae and Streptococcus 1989;102-103.
pneumoniae to trimethoprim and trimethoprim-sulfamethoxazole. J Clin Microbiol. 9. Gottschling HD, Reuter W, Ronquist G, et al. Multicentre evaluation of a non-wipe
1997;35:125-131. system for the rapid determination of total cholesterol in capillary blood, Accutrend
2. Houdijk WP. Warning: Simplate II, lack of standardization in standardized bleeding cholesterol on Accutrend GC. Eur J Clin Chem Clin Biochem. 1995;33:373-381.
time devices—reply: Lot-to-lot variation in ejection force has been rectified. Thromb 10. Boerma GJ, Gelderland J, van Gorp I, et al. Use of the Reflotron system for
Haemost. 1991;66:625-626. cholesterol assay in capillary blood, venous blood, and serum—evaluation of
3. Anonymous. COULTER lyse S III diff: Reagent lot gives erroneous results. Health accuracy and lot-to-lot reagent comparability. Clin Chem. 1988;34:2117-2119.
Devices. 1991;20:445-446. 11. Hill SA, Heathcote JC, McQueen MJ. The significance of matrix effects on the
4. Carey RN, Frye RM, Cook JD, et al. Between-lot/between-instrument variations of measurement of lactate dehydrogenase (LD) activity using Kodak dry slide
the Abbott IMx method for prostate-specific antigen. Clin Chem. 1992;38:2341- technology in the Ontario Laboratory Proficiency Testing Program. Clin Biochem.
2343. 1990;23:179-181.
5. Wener MH, Daum PR, Brawer MK. Variation in measurement of prostate-specific 12. Department of Health & Human Services Health Care Financing
antigen: importance of method and lot variability. Clin Chem. 1995;41:1730-1737. Administration Public Health Service 42 CFR Part 405, et al. Clinical
6. Dufour DR. Lot-to-lot variation in anti-hepatitis C signal-to-cutoff ratio. Clin Laboratory Improvement Amendments of 1988; final rule. Federal Register
Chem. 2004;50:958-960. Friday February 28, 1992.
7. Goodman DB, Bulley M, Hendricks M, et al. Assessment of the Abbott IMx assay 13. Ricos C, Alvarez V, Cava F, et al. Current databases on biologic variation: pros,
system for the measurement of human chorionic gonadotropin levels in the cons and progress. Scand J Clin Lab Invest. 1999;59:491-500. (updated in
treatment of ectopic pregnancy. Arch Pathol Lab Med. 1993;117:701-703. http://www.westgard.com/biodatabase1.htm)

labmedicine.com June 2006 䊏 Volume 37 Number 6 䊏 LABMEDICINE 351