10.

Assessment—
external quality
assessment
 10-1: Overview
Role in quality Assessment is a critical aspect of
management laboratory quality management, and
Organization Personnel Equipment
system it can be conducted in several ways.
One of the commonly employed
assessment methods is that of external
quality assessment (EQA). Purchasing
Process Information
and
control management
inventory

Documents
Occurrence
and Assessment
management
records

Facilities
Process Customer
and
improvement service
safety

Definition of The term EQA is used to describe a method that allows for comparison of
EQA a laboratory’s testing to a source outside the laboratory. This comparison
can be made to the performance of a peer group of laboratories or to the
performance of a reference laboratory. The term EQA is sometimes used
interchangeably with proficiency testing; however, EQA can also be carried
out using other processes.

EQA is here defined as a system for objectively checking the

laboratory’s performance using an external agency or facility.

Types of EQA Several EQA methods or processes are commonly used. These include:
1. Proficiency testing—external provider sends unknown samples for testing
to a set of laboratories, and the results of all laboratories are analyzed,
compared and reported to the laboratories.
2. Rechecking or retesting—slides that have been read are rechecked by
a reference laboratory; samples that have been analyzed are retested,
allowing for interlaboratory comparison.
3. On-site evaluation—usually done when it is difficult to conduct traditional
proficiency testing or to use the rechecking/retesting method.

Another method of interlaboratory comparison is the exchange of samples

among a set of laboratories, usually reserved for specialized tests for which
no proficiency testing is available. This method is used by very specialized or
sophisticated laboratories and therefore will not be further discussed in this
chapter.

112 Laboratory Quality Management System

 10-1: Overview

EQA Participation in an EQA programme provides valuable data and information, which:
benefits  allows comparison of performance and results among different test sites;
 provides early warning for systematic problems associated with kits or
operations;
 provides objective evidence of testing quality;
 indicates areas that need improvement;
 identifies training needs.

EQA helps to ensure customers, such as physicians, patients and health authorities,
that the laboratory can produce reliable results.

Individual laboratories can use EQA to identify problems in laboratory practices,

allowing for appropriate corrective action. EQA participation will help to evaluate
reliability of methods, materials and equipment, and to evaluate and monitor
training impact.

For laboratories performing public health–related testing, EQA can help to ensure
that results from different laboratories during surveillance activities are comparable.
EQA participation is usually required for accreditation. Also, EQA participation
creates a network for communication, and can be a good tool for enhancing a
national laboratory network. Samples received for EQA testing, as well as the
information shared by the EQA provider, are useful for conducting continuing
education activities.

Principal EQA programmes vary, but principal characteristics include the following:
characteristics  EQA programmes can either be free of charge or require a fee. Free EQA
of an EQA programmes include those offered by a manufacturer to ensure equipment is
scheme working correctly, and those organized by a regional or national programme for
quality improvement.
 Some EQA programmes are obligatory, either required by an accrediting
body or by law. Others are voluntary, and the quality manager may choose to
voluntarily participate in an EQA programme in order to achieve improvement
in the quality of the laboratory’s performance.
 The EQA programme can be organized at different levels: regional, national or
international.
 Individual laboratory results are kept confidential, and generally are only known
by the participating laboratory and the EQA provider. A summary is generally
provided and allows comparison to the overall group.

Laboratory Quality Management System 113

 10-1: Overview

 Some EQA schemes may address a single disease; for example, the EQA
programme for tuberculosis. Others may address many kinds of laboratory
tests, looking at the overall testing performance for microbiology. An example
of this multidisease or test programme is the national microbiology EQA in
France, which is obligatory.

Successful performance in an EQA programme reflects the effectiveness of the

laboratory’s quality management, and allows for recognition of laboratory quality
by external groups.

EQA is important for improvement of the laboratory quality

management system, as it is a measure of laboratory performance.

114 Laboratory Quality Management System

 10-2: Proficiency testing
Definitions Proficiency testing, or PT, has been in use by laboratories for many years. It is the
most commonly employed type of EQA, as it is able to address many laboratory
methods. PT is available for most of the commonly performed laboratory tests,
and covers a range of chemistry, haematology, microbiology and immunology
testing. Most laboratorians are familiar with the PT process, and many laboratories
employ some kind of PT.

Standards organizations recognize the importance of this tool, and the following
are examples of formal definitions that are in use.
• ISO/IEC Guide 43-1:1997:“Proficiency testing schemes (PTS) are interlaboratory
comparisons that are organized regularly to assess the performance of analytical
laboratories and the competence of the analytical personnel”.
• Clinical and Laboratory Standards Institute: “A program in which multiple
samples are periodically sent to members of a group of laboratories for analysis
and/or identification; whereby each laboratory’s results are compared with those
of other laboratories in the group and/or with an assigned value, and reported to
the participating laboratories and others”.

Proficiency In the PT process, laboratories receive samples from a PT provider. This provider
testing process may be an organization (non-profit or for-profit) formed specifically to provide PT.
Other providers of PT include central reference laboratories, government health
agencies, and manufacturers of kits or instruments.

In a typical PT programme, challenge samples are provided at regular intervals.

An optimal frequency will be 3–4 times yearly. If the programme cannot provide
challenges with this frequency, the laboratory may be able to seek additional
sources.

The laboratories participating in the programme analyze the samples and return
their results to the central organization. Results are evaluated and analyzed, and
the laboratories are provided with information about their performance and
how they compared with other participants. The participating laboratories use
the information regarding their performance to make appropriate changes and
improvements.

Role of the To be successful, PT instructions must be followed carefully, all paper work
laboratory completed accurately and results submission deadlines met. All PT results, as
well as corrective actions, should be recorded and the records maintained for an
appropriate period of time.

Laboratory Quality Management System 115

 10-2: Proficiency testing

PT is a tool to measure laboratory performance. Therefore, there must be no

difference in the treatment of PT samples and the patient’s sample. PT providers
make every effort to produce samples that exactly mimic, or closely resemble,
usual samples received from patients. PT samples must be processed by normal
testing method(s) and involve personnel who routinely perform the testing.

When PT is used for any purpose other than internal quality improvement, the
provider or central organization generally prohibits the discussion of results with
other laboratories. Some PT organizers send different samples to different groups
of laboratories to avoid interlaboratory discussion.

PT participation is valuable only if the information received is directed

to improvement in the laboratory.

Limitations It is important to remember that PT does have some limitations and it is not
appropriate to use PT as the only means for evaluating the quality of a laboratory.
PT results are affected by variables not related to patient samples, including
preparation of the sample, matrix effects, clerical functions, selection of statistical
methods of evaluation, and peer group definition. PT will not detect all problems
in the laboratory, particularly those that address the pre-examination and post-
examination procedures.

A single unacceptable result does not necessarily indicate that a problem exists
in the laboratory.

116 Laboratory Quality Management System

 10-3: Other external quality assessment methods
Using other EQA In situations where it is difficult to provide appropriate external samples, or
methods sometimes when normal laboratory quality control methods cannot be applied,
other procedures have been developed and used for EQA. The primary examples
and their uses are as follows:
 Rechecking/retesting has been used traditionally for EQA for microscopic slides
for acid-fast bacilli (AFB), and for human immunodeficiency virus (HIV) rapid
testing. It can also be used in other situations, but is not usually employed if
traditional PT is feasible.
 On-site evaluation has proven a useful technique for the same situations—AFB
examination and HIV rapid testing. It allows for an external evaluation of quality
on-site, and can be conducted in conjunction with PT or rechecking/retesting.

These procedures can be time-consuming and costly, and so are used only when
there are not good alternatives. It is essential to have a reference laboratory
with the capacity to do the repeat testing; the use of a reference laboratory
gives assurance that the re-examination process will give a dependable result.The
turnaround for the retesting must be accomplished in a timely manner, allowing
for immediate corrective actions. In some settings, transport of samples or slides
to the reference laboratory will present problems.

This EQA method is used for HIV rapid testing. HIV rapid testing presents some
special challenges, because it is often performed outside a traditional laboratory,
Retesting process and by persons who are not trained in laboratory medicine. Additionally, the kits
are single use, and cannot be subjected to the usual quality control methods that
laboratories employ. Therefore, retesting of some of the samples using a different
process such as enzyme immunoassay (EIA) or enzyme-linked immunosorbent
assay (ELISA) helps to assess the quality of the original testing.

Characteristically, the retesting is:

 done by a reference laboratory to ensure quality;
 performed on dried blood spots or serum collected at the time of the rapid
test performance;
 not performed as a blinded process, as this is unnecessary.

The number of samples retested must provide statistically significant data in

order to detect error. This becomes difficult in settings where small numbers of
rapid tests are performed. A full discussion of the statistical issues in retesting
is found in the Centres for Disease Control and Prevention and World Health
Organization Guidelines for assuring the accuracy and reliability of HIV rapid testing:
applying a quality system approach.

Laboratory Quality Management System 117

 10-3: Other external quality assessment methods

Rechecking This method is most commonly used for acid-fast smears; the slides that were read
process in the original laboratory are “rechecked” in a central or reference laboratory.
This allows for the accuracy of the original report to be evaluated, and also allows
assessment of the quality of the slide preparation and staining.

The following principles are important when performing recheck procedures:

 The slides for re-examination must be collected randomly. Every effort should
be made to avoid systematic sampling bias.
 Rechecking must be based upon statistical considerations. A common method
is for the central laboratory to recheck 10% of negative and 100% of positive
slides.
 When discrepancies occur, there should be procedures in place to resolve them.
 The outcome of rechecking must be analyzed for effective and timely feedback.

Advantage of It is usually recommended that rechecking be done in a blinded fashion, so that

performing blind the laboratorian performing the retest does not know the original results. In
recheck the study carried out by Martinez et al.1, random blinded rechecking provided
more accurate estimates of AFB microscopy results than on the nonrandomly
selected, nonblinded smears. This resulted in improved diagnosis and monitoring
of treatment response.

On-site A periodic visit by evaluators for on-site laboratory assessment is a type of EQA
evaluation that has been used when other methods of EQA are not feasible or effective.
Again, this method has most frequently been employed for assessment of sites
performing AFB smears and those performing HIV rapid testing.

On-site evaluation can be a valuable tool to:

• obtain a realistic picture of laboratory practices by observing the laboratory
under routine conditions in order to check that it is meeting quality requirements;
• provide information for internal process improvement;
• measure gaps or deficiencies—learn “where we are”;
• assist the laboratory in collecting information for planning and implementation
of training, monitoring and corrective actions.

On-site evaluation for the purpose of EQA may be conducted by a central

reference laboratory or other health authorities. On-site evaluation can be used
together with retesting and rechecking schemes to provide more information
about performance.

1 Martinez A et al. Evaluation of new external quality assessment guidelines involving random blinded rechecking of acid-fast bacilli
smears in a pilot project setting in Mexico. International Journal of Tuberculosis and Lung Diseases, 2005,9(3):301–305.

118 Laboratory Quality Management System

 10-4: Comparison of external quality assessment methods
Comparison Some of the characteristics of PT and rechecking are compared in the table below.
of some
characteristics Comparison of proficiency testing (PT) and rechecking/retesting (RC)
Method/characteristics PT RC
Interlaboratory comparison Yes Yes
Simulated samples Yes No
Real samples Yes/No Yes
Time and resources needed Less More
Analytes evaluated Many Few

Summary of Proficiency testing:

comparison  gives a good, objective measure of the laboratory performance
 can be organized to address most kinds of laboratory testing
 is cost-effective and can therefore be used frequently.

Retesting/rechecking:
 is useful when it is difficult or impossible to prepare samples to test all of the
testing process;
 is expensive and uses considerable staff time.

On-site evaluation:
 can give a true picture of a laboratory’s overall performance, and offer real-time
guidance for improvements that are needed;
 is probably the most costly, requiring staff time, travel time and expenses of those
performing the evaluation.

Laboratory Quality Management System 119

 10-5: Managing external quality assessment in the
laboratory
Participation in All laboratories should participate in EQA challenges, and this should include EQA
EQA for all testing procedures performed in the laboratory, if possible. The benefits of
this participation are considerable, and EQA provides the only means available
to a laboratory to ensure that its performance is comparable to that of other
laboratories.

For laboratories that are accredited, or that plan to seek accreditation, EQA
participation is essential. ISO 15189 addresses EQA requirements for laboratories
as follows.
 There is a requirement that the laboratory participate in interlaboratory
comparisons.
 Where an established EQA scheme is not available, an alternate EQA mechanism
will have to be considered for interlaboratory comparison, such as exchange of
samples with other laboratories.
 The laboratory management shall monitor the results of EQA and participate
in the implementation of corrective actions.
Management
process When participating in EQA programmes, the laboratory needs to develop a
process for the management of the process. A primary objective is to assure
that all EQA samples are treated in the same manner as other samples tested.
Procedures should be developed that address:
 Handling of samples—These will need to be logged, processed properly and
stored as needed for future use.
 Analyses of samples—Consider whether EQA samples can be tested so that
staff do not recognize them as different from patient samples (blinded testing).
 Appropriate record keeping—Records of all EQA testing reporting should be
maintained over a period of time, so that performance improvement can be
measured.
 Investigation of any deficiencies—For any challenges where performance is not
acceptable.
 Taking corrective action when performance is not acceptable—The purpose of
EQA is to allow for detection of problems in the laboratory, and to therefore
provide an opportunity for improvement.
 Communication of outcomes to all laboratory staff and to management.

120 Laboratory Quality Management System

 10-5: Managing external quality assessment in the laboratory

EQA If the laboratory performs poorly on EQA, the problems may lie anywhere along
performance the path of workflow. All aspects of the process will need to be checked. Some
problems examples of problems that may be identified include the following.

Pre-examination:
 The sample may have been compromised during preparation, shipping, or after
receipt in the laboratory by improper storage or handling.
 The sample may have been processed or labelled improperly in the laboratory.

Examination:
 The EQA challenge materials may exhibit a matrix effect in the examination
system used by the participating laboratory.
 Possible sources of analytical problems include reagents, instruments, test
methods, calibrations and calculations.Analytical problems should be investigated
to determine whether error is random or systemic.
 Competency of staff will need to be considered and evaluated.

Post-examination:
 The report format can be confusing.
 Interpretation of results can be incorrect.
 Clerical or transcription errors can be sources of error.

Incorrect data captured by the EQA provider is another possible source of error.

Laboratory Quality Management System 121

 10-6: Summary
Summary EQA is a system for objectively checking the laboratory’s performance using an
external agency or facility. All laboratories should participate in an EQA process
for all tests performed, whenever possible. Accredited laboratories are required
to participate in EQA.

There are several methods for conducting EQA.Traditional PT is available for many
tests, is cost-effective and provides useful information.When PT is not practical or
does not provide enough information, other methods should be employed.

There must be no difference in the treatment of a PT sample and a patient sample.

The normal testing methods must be followed and the procedure must involve
personnel who routinely perform the testing.

Key messages  As EQA uses valuable resources, the laboratory should make the best use
possible of its participation in EQA.
 EQA should not be punitive. It should be viewed as educational and used as a
tool to help direct improvement efforts in the laboratory.
 EQA is one of the critical elements of a laboratory quality management system.

122 Laboratory Quality Management System