CDC 79046 DS1

HHS Public Access
Author manuscript
Saf Sci. Author manuscript; available in PMC 2020 August 01.
Author Manuscript
Published in final edited form as:

Saf Sci. 2019 August ; 117: 205–216. doi:10.1016/j.ssci.2019.04.003.
Development and implementation of evidence-based laboratory

safety management tools for a public health laboratory
M.S. Kecklera,b,*, K. Andersona, S. McAllistera, J.K. Rasheeda, and J. Noble-Wanga
aCenters for Disease Control and Prevention, National Center for Emerging Zoonotic and
Infectious Diseases, Division of Healthcare Quality Promotion, Clinical and Environmental
Microbiology Branch, United States
Author Manuscript
bCenters for Disease Control and Prevention, Center for Surveillance, Epidemiology and
Laboratory Services, Laboratory Leadership Service Fellowship, United States
Abstract
We developed an evidence-based continuous quality improvement (CQI) cycle for laboratory
safety as a method of utilizing survey data to improve safety in a public health laboratory setting.
• Expert Opinion: The CQI cycle begins with the solicitation of laboratory staff input via an
annual survey addressing potential chemical, physical and radiological hazards associated with
multiple laboratory activities. The survey collects frequency, severity and exposure data related to
these activities in the context of the most pathogenic organisms handled at least weekly.
• Gap Analysis: Step 2 of the CQI cycle used survey data to identify areas needing improvement.
Author Manuscript
Typically, the traditional two-dimensional risk assessment matrix is used to prioritize mitigations.
However, we added an additional dimension – frequency of exposure – to create three-dimensional
risk maps to better inform and communicate risk priorities.
• Mitigation Measures: Step 3 of the CQI cycle was to use these results to develop mitigations.
This included evaluating the identified risks to determine what risk control measures (elimination,
substitution, engineering, administrative or PPE) were needed. In the 2016 iteration of the CQI
cycle described here, all mitigations were based on administrative controls.
• Evaluation and Feedback: The last step of the CQI cycle was to evaluate the inferred effects of
interventions through subsequent surveys, allowing for qualitative assessment of intervention
effectiveness while simultaneously restarting the cycle by identifying new hazards.
Author Manuscript
Here we describe the tools used to drive this CQI cycle, including the survey tool, risk analysis
method, design of interventions and inference of mitigation effectiveness.
Keywords
Laboratory safety; Occupational health; Evidence-based interventions; Survey; Continuous
improvement; Quality management
*
Corresponding author. hbq0@cdc.gov (M.S. Keckler).
Appendix A. Supplementary material
Supplementary data to this article can be found online at https://doi.org/10.1016/j.ssci.2019.04.003.
Keckler et al. Page 2
Author Manuscript
1. Introduction
Laboratory safety is first, and most importantly, an occupational health concern for the
estimated 290,988 public health workers in the United States (Beck et al., 2014). However,
contaminated or infected employees can also transmit occupationally acquired pathogens
outside the laboratory, making any actual or perceived safety breach in a public health
laboratory a serious concern (Fleck, 2004; Blaser and Lofgren, 1981; Hawkes, 1979). In
2014, a series of safety incidents among multiple federal agencies drew extensive national
media attention to the issue of safety in public health laboratories (McCarthy, 2014). These
safety failures can erode trust in the public health system (Cohen, 2014), which has the
potential to decrease compliance with public health agency recommendations (Ward, 2017).
Therefore, the consequences of laboratory incidents in public health laboratories can be
severe and widespread, even when occupational health risks are low (Centers for Disease
Author Manuscript
Control and Prevention, 2014).
While incidents involving biological hazards are often the focus of laboratory-related safety
discussions, it is well known that laboratories contain many potential hazards - including
chemical, physical, and radiological (World Health Organization, 2004; Chosewood and
Wilson, 2009; Occupational Health and Safety Administration, 2011). Unfortunately, current
data about laboratory incidents is difficult to obtain as there is not yet a standardized system
for reporting of laboratory incidents (Chamberlain et al., 2009; Dirnagl et al., 2016; Blaine,
2012). However, some insight into laboratory incidents can be gained using Bureau of Labor
statistics, which show that of the incidence rate (2011–2016) of nonfatal occupational
injuries and illnesses involving days away from work in medical and diagnostic labs is
100/10,000 full-time workers. Of these 100 illnesses or injuries, the source of 1% were
Author Manuscript
directly related to chemicals and chemical products. The other 99% of illnesses and injuries
came from a variety of potential chemical, physical and biological hazards which underlie
the Occupational Injury and Illness Classification System 2.01 source categories of
containers, furniture and fixtures (15%), machinery (5%), parts and materials (8%), persons,
plants, animals and minerals (26%), structures and surfaces (17%), tools, instruments and
equipment (4%), vehicles (10%) and other sources (13%). While it is difficult to relate these
reported source categories with the underlying hazards, the breadth of incident sources does
make obvious the need for laboratory safety risk assessments to consider all-hazards – not
just biological. In addition to hazard types, there is a growing body of knowledge about
various contributors to workplace safety (e.g. the effects of mental workload (Charles and
Nixon, 2019); the need for leadership training (Gravina et al., 2019); the importance of
occupational ergonomics (Fasanya and Shofoluwe, 2019) and the effects of worker
Author Manuscript
personality on safety behavior (Jong-Hyun et al., 2018), which have not been well-studied in
the laboratory setting.
While laboratory safety has long been a priority in public health laboratories (Moskowitz,
1948; Cook, 1961; Fuscaldo et al., 1980), multiple gaps remain between published best
practices and the actual implementation of these practices in laboratories (Westgard, 2017;
Herrmann-Werner et al., 2013; Van Noorden, 2013). There are many regulations, guidelines
and standards relevant to the work performed in laboratories, but strategies for

implementation of these guidelines are left to individual laboratories to develop (World

Author Manuscript
Health Organization, 2004; Chosewood and Wilson, 2009; Richmond and Nesby-O’Dell,
2002; Ned-Sykes et al., 2015; 7 CFR Part 331, 9 CFR Part 121, and 42 CFR Part 73 - Select
Agent Regulations, 2018; International Organization for Standardization, 2017; Miller et al.,
2012; 42 CFR 493, 2018; United States Code, 1988; International Organization for
Standardization, 2018; International Organization for Standardization, 2003; International
Organization for Standardization, 2012; 29 USC, 1910, 2018). Laboratory Quality
Management Systems (LQMS) can provide a framework for document and process controls,
as well as risk assessment and monitoring procedures to improve laboratory safety (Ahlin
and Weiss, 2007; Lord, 1990; Nichols, 2011); however, LQMS in public health laboratories
are frequently focused on patient safety and test result accuracy as opposed to occupational
health and safety (Allen, 2013; Lippi and Guidi, 2007; Njoroge and Nichols, 2014).
The laboratory managers and staff who develop, document and implement laboratory
Author Manuscript
procedures bring their own beliefs, knowledge, education, training, attitudes and experience
to their work, and this can affect how they identify and interpret laboratory hazards (Buxton
et al., 2011; Steelman and Alexander, 2016; Senthil et al., 2015). Laboratory risk
assessments are complex and differ significantly from laboratory to laboratory making a
standardized risk assessment approach difficult. However, obtaining a measure of worker
perceptions regarding laboratory safety can improve risk management (Xia et al., 2017;
Tziaferi et al., 2011). These are compelling reasons why better integration of safety and
quality management in public health laboratories is needed (Sciacovelli et al., 2007).
To determine a standardized process of assessing and mitigating laboratory risk, we chose to

develop a standardized, evidence-based continuous quality improvement (CQI) cycle. The
cycle starts with the solicitation of expert opinion regarding laboratory hazards through an
Author Manuscript
annual survey. Next is the gap analysis of the survey data to identify potential laboratory
hazards and perform risk analysis. Based on the gap analysis, we then design and implement
targeted mitigation measures. With the subsequent annual survey, the cyclic process ends
when data is compared with the previous year to infer the effectiveness of the mitigation
interventions and begins again by identifying new hazards or risks to target. Here we
describe the development of the survey tool and risk assessment method and the application
of these tools to design and evaluate evidence-based interventions.
1.1. Significant results summary

• Developed practical tools for use by laboratory staff and safety personnel
including:
– Annual all-hazards survey tool designed to gather laboratory safety

Author Manuscript
data.
♦ Tailored to the highest risk group pathogen handled at least

weekly
♦ Collected information on severity, probability and exposure

for biological, chemical, physical and radiological hazards
associated with equipment and processes performed in the lab.

– Method for stratifying the risk of microorganisms handled in a BSL-2

Author Manuscript
laboratory
– Three-Dimensional Risk Assessment Tool designed for analysis of

survey data
• Identified laboratory-specific risks for targeted intervention
– Potential underutilization of engineering controls
– Potential underreporting of near-misses
– Potentially low awareness of process hazards
• Measured significant differences in survey responses after interventions
– Staff training increased use of engineering controls

Author Manuscript
– Increased incident reporting burden may cause lower near-miss

reporting
– Hazard awareness was increased through presentation of survey results
2. Materials and methods

2.1. CQI stage one - expert opinion
To start the CQI cycle, data is collected from our laboratory staff regarding their perceptions
related to laboratory safety using a survey, which utilizes a non-random sampling
methodology that is purposive to assess risk and develop mitigation measures in a specific
federal public health laboratory. The anonymous survey is offered to all staff in the
laboratory on an annual basis, but its completion is not mandated. Because the sampling is
Author Manuscript
purposive, results from this survey are not generalizable to other laboratories. However, the
survey tool can be tailored to any laboratory.
2.1.1. Survey development—Reviews of multiple guidelines for laboratory safety

(World Health Organization, 2004; Chosewood and Wilson, 2009; Miller et al., 2012; 29
USC, 1910, 2018) were completed to formulate survey questions in the following categories:
Regulatory Requirements, Biologicals (to identify the highest risk organisms worked with at
least weekly), Equipment (inclusive of biological, chemical, radiological and physical
hazards), Process (inclusive of biological, chemical, radiological and physical hazards),
Mitigation (inclusive of elimination, substitution, engineering controls, administrative
controls and personal protective equipment (PPE)), Quality Controls, Safety Culture and
Safety Communications. We chose these categories to organize the survey with a goal
Author Manuscript
toward obtaining a comprehensive picture of laboratory staff perceptions related to

laboratory safety. The full text of the survey can be found in the supplementary information
Appendix A. The specific focus for each category in the survey follows.
2.1.1.1. Regulatory requirements (Questions 1 and 2).: Question 1 (Q1) elicited data for
the identification of work that requires external oversight. This included identification of
work involving recombinant DNA, clinical tests, animals, humans, dual use research of

concern or transfer of materials from BSL3 or BSL4 level laboratories. An additional

Author Manuscript
question collected self-reported data on compliance with these regulations (Q2).
2.1.1.2. Biologicals (Questions 3 through 12).: To elicit data for evidence-based risk
assessment of microbiological hazards in the laboratory, we formulated questions to obtain
self-reported data on work with pathogens (Q3) using two techniques.
2.1.1.2.1. Definition of stratified risks within risk group 2.: Risk Group 2 (RG2)
microorganisms that have been handled in the laboratory over the last 5 years were
subdivided into three substrata (High, Moderate, Low) based on a combination of literature
review and expert opinion. We categorized organisms as high risk (Q4) if they were on the
Department of Health and Human Services (DHHS) and United States Department of
Agriculture (USDA) Select Agents and Toxins List (7 CFR Part 331, 9 CFR Part 121, and 42
CFR Part 73 - Select Agent Regulations, 2018) or have been reported as the confirmed
Author Manuscript
source of a laboratory-associated infection or exposure (LAI/E) in relevant reviews of the

LAI/E scientific literature (Baron and Miller, 2008; Singh, 2009; Wurtz et al., 2016; Silver,
2015; Miller et al., 1987; Campbell et al., 2015; Sullivan et al., 1978; Pike, 1976; Pike and
Sulkin, 1951; Vesley and Hartmann, 1988). Microorganisms that have been confirmed as
foodborne infections (FI) (Powell, 2016) or healthcare-associated infections (HAI) (Magill
et al., 2014) were categorized as medium risk (Q6). The remainder of the RG2
microorganisms were categorized as low risk (Q8). The genus name of all microorganisms
in each substrata category were included in the survey text to aid respondents in answering
questions about these risk categories.
2.1.1.2.2. Use of skip logic.: Since most public health laboratory staff work with multiple
pathogens, we designed the survey to use conditional logic to obtain the highest risk
Author Manuscript
category microbes (Q4, Q6 and Q8) with which the respondent worked with at least weekly
(Q5, Q7 and Q9). Additional questions (Q10, Q11, and Q12) provided free-form entry of
additional pathogens but since these only yielded two responses which were included in the
low risk category, they were not included separately in this analysis. A flowchart of the
conditional logic structure is included in the Supplemental Materials Appendix B. This
allowed the responses of each individual to identify the work to consider when answering
the remainder of the survey.
2.1.1.3. Equipment (Question 13).: Question 13 asked for responses related to the
frequency of usage, frequency of error and severity of error for potentially hazardous
laboratory equipment. This was an all-hazards exercise as each piece of equipment
potentially represents biological (i.e. aerosol production or difficult to clean), chemical (i.e.
Author Manuscript
use of hazardous chemicals or potential for spills) or physical hazards (presence of high
voltage or pinch points). Responses were used to calculate a risk score using a modification
of the Severity, Probability and Exposure (SPE) Model (Cram, 2004) as explained in Section
2.2.1. Three-Dimensional Risk Analysis.
2.1.1.4. Process (Question 17).: Question 17 asked staff about the frequency of
performance, frequency of error and severity of error for potentially hazardous laboratory
activities and processes. Each process contains some unique combination of biological,

chemical or physical hazards. Responses were used to calculate a risk score as explained in
Author Manuscript
Section 2.2.1. Three-Dimensional Risk Analysis.
2.1.1.5. Mitigation (Question 12, 14 and 18).: These questions asked staff about the
frequency of use of multiple risk controls including administrative controls (e.g., procedures,
job aids, training), engineering controls (e.g., biosafety cabinets, chemical fume hoods, dead
air boxes) and Personal Protective Equipment (PPE) (e.g., gloves, lab coat, safety glasses)
when working with biological agents (Q12), equipment (Q14) and processes (Q18).
Questions 14 and 18 were not included in our analysis as they were removed from the 2017
survey since the results did not differ from the results of question 12 in the 2016 survey,
indicating that they were of little added value.
2.1.1.6. Quality controls (Questions 15, 16 and 19).: These questions obtained data from
staff regarding the frequency of availability of written protocols (Q15), proportion of these
Author Manuscript
protocols under document control (Q16) and frequency of adherence to these protocols
(Q19).
2.1.1.7. Safety Culture (Questions 20, 21 and 22).: These questions obtained data from
staff regarding the frequency of occurrence of safety culture-related issues in the laboratory
(Q20), the frequency of notification of incidents and near misses by scenario (Q21) and the
perceived priority of safety at various institutional management levels (Q22).
2.1.2. Safety survey reliability and validity

2.1.2.1. Content validity.: The content validity of the survey was assessed by a team of
six subject matter experts (SME) which included non-supervisory laboratory scientists with
additional expertise in safety and quality, as well as supervisory laboratory scientists with
Author Manuscript
experience in managing both laboratory quality and safety. As a group, the education level
of the SME was 67% PhD and 33% MS with general laboratory experience ranging from 7
years to over 30 years. The group met multiple times to discuss what should be measured by
the survey and then each SME reviewed draft survey question and answer sets through three
iterations to finalize the survey. Lastly, each SME took the online survey in its final form as
a small pilot to address any remaining issues before release of the survey in 2016.
2.1.2.2. Internal consistency reliability (coefficient alpha).: The internal consistency

reliability (coefficient alpha) of the survey and the Cronbach’s alpha for each of the scales
used in the survey was determined using IBM SPSS Statistical Software Version 24 (IBM,
California). Ideally, the Cronbach’s alpha should be no less than 0.70 and no more than 0.90
indicating that the scales used are reliable but not redundant (Streiner, 2003). Table 1 lists
Author Manuscript
the internal reliability of the scales used in this survey.
2.1.3. Safety survey administration—The 2016 laboratory safety survey contained

33 questions and was sent to 36 laboratory staff using an anonymous online survey through
the SurveyMonkey® platform. In 2017, the survey was 31 questions and was sent to 54
laboratory staff. The respondents were given two weeks to complete the survey, which took
an average of 16 min to complete (n = 83). In May 2016, 34 staff responded (Response Rate

= 94%) while in August 2017, 49 staff responded (Response Rate = 91%). In accordance
Author Manuscript
with the Paperwork Reduction Act, no more than 9 non-federal employees were surveyed in
either year.
2.1.4. Demographic questionnaire—To define the sample population in more detail

without compromising the anonymity of the laboratory safety survey, we administered a
separate questionnaire to collect demographic information about our staff in-between the
2016 and 2017 safety surveys. Respondents answered questions about their age category,
education and experience. The demographics questionnaire was sent in March of 2017 (10
months after the 2016 survey and 2 months prior to the 2017 survey) using an anonymous
online survey through the SurveyMonkey® platform. It was sent to 40 laboratory personnel
of which 36 responded (Response Rate = 90%). In accordance with the Paperwork
Reduction Act, no more than 9 non-federal employees were surveyed. The full text of the
demographic survey is in the supplementary information Appendix C.
Author Manuscript
2.2. CQI stage two - gap analysis

Once the survey data was collected, the next step was to perform a gap analysis. This
analysis included reviewing the responses to all survey questions with a team of SMEs to
identify areas for improvement, performing a three-dimensional risk analysis to identify
areas of higher relative risk among equipment and processes, and reviewing the most current
biosafety guidance and scientific literature. Gap analysis results (i.e. risk maps and areas
identified for improvement) were reviewed by laboratory leadership for awareness and
adjustment if needed.
2.2.1. Three-dimensional risk analysis—To standardize the approach to prioritizing

interventions and to maintain the usability of this survey as a practical tool with which to
Author Manuscript
improve laboratory safety in a public health laboratory, a method was needed to perform risk
analyses using survey data. Our goal in this effort was to establish a reproducible method for
analyzing survey results to prioritize safety interventions. In addition, we wanted a method
that would be straightforward to perform, provide semi-quantitative results for trend
analysis, and provide an easily interpreted output (a risk map) for use in prioritizing safety
interventions. To expand the granularity of the more traditional two-dimensional risk matrix,
we chose to utilize the Severity, Probability and Exposure (SPE) Model of risk assessment
and requested that respondents provide their estimation of:
• Severity (s) - How likely it is that an error in equipment usage or process

performance would lead to injury or exposure.
• Probability of Error (p) - How likely it is for an error to occur while using a
Author Manuscript
specific piece of equipment or performing a specific process.
• Exposure (e) - How often a piece of equipment is used or a process is

performed.
The scale used for perceived probability of error (p) and severity (s) was weighted as
follows: Highly Likely (75–100%) = 5; Moderately Likely (50–74%) = 4; Somewhat Likely
(25–49%) = 3, Not Likely (1–24%) = 2 and Not Used (0%) = 1. Exposure (e) was weighted

as follows: Daily = 4, Weekly = 3, Monthly = 2 and Yearly = 1. These weights are designed
Author Manuscript
to provide a maximum risk score of 100 for ease of comparison and ranking of risk scores
for prioritizing planning interventions. We then applied this model to various pieces of
laboratory equipment (n = 25) and multiple laboratory processes (n = 14). We calculated the
mean of all responses for each variable ((e), (p)and (s)) for each activity or instrument and
the variables were plugged into the following equation:
Relative Risk = Severity (s) × Probability (p) × Exposure (e)
This method allowed all data to be analyzed in a standardized manner and provides a
method by which laboratory risks can be compared and prioritized. Risk scores of 80–100
are considered critical, 60–79 very high, 40–59 high, 20–39 moderate and 1–19 low and are
graphed as radar maps for ease of risk visualization.
Author Manuscript
2.3. CQI stage three – mitigation measures

Safety staff and laboratory scientists reviewed and discussed the biological, chemical and
physical hazards identified during the gap analysis to determine if it was possible to
eliminate, substitute or isolate the hazards. Engineering controls, administrative controls and
PPE requirements were also considered and a list of suggested mitigations was prepared.
The suggestions from safety staff were discussed with additional laboratory staff to
determine the best mitigation to implement.
2.4. CQI stage four – evaluation and feedback

The final stage of the CQI cycle in 2016 was to present the results of this process to the
laboratory staff and solicit feedback on the survey and the process. However, in 2017, we
Author Manuscript
presented results, solicited feedback and compared the survey data from 2016 and 2017 to
analyze the efficacy of the 2016 mitigation measures.
2.4.1. Efficacy analysis—We used two-tailed, two sample equal variance t-tests to
compare 2016 and 2017 survey data to detect significant differences (p < 0.05) in responses
from one year to another. Because the sample was non-random and the data was
observational in nature, we cannot prove causality and can only infer that the interventions
were the cause of changes in survey responses.
3. Results
3.1. Demographics
To describe the sample population while maintaining respondent anonymity, we collected
Author Manuscript
demographic data in a separate survey presented in Table 2. These demographics were

collected in March of 2017 (10 months after the 2016 survey and 2 months prior to the 2017
survey) and therefore the demographics are most representative of the population sampled in
2017. The average respondent (see bolded categories) was a non-supervisory laboratory
employee; 25–34 years of age with a Master’s-level degree and 6–10 years of overall
laboratory experience with 1–3 years in a federal public health laboratory. All respondents
reported being vaccinated for Hepatitis B and 77% of respondents have never been injured in

a laboratory incident in any setting. Responses to the demographic questionnaire are

Author Manuscript
presented as both total respondents and percent of sample population for each category.
3.2. Regulatory requirements (Questions 1 and 2)

Our laboratory functions in the past have included projects involving clinical testing,
recombinant DNA research, dual use research of concern, human subjects research, animal
research and transfer of materials from high-containment laboratories. Each of these
functions require oversight from outside our laboratory. To gauge awareness of the
requirements for external oversight, respondents were asked to identify which regulatory
requirements were relevant to their work. There were no statistically significant changes
from 2016 to 2017 in either the percent of staff doing work that requires external oversight,
or in self-reported compliance using a two-tailed, two sample equal variance t-test.
Fig. 1 illustrates that the highest percentage of our staff (38% in 2016 and 50% in 2017)
Author Manuscript
perform clinical laboratory testing that is overseen by the Centers for Medicare & Medicaid
Services through the Clinical Laboratory Improvement Amendments (CLIA) regulations
(United States Code, 1988). A lower percentage of staff work on recombinant DNA research
(13% in 2016 and 9% in 2017), which is overseen by the Institutional Biosafety Committee
(IBC) as required by the NIH Guidelines (Federal Register, 2016). This was followed by
some staff performing work involving dual use research of concern (9% in 2016 and 13% in
2017) which is overseen by the Institutional Biosecurity Board (IBB) in accordance with the
United States Government Policy for Institutional Oversight of Life Sciences Dual Use
Research of Concern (Federal Register, 2014) and work involving human subjects research
(6% in 2016 and 9% in 2017) which is overseen by the Institutional Review Board (IRB) in
accordance with the Protection of Human Subjects (Common Rule) (Code of Federal
Regulations, 2018). The least number of staff reported that they performed work related to
Author Manuscript
the transfer of infectious materials out of high containment laboratories (0% in 2016 and 3%
in 2017) overseen by the Laboratory Safety Review Board (LSRB) as per institutional policy
or animal research work (6% in 2016 and 0% in 2017) which is overseen by the Institutional
Animal Care and Use Committee (IACUC) in accordance with the Animal Welfare Act
(Code, U.S., 1966).
3.3. Handling of microorganisms (Questions 3 through 12)

Staff were also asked a series of questions about RG2 microorganisms they handle in the
laboratory and the frequency with which they worked with those microorganisms. As
described in the methods section, the RG2 organisms were arranged into three substrata:
High (select agents and known etiologic agents of LAI), Moderate (etiologic agents of FI
and HAI) and Low (remaining organisms). Results reported in Fig. 2 show that while 12
Author Manuscript
(2016) and 17 (2017) persons work with high risk organisms, only a small portion of those
staff work with these organisms at least weekly (3 and 4 in 2016 and 2017, respectively).
Moderate risk organisms were handled by 21 (2016) and 27 (2017) staff and almost all 21
(2016) and 23 (2017) work with those pathogens at least weekly. Low risk organisms were
handled by 2 and 3 staff in 2016 and 2017, respectively at least weekly. There were no
statistically significant changes in the percent of staff self-reporting at least weekly work

with any risk group of pathogens from one year to another using a two-tailed, two-sample
Author Manuscript
equal variance t-test.
3.4. Equipment (Question 13)

Respondents were given a list of 25 types of equipment and were asked to identify the
frequency at which they used that type of equipment. In addition, they were asked to
categorize the likelihood of an error occurring and the likelihood of that error leading to an
injury or exposure. The responses were weighted and calculated as detailed in the Section
2.2.1. Three-Dimensional Risk Analysis. Results are reported as single and combined risk
maps using relative risk scores in Fig. 3. The top five perceived risks in 2016 (Fig. 3A) were
1. Freezers, 2. Waterbaths, 3. Electrical, 4. Needles and 5. Heat. In 2017 (Fig. 3B) they were
1. Machinery, 2. Compressed Gas, 3. Glass, 4. Electrical and 5. Needles. Further analysis
demonstrated that the largest increases in perceived risk involved autoclaves (Fig. 3C). The
Author Manuscript
risk map for autoclaves illustrated that the increased perceived risk was due to an increase in
both the reported frequency of usage and frequency of error (Fig. 3D).
3.5. Process (Question 17)

Respondents were given a list of 14 laboratory processes and were asked to identify the
frequency at which they performed those processes. In addition, they were asked to rank the
likelihood of an error occurring and the likelihood of that error leading to an injury or
exposure. The responses were weighted and scored as detailed in the Section 2.2.1. Three-
Dimensional Risk Analysis. Results are reported as single and combined risk maps using
relative risk scores in Fig. 4. The top five perceived risks in 2016 (Fig. 4A) were 1.
Explosion, 2. Repetitive Motion, 3. Spills, 4. > 500 ml Cultures and 5. Aerosol. In 2017
(Fig. 4B) they were 1. Spills, 2. Repetitive Motion, 3. Explosion, 4. Aerosol and 5.
Author Manuscript
Biological Inactivation. Further analysis demonstrated that the largest increases in perceived
risk involved biological inactivation (Fig. 4C). The risk map for biological inactivation
illustrated that the increased perceived risk was due to an increase in both the reported
frequency of usage and severity (Fig. 4D).
3.6. Mitigation (Question 12, 14 and 18)

Respondents were asked to indicate the different mitigation strategies they use to control risk
in the laboratory. Fig. 5 presents the self-reported data showing that in 2016, 96% of staff
used administrative controls (e.g. training, procedures) while only 83% used engineering
controls. For PPE, 100% of staff used body and hand PPE but only 31% used eye PPE. All
respondents (100%) had been vaccinated previously for Hepatitis B. In 2017, 100% of staff
used administrative and engineering controls. PPE use for body and hands remained at 100%
Author Manuscript
while eye PPE usage increased to 97%. All respondents (100%) again reported that they had
been previously vaccinated for Hepatitis B. Increases in the use of engineering controls (e.g.
biosafety cabinet, chemical fume hood) and eye and face PPE were significant by two-tailed,
two-sample equal variance t-test (p < 0.05). Respiratory protection is not necessary for most
of our procedures and the percentage of use remained essentially the same from 2016 (37%)
to 2017 (40%).

3.7. LQMS indicators (Questions 15, 16 and 19)

Author Manuscript
Respondents were asked to gauge the frequency at which: (1) their work processes are
documented in written protocols, (2) these protocols are in a document control system and
(3) staff members are adhering to these protocols. Allowable answers were Always (100%);
Mostly (60–99%); About half of the time (40–59%); Seldom (1–39%) and Never (< 1%). As
shown in Fig. 6, the median response in 2016 corresponded to work processes always being
documented and these protocols mostly being in a document control system. Responses also
indicated that staff mostly adhered to the protocols. In 2017, the only change was a decrease
in the number of processes that are documented from always to mostly but there were no
statistically significant changes by two-tailed, two-sample equal variance t-test.
3.8. Safety culture (Questions 20, 21 and 22)

Respondents were asked to categorize the frequency of occurrence of unsafe behaviors
Author Manuscript
(disorganized work spaces, inattention, misuse of PPE, non-aseptic practices, lack of

foresight, intentional risk taking, failure to report incidents) and workplace issues (rushed
staff, facility issues, lack of resources, equipment issues, lack of administrative controls) that
could contribute to the probability of a safety incident (data not shown). Allowable answers
were “Daily, Weekly, Monthly or Yearly”. None of the median responses corresponded to
“Daily” or “Weekly” Disorganized work spaces and rushed staff had median responses that
corresponded to “Monthly” for both 2016 and 2017. The median response for all other
unsafe behaviors or workplace issues corresponded to “Yearly”.
We also elicited opinions on the perceived priority of safety at various institutional

management levels. Allowable answers included “Safety is a top priority; Safety is low
priority; Safety is not considered; or Who is this group?”. The results were aggregated into
Author Manuscript
levels corresponding to non-supervising staff (Level 1) through the various levels of

supervisory management (Level 2, 3, 4 and 5). For all levels, the median response indicated
that safety is a top priority (data not shown). There were no significant changes from 2016 to
2017 by two-tailed, two-sample equal variance t-test for either of these questions.
Lastly, we asked for staff perspectives about how frequently they would expect that various
incidents would be reported (Absolutely would report (100%); Most likely (70–99%); It
could go either way (30–69%); Doubtful (1–29%); Would not report (0%)). For both 2016
and 2017, the median response corresponded to absolutely would report for any probable
exposure, possible exposure, release or a major injury. In 2016, respondents indicated that
they absolutely would report a minor injury but this value dropped to most likely would
report in 2017. For near misses, both 2016 and 2017 median responses corresponded to “It
could go either way”. Only the median responses for minor injury reporting were
Author Manuscript
significantly decreased (p > 0.05) from 2016 to 2017 by two-tailed, two-sample equal
variance t-test (Fig. 7).
4. Discussion
Designing an effective laboratory safety program requires data to formulate safety
interventions that are evidence-based (Cote et al., 2016; Yarahmadi et al., 2016; Smith and

Morrato, 2014; Kimman et al., 2008; Birnbaum et al., 2016), but data on laboratory safety
Author Manuscript
are limited in various ways. Limitations of published safety data include only being relevant
to a specific pathogen (Leunda et al., 2013; Tyshenko et al., 2011; Rozell, 2015; Wagar,
2016; Li et al., 2012; Pedrosa and Cardoso, 2011; Le Duc and Franz, 2012), a single type of
laboratory (Shurtleff et al., 2012; Dickmann et al., 2015; Elduma, 2012; Higgins et al., 2013)
or a single laboratory process (Ahlin and Weiss, 2007; Burke, 1993; Serafini et al., 2016;
Wedum, 1964; Gillespie and Gibbons, 1975; Nimunkar et al., 2017). These lessons learned
are difficult to apply in public health laboratories due to the complexity of working with a
large number of microorganisms and performing a breadth of testing services using a variety
of laboratory equipment and processes while fully complying with numerous standards,
regulations and guidelines. More comprehensive and standardized strategies are needed to
effectively integrate evidence-based safety with quality management systems in public
health laboratories (Pedrosa and Cardoso, 2011; Salerno and Gaudioso, 2015; Westgard,
Author Manuscript
2013; Person, 2013; Jairaman et al., 2017; Janssens, 2014; Lentz et al., 2015). We have
presented results that incorporated the expert opinion of our laboratory staff to address risk
and create a laboratory safety-specific continuous improvement cycle to inform laboratory
safety interventions.
We chose to begin the CQI cycle by collecting expert opinions about laboratory-specific
hazards through a survey of staff members for two reasons. First, it is the laboratory staff
who have the most expertise and experience in laboratory activities and the safety in the
laboratory ultimately rests with them. Second, understanding how our staff perceive risks is
critical to implementing and maintaining effective laboratory safety programs (Xu et al.,
2014; Robertson et al., 2015; Kleiner et al., 2015; Schulte et al., 2012; Chung et al., 2015).
The survey was designed to improve our understanding of risk in the laboratory setting.
First, although all microorganisms are handled in our biosafety level 2 (BSL2) laboratory
Author Manuscript
(Charles and Nixon, 2019) and are RG2 agents, some of our microorganisms are considered
to be more pathogenic or transmissible than other RG2 microorganisms. While biosafety
levels are invaluable benchmarks for basic laboratory safety (World Health Organization.,
2004; Chosewood and Wilson, 2009), they are, by necessity, very general and should mark
the beginning of a more rigorous assessment of microorganism risk (Gravina et al., 2019;
Buxton et al., 2011). To seek more precision in how we managed biological risk, we sought
to stratify our RG2 microorganisms into high, moderate and low risk categories. Second, we
also utilized skip logic in the survey to ascertain the most pathogenic microorganisms that
each staff member works with at least weekly. This enabled us to obtain frequency and
severity data to better define our microorganism-associated risks and provided the biological
context for respondents to use when responding to the remainder of the survey (Fig. 2).
Author Manuscript
The CQI cycle continued with a gap analysis to identify and prioritize risks. We chose a
three-dimensional risk analysis method using data on the probability of error, severity of
error and likelihood of exposure, as opposed to the traditional two-dimensional matrices
(probability and severity of error) (Manuele, 2001). This provided a method to identify risks
at a more granular level and to better understand how staff in our laboratories perceive
equipment (Fig. 3) and process (Fig. 4) risks. The output of this method is a risk map that
offers a graphical presentation of risk by probability, severity and exposure. Graphical risk
communications have been shown to contribute to successful safety interventions (Severtson

and Henriques, 2009). This approach to hazard identification, risk analysis and risk
Author Manuscript
communication was successful in raising awareness of laboratory safety amongst our staff
and engaging them in these processes.
Understanding and prioritizing the risks leads to the next step in the CQI cycle – the design
of mitigation measures. For example, in 2016, we identified the following as areas for
targeted intervention:
• Potential underutilization of engineering controls (Fig. 5)
• Potential underreporting of near-misses (Fig. 7)
• Potentially low awareness of process hazards (Fig. 4)
To address engineering controls, we performed refresher training for all staff on biosafety
cabinet usage. We also developed and implemented a laboratory-specific risk assessment
Author Manuscript
procedure, which was used to expand and tailor the safety section of multiple laboratory
procedures to raise awareness of hazards specific to those activities. Lastly, we implemented
a new laboratory incident response and reporting notification program to address near miss
reporting.
To close out the CQI cycle, we elected to utilize the data from the subsequent annual survey
to evaluate the effects of the interventions. In 2017, engineering control use (Fig. 5) and
process awareness was increased (Fig. 4) which was consistent with successful interventions
in this area. In both cases, institutional efforts in laboratory safety and risk assessment
training occurred in the same timeframe and most likely amplified the effectiveness of our
efforts. Interestingly, responses indicated that staff were less likely to report minor injuries in
2017 (Fig. 6) despite issuance of both institutional and laboratory-specific policies regarding
Author Manuscript
incident response and notification. It has been shown in other laboratory settings that
additional restrictions and requirements on laboratory activities are not always beneficial
(Shurtleff et al., 2012) so it is important to determine if the incident notification intervention
– as designed – should be reevaluated. The collection of data on an annual basis also allows
evaluation of interventions that are generated from outside this process. For example, in
2017 we observed a significant increase in eye PPE (Fig. 5) due most likely to a new
institutional policy requiring the use of safety glasses in all laboratories. Evaluating the
efficacy of interventions using data from a subsequent survey is an inferential process. The
data we collect is non-random and any changes that occur after an intervention may not
necessarily be caused by the intervention. However, the evaluation of data on an annual basis
allows us to infer how perceptions of laboratory safety are changing which restarts the CQI
cycle by identifying new hazards and risks to address.
Author Manuscript
We have centralized requests for IBB, IBC, LSRB, IACUC and IRB protocol approvals
which allows us to compare the data we collected about external oversight to what our
records show indicating that staff are aware of these oversight requirements (Fig. 1). We also
collected data consistent with our current understanding of barriers to safety. For example, a
major barrier to staff compliance with laboratory safety directives is lack of time (Shakoor et
al., 2016). Our data is consistent with this as rushed laboratory staff, along with disorganized

laboratory areas and facility issues, were the most frequently reported workplace conditions
Author Manuscript
that could potentially lead to a safety incident.
There are obvious limitations to this method of data collection and analysis. We observed a
potential for self-reporting bias as 100% of staff reported using administrative controls,
while also reporting that only 60–99% of other staff members were doing the same. Future
surveys will use a new scale with smaller ranges for this answer to try to detect the extent of
this bias (i.e. was the reported 60–99% of staff who use administrative controls actually
closer to 60% or closer to 99%). In addition, inferring efficacy of interventions is also
limited as there is no way to prove that any change in responses is due to a targeted
intervention as opposed to be due to changes in the workforce, laboratory goals or external
directives.
Successful interventions to improve safety include training (Coelho and García Díez, 2015;
Author Manuscript
Olson et al., 2009; Pallozzi et al., 2003) and the establishment of core competencies for
biosafety (Chamberlain et al., 2009). We utilized training interventions with success in
significantly increasing the reported use of engineering controls (Fig. 5) and raising the level
of awareness regarding laboratory process risks among laboratory staff (Fig. 4). We also
implemented a knowledge transfer assessment after safety training, which allowed additional
trainings and interventions to identify and address gaps in trainer expertise as well as staff
knowledge. It is vital that public health laboratories establish and maintain high safety
standards. However, the diversity and complexity of the work performed in public health
laboratories makes it difficult to maintain, monitor and continuously improve laboratory
safety management. The tools and observations described here can be used to enhance
engagement of expert staff and provide a means of generating practical evidence-based
safety interventions within a Quality Management framework.
Author Manuscript
Supplementary Material
Refer to Web version on PubMed Central for supplementary material.
Acknowledgements
This work was supported by the Centers for Disease Control and Prevention, Office of the Associate Director for
Laboratory Science and Safety [Laboratory Safety Science and Innovation (LaSSI) Intramural Research Fund]. We
would also like to acknowledge the Laboratory Leadership Service Fellowship program and specifically, Ren
Salerno, for providing invaluable training in the area of risk assessment. Lastly, we wish to acknowledge the
participation of our laboratory staff in the development of these tools.
The findings and conclusions in this report are those of the authors and do not necessarily represent the official
position of CDC.
Author Manuscript
References
29 USC, 1910, 2018 29 USC 1910. Occupational Safety and Health Standards for Laboratories
Department of Health and Human Services, Editor.
42 CFR 493, 2018 42 CFR 493 - Standards and Certification: Laboratory Requirements Department of
Health and Human Services, Editor.
Ahlin PA, Weiss RL, 2007 Risk management and compliance in pathology and laboratory medicine.
Clin. Lab. Med 27 (4), 859–873 vii. [PubMed: 17950902]

Allen LC, 2013 Role of a quality management system in improving patient safety - laboratory aspects.
Clin. Biochem 46 (13–14), 1187–1193. [PubMed: 23648455]
Author Manuscript
Association of State and Territorial Public Health Laboratory Directors (ASTPHLD), 1997 Regulatory
closure of cervical cytology laboratories: recommendations for a public health response. MMWR
Recomm. Rep 46 (Rr-17), 1–19.
Baron EJ, Miller JM, 2008 Bacterial and fungal infections among diagnostic laboratory workers:
evaluating the risks. Diagn. Microbiol. Infect. Dis 60 (3), 241–246. [PubMed: 17997259]
Beck AJ, Boulton ML, Coronado F, 2014 Enumeration of the governmental public health workforce,
2014. Am. J. Prev. Med 47 (5 Suppl 3), S306–S313. [PubMed: 25439250]
Birnbaum LS, Burke TA, Jones JJ, 2016 Informing 21st-century risk assessments with 21st-century
science. Environ. Health Perspect 124 (4), A60–A63. [PubMed: 27035154]
Blaine JW, 2012 Establishing a national biological laboratory safety and security monitoring program.
Biosecur. Bioterror 10 (4), 396–400. [PubMed: 23210833]
Blaser MJ, Lofgren JP, 1981 Fatal salmonellosis originating in a clinical microbiology laboratory. J.
Clin. Microbiol 13 (5), 855–858. [PubMed: 7240398]
Burke RL, 1993 Managing risk under CLIA (Clinical Laboratory Improvement Amendments). Med.
Author Manuscript
Group Manage. J 40 (3), 30–34.

Buxton JA, et al., 2011 Using qualitative methods to investigate risk perception of Canadian medical
laboratory workers in relation to current prion disease infection control policies. J. Toxicol.
Environ. Health A 74 (2–4), 241–247. [PubMed: 21218349]
Campbell MJ, et al., 2015 Characterizing accidents, exposures, and laboratory-acquired infections
reported to the National Institutes of Health’s Office of Biotechnology Activities (NIH/OBA)
involving recombinant DNA biological agents. Dissertation Abstr. Int.: Section B: Sci. Eng
Centers for Disease Control and Prevention, 2014 CDC Report on the Potential Exposure to Anthrax.
Int. Microbiol, 17(2), pp. 119–127. [PubMed: 26418856]
Chamberlain AT, et al., 2009 Biosafety training and incident-reporting practices in the United States: a
2008 survey of biosafety professionals. Appl. Biosaf.: J. Am. Biol. Saf. Assoc 14 (3), 135–143.
Charles RL, Nixon J, 2019 Measuring mental workload using physiological measures: A systematic
review. Appl. Ergon 74, 221–232. [PubMed: 30487103]
Chosewood LC, Wilson DE, 2009 Biosafety in microbiological and biomedical laboratories 5th ed.
Author Manuscript
HHS publication. U.S. Dept. of Health and Human Services, Public Health Service, Centers for
Disease Control and Prevention, National Institutes of Health, Washington, D.C. xxii, 415 p.
Chung D, et al., 2015 Social signals of safety and risk confer utility and have asymmetric effects on
observers’ choices. Nat. Neurosci 18 (6), 912–916. [PubMed: 25984890]
Code of Federal Regulations, 2018 45 CFR 46 - Protection of Human Subjects (Common Rule)
Code, U.S., 1966 7 U.S.C. § 2131 Animal Welfare Act (AWA)
Coelho AC, García Díez J, 2015 Biological risks and laboratory-acquired infections: a reality that
cannot be ignored in health biotechnology. Front. Bioeng. Biotechnol 3, 56. [PubMed: 25973418]
Cohen J, 2014 Lab safety. Alarm over biosafety blunders. Science 345 (6194), 247–248. [PubMed:
25035464]
Cook EB, 1961 Safety in the public health laboratory. Public Health Rep 76, 51–56. [PubMed:
13695240]
Cote I, et al., 2016 The next generation of risk assessment multi-year study—highlights of findings,
applications to risk assessment, and future directions. Environ. Health Perspect 124 (11), 1671–
Author Manuscript
1682. [PubMed: 27091369]

Cram RS, Anadarko Algeria Corporation LLC, 2004 3D model for qualitative risk assessment. In:
Seventh SPE International Conference on Health, Safety and Environment in Oil and Gas
Exploration and Production Society of Petroleum Engineers Inc, Calgary, Alberta, Canada.
Dickmann P, Sheeley H, Lightfoot N, 2015 Biosafety and biosecurity: a relative risk-based framework
for safer, more secure, and sustainable laboratory capacity building. Front. Public Health 3, 241.
[PubMed: 26539427]
Dirnagl U, et al., 2016 A laboratory critical incident and error reporting system for experimental
biomedicine. PLoS Biol 14 (12), e2000705. [PubMed: 27906976]

Elduma AH, 2012 Assessment of biosafety precautions in Khartoum state diagnostic laboratories,
Sudan. Pan Afr. Med. J 11, 19. [PubMed: 22514753]
Author Manuscript
Fasanya BK, Shofoluwe M, 2019 Occupational ergonomics: emerging approaches toward improved
worker productivity and injury reduction. In: In: Goonetilleke R, Karwowski W (Eds.), Advances
in Physical Ergonomics & Human Factors. AHFE 2018. Advances in Intelligent Systems and
Computing, vol. 789 Springer, Cham.
Federal Register, 2014 79 FR 57589 Policy for Institutional Oversight of Life Sciences Dual Use
Research of Concern U.S. Government, Editor.
Federal Register, 2016 81 FR 22286 NIH Guidelines for Research Involving Recombinant or Synthetic
Nucleic Acid Molecules (NIH Guidelines), N.I.H. (U.S.), Editor.
Fleck F, 2004 SARS outbreak over, but concerns for lab safety remain. Bull. World Health Organ 82
(6), 470.
Fuscaldo AA, Erlick BJ, Hindman B, 1980 Laboratory safety: theory and practice Academic Press,
New York xiv, 357 p.
Gillespie EH, Gibbons SA, 1975 Proceedings: Laboratory autoclaves–dangers and safety. J. Clin.
Pathol 28 (9), 754.
Author Manuscript
Gravina NE, King A, Austin J, 2019 Training leaders to apply behavioral concepts to improve safety.
Safety Sci 112, 66–70.
Hawkes N, 1979 Smallpox death in Britain challenges presumption of laboratory safety. Science 203
(4383), 855–856. [PubMed: 419409]
Herrmann-Werner A, et al., 2013 Best practice“ skills lab training vs. a ”see one, do one“ approach in
undergraduate medical education: an RCT on students’ long-term ability to perform procedural
clinical skills. PLoS One 8 (9), e76354. [PubMed: 24086732]
Higgins JJ, et al., 2013 Implementation of a personnel reliability program as a facilitator of biosafety
and biosecurity culture in BSL-3 and BSL-4 laboratories. Biosec. Bioterrorism: Biodefense
Strategy, Pract., Sci 11 (2), 130–137.
International Organization for Standardization, 2003 ISO 15190:2003 Medical Laboratories -
Requirements for Safety
International Organization for Standardization, 2012 ISO 15189:2012 Medical Laboratories -
Requirements for Quality and Competence
Author Manuscript
International Organization for Standardization, 2017 ISO 17025:2017 General Requirements for the
Competence of Testing and Calibration Laboratories
International Organization for Standardization, 2018 ISO 45001:2018 Occupational Health and Safety
Jairaman J, Sakiman Z, Li LS, 2017 Sunway medical laboratory quality control plans based on six
sigma, risk management and uncertainty. Clin. Lab. Med 37 (1), 163–176. [PubMed: 28153364]
Janssens PM, 2014 Practical, transparent prospective risk analysis for the clinical laboratory. Ann.
Clin. Biochem 51 (Pt 6), 695–704. [PubMed: 24553437]
Jong-Hyun L, et al., 2018 The effects of personality types on self-reported safety behavior: Focused on
plant workers in Korea. Accid. Anal. Prev 121, 20–27. [PubMed: 30205283]
Kimman TG, Smit E, Klein MR, 2008 Evidence-based biosafety: a review of the principles and
effectiveness of microbiological containment measures. Clin. Microbiol. Rev 21 (3), 403–425.
[PubMed: 18625678]
Kleiner BM, et al., 2015 Sociotechnical attributes of safe and unsafe work systems. Ergonomics 58 (4),
635–649. [PubMed: 25909756]
Author Manuscript
Le Duc JW, Franz DR, 2012 Genetically engineered transmissible influenza A/H5N1: a call for
laboratory safety and security. Biosec. Bioterrorism 10 (1), 153–154.
Lentz TJ, et al., 2015 Aggregate exposure and cumulative risk assessment—integrating occupational
and non-occupational risk factors. J. Occup. Environ. Hygiene 12 (sup1), S112–S126.
Leunda A, et al., 2013 Laboratory activities involving transmissible spongiform encephalopathy
causing agents: risk assessment and biosafety recommendations in Belgium. Prion 7 (5), 420–433.
[PubMed: 24055928]
Li Z, et al., 2012 Aerosolized avian influenza virus by laboratory manipulations. Virology J 9 p. 146–
146. [PubMed: 22866888]

Lippi G, Guidi GC, 2007 Risk management in the preanalytical phase of laboratory testing. Clin.
Chem. Lab. Med 45 (6), 720–727. [PubMed: 17579523]
Author Manuscript
Lord JT, 1990 Risk management in pathology and laboratory medicine. Arch. Pathol. Lab. Med 114
(11), 1164–1167. [PubMed: 2241529]
Magill SS, Edwards JR, Fridkin SK, 2014 Survey of health care-associated infections. N. Engl. J. Med
370 (26), 2542–2543.
Manuele FA, 2001 Innovations in Safety Management: Addressing Career Knowledge Needs Wiley,
New York xiv, 251 p.
McCarthy M, 2014 Biosafety lapses prompt US CDC to shut labs and launch review. BMJ 349, g4615.
[PubMed: 25022658]
Miller JM, et al., 2012 Guidelines for safe work practices in human and animal medical diagnostic
laboratories. Recommendations of a CDC-convened, Biosafety Blue Ribbon Panel. MMWR Suppl
61 (1), 1–102. [PubMed: 22217667]
Miller CD, Songer JR, Sullivan JF, 1987 A twenty-five year review of laboratory-acquired human
infections at the National Animal Disease Center. Am. Ind. Hyg. Assoc. J 48 (3), 271–275.
[PubMed: 3578038]
Author Manuscript
Moskowitz S, 1948 Safety for laboratory technicians. Mon. Rev. New York State Dep. Labor. Div. Ind.
Hyg 27 (8), 29 passim.
Ned-Sykes R, et al., 2015 Competency guidelines for public health laboratory professionals: CDC and
the association of public health laboratories. MMWR Suppl 64 (1), 1–81.
Nichols JH, 2011 Laboratory quality control based on risk management. Ann. Saudi Med 31 (3), 223–
228. [PubMed: 21623049]
Nimunkar AJ, et al., 2017 Reducing thumb extensor risk in laboratory rat gavage. Appl. Ergon 58,
151–155. [PubMed: 27633208]
Njoroge SW, Nichols JH, 2014 Risk management in the clinical laboratory. Ann. Lab. Med 34 (4),
274–278. [PubMed: 24982831]
Occupational Health and Safety Administration, 2011 Laboratory Safety Guidance
Olson R, et al., 2009 Observational learning and workplace safety: the effects of viewing the collective
behavior of multiple social models on the use of personal protective equipment. J. Safety Res 40
(5), 383–387. [PubMed: 19932320]
Author Manuscript
Pallozzi WA, et al., 2003 Novel training concepts and techniques used to increase safety awareness in
the laboratory animal facility. Lab. Anim. (NY) 32 (6), 38–42.
7 CFR Part 331, 9 CFR Part 121, and 42 CFR Part 73 - Select Agent Regulations, 2018 Department of
Health and Human Services and the United States Department of Agriculture, Editor.
Pedrosa PB, Cardoso TA, 2011 Viral infections in workers in hospital and research laboratory settings:
a comparative review of infection modes and respective biosafety aspects. Int. J. Infect. Dis 15 (6),
e366–e376. [PubMed: 21497126]
Person NB, 2013 Developing risk-based quality control plans: an overview of CLSI EP23-A. Clin.
Lab. Med 33 (1), 15–26. [PubMed: 23331726]
Pike RM, 1976 Laboratory-associated infections: summary and analysis of 3921 cases. Health Lab. Sci
13 (2), 105–114. [PubMed: 946794]
Pike RM, Sulkin SE, 1951 Laboratory-acquired infections in the United States. Tex Rep. Biol. Med 9
(2), 346–347. [PubMed: 14835469]
Powell MR, 2016 Trends in Reported Foodborne Illness in the United States; 1996–2013. Risk Anal
Author Manuscript
36 (8), 1589–1598. [PubMed: 26709453]

Richmond JY, Nesby-O’Dell SL, 2002 Laboratory security and emergency response guidance for
laboratories working with select agents. Centers for Disease Control and Prevention. MMWR
Recomm. Rep 51 (Rr-19), 1–6.
Robertson MM, et al., 2015 Sociotechnical approaches to workplace safety: Research needs and
opportunities. Ergonomics 58 (4), 650–658. [PubMed: 25728246]
Rozell DJ, 2015 Assessing and managing the risks of potential pandemic pathogen research. mBio 6
(4), e01075–e1115. [PubMed: 26199335]

Salerno RM, Gaudioso JM, 2015 Laboratory Biorisk Management: Biosafety and Biosecurity CRC
Press, Taylor & Francis Group, Boca Raton xxii, 242 pages.
Author Manuscript
Schulte PA, et al., 2012 Interaction of occupational and personal risk factors in workforce health and
safety. Am. J. Public Health 102 (3), 434–448. [PubMed: 22021293]
Sciacovelli L, et al., 2007 Risk management in laboratory medicine: quality assurance programs and
professional competence. Clin. Chem. Lab. Med 45 (6), 756–765. [PubMed: 17579529]
Senthil A, et al., 2015 Perception and prevalence of work-related health hazards among health care
workers in public health facilities in southern India. Int. J. Occup. Environ. Health 21 (1), 74–81.
[PubMed: 25482656]
Serafini A, et al., 2016 Use of a systematic risk analysis method (FMECA) to improve quality in a
clinical laboratory procedure. Ann. Ig 28 (4), 288–295. [PubMed: 27479765]
Severtson DJ, Henriques JB, 2009 The effect of graphics on environmental health risk beliefs,
emotions, behavioral intentions and recall. Risk Anal.: Off. Publ. Soc. Risk Anal 29 (11), 1549–
1565.
Shakoor S, et al., 2016 Barriers to implementation of optimal laboratory biosafety practices in
Pakistan. Health Security 14 (4), 214–219. [PubMed: 27400192]
Author Manuscript
Shurtleff AC, et al., 2012 The impact of regulations, safety considerations and physical limitations on
research progress at maximum biocontainment. Viruses 4 (12), 3932–3951. [PubMed: 23342380]
Silver S, 2015 Laboratory-acquired lethal infections by potential bioweapons pathogens including
Ebola in 2014. FEMS Microbiol. Lett 362 (1), 1–6.
Singh K, 2009 Laboratory-acquired infections. Clin. Infect. Dis 49 (1), 142–147. [PubMed: 19480580]
Smith MY, Morrato E, 2014 Advancing the field of pharmaceutical risk minimization through
application of implementation science best practices. Drug Saf 37 (8), 569–580. [PubMed:
25005707]
Steelman ED, Alexander JL, 2016 Laboratory animal workers’ attitudes and perceptions concerning
occupational risk and injury. J. Am. Assoc. Lab. Anim. Sci 55 (4), 419–425. [PubMed: 27423148]
Streiner DL, 2003 Starting at the beginning: an introduction to coefficient alpha and internal
consistency. J. Pers. Assess 80 (1), 99–103. [PubMed: 12584072]
Sullivan JF, Songer JR, Estrem IE, 1978 Laboratory-acquired infections at the National Animal
Disease Center 1960–1976. Health Lab. Sci 15 (1), 58–64. [PubMed: 565343]
Author Manuscript
Tyshenko MG, et al., 2011 Expert elicitation for the judgment of prion disease risk uncertainties. J.
Toxicol. Environ. Health A 74 (2–4), 261–285. [PubMed: 21218351]
Tziaferi SG, et al., 2011 Risk assessment of physical hazards in greek hospitals combining staff’s
perception, experts’ evaluation and objective measurements. Safety Health Work 2 (3), 260–272.
United States Code, 1988 42 U.S.C. § 263a - Clinical Laboratory Improvement Amendments
Van Noorden R, 2013 Safety survey reveals lab risks. Nature 493 (7430), 9–10. [PubMed: 23282341]
Vesley D, Hartmann HM, 1988 Laboratory-acquired infections and injuries in clinical laboratories: a
1986 survey. Am. J. Public Health 78 (9), 1213–1215. [PubMed: 3407823]
Wagar E, 2016 Bioterrorism and the role of the clinical microbiology laboratory. Clin. Microbiol. Rev
29 (1), 175–189. [PubMed: 26656673]
Ward PR, 2017 Improving access to, use of, and outcomes from public health programs: the
importance of building and maintaining trust with patients/clients. Front. Public Health 5, 22.
[PubMed: 28337430]
Wedum AG, 1964 Laboratory safety in research with infectious aerosols. Public Health Rep 79, 619–
Author Manuscript
633. [PubMed: 14177793]

Westgard JO, 2013 Perspectives on quality control, risk management, and analytical quality
management. Clin. Lab. Med 33 (1), 1–14. [PubMed: 23331725]
Westgard SA, 2017 Rhetoric versus reality? Laboratory surveys show actual practice differs
considerably from proposed models and mandated calculations. Clin. Lab. Med 37 (1), 35–45.
[PubMed: 28153368]
World Health Organization, 2004 Laboratory biosafety manual, 3rd ed. World Health Organization,
Geneva viii, 178 p.

Wurtz N, et al., 2016 Survey of laboratory-acquired infections around the world in biosafety level 3
and 4 laboratories. Eur. J. Clin. Microbiol. Infect. Dis 35 (8), 1247–1258. [PubMed: 27234593]
Author Manuscript
Xia N, et al., 2017 Do we see how they perceive risk? An integrated analysis of risk perception and its
effect on workplace safety behavior. Accid. Anal. Prev 106, 234–242. [PubMed: 28645020]
Xu Y, et al., 2014 Controlled versus automatic processes: which is dominant to safety? The moderating
effect of inhibitory control. PLoS ONE 9 (2), e87881. [PubMed: 24520338]
Yarahmadi R, Moridi P, Roumiani Y, 2016 Health, safety and environmental risk management in
laboratory fields. Med. J. Islam Repub. Iran 30, 343. [PubMed: 27284544]
Author Manuscript
Author Manuscript
Author Manuscript

Author Manuscript
Author Manuscript
Fig. 1.
Work requiring external oversight. Total percent of staff who self-report that their work
requires external oversight by Clinical Laboratory Improvement Amendments (CLIA),
Institutional Biosafety Committee (IBC), Institutional Biosecurity Board (IBB), Institutional
Review Board (IRB), Institutional Animal Care and Use Committee (IACUC), and
Laboratory Safety Review Board (LSRB) for 2016 (black) and 2017 (grey).
Author Manuscript
Author Manuscript

Author Manuscript
Author Manuscript
Fig. 2.
Work with pathogens. Total percent of staff self-reporting work with high (black columns),
moderate (white columns) and low risk organisms (grey columns) with number of persons
working with high risk organisms (black diamonds) and moderate risk organisms (white
squares) at least weekly for 2016 (left) and 2017 (right).
Author Manuscript
Author Manuscript

Author Manuscript
Author Manuscript
Author Manuscript
Fig. 3.
Equipment risks. Types of laboratory equipment assessed for relative risk for 2016 (A) and
2017 (B) with asterisks denoting the top five perceived risks. Changes in perception (C) and
root causes of these perception changes from 2016 (black) and 2017 (gray) (D) are shown
with asterisks denoting the greatest change in perceived risk (C) and the primary root cause
(D).
Author Manuscript

Author Manuscript
Author Manuscript
Author Manuscript
Fig. 4.
Process risks. Types of laboratory processes assessed for relative risk for 2016 (A) and 2017
(B) with asterisks denoting the top five perceived process risks. Changes in perception (C)
and root causes of these perception changes from 2016 (black) and 2017 (gray) (D) are
shown with asterisks denoting the greatest change in perceived risk (C) and the primary root
cause (D).
Author Manuscript

Author Manuscript
Author Manuscript
Fig. 5.
Use of risk controls. Total percent of staff that handle pathogens and report the use of risk
controls of various types for 2016 (black) and 2017 (gray). Brackets and p-values indicate
significant increases in self-reported use of engineering controls and eye and face PPE from
2016 to 2017 based on a two-tailed, two sample equal variance t-test.
Author Manuscript
Author Manuscript

Author Manuscript
Author Manuscript
Fig. 6.
LQMS indicators. Bar graph presentation of the median response to questions about the
frequency of work being covered by a written protocol (black), frequency of protocols being
in a document control system (white) and frequency with which staff adhere to protocols
(grey) for 2016 (top) and 2017 (bottom). Np statistically significant changes were observed.
Author Manuscript
Author Manuscript

Author Manuscript
Author Manuscript
Fig. 7.
Safety culture for reporting incidents. Responses regarding staff likeliness to report various
Author Manuscript
events for 2016 (top) and 2017 (bottom). For each scenario, the percent of responses
corresponding with each answer are shown. Absolutely Report (100%) (black); Most Likely
(70–99%) (dark gray); Either Way (30–69%) (medium gray); Doubtful (1–29%) (light gray);
and Absolutely No Report (0%) (white). Only minor injury reporting changes from 2016 to
2017 were significant (p > 0.05) by two-tailed, two-sample equal variance t-test.
Author Manuscript

Table 1
Internal consistency reliability (coefficient alpha).

Author Manuscript
Scale name Questions Cronbach’s alpha

Regulatory Compliance Frequency Scale Q2 0.900
Biologics Frequency Scale Q5, Q7 and Q9 0.889
Frequency of Use Scale Q13 and Q17 0.786
Frequency of Error Scale Q13 and Q17 0.875
Severity of Consequences Scale Q13 and Q17 0.778
Author Manuscript
Author Manuscript
Author Manuscript

Table 2
Sample population demographics (n = 36).

Author Manuscript
Demographic Category Total Percent

Age Group < 24 0 0.0%
25–34 18 51.4%
35–44 8 22.9%
45–54 6 17.1%
> 54 3 8.6%
Highest Education Level High School 0 0.0%
Some College 0 0.0%
Associates 0 0.0%
Bachelors 8 22.2%
Masters 16 44.4%
Author Manuscript
Doctorate 12 33.3%
Management Level Supervisory 8 22.2%
Non-Supervisory 28 77.8%
Public Health Laboratory Experience <1 7 20.0%
1–3 8 22.9%
4–5 6 17.1%
6–10 7 20.0%
> 10 7 20.0%
Non-Public Health Laboratory Experience <1 2 7.4%
1–3 6 22.2%
4–5 5 18.5%
6–10 10 37.0%
Author Manuscript
> 10 4 14.8%
Hepatitis B Vaccination Yes 36 100.00%
No 0 0.0%
Previous Laboratory Injury Yes 8 22.9%
No 27 77.1%
Type of Public Health Experience None 9 25.0%
Federal 25 69.4%
State 2 5.6%
Local 0 0.0%
Type of Non-Public Health Laboratory Experience None 5 11.6%
Academia 20 46.5%
Clinical 9 20.9%
Author Manuscript
Private 9 20.9%

CDC 79046 DS1

Uploaded by

Copyright:

Available Formats

CDC 79046 DS1

Uploaded by

Document Information

Original Title

Copyright

Available Formats

Share this document

Share or Embed Document

Sharing Options

Did you find this document useful?

Is this content inappropriate?

Copyright:

Available Formats

CDC 79046 DS1

Uploaded by

Copyright:

Available Formats

HHS Public Access

Published in final edited form as:

Development and implementation of evidence-based laboratory

Control and Prevention, 2014).

Saf Sci. Author manuscript; available in PMC 2020 August 01.

implementation of these guidelines are left to individual laboratories to develop (World

To determine a standardized process of assessing and mitigating laboratory risk, we chose to

1.1. Significant results summary

– Annual all-hazards survey tool designed to gather laboratory safety

♦ Tailored to the highest risk group pathogen handled at least

♦ Collected information on severity, probability and exposure

Saf Sci. Author manuscript; available in PMC 2020 August 01.

– Method for stratifying the risk of microorganisms handled in a BSL-2

– Three-Dimensional Risk Assessment Tool designed for analysis of

• Identified laboratory-specific risks for targeted intervention

– Potential underutilization of engineering controls

– Potential underreporting of near-misses

– Potentially low awareness of process hazards

• Measured significant differences in survey responses after interventions

– Staff training increased use of engineering controls

– Increased incident reporting burden may cause lower near-miss

– Hazard awareness was increased through presentation of survey results

2. Materials and methods

2.1.1. Survey development—Reviews of multiple guidelines for laboratory safety

toward obtaining a comprehensive picture of laboratory staff perceptions related to

Saf Sci. Author manuscript; available in PMC 2020 August 01.

concern or transfer of materials from BSL3 or BSL4 level laboratories. An additional

question collected self-reported data on compliance with these regulations (Q2).

source of a laboratory-associated infection or exposure (LAI/E) in relevant reviews of the

Saf Sci. Author manuscript; available in PMC 2020 August 01.

Section 2.2.1. Three-Dimensional Risk Analysis.

2.1.2. Safety survey reliability and validity

2.1.2.2. Internal consistency reliability (coefficient alpha).: The internal consistency

the internal reliability of the scales used in this survey.

2.1.3. Safety survey administration—The 2016 laboratory safety survey contained

Saf Sci. Author manuscript; available in PMC 2020 August 01.

2.1.4. Demographic questionnaire—To define the sample population in more detail

2.2. CQI stage two - gap analysis

2.2.1. Three-dimensional risk analysis—To standardize the approach to prioritizing

• Severity (s) - How likely it is that an error in equipment usage or process

specific piece of equipment or performing a specific process.

• Exposure (e) - How often a piece of equipment is used or a process is

Saf Sci. Author manuscript; available in PMC 2020 August 01.

Relative Risk = Severity (s) × Probability (p) × Exposure (e)

2.3. CQI stage three – mitigation measures

2.4. CQI stage four – evaluation and feedback

demographic data in a separate survey presented in Table 2. These demographics were

Saf Sci. Author manuscript; available in PMC 2020 August 01.

a laboratory incident in any setting. Responses to the demographic questionnaire are

3.2. Regulatory requirements (Questions 1 and 2)

3.3. Handling of microorganisms (Questions 3 through 12)

Saf Sci. Author manuscript; available in PMC 2020 August 01.

equal variance t-test.

3.4. Equipment (Question 13)

3.5. Process (Question 17)

3.6. Mitigation (Question 12, 14 and 18)

Saf Sci. Author manuscript; available in PMC 2020 August 01.

3.7. LQMS indicators (Questions 15, 16 and 19)

3.8. Safety culture (Questions 20, 21 and 22)