Research and applications
Reduction in medication errors in hospitals due to
adoption of computerized provider order entry
systems
David C Radley,1 Melanie R Wasserman,2 Lauren EW Olsho,2 Sarah J Shoemaker,2
Mark D Spranca,2 Bethany Bradshaw3
1
Institute for Healthcare
Improvement, Cambridge,
Massachusetts, USA
2
US Health Division, Abt
Associates Inc, Cambridge,
Massachusetts, USA
3
School of Public Health, Johns
Hopkins University, Baltimore,
Maryland, USA
Correspondence to
Dr Lauren Olsho, US Health
Division, Abt Associates, Inc,
55 Wheeler St, Cambridge,
MA 02138, USA;
lauren_olsho@abtassoc.com
Received 2 August 2012
Revised 19 December 2012
Accepted 10 January 2013
Published Online First
21 February 2013
ABSTRACT
Objective Medication errors in hospitals are common,
expensive, and sometimes harmful to patients. This
study’s objective was to derive a nationally representative
estimate of medication error reduction in hospitals
attributable to electronic prescribing through
computerized provider order entry (CPOE) systems.
Materials and methods We conducted a systematic
literature review and applied random-effects metaanalytic techniques to derive a summary estimate of the
effect of CPOE on medication errors. This pooled
estimate was combined with data from the 2006
American Society of Health-System Pharmacists Annual
Survey, the 2007 American Hospital Association Annual
Survey, and the latter’s 2008 Electronic Health Record
Adoption Database supplement to estimate the
percentage and absolute reduction in medication errors
attributable to CPOE.
Results Processing a prescription drug order through a
CPOE system decreases the likelihood of error on that
order by 48% (95% CI 41% to 55%). Given this effect
size, and the degree of CPOE adoption and use in
hospitals in 2008, we estimate a 12.5% reduction in
medication errors, or ∼17.4 million medication errors
averted in the USA in 1 year.
Discussion Our findings suggest that CPOE can
substantially reduce the frequency of medication errors in
inpatient acute-care settings; however, it is unclear
whether this translates into reduced harm for patients.
Conclusions Despite CPOE systems’ effectiveness at
preventing medication errors, adoption and use in US
hospitals remain modest. Current policies to increase
CPOE adoption and use will likely prevent millions of
additional medication errors each year. Further research
is needed to better characterize links to patient harm.
which may further reduce errors. There is also
some evidence that CPOE may cause errors.5
CPOE’s impact on medication errors and outcomes
remains uncertain because of the varied clinical settings, CPOE system origins (commercial vs created
in-house), and quality of existing studies.6
With its Healthcare Information Technology for
Economic and Clinical Health (HITECH) provision, the American Recovery and Reinvestment Act
of 2009 authorized US$20 billion in funding to
assist in the development of a robust health information technology (health IT) infrastructure to
improve healthcare safety and quality. Among the
HITECH Act’s provisions are incentive payments
to outpatient physicians and hospitals to support
health IT implementation, including CPOE implementation as a core requirement.7
The ultimate goal of CPOE is improved safety,
quality, and value of patient care. Medication
errors are an important intermediate, measurable
outcome in pursuit of that goal. In this investigation, we examine the impact of CPOE on medication error frequency.
OBJECTIVE
The objective of this study was to provide a baseline national estimate of medication errors averted
in hospitals due to use of CPOE, using data on
CPOE use in 2008, before implementation of the
HITECH Act. The baseline estimate and methodology may be useful to track national progress on
CPOE adoption, use, and outcomes, and to inform
the evolving federal strategy to build an effective
health IT infrastructure.
MATERIALS AND METHODS
BACKGROUND AND SIGNIFICANCE
Open Access
Scan to access more
free content
To cite: Radley DC,
Wasserman MR, Olsho LEW,
et al. J Am Med Inform
Assoc 2013;20:470–476.
470
The Institute of Medicine estimates that, on
average, hospitalized patients are subject to at least
one medication error per day.1 Medication errors
are expensive and sometimes harmful to patients.2 3
The Institute of Medicine estimates that at least a
quarter of all medication-related injuries are preventable, and recommends electronic prescribing
(e-prescribing) through a computerized provider
order entry (CPOE) system as one way to reduce
medication errors and patient harm.4 Electronic
entry of medication orders through CPOE may
reduce errors from poor handwriting or incorrect
transcription. CPOE systems often include functionalities such as drug dosage support, alerts about
harmful interactions, and clinical decision support,
This study was conducted in two phases. First, we
developed supporting statistics describing CPOE
adoption and implementation, the volume of medication orders processed in our target population of
hospitals, the number and proportion of medication orders processed through CPOE, the expected
error rate without CPOE, and the expected percentage reduction in medication error rates resulting from CPOE. Second, we used supporting
statistics to derive two nationally representative
outcome estimates: the percentage and absolute
reduction in medication errors in acute-care hospitals over a 1-year period due to CPOE use. Below
we describe the data sources and target population
for this study, followed by the data elements and
analytic techniques used for each statistic.
Radley DC, et al. J Am Med Inform Assoc 2013;20:470–476. doi:10.1136/amiajnl-2012-001241
Research and applications
Data sources
Study data were drawn from the following: the 2007 American
Hospital Association (AHA) Annual Survey (AHA survey); the
AHA Hospital Electronic Health Record (EHR) Adoption
Survey (EHR survey), collected in 2008 as a supplement to the
2007 AHA survey; the 2006 American Society of Health-System
Pharmacists (ASHP) national survey of hospital pharmacies8;
and a systematic review of the peer-reviewed literature.
Target population
Hospitals represented in the AHA survey were included if they
provided general or pediatric acute medical and surgical care
and self-identified as private-for-profit, private not-for-profit, or
public. We excluded long-term care and federally owned hospitals, and hospitals outside the 50 states or the District of
Columbia. Our final sample included 4701 hospitals.
Supporting statistics
CPOE adoption and implementation
EHR survey data were used to estimate CPOE adoption and
implementation for a nationally representative sample of hospitals. The survey’s methodology has been described previously.9
The response rate from the 4701 eligible hospitals was 60.9%
(N=2864), and the response rate to CPOE questions was
slightly lower at 60.3% (N=2833).
CPOE adoption was estimated using EHR survey items on
the presence of CPOE, number of units in which it had been
implemented, and its functionalities. A hospital was counted as
a CPOE adopter if it had in at least one unit an operational
CPOE system capable of processing prescription drug orders.
We used regression imputation to estimate CPOE adoption
among non-responding hospitals. A logistic regression model
identified hospital characteristics associated with CPOE adoption from the AHA survey; estimated model parameters were
used to derive the predicted probability of CPOE adoption
among non-responding facilities.
CPOE implementation was estimated using EHR survey items
asking respondents to estimate the fraction of drug orders processed via CPOE. Respondents had five response options: 0%,
1–25%, 26–50%, 51–90%, or 91–100%. Responding hospitals
were assigned the mid-point of their reported range as their
CPOE implementation measure. Responders’ mean value
(58.8%) was imputed to facilities with CPOE not responding to
this question. For non-responding facilities, the mean value of
58.8% was multiplied by the predicted probability of having
CPOE. Sensitivity analyses to evaluate the impact of this
approach are reflected in reported estimate bounds.
Medication order volume
We used data from the ASHP survey to estimate the volume of
medication orders processed each year by US hospitals. The
ASHP survey is a nationally representative stratified random
sample of hospital pharmacy directors at general and children’s
hospitals. Pedersen and colleagues provide details on survey
methodology, implementation and results.8 Published results
include the average number of prescription drug orders per
patient-day for hospitals stratified by bed size. For each targeted
hospital,
we
multiplied
the
appropriate
average
prescription-drug-order-per-patient-day statistic by AHA survey
estimates of total bed days to estimate the total number of medication orders processed during a year.
Number and proportion of medication orders processed through
CPOE
We then combined hospital-specific estimates of CPOE adoption
and implementation from the 2008 EHR survey with estimates
of medication order volume from the 2006 ASHP survey to
obtain a nationally representative estimate of the total number
and proportion of medication orders processed using CPOE.
We interpret these composite estimates as reflective of the year
2008, implicitly assuming that 2006 medication order volume
estimates did not substantially change over the intervening
2-year period.
Expected medication error rate without CPOE and expected
reduction in medication error rates resulting from CPOE
To date, no nationally representative dataset exists linking
CPOE use to medication errors. Thus, we extracted data from a
systematic literature review and used meta-analytic random
effects techniques to estimate three parameters: medication
error rates when CPOE is not used, medication error rates when
CPOE is used, and the percentage difference between them.
Our literature search strategy was adapted from a review by
Ammenwerth and colleagues.6 Study inclusion criteria were as
follows: the intervention included e-prescribing functionality;
an e-prescribing system was compared with handwritten ordering; the study setting was an inpatient section of an acute-care
hospital; providers with prescribing authority were the primary
users of the e-prescribing system; the study used an experimental or quasi-experimental design, including randomized controlled trial, non-randomized controlled trial, before–after trials,
or repeated time-series analysis; and the mode of medication
error detection was consistent before and after intervention.
Finally, since our goal was to construct aggregated medication
error rates before and after CPOE implementation, we retained
only studies that either reported medication error rates per
order or provided sufficient data from which to calculate these
rates. Studies conducted outside the USA were excluded.
Using the search terms of Ammenwerth et al, we updated the
search using PubMed in February 2009, identifying 390 studies.
Each was reviewed by two study authors (MRW and DCR).
After applying the a priori inclusion/exclusion criteria, 10
studies were retained.10–19 Based on later expert reviewer feedback, we eliminated one additional study that solely used a voluntary reporting method for error detection,19 leaving nine
studies for our final pooled analysis.
The outcome of interest was medication error frequency. For
each included study, we used the author’s definition of a medication error, encompassing errors in ordering, transcribing, dispensing, administration, and monitoring.10 11 Some reviewed
studies also reported adverse drug events (ADEs) and potential
adverse drug events (PADEs). These studies were not excluded
from our literature review, but we used information provided in
reviewed studies to exclude ADEs and PADEs from data aggregation procedures whenever feasible, since both are considered
distinct from more broadly defined medication errors.
Relevant data elements were extracted for pre- and postintervention periods from each study. These included duration
(in months) of pre- and post-intervention periods, and numbers
of prescription drug orders and medication errors recorded. For
each study, we calculated pre- and post-CPOE medication error
rates per order per month. These rates were used to calculate
the percentage change in medication error rates. Pooled
summary statistics were calculated as the weighted average of
pre-CPOE medication error rates, and the weighted average
Radley DC, et al. J Am Med Inform Assoc 2013;20:470–476. doi:10.1136/amiajnl-2012-001241
471
Research and applications
difference between pre- and post-CPOE medication error rates.
Cochran’s Q was significant ( p<0.001), indicating heterogeneity across the nine studies used to calculate the pooled effect.
To accommodate within- and between-study variance, we therefore used the DerSimonian–Laird random effects model to pool
data.20
Outcome statistics
The percentage reduction in medication error frequency due to
CPOE is the product of the expected percentage reduction in
medication error rates resulting from CPOE, as estimated from
our systematic literature review using meta-analytic techniques,
and the proportion of medication errors processed through
CPOE in hospitals that have adopted and use CPOE, as calculated using estimates from nationally representative surveys
described above. We assumed a 0% reduction in medication
error rates due to CPOE for orders processed in settings
without CPOE.
The absolute reduction in medication error frequency due to
CPOE is the product of the first outcome statistic, the expected
percentage reduction in medication error rates resulting from
CPOE, and the expected total number of medication errors if
no CPOE were used (ie, total number of medication orders processed during a year multiplied by the expected medication
error rate without CPOE), as calculated using estimates from
nationally representative surveys as described above.
Input supporting statistics were annualized to facilitate calculation of outcome statistics. These outcome statistics thus represent estimated medication error reduction during a 1-year
period based on observed levels of CPOE adoption and use in
2008, medication order volume in 2006, and error rates and
estimated reductions from our meta-analysis. Varying assumed
levels of CPOE adoption and use from observed 2008 levels
allows us to extrapolate to expected reductions over a 1-year
period as adoption and use increase. These calculations assume
that the effect of CPOE adoption and use on medication error
rates remains constant as the number of hospitals adopting and
implementing CPOE increases. This assumption may not hold if
later CPOE adopters differ systematically from earlier adopters,
or if medication orders currently processed through CPOE
differ systematically from medication orders not currently processed through CPOE. We also assumed that CPOE adoption
does not change the total volume of medication orders processed in hospitals, a simplifying assumption necessary for converting the estimated percentage reduction in medication errors
into an estimated absolute reduction.
More generally, our estimates proceed by applying estimated
parameters from our meta-analysis to nationally representative
survey measures. This approach implicitly assumes that the estimated error rates from the literature generalize to US hospital
settings as a whole.
Estimate bounds
Constructing probability-based CIs around these outcome statistics was not feasible because of non-independence between
several sources of variability embedded in supporting statistics.
Sources of variability included within- and between-study variation in our meta-analysis, sampling variance on prescription
drug order estimates from the ASHP survey, and imputation
procedures used to estimate CPOE adoption and implementation. Instead of probability-based CIs, we therefore constructed
logical estimate bounds based on reasonable assumptions about
underlying variability in these measures.
472
Our approach for pooling data from the systematic literature
review allowed us to derive a probability-based CI around our
estimate of the average percentage reduction in medication
error rates due to CPOE. In addition, sampling variance on
medication order estimates from the ASHP survey influenced
our estimates for the total number of medication orders and the
number of medication orders processed with CPOE.
ASHP-reported SEs on mean medication orders per bed-day
were used to calculate probability-based lower and upper
bounds on estimates of medication order volume. There was no
credible approach for deriving probability-based estimates of
variance for measures of CPOE adoption and implementation.
Instead, we set logical bounds around each by imposing assumptions about CPOE use for hospitals not responding to the EHR
survey. The lower bound assumed that no non-responding hospitals adopted CPOE, and the upper bound assumed that all
non-responding hospitals adopted CPOE. Each supporting statistic described above was recalculated using lower and upper
bounds as inputs. Upper- and lower-bound supporting statistics
were in turn used as inputs to calculate upper- and lower-bound
summary statistics.
Our intentionally conservative approach resulted in point estimates with relatively wide bounds. Note, however, that, as these
are not probability-based CIs, the point estimates do not have
an equal probability of taking on any value between reported
bounds. Instead, the point estimates give a reasonable approximation of the true value, while the bounds represent possible
extreme values that could have been derived from given inputs.
RESULTS
CPOE use
Approximately 34% (1589 of 4701) of US acute-care hospitals
had adopted CPOE in 2008. Among the 2833 hospitals
responding to the EHR survey, larger hospitals (≥400 beds)
were more likely to have adopted CPOE (56%) compared with
medium-sized or small hospitals (35% and 30%, respectively).
CPOE adoption was more common among urban hospitals
(41% vs 28% among rural hospitals, p<0.001) and major teaching hospitals (53% vs 32% in non-teaching hospitals, p<0.001).
CPOE adoption was higher among private not-for-profit hospitals (37%) compared with public hospitals (31%) and private
for-profit hospitals (32%). CPOE adoption did not significantly
differ between independent and health system-affiliated hospitals (34% vs 36%, p=0.13). Table 1 summarizes CPOE adoption in 2008 by hospital characteristic.
Table 2 describes 2008 CPOE implementation, the proportion
of prescription drug orders processed using CPOE in hospitals
with a CPOE system. Many CPOE adopters (39.0%) indicated a
very high degree of implementation (>90% of orders processed
by CPOE). Still, 42.4% of responding hospitals using CPOE
reported <50% implementation. Assuming the midpoint value
for each range, and averaging across all hospitals providing
CPOE data, mean CPOE implementation was 58.8%. Among
hospitals adopting CPOE, there was no statistically significant
association between bed size and CPOE implementation levels.
Total annual medication order volume was estimated at
1 757 886 464 orders per year based on the 2007 AHA survey
and the 2006 ASHP survey. Factoring this with CPOE adoption
and implementation in 2008, and extrapolating to all US acutecare hospitals, we estimate that ∼26.1% (bounds 16.0–53.6%)
of medication orders in acute-care hospitals were processed
using CPOE.
Radley DC, et al. J Am Med Inform Assoc 2013;20:470–476. doi:10.1136/amiajnl-2012-001241
Research and applications
medication error rate per order of 0.079 without CPOE.
Medication error rates were ∼48% (95% CI 41% to 55%)
lower after CPOE implementation (table 4).
Table 1 Computerized provider order entry (CPOE) adoption by
hospital characteristic, 2008
Characteristic
All acute hospitals
Pediatric specialty hospitals
Bed size
Small (6–99)
Medium (100–399)
Large (≥ 400)
Census region*
Northeast
Midwest
South
West
Ownership type
Public
Not-for-profit
For-profit
Member of a health system
No
Yes
Location
Rural
Urban
Teaching status
Non-teaching hospital (0 full-time
residents)
Minor teaching hospital (between 1
and 20 full-time residents)
Major teaching hospital (more than
20 full-time residents)
With
CPOE
Without
CPOE
p Value
992 (35)
14 (100)
1841 (65)
(0)
NA
NA
389 (30)
430 (35)
173 (56)
910 (70)
795 (65)
136 (44)
173
298
356
141
224
633
639
297
(44)
(32)
(36)
(32)
(56)
(68)
(64)
(68)
<0.001
0.001
211 (31)
670 (37)
111 (32)
471 (69)
1131 (63)
239 (68)
452 (34)
540 (36)
894 (66)
947 (64)
0.128
360 (28)
632 (41)
928 (72)
913 (59)
<0.001
705 (32)
1505 (68)
98 (37)
169 (63)
189 (53)
167 (47)
0.005
Summary and outcome statistics are summarized in table 4. At
the rate of CPOE adoption and implementation in 2008, our
findings suggest that medication errors were reduced by
∼12.5% (bounds 10.6–14.4%). This equates to ∼17.4 million
(bounds 0.09–27.1 million) fewer medication errors over a
1-year period than would be expected without CPOE.
Figure 1 illustrates observed (under the analytic assumptions
of this study) and potential reductions in medication errors
during a 1-year time frame associated with varying levels of
CPOE adoption. The y-axis is the number of averted errors, and
the x-axis is the proportion of hospitals adopting CPOE. The
slope of the line for each wedge represents CPOE implementation ( proportion of drug orders processed using CPOE). The
black wedge represents medication error reductions associated
with CPOE use in 2008. If all US hospitals adopted CPOE,
assuming constant implementation levels of ∼60%, as many as
51 million medication errors per year could be averted compared with what would have been expected without CPOE
(hatched area). If the proportion of medication orders processed
using CPOE were to increase in hospitals that already have
CPOE in place (grey shaded area), as many as 36 million errors
per year might be averted. Finally, if all hospitals were to use
CPOE to process all medication orders, as many as 104 million
medication errors per year could be averted (dotted area).
<0.001
DISCUSSION
Values are number (%). Data in this table are aggregated only from the 2833
hospitals that provided responses to the EHR adoption database supplement
questions regarding CPOE adoption.
*Geographic region was missing for 72 hospitals.
EHR, electronic health record.
Expected medication error rate without CPOE and expected
change in error rate associated with CPOE use
Table 3 summarizes literature review findings.
Among the nine pooled studies, eight found a decrease in
medication error frequency after CPOE implementation. The
last, Walsh et al,18 reported an increase (23%) in medication
errors. Pooling data across these studies, we find an expected
Table 2 CPOE implementation among hospitals that report having
a CPOE system according to hospital size, 2008
Proportion of medication orders processed using
CPOE
Hospital size, beds
1–25%
26–50%
51–90%
≥ 90%
<200
200–299
300–399
≥ 400
Total (row %)
110
28
22
45
205
46 (13)
12 (12)
10 (14)
16 (11)
84 (12)
78
20
13
16
127
126
42
24
74
266
(31)
(27)
(32)
(30)
(30)
Percentage and absolute reduction in medication errors
(22)
(20)
(19)
(11)
(19)
(35)
(41)
(35)
(49)
(39)
Values are number (%). These data represent reported CPOE implementation among
the 682 hospitals that indicated having a CPOE system in place and gave responses to
CPOE implementation questions in the EHR survey.
CPOE, computerized provider order entry; EHR, electronic health record.
To our knowledge, our study represents the first attempt to generate a nationally representative estimate of the effect of CPOE
on medication error frequency. Our findings suggest that CPOE
can substantially reduce medication errors in hospitals. In 2008,
∼34% of US acute-care hospitals had adopted CPOE capable of
processing prescription orders. At these adoption and implementation levels, we estimate 17.4 million medication errors per
year avoided due to CPOE—a 12.5% reduction nationally.
Given the modest adoption and implementation rates to date,
there is still great potential for this technology to reduce medication errors.
The projected reduction in medication errors represents an
important intermediate indicator of potential gains as health IT
systems are expanded and more deeply integrated in care delivery systems nationwide. However, it is unclear whether reduced
medication errors would translate into reduced patient harm
from medications. Several studies reviewed for this analysis
provide insight into the potential for harm reduction. Bates
et al,10 Evans et al,14 and Igboechi et al15 all report fewer ADEs
and fewer serious medication errors with potential to harm
patients after CPOE implementation. However, insufficient
information was available to construct meta-analytic estimates
of CPOE impacts on serious medication errors, PADEs, or
ADEs.
While our conclusions reflect the fact that the majority of peerreviewed studies find reductions in medication errors associated
with CPOE, a few studies have found increases. As noted above,
Walsh et al18 found that error rates increased from 18 to 22 per
1000 medication orders after CPOE adoption, although this difference was not statistically significant, and the authors also found
fewer non-intercepted serious medication errors and fewer PADEs
in the post-CPOE implementation period. Bradley and colleagues
illustrate the risk of unintended consequences from CPOE: while
Radley DC, et al. J Am Med Inform Assoc 2013;20:470–476. doi:10.1136/amiajnl-2012-001241
473
Research and applications
Table 3 Summary data from systematically peer-reviewed literature evaluating medication error frequency before (pre) and after (post)
implementation of computerized provider order entry (CPOE)
Duration
(months)
Pre
Author (year of
publication)
CPOE implementation and study setting
(hospital department)
Bates et al (1999)10
Select medical and intensive care units
(inpatient)
Emergency department
NICU
Intensive care unit
Hospital wide (inpatient)
2
6
24
24
Pediatric oncology unit
Hospital wide (inpatient)
NICU
NICU, PICU, select pediatric medical and
surgical units (inpatient)
Bizovi et al (2002)12
Cordero et al (2004)13
Evans et al (1998)14
Igboechi et al
(2003)15
Kim et al (2006)16
Mahoney et al
(2007)17
Taylor et al (2008)11
Walsh et al (2008)18
1.7
Post
5.6
Medication orders
Medication
errors
Rate per
1000
orders
Pre
Pre
Post
Pre
Post
Post
Percentage difference
(unweighted)*
10070
42516
255
340
25
8
−68
2
6
12
12
2326
136
1813
1868274
2169
117
942
934137
54
16
787
5441
11
0
134
1247
23
118
434
3
5
0
142
1
−78
−100
−67
−54
8
12
9.9
12
1259
1452346
1116
1390789
84
4815
69
2227
67
3
62
2
−7
−52
11
7
9
9
254
5777
272
6895
50
106
31
155
197
18
114
22
−42
23
*Our calculated summary statistic (presented in table 4) used the DerSimonian–Laird method (DL) to pool these data, where each study’s DL weight was multiplied by the unweighted
percentage difference shown. The DL effect sizes are not included here, as they are not scaled in a meaningful way.
NICU, neonatal intensive care unit; PICU, pediatric intensive care unit.
error types reported during the study’s pre-period were reduced
by 65%, of the 164 errors reported during the study’s post-CPOE
implementation period, 117 were related to the use of the CPOE
system itself.19 The authors observed a range of CPOE-related
errors, including prescribing errors, where clinicians chose the
wrong drug or dose from a pull-down menu, attributed a prescription to the wrong patient, or entered duplicate orders, and transcribing errors, where pharmacy staff incorrectly recorded
medication order information. As noted above, data from the
Bradley et al study were not used in our meta-analysis, because the
study used only a voluntarily reported error detection method. As
a sensitivity analysis, we produced estimates including Bradley
et al. Pooling data from Bradley et al with estimates from the other
literature would decrease our estimate of 12.5% (bounds 10.6–
14.4%) error reduction nationally to 9.2% (bounds 0.33–36.7%),
implying ∼3.1 million fewer averted medication errors per year.
Reviewed studies used various medication error detection
methods. Research suggests that the highest error rates are
found through direct observation,21 followed by chart review,
then automated surveillance, and voluntary reporting.22 There
exists limited overlap in errors reported through each modality,
and studies using a combination of methods detect the highest
Table 4
error rates.23 We extracted information on detection mode from
each study, and excluded studies relying solely on voluntary
reporting or employing inconsistent detection modes before and
after CPOE implementation. While there is a risk that errors
will be double-counted in studies using multiple modes,22
studies that use single and/or less rigorous detection modes may
seriously undercount medication errors.
Definitions of medication errors also varied between studies
in our review. For example, most studies counted wrong doses
as medication errors, but only some counted drug–drug interactions. In addition, while some studies reported stratified results
by error type (eg, frequency, route), others stratified by process
stage (eg, prescribing, transcribing). Finally, some studies defined
medication errors as events with potential to cause patient
harm, while others counted errors corrected by the hospital’s
internal redundant systems before reaching the patient.
Unfortunately, in aggregating these data, it was not possible to
parse out these differences.
Several limitations should be noted when interpreting results.
As with any meta-analysis, some limitations extend from limitations in the primary literature upon which our estimates relied.
First, as noted above, detection mode and medication error
Outcome and supporting statistics
Calculated metric
Supporting statistics
Mean % reduction in medication error rates conditional on
using CPOE to prescribe the order
Proportion of medication orders that are ordered using a
CPOE system
Outcome statistics
Percentage reduction in medication error frequency
resulting from CPOE use to process medication orders
Absolute reduction in medication error frequency resulting
from CPOE use to process medication orders
Data source
Point estimate
Estimate bound
Peer-reviewed literature
−48%
(−55% to −41%)
Hospital surveys (2007 AHA annual survey, 2008 EHR
adoption database, 2006 ASHP National Survey)
26.10%
(16.0% to 53.6%)
Calculated from supporting statistics
−12.5%
(−14.4% to −10.6%)
Calculated from supporting statistics
17390443
(88058 to 27094038)
AHA, The American Hospital Association; ASHP, The American Society of Health-System Pharmacists; CPOE, computerized provider order entry; EHR, electronic health record.
474
Radley DC, et al. J Am Med Inform Assoc 2013;20:470–476. doi:10.1136/amiajnl-2012-001241
Research and applications
Figure 1 Estimated medication errors averted due to observed and increased use of computerized provider order entry in inpatient acute-care
hospitals in a 1-year period.
definitions varied across included studies. We attempted to
address these and other sources of study heterogeneity by implementing inclusion criteria to ensure a minimum standard for
comparability, and by using statistical adjustments on pooled
estimates. However, it was not possible to fully account for all
methodological differences. Second, hospitals in included
studies may not be broadly representative of all US hospitals,
particularly since there was a preponderance of large, urban,
academic medical centers; Hug and colleagues found a higher
incidence of ADEs in community hospitals than in academic
medical centers.24 For this reason, the medication error rates
and the mix of medication order types in included studies may
vary from that in all US hospitals. Similarly, while studies have
found that specific CPOE functionalities or accompanying clinical decision support can affect error rates,25 26 we could not
systematically capture and quantify these factors for our estimate. It was difficult to observe and quantify other factors that
may modify CPOE effects across hospitals (eg, implementation
duration, commitment to quality improvement, severity of
illness among patient population). Finally, we note an additional
limitation not stemming from our meta-analytic approach. Our
calculations assume that the effect of CPOE adoption on medication error rates is constant as the number of hospitals adopting CPOE increases. This assumption may not hold if later
CPOE adopters differ systematically from earlier adopters.
Our findings and the limitations of our estimate point to key
areas for future research; as the relevant source literature grows
in breadth and methodological consistency to overcome these
limitations, the base methodology described here can be refined
and expanded to improve on existing estimates. First, further
research is needed to characterize the effect of CPOE
implementation on order volume and patterns, and heterogeneity in the likelihood of medication errors for different types of
medications, orders, and settings. In addition, common and consistently applied definitions of medication errors and serious
medication errors, as well as consistent stratification of errors by
type and/or ordering process stage, will ensure greater comparability across studies. Second, further work is needed to explain
variation in findings across studies; for example, authors have
noted an increase in medication errors due to CPOE19 and unintended consequences of health IT.27 28 Even among the eight
studies in our meta-analysis showing reductions in errors due to
CPOE, variation in the magnitude of impacts across study settings may indicate that not all patient populations will benefit
equally from CPOE’s apparent error risk reduction. Third, given
the variation in detected error rates by detection mode, future
model refinements might weight analyses according to detectionmode sensitivity. Making scaling decisions will necessarily
require a judgment about the relative value of more or less rigorous detection modes. Finally, additional evidence is needed to
establish more concrete links between medication errors, ADEs,
and patient harm; while Bates et al29 found that 0.9% of medication errors result in ADEs, few other such estimates are currently
available, either in general or stratified by CPOE system functionality, and even less is known about the frequency with which
medication errors result in actual patient harm.
CONCLUSION
Our rigorously developed meta-analytic estimate is in keeping
with the earlier, evidence-based heuristic of Bates et al,23 and,
more importantly, is a much-needed addition to our knowledge
of the effect of CPOE on medication errors. Our estimation
Radley DC, et al. J Am Med Inform Assoc 2013;20:470–476. doi:10.1136/amiajnl-2012-001241
475
Research and applications
approach may be useful for estimating other health IT-related
impacts on the healthcare system and patient outcomes. Future
research in this area will be critically important to inform policy
and funding decisions regarding the development and implementation of CPOE in care delivery.
7
Acknowledgements We thank our Task Order Officer, Robert Mayes (Agency for
Healthcare Research and Quality), who helped conceptualize this paper. Ashish Jha
(Department of Health Policy and Management, Harvard School of Public Health)
provided expertise on CPOE and medication errors and contributed to the literature
review. Catherine Desroches and Eric Campbell (Institute for Health Policy,
Massachusetts General Hospital) provided expertise on CPOE and data management
for the American Hospital Association’s 2008 EHR Adoption Database. William
Rhodes and K P Srinath (Abt Associates) contributed to the methodology to develop
estimate bounds. We thank Jacob Klerman and the Abt Associates Journal Authors
Support Group for financial support to revise and resubmit this manuscript. Finally,
Rena Kirsch and Suzanne Erfurth (Abt Associates) provided editorial assistance.
9
Funding This project was funded under PSC contract No 233–02–0088, Task
Order # HHSP233200700008T from the Agency for Healthcare Research and Quality
(AHRQ), US Department of Health and Human Services. The opinions expressed in
this document are those of the authors and do not reflect the official position of
AHRQ or the US Department of Health and Human Services.
8
10
11
12
13
14
15
Competing interests All authors have completed the Unified Competing Interest
form at http://www.icmje.org/coi_disclosure.pdf (uploaded through manuscript
central) and declare: no support from any organization for the submitted work; no
financial relationships with any organizations that might have an interest in the
submitted work in the previous 3 years; no other relationships or activities that could
appear to have influenced the submitted work.
16
Contributors MS and MW designed the study in response to the AHRQ RFP (the
idea for this paper). MW developed the literature review method, with input from
DR. MW drafted the original paper with DR and LO. DR led the analysis with
substantial input from LO. SS and MS reviewed the original paper. SS and BB
revised the paper for submission. SS, LO and DR responded to peer reviewers’
comments.
18
Provenance and peer review Not commissioned; externally peer reviewed.
Open Access This is an Open Access article distributed in accordance with the
Creative Commons Attribution Non Commercial (CC BY-NC 3.0) license, which
permits others to distribute, remix, adapt, build upon this work non-commercially,
and license their derivative works on different terms, provided the original work is
properly cited and the use is non-commercial. See: http://creativecommons.org/
licenses/by-nc/3.0/
17
19
20
21
22
23
REFERENCES
1
2
3
4
5
6
476
Aspden P, Wolcott J, Bootman J, et al. Preventing medication errors. Washington,
DC: National Academic Press, 2007.
Bates DW, Spell N, Cullen DJ, et al. The costs of adverse drug events in hospitalized
patients. Adverse Drug Events Prevention Study Group. JAMA 1997;277:307–11.
Lesar TS, Briceland LL, Delcoure K, et al. Medication prescribing errors in a teaching
hospital. JAMA 1990;263:2329–34.
Committee on Quality of Health Care in America. Crossing the quality chasm: a
new health system for the 21st century. Washington, DC: Institute of Medicine,
National Academy Press, 2001.
Berger RC, Kichak BA. Computerized physician order entry: helpful or harmful? J
Am Med Inform Assoc 2004;11:100–3.
Ammenwerth E, Schnell-Inderst P, Machan C, et al. The effect of electronic
prescribing on medication errors and adverse drug events: a systematic review. J Am
Med Inform Assoc 2008;15:585–600.
24
25
26
27
28
29
Department of Health and Human Services, Centers for Medicare and Medicaid
Services. 42 CFR parts 412, 413, 422 et al. Medicare and Medicaid Programs:
Electronic Health Record Incentive Program; Final Rule. 2010. http://edocket.access.
gpo.gov/2010/pdf/2010-17207.pdf (accessed 19 Aug 2010).
Pedersen CA, Schneider PJ, Scheckelhoff DJ. ASHP national survey of pharmacy
practice in hospital settings: monitoring and patient education–2006. Am J Health
Syst Pharm 2007;64:507–20.
Jha AK, DesRoches CM, Campbell EG, et al. Use of electronic health records in U.S.
hospitals. N Engl J Med 2009;360:1628–38.
Bates DW, Teich JM, Lee J, et al. The impact of computerized physician order entry
on medication error prevention. J Am Med Inform Assoc 1999;6:313–21.
Taylor JA, Loan LA, Kamara J, et al. Medication administration variances before and
after implementation of computerized physician order entry in a neonatal intensive
care unit. Pediatrics 2008;121:123–8.
Bizovi KE, Beckley BE, McDade MC, et al. The effect of computer-assisted
prescription writing on emergency department prescription errors. Acad Emerg Med
2002;9:1168–75.
Cordero L, Kuehn L, Kumar RR, et al. Impact of computerized physician order entry
on clinical practice in a newborn intensive care unit. J Perinatol 2004;24:88–93.
Evans RS, Pestotnik SL, Classen DC, et al. A computer-assisted management
program for antibiotics and other antiinfective agents. N Engl J Med
1998;338:232–8.
Igboechi C, Ng C, Yang C, et al. Impact of computerized prescriber order entry on
medication errors at an acute tertiary care hospital. Hosp Pharm 2003;38:227–31.
Kim GR, Chen AR, Arceci RJ, et al. Error reduction in pediatric chemotherapy:
computerized order entry and failure modes and effects analysis. Arch Pediatr
Adolesc Med 2006;160:495–8.
Mahoney CD, Berard-Collins CM, Coleman R, et al. Effects of an integrated clinical
information system on medication safety in a multi-hospital setting. Am J Health
Syst Pharm 2007;64:1969–77.
Walsh KE, Landrigan CP, Adams WG, et al. Effect of computer order entry on
prevention of serious medication errors in hospitalized children. Pediatrics
2008;121:e421–7.
Bradley VM, Steltenkamp CL, Hite KB. Evaluation of reported medication errors
before and after implementation of computerized practitioner order entry. J Health
Inf Manag 2006;20:46–53.
Egger M, Smith G, Altman D. Systematic reviews in health care: meta-analysis in
context. 2nd edn. London: BMJ Publishing Group, 2001.
Flynn EA, Barker KN, Pepper GA, et al. Comparison of methods for detecting
medication errors in 36 hospitals and skilled-nursing facilities. Am J Health Syst
Pharm 2002;59:436–46.
Jha AK, Kuperman GJ, Teich JM, et al. Identifying adverse drug events:
development of a computer-based monitor and comparison with chart review and
stimulated voluntary report. J Am Med Inform Assoc 1998;5:305–14.
Bates DW, Leape LL, Cullen DJ, et al. Effect of computerized physician order entry
and a team intervention on prevention of serious medication errors. JAMA
1998;280:1311–16.
Hug BL, Witkowski DJ, Sox CM, et al. Adverse drug event rates in six community
hospitals and the potential impact of computerized physician order entry for
prevention. J Gen Intern Med 2009;25:31–8.
Metzger J, Welebob E, Bates DW, et al. Mixed results in the safety performance of
computerized physician order entry. Health Aff (Millwood) 2010;29:655–63.
Nanji KC, Rothschild JM, Salzberg C, et al. Errors associated with outpatient
computerized prescribing systems. J Am Med Inform Assoc 2011;18:767–73.
Sittig DF, Ash JS, Guappone KP, et al. Assessing the anticipated consequences of
computer-based provider order entry at three community hospitals using an
open-ended, semi-structured survey instrument. Int J Med Inform 2008;77:440–7.
Ash JS, Sittig DF, Dykstra R, et al. Exploring the unintended consequences of computerized
physician order entry. Stud Health Technol Inform 2007;129(Pt 1):198–202.
Bates DW, Boyle DL, Vander Vliet MB, et al. Relationship between medication errors
and adverse drug events. J Gen Intern Med 1995;10:199–205.
Radley DC, et al. J Am Med Inform Assoc 2013;20:470–476. doi:10.1136/amiajnl-2012-001241