G Model

HEAP-3604; No. of Pages 11 ARTICLE IN PRESS

Health Policy xxx (2016) xxx–xxx

Contents lists available at ScienceDirect

Health Policy
journal homepage: www.elsevier.com/locate/healthpol

Mechanisms and effects of public reporting of surgeon

outcomes: A systematic review of the literature
Katja Behrendt a , Oliver Groene a,b,∗
a
Department of Health Services Research and Policy, London School of Hygiene and Tropical Medicine, London, UK
b
Department of Health Services Research and Policy, London School of Hygiene and Tropical Medicine, London, UK; and Optimedis AG,
Hamburg, Germany

a r t i c l e i n f o a b s t r a c t

Article history: Background: Public reporting of surgeon outcomes has become a key strategy in the English
Received 21 October 2015 NHS to ensure accountability and improve the quality of care. Much of the evidence that
Received in revised form 3 August 2016 supported the design of the strategy originates from the USA. This report aims to assess
Accepted 4 August 2016
how the evidence on public reporting could be harnessed for cross-country translation of
this health system strategy; in particular, to gauge the expected results of the UK surgeon
Keywords: outcome initiative and to propose criteria that elucidate that prerequisites and factors that
Surgeons
are needed to public reporting effective.
Outcome assessment
Methods: A systematic search of academic databases was followed by snowballing from the
Public reporting
Quality of health care reference lists. Only peer-reviewed articles and primary studies were included.
Quality improvement Results: 25 studies from the USA (n = 22) and the UK (n = 3) were included. Suggestive
Task performance and analysis evidence of a negative effect on access to surgery was found for high-risk patients and
non-whites; one survey indicated presence of gaming. There was anecdotal evidence of
quality improvement measures adopted by low-rated hospitals in New York. Most studies
reported only on the effectiveness of public reporting, rather than addressing how effects
accrue. This limits cross-country transferability of policy lessons. Based on our analysis,
we propose factors impacting on the transferability of the evidence underlying the public
reporting of surgeon outcomes, which may inform the adoption of this strategy in other
health systems.
Conclusions: There is some evidence that public reporting can be an incentive for low per-
forming surgeons to improve quality. Negative incentive on patient selection as suggested
in the USA have not yet been observed in the UK.
© 2016 Elsevier Ireland Ltd. All rights reserved.

1. Introduction outcomes [1,2]. In December 2012, NHS England published

its 2013/14 planning guidance specifying – amongst other
In the UK, an ambitious public reporting (PR) pro- – the aim to improve patient care through better com-
gramme has been implemented in the last few years, missioning of services, and to support patients in making
steered by various high profile scandals about quality care informed choices, implemented via public reporting of
(initially) ten specialities (better known as ‘Offer 2’) [3].
Today, PR is seen as a central tool for quality improve-
∗ Corresponding author at: Department of Health Services Research ment and since 2013 individual surgeons’ outcomes are
and Policy, London School of Hygiene and Tropical Medicine, London, UK. made public via the patient portal ‘NHS choices’ [4]. Data
E-mail addresses: Behrendt.katja@gmail.com (K. Behrendt),
has been published for more than 5000 consultant sur-
oliver.groene@lshtm.ac.uk (O. Groene).

http://dx.doi.org/10.1016/j.healthpol.2016.08.003
0168-8510/© 2016 Elsevier Ireland Ltd. All rights reserved.

Please cite this article in press as: Behrendt K, Groene O. Mechanisms and effects of public reporting of surgeon outcomes:
A systematic review of the literature. Health Policy (2016), http://dx.doi.org/10.1016/j.healthpol.2016.08.003
G Model
HEAP-3604; No. of Pages 11 ARTICLE IN PRESS
2 K. Behrendt, O. Groene / Health Policy xxx (2016) xxx–xxx

geons in 12 specialties (adult cardiac surgery, bariatric Table 1

Inclusion and exclusion criteria.
surgery, colorectal surgery, endocrine and thyroid surgery,
head and neck cancer surgery, interventional cardiology, Inclusion criteria Exclusion criteria
lung cancer, neurosurgery, orthopaedic surgery and upper - Primary studies - (Systematic) literature
gastro-intestinal surgery). Data sources and outcome mea- - All clinical areas reviews
sures vary among specialties, but all include mortality rates - Focus on supply side - Consumer ratings only (see
(MR) of patients. (healthcare providers) definition of outcome)
- Main focus on surgery, either - Focus on demand side
The idea to publicly report provider outcomes in order
individual surgeons or - (Duplication of findings in
to ensure and improve quality is not new. Nearly 30 years surgery departments revision phase of manuscript
ago, the Healthcare Financing Authority (HCFA) published - Mandatory PR or PR in preparation)
annual mortality rates of Medicare patients in hospitals in general
the USA in 1987. Shortly after that, in 1989, the most stud- - Information about possible
consequences of PR, the
ied PR scheme, the New York Cardiac Surgery Reporting how?
System (CSRS) was initiated. In New York State, hospi- - Peer reviewed journals
tals needed to obtain a certificate in order to offer cardiac - Full text available
surgery, and mandatory PR outcomes have been enforced
since 1991. From 1989 to 1992 mortality rates dropped
from 3.5% to 2.8% (41% decrease of risk-adjusted in-hospital
evidence on purchasing models for surgeon outcomes, even
mortality) [5], while nationally it only decreased 18%
though this is an area of increasing importance as many
between 1987–1990 (30-day MR). In the 1990s, Pennsyl-
health systems start to focus on purchasing for value rather
vania and Massachusetts also started surgery-focused PR.
than services. Research suggests that a large part of the
Today, PR is a widespread tool to measure and advertise
impact of PR is amongst providers themselves [15–19].
the quality of nursing homes, hospitals and healthcare-
There is a growing body of evidence about what the
maintenance-organisations (HMOs) [6]. While in the
impacts of PR might be. Yet, a large volume of previous
English NHS surgeon outcomes were available before,
research on public disclosure focused at the hospital level,
they were not specifically released to the patients (but
rather than individual surgeon level, and did not suffi-
reported embedded on hospital web sites), not systemat-
ciently focus on the question by way of which mechanisms
ically reported in a common, standardized platform and
PR influences quality and which health system structures,
not covering the wide range of specialities involved now.
governance arrangements and, broadly, prerequisites need
Thus, the recent publication of surgeon outcomes has gen-
to be in place for public reporting to be effective. However,
erated substantial debate in academia and policy since its
this very question is a key to understanding the potential
publication on NHS Choices in 2013/2014.
effects of public reporting and assessing whether research
The assumption is that public reporting unfolds its
evidence stemming from one health system can be applied
effects via a range of pathways [7,8]. Generally, a distinction
to another (here: the NHS in the UK).
is made between pathways referring to the principle–agent-
The aims of this review are therefore to assess the evi-
relationship where patients are empowered by public
dence on whether and how public reporting of outcomes
reporting to make better choices, or pathways related
can be an incentive for surgeons to improve quality. Fur-
to competition theory, whereas providers would improve
thermore, we investigate whether the literature contains
outcomes to compete for patients. In practice, there are
evidence regarding two key tenets of the UK surgeon-
alternative mechanisms through which public reporting
level outcomes public reporting initiative: (i) surgeons
could affect outcomes. Werner and Asch [8] outline 4 pos-
discussing their outcomes with patients, GPs, CCGs or man-
sible mechanisms: (1) patients might choose high-quality
agers in order to improve the quality of care (as indicated in
providers; (2) GPs might refer to high-quality providers;
the NHS planning document “Offer 2”), or (ii) public report-
(3) purchasers select high-quality providers; (4) providers
ing leading to adverse patient selection, as feared by the
themselves react in response to PR. In terms of patient
surgical specialties [20]. We then (iii) discuss the findings
choice, a growing body of evidence shows that PR of out-
and consider the potential impact of the current UK surgeon
comes only have a limited impact: patients have difficulties
level reporting initiative and (iv) propose some key lessons
understanding comparative PR data, especially if differ-
and contextual factors that should be considered by other
ent measures are contradicting [9,10]. Other studies have
countries that are in the process of devising a surgeon level
found that consumers might value own experience or
public disclosure strategy.
recommendation from family and friends more than com-
parative outcomes information [11]. In one study, fewer
than 10% of patients looked at performance data [12]; in 2. Methods
another, only 12% of patients who had undergone Coro-
nary Artery Bypass Graft (CABG) surgery in NY knew about We gathered and assessed the literature on public
PR and fewer than 1% were able to state the performance reporting of surgeon outcomes, and synthesised the effects
of their surgeon correctly [13]. In terms of GPs referral to and mechanisms from this literature. Methods used to col-
high quality providers, the proposition is that they would lect and analyse the information contributing to this review
act as agents for their clients, but in fact evidence suggests followed the PICO framework [21], PRISMA statement [22]
that GPs also value informal sources of quality information and Hawker et al. [23] tool for quality assessment of the
more than performance ratings [14]. There is insufficient retrieved papers.

Please cite this article in press as: Behrendt K, Groene O. Mechanisms and effects of public reporting of surgeon outcomes:
A systematic review of the literature. Health Policy (2016), http://dx.doi.org/10.1016/j.healthpol.2016.08.003
G Model
HEAP-3604; No. of Pages 11 ARTICLE IN PRESS
K. Behrendt, O. Groene / Health Policy xxx (2016) xxx–xxx 3

2.1. Search strategy 9 categories (abstract, introduction, methods, sampling,

results, bias, analysis, transferability and implications) that
Translating the question into search terms was can be rated with 1–4 points in their quality and added up
informed by the PICO framework that has been recom- to a cumulative score between 9 and 36. The results can be
mended for literature review about clinical questions [21]. seen in Appendix C in the Supplementary material.
The framework includes the patient target group (P), inter-
vention (I), comparison group (C) and outcome (O). For
the purpose of this review, a comparison group was not 2.4. Data extraction
included into the search terms.
Target population were individual surgeons or surgery Again in accordance with Cochrane recommendations,
departments. To extend the number of potential hits, all the Cochrane data extraction worksheet [25] was adapted
known US states with a PR system or known USA-wide PR to fit the question and scope of this review. A meta-analysis
systems were included, as they might be mentioned in the of the study data was not attempted due to the various
title rather than “surgeon”. This approach was informed by study designs and backgrounds.
the review by Totten et al. on a related topic [24]. In this
way, all present literature, also from outside the USA, could 3. Results
be included.
As PR was the intervention, all synonyms for PR or out- 3.1. Studies found and evaluated
comes were included in the search. As outcomes, all terms
referring to consequences and assessment were included The database search led to initially 710 publications
(see Appendix A in the Supplementary material). being found, of which 663 remained after exclusion of
Starting from these initial search terms, related Emtree duplicates. After screening titles, 112 studies were iden-
and MESH terms were compiled and linked using Bolean tified as potentially relevant, of which 50 were chosen for
operators: search terms were linked with OR and con- full text reading after reviewing abstracts. 11 of these were
cepts with AND. The search was conducted on 15.07.2015 included in the review. The screening of the reference list
(Appendix B in the Supplementary material). The search resulted into another 14 studies being located. Thus, in
was conducted in Medline, EconLit and Embase. The search total, 25 studies were included into the review (Fig. 1).
was limited to all studies published after 1980, as the first Of the 25 studies identified, 8 had a survey or interview
PR initiative started in 1989. Further limits were human design, 8 an interrupted time series or difference-in dif-
and English language. Due to the number of results, the ferences approach, one case study, a case control study, 5
search was further limited in Medline and Embase to title cohort studies and 2 with a before-after design. 22 stud-
and abstracts. For the search in EconLit, only 3 concepts ies were from the USA and 3 from the UK. The period of
were applied and no limits, due to the much smaller num- analysis covered 1987 until 2012.
ber of results than in Medline and Embase.

2.2. Study selection 3.2. Key themes in the literature

The initial results were screened according to their title The most frequent issue of PR addressed in the litera-
and potential relevance to PR. In a second step the abstracts ture is the potential detrimental effect of PR on access to
were considered. Of the remaining studies, the full text was surgery—the assumption that surgeons might reject high
retrieved if possible and pre-defined inclusion and exclu- risk patients because of the fear of becoming an outlier in
sion criteria applied (Table 1). To make sure all relevant the mortality statistics. All other themes were addressed far
studies would be included, even if not found in the system- less commonly, particularly those related to the assump-
atic search, reference lists of included studies were perused tions of “Offer 2”, namely that patients use the information
to find further relevant studies (snowballing). For the pur- to choose a surgeon or that GPs and purchasers use the
pose of this review, only primary studies were considered. reports to contract surgeons. Likewise, we did not find stud-
Literature reviews were excluded, but the reference lists of ies that describe the consequences of patients confronting
systematic reviews were used to further check that no rele- the hospital/surgeon and requesting, because of their out-
vant study was left out. To ensure the quality of the review, comes, a reassignment. Table 2 provides an overview on
only peer-reviewed articles were included. the studies included and on the outcomes of interest.
Adverse patient selection was the most frequently stud-
2.3. Quality assessment ied outcome of public reports. Of the studies included in
this review, 5 reported a positive or no effect on access to
Cochrane Effective Practice and Organisation of Care surgery, one a transient effect. The majority of the stud-
Group (EPOC) provides detailed guidelines on how to assess ies (n = 14) however reported a negative effect on patient
the quality of studies, mostly clinical trials [25]. Due to the selection, suggesting that surgeons were less likely to
diverse nature of the studies included, the tool was found accept high risk patients after the introduction of public
unfit to assess the quality of qualitative studies. To do jus- reporting.
tice to the different types of studies, a checklist developed Table 3 summarizes the result of the studies on adverse
by Hawker et al. [23,26] was used, which can be applied to selection as a consequence of public reporting of surgeon
studies with qualitative and quantitative methods. It has outcomes.

Please cite this article in press as: Behrendt K, Groene O. Mechanisms and effects of public reporting of surgeon outcomes:
A systematic review of the literature. Health Policy (2016), http://dx.doi.org/10.1016/j.healthpol.2016.08.003
 4

HEAP-3604; No. of Pages 11

G Model
A systematic review of the literature. Health Policy (2016), http://dx.doi.org/10.1016/j.healthpol.2016.08.003
Please cite this article in press as: Behrendt K, Groene O. Mechanisms and effects of public reporting of surgeon outcomes:

Table 2
Included studies and outcomes of interest.

Studies included Outcome of interest Quality

score

Name Year Country Identified in Study Patient Complementary Gaming Surgical Discuss Discuss Organisational Intrinsic Caseload/ Leaving
systematic type selec- treatments train- data data with changes moti- volume profes-
review or tion ing with GPs and vation sion
snoballing patients CCGs

Apolito et al. [43] 2008 USA, NY Sno CC x 34

Bridgewater et [42] 2007 UK Sno IST x 35
Burack et al. [46] 1999 USA, NY Sno Q x x 31

ARTICLE IN PRESS
K. Behrendt, O. Groene / Health Policy xxx (2016) xxx–xxx
Chassin [49] 2002 USA, NY Sys Q x 19
Dranove et al. [32] 2003 USA, NY Sys DID x x 28
Dziuban et al. [48] 1994 USA, NY Sno C x 23
Glance et al. [31] 2007 USA, NY Sno C x 35
Hannan et al. [45] 1995 USA, NY Sno C x x 34
Hannan et al. [30] 1997 USA; NY Sno C x 34
Hannan et al. [27] 1997 USA, NY Sys Q x x 30
Jha and Epstein [51] 2006 USA, NY Sys Q x 25
Joynt et al. [44] 2012 USA, NY Sno ITS/DID x 34
Mass
Penn
Khan et al. [70] 2007 UK Sys ITS x 31
Kolstad [50] 2013 USA, Sys ITS x 29
Penn
Li et al. [35] 2009 USA, Cal Sno BA x 34
Maytham and 2010 UK Sys Q x (x) 29
Kessaris [39]
Moscucci et al. [33] 2005 USA, NY Sno C x 26
Mukamel et al. [38] 2006 USA, NY Sys C x 26
Narins et al. [28,40] 2005 USA; NY Sys Q x x 29
Omoigui et al. [45] 1996 USA; Sno ITS/CBA x 33
Ohio
Peterson and DeLong 1998 USA; NY Sno ITS x 27
[34]
Romano et al. [36] 2011 USA, Cal Sno BA x 33
Schneider and 1996 USA; Sno Q x 32
Epstein [29] Penn
Sherman et al. [71] 2013 USA Sys Q x x 31
Werner et al. [37] 2005 USA; NY Sys ITS/DID x 32

NY = New York; Penn = Pennsylvania; Mass = Massachusetts; sno = snowballing; sys = systematic review; ITS = Interrupted time series; Q = survey or interview; C = cohort study; CC = case-control study;
DID = difference-in-difference analysis; CBA = controlled before-after study; CR = case report; BA = before-after design
 Table 3
Patient selection and access to care.

HEAP-3604; No. of Pages 11

G Model
A systematic review of the literature. Health Policy (2016), http://dx.doi.org/10.1016/j.healthpol.2016.08.003
Please cite this article in press as: Behrendt K, Groene O. Mechanisms and effects of public reporting of surgeon outcomes:

Author Apolito et al. [43] Bridgewater et al. Burack et al. [46] Dranove et al. Glance et al. [31] Hannan et al. [30] Hannan et al. [27] Joynt et al. [44] Kolstad [50] Li et al. [35]
[42] [32]
Year 2008 2007 1999 2003 2007 1997 1997 2012 2013 2009
Country USA, NY UK USA, NY USA, NY USA, NY USA, NY USA, NY USA, USA, Penn USA, Cal
NY + Mass + Penn
Study type CC ITS Q DID C C Q ITS/DID ITS BA
Level Pat Hosp Sur Hosp, Pat Pat Sur Sur Pat Sur Pat
Results ↓ Rates of ↑ Numbers and 70% reported no ↓ Relative illness High risk CABG No sign of limited 38% NY ↓ Likely to get PCI Komplex 27% less patients
coronary percentages of change in severity among patients were access to surgery cardiologists with acute MI, economic model, underwent CABG
angiography for low, high and practice, but 64% PR patients, as found to be more or gaming stated the report especially with in which there in 2006 than
patients with very high risk claimed to have well as ↓ in the likely to recieve had an influence ST-elevated MI or was a minor 2003; the case
shock and acute patients refused at least within-hospital surgery from a on their referral cardiogenic effect of mix stayed
MI; ↑ waiting significantly one CABG patient heterogeneity of high quality practice, but only shock or cardiac low-performing unchanged
times CABG. ↑ increased because of PR. AMI patients. ↑ surgeon 6% said they arrest. ↔ Overall surgeons to avoid however.
Mortality among More high risk High risk patiens were very much mortality in high-risk Observed MR
NY patiens who patients were attending influenced by the reporting and patients but this was lower in the

ARTICLE IN PRESS
K. Behrendt, O. Groene / Health Policy xxx (2016) xxx–xxx
were not refused CABG teaching reports non-reporting only accounted high-risk
revascularized surgery than hospitals. ↑ states. In for 5% of the quartiles in 2006
surgery for aortic Waiting times for Massachusets, imporvement of than 2003. All 4
dissection, which CABG and PCI. odds of recieving quality outlier surgeons
is not publicly Increased PCI ↓ after PR had of 2006 met the
reported expenditures for begun expected MR in
AMI and CABG 2006
patients
Effect — + − — + 0 − – − 0

Author Maytham and Moscucci et al. Mukamel et al. Narins et al. Omoigui et al. Peterson and Romano et al. Schneider and Sherman et al. Werner et al. [37]
Kessaris [39] [33] [38] [28,40] [45] DeLong [34] [36] Epstein [29] [71]
Year 2010 2005 2006 2005 1996 1998 2011 1996 2013 2005
Country UK USA, NY USA, NY USA, NY USA, Ohio USA, NY USA, Cal USA, Penn USA, IL USA, NY
Study type Q C C Q ITS/CBA IST BA Q Q ITS/DID
Level Sur Pat Pat Sur Pat Pat Pat Sur Sur Pat
Results ↓ Surgeons ↓ Patients with Whites were >79% agreed that Patients referred A smaller High-mortality 59% of Majorities show Before PR was
claiming they AMI and more likely to (1) knowledge of from NY had a percentage of hospitals cardiologists concern with introduced,
will avoid cardiogenic have access to RAMR PR were sicker; ↑ patients from NY operated on found it more or surgeons white patients
high-risk shock underwent low-RAMR influenced average yearly recieved CABG slightly less much more refusing were more likely
patients, ↑ PCI, ↓ rates of surgeons. Level decision to volume of surgery outside high-risk difficult to found high-risk to recieve CABG
claiming they comorbidities. of access also perform referrals from of NY (from patients (25% a surgeon to patients, that than black
will not change Unadjusted MR ↓ dependent on angioplasy, (2) NY, while other 12.5% to 11.3%). decrease in operate on a high risk patients patients. In the 9
practice, ↑ but after health insurance patients might referrals ↓, from After PR was expected severly ill might be shifted years following
thinking PR will adjustment the type (for HMO not get it due to 1989 MR among introduced, the mortality) patient; 66% of to safety-net the first PR, the
improve difference not patients worse PR RAMR; (3) NY referrals ↑ odds of an older cardiac surgeons hospitals racial disparities
outcomes significant any than for FFS RAMR PR than from other patient with AMI were less or were wider. Then
more patients) influences states or Ohio to recieve much less willing it got back to
decision on surgery increased to operate on pre-PR level
wether to significantly severly ill
intervene on patients
patients with (compared to 3
high expected years earlier
mortality
Effect − – − — — + − − − ±

NY = New York; Penn = Pennsylvania; Mass = Massachusetts; IL = Illinios;sno = snowballing; sys = systematic review; ITS = Interrupted time series; Q = survey or interview; C = cohort study; CC = case-control study;

5
DID = difference-in-difference analysis; CBA = controlled before-after study; Case = case report; BA = before-after design; Pat = patient; Sur = surgeon; Hos = hospital; RAMR = risk-adjusted mortality rate.
G Model
HEAP-3604; No. of Pages 11 ARTICLE IN PRESS
6 K. Behrendt, O. Groene / Health Policy xxx (2016) xxx–xxx

Fig. 1. Flow chart of study selection process.

3.3. Adverse selection as a potential consequence of of comorbidities in AMI patients, while Omoigui et al.
public reporting [45] found that patients referred from NY to Ohio were
sicker than other referrals or patients from Ohio itself.
Of the studies addressing adverse selection, quality and Glance et al. Hannan et al. and Peterson et al. on the
strength of evidence varied considerably: For example, other hand could not identify such effects in their studies
Hannan et al. [27] reported a minor effect of PR, with [30,31,34].
40% of NY cardiologists influenced by PR, but only 6% Results from California are mixed: Li et al. [35] and
claiming to be strongly influenced. Narins et al. [28] and Romano et al. [36] found no evidence of risk selection
Schneider and Epstein [29] both reported a majority of when they looked at patient case mix, but argue that high-
cardiac surgeons being less likely to accept a high risk mortality hospitals might have avoided high-risk patients.
patient. Hannan et al. [30] and Glance et al. [31] found in Moreover, Werner et al. [37] found a transient negative
a sample of 31 hospitals and 87 surgeons that high risk effect on access to CABG surgery for African American
patients were less likely operated on by surgeons with patients, as they were less likely to receive CABG surgery.
high performance. This is consistent with the findings of In the consecutive year (1996), Mukamel et al. [38] found
Dranove et al. [32], we found that the relative illness sever- that non-whites are more likely to receive surgery from
ity among AMI patients in NY and Pennsylvania declined, low-quality surgeons. Overall, although study results are
Moscucci et al. [33] who detected a decline in the rates contradictory in details, there was some evidence that PR

Please cite this article in press as: Behrendt K, Groene O. Mechanisms and effects of public reporting of surgeon outcomes:
A systematic review of the literature. Health Policy (2016), http://dx.doi.org/10.1016/j.healthpol.2016.08.003
G Model
HEAP-3604; No. of Pages 11 ARTICLE IN PRESS
K. Behrendt, O. Groene / Health Policy xxx (2016) xxx–xxx 7

reduced access to surgery for very sick patients and patients support staff was identified, the other two identified the
from different ethnic backgrounds. approach to the treatment of emergency cases as the cause
However, adverse selection is a topic that is difficult of the high mortality rate. Strikingly, Dziuban reported an
to assess, prone to a number of biases, and vulnerable increased team cooperation as one of the salient positive
to the availability of high quality data. Maythams and outcomes of the investigation into the high mortality rate.
Kessaris study [39] suffered from potential responder bias The reports demonstrate that the reasons for high MRs are
due to low response rate of 50%, and, generally, the sur- complex and may not only be attributable to the surgeons
veys conducted represent weaker evidence due to social themselves.
acceptability bias. Narins et al. [40] reported that 88% A proposition put forward by the proponents of public
agreed or strongly agreed that “physicians may report reporting is that learning about their performance com-
higher risk conditions to improve their outcomes”, while pared to peers might lead to increased intrinsic motivation
57% of respondants in Brown et al.’s study felt that sur- of surgeons to perform better as opposed to extrinsic moti-
geons and hospitals can manipulate the data [41]. Further, vation due to remuneration. We did not find research to
studies fail to clearly attribute exposure or provide con- support this proposition. Economic models exist to demon-
trol groups [42], suffer from missing data [43] or were strate the influence of intrinsic motivation on surgeon
conducted based on patient [44] or provider subgroups performance [50], however, such models rely on a large
[45]. Moreover, even if adverse selection took place, this number of assumptions. A more pertinent implication of
may not necessarily suggest lower quality of care: it has surgeon reports might be that low-performing surgeons
been suggested that the less invasive percutaneous coro- cease practicing. Jha et al. [51] and Hannans et al. [47]
nary intervention (PCI) might be a substitute treatment studies indicate that a larger percentage of bottom quartile
for CABG patients being considered too sick to be offered surgeons stopped practice in NY than top quartile. Simi-
surgery. Dranove et al. [32] looked at PCI as a substitute for larly, in Hannans study, the decline in mortality rate was
CABG in NY, where only CABG outcomes were reported. partly attributable to low-volume surgeons with high mor-
They also found that percutaneous transluminal coronary tality rate that ceased to practice [47].
angioplasty (PTCA) procedure numbers decreased and that
hospitals might have taken general measures to avoid 4. Discussion
high risk patients. Their dataset was, however, restricted
to Medicare claims. Consequently, because of the diverse The aim of this study was to assess the international
nature of study designs applied, systems and patient groups evidence on public reporting with a special focus on the
studied, the evidence of consequences of public reporting two contrasting theses surrounding the introduction of
is limited and its impact on quality of care outcomes needs the UK surgeon-level outcomes public reporting initiative:
to be interpreted with great caution. improved quality through better reflection on outcomes by
patients, GPs, CCGs or managers and adverse consequences
3.4. Use of data by providers to improve quality of care of public reporting on patient selection.
We found a substantial literature assessing potential
A key claim of “Offer 2” is that public reports could be and observed adverse consequences of public report-
used by the public to choose a provider, by GPs to inform ing and less literature addressing the mechanisms that
referrals and by commissioners to contract for outcomes. translate public reporting into quality improvement initia-
This review found limited evidence for such interactions tives, with the largest effects to be expected amongst the
in health systems with a history of surgeon reporting. lowest performing providers. The findings need to be inter-
Two surveys assessed the usage of the data in interactions preted with great caution. The variety of methodological
between surgeons and patients. Burack et al. found that approaches (design with/without control group or surveys
29% discussed the dataless than weekly with their patients with risk of bias) make it difficult to reach a final conclu-
and 44% frequently (less than weekly) with colleagues sion on the presence and extent of adverse selection of
[46]. Moreover, the study suggests that understanding of patients according to risk [8]. Questions have also been
the methods underlying the published outcomes was lim- raised concerning the reliability of underlying data sources,
ited amongst the surgeons. In the study by Hannan et al. the extent and impact of missing data, the comparabil-
[47], 22% agreed that they routinely discussed the infor- ity of clinical and administrative sources, and variations
mation in the report with their patients. Based on the in approaches to coding the data. Moreover, the mortality
literature, only a minority of surgeons seem to discuss risk of a patient is influenced by the patient (e.g. sever-
their outcomes with patients on a regular basis. It has to ity of illness or comorbidities) as well as the treatment.
be taken into account that all these studies were surveys Since the beginning of PR, the appropriateness of the risk-
from the USA; there was no data available yet from UK adjustment procedures has been questioned and criticized.
since the introduction of “Offer 2”. However, investiga- The NYCS has updated their risk-adjustment several times
tions and organizational changes amongst hospitals with [52,53] but research suggests that different methods of risk
suspicious surgeon outcomes could be a relevant implica- adjustments will lead to different results [54,55].
tion of public reporting. Dziuban et al. [48] and Chassin More fundamentally, however, is the question to what
[49] interviewed staff from hospitals which were published extent the literature (mostly derived from the experience
as having high mortality rates. All hospitals were sub- of public reporting of cardiac surgeon outcomes in NY state)
jected to special measures and required to undergo a case supports a generalizability of the findings to the UK context.
review. In two cases a lack of dedicated cardiac surgery In the US healthcare system with its many health insur-

Please cite this article in press as: Behrendt K, Groene O. Mechanisms and effects of public reporting of surgeon outcomes:
A systematic review of the literature. Health Policy (2016), http://dx.doi.org/10.1016/j.healthpol.2016.08.003
G Model
HEAP-3604; No. of Pages 11 ARTICLE IN PRESS
8 K. Behrendt, O. Groene / Health Policy xxx (2016) xxx–xxx

ances and where surgeons are not necessarily employed of more than 5000) who were found to be negative out-
by a hospital [56], PR might offer a stronger incentive for liers in the UK so far faced an individual review as well
patient risk selection than in the UK, where surgeons are as a review of their whole unit [62]. Organizational mea-
less dependent on the market. Thus, this evidence might sures were taken rather than re-training of the surgeon
not be applicable to the UK NHS (the only UK study about [63–65]. On the other hand, a clear accountability can also
patient access to care could not found evidence of gaming). be considered as a requirement to improve team efforts
In order to assess the transferability of the results, the pro- [66].
cesses through which public reporting should lead to the The absence of controlled studies means that is dif-
desired effect will be elucidated in more detail below. Sub- ficult to separate the effect of public reporting from
sequently we will present some factors that policy makers other strategies. There is also evidence that non-PR
may consider when devising a surgeon outcome initiative feedback can improve quality as well: in New England
in their own country. outcomes feedback, hospital visits and training in qual-
Whether public reporting can be an incentive for qual- ity improvement were combined. The mortality rate went
ity improvement can not be easily answered. From a down as various measures were taken at the hospi-
behavioural economics perspective, PR data can be seen tal level—multiprofessional reviews on clinical processes,
as a ‘nudge’ [57], that means providing feedback to intrin- protocol implementation, reviews of deaths, hiring dedi-
sically motivated surgeons, who will then act accordingly cated cardiac surgery staff, training and implementation
and try to improve [58]. Kolstad’s study showed a higher of checklists [67]. This then also raises the question of
impact of intrinsic motivation on NY surgeons’ quality whether an individual surgeon represents the right unit
improvement than their revenue had [50]. In fact surgeons of analysis. In the USA, where surgeons might not be
were willing to forego parts of their revenue to improve employed by the hospital, and in cardiac surgery where
quality. An alternative behavioural economics mechanism procedure volume is high, the answer might be different
for public reporting to drive quality improvement is via from in the UK, and for a different procedure (say, upper
cognitive bias that can come into play leading individu- gastro-intestinal surgery, with its much lower caseload
als to fear deviating from accepted standards more than per surgeon). Whether the statistical power is sufficient
appreciating the potential to improve their care [59]. The to detect variations in quality outcomes is an issue we
prospect of being ‘named & shamed’ might lead surgeons to have raised previously, and a prerequisite for public reports
change their choice of patients in order to avoid becoming to unfold their effects [68]. This depends mostly on the
an outlier, as discussed above. According to Kolstad, moti- expected outcome, the volume of cases per surgeon and
vation is highest when expected and observed performance the number of surgeons performing the operation. These
are far apart, whether better or worse than expected, yet seemingly technical details are a good example to illustrate
the UK PR data as presented online mainly distinguishes the limitations of transferring research evidence from one
between ‘OK’ performance and ‘negative outlier’. As the setting to another.
media focuses on the negative outliers, this is a plausi- Ideally, in order to appraise research evidence to sup-
ble mechanism for PR functioning. Chassin showed that port translating a health system strategy from one setting
low performers are more likely to leave the practice, so to another one should a-priori consider the key factors
the PR might indeed have had an effect [49]. Transparency that determine the translation. At the conceptual level (as
and consumer choice are also often mentioned as rea- reflected in the “generic quality criteria for cross-country
sons to justify PR efforts [6]. Some doctors go as far as comparisons of health systems and policies”), this implies
claiming that consent to a procedure can only be given, appropriate use of theory, explicit selection of country
and be valid, if the patient is aware of the surgeon’s per- comparator, rigour in the comparison design, attention to
formance [60]. Survey results from the USA suggest that the complexity of the cross-national comparison, rigour of
mortality data is not often discussed between doctors and the research method and a clear contribution of knowledge
patients. This finding is in line with other research that for both theory development and policy learning [69]. In
shows that patients found it difficult to understand out- the context of public reporting of surgeon outcomes, we
come measures and value more highly recommendations have outlined in this review a wide range of the factors
from family and friends [10]. In the UK, where patients underlying the effectiveness of public reporting, a diverse
traditionally have less choice, data usage might be even set of theories to postulate the expected results and to
less significant and, should this be the case, PR a weaker formulate the implicit causal pathways, all of which will
incentive. determine to what extent the published evidence is appro-
An assessment of whether public disclosure of surgeon priate to justify the proposed health system strategy, in this
outcomes can be translated to other settings also needs case the formulation of “Offer 2”.
to consider the health care delivery system in which the Based on the published literature and our analysis pre-
surgeon operates. In this sense, public reporting can be con- sented here, we suggest specific criteria for cross-country
sidered as a complex improvement intervention of which learning from public reporting initiatives, which may sup-
the active ingredient is not well understood. For example, port other countries to assess the potential impact of public
case studies of US hospitals that faced high MR show evi- reporting of surgeon outcomes and to devising a public
dence that the published outcomes are rarely attributable reporting strategy. These criteria cover health system fac-
to the surgeon alone. Organisational structures and a lack of tors, surgeon factors, data related factors, patient factors,
dedicated staff and procedures can also have a large impact and organizational factors (Box 1).
on patient outcomes [61]. Similarly, the three surgeons (out

Please cite this article in press as: Behrendt K, Groene O. Mechanisms and effects of public reporting of surgeon outcomes:
A systematic review of the literature. Health Policy (2016), http://dx.doi.org/10.1016/j.healthpol.2016.08.003
G Model
HEAP-3604; No. of Pages 11 ARTICLE IN PRESS
K. Behrendt, O. Groene / Health Policy xxx (2016) xxx–xxx 9

Box 1: Factors impacting on the transferability 5. Limitations

of the evidence underlying the public reporting
of surgeon outcomes. This review has several limitations. Due to the diverse
Generic health system factors: range of studies and outcomes included, a meta-analysis
could not be conducted. Some studies were not included
- Coverage and access: do all patients have equal due to lack of access, or due to not being published in a peer-
access to the procedure? reviewed journal. The studies included showed varying
- Choice: do patients have a choice of the provider quality. No study addressed the the question of this review
(hospital, surgeon)?
entirely, so that different pieces of information had to be
- Existing public reporting system: has some data (say
put together and evaluated in relation to each other. Pub-
hospital level outcomes) already been puiblished?
- What is the role of the GP in the referral system (gate- lic reporting in the literature so far has mostly addressed
keeping, direct referral) and are there other means of mortality outcomes. The public reporting mechanisms will
patient referral or relevance in the context of public be the same for other outcomes, yet the effectiveness of
reporting? mechanisms might differ. For example, patient reported
- Purchasing mechanisms: how are services con- outcomes might offer better metric properties (in the con-
tracted for—based on volume or based on outcomes. text of surgeons outcomes, where denominators are small)
- Number of surgeons conducting the procedure: how and might also lead for a more active engagement of
many surgeons care conducting the operation in the patients [68]. Finally, most studies (n = 22) found were from
health system.
the USA with limited applicability to a UK context, there-
- How will the results be published: is there a single
entry portal? fore, we have emphasized the transferability of the results
- Does the single entry portal contain patient educa- to different health system contexts.
tion material beyond the outcome data?
6. Conclusions
Surgeon factors:
PR represents a well established but not yet fully under-
- Employed or independent: are surgeons on a fixed- stood approach to improving quality of healthcare. As
appointment with a hospital or clinic, or do they
most research on PR is from a US context, the findings
freely operate in different organizations.
- Competition: do surgeons compete for patients? cannot easily be transferred to the potential use by GPs
- Type and volume of procedures covered: which pro- and clinical commissioning groups in the NHS (or health
cedure should be publicly reported? Depending on care providers and purchasing agencies in different health
the case volume and accountability there will be systems). Therefore, if public reporting is introduced at
major implications for the public reporting initiative. population level, access to surgery and potential nega-
tive effects (adverse selection) should be monitored. Policy
Data related factors: makers devising a strategy to publish surgeon outcomes in
their own health systems may benefit from contemplating
- Is quality data available (clinical registry or high qual-
the factors presented here that address the transferability
ity administrative data)?
of the evidence underlying the public reporting of surgeon
- Have appropriate risk adjustment mechanisms for
the procedure been developed? outcomes and specifically, whether prevailing health sys-
- What is the median delay from procedure to regis- tem conditions are supportive of such a strategy.
tration to analysis to public report?
Appendix A. Supplementary data
Patient factors:
Supplementary data associated with this arti-
- Does the entire population have easy access to the cle can be found, in the online version, at
Internet?
http://dx.doi.org/10.1016/j.healthpol.2016.08.003.
- How is the health literacy level amongst the target
population?
- Are there major regional variations in access surgery References
(limiting patient choice)?
[1] Marshall MN, Shekelle PG, Davies HTO, Smith PC. Public reporting on
quality in the United States and the United Kingdom. Health Affairs
Organizational and professional:
2003;22:134–48, http://dx.doi.org/10.1377/hlthaff.22.3.134.
[2] Department of Health. Learning from Bristol: the DH response
- Do hospitals/clinics have the capacity for team and to the report of the public inquiry into children’s heart
hospital level discussions to learn from outlying per- surgery at the Bristol Royal Infirmary 1984–1995, executive
formers? summary; 2002, 9 https://www.gov.uk/government/uploads
- Is there a national agency in charge of liaising with /system/uploads/attachment data/file/273320/5363.pdf.
low performers to ensure that appropriate investiga- [3] NHS England. Putting patients first—the NHS England business plan
for 2013/14–2015/16. NHS England; 2013.
tions, where appropriate, are initiated? [4] NHS. NHS choices; 2014 https://www.nhs.uk/service-search
/performance/Consultants#view-the-data.
[5] Hannan EL, Kilburn H, Racz M, Shields E, Chassin MR.
Improving the outcomes of coronary artery bypass
surgery in New York State. JAMA 1994;271:761–6,
http://dx.doi.org/10.1001/jama.1994.03510340051033.

Please cite this article in press as: Behrendt K, Groene O. Mechanisms and effects of public reporting of surgeon outcomes:
A systematic review of the literature. Health Policy (2016), http://dx.doi.org/10.1016/j.healthpol.2016.08.003
G Model
HEAP-3604; No. of Pages 11 ARTICLE IN PRESS
10 K. Behrendt, O. Groene / Health Policy xxx (2016) xxx–xxx

[6] Marshall MN. Dying to know: public release of information about [27] Hannan EL, Stone CC, Biddle TL, DeBuono BA. Public release of car-
quality of health care. London: RAND, The Nuffield Trust; 2000. diac surgery outcomes data in New York: what do New York state
[7] Berwick DM, James B, Coye MJ. Connections between qual- cardiologists think of it? American Heart Journal 1997;134:1120–8.
ity measurement and improvement. Medical Care 2003;41:I30–8, [28] Narins CR, Dozier AM, Ling FS, Zareba W. The influence of
http://dx.doi.org/10.1097/00005650-200301001-00004. public reporting of outcome data on medical decision mak-
[8] Werner RM, Asch DA. The unintended consequences of pub- ing by physicians. Archives of Internal Medicine 2005;165:83–7,
licly reporting quality information. JAMA 2005;293:1239–44, http://dx.doi.org/10.1001/archinte.165.1.83.
http://dx.doi.org/10.1001/jama.293.10.1239. [29] Schneider EC, Epstein AM. Influence of cardiac-surgery performance
[9] Zwijnenberg NC, Hendriks M, Damman OC, Bloemendal E, Wendel reports on referral practices and access to care. A survey of cardiovas-
S, de Jong JD, et al. Understanding and using comparative healthcare cular specialists. New England Journal of Medicine 1996;335:251–6,
information; the effect of the amount of information and consumer http://dx.doi.org/10.1056/NEJM199607253350406.
characteristics and skills. BMC Medical Informatics and Decision [30] Hannan EL, Siu AL, Kumar D, Racz M, Pryor DB, Chassin MR. Assess-
Making 2012;12:101, http://dx.doi.org/10.1186/1472-6947-12-101. ment of coronary artery bypass graft surgery performance in New
[10] Damman OC, Hendriks M, Rademakers J, Delnoij DMJ, Groe- York. Is there a bias against taking high-risk patients? Medical Care
newegen PP. How do healthcare consumers process and 1997;35:49–56.
evaluate comparative healthcare information? A qualitative [31] Glance LG, Dick A, Mukamel DB, Li Y, Osler TM, et al. Are
study using cognitive interviews. BMC Public Health 2009;9:423, high-quality cardiac surgeons less likely to operate on high-
http://dx.doi.org/10.1186/1471-2458-9-423. risk patients compared to low-quality surgeons? Evidence from
[11] Schauffler HH, Mordavsky JK. Consumer reports New York State. Health Services Research 2008;43:300–12,
in health care: do they make a difference? http://dx.doi.org/10.1111/j.1475-6773.2007.00753.x.
Annual Review of Public Health 2001;22:69–89, [32] Dranove D, Kessler D, McClellan M, Satterthwaite M. Is more infor-
http://dx.doi.org/10.1146/annurev.publhealth.22.1.69. mation better? The effects of “report cards” on health care providers.
[12] Dixon A, Robertson R, Appleby J, Burge P, Devlin N, Magee H. Patient Journal of Political Economy 2002;111:555–88.
choice. Kings Fund; 2010. p. 215. [33] Moscucci M, Eagle KA, Share D, Smith D, De Franco AC,
[13] Schneider EC, Epstein AM. Use of public performance reports: O’Donnell M, et al. Public reporting and case selection for per-
a survey of patients undergoing cardiac surgery. JAMA cutaneous coronary interventions: an analysis from two large
1998;279:1638–42. multicenter percutaneous coronary intervention databases. Jour-
[14] Coulter A. Do patients want a choice and does it work? BMJ nal of the American College of Cardiology 2005;45:1759–65,
2010;341:c4989, http://dx.doi.org/10.1136/bmj.c4989. http://dx.doi.org/10.1016/j.jacc.2005.01.055.
[15] Marshall MN, Romano PS, Davies HTO. How do we maximize [34] Peterson E, DeLong E. The effects of New York’s bypass surgery
the impact of the public reporting of quality of care? Interna- provider profiling on access to care and patient outcomes
tional Journal for Quality in Health Care 2004;16(Suppl. 1):i57–63, in the elderly. Journal of the American College of Cardiology
http://dx.doi.org/10.1093/intqhc/mzh013. 1998;32:993–9.
[35] Li Z, Carlisle DM, Marcin JP, Castellanos LR, Romano PS, Young
[16] Williamson OE. The economic institutions of capitalism. New York:
JN, et al. Impact of public reporting on access to coronary
Free Press; 1985, http://dx.doi.org/10.2307/2392889.
artery bypass surgery: the California Outcomes Report-
[17] Holmstrom B, Milgrom P. Multitask principal-agent analy- ing Program. Annals of Thoracic Surgery 2010;89:1131–8,
ses: incentive contracts, asset ownership, and job design. http://dx.doi.org/10.1016/j.athoracsur.2009.12.073.
Journal of Law, Economics, and Organization 2001;7:24–52.
[36] Romano P, Marcin JP, Dai JJ, Yang XD, Kravitz RL, Rocke DM, et al.
http://www.jstor.org/stable/764957?seq=1#age scan tab contents
Impact of public reporting of coronary artery bypass graft surgery
[accessed 02.07.15].
performance data on market share, mortality, and patient selection.
[18] Rosenthal MB, Fernandopulle R, Song HR, Landon B. Paying for qual-
Medical Care 2011;49:1118–25.
ity: providers’ incentives for quality improvement. Health Affairs
[37] Werner RM, Asch DA, Polsky D. Racial profiling: the
2004;23:127–41, http://dx.doi.org/10.1377/hlthaff.23.2.127.
unintended consequences of coronary artery bypass
[19] Hanson K. Delivering health services: incentives and information in graft report cards. Circulation 2005;111:1257–63,
supply-side innovations. In: Smith RD, Hanson K, editors. Health sys- http://dx.doi.org/10.1161/01.CIR.0000157729.59754.09.
tems in low- and middle-income countries: an economic and policy
[38] Mukamel DB, Murthy AS, Weimer DL. Racial differences in access
perspective. Oxford: Oxford University Press; 2011. p. 103–24.
to high-quality cardiac surgeons. American Journal of Public Health
[20] Surgeons ask NHS England to rethink policy of publish-
2000;90:1774–7.
ing patients’ death rates. The Guardian; 2016, 30.1.2015,
[39] Maytham G, Kessaris N. A change in opinion on surgeon’s perfor-
https://www.theguardian.com/society/2015/jan/30/surgeons-
mance indicators. Interactive Cardiovascular and Thoracic Surgery
nhs-england-patients-death-rates-bruce-keogh-jeremy-hunt-health
2011;12:586–9, http://dx.doi.org/10.1510/icvts.2010.257857.
[accessed 3.8.2016].
[21] Schardt C, Adams MB, Owens T, Keitz S, Fontelo P. Utilization of the [40] Narins CR, Dozier AM, Ling FS, Zareba W. The influence of
PICO framework to improve searching PubMed for clinical ques- public reporting of outcome data on medical decision mak-
tions. BMC Medical Informatics and Decision Making 2007;7:16, ing by physicians. Archives of Internal Medicine 2005;165:83–7,
http://dx.doi.org/10.1186/1472-6947-7-16. http://dx.doi.org/10.1001/archinte.165.1.83.
[22] Moher D, Liberati A, Tetzlaff J, Altman DG. Preferred report- [41] Brown DL, Epstein AM, Schneider EC. Influence of cardiac surgeon
ing items for systematic reviews and meta-analyses: report cards on patient referral by cardiologists in New York state
the PRISMA statement. PLoS Medicine 2009;6:e1000097, after 20 years of public reporting. Circulation: Cardiovascular Quality
http://dx.doi.org/10.1371/journal.pmed.1000097. and Outcomes 2013;6:643–8.
[42] Bridgewater B, Grayson AD, Brooks N, Grotte G, Fabri BM, Au J, et al.
[23] Hawker S, Payne S, Kerr C, Hardey M, Powell J. Appraising
Has the publication of cardiac surgery outcome data been associ-
the evidence: reviewing disparate data systemati-
ated with changes in practice in northwest England: an analysis of
cally. Qualitative Health Research 2002;12:1284–99,
25,730 patients undergoing CABG surgery under 30 surgeons over
http://dx.doi.org/10.1177/1049732302238251.
eight years. Heart 2007;93:744–8.
[24] Totten A, Wagner J, Tiwari A. Public reporting as a quality improve- [43] Apolito R, Greenberg M, Menegus M, Lowe AM, Sleeper L, Gold-
ment strategy closing the quality gap: revisiting the state of the berger MH, et al. Impact of the New York State Cardiac Surgery and
science. Agency for Healthcare Research & Quality 2012;(July). Percutaneous Coronary Intervention Reporting System on the man-
[25] Cochrane Effective Practice and Organisation of Care Group. EPOC- agement of patients with acute myocardial infarction complicated
specific resources for review authors | Cochrane Effective Practice by cardiogenic shock. American Heart Journal 2008;155:267–73,
and Organisation of Care Group. Nor Knowl Cent Heal Serv; 2015. http://dx.doi.org/10.1016/j.ahj.2007.10.013.
http://epoc.cochrane.org/epoc-specific-resources-review-authors
[44] Joynt KE, Blumenthal DM, Orav EJ, Resnic FS, Jha AK. Association of
[accessed 03.08.15].
public reporting for percutaneous coronary intervention with uti-
[26] Groene O, Botje D, Suñol R, Lopez MA, Wagner C. A systematic review
lization with acute myocardial infarction. JAMA 2012:02115.
of instruments that assess the implementation of hospital quality
[45] Omoigui NA, Miller DP, Brown KJ, Annan K, Cosgrove D, Lytle B,
management systems. International Journal for Quality in Health
et al. Outmigration for coronary bypass surgery in an era of pub-
Care 2013;25:525–41, http://dx.doi.org/10.1093/intqhc/mzt058.

Please cite this article in press as: Behrendt K, Groene O. Mechanisms and effects of public reporting of surgeon outcomes:
A systematic review of the literature. Health Policy (2016), http://dx.doi.org/10.1016/j.healthpol.2016.08.003
G Model
HEAP-3604; No. of Pages 11 ARTICLE IN PRESS
K. Behrendt, O. Groene / Health Policy xxx (2016) xxx–xxx 11

lic dissemination of clinical outcomes. Circulation 1996;93:27–33, [58] Kahneman D, Tversky A. Prospect theory: an analysis
http://dx.doi.org/10.1161/01.CIR.93.1.27. of decision under risk. Econometrica 2007;47:263–92,
[46] Burack JH, Impellizzeri P, Homel P, Cunningham JNJ. Public report- http://dx.doi.org/10.2307/1914185.
ing of surgical mortality: a survey of New York State cardiothoracic [59] Bevan G. Performance measurement of “knights” and “knaves”:
surgeons. Annals of Thoracic Surgery 1999;68:1192–5. differences in approaches and impacts in British countries after
[47] Hannan EL, Siu AL, Kumar D, Kilburn H, Chassin MR. The devolution. Journal of Comparative Policy Analysis 2010;12:33–56,
decline in coronary artery bypass graft surgery mortality in New http://dx.doi.org/10.1080/13876980903076187.
York State. The role of surgeon volume. JAMA 1995;273:209–13, [60] Clarke S, Oakley J. Informed consent and surgeons? Perfor-
http://dx.doi.org/10.1001/jama.1995.03520270043029. mance. Journal of Medicine and Philosophy 2004;29:11–35,
[48] Dziuban SW, McIlduff JB, Miller SJ, Dal Col RH. How http://dx.doi.org/10.1076/jmep.29.1.11.30415.
a New York cardiac surgery program uses outcomes [61] Pace N. Surgeons should beware of plans to
data. Annals of Thoracic Surgery 1994;58:1871–6, print individual performance. BMJ 2002;324:979,
http://dx.doi.org/10.1016/0003-4975(94)91730-2. http://dx.doi.org/10.1136/bmj.324.7343.974.
[49] Chassin MR. Achieving and sustaining improved quality: lessons [62] Taylor R. Surgeon performance data “misses the mark”. BBC News;
from New York state and cardiac surgery. Health Affairs 2014. http://www.bbc.co.uk/news/health-30057602 [accessed
2002;21:40–51, http://dx.doi.org/10.1377/hlthaff.21.4.40. 21.10.15].
[50] Kolstad JT. Information and quality when motivation is intrinsic: evi- [63] Perry L. Just three surgeons named as having poor death rates as NHS
dence from surgeon report cards. The American Economic Review publishes data for more than 5,000 doctors. Mail 2014:17.
2013;103:2875–910. [64] Ben James. Brighton heart surgeon named as having death rates
[51] Jha AK, Epstein AM. The predictive accuracy of the New York State above national average. Arg; 2014. http://www.theargus.co.uk/
coronary artery bypass surgery report-card system. Health Affairs news/11616753Brighton heart surgeon named as having death rates
2006;25:844–55, http://dx.doi.org/10.1377/hlthaff.25.3.844. above national average/ [accessed 27.08.15].
[52] Peterson ED. The need for “compassionate provider profiling”: [65] Donnelly L. Just three surgeons named as
refining risk assessment for percutaneous coronary intervention. having high death rates. Telegraph; 2014.
Journal of the American College of Cardiology 2011;57:912–3, http://www.telegraph.co.uk/news/nhs/11240241/PIC-AND-HOLD-Just
http://dx.doi.org/10.1016/j.jacc.2010.10.022. -three-surgeons-named-as-having-high-death-rates.html [accessed
[53] Resnic FS, Normand S-LTLT, Piemonte TC, Shubrooks SJ, Zelevinsky 27.08.15].
K, Lovett A, et al. Improvement in mortality risk prediction after [66] Beed M, Brindley PG. Publication of surgical outcomes-data: whose
percutaneous coronary intervention through the addition of a com- team are we on? British Journal of Anaesthesia 2014;112:615–7,
passionate use variable to the National Cardiovascular Data Registry http://dx.doi.org/10.1093/bja/aet432.
CathPCI Dataset: a study from the Massachusetts Angioplasty Reg- [67] O’Connor GT. A regional intervention to improve
istry. Journal of the American College of Cardiology 2011;57:904–11, the hospital mortality associated with coronary
http://dx.doi.org/10.1016/j.jacc.2010.09.057. artery bypass graft surgery. JAMA 1996;275:841,
[54] Shahian DM, Wolf RE, Iezzoni LI, Kirle L, S.-L.T Normand. http://dx.doi.org/10.1001/jama.1996.03530350023029.
Variability in the measurement of hospital-wide mortality [68] Walker J, Neuburger J, Groene O, Cromwell D, van der Meulen J. Public
rates. New England Journal of Medicine 2010;363:2530–9, reporting of surgeon outcomes: low number of procedures leads to
http://dx.doi.org/10.1056/NEJMsa1006396. false complacency. The Lancet 2013;382:1674–7.
[55] Grant SW, Grayson AD, Jackson M, Au J, Fabri BM, Grotte G, et al. [69] Cacace M, Ettelt S, Mays N, Nolte E. Assessing quality in cross-country
Does the choice of risk-adjustment model influence the outcome comparisions of health systems and policies: towards a set of generic
of surgeon-specific mortality analysis? A retrospective analysis quality criteria. Health Policy 2013:156–62.
of 14,637 patients under 31 surgeons. Heart 2008;94:1044–9, [70] Khan OA, Iyengar S, Pontefract DE, Rogers V, Ohri SK, Livesey SA.
http://dx.doi.org/10.1136/hrt.2006.110478. Impact of surgeon-specific data reporting on surgical training. Annals
of the Royal College of Surgeons of Engl 2007;89:796–8.
[56] Thaler R, Sunstein C. Nudge: improving decisions about health,
[71] Sherman KL, Gordon EJ, Mahvi DM, Chung J, Bentrem DJ, Holl JL, Bil-
wealth, and happiness. Yale Univ Press; 2008. p. 53–71,
imoria KY. Surgeons’ perceptions of public reporting of hospital and
http://dx.doi.org/10.1007/s10602-008-9056-2.
individual surgeon quality. Medical Care 2013;51:1069–75.
[57] Bevan G, Fasolo B. In: Adam Oliver, editor. Models of Governance of
Public Services: Empirical and Behavioural Analysis of “Econs” and
“Humans”. Cambridge University Press; 2013. p. 530–1. Behav Public
Policy.

Please cite this article in press as: Behrendt K, Groene O. Mechanisms and effects of public reporting of surgeon outcomes:
A systematic review of the literature. Health Policy (2016), http://dx.doi.org/10.1016/j.healthpol.2016.08.003