View metadata, citation and similar papers at core.ac.uk
brought to you by
CORE
provided by Radboud Repository
PDF hosted at the Radboud Repository of the Radboud University
Nijmegen
The following full text is a publisher's version.
For additional information about this publication click this link.
http://hdl.handle.net/2066/147494
Please be advised that this information was generated on 2018-07-07 and may be subject to
change.
For reasons of consistency within this thesis, some terms have been standardized throughout the text. As
a consequence the text may differ in this respect from the articles that have been published.
The studies presented in this thesis have been performed at the Scientific Institute for Quality of
Healthcare (IQ healthcare). This institute is part of the Radboud Institute for Health Sciences (RIHS), one of
the approved research institutes of the Radboud university medical center.
The studies in this thesis were financially supported by the CZ Fund [grant AFVV08‐156AFVV08‐156].
Financial support by IQ healthcare for the publication of this thesis is gratefully acknowledged.
ISBN: 9789462799905
Nijmegen, 2015
Cover: Manon den Hartog, Omdat Ontwerp
Lay‐out: Jolanda van Haren / Nicole Ketelaar
Print: GVO drukkers en vormgevers B.V. | Ponsen & Looijen, Ede
Choosing healthcare providers
Healthcare consumers’ use of comparative performance
information
Proefschrift
ter verkrijging van de graad van doctor
aan de Radboud Universiteit Nijmegen
op gezag van de rector magnificus
volgens besluit van het college van decanen
in het openbaar te verdedigen op woensdag 2 december 2015
om 10.30 uur precies
door
Nicole Antonia Berendina Maria Ketelaar
geboren op 26 maart 1982
te Doetinchem
Promotoren:
Prof. dr. G.P. Westert
Prof. dr. G. Elwyn (Dartmouth College, Verenigde Staten)
Copromotoren:
Dr. J.C.C. Braspenning
Dr. M.J. Faber
Manuscriptcommissie:
prof. dr. J.W.A. Smit
prof. dr. W.J.J. Assendelft
prof. dr. D. Delnoij (Tilburg University)
Contents
Chapter
Title
page
Chapter 1
Introduction
Chapter 2
Exploring consumer values of comparative performance
information for hospital choice
Quality in Primary Care 2014; 22(2): 81-9.
19
Chapter 3
Patients’ expectations of variation in quality of care relates
to their search for comparative performance information
BMC Health Services Research 2014; 14: 617.
33
Chapter 4
Recognition of physiotherapists’ expertise in Parkinson’s
disease
BMC Health Services Research 2013; 13: 430.
47
Chapter 5
Public release of performance data in changing the behaviour
of healthcare consumers, professionals or organizations
(Review)
Cochrane Database Systematic Review 2011; (11).
63
Chapter 6
Comparative performance information plays no role in the
referral behaviour of GPs
BMC Family Practice 2014; 15(1): 146.
119
Chapter 7
General discussion
135
Summary
153
Samenvatting
159
Dankwoord
165
Curriculum Vitae
169
Portfolio Radboud Institute for Health Sciences
173
7
Chapter
1
Introduction
Chapter 1
INTRODUCTION
Patients are increasingly referred to as ‘healthcare consumers’, which acknowledges
their active position when they are making choices.1 This position has become much
more prominent since the introduction of demand-driven care and market-based forces
in the current healthcare system in the Netherlands. An emancipatory movement
emphasizes an active position as well, which the increased focus on patients’ interests,
needs, and attributes in the healthcare system reflects.2 Concepts in which the role of
patients is empathetically defined and acknowledged are patient-centredness, shared
decision-making and demand-driven care.3,4
In demand-driven healthcare systems consumers are expected to have an active role
and to choose healthcare providers that offer high quality and ignore the ones that
provide low quality.5,6 If they do so, consumers can influence the average levels of
quality of care, provided that they act as rational consumers.7,8 However, as decisions
become increasingly complicated and consequential, the assumption of a rational
consumer becomes less realistic. Marshall argues that consumers take more time to get
the best deal for a hotel stay or a car than for the best healthcare setting.9 Even so,
encouraging consumers to actively choose a provider can be considered a goal in itself
because it enhances patient empowerment, consumer activation, and selfdetermination.
Active provider choice for consumers leads to an urgent demand for accessible,
comprehensible information about differences in the performance of healthcare
providers10-12, the comparative performance information (CPI). Other stakeholders need
transparency of the quality of the hospital or the provider’s performance as well.
Payers would like to use CPI for purchasing care based on quality instead of quantity;
the inspectorate, for monitoring the minimum level of care delivered; and the
healthcare services and providers, for improving quality of care processes. Whether all
these goals can be reached simultaneously remains to be seen. However, in this thesis,
the focus is on the 1 healthcare consumer’s use of CPI.
The basic assumption is that healthcare consumers like to make their own choices in
healthcare. The existing literature shows that they want more information about the
quality of care.13,14 Nevertheless, the choice of providers does not seem to be a high
priority for consumers.15 While a growing mass of CPI is freely available, consumers
hardly use this information to choose their healthcare provider.11,16-21
At least three relevant topics have emerged from the healthcare reforms and the
provision of CPI to healthcare consumers. First of all, why a majority of patients ignore
CPI is unknown. Some research into the presentation of CPI and the patients’
8
Introduction
understanding of it has been carried out 17,22-24, but little is known about the search
drivers and factors that affect it. Insight into the design and dissemination is needed to
overcome these barriers to valuable usage. Second, CPI can empower patients in their
choice of providers, but how much this affects actual behaviour is still unclear.25 Third,
healthcare consumers probably need help in interpreting the CPI. CPI has been
described as not easily accessible.10 Perhaps structural guidance, e.g. from healthcare
professionals (GPs), can increase the use of CPI.
This thesis studies healthcare consumers’ limited search for and use of CPI for the
purpose of choosing a healthcare provider. The introduction briefly describes the
changing position of healthcare consumers, provider choice in the healthcare system,
the kind of the comparative performance information (CPI) available for consumers,
and a further description of the three research themes. This chapter ends with the set
of research questions that have been studied in the five consecutive studies.
Provider choice in healthcare systems
The way provider choice is structured in a healthcare system directly affects the
consumer’s actual freedom of choice. There are several ways to organise provider
choice. One option is to regulate provider choice either via a gatekeeper system or via
health plans. Another option is free provider choice and patient access to specialised
care without any interference. Countries differ in this organizational aspect of the
healthcare system. Within Europe, the regulated form arranged by the gatekeeper
system and free provider choice are common. The UK and the Netherlands have strong
systems of primary-care gate-keeping, which means that the general practitioner (GP)
has an important role in referral and the choice of a provider.26 As a result, consumers
cannot easily consult a professional or a specialist, or visit a hospital, without seeing a
GP first. In Sweden, provider choice is based on agreements between the government
and the federation of county councils.27 In Germany, patients have free access to
specialists and hospitals of choice. In France, patients are free to visit any GP or
specialist practising privately or in a hospital.28 In the USA, the gatekeeper system also
plays a role, but in a different way, as provider choice is mainly regulated by health
plans.29 Healthcare consumers choose a health plan and pay for the care within the
network. Plans that restrict provider choice cost less, while flexible plans with an
extension of provider choice cost more.30 A second form of the American gatekeeper
system demands that patients see their GP first to avoid excessive out-of-pocket
payment charged by their health plan.
Provider choice has changed in Western countries (including the USA, UK, Denmark,
Norway, the Netherlands, and Sweden) in recent years, in favour of free provider
choice. The wish to empower and activate consumers in healthcare goes hand in hand
9
1
Chapter 1
with these developments. The Dutch healthcare system has been actively encouraging
consumers to choose a provider since 2006. The Health Insurance Act (HIA) made it
possible for healthcare consumers to choose their preferred type of health insurance
and their own healthcare providers. The GPs continued to act as the gatekeepers of
specialised care. However, dental hygienists and physiotherapists have been freely
accessible to consumers without an intervening consultation with a GP since the
implementation of the HIA.31 Box 1 describes how provider choice has been expanded.32
Box 1.
Important elements of the Dutch healthcare system after the introduction of the Health
Insurance Act in 2006
Important elements of the Dutch healthcare system:
Healthcare insurance is now obligatory for every Dutch resident
The distinction between private and public healthcare insurance disappeared. All 18+ have to buy
private insurance
Provider choice for healthcare consumers has expanded
Healthcare insurers are authorised to selectively contract providers
Healthcare organisations must inform the public about price, quality, and other characteristics of
care
Both consumers’ choices and the insurers’ selective contracts are seen as main drivers in healthcare
providers’ improvement of the quality of care at lower costs
The wish to empower and activate patients in healthcare goes hand in hand with the
societal demand for accountability in healthcare, transparency about the quality of
hospital care, and provider performance.33,34 More information about the quality of
care is becoming available in the public domain, not only for consumers, but also for
other end-users: the health inspectorate, healthcare providers, and insurance
companies. Each group has its own goal in using this type of information, and the
question is whether these goals are compatible (Box 2).25
Box 2.
Users and goals of different end-users of comparative performance information, based on
Smith et al., 2009
Users of comparative
performance information
Government
Health inspectorate
Healthcare providers
Health insurance
Patients and healthcare consumers
10
Goals in using comparative performance information
Measuring the quality of healthcare
Monitoring regulatory effectiveness and efficiency
Ensuring the development of comparative performance
information
Starting healthcare improvement projects
Selectively referring to high-quality providers
Selectively contracting the best health provider balanced
against low cost
Addressing information asymmetry
Making an informed provider choice
Selecting high-quality providers
Introduction
Theoretically, there are a number of pathways through which the public release of CPI
may have an impact on providers and the quality of care.35 Berwick and colleagues’
framework shows two pathways. The ‘pathway of selection’ implies that healthcare
consumers drive healthcare quality improvement by using the release of comparative
performance data to make choices between hospitals or providers. Consumers will
identify the value of the outcomes, search for performance information about a
healthcare provider, and then make an informed choice. The assumption is that they
will try to obtain the best quality or avoid the worst, and select (or reward or
recognise) the best hospital or provider (Figure 1). In the ‘pathway of change’, the
extrinsic motivation is necessary to effect changes. Change may occur due to pressure
to avoid being identified as a provider of poor-quality care or to improve a provider’s
reputation in relation to other providers.36
Figure 1.
Two pathways for improving performance by means of the public release of performance
data
Publicly reported performance data
Knowledge
Selection
Pathway 1
Motivation
Change
Pathway 2
Performance
The ‘pathway of change’ assumes that benchmarking healthcare organisations will
encourage hospitals and healthcare providers to improve their performance. The
pathways are strongly connected to one other, and for their effects, both mechanisms
rely on the availability of reliable performance information that all users of the
healthcare system can use and act upon. We include both pathways, with an emphasis
on the pathway of selection.
Comparative performance information
Many different terms have been introduced to refer to information about the quality of
the care delivered: comparative quality information, performance information, quality
information, and healthcare information. This thesis uses the term ‘comparative
performance information’ (CPI) within the following definition:
“Systematically collected, publicly available information about the
performance of health organizations or healthcare providers that can be
compared objectively. The purpose of CPI is to inform and enable healthcare
consumers to make their choices based on their own values”
11
1
Chapter 1
This kind of information has been developed in many countries, especially in the USA
and the UK, where the production and dissemination of such information has been high
on the agenda since the 1980s.37,38
One can differentiate three types of CPI: information about healthcare provider
characteristics and services, experiences of healthcare consumers, and performance
indicators (Table 1).39
Table 1. Types of comparative performance information
Type
1
2
3
Description
Information about healthcare
provider characteristics and
services
Information from the
healthcare consumer’s
perspective and experience
Performance indicators
(process, structure, or outcome
indicators)
Examples
Travelling distance, region, type of provider, provider
specialties, treatment specialties, costs, waiting times,
number of hospital beds, and personnel
Experiences with treatment in a healthcare organisation
(e.g. hospital and nursing home), satisfaction with food,
privacy, communication, and the consumer quality index
Hip-fracture patients having surgery 24 hours after admission
(process), number of patients with pressure wounds,
treatment volumes, methods of anaesthesia, and the
hospital standardised mortality ratio
These different types of information are often based on various sources and collected
by a range of instruments. Data can be collected from healthcare providers, patients,
and insurance companies. The instruments can be observation lists, medical records,
interview protocols, or surveys. The Dutch government initiated placing CPI on
websites, including www.kiesbeter.nl (make better choices) in 2005. This website
includes quality information and assessments of individual hospital performance. It
started with mainly a combination of types 1 and 2, but in recent years type 3 has also
been presented on this website (Table 1). Other Dutch websites also offer comparisons
of hospitals and other healthcare services (www.independer.nl, www.mediquest.nl,
and www.zorgkaartnederland.nl). The Dutch daily newspaper Algemeen Dagblad
produces a list of the 100 best Dutch hospitals every year, and the weekly magazine
Elsevier publishes a list of best hospitals; their assessments are based on various
indicators.
Three emerging research themes of comparative performance information
Search drivers and the possible determinants
Discussions about selective provider choice usually start with the assumption that
healthcare consumers are keen to choose their own healthcare providers. Researchers
uncritically adopt this assumption, and then begin assessing the availability of the right
information, the content of such information, and the influence of the way the CPI is
presented.10,15,40 These elements are, without doubt, important. The debate focuses on
12
Introduction
understanding how to present quality information in ways that are meaningful to
consumers.38 However, the basic barriers to consumers using and trusting CPI are
underexposed. The fundamental questions of how consumers perceive, value, and
appraise CPI for the purpose of choosing healthcare providers are often neglected and
poorly understood. An important factor that could influence the value of CPI, and
therefore the search driver, is the patients’ recognition of quality-of-care differences.
For this thesis, we studied whether expectations about the quality of care differences
had an impact on the CPI search behaviour of patients. Since the context of the
healthcare can affect the results, we did one study about elective surgery in a hospital
and another about the chronic care setting in primary care.
The impact of comparative performance information on behaviour
From the time that CPI was first published, there have been doubts whether the
consumers would use this information and how it would impact their choice
behaviour.37 Research seems to confirm these doubts: it shows that few consumers
actually use CPI. For example, the proportion of Americans using comparative data
about physicians grew from 4% to 6% between 1996 and 2008.18 Research highlighted
several barriers to the use of CPI. Examples are people’s distrust of hospital ranking
and the reliability of health information on the internet, in the media, or from the
government. Consequently, the relevance of CPI consistently ranks far below the
information from family and friends.11,20,41 Experiences of family and friends reported
by word of mouth have more impact than objective, measurable performance data.42,43
Nevertheless, there is still a firm belief that CPI will result in better informed decisionmaking. The literature does not support this hypothesis. Except for studies that used
hypothetical scenarios to examine the effect of public reporting,23,44-48 when this
research project began in 2009 there were still no systematic data available about the
effects of public reporting of the actual behaviour regarding the selection of healthcare
providers, quality improvement, and patient outcomes.49
Support of the consumer’s choice behaviour
The literature teaches us that not all consumers have the same competencies to deal
with complex choices50; consumers may need help to interpret such information. The
availability of someone who provides additional choice expertise can help consumers
choose.51 Because the primary-care system has a strong position in Dutch healthcare
and Dutch GPs traditionally act as gatekeepers52, the GPs would be the logical partners
to help Dutch consumers make choices. They can help consumers exercise their patient
rights by offering a respectful relationship.2 Research from the UK indicates that GPs
are sceptical about the use of objective information and prefer to rely on soft
knowledge such as relationships with providers and the experiences of patients whom
they had referred previously.53 However, it is not yet clear how GPs view their choice13
1
Chapter 1
supporting role when it comes to using CPI in consultation time, and in which ways they
could help consumers.
OBJECTIVES AND AIM OF THIS THESIS
The aim of this thesis is to make a contribution to the empirical evidence about how
consumers use CPI and how they choose a provider. The main focus is to make the
perception of the consumer visible. The research questions for the three research
themes that this thesis addresses are:
What are the barriers and facilitators for healthcare consumers to search and use
CPI?
To what extent does publicly released performance information changes the choice
behaviour of healthcare consumers, providers, and purchasers?
To what extent do GPs advocate and encourage patients to choose providers on the
basis of CPI?
The research focuses on the effects of public reporting, and the aim is to discover the
conditions that either constrain or facilitate consumers’ searches and use of
information that ultimately helps them choose a healthcare provider. In particular, we
focus on:
Identifying the basic values, attitudes, and ideas regarding CPI, as well as choice
behaviour among healthcare consumers;
Exploring whether expectations of quality of care affects consumers’ behaviour in
their search for CPI;
Testing whether expectations of performance and recognition of healthcare
providers’ expertise can foster the search for CPI and selective provider choice;
Assessing how the effects of public reporting changes the behaviour of consumers,
professionals, healthcare organisations and purchasers of care (insurance
companies);
Exploring GPs’ views of the use of CPI at the point of referral in primary care and
their views of their own role of choice-supporting behaviour.
OUTLINE OF THIS THESIS
Chapter 2 reports the results of six focus groups with healthcare consumers. The main
goal of this study is to clarify the value that CPI gives the consumers. Most research in
this field is oriented towards the content and the presentation format of CPI, while
little is known about how consumers value CPI and use this information. This chapter
considers the awareness of consumers, current sources of decision-making, the value of
CPI, and the effects of CPI.
14
Introduction
Chapters 3 and 4 report how consumers’ awareness of the availability of variation in
quality of care affects their use of CPI to find a hospital or healthcare provider for
elective surgery or a progressive chronic-care condition. Chapter 3 deals with patient
expectations about the variation in quality of care. Postal questionnaires were sent to
475 patients undergoing a total hip or knee replacement in three types of hospitals to
obtain the necessary data. The objective of Chapter 4 is to explore the ability of
people with Parkinson’s disease to recognise expertise among physiotherapists and to
determine to what extent patients selectively choose expert physiotherapists.
Chapter 5 describes a systematic review of the public release of performance data.
The review concerns the content and effectiveness of determinants of behaviour
change that prompt the actual behaviour in choosing.
Using the suggestion to provide an advocate who can help consumers with their choice
of hospital, Chapter 6 investigates how GPs view their role in using CPI during
consultation time.
The general discussion in Chapter 7 summarises the results in this thesis and discusses
our findings in view of several methodological issues, implications for practice, and
aims for future research.
15
1
Chapter 1
REFERENCES
1.
2.
3.
4.
5.
6.
7.
8.
9.
10.
11.
12.
13.
14.
15.
16.
17.
18.
19.
20.
21.
22.
23.
24.
16
Williamson C. The patient movement as an emancipation movement. Health Expect
2008;11(2):102-12.
Dieterich A. The modern patient - threat or promise? Physicians' perspectives on patients’ changing
attributes. Pat Educ Couns 2007;67(3):279-85.
de Haes H. Dilemmas in patient centeredness and shared decision making: a case for vulnerability.
Pat Educ Couns 2006;62(3):291-98.
Elwyn G. Idealistic, impractical, impossible? Shared decision making in the real world. Br J Gen
Pract 2006;56(527):403-4.
Mukamel DB, Weimer DL, Zwanziger J, Gorthy SF, Mushlin AI. Quality report cards, selection of
cardiac surgeons, and racial disparities: a study of the publication of the New York State Cardiac
Surgery Reports. Inquiry 2004;41(4):435-46.
Rothberg MB, Morsi E, Benjamin EM, Pekow PS, Lindenauer PK. Choosing the best hospital: the
limitations of public quality reporting. Health Affairs 2008;27(6):1680-87.
Botti S, Iyengar SS. The Dark Side of Choice: When Choice Impairs Social Welfare. J Public Policy
Mark 2006;25(1):24-38.
Glazer J, McGuire TG. Optimal quality reporting in markets for health plans. J Health Econ
2006;25(2):295-310.
Marshall M, McLoughlin V. How do patients use information on health providers? BMJ
2010;341:c5272.
Hibbard JH, Peters E. Supporting informed consumer health care decisions: data presentation
approaches that facilitate the use of information in choice. Ann Rev Public Health 2003;24:413-33.
Magee H, Davis LJ, Coulter A. Public views on healthcare performance indicators and patient
choice. J Roy Soc Med 2003;96(7):338-42.
Shaller D, Sofaer S, Findlay SD, Hibbard JH, Lansky D, Delbanco S. Consumers and quality-driven
health care: a call to action. Health Affairs 2003;22(2):95-101.
Fotaki M, Boyd A, Smits L, McDonald R, Sheaff R, Edwards A, et al. Patient Choice and the
Organisation and Delivery of Health Services: Scoping review. University of Manchester: Report for
the National Co-ordinating Centre for NHS Service Delivery and Organisation R&D (NCCSDO), 2005.
Damman OC, Hendriks M, Rademakers J, Spreeuwenberg P, Delnoij DM, Groenewegen PP.
Consumers’ interpretation and use of comparative information on the quality of health care: the
effect of presentation approaches. Health Expect 2012;15(2):197-211.
Fung CH, Lim YW, Mattke S, Damberg C, Shekelle PG. Systematic review: the evidence that
publishing patient care performance data improves quality of care. Ann Intern Med
2008;148(2):111-23.
Fotaki M, Roland M, Boyd A, McDonald R, Scheaff R, Smith L. What benefits will choice bring to
patients? Literature review and assessment of implications. J Health Serv Res Policy
2008;13(3):178-84.
Harris-Kojetin LD, Uhrig JD, Williams P, Bann C, Frentzel EM, McCormack L, et al. The “choose
with care system” - development of education materials to support informed Medicare health plan
choices. J Health Commun 2007;12(2):133-56.
Kaiser Family Foundation. Update on consumers’ view of patient safety and quality information.
Menlo Park, California: Agency for Healthcare Research and Quality, 2008.
Schneider EC, Epstein AM. Use of public performance reports: a survey of patients undergoing
cardiac surgery JAMA 1998;279(20):1638-42.
Schwartz LM, Woloshin S, Birkmeyer JD. How do elderly patients decide where to go for major
surgery? Telephone interview survey. BMJ 2005;331(7520):821-24.
Werner RM, Asch DA. The unintended consequences of publicly reporting quality information. JAMA
2005;293(10):1239-44.
Damman OC. Public reporting about healthcare users’ experiences. Utrecht: University of Tilburg,
2010.
Peters E, Dieckmann N, Dixon A, Hibbard JH, Mertz CK. Less is more in presenting quality
information to consumers. Med Care Res Rev 2007;64(2):169-90.
Fasolo B, Reutskaja E, Dixon A, Boyce T. Helping patients choose: how to improve the design of
comparative scorecards of hospital quality. Pat Educ Couns 2010;78(3):344-49.
Introduction
25.
26.
27.
28.
29.
30.
31.
32.
33.
34.
35.
36.
37.
38.
39.
40.
41.
42.
43.
44.
45.
46.
47.
48.
49.
50.
51.
Smith PC, Mossialos E, Papanicolas I, Leatherman S. Performance measurement for health system
improvement. Experiences, challenges and prospects. Cambridge: University Press Cambridge,
2009.
Dixon A, Robertson R, Bal R. The experience of implementing choice at point of referral: a
comparison of the Netherlands and England. Health Econ Policy Law 2010;5(3):295-317.
Thomson S, Dixon A. Choices in health care: the European experience. J Health Serv Res Policy
2006;11(3):167-71.
Tuffs A. Germany plans to make GPs gatekeepers. BMJ 1999;318(7189):961.
Harris-Kojetin LD, Stone RI. The role of consumer satisfaction in ensuring quality long-term care:
opportunities and challenges. J Aging Soc Policy 2007;19(2):43-61.
Medline Plus. Trusted Health Information for you http://www.nlm.nih.gov/medlineplus2014 [3003-2014].
Groenewoud AS. It’s your Choice! A study of search and selection processes, and the use of
performance indicators in different patient groups. Veenendaal: Erasmus University Rotterdam,
2008.
Westert GP, Burgers JS, Verkleij H. The Netherlands: regulated competition behind the dykes? BMJ
2009;339:b3397.
Marshall MN, Shekelle PG, Leatherman S, Brook RH. Public disclosure of performance data:
learning from the US experience. Qual Health Care 2000;9(1):53-57.
Sinaiko AD, Eastman D, Rosenthal MB. How report cards on physicians, physician groups, and
hospitals can have greater impact on consumer choices. Health Affairs 2012;31(3):602-11.
Berwick DM, James B, Coye MJ. Connections between quality measurement and improvement. Med
Care 2003;41(1 Suppl):I30-38.
Shekelle PG, Lim YW, Soeren M, Damberg C. Does public release of performance results improve
quality of care? Southern California: The Health Foundation, 2008.
Marshall MN, Romano PS, Davies HT. How do we maximize the impact of the public reporting of
quality of care? Int J Qual Health Care 2004;16(Suppl 1):i57-63.
Marshall MN, Shekelle PG, Davies HT, Smith PC. Public reporting on quality in the United States
and the United Kingdom. Health Affairs 2003;22(3):134-48.
Van Loon AJM, Tolboom RAL. Design report “Choose Better” 2006-2007 [In Dutch: Ontwerprapport
KiesBeter.nl 2006-2007]. Bilthoven: RIVM, 2005.
Stein BD, Kogan JN, Essock S, Fudurich S. Views of mental health care consumers on public
reporting of information on provider performance. Psychiatric Services 2009;60(5):689-92.
RAND Health. Consumers and health care quality information: need, availability, utility. Oakland,
CA: California Healthcare Foundation, 2001.
Coulter A. Do patients want a choice and does it work? BMJ 2010;341:c4989.
Harris KM, Beeuwkes Buntin M, The RAND Corporation. Choosing a health provider: the role of
quality information. Princeton: Robert Wood Johnson Foundation, 2008.
Hibbard JH, Slovic P, Peters E, Finucane ML. Strategies for reporting health plan performance
information to consumers: evidence from controlled studies. Health Serv Res 2002;37(2):291-313.
Schoenbaum M, Spranca M, Elliott M, Bhattacharya J, Short PF. Health plan choice and information
about out-of-pocket costs: an experimental analysis. Inquiry 2001;38(1):35-48.
Spranca M, Kanouse DE, Elliott M, Short PF, Farley DO, Hays RD. Do consumer reports of health
plan quality affect health plan selection? Health Serv Res 2000;35(5 Pt 1):933-47.
Uhrig JD, Harris-Kojetin L, Bann C, Kuo TM. Do content and format affect older consumers’ use of
comparative information in a Medicare health plan choice? Results from a controlled experiment.
Med Care Res Rev 2006;63(6):701-18.
Uhrig JD, Short PF. Testing the effect of quality reports on the health plan choices of Medicare
beneficiaries. Inquiry 2002;39(4):355-71.
Shekelle PG. Public performance reporting on quality information. In: Smith PC, Mossialos E,
Papanicolas I, Leatherman S (eds). Performance measurement for health system improvement:
Experiences, challenges and prospects Cambridge: University Press Cambridge, 2009.
Hibbard JH, Peters E, Dixon A, Tusler M. Consumer competencies and the use of comparative
quality information: it isn’t just about literacy. Med Care Res Rev 2007;64(4):379-94.
Schlesinger M. Choice cuts: parsing policymakers’ pursuit of patient empowerment from an
individual perspective. Health Econ Policy Law 2010;5(3):365-87.
17
1
Chapter 1
52.
53.
18
Berendsen AJ, Kuiken A, Benneker WH, Meyboom-de Jong B, Voorn TB, Schuling J. How do general
practitioners and specialists value their mutual communication? A survey. BMC Health Serv Res
2009;9:143.
Rozen R, Florin D, Hutt R. An anatomy of GP referral decisions. A qualitative study of GP's view on
their role in supporting patient choice. London: The King’s Fund, 2007.
Chapter
2
Exploring consumer values of comparative
performance information for hospital choice
Nicole Ketelaar
Marjan Faber
Gert Westert
Glyn Elwyn
Jozé Braspenning
Quality in Primary Care 2014;22(2):81–9.
Chapter 2
ABSTRACT
Background: In many countries, market orientation in healthcare has resulted in the
publication of comparative performance information (CPI). Most of the research in this
field is oriented towards the content and the presentation format of CPI while little is
known about how consumers value CPI and the use of this information.
Aim: The aim of this study was to clarify the perceived value that CPI brings for
consumers of healthcare.
Methods: Qualitative research using six focus group interviews. Twenty-seven
healthcare consumers were recruited using a mailing list and by personal invitation.
Data from focus group interviews were transcribed and thematic analysis undertaken.
Results: Most participants were unaware of CPI, and valued alternative sources of
information more than CPI. Through discussion with other consumers and by means of
examples of CPI, respondents were able to express the values and perceived effects of
CPI. Numerous underlying values hindered consumers’ use of CPI, and therefore
clarification of consumer values gave insights into the current nonusage of CPI.
Conclusions: CPI is marginally valued, partly because of conflicting values expressed by
consumers and, as such, it does not yet provide a useful information source on hospital
choice beyond consumers’ current selection routines in healthcare. Future research
should be more focused on the values of consumers and their impact on the use of CPI.
20
Exploring consumer values of comparative performance information
BACKGROUND
Provider choice for healthcare consumers has emerged as a key policy focus in modern
Western healthcare systems. The architects of the policy outlined several reasons to
promote choice among healthcare consumers. First, to encourage competition between
providers which was expected to improve the efficiency and quality of healthcare;1,2
and second, to increase patient empowerment and improve the position of consumers
in healthcare.3,4 It is based on the expectation that patients will choose the best
healthcare provider once they are informed.
Systematically collected, publically available information about the performance of
healthcare providers, called comparative performance information (CPI), can be used
as a tool to inform healthcare consumers to enable them to make informed provider
choices. The information consists of, for example, patient volumes, treatment
methods, waiting lists and patient experiences, and can be found in leaflets,
magazines, books and on the internet.
Despite the efforts and resources that go into the collection, production and
dissemination of information for the public,5 there is little evidence that CPI affects
consumer choices. Many people do not understand the information, or do not view the
information as useful.6 British and American studies have shown that as few as 5–7% of
consumers use CPI to choose a hospital7,8 and only 12% use CPI to select a primary care
physician,9 although awareness of the right to choose a provider has increased. The
overall conclusion is that consumer usage of CPI to make an informed choice is still
limited. Instead, consumers rely on information from family and friends or base their
choice of hospital on recommendations from their referring primary care physician.10,11
The choice is largely driven by familiarity with a certain hospital12,13 or the distance
between home and hospital.12,14 Using CPI is also a difficult and complex task for
consumers, which limits its impact on consumer choice even more.14,15
Because provider choice by healthcare consumers is positively associated with
autonomy and self-determination, both important principles of patient activation,16 the
need to explain the limited exercise of choice by healthcare consumers might improve
our understanding of active patient participation in healthcare.
Previous studies have questioned whether this limited usage of CPI is caused by the
absence of the right information content17 or whether CPI presentation formats do not
support decision-making tasks.7 Evidence for both assumptions has been found.
Damman and colleagues conclude that the presentation of CPI facilitates consumers’
correct interpretation as well as effective use, meaning the ability to choose the best
performing provider.18 Although these are without doubt crucial elements in facilitating
21
2
Chapter 2
consumers in using such information, fundamental questions over how CPI is perceived,
valued and appraised by healthcare consumers for their choice of hospital are
overlooked. Some theory and evidence suggest that healthcare consumers are
insufficiently informed about what is important for them.5,19 Current shared decisionmaking models for clinical treatment acknowledge the need to clarify patients’ values
to promote active patient participation in decision-making.20,21 The aim of this study
was to identify consumers’ values, appraisal, understanding, opinions and judgement of
CPI. We conducted a series of focus group interviews.
METHODS
Approach
The current study was a qualitative focus group study in 2009. The method allows
researchers to capitalise on communication between participants to generate pertinent
information.22 The focus group method was chosen because of its flexibility to explore
unanticipated issues, and to make use of the interaction between group members. We
encouraged the participants to discuss the subject of CPI, ask each other questions,
exchange opinions and views, and share experiences.23 We presented the ‘values’ of
CPI initially using terms such as ‘meaning’, ‘sense’, ‘ideals’ and ‘principles’ to explain
their meaning for healthcare consumers. We introduced three real-world CPI examples
to align the mindset of the participants in the discussion (Table 1). The example shown
in Table 1 combined stars with percentages, the second used different kinds of bullets,
and the third one showed the availability of a service using ‘yes’ or ‘no’.
Table 1.
Example of comparative performance information related to three quality indicators of
hospital A, B and C
Quality indicator
Hospital A
Hospital B
Hospital C
Cancelled surgeries (%)a
1.2
3.3
0.7
Pressure ulcers (%)b
8.1
2.5
1.5
Electronic personal health record (%)c 9.5
10.0
7.0
a
Surgeries cancelled within 24 hours
b
New patients with pressure ulcers. All patients underwent total hip replacement
c
Availability of electronic personal health record including lab results, medication use, X-rays
= hospital performance was less than average; = hospital performance was average; =
hospital performance was better than average
Participants and sampling
Our initial idea was to approach patients with a specific condition. However, a
commonly cited concern is that patients predominantly focus on their personal context
and return to the hospital where they were treated previously.24 Also, those who are
most dependent on care services and who could benefit most from a ‘good choice’ are
also those who tend to have more problems associated with the capacity to manage
informed choices about care providers.25 We therefore decided to recruit a crosssection of healthcare consumers from the general population. By ‘healthcare consumer’
22
Exploring consumer values of comparative performance information
we mean anybody who is a user or potential user of healthcare. In order to increase the
potential use of healthcare for themselves, family or friends, we selected respondents
aged over 35 years. Our participants responded to an invitation letter sent to a random
sample of 480 people listed at a large health insurance company. We used a
stratification procedure to include a representative age distribution. We sent 160
letters to people aged 35–50 years and 320 to people aged 50–65 years (ratio: 1/3 to
2/3). Because the response rate was low, an additional approach was undertaken,
namely a personal invitation, made by members of the research team, to healthcare
consumers, including people sitting in a general practitioner’s (GP’s) waiting room.
Participants received a small incentive in exchange for their efforts. No medical
information was asked for, so approval by an ethics committee was not needed.
Data collection
First, an interview guide was developed by members of the research team (NK, MF, JB)
and influenced by the available literature about this topic. The interview guide was
used to guarantee consistency among groups. The interview guide is available in an
online appendix.
Before beginning each session, the aims and methods to be used were explained by the
moderator (NK). Participants were asked to give their permission for audiotaping and
transcribing. At the beginning of the focus group session, the moderator attempted to
create a thoughtful, non-threatening atmosphere and set the tone for the discussion.
Participants were encouraged to respond to all the issues raised by the moderator or
other participants. Every effort was made to create an environment that encouraged
individuals to participate fully in the discussion. After the introduction and the first
items of the interview guide, real-world examples were explained by the moderator to
encourage involvement and to support respondents in their discussion about big themes
as values. Participants were asked to look at the examples and to answer the following
questions:
Do you understand what the performance information is showing you?
Could this information benefit you in any way when you would be in a situation to
make a comparison between different providers?
What benefit could you gain from this information?
All focus group meetings lasted about 1.5 hours and were audiotaped. In addition, an
assistant made notes during each session. Data saturation, the point at which the
participants provided no new information to the researchers, was reached after focus
group five.
23
2
Chapter 2
Analysis
All audiotaped sessions were transcribed verbatim into a Microsoft Word file. Analysis
of the transcripts was facilitated by the use of a qualitative software program
Atlas.ti.5.2 to manage the data.25 Thematic analysis was undertaken to reveal core
consistencies and meanings in the data by identifying and analysing themes, which are
abstract categories of meaningful data fragments. In themes, several fragments, known
as codes are connected to each other and recur in a patterned way.26
Data were collected and transcripts read thoroughly to estimate data saturation. The
transcripts were read several times by two investigators (NK, MF) and the main themes
were extracted. Participants’ statements referring to a particular theme were grouped
by similar codes, and further explored. The analysis followed an inductive approach,
which was emergent and strongly linked to the data because assumptions were
datadriven. This means that the process of coding occurs without trying to fit the data
into a pre-existing model or frame.27
Statements about how many people have said something can leave readers unsure how
to interpret quantitative numbers in a qualitative study. However, a relatively high
frequency may also signify the importance of a finding. In describing something in
between, we avoid actual concrete numbers and used terms such as, for example,
‘many’, ‘most’ or ‘a minority’. We present the focus groups discussion in sufficient
detail supported by quotes which can be read in the boxes after every theme, to allow
readers check the interpretation made during the analysis.26 Every quotation used in
the boxes is followed by the number of the focus group (FG), the gender and age of the
participant.
Trustworthiness
We took several measures to ensure the trustworthiness of this study, including
multiple methods of recruitment; multiple researches to reflect on the analysis
process; multiple rounds in which data were read, analysed, compared and contrasted;
project team meetings to review and explore scientific and organisational aspects of
the project.
RESULTS
Thirty-seven consumers agreed to participate in the focus group interviews. Twenty
people responded to the invitation letter. Seventeen consumers were recruited having
been approached personally; two in the GP’s waiting room. Of the 37 participants, 27
finally participated in one of the six pre-planned focus group interviews (7 men and
20 women). The mean age of the participants was 59 years. We describe four themes
that emerged from the analysis.
24
Exploring consumer values of comparative performance information
Theme 1: awareness (Box 1)
Most participants were not familiar with CPI as a tool to guide informed choice or to
compare hospitals. Only a minority said they had seen CPI before. Although they
believed themselves to be skilled internet users, most participants did not know where
to find CPI. They said, ‘somewhere on the internet’, but they could not specify. Some
were aware of the national website presenting CPI for hospitals in the Netherlands, a
site the Dutch government initiated in 2005 (www.kiesbeter.nl).
Despite a lack of awareness of CPI as a tool to compare hospitals, most participants
reported that they knew about the possibility of comparing hospitals and were aware of
their free choice of providers. Nevertheless, they mostly seemed to have a low level of
interest in CPI. They said that, being in good health, they were not interested in
hospital choice, and they did not feel a sense of urgency to look for CPI. The
participants also agreed that if they became ill, they probably would not have the time
and energy to look for CPI. During the focus group meetings, the participants realised
that, because of a low level of awareness, there was a vicious circle in which they
continued to be unaware of the potential value of CPI to help them.
Box 1.
Awareness
‘I do not know where to find such information’ (FG6, female, 58). ‘No, I did not know either. And on
which things do we need to compare? That is also a question for me.’ (FG6, female, age 60)
‘If you feel healthy, you will not start looking for a hospital just in case you might need one.’ (FG1,
female, age 53)
‘Only, if it is really necessary then is the chance big enough that I would use this kind of
information’ (FG2, female, 57) ‘For me as well, I have better things to do.’ (FG2, female, age 56)
Theme 2: current sources for decision-making (Box 2)
Most of the participants said that their hospital choice was fixed: essentially, they
always went to the same hospital. This hospital was usually the closest one, as distance
was an important choice attribute. They did not have any reason to change this
routine, and questioned the added value of CPI. There were other significant sources
they currently used for their decision making. Consumers said that their own previous
experiences were important and outweighed the impact of CPI. They also highly valued
the advice of family and friends. The consumers trusted their own GP as a source of
information. There was some disagreement about whether to follow the GP’s advice
and the GP’s role in providing information. In one session, some said they would not
argue with their GP’s advice because they felt it could damage their relationship with
their GP.
25
2
Chapter 2
Box 2.
Current sources for decision making
‘I think, well I would go to the hospital I always go to. I would not go searching. I am not sure if this
wise, I have been in the [name] hospital a zillion times.’ (FG2, female, age 57)
‘I just feel save in that hospital.’ (FG3,female, age 75)
‘I have had personal experience with that hospital and I will never go back! And that is the way you
choose. The list may point out that it is a great hospital, but if your personal experience was
unpleasant, you simply do not believe the list.’ (FG1, female, age 63)
‘It’s simple, you rely on the experiences of the people you know.’ (FG1, female, age 53a)
I don’t know how my PCP would react if I go to a different hospital than what he refers me to. I don’t
know if it would affect the relationship. I still think you will be tend to listen to your PCP.’ (FG6,
female, age 63)
‘It would be nice, if my PCP would help me to remember that I have a choice, and that he would
provide me with an overview or refer me to a website where I could look for it’ (FG4, female, age
54). ‘But, you can’t expect that the PCP know everything about this.’ (FG1, female, age 53b)
‘If my PCP said, “I have faith in that specialist for these reasons” or “I would not rely on that
specialist”, then I would indeed switch to another hospital.’ (FG1, female, age 53a).
Theme 3: value of CPI (Box 3)
Most participants had never seen CPI before attending the focus group interviews. They
had some doubts and felt a little confused after being confronted with the CPI
examples. The examples caused a variety of reactions among the participants. The
inventory of these reactions, including start-up questions, was supportive for
participants in the discussion that followed. At first, they gave a reaction to the
examples and their usefulness in general. Some participants immediately tried to
interpret what they saw, others were primarily looking to see if the examples
confirmed their ideas or their own experiences. Some consumers questioned the added
value of such information.
During the discussion, reactions evolved to more specific goals and values by using the
examples. Participants reported that having the ability to select a healthcare provider
might involve a significant effort to find and compare information. They argued that
choosing a hospital introduced a new responsibility for patients, including feelings of
distrust and anxiety, since they could not foresee the consequences that they might be
held responsible for. Not using CPI was sometimes explained as a strategy to prevent
regret for a wrong decision.
The participants pinpointed an important paradox: the more you know, the more
uncertain you become, especially if information from different sources is inconsistent.
The participants were afraid of losing their trust in certain aspects of healthcare that
they had previously assumed to be good.
An important theme was the importance consumers assigned to the reliability and
trustworthiness of the information. Respondents complemented each other in listing
26
Exploring consumer values of comparative performance information
conditions CPI must fulfill before it became of value for them and before they would
start using this information. The discussion included the objectivity of the information,
how the information was collected, which groups of patients were compared, and at
what level the CPI was published. Most participants stressed the importance of the
reliability of CPI, but at the same time they found it difficult to determine on what
grounds CPI could be considered reliable and trustworthy. Our participants wished to
see that sources presenting CPI would include disclaimers about reliability aspects and
declare any conflicts of interest. Also, up-to-date information was of much value for
consumers. The remarks regarding to this can be summarised with the question: ‘How
do I know if what was good then is good for me right now?’ Some participants wanted a
single composite indicator of overall quality to compare hospitals rapidly and easily.
Other participants stated that the level of aggregation was too general in the realworld examples. The examples referred to an entire hospital, whereas participants
wanted to have information at the level of departments, e.g. cardiology and
orthopaedics, or at the level of individual doctors.
Box 3.
Values of comparative performance information
‘I had to take a look at this for a second. What is this!? All these dots and circles. I'm just amazed by
the way we are supposed to believe that this is useful for us as patients.’ (FG1, female, age 53a)
‘What happens now is that someone tells me: “you are the patient, so make your own choice.” (..).
The responsibility is passed on to me, the patient. This fits with the image of the consumer-driven
health care system, but the question is whether you can deal with this and what happens if
something goes wrong?’ (FG2 male, age 53)
‘You read something and it stays in your head. But if you had not known it, you wouldn’t have that
problem. Knowing everything is not that great either.’ (FG2, female, age 57)
Theme 4: perceived effect of CPI (Box 4)
The values identified stimulated the participants to mention possible effects of CPI:
patient empowerment, waste, freedom of choice, benchmarking, changed perceptions,
and a dichotomy in society. Patient empowerment was mentioned in several sessions,
so was the counterpart that CPI is a waste. Some respondents found that using CPI to
choose providers went beyond what is necessary in healthcare in terms of good care.
Increasing freedom of choice was noted as a positive effect, as well as an attitude
among some participants who felt their perceptions of quality of care had changed.
They declared raising more awareness about the quality of care after the recent
introduction of choice for consumers. Consumers supported the effect of
benchmarking, so that professionals and organisations could compare each other in
terms of quality of care. Finally, some participants had concerns about whether using
this information might cause a further dichotomy in society by increasing inequity in
healthcare. They foresaw that the ability to use information required skills of
healthcare consumers for which some would be better than others.
27
2
Chapter 2
Box 4.
Perceived effect of comparative performance information
‘You now look differently. When I, recently, came to the Emergency Room with my mother, you
start to look around, how is it here? How are the nurses doing things? How do they do that? What are
the stories of my mother?’ Moderator: ‘Do you now look at things in health care from a different
perspective?’ ‘Yes, more as a purchaser of care.’ (FG3, female, age 56)
[..] ‘So you go on this path, and you might get a dichotomy in society, that is a risk with this market
in health care’ (female, age 57). ‘Yes’ [approvingly] (female, age 56). ‘You have people who can do
this’ (female, age 57). ‘Yes, yes’ (female, age 56). ‘People who have the skills, but for others there
will be less possibilities’ (female, age 57). ‘Yes, I am worry about that too.’ (FG3, female, age 56)
‘I think patients become more assertive. Normally, you would take things for granted if they say ‘go
to that hospital. While if you later find out you had a choice, you might say: if I could choose, I had
perhaps done things differently and if I had known, I might have gone elsewhere; so I think this has
an effect on empowerment of patients.’ (female, age 60) ‘Yes, I do think it is useful information
that they receive.’ (FG6, male, age 63)
DISCUSSION
This study explored the values, thoughts, understanding and evaluations of CPI for
hospital choice among healthcare consumers in the Netherlands. By means of realworld examples, healthcare consumers were able to express their views about CPI. The
four themes (awareness, current sources of decision-making, value of CPI and
perceived effect of CPI) from the data suggest that there are numerous underlying but
conflicting values, which are important for healthcare consumers relating to their use
of CPI. The CPI was only marginally valued, due to consumers’ values during their
processing of CPI, and wider principles that limited consumers even considering the use
of the information.
Most participants were unaware of CPI, and did not use this kind of information.
Therefore, participants could not give a direct answer when we asked them what kind
of information they would like to have or what matters most to them at the beginning
of the focus group sessions. It seemed that the participants had never reflected on the
information they would like to have while comparing hospital care. This is a general
finding in judgement and decision-making research because decision-makers often do
not know their own values.28,29 The examples were needed to clarify information
preferences and led to active debates during the focus group session and to a deeper
understanding of the consumers’ values around CPI. In Moser and colleagues’ study, the
use of concrete examples led to similar positive results.30 Hibbard et al,5 stressed the
need for consumers to develop a better understanding of quality of care and current
measures of quality. A recent study in clinical decision-making31 focuses on the social
influence of interaction in decision-making. The concept of ‘shared mind’ as an
underlying process for clarifying individual values between two or more people, in
which new ideas and perspectives emerge, corresponds with our findings using focus
28
Exploring consumer values of comparative performance information
groups. Epstein also underlines the power of multiple perspectives of patients, family,
physicians or other members of the healthcare team.31
Our study showed that consumers expressed values on several levels and that these
values sometimes conflicted with each other. Some values were in favour of the use of
CPI, while other doubts, concerns and principles negatively affected consumers’ views
of CPI. Participants relied on previous routines such as consulting GP, family, friends,
and personal experiences as the basis for choosing a hospital.32,33 Nonetheless, they
were also keen on having a choice. Respondents also appreciated the increased
transparency in care, and the effect on patient empowerment.34,35 However, the
responsibility was difficult, and participants felt that choosing a hospital was a bit of a
burden. Fear of disrupting existing relationships was another consideration that
prevented people from using CPI to choose a hospital. The relationship with their GP
was of much more value to them, which corresponds with UK and Dutch studies.35–37 A
new perspective that was raised in our study is that not using CPI was explained as a
strategy to prevent regret for a wrong decision as participants could not foresee the
consequences of using CPI. Trusted others can help them to clarify the possible
consequences and compare this with personal values. Finally, as in other studies our
participants placed much value on reliability and distrust the current CPI.24,30
The use of CPI by healthcare consumers is a complex process in which values, rather
than rationality, play an important role. Our findings show that consumers need help
from others to solve conflicting values, to develop a firmer understanding of the quality
of care concept, and to move forward in making active and informed hospital choices.
Limitations and strengths
Our study adds to the understanding of the role of values, appraisal and judgement of
CPI among Dutch consumers. However, the study has a number of limitations that limit
the generalisability. First, the small sample size. We intended to include more
participants, but recruitment was problematic, despite the use of various strategies.
Owing to this difficult recruitment, we were not able to achieve our planned age
distribution for the study. Moreover, 25% of the recruited participants did not attend
the focus group session, underlining our finding that current CPI is valued marginally.
Second, the data collection and analysis took place simultaneously, as the time frame
of three months for focus group meetings was short. A strength of this study is the use
of real-world examples, which was very helpful for the discussion.
Implications
Our results make clear that simply providing information is not enough and will not
enhance the usage of CPI. CPI is valued only marginally, and as such it is not yet used
by service users as an additional information source, nor does it challenge healthcare
29
2
Chapter 2
consumers’ current selection of hospitals. Our findings also show that more focus is
needed on eliciting the underlying values of consumers. Several studies, including ours,
have stressed the need for an agent that can support healthcare consumers in choosing
and can coordinate on their behalf when bringing the choice into practice.38,39 Such an
agent might be able to elicit consumers’ preferences, clarify the values of CPI and
preferably integrate this into the referral processes of primary care providers.
30
Exploring consumer values of comparative performance information
REFERENCES
1.
2.
3.
4.
5.
6.
7.
8.
9.
10.
11.
12.
13.
14.
15.
16.
17.
18.
19.
20.
21.
22.
23.
24.
25.
26.
27.
Robertson R, Burge P. The impact of patient choice of provider on equity: analysis of a patient
survey. J Health Serv Res Policy 2011;16(Suppl 1):22–8.
Victoor A, Delnoij DM, Friele RD, Rademakers JJ. Determinants of patient choice of healthcare
providers: a scoping review. BMC Health Serv Res 2012;12:272.
Victoor A, Friele RD, Delnoij DM, Rademakers JJ. Free choice of healthcare providers in the
Netherlands is both a goal in itself and a precondition: modelling the policy assumptions underlying
the promotion of patient choice through documentary analysis and interviews. BMC Health Serv
Res 2012;12(1):441.
Fredriksson M. Is patient choice democratising Swedish primary care? Health Policy 2013;111(1):95–
98.
Hibbard JH, Greene J, Daniel D. What is quality anyway? Performance reports that clearly
communicate to consumers the meaning of quality of care. Med Care Res Rev 2010;67:275–93.
Werner RM, Asch DA. The unintended consequences of publicly reporting quality information. JAMA
2005;293:1239–44.
Fasolo B, Reutskaja E, Dixon A, Boyce T. Helping patients choose: how to improve the design of
comparative scorecards of hospital quality. Pat Educ Couns 2010;78:344–49.
Kaiser Family Foundation. Update on Consumers’ View of Patient Safety and Quality Information.
Agency for Healthcare Research and Quality: Menlo Park, CA, 2008.
Fanjiang G, von Glahn T, Chang H, Rogers WH, Safran DG. Providing patients web-based data to
inform physician choice: if you build it, will they come? J Gen Intern Med 2007;22:1463–66.
Harris KM, Buntin MB. Choosing a Health Care Provider: The role of quality information. Princeton,
NJ: Robert Wood Johnson Foundation, 2008.
Berendsen AJ, de Jong GM, Meyboom-de Jong B, Dekker JH, Schuling J. Transition of care:
experiences and preferences of patients across the primary/secondary interface – a qualitative
study. BMC Health Serv Res 2009;9:62.
Groenewoud AS. It’s Your Choice! A study of search and selection processes, and the use of
performance indicators in different patient groups. Veenendaal: Erasmus University Rotterdam,
2008.
Spranca M, Kanouse DE, Elliott M, Short PF, Farley DO, Hays RD. Do consumer reports of health
plan quality affect health plan selection? Health Serv Res 2000;35:933–47.
Kolstad JT, Chernew ME. Quality and consumer decision making in the market for health insurance
and health care services. Med Care Res Rev 2009;66(1 Suppl):28S–52S.
Faber M, Bosch M, Wollersheim H, Leatherman S, Grol R. Public reporting in health care: how do
consumers use quality-of-care information? A systematic review. Med Care 2009;47:1–8.
Rademakers J, Nijman J, Brabers AE, de Jong JD, Hendriks M. The relative effect of health literacy
and patient activation on provider choice in the Netherlands. Health Policy 2014;114: 200–6.
Stein BD, Kogan JN, Essock S, Fudurich S. Views of mental health care consumers on public
reporting of information on provider performance. Psychiatric Services 2009;60:689–92.
Damman OC, Hendriks M, Rademakers J, Spreeuwenberg P, Delnoij DM, Groenewegen PP.
Consumers’ interpretation and use of comparative information on the quality of health care: the
effect of presentation approaches. Health Expect 2012;15: 197–211.
Slovic P. The construction of preference. American Psychologist 1995;50:364–71.
Epstein RM, Peters E. Beyond information: exploring patients’ preferences. JAMA 2009;302:195–97.
Lee YK, Low WY, Ng CJ. Exploring patient values in medical decision making: a qualitative study.
PloS One 2013;8(11): e80051.
Krueger RA, Casey MA. Focus Groups: A practical guide for applied researchers (3e). Thousand
Oaks, CA: SAGE, 2000.
Barbour R. Doing Focus Groups. SAGE: London, 2007.
Schwartz LM, Woloshin S, Birkmeyer JD. How do elderly patients decide where to go for major
surgery? Telephone interview survey. BMJ 2005;331(7520):821.
Sheon N (2007) Overview of Atlas.ti.5.2. San Francisco. www.palmpal.org/atlas.pdf (accessed
05/03/14).
Buetow S. Thematic analysis and its reconceptualization as ‘saliency analysis’. J Health Serv Res
Pol 2010;15:123–25.
Jepson R, Harris FM, Bowes A, Robertson R, Avan G, Sheikh A. Physical activity in South Asians: an
in-depth qualitative study to explore motivations and facilitators. PloS One 2012;7(10):e45333.
31
2
Chapter 2
28.
29.
30.
31.
32.
33.
34.
35.
36.
37.
38 .
39.
32
Lichtenstein S, Slovic P. The Construction of Preference. Cambridge: Cambridge University Press,
2006.
Peters E, Dieckmann NF, Vastfjall D, Mertz CK, Slovic P, Hibbard JH. Bringing meaning to numbers:
the impact of evaluative categories on decisions. J Exp Psychol Appl 2009;15:213–27.
Moser A, Korstjens I, van der Weijden T, Tange H. Themes affecting health-care consumers’ choice
of a hospital for elective surgery when receiving web-based comparative consumer information.
Pat Educ Couns 2010;78:365–71.
Epstein RM. Whole mind and shared mind in clinical decision-making. Pat Educ Couns 2013;90:200–
6.
Kaiser Family Foundation. A National Survey on Consumers’ Experiences with Patient Safety and
Quality Information. Menlo Park, CA: Kaiser Family Foundation, 2004.
Sinaiko AD. How do quality information and cost affect patient choice of provider in a tiered
network setting? Results from a survey. Health Serv Res 2011; 46:437–56.
Coulter A. Engaging Patients in Healthcare. Buckingham: Open University Press, 2011.
Dixon A, Robertson R, Appleby J, Burge P, Devlin N, Magee H. How Patients Choose and How
Providers Respond. London: The King’s Fund, 2010.
Groot de IB Otten W Smeets HJ et al. Is the impact of hospital performance data greater in
patients who have compared hospitals? BMC Health Serv Res 2011;11:214.
Lako CJ, Rosenau P. Demand-driven care and hospital choice. Dutch health policy toward
demanddriven care: results from a survey into hospital choice. Health Care Analysis 2009;17:20–35.
Meinow B, Parker MG, Thorslund M. Consumers of eldercare in Sweden: the semblance of choice.
Soc Sci Med 2011;73:1285–89.
Schlesinger M. Choice cuts: parsing policymakers’ pursuit of patient empowerment from an
individual perspective. Health Econ Policy Law 2010;5:365–87.
Chapter
3
Patients’ expectations of variation in quality of care
relates to their search for comparative performance
information
Nicole Ketelaar
Marjan Faber
Jozé Braspenning
Gert Westert
BMC Health Services Research 2014; 14: 617.
Chapter 3
ABSTRACT
Background: Choice of hospital based on comparative performance information (CPI)
was introduced for Dutch healthcare consumers at least 5 years ago, but CPI use has
not yet become commonplace. Our aim was to assess the role of patients’ expectations
regarding variation in the quality of hospital care in determining whether they search
for CPI.
Methods: A questionnaire (for a cross-sectional survey) was distributed to 475
orthopaedic patients in a consecutive sample, who underwent primary hip or knee
replacement in a university, teaching, or community hospital between September 2009
and July 2010.
Results: Of the 302 patients (63%) who responded, 13% reported searching for CPI to
help them choose a hospital. People who expected quality differences between
hospitals (67%) were more likely to search for CPI (OR =3.18 [95% CI: 1.02–9.89]; p
<0.04) than those who did not. Quality differences were most often expected in
hospital reputation, distance, and accessibility. Patients who did not search for CPI
stated that they felt no need for this type of information.
Conclusion: Patients’ expectations regarding variation in quality of care are positively
related to their reported search for CPI. To increase the relevance of CPI for patients,
future studies should explore the underlying reasoning of patients about meaningful
quality-of-care variation between hospitals.
34
Patients’ expectations of variation in quality of care
BACKGROUND
The public release of comparative performance information (CPI) is common in many
countries.1 The 2006 Healthcare Market Regulation Act led to better availability of CPI
in the Netherlands. Choice for consumers and providers became a cornerstone of this
new healthcare system based on market elements and competition. CPI can include
information about service, patient experiences, and quality indicators for clinical care
(structure, process, and patient outcomes). The CPI in the Netherlands includes items
related to the hospital (e.g. ranking of 100 hospitals) and to condition-related factors
(e.g. patient experiences, waiting lists, and annual patient volumes). The purpose of
making this information publicly available is to enable healthcare consumers to choose
high-quality healthcare2 and to empower them to make an informed choice about
healthcare.3
There is no firm evidence that CPI influences patient choices.4 American and British
studies have shown that the actual use of CPI for hospital care is restricted to 4–14% of
the consumers5,6, while the idea of choice appeals to most consumers.7 Several Dutch
studies of CPI for total hip or knee replacements have been performed.8-11 The patients
in Moser and colleagues’ study considered CPI to be an additional source of information
when they were preparing for a doctor’s appointment. They benefited from the
information most when they had to undergo a total hip or knee replacement for the
first time.9 American patients who report a lack of hospital choice for total hip or knee
replacement are more likely to be dissatisfied with their surgery.8 This observation
suggests that encouraging patients to engage in provider selection based on quality of
care would improve their satisfaction.
The low level of CPI search behaviour for selecting a hospital is partly due to the
consumer’s previous experience with a specific hospital12-14 and the unfamiliarity of
using CPI for hospital selection15,16, as well as the role of the referring physician. Dutch
research shows that many patients prefer their primary care physician to be involved in
the choice of hospital so they can either take the physician’s advice or delegate the
decision.17-19 Furthermore, many consumers are unaware of the opportunity to consult
CPI.6,20,21
To raise the level of CPI awareness (an important step in a causal chain towards using
CPI) an attendant motive is required so that consumers feel a need for this
information.21 One consumer choice model takes awareness as a starting point.18
Consumers must be aware that there is CPI, and that it is possible to make a choice.
We presume that patients’ expectations are an important pre-step and have a
subjective influence on awareness. Studies show that, when consumers start searching
for information, the questions and expectations already in their minds will drive the
35
3
Chapter 3
direction of their search.22-24 Having expectations about the practice variation of
hospitals and therefore perceiving a risk of receiving poor care might be a motive for
using CPI.
Despite on-going efforts and investments in the collection, production, and
dissemination of CPI for the public15, there is no firm evidence that CPI influences
patient choices.4 In an attempt to bridge this gap, we hypothesise that consumers who
expect to find quality differences between hospitals are those who search for CPI. We
tested whether patients’ expectations of variation in quality of care affect their
reported search behaviour for hospital performance information, then we adjusted for
potential confounders. Furthermore, respondents were asked in what ways they
expected hospitals to differ. We also asked them about the most important reasons why
they did not search for CPI. We obtained data from patients who had recently been
admitted to hospital for an elective total hip or knee replacement. Admission to a
hospital for elective surgery can be planned in advance which gives patients time to
search for and look into CPI. These replacement procedures are provided at all 87
hospitals in the Netherlands.
METHODS
In a cross-sectional study of three types of hospitals, we used a consecutive sampling
strategy to recruit 475 patients undergoing a total hip or knee replacement, and we
invited them to participate in a paper-based survey. To make our sample
representative, we included patients from a university hospital, a teaching hospital,
and a community hospital. We included adult patients undergoing a primary hip or knee
replacement because we expected that previous surgical experience (as in the case of a
secondary replacement) would bias the selection. The annual patient volumes were 53
for primary hip replacements and 64 for knee replacements in the university hospital;
232 and 170, respectively, in the teaching hospital; and 236 and 153, respectively, in
the community hospital. Data collection took from 5 to 9 months (September 2009
through July 2010). The nurse or anaesthetist who prepared the patient for surgery
personally gave the survey questionnaire to the patient at the preoperative
appointment 4 weeks prior to the operation. Reminders were sent 2–3 weeks later in
the teaching and community hospitals, but no reminders were sent in the university
hospital because permission for this was not granted.
The institutional ethics committee reviewed the study protocol in accordance with
local regulations in the Netherlands, and they concluded that the study was not subject
to the Dutch Medical Research Involving Human Subjects Act.
36
Patients’ expectations of variation in quality of care
The primary outcome measure was the self-reported search for CPI, i.e. ‘Did you search
for additional CPI to compare hospitals after it became clear that you needed surgery?’
(answer: yes or no). Respondents did not search for CPI were asked to select their
motive from a list of seven pre-listed reasons with an option to add one. The reasons
were based on a literature search (unpublished search) and focus group interviews with
consumers.25 Other measures concerned the previous treatment in the selected
hospital: consumers’ perception of being well-informed to make a decision, the general
practitioner’s (GP’s) role in advising a choice of hospital (all dichotomous variables),
and expectations of variation in quality of care between hospitals (large, small, or no
difference in quality of care). If respondents expected quality-of-care differences, they
were asked to specify these differences for 14 pre-listed factors that were available
from the Dutch internet sites www.kiesbeter.nl (make better choices) and
www.independer.nl. Both sites based the factors they listed on a set of quality
indicators that the Health Care Transparency Programme listed in 2009.26 Focus group
interviews with consumers25 and the annual list of a Dutch magazine, Elsevier27, also
presented factors to be considered. The factors included the available CPI for hip and
knee replacements in the Netherlands, which gave the name of the organisation and
the clinical performance of the care providers. Formally, ‘distance’ and ‘reputation’
are not CPI, but they were included because they are important to patients.9,11,28 In this
sense, such information can be seen as part of the performance of the healthcare
system. We used a broad definition of performance, as did Van Loon and Tolboom, who
defined three information types: (1) factual information (names, addresses, and type of
provider), (2) quality information based on performance, and (3) quality information
based on consumer experience.29 The questionnaire contained items about
demographic variables (age, education, and type of replacement [hip or knee]), and
hospital characteristics (type and patient volumes for hip and knee replacements).
We used descriptive statistics and frequency tables to describe our study population’s
demographic variables (age, gender, and education), previous treatment in the current
hospital, awareness that hospital performance information is available for comparison,
receipt of choice options from GPs, the search for CPI, and expectations of quality
differences in hospital care (Table 1). To address the issue of representativeness, we
compared the characteristics of the participants in our study with the characteristics of
a larger sample of 1508 Dutch patients who underwent a hip or knee replacement.30
We included data only for those respondents who provided valid answers for the core
items of our study: search for CPI and expectations of quality differences in hospital
care. The respondents were dichotomised into a group 65 years or younger, and a group
older than 65 years. The education variable (the level of education) was measured on a
37
3
Chapter 3
four-point scale (none, low, middle, and high), and it was dichotomised for analytical
purposes into low and high levels of education.
We used logistic regression to analyse the relationship between expectations of quality
differences (independent variable) and searching for CPI (dependent variable) for
patients who underwent a hip or knee replacement (Table 2, model 1). In order to
correct for possible confounders, we also performed this univariate analysis for the
demographic variables (e.g. age, gender, education, and type of replacement), hospital
type, GPs’ role in advising choice of hospital, awareness of available information for
comparing hospital performance, and previous treatment at the current hospital.
Potential confounders (p <0.2) in univariate analyses were added to the multivariate
model, and we examined their effects on the beta coefficients. Any variables resulting
in a change in the beta coefficient of more than 10% were included in the final model.
We compared the univariate analysis (model 1) with the multivariate analysis (model 2)
for odds ratios (ORs) with 95% confidence intervals (95% CIs). An association was
considered statistically significant for p <0.05.
We present descriptive statistics for the factors that we expected to differ between
hospitals, as well as statistics for the reasons for not searching CPI. In describing these
reasons, we distinguished between respondents who expected differences in the quality
of hospitals and those who did not. We used SPSS 18.0 for all analyses.
RESULTS
Study population
Of the 475 questionnaires sent out, 302 were returned completed (response rate 63%).
279 questionnaires had valid answers for the core items: search for CPI and
expectations of quality differences in hospital care. Table 1 shows a comparison of
consumer characteristics differentiated by the type of hospital. We compared our
respondents with a sample of 1508 Dutch patients who underwent a hip or knee
replacement.30 The age distribution of our respondents was 40% for those aged 65 years
or less and 60% for those older than 65 years.
For the sample of 1508 patients, the age distribution was 30% and 70%, respectively.
While this was more or less similar, the samples showed greater differences for gender:
43% were male in our study versus 28% in the sample of 1508 patients; and for a low
level education, 19% and 28%, respectively.
38
Patients’ expectations of variation in quality of care
Table 1. Characteristics of the participants differentiated by type of hospitala
Total
University
hospital
n
%
45
55
37
45
45
56
35
44
64
79
17
21
58
72
23
28
27
34
53
66
58
72
23
28
22
27
Teaching
hospital
N
%
65
57
49
43
42
39
67
62
86
76
27
24
69
61
44
39
18
16
98
84
94
84
18
16
23
26
n
%
Women
158
57
Men
121
43
Age ≤ 65*
107
40
>65*
163
60
Low level of education
222
81
High level of education
52 19
Total primary hip replacementsb
179
65
Total primary knee replacements
97 35
Received no previous treatment in current hospital
59 21
Received previous treatment in current hospital
217
79
Did not receive hospital choice options from GPs
217
80
Received hospital choice options from GPs
55 20
24
Unaware that they could compare hospital
67
performance
77
87
Aware that they could compare hospital performance 210
76
60
73
Did not search for CPI
242
87
68
83
97
85
Searched for CPI
37 13
14
17
17
15
Expectations of quality differences in hospital care
32
36
No differences
91 33
17
21
52
59
Yes, small differences
137
49
37
45
17
19
Yes, large differences
51 18
28
34
a
Data were based on answers from eligible respondents about the search for comparative
information and expectations of quality differences in hospital care
b
Not every score accumulates to 279 because of missing characteristic data
CPI: comparative performance information; GP: general practitioner
Community
hospital
n
%
48
58
35
42
20
25
61
75
72
90
8
10
52
63
30
37
14
17
68
83
65
82
14
18
23
19
69
77
6
77
93
7
38
46
41
49
4
5
performance
Awareness of hospital comparison information and choice
Most respondents (76%) reported that they were aware of the possibility of comparing
hospital quality, and they (72%) were aware that they actually could choose a hospital
for their surgery. Most respondents (73%) reported that they had a choice option. Most
(89%) reported being well-informed about the choice of hospital. Of the total group of
respondents, only 13% said they had searched for hospital CPI before choosing a
hospital for the operation. Among those who did not have a choice option, a minority
(14%) searched for information once they knew they needed surgery. It did not seem to
matter whether patients had a choice option or not; a minority in each group searched
for information.
Expectations of differences in quality of hospital care
Most participants (67%) expected to find differences in hospital quality of care. The
variables gender, type of replacement, GP role in advising a choice of hospital, and
being aware that they could compare hospital performance contributed to the
39
3
Chapter 3
explanation model with p <0.2. However, these variables did not effect a substantial
change (>10%) in the beta coefficients, so were excluded from the final model. Table 2
shows the results of the univariate and multivariate logistic regression analyses for the
relationship between the quality expectations of hospital care and the reported search
for CPI.
Table 2. The relationship between quality expectations and searching for comparative performance
information by patients who underwent a hip or knee replacement (model 1), controlled for
patient and hospital characteristics and previous treatment (model 2)
Model 1a
Univariate analysis
OR [95% CI]
p
Model 2b
Multivariate analysis
OR [95% CI]
Age
<65
1.18 [0.55–2.53]
≥65 (reference)
Hospital type
1.27 [0.54–2.95]
Teaching
Community
0.73 [0.24–2.22]
University (reference)
Previous treatment in current
hospital
0.32 [0.15–0.70]
Yes
No (reference)
Expectations regarding quality
0.00c
differences in hospital care
Yes, small differences
3.71 [1.22-11.27] 0.02c
3.18 [1.02–9.89]
Yes, large differences
7.44 [2.28-24.30] 0.00c
5.05 [1.44–17.77]
No, differences (reference)
a
Based on the answers of 279 respondents; b Based on the answers of 263 respondents
c
P < 0.05; OR [95% CI]: odds ratio [95% confidence interval]
p
0.66
0.58
0.58
0.00c
0.04c
0.04c
0.01c
Previous treatment in the current hospital also appeared to significantly influence the
reported search for CPI.
The univariate model shows that respondents who expected small differences in
hospital quality were more likely to search for CPI than those who did not (OR = 3.71
[95% CI: 1.22–11.27]). For people who expected large quality-of-care differences, this
effect was even greater (OR = 7.44 [95% CI: [2.28–24.3]). The adjusted ORs were 3.18
[95% CI: 1.02–9.89] for the group who expected small differences, and 5.05 [95% CI:
1.44–17.77] for the group who expected large differences. Respondents with previous
treatment in the same hospital less often searched for CPI than those who had no
previous treatment (OR =0.32 [95% CI: 0.15–0.70]).
The participants’ rating of expected hospital quality differences in reputation,
distance, and accessibility are ranked the highest (Table 3).
40
Patients’ expectations of variation in quality of care
Table 3. Factors influencing choice of hospital for total hip and knee replacementsa
Hospital-related factors
1.
Reputation
2.
Distance
3.
Accessibility
4.
Type of hospital (university, teaching, or community)
5.
Ranking list of ‘100 hospitals’
6.
Hospital size
7.
Number of cancelled operations
Condition-related factors
8.
Plan for pre-operative schedules on 1 day
9.
Orthopaedic specialism
10. Patient experiences
11. PROMs
12. Waiting list
13. Annual patient volume
14. Infection rates
a
Data were based on 191 answers from eligible respondents
PROMs: Patient-Reported Outcome Measures
n
107
95
79
51
41
20
1
n
51
50
49
49
45
42
16
3
Reasons for not searching for comparative performance information
Of the respondents who did not search for additional information, 179 said they felt no
need for more information. Others gave far less common reasons: 19 had no internet
access at home, 15 felt more information would create more doubts, 13 said searching
for information was an extra burden, 7 thought choosing a hospital based on CPI was
too much responsibility, 8 did not know where to look, 6 had no skills how to look, 8
had no hospital choice options, and 5 had no time.
Feeling no need was by far the most important reason for not searching for hospital
CPI. Though, this reason was not significant related to the expectations of quality-ofcare differences.
DISCUSSION
Thirteen per cent of our study population searched for CPI to compare hospitals. This is
consistent with the results of American and Dutch studies for similar populations.14,31
Our hypothesis that patients who expect quality differences are those who search for
CPI was confirmed. Previous experience with the hospital is another factor influencing
the search for CPI. Expecting quality of care differences in hospital performance
appears to be a stimulus for searching for CPI, although respondents who underwent
previous treatment in the hospital tended to search less for this information.
In our study, the impact of previous experience on hospital choice was consistent with
Dixon and colleagues’ results.32 They compared the effect of consumer choice during
the referral process in the Netherlands and England. For Dutch patients, ‘being in the
neighbourhood’ or ‘having been there before’ were the most important reasons for
41
Chapter 3
choosing a hospital: patients usually returned to the same hospital.33 Interestingly,
patients prefer being treated at their current hospital, even if they could choose a
better alternative with higher-quality care.34,35 Choosing a familiar hospital instead of
an unknown one suggests that personal experience is a value in itself. While CPI may
indicate the best hospitals, patients may optimise the factors they value most rather
than objectively maximize quality.
Our respondents expected most differences to be in reputation, distance, and
accessibility. As reported in other studies6,12,19, these are known choice factors in
decision-making. Our study population expected differences mainly in the general
performance of the healthcare system rather than in specific condition-related factors
(total hip or knee replacement). The factors for which consumers expect quality
differences may change in the future as consumers become more knowledgeable about
CPI. Dutch studies among patients with a total hip or knee replacement found that both
interpersonal aspects (conduct of doctors) and more technical ones (for example, the
prevention of adverse effects of thrombosis and the specialist area of orthopaedists)
are important to patients.11 Making the concept of quality more meaningful may also
increase consumer interest and need for CPI.15 Bozic and colleagues have confirmed
this statement: they have recently found that patients were very motivated to search
for provider quality. In their study, physician manner and surgical outcomes appeared
to be the most important considerations for selecting a provider for elective total joint
arthroplasty.36
Some studies6,18 led us to expect that unawareness is an important reason why hospital
performance information has little influence on decision-making. An information need
and a sense of urgency are necessary ingredients for awareness and interest in this type
of information. Although most of our respondents (76%) were aware that they could
compare hospital performance and choose a hospital (72%), they still did not use CPI.
This discrepancy might be due to their feeling no need for such information. Having
alternative information sources9 or doubt about the trustworthiness of such
information2 could also contribute to this feeling. This discrepancy also implies the
need for a more outspoken reporting of hospital quality that emphasizes differences in
quality. If rational patients assume that these differences are small, then they cannot
be expected to look for and use such information. The ongoing efforts and investments
that go into the collection, production, and dissemination of CPI would then be useless.
Implications
Our study shows that merely making CPI available in the public domain does not result
in its use. This implies that further action, such as applying an implementation
42
Patients’ expectations of variation in quality of care
strategy, is necessary. Other studies see the need for an infrastructure that provides
patients with advice about their choices and helps them in actually choosing.37,38
Future research should explore the concept of patients’ expectations more
comprehensively because the fact that patients’ expectations that quality differences
exist affects their search for CPI, but does not affects their need for CPI. Research
should use the resulting information to determine how fragile or robust these
expectations are.39 Whereas CPI is based on measurable factors, consumer expectations
are more diffuse and individually determined. More development of tailored methods
to assess the understanding of variation in quality of care as a precondition for
acquiring awareness, knowledge, and interest is necessary.
Finally, further research should explore whether the sense of urgency for this
information will increase if the concept of quality of care becomes more meaningful
and patients start to realise that quality and outcomes of care do vary for both
treatment and hospital factors.
Limitations
One strength of our study is the recruitment of respondents in three types of hospitals.
Consumer characteristics differed somewhat among the three settings (Table 1), which
confirmed the validity of our decision to include the three types. One limitation of our
study is that only a minority reported searching for CPI, so we could not make precise
estimations, as is reflected in the large confidence intervals in Table 2. We would have
preferred a more balanced dataset for better data modelling. Another limitation is the
self-reporting of our main outcome, which may have introduced a recall bias. However,
we limited the time between selecting a hospital and completing the questionnaire by
giving it to the participants at their appointments with a nurse or anaesthetist before
surgery. This tactic minimized the time lag between choosing a hospital choice and the
date of surgery.
Conclusions
CPI makes the variation in quality of care between hospitals transparent. This study
shows that the number of people who report having searched for CPI is still limited, but
may increase if patients become more aware of the quality-of-care variation of
hospitals. However, this will be difficult to achieve because people who feel no need
for more information – e.g. based on a lack of expected differences in quality of care –
do not search for CPI. Awareness as a prerequisite for the use of CPI should not be
limited to having knowledge about the existence of CPI and where to find it; awareness
should also extend to the quality-of-care variation of hospitals.
43
3
Chapter 3
REFERENCES
1.
2.
3.
4.
5.
6.
7.
8.
9.
10.
11.
12.
13.
14.
15.
16.
17.
18.
19.
20.
21.
22.
23.
24.
25.
44
Victoor A, Delnoij DM, Friele RD, Rademakers JJ. Determinants of patient choice of healthcare
providers: a scoping review. BMC Health Serv Res 2012; 12:272.
Marshall M, McLoughlin V. How do patients use information on health providers? BMJ 2010;
341:c5272.
Fasolo B, Reutskaja E, Dixon A, Boyce T. Helping patients choose: how to improve the design of
comparative scorecards of hospital quality. Patient Educ Couns 2010; 78(3):344–349.
Ketelaar NABM, Faber MJ, Flottorp S, Rygh LH, Deane KHO, Eccles MP. Public release of
performance data in changing the behaviour of healthcare consumers, professionals or
organisations. Cochrane Database Syst Rev 2011; 11:CD004538.
Fung CH, Lim YW, Mattke S, Damberg C, Shekelle PG. Systematic review: the evidence that
publishing patient care performance data improves quality of care. Ann Intern Med
2008;148(2):111–123.
Harris KM, Buntin MB. Choosing a Health Care Provider: The Role of Quality Information. Princeton,
NJ: Robert Wood Johnson Foundation, 2008.
Coulter A. Engaging Patients in Healthcare. London: Open University Press, 2011.
Losina E, Plerhoples T, Fossel AH, Mahomed NN, Barrett J, Creel AH, Wright EA, Katz JN. Offering
patients the opportunity to choose their hospital for total knee replacement: impact on
satisfaction with the surgery. Arthritis Rheum 2005;53(5):646–652.
Moser A, Korstjens I, Van der Weijden T, Tange H. Patient’s decision making in selecting a hospital
for elective orthopaedic surgery. J Eval Clin Pract 2010;16(6):1262–1268.
Moser A, Korstjens I, van der Weijden T, Tange H. Themes affecting healthcare consumers’ choice
of a hospital for elective surgery when receiving web-based comparative consumer information.
Patient Educ Couns 2010;78(3):365–371.
Zwijnenberg NC, Damman OC, Spreeuwenberg P, Hendriks M, Rademakers JJ. Different patient
subgroup, different ranking? Which quality indicators do patients find important when choosing a
hospital for hip- or knee arthroplasty? BMC Health Serv Res 2011;11:299.
Kolstad JT, Chernew ME. Quality and consumer decision making in the market for health insurance
and health care services. Med Care Res Rev 2009;66(1 Suppl):28S–52S.
Marang van de Mheen PJ, Dijs-Elsinga J, Otten W, Versluijs M, Smeets HJ, Vree R, van der Made
WJ, Kievit J. The relative importance of quality of care information when choosing a hospital for
surgical treatment: a hospital choice experiment. Med Decis Making 2011;31(6):816–827.
Schwartz LM, Woloshin S, Birkmeyer JD. How do elderly patients decide where to go for major
surgery? Telephone interview survey. BMJ 2005;331(7520):821.
Hibbard JH, Greene J, Daniel D. What is quality anyway? Performance reports that clearly
communicate to consumers the meaning of quality of care. Med Care Res Rev 2010; 67(3):275–293.
Trigg L. Patients’ opinions of health care providers for supporting choice and quality improvement.
J Health Serv Res Policy 2011;16(2):102–107.
Damman OC, Spreeuwenberg P, Rademakers J, Hendriks M. Creating compact comparative health
care information: what are the key quality attributes to present for cataract and total hip or knee
replacement surgery? Med Decis Making 2012;32(2):287–300.
Faber M, Bosch M, Wollersheim H, Leatherman S, Grol R. Public reporting in health care: how do
consumers use quality-of-care information? A systematic review. Med Care 2009;47(1):1–8.
Lako CJ, Rosenau P. Demand-driven care and hospital choice: Dutch health policy toward demanddriven care: results from a survey into hospital choice. Health Care Anal 2009;17(1):20–35.
Jha AK, Epstein AM. The predictive accuracy of the New York State coronary artery bypass surgery
report-card system. Health Aff 2006;25(3):844–855.
Wilson TD. Trends in…a critical review: information behaviour: an interdisciplinary perspective.
Inform Process Manag 1997;33(4):551–572.
Adam JA, Khaw FM, Thomson RG, Gregg PJ, Llewellyn-Thomas HA. Patient decision aids in joint
replacement surgery: a literature review and an opinion survey of consultant orthopaedic
surgeons. Ann R Coll Surg Engl 2008;90(3):198–207.
Brashers DE. Communication and uncertainty management. J Commun 2001;51:477–497.
Kivits J. Researching the ‘informed patient’. Information, Communication & Society 2004;7(4):510–
530.
Ketelaar NABM, Faber MJ, Westert GP, Elwyn G, Braspenning JC. Exploring consumer values of
comparative performance information for hospital choice. Qual Prim Care 2014; 22(2):81–89.
Patients’ expectations of variation in quality of care
26.
27.
28.
29.
30.
31.
32.
33.
34.
35.
36.
37.
38.
39.
Health Care Transparency Programme. Kwantitatieve Analyse Indicatoren Zichtbare Zorg.
Ziekenhuizen - 2008 Data. Den Haag: Healthcare Inspectorate, 2009:1–443.
Elsevier. The Best Hospitals 2008. In Elsevier, Volume 74. Amsterdam: Reed Elsevier, 2008.
Epstein AJ. Effects of report cards on referral patterns to cardiac surgeons. J Health Econ 2010;
29(5):718–731.
Van Loon AJM, Tolboom RAL. Design Report “Choose Better” 2006–2007 [In Dutch: Ontwerprapport
KiesBeter.nl 2006–2007]. Bilthoven: RIVM, 2005.
Stubbe JH, Gelsema T, Delnoij DM. The Consumer Quality Index Hip Knee Questionnaire measuring
patients’ experiences with quality of care after a total hip or knee arthroplasty. BMC Health Serv
Res 2007; 7:60.
de Groot IB, Otten W, Smeets H, Marang-van de Mheen P, The Choice 2 Study Group. Is the impact
of hospital performance data greater in patients who have compared hospitals? BMC Health Serv
Res 2011; 11:214.
Dixon A, Robertson R, Bal R. The experience of implementing choice at point of referral: a
comparison of the Netherlands and England. Health Econ Policy Law 2010;5(3):295–317.
Dijs-Elsinga J, Otten W, Versluijs MM, Smeets HJ, Kievit J, Vree R, van der Made WJ, Marang-van
de Mheen PJ. Choosing a hospital for surgery: the importance of information on quality of care.
Med Decis Making 2010;30(5):544–555.
Boonen LH, Donkers B, Schut FT. Channeling consumers to preferred providers and the impact of
status quo bias: does type of provider matter? Health Serv Res 2011;46(2):510–530.
Laverty AA, Smith PC, Pape UJ, Mears A, Wachter RM, Millett C. High-profile investigations into
hospital safety problems in England did not prompt patients to switch providers. Health Aff
2012;31(3):593–601.
Bozic KJ, Kaufman D, Chan VC, Caminiti S, Lewis C. Factors that influence provider selection for
elective total joint arthroplasty. Clin Orthop Relat Res 2013;471(6):1865–1872.
Meinow B, Parker MG, Thorslund M. Consumers of eldercare in Sweden: the semblance of choice.
Soc Sci Med 2011;73(9):1285–1289.
Schlesinger M. Choice cuts: parsing policymakers’ pursuit of patient empowerment from an
individual perspective. Health Econ Pol Law 2010; 5(3):365–387.
Bowling A, Rowe G, Lambert N, Waddington M, Mahtani KR, Kenten C, Howe A, Francis SA. The
measurement of patients’ expectations for health care: a review and psychometric testing of a
measure of patients’ expectations. Health Technol Assess 2012;16(30):i–xii. 1–509.
45
3
Chapter
4
Recognition of physiotherapists’ expertise in
Parkinson’s disease
Nicole Ketelaar
Marten Munneke
Bas Bloem
Gert Westert
Marjan Faber
BMC Health Services Research 2013; 13: 430.
Chapter 4
ABSTRACT
Background: Publicly available information comparing performance across quality and
costs has proliferated in recent years, both about individual healthcare professionals
and hospitals. This type of information is now becoming increasingly available for
physiotherapists with expertise in Parkinson’s disease (PD). Our study aimed to explore
the ability of people with Parkinson’s disease to recognise expertise, and to what
extent respondents selectively choose such expert physiotherapists.
Methods: We used claim data from the period 2009–2010 to select customers with PD
who claimed physiotherapy. A random sample of 500 eligible respondents received a
paper-based survey. We used descriptive statistics to compare the respondent
characteristics, a qualitative programme to analyse the qualitative items, and
univariate and multivariate regression.
Results: Most respondents (89%) took their referring physician’s advice when selecting a
physiotherapist, although this advice rarely was supported with arguments. The
remaining respondents (11%) searched for comparative performance information about
physiotherapists. Respondents who recognised the added value of PD expertise among
physiotherapists were 3.28 times as likely to search for comparative performance
information as those who did not understand. Respondents were willing to switch to an
expert physiotherapist (68%), and this willingness increased if they recognised the value
of PD expertise (p < .001).
Conclusion: The participants were able to recognise certain aspects of expertise.
Though they showed relatively few signs of selectively choice behaviour for expert
physiotherapists. Both respondents and referring professionals need more
understanding about the added value of an expert physiotherapist, to foster selective
provider choice.
48
Recognition of physiotherapists’ expertise in Parkinson’s disease
BACKGROUND
The release of information in the public domain about the performance of healthcare
providers has become a strategy for improving the quality of healthcare.1 Providing
such comparative performance information (CPI) may empower and enable patients to
identify and choose high-quality providers.2,3 While we have an increasing
understanding of motivating factors for selective provider choice regarding elective
conditions4-6, or primary care physicians7 there is no knowledge available about
whether this evidence also applies to people with a chronic disorder such as Parkinson’s
disease (PD). Our aim is to explore whether people are aware of, and able to recognise
expertise, as an expression of quality of care, in PD among physiotherapists.
There are an estimated 1.2 million people with Parkinson’s disease (PwP) in Europe.8
Physiotherapy is part of the treatment for many PwP.9 In the Netherlands and the
United Kingdom, 54–60% of all PwP receive physiotherapy.10,11 In the former,
physiotherapy is mainly provided in community-based settings, in provider-owned
practices. The reimbursement of the first 21 physiotherapy sessions, on annual basis,
depends on the consumers additional insurance package. Though, for the treatment of
PD there is an unlimited reimbursement out of the basic insurance after these twentyone treatments. Dutch patients have free access to physiotherapy since 2006 so they
can self-refer to a physiotherapist, which is considered to be a structure premise for
selective provider choice.12
More than 75% of the Dutch allied health professionals report a lack of PD-specific
expertise. More than half of them were unfamiliar with the treatment options of other
professionals, and they had not participated in relevant educational programmes.9 In
response to this gap of knowledge, a Dutch multidisciplinary network, ParkinsonNet,
was developed and implemented to increase the PD expertise of physiotherapists.13
ParkinsonNet is a multifaceted intervention that includes several elements (Table 1).
Table 1. Five core elements of ParkinsonNet14
1. Delivering care according to evidence-based guidelines
2. Continuous education and training of therapists (including physiotherapists, occupational therapists,
and speech therapists)
3. Structured and preferred referral to ParkinsonNet therapists by neurologists, enabling each therapist
to attract a large volume of patients to increase expertise
4. Encouraging communication and regional collaboration with referring physicians
5. Promoting visibility of the available expertise for both patients and professionals
Launching of the sixty-fifth regional ParkinsonNetnetwork in 2010, national coverage
was achieved. Currently, about 2700 professionals are connected throughout the
49
4
Chapter 4
Netherlands.14 Although, physiotherapists can be expert in PD without being connected
to this multidisciplinary network, the term ‘expert physiotherapist’ is exclusively used
throughout this paper for physiotherapists who are connected to ParkinsonNet. An
expert physiotherapist received additional training in treating PwP. Evidence shows
that specific treatment options provided by expert physiotherapists are more beneficial
to patients.15,16 Also, expert physiotherapists show better guideline adherence scores
compared to generically active physiotherapists and their PwP volume is larger.9
Awareness of referral options for other professional disciplines17 and knowledge and use
of the Dutch PD guideline13 are necessary to attain expertise in PD. The implementation
of ParkinsonNet increased the number of PwP treated by each physiotherapist.18
However, approximately 70% of the PwP are still treated by a general physiotherapist.18
This raises the questions whether PwP are aware of PD expertise in physiotherapy and
whether they understand its value, i.e. these are, apart from free provider choice, two
additional relevant conditions for selectively choosing a provider.
This paper focuses on ability of PwP to recognise expertise in PD among
physiotherapists, as a potential indicator of selective provider choice. We consider this
in several ways: by seeking the perspective of PwP about the PD expertise of their
physiotherapist, by ascertaining the descriptions of PD expertise from the perspective
of PwP in a qualitative way, and whether physiotherapists’ PD-related expertise
influences the search for CPI among PwP. In addition, we examine the role of referring
physicians. We take into account factors that might affect provider selection such as
consumer characteristics (e.g. age, education, and internet use), knowing where to
search for CPI, the willingness to switch, and respondents’ expectations regarding
variation in quality of care.
METHODS
Design and study population
Our paper-based survey focused on the selective provider choice for expert
physiotherapists in a cohort of people with PD. We selected PwP on the basis of claim
data in the period 2009–2010 of a Dutch healthcare insurance company that accounts
for 20% of the Dutch insurance market. We approached eligible candidates, that is,
people who were registered as having PD in the diagnosis–treatment combination
(Dutch version of diagnosis related groups) combined with an episode of physiotherapy
in the year prior to our survey. Only consumers who received physiotherapy treatment
for PD were included because we wanted to focus on peoples’ capacity to recognise
physiotherapists’ PD expertise. People who had had physiotherapy for PD for several
years were excluded; the decision-making process (deliberate or not) had to be recent
to exclude recall bias. A total of 886 patients met these eligibility criteria. We sent
50
Recognition of physiotherapists’ expertise in Parkinson’s disease
surveys to 500 participants randomly selected from this group. Due to privacy
legalities, the insurance company drew the sample. According to local regulations in
the Netherlands (Commission involving human subjects research) (CMO) region ArnhemNijmegen this study did not need approval of the ethical review board.
Measures and data collection
The survey consisted of 37 items. The first items ascertained the treatment with
physiotherapy for PD in the past year to satisfy our inclusion criteria. We used five
items to operationalise the recognition of physiotherapists’ PD expertise. Two items
explored the level of awareness and the descriptions of PD expertise from the
perspective of the participants: awareness of the existence of expert PD
physiotherapists (dichotomous variable), and participants were asked to estimate
whether their physiotherapist was an expert in PD. This latter item was answered on a
three-point scale: yes, no, or do not know. We compared the participants’ views of
physiotherapists’ expertise on the basis of claim data with the ParkinsonNet data to see
if they correlated. The respondents answered an openended question that dealt with
their assessment of physiotherapists’ expertise. We categorised the answers into eight
core themes based on a framework for patient centeredness in PD19 and quality of care
domains as formulated by the Institute of Medicine.20 We estimated the search for CPI
by addressing the last two aspects used to operationalise the recognition of
physiotherapists’ PD expertise: recognition of the added value of PD expertise, and
whether participants paid attention to PD expertise among physiotherapists when they
selected a provider (all dichotomous variables). Knowing where to look for CPI was a
dichotomous variable. Expectations of variation in the quality of care between a
generic physiotherapist and an expert physiotherapist formed a categorical variable,
determined on a four-point scale: yes, large differences; yes, small differences; do not
know; or no differences. The willingness to switch to an expert physiotherapist was
determined on a five-point scale: most likely, likely, unlikely, most unlikely, and do not
know. We dichotomised this into likely (most likely, likely) versus unlikely (most
unlikely, unlikely, do not know) because the distribution of this variable was positively
skewed.
We assessed the role of referring physicians with the following questions: do referring
physicians provide you multiple choice options for physiotherapists about where you
could go to? Did your referring physicians give you an advice to which provider you
should go for the best treatment? If so, is this advice accompanied by arguments and by
CPI, and do you take this advice? (All of these are dichotomous variables).
We asked the participants which attributes they searched for when choosing a
physiotherapist, and for which of these attributes referring providers supplied them
51
4
Chapter 4
with information. The attributes contain items about services and quality of care. The
first three items were available for PwP at the time the survey was send out, we added
items 4 and 5 because we expected that these items soon become available as well
and/or these items were common items with other (elective) conditions.
1. ParkinsonNet membership, e.g. practices connected to this network
2. Information regarding the added value of being treated by an expert PD
physiotherapist
3. Distances to physiotherapy practices
4. Physiotherapists with PD expertise
5. Experiences of PwP who were treated by an expert physiotherapist.
We also ascertained the demographics of the study participants. We treated age as a
continuous variable. The variable of educational level was described as none/low,
average, or high. We used the stages defined by Hoehn and Yahr21 to describe the selfreported disease characteristics.
The Hoehn and Yahr (HY) stages range from no PD signs in stage 0 to needing a
wheelchair or being bedridden in the most severe stage 5. Patients’ disease severity
was classed as mild (HY stages 0–1), moderate (HY stages 2–3), or severe PD (HY stages
4–5). The survey was field tested and optimised by ten patients and four PD
researchers. The final survey was sent by post, and a reminder was posted 2 weeks
later. Data were collected in October and November 2010.
Analysis
The survey contained four qualitative questions. The filled-out paper-copy surveys were
scanned and transcribed into an electronic format, by means of an automated process
(Teleform). We did not use a separate programme for the qualitative analysis, such as
Atlas.ti, as it did not bring much benefit for the rather small amount of qualitative
data. Though we did apply the principles of thematic analysis (conducted by NK and
MF), supported by the PD-specific framework for patient centeredness19 and a general
framework for quality of care.20 Descriptive statistics were calculated for each survey
item and compared with respondent characteristics. We explored the association
between how patients value PD expertise and the search for CPI (treated as a
dependent variable determined by ‘yes’ or ‘no’). We used univariate logistic regression
analysis to separately examine the associations between the independent variables and
the search for CPI. The independent variables were: age, education, internet use,
awareness of the existence of expert PD physiotherapists, an understanding of the
added value of an expert PD physiotherapist, prior attention to physiotherapists’
expertise, knowing where to search, expected quality differences between generic
physiotherapists and expert physiotherapists, willingness to switch, and the provider
options named by referring physicians. Statistically significant variables (p < 0.05) were
52
Recognition of physiotherapists’ expertise in Parkinson’s disease
included in the stepwise forward multivariate logistic regression analysis. We
calculated the outcomes separately for each independent variable while controlling for
the other variables in the model. We presented these outcomes with odds ratios (ORs)
and 95% confidence intervals (CIs). We used SPSS 18.0 to carry out the analyses.
RESULTS
Demographics
In total, 380 respondents completed the survey (gross response rate=76%), and
320 surveys were analysed (net response rate=64%). The 60 participants whose surveys
were excluded had not received physiotherapy for PD, but for another medical
condition. Table 2 presents the background characteristics of the study population.
Table 2. Background characteristics of study population
General characteristics
Gender
Mean age in years ± SD
Level of education
Male
n
147
72
%
56
±9
None/low
Average
High
165
91
39
56
31
13
55
211
36
85
18
71
11
28
4
Residential status
Alone
Together
Other situation
Internet use
Yes
Specific Parkinson’s disease characteristics
Diagnosis
Parkinson’s disease
Atypical parkinsonism
Unknown
Hoehn and Yahr stage (self-reported)
Mild (0–1)
Moderate (2–3)
Severe (4–5)
Yes
Use of physiotherapy in the past year
Because of missing data in the background characteristics, not every
320
292
10
4
95
3
2
83
30
127
41
87
29
320
100
score accumulates to the total of
Quality of care for Parkinson’s disease
Sixty percent of the study population expected quality differences in the care provided
by generic physiotherapists and expert physiotherapists, and 34% of them expected
these variations would be large. A minority of participants (5%) expected no quality
differences, and the remaining respondents (35%) said they did not know. Of those who
expected quality differences, 33% did not know what kind of value an expert
physiotherapist could add. In total, about half the study population (51%) did not know
53
Chapter 4
the added value of an expert physiotherapist. Participants who expected to find quality
differences were younger than participants who did not expect quality differences (71 ±
10 years versus 74 ± 8 years, p = .009).
More than two-thirds of the respondents (68%) were willing to switch to an expert
physiotherapist if it turned out that their current physiotherapist had no PD expertise.
The distance the participants were willing to travel to see an expert physiotherapist
was 5 km (interquartile range: 2–11 km). Respondents who had previously heard about
expert physiotherapists were more willing to switch (82% versus 54%, p < .001) and
respondents expecting differences in the quality of care were also more likely to switch
(87% versus 31%; p < .001).
Recognition of physiotherapist expertise in Parkinson disease
Most participants (74%) had already heard about expert physiotherapists. Fewer
participants (46%) said they had previously paid attention to whether the
physiotherapist was an expert in PD before selecting a physiotherapist. Participants
who had previously heard about physiotherapists with PD expertise had a higher
educational level than those who did not know about expert physiotherapists (p =
.001). Awareness of expert physiotherapists was also related to age. Participants who
were aware of expert physiotherapists were younger than those who were unaware
(71 ± 10 years versus 76 ± 8 years, p = .001).
We asked those who had already heard about expert physiotherapists whether they
were treated by an expert PD physiotherapist (n=229). More than 70% asserted they
were treated by an expert, 12% stated that they were not, and 17% said they did not
know. A comparison of the answers of our study population with ParkinsonNet data
showed that 28% of our respondents were being treated by a ParkinsonNet-affiliated
physiotherapist.
Participants reported various themes describing what physiotherapists’ expertise and
knowledge stands for. Table 3 presents the core themes and the underlying descriptions
that the respondents gave. Thirty percent of the respondents said that
physiotherapists’ treatment, exercises, and information express a degree of expertise.
54
Recognition of physiotherapists’ expertise in Parkinson’s disease
Table 3. Themes related to the perceived expertise of physiotherapists from the patients’
perspective
n
69
Treatment and exercises
‘Expertise is related to the good exercises I have to do
at home’ (Woman, 82 years old)
‘Right exercises for posture and movement’ (Man,
76 years old)
Tailored information and communication
Information about the disease, (practical) ‘All arguments with the given instructions are correct’
advice, counselling, and communication
(Man, 64 years old)
with physiotherapist
‘He tries to find matching exercises for me in person’
(Man, 70 years old)
‘She knows a lot of Parkinson Disease, I receive good
answers to my Parkinson Disease related questions’
(Woman, 74 years old)
Knowledge of Parkinson disease
Knowledge about and expertise in
‘He knows exactly what to do’ (Man, 78 years old)
‘He knows the Parkinson disease signs and tries the
Parkinson disease symptoms. Noticing
recommended treatment’ (Man, 57 years old)
changes of condition
‘She knows the limitations Parkinson disease imposes
on me, and gives useful suggestions’ (Woman, 82 years
old)
‘Works with several patients who have Parkinson
disease’ (Man, 60 years old)
Effectiveness
Improved condition, increased mobility,
‘I can to move better and my muscles are less stiff
since the treatment’ (Woman, 82 years old)
decreased freezing
‘My condition has improved since this treatment’ (Man,
75 years old)
Cooperation with caregivers
Need for interdisciplinary care. Working
‘He works with the neurologist’ (Man, 73 years old)
‘He is involved in the Park fit studies’ (Man, 71 years
within a network with allied healthcare
old)
providers
Emotional support, empathy, and respect
‘Shows great interest’ (Man, 74 years old)
‘Shows interest, takes me seriously’ (Woman, 77 years
old)
‘Is very patient’ (Man, 88 years old)
Unable to define the expertise of the physiotherapist
Difficult to access the providers’ expertise ‘I do not know which exercises are the best for
patients with Parkinson disease’ (Man, 61 years old)
Treatment evaluation
Providers ask questions about the
‘Takes the time to evaluate the treatment process’
(Man, 70 years old)
treatment
Accessibility of healthcare
Treatment or physiotherapy at home
‘Treats me at home’ (Man, 70 years old)
Total
66
34
25
19
16
4
2
2
103
The role of referring physicians
The respondents were referred by: neurologists (49%), general practitioners (GPs; 18%),
and specialist Parkinson’s nurses (18%). Another 15% of the respondents saw a
physiotherapist on their own initiative. About half the participants who received a
55
4
Chapter 4
referral were provided with additional information regarding expert PD
physiotherapists. Only a minority of the respondents (25%) received multiple choice
options for a physiotherapist. Although 85% of the participants found it important to
choose their own physiotherapist, most (89%) of those with a physician referral took
their physician’s advice when selecting a physiotherapist.
The search for comparative performance information
Most respondents (89%) reported not having searched for CPI when it became clear they
needed physiotherapy. Some respondents gave more than one reason for not searching.
The most important reason was not perceiving any need for more information (60%).
Other reasons were: no internet access at home (29%), not knowing where to search
(15%), not knowing how to search (9%), lack of motivation (13%), not knowing how to
look for information (12%). A smaller group found that more information led to more
doubt (7%), some felt that it was too much responsibility (4%), and some had no time
(3%).
Participants who searched for information (11%) wanted to know about: practices
connected to ParkinsonNet (n=13, 4%), expert PD physiotherapists (n=18, 6%),
physiotherapist practices close to home (n=19, 6%), experiences of patients who
received treatment from an expert physiotherapist (n=7, 2%), and what added value
physical treatment from an expert physiotherapist can give a person with PD (n=11,
3%). Some declared they did not find the information they would have like to have.
Others reported that the information was too general and not trustworthy.
Univariate logistic regression analyses revealed that several variables are associated
with the search for CPI. Respondents’ awareness of expert PD physiotherapists, an
understanding of the added value of an expert physiotherapist, and the willingness to
switch to an expert PD physiotherapist were statistically significant, as were consumer
characteristics (age and internet use). These variables were included in the
multivariate regression analysis.
The stepwise multivariate logistic regression analysis revealed that recognising of
added value of an expert PD physiotherapist was the most important predictor for
search for physiotherapists’ CPI (Table 4). The likelihood of people who recognised
added value searching for information was 3.28 times as great as the likelihood
those who did not recognise (OR=3.28 [95% CI 1.42–7.58]).
56
the
the
the
for
Recognition of physiotherapists’ expertise in Parkinson’s disease
Table 4. Univariate and forward stepwise multivariate regression relationship of searching for
comparative performance information versus background characteristics, awareness, and
understanding physiotherapists’ expertise in Parkinson’s disease
Univariate analyses
OR [95% CI]
P
0.96 [0.93–0.99] 0.02*
Multivariate analyses
OR [95% CI]
P
Age
Educational level
Low (reference)
Middle
1.49 [0.69–3.23] 0.32
High
0.73 [0.20–2.61] 0.62
No internet use
2.12 [1.02–4.39] 0.04*
Not aware of expert PD physiotherapists
3.76 [1.11–12.70] 0.03*
No recognition of added value of expert
3.29 [1.47–7.31] 0.01*
3.28 [1.42–7.58] 0.01*
physiotherapist
No prior attention to physiotherapist expertise
0.57 [0.18–1.82] 0.34
Not knowing where to look for CPI
1.13 [0.30–4.16] 0.86
Quality differences between generic and expert physiotherapists
No differences (reference)
Yes, large differences
1.41 [0.29–6.80] 0.67
Yes, small differences
0.91 [0.17–4.73] 0.92
Do not know
0.26 [0.04–1.56] 0.14
Not willing to switch
3.86 [0.89–16.78] 0.07
Received provider options from referring physician 1.79 [0.77–4.14] 0.17
The total number of respondents included in the analysis was 279. *p < 0.05
CI: Confidence interval; CPI: comparative performance information; OR: odds ratio; PD: Parkinson’s
Disease
DISCUSSION
The ultimate goal of releasing CPI about the quality of expert physiotherapists is to
improve quality of care for people with PD. Consumers of health care have the power
to make a contribution to quality of care in competitive health care system by selective
provider choice. This study shows that PwP identify aspects of expertise that appeared
to align with the IOM-framework for quality of care, and mostly with patient
centeredness. Moreover, the majority of participants (74%) were aware of expert and
non-expert PD physiotherapists. Participants were able to describe what the PD-specific
expertise and knowledge of their physiotherapist means to them. However, we found
little evidence suggesting that the influence on how patients value expertise among
physiotherapists influences the search for CPI and selective provider choice.
Recognition of the additional value of a PD physiotherapist was a strong predictor of
such a CPI search. Yet, about half the patients (51%) had this understanding; therefore,
this situation can be improved. Our study shows that PwP hardly ever selectively chose
a physiotherapist with PD expertise. Most took the physicians’ referral advice (89%),
and the influence of CPI in the decision-making process was limited because only a
minority searched for such information (11%).
57
4
Chapter 4
In terms of the way forward, we first discuss the selective referral behaviour of
physicians. Currently, very few physicians’ selective referrals to expert physiotherapists
occur. Only half the participants were given additional information regarding expert PD
physiotherapists. A King’s Fund publication22 shows that most GPs in the United
Kingdom did not give information about their referrals either. Knowing and recognising
PD expertise are necessary conditions for providers’ selective referrals. Without these
conditions, it is difficult to provide patients with information. It is also important that
referring physicians (e.g. neurologists and GPs) proactively recall this knowledge when
they advise and refer patients to a physiotherapist. Referring physicians should support
consumers choose selectively by discussing their referral options, so that the choice
becomes a matter of shared decision-making.
Second, we focus on the consumers’ selective choice behaviour. Previous studies
suggest that, although consumers value quality information23,24, the use of CPI is
limited1,25,26 among different populations and for a diversity of conditions. Our study
confirms this discrepancy. The respondents valued free provider choice as important,
but usually followed the referring provider’s advice and did not use CPI to choose their
physiotherapist. Previous research shows that once people understand the concept of
quality of care, they give a higher value to the measures of quality performance.27
Further research should address in what way this discrepancy can be countered, for
example by improving the circumstances so that consumers’ intentions and their
behaviour coincide.
Characteristics of expertise, as they were perceived and expressed by PwP in their own
words (Table 3), are connected with the definition of patient-centred care for PD.19
Other quality criteria, like a large PwP volume, having followed specific training in
treating PD and a connection to an expert network, which is important for the members
of ParkinsonNet, were less frequently mentioned. We therefore conclude that the
definition of expertise among respondents was rather narrow. The aspects that were
less frequently mentioned by respondents, might need more attention since recent
evidence shows that PwP allocated to multidisciplinary PD care have better quality of
life, better motor scores, and less depression.28
Moreover, at a time when cost control dominates the health policy agenda, it is more
urgent than ever to support consumers become a force for improving the quality of
healthcare.29 Since the concept of ParkinsonNet is cost-effective guiding patients to
physiotherapists that are connected to ParkinsonNet, might contribute to the
containment of costs in our health care system. In regions where ParkinsonNet was
active, PwP received more physical therapy, there were fewer admissions to nursing
homes, fewer people needed revalidation treatment, and the reimbursed costs were
58
Recognition of physiotherapists’ expertise in Parkinson’s disease
lower.30 These are all reasons for supporting initiatives that enhance knowledge among
PwP about the added value of expert physiotherapists, by means of CPI that covers a
broad definition of expertise and a range of quality of care criteria.
Twenty-eight percent of our study population received treatment from ParkinsonNetaffiliated physiotherapists, while 70% claimed treatment from expert physiotherapists.
This percentage of 28 is in line with previous evidence.18 Physiotherapists can be expert
in PD without being connected to the multidisciplinary network, but the chance of
being treated by an expert outside the network is smaller. Moreover, the
multidisciplinary network has been implemented across the entire country. It is most
likely that respondents overestimate their physiotherapists’ expertise in PD, otherwise
they would have to admit receiving treatment from a physiotherapist who might not
provide them with the best possible healthcare. Their overestimation makes switching
to a physiotherapist with PD knowledge unlikely, as in their view, physiotherapists are
already experts in PD. More consumer knowledge and recognition of expert
physiotherapists might lead to an adjustment of consumers’ views towards the level of
PD expertise among their physiotherapists and eventually to a search of CPI.
Implications
Demonstrating the value of expert PD physiotherapists to referring physicians is
necessary. Further research should explore whether this will encourage selective
referral behaviour. In terms of patient participation, more attention is required to
clarify physicians’ decisions to refer to an expert physiotherapist.
A practical implication is that CPI should be extended and made more accessible for
PwP and their informal caregivers (family and friends). For example by spreading the
information through the patient organisations for PwP on their website or magazine, or
flyers in the waiting room of primary care practices. Further research is needed to
explore how to encourage consumers to use the information. The CPI should also
emphasise in more detail how and in what ways expert PD physiotherapists distinguish
themselves from generic physiotherapists. Both the added value of expert
physiotherapy and the multidisciplinary element should be emphasised.
Limitations
This study is not without shortcomings. First, as the understanding of the added value
of an expert physiotherapist is related to the search for CPI, it would have been better
to have more detailed questions about the perception of added value. For this purpose,
future studies should focus on the perceived added value of PD expertise in a more
extensive way.
59
4
Chapter 4
Second, we do not know whether the participants’ overestimation of the expertise of
their physiotherapist correlates with their satisfaction with their treatment and
physiotherapist in general. We did not take the element of consumer satisfaction into
account. Future work should replicate these findings and control for consumer
satisfaction when asking about physiotherapist expertise in PD. A recent paper shows
that consumers are generally very satisfied with their physiotherapeutic care.31
Third, only a minority searched for CPI so that the influence of the variables could not
always be estimated precisely, which the large confidence intervals reflect.
Fourth, information about the number of physiotherapy episodes respondents had in
the past year would brought us more insight whether the choice for a physiotherapist
was temporally or more on and on. Future research should use items about the number
of episodes, the length of the episodes and number of physiotherapists by whom they
were treated during these episodes.
A strength is that our sample is representative: the mean age of 72 ± 9 years is in line
with the data based on the Dutch system of diagnosis-treatment combination (71 ±
10 years). The gender proportion (56% men) was also consistent with the national
number.32 According to the Dutch Parkinson guideline, there are no data available
regarding the severity of the disease.
Conclusions
This study shows that recognition of the added value of an expert physiotherapist was
found to be the strongest predictor for the search for CPI. The definition of expertise
expressed by PwP was in line with patientcenteredness, though in a rather narrow
manner, as only certain characteristics of PD expertise were recognised. In order for
PwP choosing high-quality care, improvements are needed. There is a lack of
recognition of expertise caused by a mismatch in the current available CPI. As PwP
showed relatively few signs of selectively choosing expert physiotherapists, CPI should
include additional and crucial quality of care information that matters for PwP. After
this, it is expected that expert physiotherapists become more easier recognisable for
those who want to make a selective provider choice. Furthermore, PwP heavily rely on
their referring providers, meaning that referring providers have a responsibility to act
as a coach for their patients. This fact should be used advantageously by involving the
professionals in a more active way.
60
Recognition of physiotherapists’ expertise in Parkinson’s disease
REFERENCES
1.
2.
3.
4.
5.
6.
7.
8.
9.
10.
11.
12.
13.
14.
15.
16.
17.
18.
19.
20.
21.
22.
23.
24.
Fung CH, Lim YW, Mattke S, Damberg C, Shekelle PG. Systematic review: the evidence that
publishing patient care performance data improves quality of care. Ann Intern Med 2008;148(2):
111–23.
Grabowski DC, Town RJ. Does information matter? Competition, quality, and the impact of nursing
home report cards. Health Serv Res 2011;46(6pt1):1698–719.
Werner RM, Norton EC, Konetzka RT, Polsky D. Do consumers respond to publicly reported quality
information? Evidence from nursing homes. J Health Econ 2012;31(1):50–61.
Lim JN, Edlin R. Preferences of older patients and choice of treatment location in the UK: a binary
choice experiment. Health Policy 2009;91(3):252–57.
Marang-van de Mheen PJ, Dijs-Elsinga J, Otten W, Versluijs M, Smeets HJ, Vree R, et al. The
relative importance of quality of care information when choosing a hospital for surgical treatment:
a hospital choice experiment. Med Decis Making 2011;31(6):816–27.
Sivey P. The effect of waiting time and distance on hospital choice for english cataract patients.
Health Econ 2012;21(4):444–56.
Fanjiang G, von Glahn T, Chang H, Rogers WH, Safran DG. Providing patients web-based data to
inform physician choice: if you build it, will they come? J Gen Intern Med 2007;22(10):1463–66.
Andlin-Sobocki P, Jonsson B, Wittchen HU, Olesen J. Cost of disorders of the brain in Europe. Eur J
Neurol 2005;12(1):1–27.
Nijkrake MJ, Keus SH, Oostendorp RA, Overeem S, Mulleners W, Bloem BR, et al. Allied health care
in Parkinson’s disease: referral, consultation, and professional expertise. Mov Disord 2009;24(2):
282–86.
Keus SH, Bloem BR, Verbaan D, de Jonge PA, Hofman M, van Hilten BJ, et al. Physiotherapy in
Parkinson’s disease: utilisation and patient satisfaction. J Neurol 2004;251(6):680–87.
Chartered Society of Physiotherapy: Neurology.Parkinson’s disease, multiple sclerosis and severe
traumatic brain injury. Physiotherapy Effectiveness Bulletin 2001;3:1–11.
Victoor A, Delnoij DM, Friele RD, Rademakers JJ. Determinants of patient choice of healthcare
providers: a scoping review. BMC Health Services Res 2012;12:272.
Nijkrake MJ, Keus SH, Overeem S, Oostendorp RA, Vlieland TP, Mulleners W, et al. The
ParkinsonNet concept: development, implementation and initial experience. Mov Disord Soc 2010;
25(7):823–29.
Keus SH, Oude Nijhuis LB, Nijkrake MJ, Bloem BR, Munneke M. Improving community healthcare for
patients with Parkinson’s disease: the dutch model. Parkinsons Dis 2012; 2012:543426.
Keus SH, Bloem BR, Hendriks EJ, Bredero-Cohen AB, Munneke M. Evidence-based analysis of
physical therapy in Parkinson’s disease with recommendations for practice and research.
Movement Disord: Official J Movement Disord Soc 2007;22(4):451–60.
Nijkrake MJ, Keus SH, Kalf JG, Sturkenboom IH, Munneke M, Kappelle AC, et al. Allied health care
interventions and complementary therapies in Parkinson’s disease. Parkinsonism Relat Disord 2007;
13(3):S488–S494.
Bloem BR, van Laar T, Keus SHJ, de Beer H, Poot E, Buskens E, et al. Multidisciplinary guideline in
Parkinson disease [in dutch: multidisciplinaire richtlijn ziekte van Parkinson]. Alphen a/d Rijn: Van
Zuiden Communication, 2010.
Munneke M, Nijkrake MJ, Keus SH, Kwakkel G, Berendse HW, Roos RA, et al. Efficacy of
community-based physiotherapy networks for patients with Parkinson’s disease: a clusterrandomised trial. Lancet Neurol 2010;9(1):46–54.
van der Eijk M, Faber MJ, Al Shamma S, Munneke M, Bloem BR. Moving towards patient-centered
healthcare for patients with Parkinson’s disease. Parkinsonism Relat Disord 2011;17(5):360–64.
Institute of Medicine. Crossing the quality chasm: a new health system for the 21st century.
Wasinghton: National Academy Press, 2001.
Hoehn MM, Yahr MD. Parkinsonism: onset, progression and mortality. Neurology 1967;17(5):427–42.
Dixon A, Robertson R, Appleby J, Burge P, Devlin N, Magee H. How patient choose and how
providers respond. London: The King’s Fund, 2010.
Coulter A. Engaging patients in healthcare. London: Open University Press, 2011.
Losina E, Plerhoples T, Fossel AH, Mahomed NN, Barrett J, Creel AH, et al. Offering patients the
opportunity to choose their hospital for total knee replacement: impact on satisfaction with the
surgery. Arthritis Rheum 2005; 53(5):646–52.
61
4
Chapter 4
25.
26.
27.
28.
29.
30.
31.
32.
62
Harris KM, Buntin MB. Choosing a health care provider: the role of quality information. Princeton:
Robert Wood Johnson Foundation, 2008.
Kaiser Family Foundation. Update on consumers’ view of patient safety and quality information.
California: Agency for Healthcare Research and Quality, 2008.
Hibbard JH, Greene J, Daniel D. What is quality anyway? Performance reports that clearly
communicate to consumers the meaning of quality of care. Med Care Res Rev 2010; 67(3):275–293.
van der Marck MA, Bloem BR, Borm GF, Overeem S, Munneke M, Guttman M. Effectiveness of
multidisciplinary care for Parkinson’s disease: a randomized, controlled trial. Movement Disord:
Official J Movement Disord Soc 2013; 28(5):605–11.
Sinaiko AD, Eastman D, Rosenthal MB. How report cards on physicians, physician groups, and
hospitals can have greater impact on consumer choices. Health Aff (Millwood) 2012; 31(3):602–611.
Beersen N, Berg M, van Galen M, Huijsmans K, Hoeksema N. Study to the value of ParkinsonNet [in
dutch: onderzoek naar de meerwaarde van ParkinsonNet]. Zeist: Vektis and Plexus, 2011.
Calsbeek H, van Dulmen SA, Nijhuis-van der Sanden MWG, Braspenning JCC. Kwaliefy indicators:
results measures 2011 [in Dutch: kwaliefyindicatorenset: resultaten meting 2011]. FysioPraxis
2012. In Press:3.
Berg van D. Parksinon disease and parkisonism in The Netherlands. [in dutch: de ziekte van
parkinson en parkinsonisme in nederland: een schatting van de prevalentie en incidentie op basis
van data uit het DBC informatie systeem]. Amsterdam: Parkinson Vereniging, 2010:1–25.
Chapter
5
Public release of performance data in changing the
behaviour of healthcare consumers, professionals or
organizations (Review)
Nicole Ketelaar
Marjan Faber
Signe Flottorp
Liv Rygh
Katherine Deane
Martin Eccles
Cochrane Database Systematic Review 2011; 11.
Chapter 5
ABSTRACT
Background: It is becoming increasingly common to release information about the
performance of hospitals, health professionals or providers, and healthcare
organisations into the public domain. However, we do not know how this information is
used and to what extent such reporting leads to quality improvement by changing the
behaviour of healthcare consumers, providers and purchasers, or to what extent the
performance of professionals and providers can be affected.
Objectives: To determine the effectiveness of the public release of performance data
in changing the behaviour of healthcare consumers, professionals and organisations.
Search strategy: We searched the Cochrane Central Register of Controlled Trials
(CENTRAL), Cochrane Effective Practice and Organisation of Care (EPOC) Trials
Register, MEDLINE Ovid (from 1966), EMBASE Ovid (from 1979), CINAHL, PsycINFO Ovid
(from 1806) and DARE up to 2011.
Selection criteria: We searched for randomised or quasi-randomised trials, interrupted
time series and controlled before-after studies of the effects of publicly releasing data
regarding any aspect of the performance of healthcare organisations or individuals. The
papers had to report at least one main outcome related to selecting or changing care.
Other outcome measures were awareness, attitude, views and knowledge of
performance data and costs.
Data collection and analysis: Two review authors independently screened studies for
eligibility and extracted data. For each study, we extracted data about the target
groups (healthcare consumers, healthcare providers and healthcare purchasers),
performance data, main outcomes (choice of healthcare provider and improvement by
means of changes in care) and other outcomes (awareness, attitude, views, knowledge
of performance data and costs).
Main results: We included four studies containing more than 35,000 consumers, and
1560 hospitals. Three studies were conducted in the USA and examined consumer
behaviour after the public release of performance data. Two studies found no effect of
Consumer Assessment of Healthcare Providers and Systems information on health plan
choice in a Medicaid population. One interrupted time series study found a small
positive effect of the publishing of data on patient volumes for coronary bypass surgery
and low-complication outliers for lumbar discectomy, but these effects did not persist
longer than two months after each public release. No effects on patient volumes for
acute myocardial infarction were found.
One cluster-randomised controlled trial, conducted in Canada, studied improvement
changes in care after the public release of performance data for patients with acute
myocardial infarction and congestive heart failure. No effects for the composite
process-of-care indicators for either condition were found, but there were some
improvements in the individual process-of-care indicators. There was an effect on the
mortality rates for acute myocardial infarction. More quality improvement activities
64
Public release of performance data in changing behaviour
were initiated in response to the publicly-released report cards. No secondary
outcomes were reported.
Authors’ conclusions: The small body of evidence available provides no consistent
evidence that the public release of performance data changes consumer behaviour or
improves care. Evidence that the public release of performance data may have an
impact on the behaviour of healthcare professionals or organisations is lacking.
5
65
Chapter 5
BACKGROUND
It is becoming increasingly common to release information about the performance of
healthcare systems into the public domain. In the present era of accountability, costeffectiveness, quality improvement and demand-driven healthcare systems, policy and
decision-makers such as governments, regulators, purchaser and provider organisations,
health professionals and consumers of health care are becoming more interested in
measuring performance.1 The measurements may appear in consumer reports, provider
profiles or report cards. It is not always clear who the information users are or what
the release of data is expected to achieve. However, it is often assumed that the
information will affect and facilitate the decisions and behaviours of various
stakeholders and ultimately result in health system improvements.1-3
The stakeholders in this review include healthcare consumers, professionals, providers
and purchaser organisations. Accountability relationships connect all the stakeholders.
These relationships have two prominent elements, namely the ‘provision of
information’ about performance and the ‘sanctions or rewards for the accountable
party’.1 The main role of performance measurement is to keep the various agents
accountable by enabling stakeholders to make informed decisions.1 Various suggested
uses of performance measurements are linked to the accountability perspective.
Examples of underlying objectives are (1) the use of performance measurements to
promote more efficient and demand-driven healthcare4,5 and (2) applying the results as
a marketing tool.6 One user goal is to use the public disclosure of performance
measurements to encourage providers to focus on quality problems and to stimulate
performance improvement.5,7,8 From a healthcare consumer’s perspective, the data can
encourage patients to preferentially choose high-quality health care, i.e. the best
health plan or provider9-11 or to assess the performance of individual professionals.12,13
Other proposed user goals for performance measurements have been linked to
controlling costs,14,15 regulating the healthcare system16,17 and influencing the decisions
of healthcare purchasers.18-20
Consumers must overcome barriers to the use of performance data. Examples of such
barriers are the complexity of the performance data,21 lack of skills to comprehend and
use performance data22-25 and the way data are presented.8,26-28 A negative
consequence of such barriers might be related to the impact of choice on equity in
healthcare. Consumers from poorer backgrounds and with lower educational levels will
be less likely to be given a choice, less able to choose and less able to afford travel to a
better performing, but more distant, provider.29
Professionals focus on the barriers to accessibility,30 the validity of the performance
measures themselves,31,32 and the validity of implicit or explicit comparisons of
66
Public release of performance data in changing behaviour
performance.33,34 There are concerns that failure to adequately adjust for the case mix
in the data sets may lead to hospitals or clinicians who treat higher-risk patients being
labelled as poor performers, or to providers preferentially selecting lower-risk
patients.35-37 In healthcare systems where providers charge for their services, the
‘better’ performing providers may charge more,20 thereby restricting access to better
care. Publicly releasing performance data may have other unintended consequences as
well. There is a risk that the release may lead to improved reporting without
necessarily improving performance. It has been said that the care tasks that are easiest
to measure are often those least important in a quality improvement context and that
other task measurements will be neglected.38
Thus, the impact of public release of performance data may have various mechanisms.
Most commentators seem to consider the most important goal of publishing
performance data to be to cause providers to improve their performance. This goal can
be achieved in a selection pathway or a change pathway.3 Consumers, patients and
purchaser organisations that are in a position to do so can select the best healthcare
professionals and organisations. This type of selection will not change the quality of the
delivered care by itself, but it can be a stimulus for quality improvement. In a change
pathway, healthcare professionals and organisations can improve performance by
changing their work procedures or professional culture, and organisations can make
structural changes.
Description of the intervention
Public release of performance data is the release of information about the quality of
care so that patients and consumers can better decide what health care they wish to
select and healthcare professionals and organisations can better decide what to
provide, to improve or to purchase. This mechanism excludes the use of auditing and
feedback as a tool for improving professional practice and healthcare outcomes. This
subject has been reviewed elsewhere.39
Why it is important to do this review
Some systematic reviews2,8,40,41 have suggested positive effects of publicly releasing
performance data. However, none of them focuses on identifying and synthesising only
the most robust evidence available; this systematic review will do so.
Objectives
To estimate the effects of publicly releasing performance data on changing the
behaviour of three target groups: healthcare consumers (patients), providers of
healthcare (health professionals) and purchasers of healthcare.
67
5
Chapter 5
METHODS
Criteria for considering studies for this review
Types of studies
• Randomised controlled trials (RCT), including cluster-randomised controlled trials
(ClRCTs)
• Quasi randomised trials (QRT), including cluster quasi randomised trials (ClQ-RCTs)
using methods of allocation such as alternation or allocation by case note number.
• Interrupted time series (ITS) studies with at least three data points before and three
data points after the intervention.
• Controlled before-after (CBA) studies, with at least two intervention sites and two
control sites that are chosen for similarity of main outcome measures at baseline.
Types of participants
Patients or other healthcare consumers and healthcare providers, including
organisations (e.g. hospitals, practices and individual healthcare professionals) without
any restriction by type of healthcare professional, provider, setting or purchaser.
Types of interventions
We included interventions that contained the following elements.
• Performance data about any aspect of the healthcare organisations or individuals,
including process measures (e.g. waiting times), healthcare outcomes (e.g.
mortality), structure measures (e.g. presence of waiting rooms), consumer or patient
experiences (e.g. Consumer Assessment of Healthcare Providers and System (CAHPS)
data) and/or expert or peer-assessed measures (e.g. certification, accreditation and
quality ratings given by colleagues).42 The data presented may or may not provide
comparisons with similar providers or quality standards and may or may not be
adjusted for case mix. Performance data may be prepared and released by any
organisation, such as the government, insurers or consumer organisations.
• The release of performance data into the public domain in written or electronic
form, with varying degrees of accessibility, such as a report in a publicly accessible
library or more active dissemination directly to consumers in newspapers, leaflets,
personal mailings, broadcasting media, etc.
The data may be presented numerically, graphically or pictorially.
Comparators
The following comparisons were planned.
1. Public release of performance data compared to control (the control intervention
should consist of the usual practice in that setting, which may include other
interventions aimed at quality improvement, such as the internal use of the same
performance data).
68
Public release of performance data in changing behaviour
2. Different types of public release of performance data compared to each other.
We excluded studies that did not expose participants to performance data concerning
process measures, healthcare outcomes, structure measure, consumer/patient
experiences or expert or peer assessed measures. We also excluded studies that
reported only hypothetical choices.
Types of outcome measures
Main outcome measures
We planned to the primary outcome measures according to two important aims of those
publicly releasing performance data.
1. Improvement by selection
• Changes in the healthcare utilisation decisions of consumers (public and patients)
• Changes in the healthcare utilisation decisions of healthcare providers (professionals
and organisations)
• Changes in the healthcare utilisation decisions of purchasers
2. Improvement by changes in care
• Objective measures of provider performance, including those that were made public
and others that were not
• Valid measures of staff morale or behaviour (‘valid’ defined as having the
development of the assessment tool reported in a peer-reviewed journal)
Other outcome measures
If a study reported at least one main outcome measure we also collected those
concerning awareness, attitude, views, knowledge of performance data in all target
groups and cost data. Where possible, we planned to collect data about the extent to
which outcome measures varied with participant characteristics. We excluded studies
that reported awareness, knowledge, attitude or costs in the absence of objective
measures of provider performance or decision behaviour of healthcare consumers,
providers or purchasers. Where possible, we planned to collect data about the extent
to which outcome measures varied with participant characteristics.
Search methods for identification of studies
We searched the Cochrane Central Register of Controlled Trials (CENTRAL), MEDLINE
Ovid (from 1966), EMBASE Ovid (from1979), CINAHL, PsycINFO Ovid (from1806) and
DARE up to 2011. For MEDLINE, we used subject headings and the relevant quality of
healthcare MeSH terms, such as ‘process assessment’, ‘outcome assessment’, ‘quality
indicators’, ‘quality assurance’ and ‘benchmarking’. We also used text words and
phrases such as ‘performance outcome‘, ‘report card’, ‘criteria’, ‘standard’,
69
5
Chapter 5
‘disclosure’, ‘quality information’ and ‘public information’. We combined these terms
with forms of decision-making such as ‘choice behaviour’, ‘patient preferences’,
‘patient acceptance’ and ‘consumer satisfaction’. We searched the other databases
using the appropriate controlled vocabulary. In addition, we identified potentially
relevant studies in the reference lists of key articles. Appendix 1 to Appendix 5 give full
details of the search terms.
Data collection and analysis
Selection of studies
We downloaded all titles and abstracts (N=6839) retrieved in the electronic search to a
reference management database. We removed the duplicates, then two review authors
independently examined the remaining references. All review authors recorded their
assessments of abstracts with points: ‘0’ for exclusion, ‘1’ for doubtful and ‘2’ for
inclusion. Two review authors independently rated each abstract, therefore a minimum
score of ‘0’ and a maximum score of ‘4’ was possible. Abstracts with a combined score
of 0 or 1 were excluded. Studies with a combined score of 3 or 4 were included.
Two review authors resolved the fate of studies with a combined score of 2 by
discussion. A third review author (ME) decided any disagreements that remained
unresolved. We documented the reasons for exclusion. We obtained full-text copies of
papers taken from references for inclusion. Two authors of our review independently
assessed the eligibility of these papers.
Data extraction and management
After the first selection round, relevant studies were retrieved for full-text reading
(Figure 1). We distributed these studies to our authors in such way that they did not
receive studies for text reading that they already evaluated in the first round. We
extracted the data about the study design, patient and provider characteristics,
interventions, outcome measures, and healthcare choices to a form specially designed
for our review (Appendix 6). We used another form (Appendix 7) for the studies we
retrieved for a more detailed evaluation. The two review authors resolved
disagreements by discussion if possible. A third review author (ME or MF) dealt with
disagreements that the two review authors could not resolve.
Assessment of risk of bias in the included studies
We assessed the risk of bias on the basis of the Cochrane Collaboration criteria43: (i)
adequate sequence generation, (ii) concealment of allocation, (iii) blinding, (iv)
incomplete outcome data, (v) selective reporting and (vi) no risk of bias from other
sources. We used three additional criteria that the Cochrane Effective Practice and
Organisation of Care (EPOC) Group specifies44: (vii) baseline characteristic similarity,
70
Public release of performance data in changing behaviour
(viii) reliable primary outcome measures and (ix) adequate protection against
contamination. We used these nine standard criteria for ClRCTs, ClQ-RCTs and CBA
studies. We used seven criteria for ITS studies: (i) the intervention is independent of
other changes, (ii) the shape of the intervention effect is pre-specified, (iii) the
intervention is unlikely to affect data collection, (iv) knowledge of the allocated
interventions is adequately prevented during the study, (v) the outcome data are
incomplete, (vi) reporting is not selective and (vii) there is no risk of bias from other
sources. Two review authors independently examined the risk of bias assessment and
resolved disagreements by discussion. There were some disagreements about the rating
of the criterion as ‘yes’ instead of ‘unclear’ or vice versa. Sometimes the rating was
based on a different phrase in the text. A third review author (MPE or MF) dealt with
any disagreements that the two review authors could not resolve.
Unit of analysis issues
We noted whether studies randomised patients or healthcare providers. If analysis did
not allow for clustering of patients within healthcare providers, we recorded a unit of
analysis error, because such analyses tend to overestimate the precision of the
treatment effect.
Data synthesis
We report the effect sizes for each outcome for each study. Quantitative synthesis was
not possible.
RESULTS
Description of studies
See: Characteristics of included studies; Characteristics of excluded studies.
See table 1 ‘Characteristics of included studies’ and table 3 ‘Characteristics of
excluded studies’.
Results of the search
The searches found 6839 references; we excluded 6786 references because the titles
and abstracts did not meet our inclusion criteria. We retrieved the full texts of
publications listed in 53 references. Four citations were irretrievable and could not be
considered for inclusion in the review. An additional search brought two more
references to light. Altogether, we retrieved full text versions of 51 papers. Forty-two
of these papers did not fulfill the inclusion criteria.
We evaluated nine papers in more detail, and subsequently we excluded another five
papers. We reported the reasons for exclusion of these 51 studies (including the four
irretrievable citations) in the ‘Characteristics of excluded studies’ table. Four papers
met the inclusion criteria of the review. Figure 1 presents the study flow chart.45
71
5
Chapter 5
Figure 1.
Flowchart for Study Selection
References identified by database
searching – records screened
N = 6839
Potentially relevant studies identified by
screening of abstracts
N = 53
Additional searches in key articles, other
potentially relevant reviews
N=2
(49 + 2 = 51)
References excluded
N = 6786
References not available
N=4
(53 – 4 = 49)
Studies excluded after
full-text reading
N = 42
Studies identified for full-text reading
N = 51
Studies retrieved for more detailed
evaluation
Studies excluded based on
design
N=9
N=5
Studies included
N=4
Included studies
Characteristics of setting and patients/consumers
See tables 1 and 2.
72
Table 1. Characteristics of included studies
Farley 2002a
Methods
Participants
Interventions
Outcomes
Notes
Design: ClRCT
Unit of allocation: New cases (household units) were randomly assigned to either an
experimental or control group. This random assignment was independent of case size,
county of residence and initial plan assignment. The Iowa Medicaid office supplied the
authors with data files for the full sample of new beneficiaries
Unit of analysis: Medicaid beneficiaries. All analyses were corrected for clustering of
beneficiaries within cases using Huber/White corrections
Sample size calculation: Not done. Statistical significance was assessed at the 0.05 level
The number randomised into the trial: 13,077 new beneficiaries in 7016 cases; 6515
beneficiaries in the control group, 6562 beneficiaries in the intervention group
Characteristics of participating beneficiaries: 72% of ‘cases’ (family or household units) had 2 or
more members with an average of 2.8 beneficiaries per case
Age of beneficiaries: Unclear
Gender of beneficiaries: Unclear Ethnicity Unclear
Characteristics of participating health plans: Two Health Maintenance Organisations
(HMOs) under contract with the Medicaid programme and 1 primary care case
management plan (MediPass). One was the lower-scoring HMO and the other was the
higher-scoring HMO
Setting: Iowa, USA. Within the state, counties were divided into 3 groups: those that
only contained Medicaid plans and other low-rated plans; those that only contained
Medicaid plans and other high-rated plans; and those that contained Medicaid plans and
both lower and higher-rated plans
Intervention group: Standard enrolment materials and Consumer Assessment of Healthcare
Providers and Systems (CAHPS) report delivered by personal mailing to beneficiaries
Duration of intervention: February 2000 to May 2000
Intervention deliverer: The Iowa Medicaid office mailed beneficiaries a packet health plan
enrolment materials that include items such as a plan enrolment form
Control Group: standard enrolment materials delivered by personal mailing to beneficiaries
Source of funding for study: Co-operative agreement 5U18HS09204-05; the Agency for
Healthcare Research and Quality and the Center for Medicare and Medicare Services
Main outcome: For each of the other 2 counties the authors performed a multinomial logistic
regression in which for each beneficiary the outcome took on the values 1 = stay in assigned
HMO; 2 = switch to other HMO; 3 = switch to MediPass
The star charts in the CAHPS report were based on a HMO’s performance. The bar charts
included 3 charts with ratings of the health plan, healthcare and personal doctor. Five
charts were included of service by the providers or health plan
5
Risk of bias
Bias
Random sequence generation
(selection bias)
Authors’ judgement Support for judgement
Unclear risk
The new cases enrolled during the study period were randomly assigned to an
experimental or control group. This random assignment was independent of case size,
county of residence and initial plan assignment (p. 326)
Allocation concealment (selection Unclear risk
The new cases enrolled during the study period were randomly assigned to an
experimental or control group (p. 326)
bias)
Low risk
Incomplete outcome data
“Medicaid office supplied us with data files for the full sample of new beneficiaries (p. 328)”
(attrition bias) All outcomes
The full sample consisted of 13,077 new beneficiaries in 7016 cases. Results from all are
presented in Table 1 (p. 330)
Selective reporting (reporting bias) Low risk
Only one outcome which was measured in 100% of all cases/new beneficiaries (p. 330)
Other bias
Low risk
Only one outcome which was measured in 100% of all cases/new beneficiaries (p. 330)
Adequate blinding of participants, Unclear risk
“The Iowa Medicaid office supplied us with data files for the full sample of new beneficiaries.
personnel and outcome assessors?
The data identified the initially assigned (default) plan, the final plan (different from the
default plan only if an active choice was made), whether CAHPS reports were mailed, the
county of residence, and the number of beneficiaries in each case (case size)” (p. 328)
Protection against contamination
Unclear risk
The authors claim that “the mail-based administration of the process, including the
distribution of CAHPS reports, effectively isolated beneficiaries in the control group (those
not receiving CAHPS reports) from exposure to CAHPS information, be it from other
beneficiaries outside their household, benefits counsellors, or media information” (p. 327).
This statement has not been verified
Baseline characteristics similar?
Unclear risk
“Demographic variables could not be measured directly (p. 329) and as such, were not
reported for the intervention and control groups”
Reliable outcome measurements
Low risk
The primary outcome measure, i.e. plan choice, was extracted from an automated system
(P328). No reliability measure for the procedure was reported
Farley 2002b
Methods
Design: ClQ-RCT
Unit of allocation: New “cases” (family or household units) were assigned to either an
experimental or control group, based on whether the last digit of the case was odd or even.
The New Jersey Medicaid office supplied the authors with data files for the full sample of all
new 5217 enrollees
Unit of analysis: New Medicaid cases; cases are the family units that qualify for Medicaid
coverage. Medicaid-eligible family units that include an adult are referred to as adult cases,
and those in which only children are Medicaid-eligible are child cases
Sample size calculation: No justification for the size of the overall sample
Participants
Interventions
Outcomes
Notes
Risk of bias
Bias
Random sequence generation
(selection bias)
Allocation concealment (selection
bias)
The number randomised into the trial: 5217 cases. Control: 2,568 cases. Intervention: 2,649
cases
Characteristics of participating cases: Cases are the family units that qualify for Medicaid
coverage. Medicaid-eligible family units that include an adult are referred to as adult cases,
and those in which only children are Medicaid-eligible are child cases. New Jersey requires all
members of each Medicaid case to enrol in the same HMO
Age: Unclear
Gender: Men and women
Ethnicity: Whole population
Self reported health status: All health statuses
Characteristics of participating health plans: Medicaid health plans in the state of New
Jersey. The Medicaid program has a form of mandatory (auto-assignment) or voluntary
managed care programme, which includes one or more HMOs or (sometimes) a primary care
case management plan
Setting of care: Medicaid health plans choice between one or more health maintenance
organizations (HMOs) and sometimes a primary care case management plan
Country: USA, state of New Jersey
Intervention group: Standard enrolment materials and Consumer Assessment of Health Plans
Study (CAHPS) report delivered by personal mailing
Duration of intervention: A 3 week period from 25 March to 15 April 1998
Intervention deliverer: The New Jersey Medicaid office published a 7-page brochure
“choosing an HMO” that compared the Medicaid HMO with respect to the consumer ratings
and experiences reported in the CAHPS survey
Control group: Standard enrolment materials delivered by personal mailing
Source of funding: Co-operative agreements No. 5U18HS09204-05;Consumer Assessment
ofHealth Plans Study (CAHPS) from the Agency for Healthcare Research and Quality (AHRQ)
Main outcome: Auto assignment rates Plan choices
The star chart in CAHPS report were based on a HMO’s performance compared to the
average in every county of residence. The counts ranged from 20 to 29 stars. The resulting
standardized CAHPS ratings ranged from -8.40 (well below the average) to 6.26 (well above
the county average)
Authors’ judgement Support for judgement
High risk
Quote: "Based on whether the last digit of the case ID was odd or even, half the cases were
randomly assigned to an experimental group and half were assigned to a control group”
(p. 989)
High risk
Allocation concealment was based on case ID number, therefore research investigators
enrolling participants could possibly foresee assignment
5
Incomplete outcome data (attrition
bias)
Selective reporting (reporting bias)
Other bias
Adequate blinding of participants,
personnel and outcome assessors?
Low risk
Low risk
Low risk
Unclear risk
Protection against contamination
Low risk
Baseline characteristics similar?
Reliable outcome measurements
Unclear risk
Low risk
Romano 2004
Methods
Participants
Quote: “The analysis of the overall effects of CAHPS included the entire April 1998 sample of
enrollees, and is therefore not subject to non-response bias” (p.1002)
Results from all hypotheses listed in methods section are reported
No details were provided about blinding:
Quote: “The New Jersey Medicaid office supplied us with a data file that identified plan
choices, auto-assignment and demographics for the full sample” (p. 990)
No information was reported. Quote: “about half the cases mailed a CAHPS report said they
received and read the report” (p.996)
Demographic characteristics were not reported
Quote: “The primary outcome measure, i.e. plan choice, was extracted from an automated
system”(p. 994). No reliability measures for the procedure were reported
Design: Interrupted time series (ITS)
Data analysis: The data were analysed using 2 different approaches:
1) Ordinary least squares (OLS). This method does not take into account the interdependency of subsequent measurements. It is, however, adjusted for several independent
variables: state-wise hospital volume for same condition or procedure, hospital, monthly
volume before publication of report card for each hospital, interaction between state
volume and hospital, unrelated volume in each hospital.
The method is used as follows: based on model predict volume for hospital for each of the 12
months after publication of report card;
model volume based on data preceding publication of report card; aggregate predicted
volumes from all hospitals in same performance category; estimate 95% confidence intervals
(CI) for predictions; if actual volume falls outside CI, then significant effect of report;
2) Autoregressive (ARIMA) methods with P = 1, q = 0 and the same main effects. As a result
of software limitations, the authors were unable to include 2-way interactions. Second-order
autoregressive models were also tested but generated very similar results. Because the
residual autocorrelations were not significant (P > 0.10) for all but 1 model, the authors did
not estimate moving average models. The autoregressive and OLS results differed, especially
for AMI volume, so both sets of numbers were reported to demonstrate the sensitivity of our
results to different statistical methods. Autoregressive models adjust for observed
correlations, in some analyses, between current and previous volume changes; however only
the OLS models adjust for hospital-level interactions
Patients admitted to hospitals designated as outliers in reports in New York and California
Characteristics of participating patients: The total number of patients with a topic condition
or procedure, or related condition or procedure, who were admitted to a specific hospital in
a specific calendar month
California - clinical problems:
Acute myocardial infarction (AMI)
Coronary artery bypass grafting (CABG) (AMI-related)
Percutaneous coronary angioplasty (AMI-related)
Congestive heart failure (AMI-related)
Cervical discectomy (target)
Lumbar discectomy (target)
Back or neck procedures (discectomy-related)
Medical back problems (discectomy-related)
Knee arthroplasty (discectomy-related)
Hip arthroplasty (discectomy-related)
New York - clinical problems:
AMI
CABG
Percutaneous coronary angioplasty (AMI-related)
Congestive heart failure (AMI-related)
Age: The authors excluded children less than 18 years of age, different groups: < 55 years,
55 to 64 years, 65 to 74 years, > 74 years
Gender: No restriction
Ethnicity: Black, Hispanic, White
Other characteristics: Patients admitted for psychiatric conditions, injury or poisoning, or
rehabilitation were excluded. Also patients transferred from other acute care hospitals were
excluded, because transfers generally reflect the capabilities of different facilities, or
insurance arrangement, rather than consumer’s choice
Characteristics of participating hospitals: Acute care, non-federal hospitals. All analyses
were limited to hospitals that were included in the report card
Setting of care: Hospital/inpatient
Country: USA, in states of California and New York
Intervention: Annual reports on risk-adjusted outcomes; focused on specific conditions or
procedures; the reports incorporate clinical expertise and address regional concerns
Intervention duration: Using hospital months. California: study period 24 months before
publication of first report in 1993 and 12 months after. In 1996, 24 months before second
report and 7 months after
New York: hospital ratings released every 12 to 24 months since December 1990
Intervention deliverer: Report cards were published by agencies in California and New York
Source of funding: US Agency for Healthcare Research and Quality
Interventions
5
Outcomes
Notes
Main outcome: Change in the utilisation decisions of consumer, healthcare professional or
purchasers
Study period:
California: First report released in December 1993 classified hospital mortality for AMI and
complication rated for cervical and lumbar discectomy as either ‘better’ or ‘not better’ than
expected. The second report, released in May 1996 rated as either ’better’, ‘worse’, or
‘neither better nor worse’ than expected.
New York: Hospital-specific, risk-adjusted mortality rates and 3-category ratings have been
released every 12 to 24 months since December 1990
Risk of bias
Bias
Authors’ judgement Support for judgement
Incomplete outcome data (attrition Low risk
Aggregate data from administrative databases. California analysis was based on the
bias)
California Patient Discharge Data Set. The New York analysis was based on the Statewide
Planning and Research Co-operative System (SPARCS). The number of drop-outs was not
applicable here, since the databases and several independent variables were used to predict
each hospitals' patient volume after publication of a report card.
Selective reporting (reporting bias) Low risk
All outcomes and results outlined in the Methods section are reported in tables and/or text.
Results for all primary and secondary objectives are reported.
Other bias
High risk
Main analysis based on the assumption of same trend before and after intervention.
Difference from predicted values was reported, rather than change in trend and level.
Shape of intervention effect preLow risk
Quote: “We predicted what each outlier hospital’s volume should have been in each of the
12 months after publication of a report card. These predicted volumes were aggregated for
specified?
all hospitals assigned to the same risk-adjusted performance category (e.g. higher-thanexpected AMI mortality) in that report card”. (p.371)
Intervention is independent of
Unclear risk
It is not sure that the intervention occurred independently of other changes over time or
other changes?
that the outcome was not influence by other confounding variables events during study
period
Intervention unlikely to affect /
Low risk
Sources and methods of data collection were the same before and after the intervention
bias data collection?
Knowledge of the interventions
Low risk
Data collection based on administrative database and performed retrospectively
adequately prevented during the
study?
Reliable outcome measurements
Unclear risk
The outcome measures were based on a conceptual framework and hypotheses. Hypotheses
were based on a validated assumption (p.368). No clear assessment about the reliability of
outcomes measures were reported in the Method section.
Tu 2009
Methods
Design: Cluster-randomised trial
Unit of allocation: Hospitals were randomised to receive either early or delayed feedback of
a publicly released report card (p. 2331). The randomisation was stratified by type of
hospitals (p. 2332). Types of hospitals were classified as teaching hospitals, large community
hospitals, small hospitals (p. 2332).
Unit of analysis: Patients treated for acute myocardial infarction (AMI) and congestive heart
failure (CHF), taking hospital clustering into account in the analysis
Sample size calculation: The study had 84% power to detect 5% absolute difference on the
composite quality indicators. The power calculation assumed a baseline performance rate on
each composite indicator of 70% (standard deviation 10%) in each study group, and that there
would be a secular improvement of 75% (SD 7.5%) in the composite indicator, independent of
the study intervention (p. 2332).
86 hospital corporations were randomised:
44 hospitals in the early feedback report card release and 42 hospitals in the delayed
feedback report card release (Figure 1, p. 2331)
Characteristics of hospitals in the early feedback report card release: At baseline, 5676
patients were admitted with AMI and 5073 patients were admitted with CHF
Characteristics of hospitals in the delayed feedback report card release: At baseline, 5070
patients were admitted with AMI and 4220 patients were admitted with CHF
Setting: The study was conducted in Ontario, Canada. All 130 acute hospitals were assessed
for eligibility and 86 hospitals were included.
Baseline assessment: At each participating hospital a target sample of 125 charts (or all
patients if < 125 patients were treated) for patients receiving care for AMI and/or CHF
between 1 April 1999 and 31 March 2001 was abstracted. The baseline performance was
based on a set of 12 process of care indicators for AMI and 6 indicators for CHF.
Early feedback report card release: The hospitals received their baseline performance data
in October 2003 for internal validation and the results were publicly released at a press
conference and on the Web in January 2004 (p. 2332)
Duration of intervention: January 2004 to 1 April 1 2004 (inclusive the follow-up period:
January 2004 to 31 March 2005)
Delayed feedback report card release: The hospitals received their baseline performance
data in September 2005 for internal validation and the results were also publicly released on
the internet in September 2005 (p. 2332). No extensive media or associated press was
covered.
Duration of intervention: Not applicable, as the delayed feedback group received the
intervention after follow-up data were collected
Participants
Interventions
5
Outcomes
Notes
Risk of bias
Bias
Random sequence generation
(selection bias)
Allocation concealment (selection
bias)
Incomplete outcome data (attrition
bias)
Intervention deliverer: The Canadian Cardiovascular Outcomes Research Team, which is a
national team of cardiovascular outcomes researchers from across Canada. The team also
was involved, together with the Canadian Cardiovascular Society, in the development and
endorsement of the set of quality of care indicators, as used in this study.
Source of funding for study: The EFFECT study was supported by a Canadian Institutes of
Health Research team grant in cardiovascular outcomes research to the Canadian
Cardiovascular Outcomes research Team
Main outcome: There was no significant improvement in the composite AMI or CHF processof-care indicator. One out of 12 individual process of care AMI indicators improved
significant more in the early feedback group compared to the delayed feedback group. One
out of 6 of the individual process of CHF indicators improved significantly more in the early
feedback group. Regarding mortality rates, as an outcome indicator, 30-day mortality
significantly decreased in the early feedback group for AMI, while 3 other mortality-related
measures for AMI and CHF did not change.
The survey showed that the early feedback group reported significantly more often the start
of one or more quality improvement initiatives for AMI care and for CHF care
Authors’ judgement Support for judgement
Low risk
The hospitals were randomly assigned to the early feedback group or the delayed feedback
group
Low risk
Quote: “This random assignment was stratified by type of hospital and performed by a study
statistician” (p. 2332)
Unclear risk
One hospital withdrew from the baseline phase, after randomisation and 4 withdrew from
the follow-up phase, all due to resource constraints (p. 2331). No intention-to-treat analysis
was performed. Additional exclusions of patients were not reported, but cannot be verified.
Selective reporting (reporting bias) Low risk
Results from all indicators, individual and composite, are reported as well as the hospital
outcome indicators
Other bias
Low risk
Adequate blinding of participants, High risk
Quote: “It was not possible to blind the hospitals to their status” (p. 2332).
Quote: “We could not blind the delayed feedback group to the media coverage and
personnel and outcome assessors?
associated publicity surrounding the study results” (p. 2336).
Quote: “Patient charts were abstracted by an experienced research nurse” (p.2332), but it is
unclear whether or not she was blinded for allocation
Protection against contamination
High risk
Quote: “There was extensive media coverage following the release of the baseline
performance for the early feedback hospitals”(p. 2332). The authors mention that “one
unanticipated observation" was that several hospitals in the delayed feedback group
reported that they also initiated some quality improvement activities after becoming aware
of the publicly released early feedback report card” (p. 2336)
Baseline characteristics similar?
Table 1 (p. 2331) shows the baseline characteristics of the hospitals and patients. Quote:
“The hospitals were well balanced across the 2 groups in terms of clinical characteristics of
patients” (p. 2333)
Reliable outcome measurements
Quote: “The primary outcome measures were a set of national process-of-care quality
indicators for AMI and CHF care which were developed and endorsed by the Canadian
Cardiovascular Outcomes Research Team” (p. 2332). It is likely that the face validity of the
indicators is guaranteed, but no field test was performed. The last step in the validation
process was not undertaken and no verdict about the content validity is possible. Therefore,
we scores this item as high-risk
CIRT: Cluster-randomised controlled trial; ClQ-RCT: cluster quasi-randomised trial; ITS: interrupted time series; HMOs: health maintenance
organisations; CBAG: coronary artery bypass grafting; AMI: acute myocardial infarction; CHF: congestive heart failure; SD: standard deviation;
CI: confidence interval; CAHPS: Consumer Assessment of Healthcare Providers and System; ARIMA: autoregressive integrated moving average
5
Chapter 5
Table 2. Characteristics of settings and consumers (recorded as patients and Medicaid enrolees)
Study
Methods
46
Farley 2002a
Design: ClRCT
Unit of allocation: new cases (household
units)
Power calculation: not done
47
Design: ClQ-RCT
Farley 2002b
Unit of allocation: new cases (household
units)
Power calculation: unclear
Romano 200448 Design: ITS
Unit of allocation: not applicable
Power calculation: unclear
Setting of
Patients/consumers
care
Medicaid beneficiaries: 13077 Health plans:
Age: unclear
HMOs
Gender: unclear
Medicaid beneficiaries: 5878
Age: Unclear
Gender: men and women
Health plans:
HMOs
Patients: Given CABG in New Hospitals: nonYork, and treated for AMI or
federal
given post discectomy surgery hospitals
in California
Number of patients: unclear
Age: children younger than
18 years were excluded
Gender: men and women
49
Tu 2009
Design: ClRCT
Patients: 15997 patients
Hospitals:
Unit of allocation: hospitals
treated for AMI or CHF
teaching,
Power calculation: the study had 84%
Age: no restriction
community or
power to detect 5% absolute difference in Gender: men and women
small
the composite quality indicators. The
assumptions were a baseline performance
rate of 70% (SD 10%) for each composite
indicator in each study group, and a
secular improvement of 75% (SD 7.5%) in
the composite indicator, independent of
the study intervention.
CIRT: cluster-randomised controlled trial; ClQ-RCT: cluster quasi-randomised trial; ITS: interrupted time
series; HMOs: health maintenance organisations; CBAG: coronary artery bypass grafting; AMI: acute
myocardial infarction; CHF: congestive heart failure; SD: standard deviation
We included four studies46-49 comprising more than 35,000 consumers (recorded as
patients, and Medicaid enrollees), and 1560 hospitals. Three studies were conducted in
the USA and one study was conducted in Canada. Farley46 took place in Iowa and
Farley47 in New Jersey; both studies were set in health plans. Romano48 was set in
hospitals in California and New York. Tu49 was set in hospitals in Canada. Farley46
conducted their study in 35 of the 99 Iowa counties. These counties represented 60% of
the total Iowa Medicaid population. The study included MediPass and two types of
health maintenance organisations (HMOs) that differed in their performance as assessed
with CAHPS surveys scores: one high and one low-rated. The counties were subdivided
into three health plan options: type I (MediPass and two HMOs), type II (MediPass and
one HMO with a high rating) and type III (MediPass and one HMO with a low rating). The
CAHPS survey measures several dimensions of health plan performance including ratings
of health plans, primary doctors and reports of experiences with using a health plan.
The ratings are for individual items using response scales ranging from 0 to 10. The
82
Public release of performance data in changing behaviour
reports of experiences are composite scores that are averages of responses to sets of
individual items with four-category response options.
Farley47 was based on the New Jersey Medicaid programme. There was a mandatory
HMO enrolment period for Aid for Dependent Children and other welfare-related
beneficiaries in 17 of its 21 counties. In February 1998, 91% of these beneficiaries were
enrolled in Medicaid HMOs.
Romano48 was based on the California Hospital Outcomes Project (CHOP). In California
and the Cardiac Surgery Reporting System (CSRS) in New York. Trends in hospital
volumes for certain diagnoses after publication of report cards were evaluated. In
California the CHOP report published in 1993 evaluated acute myocardial infarction
(AMI) mortality at 394 hospitals, complications after lumbar discectomy at 344
hospitals, and complications after cervical discectomy at 277 hospitals. In New York,
the CSRS report evaluated 30 hospitals in December 1992 and 31 hospitals in December
1993 and June 1995.
In Canada, Tu 200949 evaluated the public release of performance data of 12 processof-care indicators for AMI and six indicators for congestive heart failure (CHF) in
86 hospitals. The hospitals were categorised by either early (2004) or delayed (2005)
feedback of a publicly released report card about their baseline performance. The
Canadian Cardiovascular Outcomes Research Team and the Canadian Cardiovascular
Society developed the indicators.
Excluded studies
In total, we excluded 47 studies after assessing full copies of the papers. The main
reasons for exclusion were: design (study was not a ClRCT, ClQ-RCT, CBA or ITS (34)),
interventions did not contain process measures, health care outcomes, structure
measures, consumer or patient experiences, expert- or peer-assessed measures (18), no
objective outcome data were recorded or available for one or both arms (seven),
and/or the study was about hypothetical choices (six). We excluded four studies
because we were unable to obtain the full-text articles (Table 3).
Table 3. Characteristics of excluded studies
Study
Alteras 2000 50
Beaulieu 2002 51
Beaulieu 2002 52
Bundorf 2009 53
Dawson 2007 54
Dranove 2008 36
Ettinger 2008 55
Reason for exclusion
Unable to retrieve
Studydesign
Studydesign
Studydesign
Studydesign
Studydesign
Studydesign/outcome measure
83
5
Chapter 5
Study
Fanjiang2007 56
Fine 1998 57
Fong 2008 58
Fotaki 2008 13
Fox 2001 59
Goldstein 2001 60
Goss 2006 61
Hannan 2003 62
Harris 2002 42
Harris-Kojetin 2007
26
63
Hibbard 1996
Hibbard 2000 64
Hibbard 2001 22
Hibbard 2002 65
Hibbard 2002 66
Hibbard 2003 67
Hibbard 2005 68
Hibbard 2005 69
Hollenbeak 2008
Jensen 2004 71
Jha 2006 72
Jian 2009 73
Knutson 1998 74
Krupat 2004
70
75
Lindenauer 2007 7
Mannion 2003 76
McCormack 2001 77
McCormack 2001 78
Moscucci 2005 79
Norem 2004 80
O’Connor 1991 81
Peters 2007 27
Peters 2009 82
Schoenbaum 2001 83
Scott 2006 84
Spranca 2000 85
Spranca 2007 86
Swaminathan 2008 87
Tai-Seale 2004 88
Uhrig2002 89
Uhrig2006 90
Wedig2002 91
Werner 2005 35
Werner 2005
92
Reason for exclusion
Studydesign
Unable to retrieve
Studydesign
Studydesign
Studydesign
Studydesign
Studydesign
Studydesign, controlled before-after, 2 intervention sites, only1 control site
Studydesign
Outcome measure/types of intervention
Outcome measure
Outcome measure
Unable to retrieve
Studydesign
Outcome measures/hypotheticaldata
Outcome measures
Studydesign
Studydesign;2 intervention and 1 control group/raw data was not reported
Studydesign
Studydesign
Studydesign
Studydesign
Studydesign
Types of intervention/outcome measures, design;2 intervention groups, 1
control group
Studydesign
Studydesign
Outcome measures
Outcome measures
Study design, controlled before-after design; no information reported from the
2 included registries. Not enough information was reported regarding the
baseline data
Studydesign/outcome measures
Unable to retrieve
Types of intervention/ outcome measures/hypotheticaldata
Studydesign
Outcome measures/hypothetical data
Studydesign
Outcome measures/hypothetical data
Studydesign/outcome measures
Studydesign
Studydesign/interventions/outcome measures
Types of intervention/hypothetical data
Outcome measures/hypotheticaldata
Studydesign, not enough data point for interrupted time series criteria
Studydesign
Studydesign, a single control and single intervention before and after
comparison
Risk of bias in included studies
We included three study designs (ClRCT, ClQ-RCT and ITS) which we rated on different
risk of bias items, we applied items as appropriate for the relevant study design.
84
Public release of performance data in changing behaviour
Allocation concealment (selection bias)
One study46 provided insufficient information about allocation of concealment to allow
judgement of the degree of the risk of bias. One study47 described a non-random
method of concealing allocation: research investigators enrolling participants could
possibly foresee assignment, therefore there is a high risk of bias. In Tu49 a statistician
randomised participating hospitals stratified by type of hospital, we rated this as a low
risk of bias.
Adequate sequence generation (selection bias)
One study46 provided insufficient information about the sequence generation for
judging the degree of the risk of bias. One study47 described a non-random method of
sequence generation (sequence determined by odd or even case record numbers), so it
is possible that selection bias occurred. A third study49 used a random method of
sequence generation to assign the hospitals to the early feedback group or the delayed
feedback group.
Blinding
Blinding of the participants was impossible because they had to see what they
received.46,47 Analysis was based on computerised discharged abstracts, for which
participants could not be blinded.48 It was also impossible to blind hospitals to their
randomisation status.49
Incomplete outcome data
Three studies46-48 had complete outcome data for the primary outcomes. The results for
the entire sample are presented. In Tu49 one of 86 hospitals withdrew from the baseline
phase after randomisation, and four withdrew from the follow-up phase, all due to
resource constraints, although they did not report a reason for the drop-out. We rated
this item as having an ‘unclear risk of bias’: five hospitals dropped out, and this
affected both intervention and control groups.
Selective reporting
We have checked two study protocols, the published reports include the expected
outcomes.46,47 As far as the other two studies were concerned,48,49 we were not able to
check whether the publications included the expected outcomes.
Other potential sources of bias
Three studies46,47,49 were free of other bias. The ITS study48 had a potential bias since
the collection periods were temporally moved about dependent upon when the hospital
became an outlier.
85
5
Chapter 5
Baseline characteristics
In two studies46,47 the risk of bias regarding the baseline characteristics is unclear, since
they did not report demographic variables for the intervention and control groups. One
study49 reported the baseline characteristics across their two groups of hospitals.
Reliable outcome measures
Two studies46,47 achieved appropriate methods for the outcome measurements, and one
study49 did not. The primary outcomes measures were developed in a national team of
experts, but the measures were not field-tested. The last step in the validation process
was not undertaken, thus the reliability of the measures was impossible to determine.
Protection against contamination
The risk of contamination in one study was unclear.46 In another study, the risk of
contamination was low because the enrolling participants received the enrolment
materials in their homes.47 It is likely that a few respondents would discuss the CAHPS
material with others, but the reality is that the risk of contamination cannot be
managed in such cases, simply because of the nature of public reporting. The third
study did not provide an explicit statement regarding the methods used to prevent
against contamination.49 There was extensive media coverage following the release of
the baseline performance data for the intervention group. The control group also
initiated some quality improvement activities after becoming aware of the release of
the performance data, which could indicate that the control group had been affected.
As in Farley,47 this is difficult to prevent because of the nature of public reporting.
However, Tu49 might have seen that the extended media coverage would affect the
hospitals in the control group.
Intervention independent of other changes
In the ITS study48 it is unclear whether the intervention occurred independently of
other changes over time or whether that the outcome was influenced by other
confounding variables and events during the study period.
Shape of intervention effect pre-specified
The Romano 2004 study adequately pre-specified the shape of the intervention effect.
Knowledge of the allocated interventions adequately prevented during the study
The Romano 2004 study dealt with the knowledge of the allocated interventions
suitably.
86
Public release of performance data in changing behaviour
Intervention unlikely to affect data collection
The Romano 2004 study appropriately managed the risk of affecting the data
collection.
Effects of interventions
See: Summary of findings for the main comparison.
Characteristics of interventions
In Farley46 conducted between February and May 2000, the control group received
standard enrolment materials by personal mailing post, including items such as
Medicaid benefits, instructions about the enrolment process, and available information
sources. The experimental group received this standard enrolment material plus the
Consumer Assessment of Healthcare Providers and System (CAHPS) report. Health plans
were categorised on the basis of their CAHPS performance, defined as high and low
performance plans. The CAHPS measures and report template used bar charts rating
the overall health plan, overall healthcare and the personal doctor. Additional charts
reported respondents’ views on five aspects of service: ‘getting needed care’, ‘getting
care without long waits’, ‘how well doctors communicate’, ‘courtesy, respect and
helpfulness’, and ‘health plan customer service’. A three-point scale (sometimes/
never, usually and always) was used. The Iowa Medicaid programme did not offer
additional proactive support to the intervention group participants for making health
plan choices.
In Farley47 conducted in March and April 1998, the control group beneficiaries received
the standard mailing of Medicaid enrolment materials. The experimental group
received the standard enrolment material plus the CAHPS report. Following the CAHPS
convention for comparative rating, a three-star rating was used with one star for plans
with survey results that scored significantly lower than average, two stars for those
that were not significantly different from the average for all other Medicaid plans in
New Jersey, and three stars for plans that scored significantly better than average. The
participants were asked to choose one HMO and sometimes a primary care casemanagement plan. The state contracted a private firm to manage the enrolment
process and assist participants in choosing their plans. They were able to call a free
phone number and ask questions about plans. The contractor also sent ‘health benefit
co-ordinators’ into county welfare offices and the community to assist participants in
choosing. The Medicaid office automatically assigned participants who did not make to
a health plan by the Medicaid office.
In Romano 2004,48 report cards were published by agencies in California and New York,
reporting on patient outcomes for coronary artery bypass grafting (CABG), acute
87
5
Chapter 5
myocardial infarction (AMI) or postdiscectomy complications. The California data began
in 1991 with the California Hospital Outcomes Project. The first report, released in
California, December 1993, used a two-category rating and classified hospital mortality
for AMI and complication rates for cervical and lumbar discectomy as either ‘better’
and ‘not better’ than expected. The second report, released in May 1996, classified
hospital mortality for AMI into three categories as ‘better’, ‘worse’ or ‘neither better
nor worse’ than expected. The analysis for California was based on the California
Patient Discharge Data Set.
The New York Cardiac Surgery Reporting System(CSRS) began in 1989 with the creation
of a special data system for cardiac surgery. In New York hospital-specific, riskadjusted mortality rates using a three-category classification have been released every
12 to 24 months since December 1990. The analysis for New York is based on the
Statewide Planning and Research Co-operative System.
In Tu 2009,49 conducted between April 1999 and April 2005, the early feedback group
(42 hospitals) received their baseline performance data of 12 process-of-care indicators
for AMI and six indicators for congestive heart failure (CHF) for internal validation
checks. The results were publicly released at a press conference and on the internet in
January 2004. The early feedback hospitals were encouraged to develop standardised
admitting orders and discharge plans, based on the baseline performance. Baseline
performance results of the delayed feedback group (N=41 hospitals) were publicly
released on the internet in September 2005 after internal validation. To determine the
effect of the public release and feedback, clinical information was collected from chart
reviews during the follow-up (1 April 2004 to 31 March 2005 inclusive 15,997 patients)
and compared with the baseline performance data (1 April 1999 to 31 March 2001
inclusive 20,039 patients). The primary outcome measures were defined as being the
difference in the mean hospital-specific performance between the two study groups on
two composite indicators, i.e. one for AMI and one for CHF.
Main outcome measures
Interventions targeting improvement through selection: changes in healthcare
utilisation decisions of consumers or healthcare providers
In Farley 200246 22.6% of the participants switched from the default health plan to
another health plan. Participants in the type I counties with three plan choices were
less likely to switch (19.9%) than those in the type II or type III counties with only two
choices (25.4% overall). Availability of CAHPS data had no effect on the switching rate;
individuals not receiving information moved from lower to higher quality plans as often
as those who did receive CAHPS data (Table 4).
88
Public release of performance data in changing behaviour
Table 4. Estimated effects of CAHPS information on enrolment choices by new beneficiaries enrolled
in the Iowa Medicaid programme
Percentages
(unadjusted frequenties)
Control
Intervention
(No CAHPS) (CAHPS)
Odds ratio (95% CI) Lower CI Upper CI
for the CAHPS group
Type I counties
Assigned to high-rated HMO
N=1717
N=1693
Stayed in HMO
84.0%
85.7%
0.80
0.58
1.09
Switched to MediPass
13.2%
10.6%
Switched to low-rated HMO
2.7 %
3.8 %
1.36
0.75
2.45
Assigned to low-rated HMO
N=1614
N=1679
Stayed in HMO
76.0%
74.7%
1.03
0.75
1.39
Switched to MediPass
14.1%
14.4%
Switched to high-rated HMO
9.9%
11.0%
1.13
0.79
1.60
Type II counties
N=1087
N=1037
Assigned to high-rated HMO
Stayed in HMO
70.5%
71.8%
0.92
0.68
1.24
Switched to MediPass
29.5%
28.2%
Type III counties
N=2097
N=2153
Assigned to low-rated HMO
Stayed in HMO
76.3%
76.4%
0.99
0.79
1.23
Switched to MediPass
23.7%
23.6%
CAHPS: Consumer Assessment of Health Plans Study; CI: Confidence Intervals; HMOs: health maintenance
organisations; MediPass: Medicaid primary care case management programme
High, low-rated: the reports or experience are composite scores that are averages of response to sets of
individual items using four-category response options. Farley 200246
Farley 200247 did not find any significant differences between the plan choices of the
enrollees in the intervention and control groups. Sixty-eight percent of the intervention
group and 69% of the control group chose a plan. The standardised CAHPS rating for
those who chose a plan were -0.03 for the intervention and 0.03 for the control groups;
28% and 27% respectively chose the dominant HMO. For those not selecting the
dominant HMO, the standardised CAHPS ratings of the selected plan were 1.80 and 1.73
respectively (Table 5).
Table 5. Plan choices for April enrollees
Proportion choosing a plan
Farley 200247
Mean or proportion
Reports (n=2649)
0.68
Control (n=2568)
0.69
Romano48 estimated time series models using ordinary least squares (OLS) data from the
states of New York and California. They re-analysed the California data with
autoregressive integrated moving average (ARIMA) methods. In autoregressive models
there were no clear patterns of effect developed between AMI report cards and
subsequent hospital volume for either AMI or related AMI conditions. There was a small
and temporary increase in volume in low complication rate hospitals for lumbar
89
5
Chapter 5
discectomy. Romano48 only report OLS results from New York because autocorrelation
was minimal in that state. The study also found a significant increase in CABG volume
for low-mortality hospitals in New York within the first month after publication and a
significant decrease in volume for high-mortality outliers in the second month after
release of the information (Table 6 and Table 7).
Table 6. Mean differences between actual and predicted monthly patient volume for the average
outlier hospital, over 4 consecutive months in New York. After publication of a riskadjusted outcome study, using ordinary least-squares regressiona
Actual minus predicted monthly patient volume (95% confidence interval)
Outlier
Month 2 (NY)
Month 3 (NY)
Month 4 (NY)
groupb Month 1 (NY)
Better 13.4d (4.3 to 22.6) 5.5 (-3.5 to –14.7) 6.7 (-1.5 to –15.0) 3.0 (-5.0 to 11.0)
(D=1.92)
Worse
-4.0 (-9.0 to –1.0) -7.1d (-12.3 to -1.9) -2.7 (-8.0 to –2.7) -0.9 (-5.9 to 4.1)
(D=1.91)
NY CABGBetter -4.9 (-12.3 to –2.4) -1.4 (-8.7 to –5.9) -1.9 (-8.7 to –4.8) 0.5 (-6.1 to 7.2)
related (AMI) (D=1.96)
Worse
-4.5c (-8.5 to -0.6) -1.2 (-5.2 to –2.8) -1.6 (-5.4 to –2.2) -6.0d (-9.8 to -2.2)
(D=1.38)
NY CABG related Better 3.7 (-3.2 to –10.8) 1.1 (-6.0 to –8.3)
0.6 (-6.1 to –7.4) -1.2 (-7.8 to 5.5)
(PTCA)
(D=2.14)
Worse
-2.6 (-7.0 to –1.8) -1.4 (-6.0 to –3.1) 0.4 (-4.2 to –4.9) -2.1 (-6.6 to 2.5)
(D=1.34)
NY CABGBetter -2.8 (-8.7 to –3.1) -4.0 (-9.9 to –2.0) -0.5 (-6.0 to –5.0) -1.7 (-7.1 to 3.7)
related
(D=1.74)
(CHF)
Worse
-1.0 (-5.8 to –3.9) -2.0 (-7.1 to –3.1) -1.7 (-6.6 to –3.2) -0.1 (-4.8 to 4.7)
(D=2.14)
a
Positive numbers indicate that hospitals in that category had more admissions than predicted; negative
numbers indicate that hospitals in that category had fewer admissions than predicted
b
The Durbin-Watson statistics in this column represent the magnitude of autocorrelation affecting OLS
models. Values close to 2 indicates the absence of autocorrelation
c
Two-tailed p < 0.005; d Two-tailed p < 0.01
AMI: Acute myocardial infarction; CABG: coronary artery bypass grafting; PTCA: percutaneous
transluminal coronary angioplasty; CHF: congestive heart failure. Romano48
State Condition or
procedure
NY CABG
(target)
90
Public release of performance data in changing behaviour
Table 7. Mean differences between actual and predicted monthly patient volume for the average
outlier hospital in California, over 4 consecutive quarters after publication of a riskadjusted outcome study, using autoregressive modelsa (ARIMA)
Actual minus predicted monthly patient volume (95% confidence
interval)
State Condition or Outlier
Quarter 2 (CA)
Quarter 3 (CA)
Quarter 4 (CA)
procedure
groupb Quarter 1 (CA)
CA AMI (target) Betterb 1.9 (-0.1 to -3.9)
-1.1 (-3.2 to –0.9) -0.6 (- 2.7 to -1.6) 1.1 (-1.3 to 3.6)
Worseb 0.7 (-1.6 to –3.0)
1.0 (-1.4 to –3.5) 0.0 (-2.3 to –2.4) 0.6 (-2.0 to 3.3)
CA AMI-related Better -1.1 (-4.9 to –2.7) 4.2 (-0.1 to –8.5) -3.8 (-0.8 to –8.3) -0.1 (-4.6 to 4.5)
Worse 1.0 (-1.5 to –3.6)
0.4 (-2.5 to –3.2) 0.4 (-3.2 to –2.5) -1.0 (-4.1 to 2.2)
CA Cervical
Better 0.2 (-1.1 to –1.5)
-0.3 (-1.8 to –1.3) -1.6c (0.0 to –3.2) -0.6 (-2.2 to 1.0)
c
discectomy
Worse -1.1 (-2.0 to -0.0) 0.3 (-0.9 to –1.6) 1.1 (-0.1 to –2.3) 0.9 (-0.4 to 2.1)
(target)
CA Lumbar
Betterb 0.6 c(0.0 to –1.1)
0.3 (-0.3 to –0.9) 0.5 (-0.2 to –1.2) 0.8c (-0.1 to 1.5)
discectomy
Worse -0.1 (-0.8 to –0.6) -0.1 (-0.9 to –0.7) -0.3 (-1.2 to –0.6) -0.5 (-1.4 to 0.3)
(target)
CA Discectomy- Betterb 0.4 (-0.1 to –1.9)
-0.9 (-2.4 to –0.7) -1.1 (-2.7 to -0.4) 0.4 (-1.4 to 2.1)
related
Worse
-1.4c (-2.4 to -0.3) 0.2 (1.1 to –1.4) 0.0 (-1.2 to –1.2) 0.2 (1.0 to 1.5)
a
Positive numbers indicate that hospitals in that category hadmore admissions than predicted; negative
numbers indicate that hospitals in that category had fewer admissions than predicted. To estimate the
total difference in patient volume for the average California hospital in each quarter, the numbers shown
should be multiplied by 3
b
The Durbin-Watson statistics in this column represent the magnitude of autocorrelation affecting OLS
models. Values close to 2 indicates the absence of autocorrelation
c
Two-tailed P < 0.005
ARIMA: autoregressive integrated moving average; AMI: Acutemyocardial infarction; CABG: coronary
artery bypass grafting; PTCA: percutaneous transluminal coronary angioplasty; CHF: congestive heart
failure. Romano48
Interventions targeting improvement through changes in care: objectives measures
of provider performance
Tu 2009 did not find significant differences in either the composite AMI indicator
(absolute change 1.5%; 95% CI -2.2% to 5.1%; P=0.43) (Table 6) or composite CHF
indicator (absolute change 0.6%; 95% CI -4.5% to 5.7%; P=0.81) (Table 7) in the early
feedback group compared with the delayed feedback group. Regarding individual
process-of-care indicators, one of the 12 for AMI and one of the six for CHF improved
significantly in the early feedback group (Table 8 and Table 9). The AMI 30-days
mortality rate was significantly lower in the early feedback group than in the delayed
feedback group (absolute change -2.5%; 95% CI -0.1% to -4.9%; P=0.045), while the oneyear mortality rates of the early, and delayed feedback groups were comparable. The
30-days and one-year CHF mortality rates did not differ significantly. In addition to the
release of a public report card, there was a hospital survey. The early feedback group
initiated more quality improvement activities in response to the publicly released
report card (for AMI 73.2% versus 46.7%; P=0.003 and for CHF 61% versus 50%; P=0.04).
91
5
Table 8. Mean changes in acute myocardial infarction (AMI) quality indicators in hospitals after publication of report cards for the early
feedback group
Early feedback (n=42)
Delayed feedback (n=39)
Absolute difference for
Baseline Follow-up Absolute change % Baseline Follow-up Absolute change % early versus delayed
%
(95% CI)
%
%
(95% CI)
feedback % (CI)*
P value
%
All 12 AMI process-of-care 57.4
65.5
8.2 (5.8 to 10.7)
56.5
63.6
7.1 (4.3 to 10.0)
1.5 (-2.2 to 5.1)
0.43
indicators
process-of-care quality indicators:
72.5
Left of standard admission 73.3
-0.8 (-5.9 to 4.3)
72.6
66.4
-6.2 (-13.7 to 1.2) 5.8 (-2.6 to 14.2)
0.17
orders
Left ventricular function 45.6
49.8
4.2 (-0.9 to 9.4)
39.3
46.9
7.6 (3.1 to 12.2)
-2.0 (-8.7 to 4.7)
0.56
assessment
Lipid test ≤ 24 h of arrival 34.1
51.1
17.0 (10.7 to 23.3) 35.7
54.9
19.2 (12.8 to 25.8) -2.9 (-10.7 to 4.9)
0.46
Fibrinolytics ≤ 30 min or 39.0
45.7
6.7 (-0.8 to 14.2)
35.9
43.1
7.2 (-0.5 to 15.1)
3.3 (-5.7 to 12.4)
0.47
primary PCI ≤ 90 min
Fibronolytic
19.9 (10.7 to 29.1) 68.8
86.3
17.5 (9.2 to 25.9) -1.6 (-9.5 to 6.4)
0.70
64.4
84.3
administration decided by
emergency department
physician
Fibrinolytics giver prior to 80.4
95.7
16.3 (7.1 to 23.7) 85.5
91.9
6.4 (0.1 to 12.7)
5.8 (1.1 to 10.5)
0.02
transfer to CCU or ICU
Aspirin ≤ 6 h of arrival
75.9
82.6
6.7 (3.7 to 9.6)
72.8
77.1
4.3 (0.2 to 8.3)
4.3 (-0.1 to 8.8)
0.06
ß-blockers ≤ 12 h of
28.3
73.7
45.4 (38.8 to 51.9) 32.2
71.3
39.1 (31.3 to 46.8) 3.1 (-5.8 to 12.1)
0.049
arrival
Aspirin at discharge
84.6
84.0
-0.6 (-4.2 to 2.7)
84.6
83.1
-1.5 (-6.5 to 3.4)
0.9 (-3.2 to 4.3)
0.75
ß-blockers at discharge
77.4
85.6
8.2 (5.4 to 11.1)
77.4
85.0
7.6 (4.1 to 11.2)
0.6 (-3.2 to 4.3)
0.75
ACE inhibitors or ARB for 75
81.7
6.7 (1.0 to 12.4)
71.6
77.0
5.4 (-0.8 to 11.5)
2.8 (-5.2 to 10.8)
0.48
left ventricular
dysfunction
Statin at discharge
57.6
85.5
27.9 (20.0 to 35.8) 57.8
85.8
28.0 (19.7 to 36.3) -0.3 (-9.0 to 8.5)
0.95
ACE: angiotensin-converting enzyme; ARB: angiotensin II receptor blocker; CCU: coronary care unit; CI: confidence interval; ICU: intensive care unit;
PCI: percutaneous coronary intervention. Positive values indicate better performance in the early feedback hospitals. Tu 200949
Table 9. Mean changes in congestive heart failure (CHF) quality indicators among hospitals after publication of report cards for the early
feedback group
Delayed feedback (n=39)
Early feedback (n=42)
Baseline
%
Follow-up
%
Absolute change
% (95% CI)
Baseline
%
Follow-up
%
Absolute difference for
Absolute change % early versus delayed
P value
feedback % (CI)a
(95% CI)
All 6 CHF process-of54.8
54.6
-0.2 (-5.0 to 4.6) 51.8
53.6
1.8 (-2.7 to 6.1)
0.6 (-4.5 to 5.7)
0.81
care indicators
Individual CHF process-of-care quality indicators
Left ventricular
47.9
55.2
-7.3 (1.5 to 13.0)
43.4
52.5
9.1 (3.5 to 14.6)
1.2 (-5.3 to 7.7)
0.72
function assessment
Daily weights recorded 14.8
24
9.2 (4.3 to 14.0)
15.1
22.7
7.6 (2.4 to 12.8)
1.8 (-5.2 to 8.8)
0.60
Counselling on ≥ 1
56.2
-10.5 (-18.2 to 2.7) -0.4 (-8.4 to 7.6)
0.92
68.4
55.3
-13.0 (-21.8 to -4.5) 66.7
aspect of CHF
ACE inhibitor or ARB for
left ventricular
88.2
92.4
4.2 (0.7 to 7.8)
86.5
86.1
-0.4 (-7.4 to 6.5)
5.9 (1.0 to 10.7)
0.02
dysfunction
ß-blocker for left
40
71.7
31.7 (22.6 to 40.9) 38.3
67.7
29.4 (18.9 to 39.8) 3.5 (-6.1 to 13.1)
0.47
ventricular dysfunctionb
Warfarin for artrial
52.4
64.2
11.8 (4.3 to 19.2)
49.3
63.6
14.3 (6.8 to 22.0) -0.2 (-6.5 to 6.2)
0.96
fibrillation
ACE: angiotensin-converting enzyme; ARB: angiotensin II receptor blocker; CCU: coronary care unit; CI: confidence interval; ICU: intensive care unit;
PCI: percutaneous coronary intervention.
a
Represents mean relative improvement in each indicator in the early feedback hospitals with the delayed feedback hospitals in the follow-up patient
cohort after adjusting for indicator performance in the baseline patient cohort and type of hospital.
Positive values indicate better performance in the early feedback hospitals.
b
Patients with documented ejection fraction of 40% or less during the index admission or within the previous 6 months were considered to have left
ventricular dysfunction.
5
Chapter 5
Other outcome measures
Awareness, attitude, views and knowledge of performance data and cost data were not
reported in three of the included studies. Farley47 reported secondary outcomes as a
result of a survey. Farley47 used a ratio of 3:1 (report versus no report) to send the
survey. They had problems with differential sampling and response rates: therefore the
data become difficult to interpret. We decided to exclude these results and so do not
report these outcomes.
Data syntheses
We have summarised the outcome data extracted from papers in a narrative format in
the section ‘Characteristics of interventions’. We did not synthesise any quantitative
data.
DISCUSSION
Summary of main results
In four studies interventions consisting of either direct (mailed) or indirect (internet)
release of performance data were focused on changes in selection or changes in care.
For changes in selection, from two studies we found no impact on choice of health plan
in Medicaid populations. From one study there was a small effect of the public release
of mortality and complication data on patient volumes for CABG surgery, and lumbar
discectomy; however, these effects did not persist for more than two months after
each release. There were no effects of releasing mortality data on patient volumes for
AMI outcomes. The changes of care were evaluated for the indirect release of
performance data for patients with AMI and CHF in one study. There were no effects on
composite process-of-care indicators for either condition, but there were some
improvements in individual process-of-care indicators for both AMI and CHF. There was
also an effect on 30 day AMI mortality rates, and more quality improvement activities
were initiated in response to the publicly-released report cards.
Overall completeness and applicability of evidence
The three studies that took place in the USA involved only a small proportion of the
numerous major reporting systems available. We included one new study from Canada49
that was published after the last systematic reviews by Shekelle,40 Fung8 and Faber.41
We excluded many of the more recent studies because they did not have a rigorous
study design or did not report the defined primary outcome measures.
Regarding overall completeness, we conclude that evaluations of public reporting
system are scarce. Only a few current reporting systems have been subjected to
scientific evaluation to determine the effects of public disclosure of quality information
94
Public release of performance data in changing behaviour
in various stakeholders.40,93 Studies that compare different reporting systems are
lacking, as are studies of purchaser behaviour.
Despite evidence that secondary outcome measures (e.g. awareness, attitude,
knowledge of performance data) are crucial since public reporting can only change
behaviour if the target population (healthcare consumers, providers or purchasers of
care) understand the provided information,21 these measures are lacking in the
included studies. Because of that it is difficult to explain the lack of effect. Faber41
demonstrated that effect of performance data was higher for those who understand the
information. Damman28 showed that comparative performance information is complex,
and consumers had difficulties in interpreting and using performance data.
One type of performance information included in our studies was about patients
experiences (CAHPS), items e.g. regarding doctor-patient communication, long waits,
respect. Other included types of performance information were mortality, and
complication data. Patient-Reported outcome measures (PROMs) were not included,
nor was performance information about services. Mortality and complication data were
included, but only for two conditions (AMI, CHF) and two surgical procedures (CABG,
discectomy).
Quality of the evidence
The quality of the evidence in this review appears to be low based on the analysis with
the Grades of Recommendations Assessment, Development and Evaluation (GRADE)
system. We downgraded the quality of the evidence for the outcomes due to some
concerns with risk of bias in the studies, loss to follow-up, and very sparse data.
There is one more source of concern regarding the quality of the included studies: we
did not have access to the complete study protocols for two of the studies, so we could
not judge the risk of selective reporting definitively.
The issue of contamination is difficult to tackle for a public reporting intervention
because it is often impossible to prevent control groups from seeing information that is
publicly available on websites and in the media. Control of exposure can be gained if
the information is only posted to consumers personally or if the control and
intervention groups are geographically separated. In one study,49 there was extensive
media coverage when hospitals in the early feedback group received their baseline
performance data. A survey among hospitals in the delayed feedback group confirmed
that these control hospitals were affected by the release of performance data in the
early feedback group.
95
5
Chapter 5
Potential biases in the review process
Although our search was comprehensive, we cannot exclude the possibility of having
missed relevant studies. We were unable to retrieve and assess four possibly relevant
studies in full text. Two review authors independently examined all the references we
found in our search. Two review authors independently extracted detailed data and
assessed the risk of bias and a third review author settled any disagreements. We did
this to exclude bias in the review process.
Agreements and disagreements with other studies or reviews
There are three relevant publications: an article by Kolstad10 and two systematic
reviews by Faber41 and Fung 2008.8 Our conclusion agrees with those of Kolstad10 and
Faber;41 we do not know the extent to which quality reporting leads to improvement of
health care quality. We also agree with the conclusion of Fung 2008;8 despite the
existence of major public reporting systems, we lack rigorous evaluations of the effects
of these systems.
AUTHORS’ CONCLUSIONS
Implications for practice
The results of this review do not enable us to make any strong recommendations for
practice. Whilst performance data may be publicly released for many reasons, we
cannot conclude from the limited evidence whether disclosure of performance
information can reliably change the behaviour of consumers, providers, purchasers or
professionals.
Implications for research
In order to understand the effectiveness of the public release of performance data, we
need more longitudinal studies with robust evaluation designs and, in particular,
studies that test for delayed or cumulative effects with continuing measurements. To
improve our insight into the current and potential impacts of public reporting, we need
to evaluate a variety of reporting systems in the USA and other countries.
As the lack of effect might be due to a missing of actual exposure to performance data,
a specific implication for future studies targeting the consumer’s choice behaviour is
that the intervention group (i.e. those provided with performance data) should actually
read and understand the performance data. Additional interventions might enhance the
impact on consumers with limited health literacy in the intervention group.25 Studies
targeting improvements effected by changes in care might benefit from baseline
performance data for the intervention group that is released repeatedly instead of only
once.
96
Public release of performance data in changing behaviour
Berwick’s model suggests that public release of performance data may improve quality
of care by means of a pathway of change or selection.3 The studies we included focused
on either one or the other of these pathways exclusively. We suggest a study design
that combines the two pathways to assess the relationship between them.
A basic assumption underlying the provision of report cards is that provider choice is a
rational decision. In other words, consumers prefer the healthcare provider or health
plan rated as the best. Evidence that confirms this assumption is limited.10,41 However,
several factors that influence the choice of consumers are known, such as established
relationships with local physicians, health plans, 9,94 hospitals, distance, and opinions of
friends, and family.29,30 Future studies should address the range, and relative impact of
factors such as these.
5
97
Chapter 5
REFERENCES
1.
2.
3.
4.
5.
6.
7.
8.
9.
10.
11.
12.
13.
14.
15.
16.
17.
18.
19.
20.
21.
22.
23.
24.
98
Smith PC, Mossialos E, Papanicolas I, Leatherman S. Performance measurement for health system
improvement. Experiences, Challenges and Prospects. Cambridge: Cambridge University Press,
2009.
Marshall MN, Shekelle PG, Leatherman S, Brook RH. The public release of performance data: what
do we expect to gain? A review of the evidence. JAMA 2000;283(14):1866-74.
Berwick DM, James B, Coye MJ. Connections between quality measurement and improvement. Med
Care 2003;41(1 Suppl):I30-38.
Bentley JM, Nash DB. How Pennsylvania hospitals have responded to publicly released reports on
coronary artery bypass graft surgery. Jt Comm J Qual Improv 1998;24(1):40-49.
Hendriks M, Spreeuwenberg P, Rademakers J, Delnoij DM. Dutch healthcare reform: did it result in
performance improvement of health plans? A comparison of consumer experiences over time. BMC
Health Serv Res 2009;9167.
Longo DR, Land G, Schramm W, Fraas J, Hoskins B, Howell V. Consumer reports in health care. Do
they make a difference in patient care? JAMA 1997;278(19):1579-84.
Lindenauer PK, Remus D, Roman S, Rothberg MB, Benjamin EM, Ma A, et al. Public reporting and
pay for performance in hospital quality improvement. N Engl J Med 2007;356(5):486-96.
Fung CH, Lim YW, Mattke S, Damberg C, Shekelle PG. Systematic review: the evidence that
publishing patient care performance data improves quality of care. Ann Intern Med 2008;148(2):
111-23.
Hibbard JH. Using systematic measurement to target consumer activation strategies. Med Care Res
Rev 2009;66(1 Suppl):9S-27S.
Kolstad JT, Chernew ME. Quality and consumer decision making in the market for health insurance
and health care services. Med Care Res Rev 2009;66(1 Suppl):28S-52S.
Werner RM, Konetzka RT, Kruse GB. Impact of public reporting on unreported quality of care.
Health Serv Res 2009;44(2 Pt 1):379-98.
Marshall MN, Romano PS, Davies HT. How do we maximize the impact of the public reporting of
quality of care? Int J Qual Health Care 2004;16 (Suppl 1):i57-63.
Fotaki M, Roland M, Boyd A, McDonald R, Scheaff R, Smith L. What benefits will choice bring to
patients? Literature review and assessment of implications. J Health Serv Res Policy 2008;13(3):
178-84.
Berwick DM, Wald DL. Hospital leaders’ opinions of the HCFA mortality data. JAMA 1990;263(2):
247-49.
Sirio CA, McGee JL. Public reporting of clinical outcomes--the data needs of health care
stakeholders. Am J Med Qual 1996;11(1): S78-81.
Rosenthal GE, Hammar PJ, Way LE, Shipley SA, Doner D, Wojtala B, et al. Using hospital
performance data in quality improvement: the Cleveland Health Quality Choice experience. Jt
Comm J Qual Improv 1998;24(7):47-60.
Schut FT, Van de Ven WP. Rationing and competition in the Dutch health-care system. Health Econ
2005;14(Suppl 1):S59-74.
Brook RH. Health care reform is on the way: do we want to compete on quality? Ann Intern Med
1994;120(1):84-86.
Hibbard JH, Jewett JJ, Legnini MW, Tusler M. Choosing a health plan: do large employers use the
data? Health Aff (Millwood) 1997;16(6):172-80.
Mukamel DB, Mushlin AI. Quality of care information makes a difference: an analysis of market
share and price changes after publication of the New York State Cardiac Surgery Mortality Reports.
Med Care 1998;36(7):945-54.
Hibbard JH, Greene J, Daniel D. What is quality anyway? Performance reports that clearly
communicate to consumers the meaning of quality of care. Med Care Res Rev 2010;67(3):275-93.
Hibbard JH, Peters E, Slovic P, Finucane ML, Tusler M. Making health care quality reports easier to
use. Jt Comm J Qual Improv 2001;27(11):591-604.
Magee H, Davis LJ, Coulter A. Public views on healthcare performance indicators and patient
choice. J R Soc Med 2003;96(7):338-42.
O’Meara J, Kitchener M, Collier E, Lyons M, de Billwiller-Kiss A, Simon LP, et al. Case study:
development of and stakeholder responses to a nursing home consumer information system. Am J
Med Qual 2005;20(1):40-50.
Public release of performance data in changing behaviour
25.
26.
27.
28.
29.
30.
31.
32.
33.
34.
35.
36.
37.
38.
39.
40.
41.
42.
43.
44.
45.
46.
47.
48.
49.
Hibbard JH, Peters E, Dixon A, Tusler M. Consumer competencies and the use of comparative
quality information: it isn’t just about literacy. Med Care Res Rev 2007;64(4):379-94.
Harris-Kojetin LD, Uhrig JD, Williams P, Bann C, Frentzel EM, McCormack L, et al. The “choose
with care system” - development of education materials to support informed Medicare health plan
choices. J Health Commun 2007;12(2):133-56.
Peters E, Dieckmann N, Dixon A, Hibbard JH, Mertz CK. Less is more in presenting quality
information to consumers. Med Care Res Rev 2007;64(2):169-90.
Damman OC, van den Hengel YK, van Loon AJ, Rademakers J. An international comparison of webbased reporting about health care quality: content analysis. J Med Internet Res 2010;12(2):e8.
Dixon A, Robertson R, Appleby J, Burge P, Devlin N, Magee H. Patient choice. How patients choose
and providers respond. In, London: The King’s Fund, 2010.
Harris KM, Beeuwkes Buntin M, Cooperation. TR. Research Synthesis Report. Choosing a healthcare
provider: the role of quality information. Princeton: Robert Wood Johnson Foundation, 2008.
Giuffrida A, Gravelle H, Roland M. Measuring quality of care with routine data: avoiding confusion
between performance indicators and health outcomes. BMJ 1999;319(7202):94-98.
Kerr EA, Hofer TP, Hayward RA, Adams JL, Hogan MM, McGlynn EA, et al. Quality by any other
name?: a comparison of three profiling systems for assessing health care quality. Health Serv Res
2007;42(5):2070-87.
Parry GJ, Gould CR, McCabe CJ, Tarnow-Mordi WO. Annual league tables of mortality in neonatal
intensive care units: longitudinal study. International Neonatal Network and the Scottish Neonatal
Consultants and Nurses Collaborative Study Group. BMJ 1998;316(7149):1931-35.
Rixom A. Performance league tables. BMJ 2002;325(7357):177-78.
Werner RM, Asch DA. The unintended consequences of publicly reporting quality information. JAMA
2005;293(10):1239-44.
Dranove D, Sfekas A. Start spreading the news: a structural estimate of the effects of New York
hospital report cards. J Health Econ 2008;27(5):1201-7.
Bardach NS, Cabana MD. The unintended consequences of quality improvement. Curr Opin Pediatr
2009;21(6):777-82.
Loeb JM. The current state of performance measurement in health care. Int J Qual Health Care
2004;16(Suppl 1)i5-9.
Jamtvedt G, Young JM, Kristoffersen DT, O’Brien MA, Oxman AD. Audit and feedback: effects on
professional practice and health care outcomes. Cochrane Database Syst Rev 2006(2):CD000259.
Shekelle PG, Lim Y-W, Mattke S, Damberg C, Southern California Evidence-based Practice Centre,
Corporation. R. Does public release of performance results improve quality of care? A systematic
review. London: The Health Foundation, 2008.
Faber M, Bosch M, Wollersheim H, Leatherman S, Grol R. Public reporting in health care: how do
consumers use quality-of-care information? A systematic review. Med Care 2009;47(1):1-8.
Harris KM. Can high quality overcome consumer resistance to restricted provider access? Evidence
from a health plan choice experiment. Health Serv Res 2002;37(3):551-71.
Higgins JPT, Green S. Cochrane Handbook for Systematic Reviews of Interventions Version 5.1.0
The Cochrane Collaboration, 2011. www.cochrane-handbook.org, 2011.
EPOC. EPOC risk of bias guideline. http://epoc.cochrane.org/epoc-resources-review-authors.
2009.
Moher D, Cook DJ, Eastwood S, Olkin I, Rennie D, Stroup DF. Improving the quality of reports of
meta-analyses of randomised controlled trials: the QUOROM statement. Quality of Reporting of
Meta-analyses. Lancet 1999;354(9193):1896-900.
Farley DO, Elliott MN, Short PF, Damiano P, Kanouse DE, Hays RD. Effect of CAHPS performance
information on health plan choices by Iowa Medicaid beneficiaries. Med Care Res Rev 2002;59(3):
319-36.
Farley DO, Short PF, Elliott MN, Kanouse DE, Brown JA, Hays RD. Effects of CAHPS health plan
performance information on plan choices by New Jersey Medicaid beneficiaries. Health Serv Res
2002;37(4):985-1007.
Romano PS, Zhou H. Do well-publicized risk-adjusted outcomes reports affect hospital volume?
Med Care 2004;42(4):367-77.
Tu JV, Donovan LR, Lee DS, Wang JT, Austin PC, Alter DA, et al. Effectiveness of public report
cards for improving the quality of cardiac care: the EFFECT study: a randomized trial. JAMA 2009;
302(21):2330-37.
99
5
Chapter 5
50.
51.
52.
53.
54.
55.
56.
57.
58.
59.
60.
61.
62.
63.
64.
65.
66.
67.
68.
69.
70.
71.
72.
73.
74.
100
Alteras TT. Health plan report cards may influence insurers more than consumers: their effect on
insurer behavior in Minnesota. Find Brief 2000;3(3):1-2.
Dean Beaulieu N, Epstein AM. National Committee on Quality Assurance health-plan accreditation:
predictors, correlates of performance, and market impact. Med Care 2002;40(4):325-37.
Beaulieu ND. Quality information and consumer health plan choices. J Health Econ 2002;21(1):4363.
Bundorf MK, Chun N, Goda GS, Kessler DP. Do markets respond to quality information? The case of
fertility clinics. J Health Econ 2009;28(3):718-27.
Dawson D, Gravelle H, Jacobs R, Martin S, Smith PC. The effects of expanding patient choice of
provider on waiting times: evidence from a policy experiment. Health Econ 2007;16(2):113-28.
Ettinger WH, Hylka SM, Phillips RA, Harrison LH, Jr., Cyr JA, Sussman AJ. When things go wrong:
the impact of being a statistical outlier in publicly reported coronary artery bypass graft surgery
mortality data. Am J Med Qual 2008;23(2):90-95.
Fanjiang G, von Glahn T, Chang H, Rogers WH, Safran DG. Providing patients web-based data to
inform physician choice: if you build it, will they come? J Gen Intern Med 2007;22(10):1463-66.
Fine A. The effect of grading and publicizing the physician groups' performance. Exec Solut Healthc
Manag 1998;1(9):2-3.
Fong J, Marsh GM, Stokan LA, Weilian S, Vinson C, Ruhl L. Hospital quality performance report: an
application of composite scoring. Am J Med Qual 2008;23(4):287-95.
Fox MH, Moore J, Zimmerman M, Hill S, Foster CH. The effectiveness of CAHPS among women
enrolling in Medicaid managed care. J Ambul Care Manage 2001;24(4):76-91.
Goldstein E, Fyock J. Reporting of CAHPS quality information to medicare beneficiaries. Health
Serv Res 2001;36(3):477-88.
Goss JR, Maynard C, Aldea GS, Marcus-Smith M, Whitten RW, Johnston G, et al. Effects of a
statewide physician-led quality-improvement program on the quality of cardiac care. Am Heart J
2006;151(5):1033-42.
Hannan EL, Sarrazin MS, Doran DR, Rosenthal GE. Provider profiling and quality improvement
efforts in coronary artery bypass graft surgery: the effect on short-term mortality among Medicare
beneficiaries. Med Care 2003;41(10):1164-72.
Hibbard JH, Sofaer S, Jewett JJ. Condition-specific performance information: assessing salience,
comprehension, and approaches for communicating quality. Health Care Financ Rev 1996;18(1):
95-109.
Hibbard JH, Harris-Kojetin L, Mullin P, Lubalin J, Garfinkel S. Increasing the impact of health plan
report cards by addressing consumers' concerns. Health Aff (Millwood) 2000;19(5):138-43.
Hibbard JH, Berkman N, McCormack LA, Jael E. The impact of a CAHPS report on employee
knowledge, beliefs, and decisions. Med Care Res Rev 2002;59(1):104-16.
Hibbard JH, Slovic P, Peters E, Finucane ML. Strategies for reporting health plan performance
information to consumers: evidence from controlled studies. Health Serv Res 2002;37(2):291-313.
Hibbard JH, Stockard J, Tusler M. Does publicizing hospital performance stimulate quality
improvement efforts? Health Aff (Millwood) 2003;22(2):84-94.
Hibbard JH, Stockard J, Tusler M. It isn’t just about choice: the potential of a public performance
report to affect the public image of hospitals. Med Care Res Rev 2005;62(3):358-71.
Hibbard JH, Stockard J, Tusler M. Hospital performance reports: impact on quality, market share,
and reputation. Health Aff (Millwood) 2005;24(4):1150-60.
Hollenbeak CS, Gorton CP, Tabak YP, Jones JL, Milstein A, Johannes RS. Reductions in mortality
associated with intensive public reporting of hospital outcomes. Am J Med Qual 2008;23(4):279-86.
Howgill M, Blaza J, Cunningham L, Foster KL. The ratings game. How important are hospital
rankings to consumers? Interview by Joyce Jensen. Mark Health Serv 2004;24(1):40-45.
Jha AK, Epstein AM. The predictive accuracy of the New York State coronary artery bypass surgery
report-card system. Health Aff (Millwood) 2006;25(3):844-55.
Jian W, Huang Y, Hu M, Zhang X. Performance evaluation of inpatient service in Beijing: a
horizontal comparison with risk adjustment based on Diagnosis Related Groups. BMC Health Serv
Res 2009;972.
Knutson DJ, Kind EA, Fowles JB, Adlis S. Impact of report cards on employees: a natural
experiment. Health Care Financ Rev 1998;20(1):5-27.
Public release of performance data in changing behaviour
75.
76.
77.
78.
79.
80.
81.
82.
83.
84.
85.
86.
87.
88.
89.
90.
91.
92.
93.
94.
Krupat E, Hsu J, Irish J, Schmittdiel JA, Selby J. Matching patients and practitioners based on
beliefs about care: results of a randomized controlled trial. Am J Manag Care 2004;10(11 Pt 1):
814-22.
Mannion R, Goddard M. Public disclosure of comparative clinical performance data: lessons from
the Scottish experience. J Eval Clin Pract 2003;9(2):277-86.
McCormack LA, Garfinkel SA, Hibbard JH, Norton EC, Bayen UJ. Health plan decision making with
new medicare information materials. Health Serv Res 2001;36(3):531-54.
McCormack LA, Anderson WL, Uhrig JD, Garfinkel SA, Sofaer S, Terrell SA. Health plan decision
making in the Medicare population: results from a national randomized experiment. Health Serv
Res 2001;36(6 Pt 2):133-49.
Moscucci M, Eagle KA, Share D, Smith D, De Franco AC, O’Donnell M, et al. Public reporting and
case selection for percutaneous coronary interventions: an analysis from two large multicenter
percutaneous coronary intervention databases. J Am Coll Cardiol 2005;45(11):1759-65.
Norem J, Moen MA. The Websites of Norwegian hospitals: do they meet national guidelines and
patient’s expectations? J Telemed Telecare 2004;10(5):272-76.
O’Connor SJ, Shewchuk RM, Bowers MR. A model of service quality perceptions and health care
consumer behavior. J Hosp Mark 1991;6(1):69-92.
Petersen LA, Woodard LD, Henderson LM, Urech TH, Pietz K. Will hypertension performance
measures used for pay-for-performance programs penalize those who care for medically complex
patients? Circulation 2009;119(23):2978-85.
Schoenbaum M, Spranca M, Elliott M, Bhattacharya J, Short PF. Health plan choice and information
about out-of-pocket costs: an experimental analysis. Inquiry 2001;38(1):35-48.
Scott IA, Ward M. Public reporting of hospital outcomes based on administrative data: risks and
opportunities. Med J Aust 2006;184(11):571-75.
Spranca M, Kanouse DE, Elliott M, Short PF, Farley DO, Hays RD. Do consumer reports of health
plan quality affect health plan selection? Health Serv Res 2000;35(5 Pt 1):933-47.
Spranca MD, Elliott MN, Shaw R, Kanouse DE. Disenrollment information and Medicare plan choice:
is more information better? Health Care Financ Rev 2007;28(3):47-59.
Swaminathan S, Chernew M, Scanlon DP. Persistence of HMO performance measures. Health Serv
Res 2008;43(6):2033-49.
Tai-Seale M. Does consumer satisfaction information matter? Evidence on member retention in
FEHBP plans. Med Care Res Rev 2004;61(2):171-86.
Uhrig JD, Short PF. Testing the effect of quality reports on the health plan choices of Medicare
beneficiaries. Inquiry 2002;39(4):355-71.
Uhrig JD, Harris-Kojetin L, Bann C, Kuo TM. Do content and format affect older consumers’ use of
comparative information in a Medicare health plan choice? Results from a controlled experiment.
Med Care Res Rev 2006;63(6):701-18.
Wedig GJ, Tai-Seale M. The effect of report cards on consumer choice in the health insurance
market. J Health Econ 2002;21(6):1031-48.
Werner RM, Asch DA, Polsky D. Racial profiling: the unintended consequences of coronary artery
bypass graft report cards. Circulation 2005;111(10):1257-63.
Smith PC, Mossialos E, Papanicolas I. Performance measurement for health system improvement:
experiences, challenges and prospects. World Health Organization, 2008.
Schwartz LM, Woloshin S, Birkmeyer JD. How do elderly patients decide where to go for major
surgery? Telephone interview survey. BMJ 2005;331(7520):821.
101
5
Chapter 5
APPENDICES
Appendix 1. MEDLINE search strategy
MEDLINE (OVID)
Syntax guide
/ - index term (MeSH heading)
exp - explode: includes narrower terms to the index term being exploded
.tw. - text word in title or abstract fields
$ - truncation/wild card: adds no or more characters
? - truncation/wild card: adds no or one character
# - truncation/wild card: retrieves alternative single character
adjx - adjacency: required words are adjacent to each other, or within x words of each other
.pt. - publication type
Description of search strategy
1. random$.tw.
2. multicenter study.pt.
3. randomized controlled trial.pt.
4. randomized controlled trial.pt.
5. clinical trial.pt.
6. intervention studies/
7. experiment$.tw.
8. (time adj series).tw.
9. (pre test or pretest or (posttest or post test)).tw.
10. random allocation/
11. impact.tw.
12. intervention?.tw.
13. chang$.tw.
14. evaluation studies/
15. evaluat$.tw.
16. effect?.tw.
17. comparative studies/
18. compar$.tw.
19. 1 or 2 or 3 or 4 or 5 or 6 or 7 or 8 or 9 or 10 or 11 or 12 or 13 or 14 or 15 or 16 or 17 or 18
20. editorial.pt.
21. letter.pt.
22. comment.pt.
23. 20 or 21 or 22
24. animals/
25. humans/
26. 24 not 25
27. 23 or 26
28. 19 not 27
29. (public release of performance data and healthcare providers).mp. [mp=title, original title, abstract,
name of substance word, subject heading word]
30. exp Primary Health Care/
31. exp Hospitals/
32. physicians/
33. health professionals.ab,ti.
34. health personnel/
35. health plans.ab,ti.
36. health plan.ab,ti.
37. insurance.ab,ti.
38. *Physician’s practice patterns/
102
Public release of performance data in changing behaviour
39. *Group Practice/
40. *Institutional Practice/
41. *Private Practice/
42. *Family Practice/
43. *Physicians/
44. *Physicians, Family/
45. *Professional Practice/
46. *Nurses/
47. *Nurse Clinicians/
48. *Nurse practitioners/
49. *Pharmacists/
50. *Pharmacies/
51. *Pharmacy/
52. *Hospitals/
53. (physician$ or GP? or doctor? or general pract$ or prescriber? or group pract$ or institutional pract$
or partnership pract$ or family pract$ or general pract$ or office pract$ or private pract$ or primary
pract$ or nurse or nurses).tw.
54. (pharmacist? or pharmacies or pharmacy).tw.
55. hospital?.tw.
56. physiotherapist.mp.
57. midwife.mp.
58. health care centre.mp.
59. dietician.mp.
60. health care provider.mp.
61. *Allied Health Personnel/
62. *Dental Clinics/
63. *Dentists/
64. *Outpatient Clinics, Hospital/
65. general pract$.tw.
66. psychologist.mp.
67. psychiatrist.mp.
68. 35 or 36 or 37
69. or/38-67
70. or/30-35
71. 69 or 70
72. 68 or 71
73. quality assurance, health care/
74. *benchmarking/
75. *“process assessment (health care)”/
76. *“outcome assessment (health care)”/
77. exp Quality Indicators, Health Care/
78. performance outcome.ab,ti.
79. (quality adj2 indicator?).tw.
80. (quality adj (criteria or criterion or standard? or norm)).tw.
81. (performance adj (indicator? or measure? or data or rating)).tw.
82. disclosure/
83. Information Services/
84. report card.ab,ti.
85. quality information.ab,ti.
86. public information.ab,ti.
87. consumer information.ab,ti.
88. patient information.ab,ti.
89. 73 or 74 or 75 or 76 or 77 or 78 or 79 or 80 or 81 or 82 or 83 or 84 or 85 or 86 or 87 or 88
90. exp Consumer Satisfaction/
91. patient preferences.ab,ti.
92. public reporting.tw.
93. consumer reports.ab,ti.
103
5
Chapter 5
94. decision making.ab,ti.
95. choice behaviour.ab,ti.
96. choice behaviour.ab,ti.
97. exp “Patient Acceptance of Health Care”/
98. ’provider profiling’.ab,ti.
99. 90 or 91 or 92 or 93 or 94 or 95 or 96 or 97 or 98
100. 28 and 72 and 89 and 99
104
Public release of performance data in changing behaviour
Appendix 2. EMBASE search strategy
EMBASE (OVID)
Syntax guide
/ - index term (EMTREE heading)
exp - explode: includes narrower terms to the index term being exploded
.tw. - text word In title or abstract fields
$ - truncation/wild card: adds no or more characters
? - truncation/wild card: adds no or one character
# - truncation/wild card: retrieves alternative single character
adjx - adjacency: required words are adjacent to each other, or within x words of each other
.pt. - publication type
Description of search strategy
1. exp consumer/ or *consumer health information/
2. patient preferences.ab,ti.
3. *patient attitude/
4. *patient participation/
5. *decision making/
6. *patient decision making/
7. 6 or 4 or 1 or 5 or 3 or 2
8. *total quality management/
9. *performance measurement system/
10. public reporting.mp.
11. *decision making/
12. *outcome assessment/
13. *interpersonal communication/
14. *health care quality/ or exp clinical indicator/ or exp “quality of nursing care”/
15. *quality control/
16. report card.ab,ti.
17. public information.mp.
18. consumer information.mp.
19. 13 or 18 or 10 or 14 or 16 or 9 or 15 or 8 or 11 or 17 or 12
20. random$.tw.
21. multicenter study.mp.
22. randomized controlled trial.mp.
23. randomized controlled trial.mp.
24. clinical trial.mp.
25. intervention studies/
26. experiment$.tw.
27. (time adj series).tw.
28. (pre test or pretest or (posttest or post test)).tw.
29. random allocation/
30. impact.tw.
31. intervention?.tw.
32. chang$.tw.
33. evaluation studies/
34. evaluat$.tw.
35. effect?.tw.
36. comparative studies/
37. compar$.tw.
38. 20 or 21 or 22 or 23 or 24 or 25 or 26 or 27 or 28 or 29 or 30 or 31 or 32 or 33 or 34 or 35 or 36 or 37
39. editorial.pt.
40. letter.pt.
41. comment.pt.
42. 39 or 40 or 41
105
5
Chapter 5
43. animals/
44. humans/
45. 43 not 44
46. 42 or 45
47. 38 not 46
48. exp Primary Health Care/
49. exp Hospitals/
50. physicians/
51. health professionals.ab,ti.
52. health personnel/
53. health plans.ab,ti.
54. health plan.ab,ti.
55. insurance.ab,ti.
56. (physician$ or GP? or doctor? or general pract$ or prescriber? or group pract$ or institutional pract$
or partnership pract$ or family pract$ or general pract$ or office pract$ or private pract$ or primary
pract$ or nurse or nurses).tw.
57. (pharmacist? or pharmacies or pharmacy).tw.
58. hospital?.tw.
59. physiotherapist.mp.
60. midwife.mp.
61. health care centre.mp.
62. dietician.mp.
63. health care provider.mp.
64. general pract$.tw.
65. psychologist.mp.
66. psychiatrist.mp.
67. exp Group practice/
68. exp benchmarking/
69. exp Institutional practice/
70. exp Physician’s Practice Patterns/
71. exp private practice/
72. exp family practice/
73. exp physicians/
74. exp Physicians, family/
75. exp professional practice/
76. exp nurses/
77. exp nurse clinicians/
78. 67 or 63 or 53 or 71 or 70 or 68 or 48 or 77 or 72 or 65 or 55 or 74 or 50 or 75 or 64 or 57 or 61 or 51
or 58 or 69 or 52 or 59 or 60 or 49 or 56 or 73 or 66 or 76 or 62 or 54
79. 7 and 19 and 78 and 47
106
Public release of performance data in changing behaviour
Appendix 3. PsycINFO search strategy
(OVID)
Syntax guide
/ - index term (APA thesaurus)
exp - explode: includes narrower terms to the index term being exploded
.tw. - text word in title or abstract fields
$ - truncation/wild card: adds no or more characters
? - truncation/wild card: adds no or one character
# - truncation/wild card: retrieves alternative single character
adjx - adjacency: required words are adjacent to each other, or within x words of each other
.pt. - publication type
Description of search strategy
1. random$.tw.
2. multicenter study.tw.
3. randomized controlled trial.tw.
4. clinical trial.tw.
5. intervention studies.mp.
6. experiment$.tw.
7. (time adj series).tw.
8. (pre test or pretest or (posttest or post test)).tw.
9. random allocation.mp.
10. impact.tw.
11. intervention?.tw.
12. chang$.tw.
13. evaluation studies.mp.
14. evaluat$.tw.
15. effect?.tw.
16. compar$.tw.
17. comparative studies.mp.
18. 1 or 2 or 3 or 4 or 5 or 6 or 7 or 8 or 9 or 10 or 11 or 12 or 13 or 14 or 15 or 16 or 17
19. editorial.tw.
20. letter.tw.
21. comment.tw.
22. 21 or 19 or 20
23. animals.mp.
24. humans.mp.
25. 23 not 24
26. 22 or 25
27. 18 not 25
28. Consumer Satisfaction.mp.
29. patient preferences.ab,ti.
30. public reporting.tw.
31. consumer reports.ab,ti.
32. decision making.ab,ti.
33. choice behaviour.ab,ti.
34. 33 or 32 or 28 or 30 or 31 or 29
35. quality assurance, health care.mp.
36. benchmarking.mp.
37. process assessment.mp.
38. outcome assessment.mp.
39. Quality Indicators.mp.
40. performance outcome.ab,ti.
41. (quality adj2 indicator?).tw.
42. (quality adj (criteria or criterion or standard? or norm)).tw.
5
107
Chapter 5
43. (performance adj (indicator? or measure? or data or rating)).tw.
44. disclosure.mp.
45. Information Services.mp.
46. report card.ab,ti.
47. quality information.ab,ti.
48. public information.ab,ti.
49. consumer information.ab,ti.
50. patient information.ab,ti.
51. 35 or 50 or 39 or 40 or 36 or 41 or 48 or 47 or 38 or 42 or 49 or 46 or 45 or 37 or 43 or 44
52. 27 and 34 and 51
108
Public release of performance data in changing behaviour
Appendix 4. CINAHL search strategy
EBSCO
Syntax guide
MH - CINAHL subject heading
MM - CINAHL major subject heading
+ - explode: includes narrower terms to the index term being exploded
TI - word in the title field
AB - word in the abstract field
* - truncation/wild card: adds no or more characters
Nx - adjacency: required words are adjacent to each other, or within x words of each other
PT - publication type
Description of search strategy
1. MM “Clinical Trials”
2. TI control* or AB control*
3. TI random* or AB random*
4. MM “Comparative Studies”
5. TI experiment* OR AB experiment
6. TI time N2 series or AB time N2 series
7. TI impact OR AB impact
8. TI intervention* OR AB intervention*
9. Ti evaluat* OR AB evaluat*
10. TI effect? OR AB effect?*
11. “Pretest-Posttest Design+”
12. “quasi-experimental studies+”
13. or/1-12
14. SO cochrane database of systematic reviews
15. 13 not 14
16. MM “Quality of Health Care” or MM “Quality of Nursing Care”
17. MM “Benchmarking”
18. MM “Process Assessment (Health Care)”
19. MM “Outcome Assessment”
20. TI performance outcome OR AB performance outcome
21. TI quality information OR AB quality information
22. TI patient information OR AB patient information
23. TI consumer information OR AB consumer information
24. TI public information OR AB public information
25. TI public reporting OR AB public reporting
26. TI disclosure OR AB disclosure
27. MM “Quality of Health Care”
28. MM “Consumer Satisfaction”
29. TI patient preferences OR AB patient preferences
30. TI consumer reports OR AB consumer reports
31. TI decision making OR AB decision making
32. TI choice behaviour OR AB choice behaviour
33. Ti ’provider profiling’ or AB ‘provider profiling’
34. TI report card or AB report card
35. S16 or S17 or S18 or S19 or S20 or S21 or S22 or S23 or S24 or S25 or S26 or S34
36. S27 or S28 or S29 or S30 or S31 or S32 or S33
37. S15 and S35 and S36 (exclude Medline records)
5
109
Chapter 5
Appendix 5. DARE & CENTRAL search strategy
COCHRANE
(PrimaryHealthCare)OR (Hospitals)OR (Health Personnel)OR (physicians)OR (Nurses)OR (Professional
Practice/)OR (Physician‘s Practice Patterns) OR (Institution Practice) OR (Nurse Clinicians)OR
(Pharmacists) OR (Pharmacy) OR (Physiotherapist) OR (Midwife):ti,ab,kw, in Clinical Trials
AND
(Consumer Satisfaction/ OR patient preferencesOR public reporting OR consumer reportsOR decision
making OR choice behaviour OR choice behaviour OR exp “Patient Acceptance of Health Care”/ OR
’provider profiling’), in Clinical Trials
AND
(quality
assurance,
health
care/)OR(benchmarking)OR(Quality
Indicators,HealthCare/)
OR(disclosure/)OR(Information Services/) OR (report card.ab,ti.) OR (performance outcome.ab,ti.) OR
(“Outcome Assessment (Health Care)”/) in Clinical Trials, in Clinical Trials
110
Public release of performance data in changing behaviour
Appendix 6. Criteria for full-text screening
Nr:
Review author:
Yes
Design: randomised controlled trials, quasi-randomised
trials, controlled before-after studies, interrupted time
series
Types of participants: health care providers and
professionals, including organisations e.g. hospitals,
practice, patients, health care insurance companies,
health plans
Types of intervention: participant is exposed to
performance information (see*)
Process measures (e.g. waiting times)
Healthcare outcomes (e.g. mortality)
Structure measure (e.g. presence of waiting rooms)
Patient experiences (like CAHPS)
Expert of peer-assessed measures
Types of outcome measures:
Primary outcomes:
Consumer choice of healthcare provider (public and
patients)
Healthcare professional choice of healthcare provider
Purchasers choice of healthcare provider
Objective measures of provider performance, including
those that were made public and others that were not
Valid measures of staff morale or behaviour (“valid”
defined as having the development of the assessment
tool reported in a peer reviewed journal)
Secondary outcomes:
Awareness, attitude, views, knowledge in all target
groups
Costs
TEMPORARILY INCLUSION
No Doubt Comments
Crucial; score: ‘no’ or
‘doubt’ = exclusion
All participants are
important for us, at least
1 type should be central in
the study
There should be a
description of at least one
primary outcome. Only
one secondary outcome
measure is insufficient.
Hypothetical behaviour =
exclusion
111
5
Chapter 5
Appendix 7. Data collection form
Systematic review: the effectiveness of the public release of performance data in changing
consumer, healthcare professional or organisational behaviour
(Comments can be made either at the question itself or on a separate sheet (please specify question
number)
Name review author:
Date:
Article:
ID article:
Title:
Authors:
Source + year:
Article found in:
MEDLINE
EMBASE
CINAHL
PsycINFO
Cochrane
Other (please specify):
Unclear
Study period:
Study design:
The design of the study is (state which):
Controlled experimental: randomised controlled trial (RCT)
If the author(s) state explicitly (usually by some variant of the term ‘random’ to describe the
allocation procedure used) that the groups compared in the trial were established by random
allocation, then the trial is classified as a ‘RCT’ (randomised controlled trial).
Controlled experimental: controlled clinical trial (CCT) or quasi-randomised studies
If the author(s) do not state explicitly that the trial was randomised, but randomisation cannot be
ruled out, the report is classified as a ‘CCT’ (controlled clinical trial). The classification ‘CCT’ is where
the method of allocation is known but is not considered strictly random, and possibly quasi-randomised
trials. Examples of quasi-random methods of assignment include alternation, date of birth and medical
record number.
Controlled experimental: controlled before-after study (CBA)
Involvement of intervention and control groups other than by random process, and inclusion of baseline
period of assessment of main outcomes.
There are two minimum criteria for inclusion of CBAs in EPOC reviews: at least two intervention sites
and two control groups are chosen to be similar in respect of the main outcome measures at baseline.
Study and control sites are comparable with respect to dominant reimbursement
system, level of care, setting of care and/or academic status.
Uncontrolled observational: interrupted time series (ITS)
A change in trend attributable to the intervention. There are two minimum criteria for inclusion of ITS
designs in EPOC reviews: clearly defined point in time when the intervention occurred. At least three
data points before and three after the intervention.
Classification of study quality:
For all study designs
Quality criteria
N/A
a) The objective measurement of performance/provider behaviour or patient (health) outcomes
Done
Not clear
Not done
Page:
b) Relevant and interpretable data presented or obtained
Done
Not clear
Not done
Page:
112
Public release of performance data in changing behaviour
For RCT and CCT
Quality criteria
N/A
1. Concealment of allocation (protection against selection bias) (the unit of allocation was by
institution, team or professional and any random process is described explicitly, e.g. the use of random
number tables, OR the unit of allocation was by patient or episode of care and there was some form of
centralised randomisation scheme, an on-site computer or sealed opaque envelopes were used)
Done
Not clear
Not done
2. Follow-up of professionals (if outcome measures obtained for 80% to 100% of health professionals
randomised in the study, do not assume 100% follow-up unless stated explicitly)
Done
Not clear
Not done
3. Follow-up of patients (if outcome measures obtained for 80% to 100% of subjects randomised or for
patients who entered the trial, do not assume 100% follow-up unless stated explicitly)
Done
Not clear
Not done
Page:
4. Blinded assessment of primary outcome(s)∗ (if the authors state explicitly that the primary
outcome variables were assessed blindly OR the outcome variables are objective, e.g. length of hospital
stay, drug levels assessed by a standardised test, medical records used)
Done
Not clear
Not done
Page:
5. Baseline measurement (if performance or patient outcomes were measured prior to the
intervention, and no substantial differences were present across study groups)
Done
Not clear
Not done
N.A.
6. Reliable primary outcome measures (if two or more raters with at least 90% agreement or kappa $
0.8 OR the outcome is obtained form some automated system, e.g. length of hospital stay, drug levels
assessed by a standardised test)
Done
Not clear
Not done
Page:
7. Protection against contamination (if allocation was by community, institution or practice and it is
unlikely that the control group received the intervention)
Done
Not clear
Not done
Page:
∗ If it is a self administered questionnaire: than it is not blinded. Not clear: contact authors
For CBA
Quality criteria
N/A
1. Contemporaneous data collection (if data collection was conducted at the same time as pre and
postintervention periods for study and control activities or sites)
Done
Not clear
Not done
Page:
2. Baseline measurement (if performance or patient outcomes were measured prior to the
intervention, and no substantial differences were present across study groups)
Done
Not clear
Not done
Page:
3. Baseline characteristics are similar for two intervention sites &two control groups in respect of
the main outcome measures
Done
Not clear
Not done
Page:
i. Characteristics for studies using second site (if characteristics of study and control providers (or
patients) are reported and similar)
Page:
Done
Not clear
Not done
N.A.
ii. Characteristics for studies using untargeted activities as controls (if study and control activities are
comparable with respect to characteristics of targeted behaviour)
Page:
Done
Not clear
Not done
N.A.
iii. Characteristics for studies using patients as control (if characteristics of study and control
providers (or patients) are reported and similar)
Done
Not clear
Not done
Page:
N.A.
113
5
Chapter 5
Quality criteria
N/A
4. Reliable primary outcome measures (if two or more raters with at least 90% agreement or kappa $
0.8 OR the outcome is obtained form some automated system, e.g. length of hospital stay, drug levels
assessed by a standardised test)
Done
Not clear
Not done
Page:
5. Follow-up of professionals (if outcome measures obtained for 80% to 100% of health professionals
randomised in the study, do not assume 100% follow-up unless stated explicitly)
Done
Not clear
Not done
Page:
6. Follow-up of patients (if outcome measures obtained for 80% to 100% of subjects randomised or for
patients who entered the trial, do not assume 100% follow-up unless stated explicitly)
Done
Not clear
Not done
Page:
7. Protection against contamination (if allocation was by community, institution or practice and it is
unlikely that the control group received the intervention)
Done
Not clear
Not done
Page:
i. Characteristics for studies using second site (if characteristics of study and control providers (or
patients) are reported and similar)
Done
Not clear
Not done
Page:
ii. Characteristics for studies using untargeted activities as controls (if study and control activities are
comparable with respect to characteristics of targeted behaviour)
Done
Not clear
Not done
Page:
iii. Characteristics for studies using patients as control (if characteristics of study and control
providers (or patients) are reported and similar)
Done
Not clear
Not done
Page:
For ITS
Quality criteria
N/A
1. Clearly defined point in time when the intervention occurred.
Page:
Done
Not clear
Not done
N.A.
2. Protection against secular changes:
i. Intervention is independent of other changes (if the intervention occurred independent of other
changes in time)
Not done
Page:
Done
Not clear
ii. Sufficient data points to enable reliable statistical inference (if at least 3 data points are recorded
before and 3 data points recorded after the intervention)
Done
Not clear
Not done
Page:
iii. Formal test for trend (if formal test for change in trend using appropriate method is reported (e.g.
Cook & Campbell ’79)
Done
Not clear
Not done
Page:
3. Protection against detection bias:
iv. Data collection is identical before and after intervention (if reported that sources and methods of
data collection identical before and after intervention)
Not done
Page:
Done
Not clear
v. Intervention unlikely to affect/bias data collection ( if reported that the intervention unlikely to
affect data collection directly)
Done
Not clear
Not done
Page:
vi. Blinded assessment of primary outcome(s) (if the authors state explicitly that the primary outcome
variables were assessed blindly OR the outcome variables are objective, e.g. length of hospital stay,
drug levels assessed by standardised test)
Done
Not clear
Not done
Page:
4. Completeness of data set (if data set covers 80% to 100% of total providers and episodes of care in
study area)
Done
Not clear
Not done
Page:
5. Reliable primary outcome measure(s) (if two or more raters with at least 90% agreement or kappa $
0.8 OR the outcome is obtained from some automated system, e.g. length of hospital stay, drug levels
assessed by standardised test)
Done
Not clear
Not done
Page:
114
Public release of performance data in changing behaviour
Risk of bias tables of studies with a separate control group (RCTs, CCTs, CBAs)
Risk of bias Item
Judgement Description
Sequence generation
Y/N/?
Allocation concealment
Y/N/?
Baseline outcome measurements similar
Y/N/?
Baseline characteristics similar
Y/N/?
Incomplete outcome data
Y/N/?
Knowledge of the allocated interventions Y/N/?
adequately prevented during the study
Protection against contamination
Y/N/?
Selective outcome reporting
Y/N/?
Free of other bias
Y/N/?
Risk of bias tables of interrupted time series studies
Risk of bias Item
Judgement
Intervention independent of other changes
Y/N/?
Shape of the intervention effect pre-specified
Y/N/?
Intervention unlikely to affect data collection
Y/N/?
Knowledge of the allocated interventions Y/N/?
adequately prevented during the study
Incomplete outcome data adequately addressed
Y/N/?
Selective outcome reporting
Y/N/?
Free of other bias
Y/N/?
Description
Characteristics of study
Type of participants
Patients/consumers/providers/purchasers (Medicaid enrollees)
Page
Number
Patient/consumer/client
Gender
Age
Clinical problem
Other characteristics
patient
Male
Mixed
Client/consumer
Female
5
1.1.1.1
Hospitals
page
Number
Size
Other characteristics
1.1.1.2
Referring physicians
page
Number
Gender
Age
Clinical specialty
Referring to
Other characteristics
Primary care
Secondary and/or tertiary care
115
Chapter 5
Purchasers of health care: insurance companies
page
Private or state?
HMO
Collective
Company that buys care
from employees
Other
private
state
yes
nol
yes
no
yes
no
Setting of care
General practice
Outpatient clinic
Community care
Hospital/inpatient
Disabled/inpatient
Elderly
Any care setting
Other (please specify): Medicaid health plans
Country
North America, including USA and Canada
South America
Europe
Australia or New Zealand
Asia
Africa
Unclear/not specified
Type of control intervention
Usual setting
Other efforts on quality improvement
Type of intervention: participant is exposed to performance information
All based on actual data
Content of performance information
Process measures (e.g. waiting times)
Patient outcomes (e.g. mortality)
Structure measure (e.g. presence of waiting rooms)
Patient experiences (like CAHPS )
Expert or peer-assessed measures
Other, specify…(‘report cards’)
Description of intervention for both intervention and control groups
Intervention group:
Control group:
Duration of the intervention:
Duration of the intervention:
Intensity:
Intensity:
Duration of follow-up:
Duration of follow-up:
Not clear
Not clear
116
Public release of performance data in changing behaviour
Information dissemination
Intervention Page Control Page
Way of data presentation (numerical, graphical, pictorial summary, star
ratings...)
How is information made available to the participants
(personal mailing, journal article, active or passive dissemination)
Who disseminated the data?
Outcome measures
Selection:
Changes in the healthcare utilisation decisions of consumer (public and patients)
Changes in the healthcare utilisation decisions of healthcare professional
Changes in the healthcare utilisation decisions of purchasers
Specify:
Choosing the best health plans
Choosing the best healthcare provider (individual physician)
Choosing the best healthcare provider (organisation)
Referring to high quality care provider (individual physician)
Referring to high quality care provider (organisation)
Other…..
Changes in care:
Objective measures of provider performance, including those that were made public and others
that were not
Valid measures of staff morale or behaviour (’valid’ defined as having the development of the
assessment tool reported in a peer
reviewed journal)
Other (for example: number of quality improvement efforts):
.........................................................
Attitudes/knowledge/views/understanding/beliefs etc.:
Awareness of information (recall receiving & seen information)
Comprehension of quality of care information (do they understand
the information?)
Knowledge about quality of care (’know who is best’)
Believes regarding quality of care information (e.g. trust,
usefulness, appreciation)
Costs versus quality consideration
Other…………………………………..
Data analyses and results
How is outcome data collected?
Number of observations for primary outcome measure
Proportion of subjects of study who 100%
Not clear (information is not available
participate out of the total number in
the sampling frame (response rate)
Number of drop-outs
Not clear
Not applicable
Reason for drop-out mentioned?
yes
no
Not clear
N.A.
Confounder or case-mix correction
yes
no
Not clear
partially
applied?
If Yes: for which variables was corrected? (for example: hospital size)
Data analysis technique (s):
State the main results of the main outcome measure(s), for each group (pre- and post values;
intervention and control groups), in natural units (mean, SD, n)
State the corrected intervention effects (mean, 95% confidence interval, P values)
Describe the main study conclusion
117
5
Chapter 5
1.Were the conclusions made by the author(s) supported by
the data and/or the analysis reported in the article?
2.What of the following is applicable to this study?
a) Conclusions inconsistent with results
b) Conclusions go beyond the data
c) No evidence interpreted as no effect
d) Implications for research inconsistent with identified
shortcoming
3.Overall (1-2), how would you rate the methods used to
analyse the findings relative to the primary question addressed
in the study?
Comments
118
Yes
Yes
No
No
Can’t tell/partially
Can’t tell/partially
Major limitations
Moderate limitations
Minor limitations
Chapter
6
Comparative performance information plays no role
in the referral behaviour of GPs
Nicole Ketelaar
Marjan Faber
Glyn Elwyn
Gert Westert
Jozé Braspenning
BMC Family Practice 2014; 15(1): 146.
Chapter 6
ABSTRACT
Background: Comparative performance information (CPI) about the quality of hospital
care is information used to identify high-quality hospitals and providers. As the
gatekeeper to secondary care, the general practitioner (GP) can use CPI to reflect on
the pros and cons of the available options with the patient and choose a provider best
fitted to the patient’s needs. We investigated how GPs view their role in using CPI to
choose providers and support patients.
Method: We used a mixed-method, sequential, exploratory design to conduct
explorative interviews with 15 GPs about their referral routines, methods of referral
consideration, patient involvement, and the role of CPI. Then we quantified the
qualitative results by sending a survey questionnaire to 81 GPs affiliated with a
representative national research network.
Results: Seventy GPs (86% response rate) filled out the questionnaire. Most GPs did not
know where to find CPI (87%) and had never searched for it (94%). The GPs reported
that they were not motivated to use CPI due to doubts about its role as support
information, uncertainty about the effect of using CPI, lack of faith in better outcomes,
and uncertainty about CPI content and validity. Nonetheless, most GPs believed that
patients would like to be informed about quality-ofcare differences (62%), and about
half the GPs discussed quality-of-care differences with their patients (46%), though
these discussions were not based on CPI.
Conclusion: Decisions about referrals to hospital care are not based on CPI exchanges
during GP consultations. As a gatekeeper, the GP is in a good position to guide patients
through the enormous amount of quality information that is available. Nevertheless, it
is unclear how and whether the GP’s role in using information about quality of care in
the referral process can grow, as patients hardly ever initiate a discussion based on
CPI, though they seem to be increasingly more critical about differences in quality of
care. Future research should address the conditions needed to support GPs’ ability and
willingness to use CPI to guide their patients in the referral process.
120
Comparative performance information plays no role in the referral behaviour of GPs
BACKGROUND
As comparative performance information (CPI) about healthcare service, patient
experiences, and quality of clinical care becomes increasingly available, questions
about its use arise, as do questions about general practitioner (GP) views of CPI at the
time of referral. In healthcare systems where the GPs are the gatekeepers of secondary
care, which they are in the Netherlands and the UK, GPs refer their patients to
specialists for further examination, diagnosis, or treatment. In doing so, they play an
important intermediary role between patient and hospital.1-3 International studies have
shown that the referral is traditionally affected by previous experiences with
specialists, perceptions of specialists’ interactions with patients, office location,
specialists’ medical skills, and patient preferences.4-7 These traditional considerations
are all understandable. The current focus on the patient’s choice of healthcare
provider, with CPI for identifying the quality of provider performance8,9, calls for GPs
to reflect anew on the current referral process.
The patient’s involvement in choosing a healthcare provider in the Netherlands has
been encouraged since regulated competition was introduced during the 2006
healthcare system reform. Publicly available CPI introduced to encourage this
competition, contains information about the performance, quality of care and is
available for various providers10, also patient experiences plays an increasingly
important role. The information covers items at the hospital level (patient volumes,
inspection scores defined by the Dutch Health Care Inspectorate, which includes
specific conditions such as waiting lists, treatment volumes, treatment methods,
methods of anaesthesia, number of specialists treating a given condition, and patient
experiences11.
The CPI can make an impact when patients select providers of high-quality care on the
basis of this kind of information. However, patients hardly use such information for
selectively choosing a provider.12,13 Bringing such a choice into practice is a difficult
and complex task for patients; e.g. they do not know how to set their own values.14,15
The CPI can be difficult to interpret, especially when it contains conflicting criteria,
shows multiplicity formats, or the presentation makes it difficult to understand.14,16,17
Given the lack of CPI usage among patients for selectively choosing a provider, we are
looking for ways to provide additional support for the patients. Schlesinger and
colleagues advise providing advocates who can help patients with their choices of
hospital and who can act on their behalf if they have difficulties putting their choices
into practice.18 From the patient’s perspective, this advocate could be the GP. Patients
do not seem to search for CPI themselves, but they do ask for advice when choosing a
healthcare provider. The GP is an important advisor for about half the Dutch patients.19
because patients consider their GP to be a reliable source of quality information.20
121
6
Chapter 6
Dutch research confirms that GPs have significant influence in directing patients: 68%
of the patients who searched for information to select a hospital noted that they based
their final decision on GP advice.21
Several studies have revealed how providers respond to performance information.22-25 A
1996 study among cardiologists and cardiac surgeons shows that the publication of
report cards for grafts bypassing the coronary artery has little credibility and therefore
little influence on referral recommendations.25 A mixed group of physicians described
several issues that made them sceptical of the data and concerned about using the
information with patients.23 Further, it appears that quality-of-care data have little
impact on referral decisions.22 This paper addresses the following research questions:
Can the GP be a choice-supporting advocate for helping patients use comparative
performance information?
What are the current referral considerations?
What is the GP’s perception of patient involvement in referral decisions and the use
of comparative performance information?
What factors constrain GPs in using comparative performance information in the
referral process?
We conducted explorative interviews to review their referral routines in which we
included the current considerations, patient involvement, and the role of CPI in
referral decisions. Using the results of these interviews, we designed and conducted a
quantitative survey with a representative sample of Dutch GPs.
METHODS
Design
We used a mixed-method, sequential, exploratory design.26 In this design, the
qualitative element is considered first for exploring the research area, then the
quantitative element is used to extend and quantify the qualitative results.26 The
methods are integrated in three ways. First we focused on building, while the interview
results are used in the data collection to build the survey.27 The second way was
merging: we used both databases for analysis and comparison. Thirdly, we transformed
qualitative data to quantitative data, then integrated the results with illustrative
quotes28. A small part of the qualitative data was not transformed in the survey, though
it will be used in the results.
Participants
For the explorative interviews, we recruited GPs from the Nijmegen University Network
of General Practitioners and from a network of innovative primary care projects
financed by a Dutch healthcare insurance company. The resulting survey questionnaire,
122
Comparative performance information plays no role in the referral behaviour of GPs
designed to quantify the issues raised in these interviews, was distributed among a
sample of 81 primary care practices affiliated with a representative national network of
general practices. Participation was voluntary, and no incentives were offered. The
study has been carried out in the Netherlands in accordance with the applicable rules
concerning the review of research ethics committees and informed consent.
Explorative interviews
We conducted explorative interviews with 15 GPs that focussed on referral routines
related to three main issues, namely (1) referral considerations (which and why), (2)
patient involvement in the referral process in general, and (3) the role of comparative
performance information during referral in terms of knowledge about CPI, attitudes
towards it, and actual usage behaviour. The first author (NK) interviewed all the GPs.
The interviews lasted from 30 to 45 minutes. The interviews were audio-recorded and
transcribed verbatim. Two researchers (NK and MF) analysed the transcripts. First, they
read the interviews to obtain a comprehensive impression of the material. Second, the
data were extensively and inductively coded. Indexing the data created a large number
of codes that were repeatedly refined and reduced in several rounds.29 Third, with
regard to the role of comparative performance information, a framework analysis was
used to approach the data deductively. Cabana and colleagues developed the general
guidelines for this framework for improvement.30 The use of CPI and the use of
professional guidelines differ, though there is a resemblance in the way they both could
be implemented. The successive steps helped us analyse the interviews, and we used
them as a guide to present the results in the section ‘Use of comparative performance
information’. Figure 1 shows the findings for this part of the interviews. We used the
analysis software Atlas.ti.5.2 to facilitate the coding process.31
Survey
We built on the issues that arose in the interviews and transformed them into a survey
questionnaire. The CPIrelated questions were added to the annual survey of the Dutch
National Information Network of General Practice (LINH). The LINH has a nationally
representative database maintaining longitudinal data derived from patients’ electronic
medical records about consultations, morbidity, drug prescriptions, and referrals.32 The
LINH consists of 81 general practices with approximately 335,000 patients. The data
were collected between September and December 2012. The survey focused on GP
considerations in the current referral process, GPs’ views towards patient involvement,
GPs’ current experiences with patient involvement, and the role of comparative
performance information in referral decisions. The seven items in GP considerations in
current referrals were patient preferences, experiences of other patients, waiting lists,
quality of care, specific treatment or techniques, patient travelling distance, and
personal contact with a specialist. The question ‘to what extent did the GP consider
123
6
Chapter 6
these items in the decision to refer a patient’ was to be answered on a five-point Likert
scale ranging from 1 (never) to 5 (always).
Five points for patient involvement were formulated: (1) to what extent did GPs agree
with patients’ needs for information about quality differences, (2) how often did
patients refer to comparative performance information during consultations, (3) what
were GPs’ evaluations of patients’ use of quality information about hospitals, (4) how
did GPs perceive the ability of patients to decide on a hospital themselves, and (5)
what about the GPs’ view that the use of CPI is the patient’s own responsibility? A fivepoint rating scale ranging from 1 (totally disagree) to 5 (totally agree) was used.
The main part was about the role of comparative performance information in referral
decisions (21 items). The findings of Figure 1 about knowledge, attitude, and behaviour
were listed in the items. The GPs were asked if they knew where to find information
about quality of care, and whether they searched for CPI (both dichotomous variables).
Ten statements about comparative performance information, containing the elements
of attitude and behaviour, were developed. They included elements of quality-of-care
differences between hospitals, GP use of comparative performance information to
select a hospital, the GPs’ role and responsibility regarding the use of comparative
performance information, effects of CPI on the continuity of care, time management
regarding comparative performance information, and GP views of the use of CPI in the
next 5 years. For the descriptive analyses, the ratings of these items were transformed
into the percentages of GPs who agreed (rating of 4 or 5 on the Likert scale). Finally,
GPs were asked to express their opinions about the currently available CPI with respect
to credibility, transparency about how information was gained, contradictory sources,
user friendliness, is it up to date, comprehensibility, the ability of CPI to show quality
of care differences, connection to patients’ wishes, and the information content. A
five-point scale ranging from 1 (totally disagree) to 5 (totally agree) was used; ‘Do not
know’ was a separate option in a separate column. We gathered general practice
characteristics about the locality of the practice (urbanization), number of patients in
the practice, and the practice type (single or group practice). Background information
about the GP included gender and the number of days a week the GP was available in
the practice (part time or full time). The data are presented in terms of means (s.d.) or
percentages (%).
RESULTS
The response rate of the survey was 86%. Table 1 presents the characteristics.
124
Comparative performance information plays no role in the referral behaviour of GPs
Table 1. Characteristics of the 70 participating general practitioners and their practices
GP characteristics
Male
Full-time GP
Practice characteristics
Single-handed practice
Duo practice
Group practice
Healthcare centre
Urbanization level
Very high
High
Moderate
Low
Rural
Number
%
55
66
80
52
38
12
12
8
54
17
17
12
20
11
14
14
11
29
16
20
20
16
Current referral considerations
In the interviews, GPs spoke in great detail about their preference for using their own
prior experiences and personal contacts with specialists or hospitals when considering
their referral. Personal contacts were important to the GPs because they provide an
opportunity to ask medical questions and to estimate colleagues’ interactions with
patients. In contrast, the surveyed GPs stated that, in deciding about a referral, they
primarily considered patients preferences for a hospital or a provider, then the quality
of care, and then the distance from the patient’s home (Table 2).
‘I mainly refer on personal grounds and experiences. This might be a really
bad thing to do. Still, I think it works this way.’ (N 1)
Table 2. The importance of factors in the referral process for selecting a hospital or a specialist
Mean (SD) of the 70 responses
Patients’ preferences for a hospital or provider
4.3 (0.7)
Quality of care
3.8 (1.0)
Patient’s travel distance to a hospital or provider
3.8 (0.9)
GP’s personal contact with a specialist
3.6 (0.9)
Waiting list
3.5 (0.8)
Specific treatment or techniques
3.5 (0.8)
Experiences of other patients
3.4 (1.0)
The responses were given on a five-point Likert scale, with ‘5’ representing ‘always taken into
consideration’, and ‘1’ as ‘never taken into consideration’. Sd: Standard deviation
Patient involvement
During the interviews, the GPs said that they always started by asking what the patient
wanted, and they repeatedly highlighted the fact that when selecting a healthcare
provider, patients valued other choice attributes than those reflected in CPI. The GPs
125
6
Chapter 6
also noted that comparing providers and making a rational trade-off based on CPI are
difficult tasks for patients.
‘Familiarity with a hospital, distance, and knowing where to go: these are
much stronger arguments for the patient than quality of care.’ (N 10)
‘The mortality rates in hospital A are better than in hospital B, but A is a
generic hospital while B is a top clinical one with an intensive care unit. For
that reason, there is a greater chance that people will die in hospital B. The
mortality rates show you the data, but you need to interpret them with the
background information in your head. As a doctor I can do that, but
patients?’ (N 6)
There was a high level of agreement between the surveyed GPs about the importance
of patients making their own hospital choice and needing to be informed about
differences in quality of care (for both items, M=4.0; SD=0.8). There was less
agreement about the statements that quality information about hospitals has added
value for patients (M=3.0; SD=0.8) and that the use of CPI is a patient’s own
responsibility (M=3.0; SD=0.7). There was little GP agreement about how often patients
refer to CPI during consultations (M=2.3; SD=0.9). Approximately half the GPs (47%)
agreed that they were ‘sceptical about patient use of CPI’ (M=3.0; SD=0.9).
Use of CPI
Knowledge
Most GPs (83%) reported that they did not know where to find performance information
to compare their regional hospitals.
Attitude: motives
The GPs varied widely in their attitude towards CPI. We distinguished four motives that
shaped GPs’ negative attitudes towards the use of CPI. First, the GPs want the best
care for their patients, and they doubt the role that CPI plays in supporting referral
choices. They suggested that they would like to have a ‘tailored’ referral process
because the extent of patient involvement in choosing a hospital varies. About half of
the surveyed GPs (47%) agreed that they have a task in supporting patients’ hospital
choice based on CPI. They recognized CPI as a type of information that can facilitate
patient involvement in choosing a hospital.
‘Patients want to steer their decisions.... When they are old and weak they
say,‘Put me in the back seat and drive me to the nearest hospital.’ Other
patients say,‘Sit next to me and tell me how, but I am the one who’s
126
Comparative performance information plays no role in the referral behaviour of GPs
driving.’ And there is a group of patients who ask,‘Where is the highway to
the best specialist in this area?’ (N 14)
A second motive reported in the interviews was that the use of CPI interferes with the
GP’s professional role. The GPs felt responsible for an optimal referral and emphasized
their role as coordinators. They expected patients to rely on their referral advice.
Therefore, they needed to adjust to the idea that patients can now propose
alternatives. They were afraid that patients could decide to go to hospitals outside
their professional network as a result of CPI. This could increase the number of medical
specialists GPs have to deal with and thereby impede communication since it is easier
to talk to someone you know. The GPs also greatly valued patients’ anecdotal reports,
so if the size of the professional network were to increase, these patient reports would
become more difficult to interpret.
‘When people came up with propositions to go elsewhere, I was unprepared.
The way I was educated to deal with my profession as a GP - it was all about
personal contact with specialists and providers, not about the arguments of
performance information gathered elsewhere or waiting lists.’ (N 13)
As a third motive in the interviews, the GPs expressed concerns regarding unintended
consequences of using CPI during the referral process. This might limit the accessibility
of certain types of care for those patients who could not easily choose to go elsewhere.
Almost half the GPs surveyed (46%) agreed that referral based on CPI could increase
fragmentation of care and threaten the continuity of care.
‘You can be treated in many hospitals for all kinds of things, but you need
to have some sort of continuity in your treatment, which often means you
end up in the same hospital.’ (N 15)
Fourth, during the interviews several GPs said that it would take too much time to
remain up to date about the CPI for multiple conditions and for a range of patient
groups, even though only a minority (23%) of surveyed GPs agreed that ‘the use of CPI
takes too much time’.
Attitude: outcome expectancy
The GPs had doubts about the outcome expectancy for CPI. In the interviews, some
questioned whether using CPI would lead to an improvement in quality of care. The
surveyed GPs were also divided in their opinions about the differences in quality of care
in hospitals: 66% disagreed with the statement that the quality of care varies greatly
between hospitals.
127
6
Chapter 6
Attitude: self-efficacy
During the interviews, some GPs said they were unsure whether they could interpret
CPI information and make a trade-off based on all the available information. The
results in Table 3 show that it is difficult for GPs to interpret CPI information.
‘It is really difficult to assess the quality of care provided by my colleagues,
and they are in the same building! Not to mention colleagues elsewhere or
specialists in the hospital. It is almost impossible to make good judgements
about that.’ (N 8)
Lack of agreement with content and validity elements
The GPs had their doubts regarding both the content itself and the validity of CPI
(Table 3). A fairly large proportion of the GPs said ‘I do not know’ when they were
asked about various CPI elements.
In the interviews, some GPs mentioned conflicts of interest. They felt that the sources
on which CPI is based should include disclaimers about various aspects of data
collection and validity, and that the sources should declare any conflicts of interest.
‘I do not know about using CPI for referral decisions, but I get the feeling
that I’m promoting a particular hospital.’ (N 9)
Table 3. General practitioners’ level of agreement about statements concerning outcome
expectancy, content, and validity of comparative performance information
Statements
Totally agree/ disagree Do not know
Mean (SD)
%
CPI is not transparent about how information is determined
3.6 (0.7)
18
CPI is not clear because of contradictory sources
3.6 (0.7)
20
20
CPI is not credible
3.2 (0.7)
27
CPI is not in line with patients’ wishes
3.2 (0.6)
21
CPI is difficult for patients to understand
3.3 (0.7)
21
CPI is not specific enough
3.3 (0.7)
3.3 (0.7)
23
CPI is not user friendly
CPI gives the wrong choice attributes
3.2 (0.6)
20
28
CPI is not up to date
3.1 (0.6)
3.0 (0.6)
20
CPI has no ability to show differences in quality of care
The 68 responses were given on a five-point Likert scale, with ‘5’ representing ‘Totally agree’ and ‘1’
representing ‘Totally disagree’. ‘Do not know’, was a separate sixth answer possibility.
SD: Standard deviation
Behaviour
Most of the GPs (94%) declared that they had never searched for hospital performance
information in their region; however, 12% reported that they had used CPI for selecting
128
Comparative performance information plays no role in the referral behaviour of GPs
a hospital. Further, approximately half the GPs agreed that they had discussed qualityof-care differences between hospitals with their patients (M=3.2; SD=0.8). The GPs
were undecided regarding the expectation that CPI will becomes a part of their referral
advice within 5 years – only a minority agreed with this statement (M=3.0; SD=0.8).
Interviewer: ‘Do you believe that patients will make more informed choices
in the future?’ GP: ‘Frankly? No, I do not think so.’ (N 4)
Environmental factors
A lack of reimbursement, time to search for CPI, the small number of hospitals to
choose from, and not having an electronic referral system containing CPI were
mentioned in the interviews. About half the surveyed GPs (48%) noted that they lacked
an electronic system to help them use CPI in the referral process.
DISCUSSION
Can the GP be a choice-supportive advocate for patients to overcome patients’ lack of
CPI usage? Our study shows that we cannot expect that a GP can play an advocate’s
role in the use of CPI. Currently, GP considerations at the point of referral are patient
preferences, quality of care, and travel distance, and there is no role for CPI as an
additional source. The GPs feel that patients should become more aware of quality
differences in general. They do not believe that current CPI has any added value for
patients. The GPs rarely see patients initiating a discussion about CPI during
consultations, and most are sceptical about the ability of patients to use CPI. The GPs’
own use of CPI is hindered by several barriers, including indecisiveness about their role
in supporting patients’ choices and their task in addressing CPI during consultations.
Comparison with existing literature
The healthcare reforms in north-western European countries have been designed to
encourage a greater role for patients in choosing a provider and to spur providers on to
support this choice. The purpose of this design is to increase the competition between
providers for the benefit of the patients.33,34 Our results show that current practice
does not yet support the concept of GPs acting as agents of patient choice and users of
CPI. The GPs in our study rarely had patients who mentioned CPI during a consultation.
To decide on a referral, the GPs focus on patient preferences informal sources (e.g.
connections with specialists), their own previous experiences, and hospital distance
from the patient’s home. Our study confirms various findings from the UK, Denmark,
and the Netherlands.24,35-37
The GPs feel responsible for coordinating care for their patients, but see no need for
using CPI during the referral process. This is partly due to not knowing where to find
129
6
Chapter 6
CPI, but there is also some ignorance regarding the content and outcome expectancy
for CPI. This ignorance may influence the ratings they gave. In another study, GPs did
not view CPI as a source of information.37 A precondition for this kind of CPI usage is
the reliability of the information and its sources. As in other studies, our GPs reported
distrust of the content and validity of CPI.16,23,25 As long as GPs do not trust CPI, other
sources of information will remain more important in their referral considerations.
Because CPI makes clear statements on quality differences and providers of highquality care, it can be seen as a powerful source of information for GPs in selectively
choosing a provider as well as in being the patient’s advocate while interpreting and
discussing the available data. Regarding selective choice of a provider, a recent Dutch
study showed that GP referral patterns were unaffected by report cards, with the
exception of outcome indicators for breast cancer.38 Thus, even if CPI highlights
differences in the quality of care, GP referral decisions are not, or are hardly,
affected. Consequently, the intended impact of CPI in enabling a selective choice of a
provider is not achieved.
Regarding the GP’s role as an advocate, it seems that patients hardly ever introduced
CPI. On the basis of our results, we can question whether the GP would use CPI if the
patient suggested it. A lack of knowledge and a certain unwillingness both seem to
contribute to the GP’s not using this kind of information during the referral process. A
recent study37 suggests an interaction between the GP’s use of CPI and patients’ use of
publicly available CPI in the decision-making discussion about referral with their GPs.
Hence, if patients were to approach their GPs with publicly available information about
quality more often, their GPs would be more likely to have consulted CPI themselves.
However, because the patients hardly use CPI, and GPs do not either, the status quo
continues. A UK study has shown that none of their participating GPs initiated a
discussion of differences between services with patients.18 Approximately half the GPs
in our study said that they discussed quality of care differences with their patients.
Given their responses, we see that these discussions are not based on publicity
available CPI. It may be that patients, despite their not using CPI, may become
increasingly critical about differences in quality of care. Ensuring that care quality
becomes an issue in the patient consultation can be considered a ‘tipping point’ in the
path towards the use of CPI in the referral process.
Despite the GPs’ restraint towards CPI, leaving the choice of provider in the hands of
the patient alone worried some GPs as well. In relation to the coordinator role, the GPs
in our study feared a further fragmentation of care, as patients might, as a result of
CPI, choose providers outside the reach of their professional network. This reasoning
has been described in another study as well39, and it makes sense because it is difficult
130
Comparative performance information plays no role in the referral behaviour of GPs
to predict how and in which cases the benefits of using CPI and the choice of highquality providers outweigh the threats to continuity of care. New in our study is that
GPs link this concern to their own professional role and to the potential weakening of
their professional network – they like to keep an overview in their role as the
coordinators in the Dutch ‘gatekeeper’ system.
Implications
Our study has various implications:
GPs should discuss whether and how to act as supportive agents for their patients
using CPI in a way that does justice to their feelings of responsibility, concerns, and
practical conditions
Education of GPs about CPI, its measures, the methodology on which information is
based, and the possible better outcomes, as well as teaching them how to discuss
CPI with patients
CPI should be publicized and made available to GPs so that they become aware of
the information, can access it easily, and recognize practice variation between
hospitals
Time is required to improve patient engagement in referral discussions (e.g. longer
consultations).
Strengths and limitations
One of this study’s strengths is the use of both qualitative and quantitative data. The
GPs interviewed came from innovative and frontrunner general practices. Even though
this might have affected the interview results, the participants were drawn from a
representative sample of Dutch general practices.40 We therefore used the survey
results to draw a picture of how Dutch GPs use CPI, while the interview results were
used mainly to illustrate the quantitative findings. A limitation was the number of CPIrelated questions that could be added to the annual LINH survey. Therefore, not every
item highlighted in the interviews could be added to the survey in order to quantify our
findings. We focussed on the barriers that the GPs encountered without explicitly
discussing facilitating factors. The GPs noted their intention to act in the patient’s best
interests in the referral considerations, the importance of the free choice of provider
for patients, and the discussion of quality of care with patients, though none of these
factors included facilitators for the use of CPI.
Conclusions
General practitioners play a key role in referring patients to hospital care. Their
decisions about referrals to hospital care are not based on systematically collected CPI
because other referral considerations are more important. CPI is assumed to be an
important factor in selective-referral behaviour, as is supporting the patient’s ability to
131
6
Chapter 6
choose a provider of high-quality care by offering more transparency. Despite policy
measures that encourage selectively choosing a provider and the expectations that both
patients and GPs will make an active and informed choice based on the increasing
availability of CPI, both are in a preliminary phase of using this data. Whether and how
the GP’s roles in CPI use and patient support should be actively stimulated and
supported is still to be determined.
132
Comparative performance information plays no role in the referral behaviour of GPs
REFERENCES
1.
2.
3.
4.
5.
6.
7.
8.
9.
10.
11.
12.
13.
14.
15.
16.
17.
18.
19.
20.
21.
22.
23.
Akbari A, Mayhew A, Al-Alawi MA, Grimshaw J, Winkens R, Glidewell E, et al. Interventions to
improve outpatient referrals from primary care to secondary care. Cochrane Database Syst Rev
2008;(4):CD005471.
Berendsen AJ, de Jong GM, Meyboom-de Jong B, Dekker JH, Schuling J. Transition of care:
experiences and preferences of patients across the primary/secondary interface - a qualitative
study. BMC Health Serv Res 2009;9:62.
Dixon A, Robertson R, Bal R. The experience of implementing choice at point of referral: a
comparison of the Netherlands and England. Health Econ Policy Law 2010;5(3):295–317.
Barnett ML, Keating NL, Christakis NA, O’Malley AJ, Landon BE. Reasons for choice of referral
physician among primary care and specialist physicians. J Gen Intern Med 2012;27(5):506–12.
Forrest CB, Nutting PA, von Schrader S, Rohde C, Starfield B. Primary care physician specialty
referral decision making: patient, physician, and health care system determinants. Med Decis
Making 2006;26(1):76–85.
Kinchen KS, Cooper LA, Levine D, Wang NY, Powe NR. Referral of patients to specialists: factors
affecting choice of specialist by primary care physicians. Ann Fam Med 2004;2(3):245–52.
Piterman L, Koritsas S. Part II. General practitioner-specialist referral process. Intern Med J 2005;
35(8):491–496.
Fung CH, Lim YW, Mattke S, Damberg C, Shekelle PG. Systematic review: the evidence that
publishing patient care performance data improves quality of care. Ann Intern Med 2008;
148(2):111–123.
Wang J, Hockenberry J, Chou SY, Yang M. Do bad report cards have consequences? Impacts of
publicly reported provider quality information on the CABG market in Pennsylvania. J Health Econ
2011;30(2):392–407.
Marshall MN, Romano PS, Davies HT. How do we maximize the impact of the public reporting of
quality of care? Int J Qual Health Care 2004;16(Suppl 1):i57–i63.
Zwijnenberg NC, Damman OC, Spreeuwenberg P, Hendriks M, Rademakers JJ. Different patient
subgroup, different ranking? Which quality indicators do patients find important when choosing a
hospital for hip- or knee arthroplasty? BMC Health Serv Res 2011;11:299.
de Groot IB, Otten W, Smeets HJ, Mharang-van de Mheen PJ, The Choice-2 study group. Is the
impact of hospital performance data greater in patients who have compared hospitals? BMC Health
Serv Res 2011;11:214.
Rothberg MB, Benjamin EM, Lindenauer PK. Public reporting of hospital quality: recommendations
to benefit patients and hospitals. J Hosp Med 2009;4(9):541–45.
Ketelaar NABM, Faber MJ, Westert GP, Elwyn G, Braspenning JC. Exploring consumer values of
comparative performance information for hospital choice. Quality in Primary Care 2014;22(2):81–9.
Lichtenstein S, Slovic P: The Construction of Preference. 1st edition. Cambridge: Cambridge
University Press, 2006.
Fasolo B, Reutskaja E, Dixon A, Boyce T. Helping patients choose: how to improve the design of
comparative scorecards of hospital quality. Patient Educ Couns 2010;78(3):344–49.
Hibbard JH, Peters E. Supporting informed consumer health care decisions: data presentation
approaches that facilitate the use of information in choice. Annu Rev Public Health 2003;24:413–
33.
Schlesinger M. Choice cuts: parsing policymakers’ pursuit of patient empowerment from an
individual perspective. Health Econ Policy Law 2010;5(3):365–87.
Van der Geest SA, Varkevisser M. Zorgconsumenten en kwaliteitsinformatie [In Dutch]. Economisch
Statistische Berichten (ESB) 2012;97(4631):174–75.
Alexander JA, Hearld LR, Hasnain-Wynia R, Christianson JB, Martsolf GR. Consumer trust in sources
of physician quality information. Med Care Res Rev 2011;68(4):421–40.
Reitsma M, Brabers A, Masman W, de Jong J. The Choosing Citizen [In Dutch: De kiezende burger].
Nivel: Utrecht; 2012.
Audet AM, Doty MM, Shamasdin J, Schoenbaum SC. Physicians’ Views on Quality of Care: Findings
from the Commonwealth Fund National Survey of Physicians and Quality of Care. New York: The
Commonwealth Fund, 2005.
Barr JK, Bernard SL, Sofaer S, Giannotti TE, Lenfestey NF, Miranda DJ. Physicians’ views on public
reporting of hospital quality data. Med Care Res Rev 2008;65(6):655–73.
133
6
Chapter 6
24.
25.
26.
27.
28.
29.
30.
31.
32.
33.
34.
35.
36.
37.
38.
39.
40.
134
Rozen R, Florin D, Hutt R. An Anatomy of GP Referral Decisions. A Qualitative Study of GP’s View
on their Role in Supporting Patient Choice. London: The King’s Fund, 2007.
Schneider EC, Epstein AM. Influence of cardiac-surgery performance reports on referral practices
and access to care. A survey of cardiovascular specialists. N Engl J Med 1996;335(4):251–56.
Bishop FL, Holmes MM. Mixed methods in CAM research: a systematic review of studies published in
2012. Evid Based Complement Alternat Med 2013;2013:187365.
Creswell JW, Klassen AC, Plano Clark VL, Smith KC, Assistance WG. Best Practices for Mixed
Methods Research in the Health Sciences. Washington, DC: Office of Behavioral and Social Sciences
Research (OBSSR), National Institutes ofHealth (NIH), 2011.
Fetters MD, Curry LA, Creswell JW. Achieving integration in mixed methods designs-principles and
practices. Health Serv Res 2013;48(6 Pt 2):2134–56.
Pope C, Ziebland S, Mays N. Qualitative research in health care. Analysing qualitative data. BMJ
2000; 320(7227):114–16.
Cabana MD, Rand CS, Powe NR, Wu AW, Wilson MH, Abboud PA, et al. Why don’t physicians follow
clinical practice guidelines? A framework for improvement. JAMA 1999; 282(15):1458–65.
Sheon N. Overview of Atlas.ti.5.2. San Franciso: University of California, 2007.
Dutch Information Network of General Practice: Dutch Information Network of General Practice
(LINH). www.nivel.nl [20-7-2014].
Botje D, Klazinga NS, Wagner C. To what degree is the governance of Dutch hospitals orientated
towards quality in care? Does this really affect performance? Health Policy 2013;113(1–2):134–41.
Victoor A, Delnoij DM, Friele RD, Rademakers JJ. Determinants of patient choice of healthcare
providers: a scoping review. BMC Health Serv Res 2012;12:272.
Birk HO, Henriksen LO. Which factors decided general practitioners’ choice of hospital on behalf of
their patients in an area with free choice of public hospital? A questionnaire study. BMC Health
Serv Res 2012;12:126.
Victoor A, Noordman J, Sonderkamp JA, Delnoij DM, Friele RD, van Dulmen S, et al. Are patients’
preferences regarding the place of treatment heard and addressed at the point of referral: an
exploratory study based on observations of GP-patient consultations. BMC Fam Pract 2013;14:189.
Doering N, Maarse H. The use of publicly available quality information when choosing a hospital or
health-care provider: the role of the GP. Health Expect 2014; DOI:10.1111/hex.12187.
Ikkersheim D, Koolman X. The use of quality information by general practitioners: does it alter
choices? A randomized clustered study. BMC Fam Pract 2013;14:95.
Coulter A. Do patients want a choice and does it work? BMJ 2010;341:c4989.
Braspenning JC, Wichers L, Faber MJ. “The running of LINH practices is typical of Dutch practices
in general”. [In Dutch: Praktijkvoering LINH-praktijken representatief]. Huisarts Wet [In Dutch]
2007;50(4):230-31.
Chapter
7
General discussion
Chapter 7
This thesis focuses on the conditions that affect healthcare consumers’ search for and
use of comparative performance information (CPI). Healthcare consumers that are
often imperfectly informed and make choices that are inconsistent with their
preferences. CPI has been developed and introduced to help consumers choose a
healthcare provider that matches their needs and preferences. Such information can be
simultaneously used to drive healthcare providers to improve the quality of care. We
address three themes in this thesis: search triggers, choice behaviour, and professional
guidance.
Figure 1.
Three themes in this thesis
search
triggers
choice
behaviour
professional
guidance
The research questions are:
What are the barriers and facilitators for healthcare consumers to search for and
use CPI?
To what extent does publicly released performance information change the choice
behaviour of healthcare consumers, providers, and purchasers?
To what extent do GPs advocate choice and encourage patients to choose providers
on the basis of CPI?
SEARCH TRIGGERS
Main findings
Consumers and their choice of healthcare providers (Chapter 2)
Most consumers are unfamiliar with CPI.
After CPI has been introduced, consumers indicate that they value other sources
more than CPI, e.g. personal experience.
136
General discussion
Consumers doubt the reliability of CPI because the information from different
sources is often inconsistent.
Patients choosing a hospital for a primary hip or knee surgery (Chapter 3)
A minority of patients search for CPI.
Patients who expect differences in quality of care are significantly more likely to
search for CPI than those who do not expect such differences.
Patients expect quality differences (mainly in general hospital characteristics such as
reputation, distance, and accessibility) rather than treatment differences (in hip
and knee surgery, for example).
Referral of patients with Parkinson disease to a physiotherapist (Chapter 4)
Most participants with Parkinson disease take the referring provider’s advice, though
this advice is rarely accompanied by supporting arguments or data.
The willingness to switch to an expert physiotherapist increases significantly when
patients recognize the added value of the specific expertise.
About half of the patients consider their own physiotherapist to be an expert, which
probably reduces their motivation to search for CPI.
Only a minority of the healthcare consumers search for CPI about hospitals or
healthcare providers.1-6 The literature reports some factors that do not stimulate a
search for CPI; some of these factors can be addressed with an adequate intervention,
while others are more or less fixed. Modifiable factors include the lack of awareness
that CPI is available7-13 and the availability of an acceptable alternative provider in the
proximity of one’s community. Fixed factors include the characteristic that women are
more prone to search than men and the fact that a long-term health condition
correlates with more consumer searching.6
The results of this thesis highlight three triggers that affect searching for CPI: the value
of CPI versus the importance of other sources, consumers’ expectations of variation of
care quality, and consumers’ recognition of expertise differences among providers.
These three search triggers can be helpful in understanding why CPI is underused.
Other studies that sought reasons for the lack of CPI usage focus on how CPI is
presented or what the content of CPI should be from the consumer’s point of view. In
this thesis, the focus is on an earlier phase, namely the search triggers. Knowing that
consumers have an interest in choosing the best doctor makes one wonder why CPI is
used so little.
The value of CPI versus the importance of other sources
As consumers hardly value CPI, they likely feel no need to search for it (Chapter 2).
People are inclined to ignore less meaningful information.14 In choosing healthcare,
137
7
Chapter 7
consumers seem to value other sources more than CPI. Existing research already states
that the impact of informal views of quality of care views of family members and
friends as well as familiar authorities such as the GP, and local reputation counts more
for most people than CPI does.15-21 The consumers use often their own personal
experience to choose a healthcare provider, and also habit plays an obvious role in
this.13,22-24 A new information source such as CPI should be perceived as a source with
added value. New in our focus group study is the extent of and insight into many of the
consumers’ underlying values and principles. Our findings suggest that the consumers
were not really interested in CPI, and the currently formulated CPI was too abstract
and not very relevant, so it seemed invaluable for them. It is debatable whether and
how CPI can be become a source alongside other sources that is meaningful. Some
studies illustrate the power of anecdotal narratives and suggest that the CPI should be
given a more story-telling form25,26, in which case the information would reflect more
of the information from the informal network. Then it would be richer in meaning and
emotional nuance.27 There is a growing interest in testing how to collect and insert
these narratives into healthcare providers’ CPI as well as how to display this material
on websites.13 Consumers are apparently only minimally aware that CPI exists, and CPI
as currently formulated seems to be irrelevant to them.
Consumers’ expectations about differences in care quality
Healthcare consumers search more for CPI when they expect to find variations in care
quality (Chapter 3). Our study is one of the first that shows how patients’ expectations
of quality of care affect the search for CPI. The finding is consistent with two other
studies that suggest that a key component in public reporting is showing consumers that
variation in quality is meaningful.28,29 So far, patients’ expectations of healthcare and
variation in quality of care seem to be underexposed in public reporting. We found that
the quality differences patients expected were oriented less toward the treatmentrelated performance indicators and more toward general issues such as reputation,
distance, and accessibility. Thus, the trigger to search could become stronger if the
patient were more aware of differences in treatment quality. The previous examination
of consumers’ interest in different types of indicators produced mixed results. Some
report that consumers are far more likely to focus on technical aspects of care than on
interpersonal skills and services30, while others suggest the opposite pattern31, or no
pattern at all; in short, all aspects seem equally important.7 Perhaps the clinical setting
or the specific condition as suggested in a narrative review of the concept and
measurement of patient expectations of quality32 could explain some of these
differences. The authors also question the robustness of the measurement and plead
for more research into the measurement itself. Although we do need more research
into the measurement and the determinants of consumers expectations about
138
General discussion
differences in quality of care, awareness of quality differences is a trigger for the
search for quality information.
Consumers’ recognition of expertise among providers
An interesting finding of our study of patients with Parkinson’s disease is that the
recognition of the value of an expert physiotherapist motivated some patients to search
for CPI (Chapter 4). As described in Chapters 2 and 3, most patients did not use CPI to
choose a healthcare provider, or in this study, specifically a physiotherapist. Chapter 3
reveals that expert awareness, just as knowledge, of quality differences motivates
consumers to search for quality information. This awareness also seems to lead to
actual behaviour. Although recognizing the value of an expert physiotherapist increased
their willingness to switch physiotherapists, half the patients still wanted to continue
the visits to their own current physiotherapist. Moreover, they characterized their own
physiotherapist as an expert. The phenomenon is linked to the consideration presented
elsewhere that your ‘own’ hospital or provider is the best.5,33 It is unlikely and
unrealistic that patients think critically about possible mistakes in ‘their’ hospital.17
Even when they are confronted with quality differences in healthcare services, being
aware of that motivated only half the patients with Parkinson disease to search for CPI.
The studies of search triggers show that awareness of quality differences and
availability of expertise can be a trigger for consumers to search for CPI, but when they
were confronted with the currently available CPI, they were sceptical. The consumers
see current CPI as rather abstract and irrelevant (low value), so they prefer more local
patterns and use reputation to choose between healthcare services. To increase its
value, CPI should be adjusted more towards the needs of the consumers. In exploring
this direction, the following finding should be taken into account. The study in Chapter
2 reveals that becoming doubtful about the quality of certain aspects of care that
consumers originally thought was good causes anxiety and distrust. This issue has been
pointed out.34 It stirs up the discussion about trust versus control in healthcare.
Transparency gives a certain kind of control, but it can also harm certain aspects of
trust.35 This is not a plea to abolish public availability of quality information, but to
balance the kind of quality information published publicly against the trust of
healthcare consumers in the competencies of the healthcare provider.
CHOICE BEHAVIOUR
Main findings (Chapter 5)
The review of the literature shows almost no effect of CPI on behaviour neither via
selection nor via improving the quality of care. The effects shown lasted no longer
than two months.
139
7
Chapter 7
The level of evidence from the prospective studies was low: only four of 6839 studies
were eligible for inclusion.
CPI and behavioural changes in healthcare performance: the underlying mechanism
Much has been invested in the development and provision of CPI since the early 1990s.
However, a systematic review as presented in Chapter 5 about the effects of the public
release of CPI on changing behaviour of healthcare consumers, providers, and
purchasers was lacking. It is often assumed that the information will affect and
facilitate decisions and behaviours of various parties and will ultimately improve the
healthcare system. Berwick and colleagues describe two pathways of how public
reporting can lead to performance change, improvement of quality of care.36 The first
pathway, ‘selection’, represents the mechanism in which consumers can vote with their
feet. Since consumers choose healthcare services on the basis of quality, healthcare
providers are tempted to improve their performance to avoid losing customers. The
second pathway, ‘change’, represents the mechanism of benchmarking and mutual
comparison. Knowing that their group is lagging behind motivates healthcare providers
to improve work procedures and professional culture.
New material
Since the review was published in 2011 and most studies were conducted in a rather
dynamic era of early development, we wondered whether more recent studies were
available able to produce additional insights. In the context of the general discussion of
this thesis, we started a narrative search of the MEDLINE (Ovid) database (15 January
2015) with the same search strategy as used earlier in Chapter 5. Only one database
was searched instead of five in the original review process. The search identified 996
relevant papers in the period of April 2011 to January 2015. Eleven studies qualified for
a full text analysis, and five of them will most likely fit in an update of the Cochrane
review. Thus, the narrative search shows that, despite a growing number of papers
published in recent years, the number of new studies that bring us additional evidencebased information is still limited. The two main reasons for exclusion were once again a
lack of description of the primary outcome measure and deficiencies in the study
design. Four studies we found in this narrative search focused on only one of the
pathways. Only one paper focussed on a combination of both pathways. The five
studies included are listed in Table 1. Two papers studied the pathway of selection.
One study of the impact of public reporting of surgery performance for coronary artery
bypass shows a decrease in the mortality rate after the release of public performance
data. The same study also states that public reporting may have caused that high-risk
patients avoid high-mortality hospitals either by provider choice or selective referral
behaviour (risk avoidance). As well that public reporting might cause an increase of
patients who went more often to low-mortality hospitals.37
140
Table 1. Overview of studies with the effectiveness of releasing public performance information about behavioural changes
Year of
publication Design
Author
Romano et al. 2011
37
Martino et al.
2012
39
2013
38
Chen et al.
Renzi et al.
Ryan et al.
40
41
Improvement by
Pathway I: Selection Pathway II:
Improvement
Interrupted time series Provider choice by
(three before-and-after patients/selective
data points)
referral behaviour by
primary care physicians
or cardiologists
An experimental design: Choice of new healthrandomized
plan members for
encouragement design
primary care physicians
based on performance
data
Use and analysis of
Better performance of
national cardiac arrest
hospitals
registry
2014
Pre- and post-evaluation
and comparative
evaluation
2012
Interrupted time series
using three models to
estimate the reduction
of mortality: 1) beforeand-after Hospital
Compare, 2) publicly
reported diagnoses, 3)
trends in non-reported
diagnoses
Impact of public
reporting on quality
indicators for acute
myocardial infarction
Patients changing from Hospital’s
lower- to higher-quality improvements of the
hospitals
publicly reported
measures (patient
mortality)
Outcomes
The release of public performance reports was
associated with increased patient volume at low
mortality in the first 6 months
No evidence of effectiveness of physician
performance
report on consumer choice
Hospitals performed better with regard to
publicly reported outcomes for common medical
conditions, though the outcomes for survival
rate of in-hospital cardiac arrest patients did not
improve
Public reporting has is associated with an
increasing proportion of patients treated in time
with percutaneous coronary interventions
The 30-day mortality rate after acute
myocardial infarction did not improve
Improvements accelerated immediately after
reporting the results, though factors other than
public reporting can plausibly explain the
improvements
Patients and physicians do not change to highquality hospitals. Mortality rates of heart attacks
and pneumonia do not change
7
Chapter 7
The other study of cardiac arrest shows no effect of physician-quality data on
consumers’ behaviour in choosing their primary-care physicians.38 Two studies present
findings about the second pathway of change.39,40 One study shows the impact of public
reporting on hospital care, though hospitals that performed better with regard to
publicly reported outcomes for common medical conditions did not necessarily have
better survival rates for cardiac arrest.39 Another study looks at the impact of public
reporting on an increased proportion of patients with myocardial infarction who
received timely treatment with a percutaneous coronary intervention. The number of
patients receiving this treatment increased significantly in the regions with public
reporting compared to other regions without public reporting.40
One study examines the effect of the public reporting of both pathways either on the
hospital’s improving the publicly reported measures (patient mortality), or on patients
changing from lower-quality to higher-quality hospitals. They show no evidence that
patients or physicians changed to high-quality hospitals nor showed they any evidence
that public reporting reduced mortality due to heart attack or pneumonia.41
Overall, the findings hint at an effect on behaviour change, although we should
probably conclude that more sound studies are needed in this area. Many studies in this
field have been conducted in the last few years, though only a few of them appear to
meet the strict criteria of the Cochrane Reviews. For the purpose of this general
discussion, we have been quite compliant about the inclusion of studies that we found
in the narrative search, but we expect that there might be one or two studies that will
not be included when the review is updated. The authors of the six recent studies
emphasize the facts that the evidence for the effectiveness of public reporting is
limited and the outcomes should be interpreted with care. Several authors of these
studies point out the difficulties of well-designed studies on public reporting, and their
attempts to design the studies as well as possible. All in all, this update and scan of the
literature is in line with the results of our review published in 2011. We conclude that
so far no firm conclusions can be drawn about the effect of CPI.
PROFESSIONAL GUIDANCE
Main findings
GPs experience of choice-supporting behaviour in the referral process (Chapter 6).
Most GPs do not know where to find CPI and have never used it before. The decisions
GPs make in the current process for referring patients to hospital care are not, or
hardly, based on CPI exchanges during consultations. Other referral considerations and
sources (e.g. personal contact with specialists, their own and patients’ experiences
with providers, and other patient experiences) are more important.
142
General discussion
Most GPs are sceptical about the ability of patients to use CPI. Several barriers also
hinder their own use of CPI. These barriers include indecisiveness about their role in
supporting patient choices, doubts about responsibility, feelings of uncertainty about
the interpretation of the information, doubts about the content of the information and
its reliability, and fear of weakening existing networks.
Our findings make two things clear. First, the pathway of selection that Berwick and
colleagues advocate as the mechanism to improve quality of care via public reporting36
does not seem to work effectively for either healthcare consumers or referring
providers. Like consumers, GPs find other sources, such as patient preferences,
personal contacts, pervious experiences with a hospital, and mouth-to-mouth
information, all of which are more important to them than CPI.42-47 Second, there is a
gap between assumptions and practice regarding professional guidance for consumers.
Ideally, GPs discuss the information and elucidate patient values before referring a
patient to a hospital.5 However, the results do not confirm the assumption that even
though patients hardly use CPI, they will benefit from performance information via
their GPs, who will bring the options for hospital choice to their attention. Other
studies find GPs similarly reluctant to make patients aware of performance
information.11,44 In general, the discussion of CPI between patients and referring
healthcare providers – researched via GPs and cardiologists – is rare.11,34,48,49 Given the
increased emphasis on a healthcare system that encourages active patient involvement
in their care process, it is remarkable that even the discussion of these quality data
with patients is so uncommon.
Discussion of GPs’ limited guidance
Consumer engagement does not come easily. There are several explanations for the
GPs limited engagement in helping patients make choices and for their lack of selective
referral behaviour.
One explanation is the lack of patient pressure on the GP.44 We found that GPs rarely
have patients initiating discussions about CPI. Others suggest that if more patients
approached their GPs with questions about this kind of information, GPs might pay
more attention to CPI themselves.44,50 The second explanation for the limited impact on
referral patterns is the lack of awareness of CPI45,51, though recent findings from the
USA43 show that even if almost every provider is aware of CPI, use does not necessarily
increase. For example, wider exposure did not convince referring cardiologists to trust
these data and use the information for referrals. The third explanation is that GPs are
reluctant to share information with which they are not familiar themselves.12 This
clashes with the claim that GPs should be capable of understanding and interpreting
data such as CPI, given their training and experience.46 This thesis reveals that GPs
143
7
Chapter 7
criticize the content of the current information for both their patients and themselves.
A recent study of Dutch GPs’ referral behaviour finds that referral patterns overall
remain unaffected by the available CPI about medical effectiveness and patient
experiences for the conditions of cataract surgery, breast cancer, and hip and knee
replacement. Nonetheless, referral patterns for the condition of breast cancer were
affected because GPs make more referrals to hospitals with a higher rating for medical
effectiveness. In line with qualitative statements, CPI about breast cancer appeared to
contain the most valid, reliable, and differentiating data.46 This calls for CPI based on
outcome indicators that shows relevant differences. Furthermore, referring providers
generally consider such data untrustworthy and not up to date; they judge the sources
as unreliable or the data as inaccurately assessing the quality of providers.43,44,52 The
findings in this thesis stress the importance of reliable performance information for
GPs. A final explanation for the limited guidance is that the findings reveal the GPs’
indecisiveness about their role in assisting patients in making choices. Further, these
explanations lead to the question of whether there are any people or professionals
other than the GPs to guide patients in using CPI.
Patients might benefit from having an advocate who makes sense of patients’
circumstances, helps them use CPI to make their choices, and advises them about their
future choices.34,53 The benefits of support have been demonstrated for other choices,
such as healthcare insurance plans and treatment.54-56 Helping consumers understand
insurance coverage conditions and supporting appeals for coverage denials are called
‘consumer assistance programmes’. In the USA, the assistance initiative has altered the
way consumers engage with healthcare insurers. It enhances their capacity for selfadvocacy and involvement, though many disempowered consumers still fail to seek
assistance.54 The effectiveness of decision coaching for patients who face a decision
between at least two treatment options has been reviewed. The findings, based on
several studies, show that decision coaching improves patient’s knowledge; such
knowledge is an important element in decision-making. However, decision coaching in
clinical care is still a novelty.55,56 As in other settings, the use of a ‘coach’, a ‘consumer
assistant’, or an ‘advocate’ in helping consumers make decisions based on CPI remains
open to debate.
METHODOLOGICAL CONSIDERATIONS
One strength of this thesis is the variety of research methods that we have used to
examine the research questions, namely, a systematic literature review, a qualitative
focus group, quantitative cross-sectional questionnaires, and a mixed-method study
with interviews and a questionnaire. Here we present our general reflections on the
methodological limitations and strengths of our work in the context of our research
methods.
144
General discussion
The first limitation is the number of healthcare consumers who participated in the
different studies. The small sample sizes threaten the validity of the outcomes. The
recruitment problem was not limited to healthcare consumers (Chapter 2); it was also
difficult to attract GPs (Chapter 6). From this we conclude that CPI is indeed not high
on the agenda of either healthcare consumers or providers.
The second limitation concerns the relatively high age (> 65 years) of the participating
consumers. The rationale for focusing on rather older participants was that these
people were more likely to make actual choices in healthcare (hip and knee
replacements for those confronted with Parkinson’s disease), and we wanted to find
out why this group of consumers had trouble seeking and using CPI. Still, to use the
words of Meinow and colleagues, ‘results suggest that those elderly people who are
most dependent on care and who could benefit most from a “good choice” are also the
ones who have the highest prevalence rates of cognitive, physical, and sensory
limitations’ p.1289.
The fact that this study used real and openly available CPI can be seen as a third
limitation. It limited us in terms of content, volume, distinctiveness, and reliability of
CPI. Though some of the attributes were not yet public, we added them because we
knew they would soon become available and we wanted to know whether people would
search for this information. At the same time, this limitation is a strength of the study:
the information was closely connected to the current practice of patient choice and the
use of CPI.
The fourth and last limitation of this study is the strict inclusion criteria we used in
conducting our systematic review (Chapter 5). We prefer to use these criteria to
measure the impact on real behaviour rather than hypothetical behaviour, and to make
a judgement based on the highest level of evidence. However, we are aware of the
large number of studies that are being published and the limited number of studies that
were suitable for inclusion in the systematic review and in the narrative search. These
elements force us to rethink our strict design criteria and consider other designs or
combinations of design for this type of research. The design must do justice to the
complex reality as well.
IMPLICATIONS
Taken together, this study and past work suggest that the provision of CPI to engage
patients and consumers in healthcare and to direct provider choice while
simultaneously driving the improvement of quality of healthcare has not shown to be
effective thus far. Nevertheless, the area of public reporting and the use of CPI do not
seem to be at a dead end. On the contrary, researchers, policy makers, insurers, and
145
7
Chapter 7
consumers maintain a continuous desire to keep moving in the direction of CPI and help
consumers improve the quality of the healthcare system by their choices of high-quality
providers despite the disappointing results.58 By far the most important reason why CPI
remains attractive is that we will not go back to the time when patients and consumers
had minor roles in the healthcare system. The source of CPI lies in accountability, the
transparency of quality performance of healthcare services, which has been positioned
as the consumer’s fundamental right to know one’s care, care options, and quality of
care.59 Even though three decades have passed since the beginning of public reporting,
we are just getting started. At the same time, there are signs that freedom of choice
for consumers might be under pressure and may become limited as the geographical
concentration of care increases. We see that the healthcare insurers are already
offering contracts that limit the number of available healthcare providers on
beforehand.
Based on findings in this thesis, we formulate several implications for those undertaking
future research, policy-makers, and consumer organizations.
Finding-based implications for future research
This thesis reveals that consumer expectation and recognition of quality differences
between hospitals and providers play an important role in the search for performance
information. More research is needed to get a better picture of how robust or fragile
healthcare consumer expectation and recognition of quality differences are, as well as
to determine what causes or influences expectation and recognition. Moreover, our
findings imply that performance information fails in the expectation of triggering
consumer interest. Studies that focus on how to strengthen the demand for CPI as well
as on finding an effective way to increase healthcare consumers’ knowledge about the
underlying concepts of quality of care and the purpose of performance information are
needed.
Our findings further reveal that performance information remains difficult to
understand. It may be that data currently presented in CPI fail to resonate with the
values and feelings that matter to consumers. It has been recently suggested that
anecdotal information from consumers13,60 or online reviews on social media sites and
commercial rating sites may be useful.61 These types of information may be better
understood than the current CPI, so they would be more valuable to consumers and
more likely to gain interest and satisfy a need. Giving consumers the ability to share
their experiences and views directly other than in patient experience surveys can be an
important aim in itself. Future research that focuses on how consumers can effectively
balance and integrate these personal commentaries would therefore be a necessary
addition to the literature of public reporting.13
146
General discussion
This study shows that most consumers still have difficulties using performance
information to choose a provider. Help and support from a professional nearby, such as
the GP, seem lacking. The GPs reported various barriers to their use of CPI in the
process of referral and to their taking a choice-supporting role. Future research into
the effectiveness of consumer support by a trusted agent is needed. Furthermore
studies that focus on the interaction between consumers and GPs or another trusted
agent while using CPI would fill a gap in the literature.
Research into the public reporting published over the last three decades is mainly
about the impact on improving the quality of care or selection of good practices in care
rather than about comparing and evaluating specific report designs to sort out the
effective and ineffective ones. A classical RCT design is not useful for capturing the
impact of public reporting. Given the public character of performance information, it is
complex and too difficult to determine whether control groups get access to CPI or
have ideas about it. Because of this, interrupted time series design and controlled
before-and-after studies are more suitable, though a lot of studies with either of these
designs do not satisfy the requirement of the multiple time points. At least three time
points are required. One task for future research is to gather more evidence about
which designs, or combinations of designs, can provide reliable information about the
impact of CPI on choice behaviour.
Finding-based implications for policy-makers and consumer organizations
This thesis contributes to the developing literature about the complex reasons why
most consumers are not prone or not eager to use CPI. The current results show that
the lack of demand for public reporting depends on the availability of other important
and more valued sources, lack of routine in using the information, outcome expectancy
of using CPI, and barriers to using the information (e.g. too much responsibility,
choosing is a burden, and questioning trustworthiness). The outcome expectancy may
be the biggest reason for some healthcare consumers’ lack of perceived need, whereas
the personal restraints of others may be their most prominent reason. Knowledge and
recognition of quality differences can probably motivate some consumers to search for
CPI. The consumers perceived professional guidance as helpful, but not necessarily
based on CPI. The GPs in our study questioned the current CPI, and they preferred
other sources to help the consumer select a healthcare provider. A series of initiatives
should counteract the concerns about the lack of information, insight, and
understanding for good quality of care. Starting to increase the urgency for consumers
and GPs, CPI need to address variation in quality of care, and visualize clearly these
differences. An issue that concerns not only for consumers and GPs, but for all
stakeholders is the design of publicly reported data. Much more effort is needed to get
more sophisticated information products.
147
7
Chapter 7
Further, providing more time in the referral process for both GPs and consumers would
be very helpful. More time might make informed choice possible for consumers and
improve the ability of GPs to help them. Currently, consumers receive referrals from
their GPs, and within minutes the choice for a hospital has been made. The moment of
actual choice has passed before the consumer has had a chance to catch up with it, let
alone actually use performance data or discuss their informed choice with their GP. We
invite policy-makers, GPs, and insurance companies to find the financial support for
extending the 10-minute standard GP consultation to allow a valuable discussion of
choice.
FINAL REMARK
CPI has been developed to help healthcare consumers choose a healthcare provider
that matches their needs and preferences and to drive healthcare providers to improve
the quality of their care. However, the evidence that CPI has an actual impact on
behaviour is still scant. Consumers say that the current CPI is novel, not well designed,
too abstract, too vague, meaningless, and too difficult to use. From this perspective,
the consumers’ non-use of the information might be reasonable, as might be their
sticking to their current selection routines. Help and support from a trusted agent can
overcome some of these issues. However, the GPs pointed out that the current state of
the CPI, fear of discontinuity and fragmentation in healthcare are important things that
stops them from using it. The promise of CPI remains attractive because of the
underlying principles of more transparency and increasing consumer involvement in
healthcare. The current state of affairs can be seen as an ongoing train, but the
current design of CPI seems to be more appropriate for policy reasons than for the
individual selection process of a healthcare provider.
CPI seems inadequate for consumers. Perhaps this is partly attributable to the fact that
consumers hardly play a role in the development of CPI. The production and
dissemination of CPI is also meant to encourage a more central position for consumers
in healthcare. Of course consumers do not suddenly achieve that position in healthcare
without effort from anyone. We call for policy-makers, CPI developers, and consumer
organizations to address the need for changing the format to help develop more
consumer-friendly and -oriented information. The value of such data might increase
once consumers become more involved for a longer time in developing performance
information.
148
General discussion
REFERENCES
1.
2.
3.
4.
5.
6.
7.
8.
9.
10.
11.
12.
13.
14.
15.
16.
17.
18.
19.
20.
21.
22.
23.
de Groot IB, Otten W, Smeets HJ, Marang-van de Mheen PJ, the CHOICE-2 study group. Is the
impact of hospital performance data greater in patients who have compared hospitals? BMC Health
Serv Res 2011;11:214.
Harris KM. How do patients choose physicians? Evidence from a national survey of enrollees in
employment-related health plans. Health Serv Res 2003;38(2): 711-32.
Rademakers J, Nijman J, Brabers AE, de Jong JD, Hendriks M. The relative effect of health literacy
and patient activation on provider choice in the Netherlands. Health Policy 2014;114(2-3): 200-6.
Robertson R, Dixon A. Choice at the point of referral. London: The King’s Fund, 2009.
Schwartz LM, Woloshin S, Birkmeyer JD. How do elderly patients decide where to go for major
surgery? Telephone interview survey. BMJ 2005;331(7520):821.
Victoor A, Rademakers J, Reitsma-van Rooijen M, de Jong J, Delnoij D, Friele R. The effect of the
proximity of patients’ nearest alternative hospital on their intention to search for information on
hospital quality. J Health Serv Res Policy 2014;19(1):4-11.
Zwijnenberg NC, Damman OC, Spreeuwenberg P, Hendriks M, Rademakers JJ. Different patient
subgroup, different ranking? Which quality indicators do patients find important when choosing a
hospital for hip- or knee arthroplasty? BMC Health Serv Res 2011;11:299.
Faber M, Bosch M, Wollersheim H, Leatherman S, Grol R. Public reporting in health care: how do
consumers use quality-of-care information? A systematic review. Med Care 2009;47(1):1-8.
Fotaki M, Roland M, Boyd A, McDonald R, Scheaff R, Smith L. What benefits will choice bring to
patients? Literature review and assessment of implications. J Health Serv Res Policy 2008;13(3):
178-84.
Damman OC, Hendriks M, Rademakers J, Delnoij DM, Groenewegen PP. How do healthcare
consumers process and evaluate comparative healthcare information? A qualitative study using
cognitive interviews. BMC Public Health 2009;9:423.
Coulter A. Do patients want a choice and does it work? BMJ 2010;341:c4989.
Dixon A, Robertson R, Appleby J, Burge P, Devlin N, Magee H. Patient choice. How patients choose
and providers respond. London: The King’s Fund, 2010.
Greaves F, Millett C, Nuki P. England’s Experience incorporating “anecdotal” reports from
consumers into their national reporting system: lessons for the United States of what to do or not
to do? Med Care Res Rev 2014;71(5 Suppl):65S-80S.
Vaiana ME, McGlynn EA. What cognitive science tells us about the design of reports for consumers.
Med Care Res Rev 2002;59(1):3-35.
Harris KM, Melinda Beeuwkes Buntin M, The RAND Corporation. Choosing a health care provider:
The role of quality information. Princeton: The Robert Wood Johnson Foundation, 2008.
Alexander JA, Hearld LR, Hasnain-Wynia R, Christianson JB, Martsolf GR. Consumer trust in sources
of physician quality information. Med Care Res Rev 2011;68(4):421-40.
Victoor A, Delnoij D, Friele R, Rademakers J. Why patients may not exercise their choice when
referred for hospital care. An exploratory study based on interviews with patients. Health Expect
2014.
Dijs-Elsinga J, Otten W, Versluijs MM, Smeets HJ, Kievit J, Vree R, et al. Choosing a hospital for
surgery: the importance of information on quality of care. Med Decis Making 2010;30(5):544-55.
Fung CH, Lim YW, Mattke S, Damberg C, Shekelle PG. Systematic review: the evidence that
publishing patient care performance data improves quality of care. Ann Intern Med 2008;148(2):
111-23.
Mittler JN, Martsolf GR, Telenko SJ, Scanlon DP. Making sense of “consumer engagement”
initiatives to improve health and health care: a conceptual framework to guide policy and
practice. Milbank Q 2013;91(1):37-77.
Moser A, Korstjens I, van der Weijden T, Tange H. Patient’s decision making in selecting a hospital
for elective orthopaedic surgery. J Eval Clin Pract 2010;16(6):1262-68.
Vonberg RP, Sander C, Gastmeier P. Consumer attitudes about health care acquired infections: a
German survey on factors considered important in the choice of a hospital. Am J Med Qual 2008;
23(1):56-59.
Trigg L. Patients’ opinions of health care providers for supporting choice and quality improvement.
J Health Serv Res Policy 2011;16(2):102-7.
149
7
Chapter 7
24.
25.
26.
27.
28.
29.
30.
31.
32.
33.
34.
35.
36.
37.
38.
39.
40.
41.
42.
43.
44.
45.
150
Fotaki M, Boyd A, Smith L, McDonald R, Roland M, Sheaff R, et al. Patient Choice and the
Organisation and Delivery of Health Services: Scoping review In, Manchester: Report for the
National Co-ordinating Centre for NHS Service Delivery and Organisation R&D (NCCSDO) 2005.
Marshall MN, Romano PS, Davies HT. How do we maximize the impact of the public reporting of
quality of care? Int J Qual Health Care 2004;16(Suppl 1):i57-63.
Huppertz JW, Carlson JP. Consumers' use of HCAHPS ratings and word-of-mouth in hospital choice.
Health Serv Res 2010;45(6 Pt 1):1602-13.
Lagu T, Lindenauer PK. Putting the public back in public reporting of health care quality. JAMA
2010;304(15):1711-12.
Stein BD, Kogan JN, Essock S, Fudurich S. Views of mental health care consumers on public
reporting of information on provider performance. Psychiatr Serv 2009;60(5):689-92.
de Groot IB, Otten W, Dijs-Elsinga J, Smeets HJ, Kievit J, Marang-van de Mheen PJ, et al. Choosing
between hospitals: the influence of the experiences of other patients. Med Decis Making
2012;32(6):764-78.
Fung CH, Elliott MN, Hays RD, Kahn KL, Kanouse DE, McGlynn EA, et al. Patients’ preferences for
technical versus interpersonal quality when selecting a primary care physician. Health Serv Res
2005;40(4):957-77.
Werner RM, Norton EC, Konetzka RT, Polsky D. Do consumers respond to publicly reported quality
information? Evidence from nursing homes. J Health Econ 2012;31(1):50-61.
Bowling A, Rowe G, Lambert N, Waddington M, Mahtani KR, Kenten C, et al. The measurement of
patients’ expectations for health care: a review and psychometric testing of a measure of
patients’ expectations. Health Technol Assess 2012;16(30): i-xii, 1-509.
Joseph-Williams N, Elwyn G, Edwards A. Knowledge is not power for patients: a systematic review
and thematic synthesis of patient-reported barriers and facilitators to shared decision making.
Patient Educ Couns 2014;94(3):291-309.
Schlesinger M. Choice cuts: parsing policymakers’ pursuit of patient empowerment from an
individual perspective. Health Econ Policy Law 2010;5(3): 365-87.
Fotaki M. Can consumer choice replace trust in the National Health Service in England? Towards
developing an affective psychosocial conception of trust in health care. Sociol Health Illn 2014;
36(8): 1276-94.
Berwick DM, James B, Coye MJ. Connections between quality measurement and improvement. Med
Care 2003;41(1 Suppl):I30-8.
Romano PS, Marcin JP, Dai JJ, Yang XD, Kravitz RL, Rocke DM, et al. Impact of public reporting of
coronary artery bypass graft surgery performance data on market share, mortality, and patient
selection. Med Care 2011;49(12):1118-25.
Martino SC, Kanouse DE, Elliott MN, Teleki SS, Hays RD. A field experiment on the impact of
physician-level performance data on consumers' choice of physician. Med Care 2012;50(Suppl):S6573.
Chen LM, Nallamothu BK, Krumholz HM, Spertus JA, Tang F, Chan PS, et al. Association between a
hospital’s quality performance for in-hospital cardiac arrest and common medical conditions. Circ
Cardiovasc Qual Outcomes 2013;6(6):700-7.
Renzi C, Asta F, Fusco D, Agabiti N, Davoli M, Perucci CA. Does public reporting improve the
quality of hospital care for acute myocardial infarction? Results from a regional outcome
evaluation program in Italy. Int J Qual Health Care 2014;26(3):223-30.
Ryan AM, Nallamothu BK, Dimick JB. Medicare'’ public reporting initiative on hospital quality had
modest or no impact on mortality from three key conditions. Health Aff (Millwood) 2012;31(3):
585-92.
Birk HO, Henriksen LO. Which factors decided general practitioners’ choice of hospital on behalf of
their patients in an area with free choice of public hospital? A questionnaire study. BMC Health
Serv Res 2012;12:126.
Brown DL, Epstein AM, Schneider EC. Influence of cardiac surgeon report cards on patient referral
by cardiologists in New York state after 20 years of public reporting. Circ Cardiovasc Qual
Outcomes 2013;6(6):643-48.
Doering N, Maarse H. The use of publicly available quality information when choosing a hospital or
health-care provider: the role of the GP. Health Expect 2014.
Epstein AJ. Effects of report cards on referral patterns to cardiac surgeons. J Health Econ 2010;
29(5):718-31.
General discussion
46.
47.
48.
49.
50.
51.
52.
53.
54.
55.
56.
57.
58.
59.
60.
61.
Ikkersheim D, Koolman X. The use of quality information by general practitioners: does it alter
choices? A randomized clustered study. BMC Fam Pract 2013;1495.
Audet AM, Doty MM, Shamasdin J, Schoenbaum SC. Physicians’ views on quality of care: Findings
from the Commonwealth Fund National Survey of Physicians and Quality of Care. New York The
Commonwealth Fund, 2005.
Hannan EL, Stone CC, Biddle TL, DeBuono BA. Public release of cardiac surgery outcomes data in
New York: what do New York state cardiologists think of it? Am Heart J 1997;134(6):1120-28.
Schneider EC, Epstein AM. Influence of cardiac-surgery performance reports on referral practices
and access to care. A survey of cardiovascular specialists. N Engl J Med 1996;335(4):251-56.
Barr JK, Bernard SL, Sofaer S, Giannotti TE, Lenfestey NF, Miranda DJ. Physicians’ views on public
reporting of hospital quality data. Med Care Res Rev 2008;65(6):655-73.
Morsi E, Lindenauer PK, Rothberg MB. Primary care physicians’ use of publicly reported quality
data in hospital referral decisions. J Hosp Med 2012;7(5):370-75.
Rozen R, Florin D, Hutt R. An anatomy of GP referral decisions. A qualitative study of GP's view on
their role in supporting patient choice. London: The King's Fund, 2007.
Shaller D, Kanouse DE, Schlesinger M. Context-based strategies for engaging consumers with public
reports about health care providers. Med Care Res Rev 2014;71(5 Suppl):17S-37S.
Grob R, Schlesinger M, Davis S, Cohen D, Lapps J. The Affordable Care Act's plan for consumer
assistance with insurance moves states forward but remains a work in progress. Health Aff
(Millwood) 2013;32(2):347-56.
Stacey D, Kryworuchko J, Belkora J, Davison BJ, Durand MA, Eden KB, et al. Coaching and guidance
with patient decision aids: A review of theoretical and empirical evidence. BMC Med Inform Decis
Making 2013;13(Suppl 2):S11.
Stacey D, Kryworuchko J, Bennett C, Murray MA, Mullan S, Legare F. Decision coaching to prepare
patients for making health decisions: a systematic review of decision coaching in trials of patient
decision AIDS. Med Decis Making 2012;32(3): E22-33.
Meinow B, Parker MG, Thorslund M. Consumers of eldercare in Sweden: the semblance of choice.
Soc Sci Med 2011; 73(9): 1285-9.
Damberg CL, McNamara P. Postscript: research agenda to guide the next generation of public
reports for consumers. Med Care Res Rev 2014;71(5 Suppl): 97S-107S.
Hussey PS, Luft HS, McNamara P. Public reporting of provider performance at a crossroads in the
United States: summary of current barriers and recommendations on how to move forward. Med
Care Res Rev 2014;71(5 Suppl): 5S-16S.
Egmond van S, Heerings M. In Dutch: Sterke verhalen uit het ziekenhuis. Leren van
patientervaringen voor goede zorg. In, Den Haag.: Rathenau Instituut, 2014.
Lagu T, Greaves F. From Public to Social Reporting of Hospital Quality. J Gen Intern Med 2015.
7
151
Summary
Summary
Chapter 1 presents an overview of this thesis, which describes the barriers and
facilitating factors of search behaviour for comparative performance information (CPI)
and healthcare consumers’ selective behaviour in choosing healthcare providers.
Providing CPI for healthcare services and healthcare providers is one way of giving
healthcare consumers insight into the quality of care. Then they can use this
information to make an informed choice of healthcare provider. The underlying idea is
that consumers who have good and reliable performance information will choose the
best healthcare provider, which will pressure healthcare providers to improve the
quality of care. Whether consumers are rational decision-makers who base their choices
of healthcare providers on cognitive considerations is the question. However, it is much
more important to provide consumers with good information so that they have more
control, influence, and power to make their own decisions. Although consumers strive
for more involvement and wish to have such information, they hardly use it. This thesis
explores three research themes: search triggers that affect the search behaviour,
choice behaviour, and professional guidance to support choice behaviour. First, we
explore why healthcare consumers do not yet use this type of information, why they
neglect it, misunderstand it, or have trouble searching for it. Second, we examine the
effect of CPI on the behaviour of various parties. Third, we examine the effect of
performance information on the behaviour of these parties. Fourth, we focus on the
GPs and their use of CPI in the current referral process, as well as their role in helping
patients make an informed choice.
Chapter 2 presents the findings of a qualitative focus group study among healthcare
consumers. Our aim was to get insight into consumers’ attitudes, opinions, value
received, and appreciation concerning CPI. Six focus groups were conducted with
27 healthcare consumers. The results showed that most consumers were unfamiliar
with the information and had not used it before. We used clear examples of CPI to help
consumers understand it. The consumers very ably put into words why they did not
search for this information. They valued other sources, including personal experience,
more than CPI. Information from different sources was often inconsistent, which cast
doubt on its reliability. This study led to new and extended insight into consumers’
distrust. There was a fear of disrupting existing relationships, loss of faith in certain
aspects of care that consumers had thought were good, and the pressure of insecurity
due to new responsibilities. This study also reveals that consumers noted positive
effects of CPI such as: more freedom of choice, more involvement for healthcare
consumers, the potential effect of benchmarking, and a new-found awareness of
quality of care in their own perception. A possible negative side-effect of CPI is that it
might cause a dichotomy in society between those who search for CPI and use it and
those who do not. Nonetheless, we conclude that the consumers valued the CPI only
154
Summary
marginally. Different values and principles limit the search for CPI, which often
precludes its use.
Chapter 3 examines the patients’ expectations concerning differences in care quality
and their search for CPI. Questionnaires were distributed among 475 patients who
underwent primary hip or knee surgery in a university, teaching, or community
hospital. Of the 302 patients who responded (response rate 63%), only a small minority
(13%) searched for CPI after they had found out they needed surgery. Most of those who
did not search for CPI felt no need of more information. Most patients (67%) expected
to find differences in the quality of care. These were the ones who were significantly
more likely to search for CPI (OR=3.18 [95% CI: 1.02–9.89]; p<0.04) than those who did
not expect such differences. The search trigger was even greater for patients expecting
large differences (OR=5.05 [95% CI: 1.44-17.77]; p<0.01). The patients expected
differences especially in general qualities such as reputation, distance, and
accessibility. They expected more differences in general performance than in specific
factors related to the patient’s disorder.
Chapter 4 reports the findings of a quantitative study about the ability of people with
Parkinson’s disease to recognize expertise, and it considers to what extent their
recognition and evaluation of this expertise triggers their search and selective choice of
experts. This survey study in a chronic-care setting focuses on physiotherapists’
expertise. Five hundred eligible respondents were invited to participate. This sample
was based on claim data of the Dutch insurance company CZ. We analysed 320 surveys
(response rate 64%). Healthcare providers’ expertise is based on a quality accreditation
for special training to work according to evidence-based recommendations as well as
structured referrals to increase the volume of patients with Parkinson’s disease whom
they treat. Most participants (89%) with Parkinson’s disease took the referring
provider’s advice, though this advice was rarely accompanied by supporting arguments.
Not many looked for additional information (11%), but the participants did recognize
some qualities of expertise. The respondents who did recognize the added value of
specific expertise were significantly more likely to search (3.28 times more often) for
performance information than those who did not. The willingness to switch to an expert
physiotherapist increased significantly (p<0.01) once patients recognized the added
value of specific expertise. About half the people with Parkinson’s disease did not
recognize the added value, and they overestimated the expertise of their own
healthcare providers. Based on the claim data, approximately 28% of the respondents
received treatment from a physiotherapist with specific expertise in Parkinson’s
disease, while 70% of the respondents claimed they received treatment from
specialized physiotherapists. Considering their own physiotherapists (without special
155
Summary
training) to be experts in Parkinson’s disease might reduce their motivation to search
for CPI or to search for another physiotherapist.
Chapter 5 presents a systematic review that estimates the effects of the public release
of CPI about changing the behaviour of healthcare consumers, providers, and
purchasers. Much has been invested in the development and provision of performance
information since the early 1990s in the USA. However, England and, in more recent
years, the Netherlands, along with many other countries, have produced no systematic
reviews about the effectiveness of such information. Because of the lack of evidence,
we analysed studies of the effects of public performance information on changing
behaviour. It is often assumed that the information will affect and facilitate decisions
and behaviours of various parties and will ultimately improve the healthcare system.
Various mechanisms in the public release of performance information may increase the
quality of care. Berwick and colleagues describe mechanisms that work via two
pathways: 1) the behaviour in choosing the best healthcare provider or organization
and 2) improving performance by changing work procedures or professional culture. We
used these two pathways as our primary outcome measures. We found 6839 relevant
studies by using an extended search strategy to search five databases for studies
published up to early 2011. Ultimately, after a strict selection procedure, as applied to
the Cochrane review (Chapter 5), we included four studies. Two studies focused on the
first pathway (selection by choice behaviour) and two studies focused on the second
pathway (improvements in care). The findings show almost no effect of CPI on
behavioural changes in either pathway. Selection and improving quality of care had
virtually no effect. Any impact we found did not persist longer than 2 months. The level
of evidence, however, was low, given the inclusion of only four studies and the analysis
based on the GRADE1 system.
Our mixed-method study with 15 interviews with GPs and a subsequent survey in 81
general practices (Chapter 6) aimed to examine the position of the GP in terms of
guidance and support of patients making choices. We explored the barriers to GPs’
search for and use of CPI. Most GPs did not know where to find CPI (87%) and had never
used it before (94%). The findings show that the decisions GPs make in the current
process for referring patients to hospital care are not, or hardly, based on CPI
exchanges during consultations. Other referral considerations and sources (e.g.
personal contact with specialists, their own and patients’ experiences with providers,
and other patient experiences) are still more important than patient choice, and there
is no role for CPI as an additional source. According to the GPs, patients hardly ever
1
Grading of Recommendations, Assessment, Development and Evaluations
http://clinicalevidence.bmj.com/x/set/static/ebm/learn/665072.html
156
Summary
initiate a discussion of CPI during consultations, though they seem to be increasingly
critical about differences in care quality. Most GPs are sceptical about the ability of
patients to use CPI; they find the current information too difficult for most patients.
Asking GPs about this matter taught us that several barriers also hinder their own use
of CPI. These barriers include indecisiveness about their role in supporting patient
choices, their task in addressing CPI during consultations, doubts about whose
responsibility this is, feelings of uncertainty about the interpretation of the
information, doubts and distrust of the content of the information and its reliability,
and fear of weakening their networks. Whether GPs can fulfil a role in providing
patients with guidance in the use of CPI is still undecided. First there must be more
clarity about the conditions.
In Chapter 7, the final chapter of this thesis, the most important findings and
conclusions of the studies in Chapters 2–6 are discussed. The results are placed in wider
perspectives and compared with other studies. The most relevant methodological
limitations are considered, practical implications are presented, and recommendations
are put forward for further research.
The purpose of CPI is 1) to facilitate the consumer’s choice of a healthcare provider
based on their requirements and personal preferences and 2) to force providers to
improve the care. However, we see little proof that CPI achieves this purpose. Both
consumers and GPs criticize the current CPI and therefore do not use it. Policy reasons
seem to be the foundation of the current design of CPI, but the promise of CPI that
provides more transparency and enables better choices can still be fulfilled and has
therefore value in itself.
157
Samenvatting
Samenvatting
Hoofdstuk 1 presenteert een overzicht van de inhoud van dit proefschrift dat de
belemmerende en bevorderende factoren van zoekgedrag naar kwaliteitsinformatie en
het keuzegedrag naar zorgaanbieders door consumenten beschrijft. Het beschikbaar
stellen van kwaliteitsinformatie over zorgaanbieders is een manier om
zorgconsumenten inzicht te geven in de kwaliteit van zorg, zodat zij een geïnformeerde
keuze kunnen maken voor een zorgaanbieder. Het onderliggende idee is dat
consumenten met goede informatie kunnen kiezen voor de beste zorgaanbieder,
waarmee zij druk uitoefenen op zorgaanbieders om de kwaliteit van zorg te
verbeteren. Of consumenten zich zo rationeel gedragen om de keuze voor een
zorgaanbieder te maken op basis van cognitieve overwegingen is de vraag. Los daarvan
is het een veel belangrijker doel consumenten van goede informatie te voorzien en hen
hierdoor meer controle, invloed en kracht te geven om hun eigen keuzes te maken.
Hoewel zorgconsumenten meer eigen regie nastreven en ook aangeven deze informatie
te wensen, gebruiken zij de informatie die hen daartoe in de gelegenheid stelt niet of
in zeer beperkte mate. In dit proefschrift exploreren we drie onderzoeksthema’s,
namelijk: prikkels die zoekgedrag beïnvloeden, keuzegedrag en ondersteuning bij het
maken van keuzes. Allereerst kijken we naar de onderliggende redenen waarom
consumenten deze informatie (nog) niet gebruiken, negeren, of moeite hebben om
deze informatie te begrijpen en om op zoek te gaan naar deze informatie. Vervolgens
kijken we naar het effect van kwaliteitsinformatie op gedrag bij diverse partijen. En
tot slot focussen we op de rol van de huisarts om consumenten te helpen bij het maken
van een geïnformeerde keuze en het gebruik van kwaliteitsinformatie tijdens het
verwijsproces.
In het tweede hoofdstuk presenteren we de bevindingen van een focusgroepstudie
onder consumenten. Het doel van deze studie was om inzicht te krijgen in waardering,
begrip, opinie en beoordeling van kwaliteitsinformatie. Zes focusgroepen vonden plaats
met consumenten (n=27). De meeste deelnemers waren niet bekend met dergelijke
informatie en hadden het niet eerder gebruikt bij het maken van een keuze binnen de
zorg. De consumenten gaven aan niet de noodzaak te voelen om te zoeken naar
kwaliteitsinformatie. Tijdens de focusgroepen gebruikten we diverse voorbeelden van
kwaliteitsinformatie om de discussie verder op gang te helpen. Als reactie daarop
konden consumenten sneller hun twijfels, terughoudendheid en kritische vragen onder
woorden brengen waarom ze niet op zoek gingen naar deze informatie. Zo gaven zij
aan andere bronnen (familie/vrienden, eigen ervaring en de huisarts) belangrijker te
vinden dan kwaliteitsinformatie. Barrières ten aanzien van de waarde van
kwaliteitsinformatie betroffen: gebrek aan toepasbaarheid omdat kwaliteitsinformatie
vanuit diverse bronnen elkaar soms tegenspreken, onzekerheid omdat dit type
informatie een beroep doet op een nieuw soort verantwoordelijkheidsgevoel waarvan
ze de gevolgen (nog) niet overzien en angst om het vertrouwen te verliezen in bepaalde
160
Samenvatting
aspecten van zorg waarvan ze dachten dat die goed zijn. Een ander belangrijk punt was
de betrouwbaarheid van de informatie. Ook liet deze studie zien dat consumenten
positieve effecten ervaren van kwaliteitsinformatie, zoals: meer keuzevrijheid, meer
betrokkenheid van en regie voor consumenten, een toename in bewustzijn over
kwaliteit, als ook (mogelijk) nadelige effecten, zoals tweedeling tussen mensen die wel
in staat zijn om de informatie te gebruiken en zij die dit niet kunnen. Op basis hiervan
kunnen we concluderen dat consumenten kwaliteitsinformatie slechts marginaal
waarderen. Verschillende waarden en principes beperken het zoeken naar
kwaliteitsinformatie, en dus ook het gebruik ervan.
In het derde hoofdstuk is onderzocht of de verwachtingen die patiënten hebben over
verschillen in kwaliteit van zorg van invloed kunnen zijn op het zoekgedrag naar
kwaliteitsinformatie. Vragenlijsten werden uitgezet onder 475 patiënten die een
electieve heup- of knieoperatie ondergingen in een academisch, topklinisch of
streekziekenhuis. Van de 302 (respons 63%) patiënten die de vragenlijst invulden, liet
een kleine minderheid weten (13%) op zoek te zijn gegaan naar kwaliteitsinformatie
nadat zij hadden gehoord een vervangende heup-of knieoperatie te moeten ondergaan.
Voor patiënten die niet op zoek gingen naar kwaliteitsinformatie was de belangrijkste
reden dat ze simpelweg geen behoefte hadden aan meer informatie. Een meerderheid
van de patiënten (67%) gaf aan verschillen te verwachten in de kwaliteit van zorg, zij
gingen vaker op zoek naar kwaliteitsinformatie (OR=3.18 [95% BI: 1.02-9.89]; p <0.04)
dan patiënten die geen verschillen in kwaliteit verwachtten. Voor patiënten die grote
verschillen verwachtten was deze relatie nog sterker ten opzichte van patiënten die
geen verschillen verwachtten (OR=5.05 [95% BI: 1.44-17.77]; p<0.01). Patiënten
verwachtten voornamelijk verschillen in reputatie, afstand en toegankelijkheid. Deze
aspecten hebben meer betrekking op algemene kenmerken dan specifieke,
aandoeninggerelateerde aspecten. Wanneer de kennis over het concept kwaliteit van
zorg toeneemt, kunnen ook de aspecten waarin patiënten verschillen in verwachtingen
mogelijk veranderen.
In hoofdstuk 4 rapporteren we de bevindingen van een kwantitatieve studie waarin we
onderzocht hebben of patiënten in staat zijn om specifieke expertise bij hun
zorgverleners te herkennen en of deze herkenning een stimulerende rol speelt bij
zoekgedrag naar informatie over een zorgverlener. Om dit doel te bereiken, hebben we
een studie opgezet binnen de chronische zorg. Rondom de Ziekte van Parkinson is de
afgelopen jaren een multidisciplinair netwerk opgezet waarin zorgverleners
(neurologen, verpleegkundigen, ergotherapeuten, fysiotherapeuten, huisartsen)
intensief met elkaar samenwerken. Zorgverleners aangesloten bij dit netwerk worden
extra getraind in het behandelen van mensen met de ziekte van Parkinson. Door het
volgen van specifieke richtlijnen en het behandelen van een hoger patiëntenvolume van
161
Samenvatting
patiënten met de ziekte van Parkinson krijgen zorgverleners meer specifieke expertise.
In deze studie hebben we ons gericht op de expertise van fysiotherapeuten. De vraag is
of mensen met de ziekte van Parkinson deze expertise herkennen en waarderen en ook
of zij deze expertise meenemen in het zoeken naar kwaliteitsinformatie om een keuze
te maken voor een fysiotherapeut. Op basis van declaratiegegevens van
zorgverzekeraar CZ nodigden we 500 mensen uit om een vragenlijst in te vullen,
waarvan we 320 ingevulde vragenlijsten (response 64%) retour kregen. Veruit de
meeste mensen met Ziekte van Parkinson in onze studie (89%) volgden het advies op
van hun verwijzer, dit advies was zelden onderbouwd met argumenten. Een kleine
groep mensen (11%) ging op zoek naar extra informatie. Respondenten bleken
verschillende aspecten van expertise te herkennen. De groep die specifieke Parkinson
expertise herkende, ging 3.28 keer vaker op zoek naar informatie dan zij die deze
toegevoegde expertise niet herkenden. Ook de bereidheid om te wisselen van
fysiotherapeut naar een in Parkinson gespecialiseerde fysiotherapeut bleek groter (p <
.001) te zijn bij respondenten die de expertise onderschreven. Op basis van de
declareergegevens zou ongeveer 28% van de respondenten onder behandeling zijn van
een gespecialiseerde fysiotherapeut, terwijl 70% van de respondenten aangaf onder
behandeling te zijn van een gespecialiseerde fysiotherapeut. Deze (mogelijke)
overschatting van de specifieke expertise in Parkinson bij de eigen fysiotherapeut kan
ook een rol spelen in het op zoek gaan naar kwaliteitsinformatie. Mensen gaan er al
vanuit dat zij bij de beste zitten en zien daarom geen noodzaak om op zoek te gaan
naar kwaliteitsinformatie en met die informatie op te zoek te gaan naar een andere
fysiotherapeut.
In het vijfde hoofdstuk presenteren we een systematische literatuurstudie naar de
effecten van openbare kwaliteitsinformatie op gedrag van zorgconsumenten,
zorgaanbieders en zorginkopers. Hoewel er sinds begin jaren ’90 in de Verenigde Staten
en Engeland - en in recente jaren in veel andere landen, waaronder Nederland - veel
geïnvesteerd is in de ontwikkeling van kwaliteitsinformatie en het beschikbaar maken
van die informatie, waren er tot dusverre geen reviews uitgevoerd naar de
effectiviteit. Daarom analyseerden we studies naar de effecten van openbaar
beschikbare kwaliteitsinformatie op gedrag. Het achterliggende idee van het openbaar
maken van kwaliteitsinformatie is dat patiënten deze informatie inzetten om de
kwaliteit van te zorg te verbeteren. Uit de literatuur gebruikten we een model met
twee paden dat laat zien hoe kwaliteitsinformatie een effect zou kunnen uitoefenen
om de zorg te verbeteren namelijk door: 1) keuzegedrag voor de beste zorgaanbieder
en 2) verbeterinitiatieven vanuit zorgaanbieders om de kwaliteit van zorg te
verbeteren. Wij gebruikten deze twee paden als gedragsuitkomstmaten. Met een
uitgebreide zoekstrategie, waarmee vijf databanken werden doorzocht tot 2011, werd
een groot aantal studies (n=6839) gevonden. Uiteindelijk konden na een strenge
162
Samenvatting
selectieprocedure vier studies geïncludeerd worden. Twee studies richtten zich op het
eerste pad (selectie door middel van keuzegedrag) en twee studies focusten zich op het
tweede pad (verbeteringen in de zorg). De impact van kwaliteitsinformatie op gedrag
was niet of nauwelijks aanwezig en die impact die wél gevonden werd, was van korte
duur. Concluderend kunnen we stellen dat kwaliteitsinformatie niet of nauwelijks een
effect heeft op gedrag. Maar, het bewijs waarop we deze conclusie baseren is beperkt
met slechts vier studies, en de analyse van de studies met het GRADE1 systeem laat
zien dat de kwaliteit van de studies laag is. Daarbij zijn alle vier studies uitgevoerd in
een relatief nieuw onderzoeksveld en in een sterk veranderende context. Concluderend
kunnen we stellen dat het effect nog weinig op een goede wijze is onderzocht en dat er
dus nog geen harde conclusie getrokken kan worden over het effect van
kwaliteitsinformatie op gedrag.
In het zesde hoofdstuk beschrijven we een mixed-method studie met als doel te
onderzoeken welke rol kwaliteitsinformatie speelt in het huidige verwijsbeleid van
huisartsen en welke rol de huisarts speelt bij het stimuleren van keuzegedrag,
gebaseerd op kwaliteitsinformatie door patiënten. We hebben gekeken welke barrières
huisartsen ondervinden bij het zoeken en gebruiken van kwaliteitsinformatie. Vijftien
diepte-interviews
vonden
plaats
met
huisartsen
over
verwijsroutines,
patiëntbetrokkenheid en de rol van kwaliteitsinformatie tijdens het verwijsproces. De
interviews werd letterlijk uitgewerkt, geanalyseerd en de resultaten werden vervolgens
verwerkt in een vragenlijst die werd uitgezet onder 81 huisartsen die verbonden waren
aan het Landelijke Informatie Netwerk Huisartsenzorg 70 huisartsen (86% respons)
vulden de vragenlijst in. De meeste huisartsen wisten niet waar ze kwaliteitsinformatie
zouden moeten zoeken (87%) en hadden het nooit eerder gebruikt (94%).
Kwaliteitsinformatie speelt geen tot een minimale rol bij het verwijzen van patiënten
naar een ziekenhuis of andere professionals, terwijl andere bronnen zoals het
persoonlijk contact met artsen, eigen ervaringen, patiëntervaringen en
patiëntvoorkeuren voor huisartsen belangrijke bronnen zijn in het verwijsproces.
Huisartsen noemden veel verschillende barrières die ervoor zorgden dat ze niet
gemotiveerd worden om kwaliteitsinformatie te gebruiken. Ze twijfelden aan hun rol
om patiënten te stimuleren in keuzegedrag, en vroegen zich af of het gebruik van
kwaliteitsinformatie niet behoort tot de eigen verantwoordelijkheid van de patiënt,
terwijl ze zich tegelijk zeer verantwoordelijk voelden voor een goed verwijsproces.
Daarnaast twijfelden ze aan de toegevoegde waarde van kwaliteitsinformatie en aan de
mogelijk positieve uitkomsten voor patiënten. Ten aanzien van het eigen gebruik van
kwaliteitsinformatie voelden huisartsen zich onzeker of zij in staat zijn om de
1
Grades of Recommendations Assessment Development and Evaluation (GRADE)
http://clinicalevidence.bmj.com/x/set/static/ebm/learn/665072.html
163
Samenvatting
informatie op een goede manier te interpreteren. Zij bekritiseerden ook de inhoud van
de informatie, omdat het volgens hen inhoudelijk gezien niet de belangrijkste
informatie bevat waaraan patiënten behoefte hebben en zij waren kritisch over de
validiteit van de informatie, waardoor ze de betrouwbaarheid ervan in twijfel trokken.
Daarnaast gaven ze aan weinig noodzaak te voelen om het te gebruiken, aangezien zij
nauwelijks
geconfronteerd
worden
met
vragen
vanuit
patiënten
over
kwaliteitsinformatie. Volgens huisartsen is een keuze maken op basis van
kwaliteitsinformatie een erg moeilijke opgave voor patiënten. Tot slot noemden zij dat
het gebruik van kwaliteitsinformatie tot een (te) grote verwijsregio (meer ziekenhuizen
en zorgverleners) zou kunnen leiden, waarmee ze aangaven daarmee ook overzicht en
belangrijke informatiebronnen te verliezen. Of huisartsen een stimulerende rol kunnen
vervullen in het gebruik van kwaliteitsinformatie door patiënten staat nog niet vast,
eerst moet meer duidelijk worden welke voorwaarden voor huisartsen gerealiseerd
moeten worden om deze rol vorm te geven.
In hoofdstuk 7, het laatste hoofdstuk van dit proefschrift, worden de belangrijkste
bevindingen van dit proefschrift uit de voorgaande hoofdstukken samengevat en
bediscussieerd. De resultaten worden in een groter perspectief geplaatst en vergeleken
met andere studies. De meest relevante methodologische overwegingen en beperkingen
worden besproken, praktische implicaties voor beleidsmakers, ontwikkelaars van
kwaliteitsinformatie worden gepresenteerd en er worden aanbevelingen gedaan voor
vervolgonderzoek.
Vergelijkende kwaliteitsinformatie is ontwikkeld om 1) consumenten en patiënten te
helpen bij het kiezen van een zorgverlener die past bij hun behoeften en voorkeuren 2)
en indirect zorgverleners hiermee te stimuleren om de kwaliteit van zorg (verder) te
verbeteren. Echter, er is weinig bewijs dat de presentatie van kwaliteitsinformatie aan
consumenten en patiënten een effect heeft op gedrag. Zowel consumenten als
huisartsen bekritiseren huidige kwaliteitsinformatie en gebruiken het mede daarom niet
of nauwelijks. Toch kan de belofte dat kwaliteitsinformatie meer transparantie biedt
en mensen in staat stelt betere keuzes te maken nog steeds vervuld worden en heeft in
dat opzicht waarde in zichzelf.
164
Dankwoord
Dankwoord
Een ware uitdaging om het grote aantal mensen dat mij geholpen heeft hier kort te
bedanken. In de eerste plaats wil ik alle respondenten (zorgconsumenten, patiënten en
huisartsen) bedanken die een bijdrage hebben geleverd.
Veel dank ben ik verschuldigd aan mijn promotoren prof. dr. Gert Westert, prof. dr.
Glyn Elwyn en copromotoren dr. Marjan Faber, dr. Jozé Braspenning.
Beste Marjan, het duurde even voordat wij met elkaar de juiste modus hadden
gevonden en goed op elkaar ingespeeld raakten. Ik in mijn rol als promovendus, jij in je
rol als copromotor voor wie ik de eerste promovendus was. Jouw inhoudelijke
expertise, schrijfkunsten, parate kennis van de literatuur en werklust zijn een bron van
inspiratie. Veel dank voor je oneindige precisie en het immer secuur lezen van de
zoveelste versie van mijn artikelen.
Beste Jozé, op heldere wijze wist je onvolkomenheden in de structuur onder de
aandacht te brengen en mij te voorzien van scherpe analyses, zodat ik nieuwe stappen
kon zetten in het schrijfproces. Heel veel dank voor je inzet de afgelopen jaren.
Beste Gert, je volgde prof. Richard Grol op als directeur van IQ en als promotor,
hierdoor stapte je iets later in het traject. Je feedback was altijd constructief en dit
zorgde voor betere artikelen. Veel dank voor je adviezen en je tijd om dit tot een
succesvol einde te brengen.
Dear Glyn, many thanks for your efforts and time spend traveling to Nijmegen. It was a
privilege to work with you.
Beste Richard, kort na de start van mijn promotietraject besloot je dat ik niet bij de
club met promovendi zat die je nog ging begeleiden ná je emeritaat. Helaas, maar het
doet me genoegen dat ik met je heb mogen werken. Ik dank je voor de start van het
traject en het gestelde vertrouwen in mij.
Dit proefschrift had niet tot stand kunnen komen zonder de medewerking en financiële
steun van zorgverzekeraar CZ, dank voor de samenwerking en support. Speciale dank
gaat uit naar Wiro Gruisen, Jolyn van Vuuren, Marjolein Morres en Linda van Mierlo.
Een woord van dank gaat uit naar de poli’s Orthopedie van de deelnemende
ziekenhuizen Radboudumc, Slingeland Ziekenhuis en Elizabeth Ziekenhuis Eindhoven.
Externe co-auteurs: Martin Eccles, Katherine Deane, Signe Flottorp, Liv Rygh, Marten
Munneke en Bas Bloem, bedankt voor jullie bijdragen en correcties bij de betreffende
artikelen.
166
Dankwoord
Collega’s van IQ healthcare
Anita Huis, Hilly Calsbeek en Simone van Dulmen, mijn drie geweldige kamergenoten,
wil ik heel hartelijk bedanken voor het meeleven van de hele rit van begin tot eind.
Goede adviezen en praktische tips bij het organiseren en plannen van de
onderzoeksactiviteiten. Hilly en Simone, wat was het bijzonder om drie gelijktijdige
zwangerschappen met elkaar te delen, gevolgd door goede pedagogische adviezen bij
het jonge moederschap. Anita, jouw ‘24 uur-crisisdienst verhalen’ en bizarre voorvallen
bezorgden mij regelmatig de slappe lach. En wat ben ik blij dat onze goede traditie van
de ‘R&A etentjes’ blijven doorgaan. Maud Heinen en Betsie van Gaal, wat ontzettend
fijn dat jullie zijn aangeschoven, input leveren voor de ‘R&A’ en meedelen in deze
traditie. Hopelijk houden we die nog heel lang vast.
Onderzoeksondersteuning en praktische hulp kreeg ik van Marc Padros, Ellen Keizer en
Reinier Akkermans, veel dank voor jullie hulp! Alice Tillema, mijn dank is groot voor je
hulp om alle databases te doorzoeken en bij het opstellen van de juiste zoektermen.
Sylvia van Roosmalen, veel dank gaat uit naar je werk van het redigeren van mijn
artikelen en je inzet bij de totstandkoming van dit manuscript. Jolanda van Haren,
dankjewel voor je hulp, het meekijken en je adviezen in de totstandkoming van het
manuscript en uiteindelijke boekje.
Speciale dank gaat uit naar mijn intervisie collega’s van IQ; Anke Oerlemans, Karin
Neeleman-van der Steen, Geertje van de Ven, Gijs Hesselink en Irene van de Glind.
Irene, dank ook voor je tijd dat we samen medevoorzitter waren van Schil. Ik kijk uit
naar jouw promotie!
Renate Jansink en Kirsten Kirschner, jullie waren al bezig met jullie promoties toen ik
bij IQ kwam. We hadden meteen een ‘klik’ en ik blij dat we die ondanks de drukke
levens proberen vast te houden. Dank voor jullie meeleven en spreekwoordelijke
dansjes bij grote en kleine lichtpuntjes. Arna van Doorn–Klomberg wil ik bedanken voor
het prettig sparren en meelezen in een fase waarin het schrijven even op slot zat.
Renske Keizer, oud studiegenote van ASW, al lang gepromoveerd en al ruim een jaar
hoogleraar. Geweldig! Hartelijk dank voor je hulp en adviezen bij het schrijven van de
discussie.
Collega’s Windesheim
Mijn aanstelling bij Windesheim combineerde ik met de afronding van mijn
proefschrift. Heel fijn om herkenbare hobbels en heuglijke feiten met jullie te kunnen
delen. Speciale dank gaat uit naar Leontine Groen–van de Ven, Marijke Span, Monique
Mensen, Jan Jukema en Carolien Smits: jullie wijsheden en aanmoedigingen ‘Trust the
167
Dankwoord
process’ en ‘Shut up and write’ zorgden ervoor dat ik ondanks de drukke
werkzaamheden bij Windesheim moedig voortwaarts bleef gaan.
Collega’s Saxion
Geen bijdrage in het traject, maar wél de vreugde van de afronding vieren. Ik ben blij
dat ik met zo’n geweldige club mensen mag werken!
Paranimfen
Mirjam, het is niet meer haalbaar om te carpoolen nu we in Nijmegen en Enschede
werken, maar wat kijk ik goed terug op onze dagelijkse evaluatiemomenten in de auto.
Jouw nuchtere blik en pragmatische houding, gecombineerd met een
wetenschappelijke analyse is onovertroffen. Veel dank dat je vandaag naast mij wilt
staan.
Lieve Anne, onze levens zijn innig met elkaar verbonden. Mijn dierbare vriendin en
schoonzus waarmee ik keihard kan lachen, kan relaxen, kan mopperen over de
dagelijkse beslommeringen en met wie ik de meest ambitieuze plannen kan ontvouwen.
Wat een eer dat je naast mij staat!
Familie
Het afronden van een promotie naast een andere baan en een gezin heeft ook één en
ander gevraagd van de mensen die dicht bij mij staan.
Lieve schoonfamilie, Anne & Roy, Els & Joop, heel erg bedankt voor het intensief
meeleven, het blijven vragen en al die extra oppasmomenten van de afgelopen tijd!
Lieve zusjes en zwagers, Marloes & Frank, Janneke & Rick, we beleefden met elkaar de
afgelopen twee jaar op z’n zachtst gezegd ‘roerige tijden’. Dank voor alle gezellige
momenten en steun.
Lieve papa en mama, wat de toekomst brengen zal is nog ongewis, maar de basis die
jullie mij hebben meegegeven is van jullie samen. Veel dank voor jullie liefde en steun!
Mama, dank voor al die onzichtbare en ontelbare dingen die er in de afgelopen tijd
voor zorgden dat ik rustig heb kunnen schrijven.
Mijn stoere ‘mannen’, Jens en Lars, ik geniet volop van jullie en ben zó blij dat ik vanaf
nu meer tijd voor jullie heb.
Lieve Martijn, jouw steun is ontzettend belangrijk geweest om dit tot einde te kunnen
brengen. Dat alles hoeft wat jou betreft niet breed uitgesponnen te worden, dat weten
is al genoeg. Het levensmotto ‘het zijn de kleine dingen…’ past je als geen ander.
DANK!
168
Curriculum Vitae
169
Curriculum Vitae
170
Curriculum Vitae
Nicole Ketelaar werd geboren op 26 maart 1982 in
Doetinchem. Na haar VWO-diploma (2001) startte zij aan de
Universiteit Utrecht met de studie Algemene Sociale
Wetenschappen. Haar masterthesis werd uitgevoerd binnen
het Slingeland Ziekenhuis in Doetinchem en richtte zich op de
transmurale zorg, in het bijzonder op de overdracht van
farmaceutische zorg van intra- naar extramurale setting. Zij
behaalde haar Master binnen de afstudeerrichting Cultuur,
Zorg en Welzijn in 2005.
Na haar studie werkte zij korte tijd (2006-2007) als
beleidsmedewerker in het Facilitair Bedrijf van het UMC
Utrecht. Zij combineerde dit met een aanstelling als
interviewer bij de stichting ‘Cliënt en Kwaliteit’. In 2007
maakte zij de overstap naar het Radboudumc, bij het Scientific Institute for Quality of
Healthcare (IQ healthcare). Halverwege 2008 startte haar promotieonderzoek, ‘De
Kiezende Zorggebruiker’, waarvan de resultaten in dit proefschrift beschreven zijn. Dit
project werd gesubsidieerd door het CZ Fonds. Naast haar promotieonderzoek was zij
betrokken bij verschillende onderzoeksprojecten en schreef zij mee aan een handboek
‘Doen bij Depressie’.
In het najaar van 2012 werd Nijmegen verruild voor Zwolle en werkte zij als
onderzoeker/docent aan de Hogeschool Windesheim, Lectoraat Innoveren in de
Ouderenzorg. In samenwerking met de Hogeschool Arnhem en Nijmegen werden er
praktische tools ontwikkeld voor casemanagers dementie binnen drie dementienetwerken. Momenteel is zij als onderzoeker verbonden aan de Hogeschool Saxion,
Academie Mens en Maatschappij, waar zij een onderzoek uitvoert naar de transitie van
het sociale domein en sociale wijkteams binnen drie Twentse gemeenten. Naast dit
onderzoeksproject neemt zij als onderzoeker deel aan het Expertise Centrum
Jeugdzorg Twente (EJT) en is zij als hoofddocent betrokken bij de Master Health Care
and Social Work.
Nicole woont samen met Martijn Kraan. Zij zijn de trotse ouders van Jens (5) en
Lars (2).
171
Portfolio Radboud Institute for Health Sciences
174
Name PhD student: N.A.B.M. Ketelaar
Department: Scientific Institute for Quality of
Health
Graduate School: Radboud Institute for Health
Sciences
PhD period: 01-06-2008 – 02-12-2015
Promotor(s):
Prof. G.P. Westert, Prof. G. Elwyn
Co-promotor(s):
Dr M.J. Faber, Dr J.C. Braspenning
TRAINING ACTIVITIES
a) Courses & Workshops
‐ Herregistratie Basiscursus Regelgeving en Organisatie voor
Klinisch onderzoekers (BROK)
‐ CaRe annual day ‘International collaboration in Primary
Health Care Research’
‐ Solliciteren en netwerken, Radboud University
‐ IQ healthcare congress, Radboudumc
‐ Masterclass Lecture: ‘Planning for future career’
‐ Academic writing (one-one-coaching), Radboud in’to
languages
‐ Cochrane review / review manager, EMGO
‐ Hulp bij subsidieaanvragen, ZonMw
‐ Instruction about literature searches
‐ Advanced conversation, Radboud in’to languages
‐ IQ healthcare congress, Radboudumc
‐ Presenteren eigen onderzoek, Radboud University
‐ Basiscursus Regelgeving en Organisatie voor Klinisch
onderzoekers (BROK)
‐ Cursus SPSS, PAO Heyendael
‐ Academic writing, Radboud in’to languages
‐ Introductie cursus CaRe
‐ PubMed introduction, Medical Library
‐ Posterpresentatie / opzetten van wetenschappelijke posters,
NCEBP
‐ Implementatie. Effectieve strategieën, IQ healthcare
‐ Wetenschapsjournalistiek, Radboud University
b) Seminars & lectures^
‐ Netwerkbijeenkomst Bruikbaar Onderzoek, ZonMw
‐ Patient Preferences in quality improvement research
(seminar) UMC St Radboud. ^ presentation
‐ Onderzoek naar keuze van patiënten met de ziekte van
Parkinson voor een gespecialiseerde fysiotherapeut. UMC St
Radboud, afdeling neurologie. ^ presentatie
‐ Zijn er effecten door het gebruik van kwaliteitsinformatie?
Zorgverzekeraar CZ, Tilburg.
‐ Wat is er al bekend over de kiezende zorggebruikers?
Onderzoeksopzet. IQ healthcare. ^ presentatie
Year(s)
ECTS
2013
0.1
2012
0.25
2012
2012
2011
2011
1.75
0.25
0.1
2
2010
2010
2010
2010
0.2
0.1
0.1
2
0.25
1.5
1.75
2010
2009
2009
2009
2009
2009
2008
2008
0.2
3
0.5
0.1
0.1
2007 - 2008
2007
2
2
2013
2011
0.1
0.25
2011
0.25
2011
0.1
2008
0.1
175
Year(s)
ECTS
2015
0.25
2011
0.5
2009
0.5
2008 - 2010
0.2
2008
1.5
2014
2011 - 2012
2012 - 2015
0.1
1
0.7
2011
2009
2008
2007
1
1
2
0.5
TRAINING ACTIVITIES
c) Symposia & congresses^
‐ Vascular Symposium, ‘Innovation in time of scarcity. Think
‘evidence’ and ‘quality’, University Hospitals Leuven, Belgium
^ presentation
‐ International Shared Decision Making Conference (ISDM),
University Maastricht, Maastricht ^ presentation
‐ International Shared Decision Making Conference (ISDM),
Harvard Medical School, Boston ^ poster
‐ Visiting several congress regarding the subject ‘Choosing
healthcare providers’ (e.e.g. Ede, Erasmus University)
^ visiting
‐ Co-organisation symposia: 5 jaar samenwerking van IQ
healthcare en Zorgverzekeraar CZ
d) Other
‐ Te gast bij journal club van NIVEL
‐ Intervisie met promovendi
‐ Review scientific publication, journals: Health Expectations,
Health Services Research, Plos One, International Journal of
Quality in Healthcare Care, BMC Medical Informatics and
Decision Making, Evaluation of Clinical Practice
‐ Lid van Kennisgroep ‘Patient Empowerment’
‐ Lid van journal club
‐ Voorziter van Schil
‐ Lid van Schil. Overleg orgaan voor junior onderzoekers en
promovendi binnen IQ healthcare
TEACHING ACTIVITIES
e) Lecturing
‐ Kiezen in Zorg. Transparantie en kwaliteitsinformatie. UMC St
Radboud. Onderwijs voor geneeskunde studenten.
f) Supervision of internships / other
‐
TOTAL
^Indicate oral or poster presentation
176
2011
-
2012
2011
2010
2012
0.4
28.4