View metadata, citation and similar papers at core.ac.uk brought to you by CORE provided by Radboud Repository PDF hosted at the Radboud Repository of the Radboud University Nijmegen The following full text is a publisher's version. For additional information about this publication click this link. http://hdl.handle.net/2066/147494 Please be advised that this information was generated on 2018-07-07 and may be subject to change. For reasons of consistency within this thesis, some terms have been standardized throughout the text. As a consequence the text may differ in this respect from the articles that have been published. The studies presented in this thesis have been performed at the Scientific Institute for Quality of Healthcare (IQ healthcare). This institute is part of the Radboud Institute for Health Sciences (RIHS), one of the approved research institutes of the Radboud university medical center. The studies in this thesis were financially supported by the CZ Fund [grant AFVV08‐156AFVV08‐156]. Financial support by IQ healthcare for the publication of this thesis is gratefully acknowledged. ISBN: 9789462799905 Nijmegen, 2015 Cover: Manon den Hartog, Omdat Ontwerp Lay‐out: Jolanda van Haren / Nicole Ketelaar Print: GVO drukkers en vormgevers B.V. | Ponsen & Looijen, Ede Choosing healthcare providers Healthcare consumers’ use of comparative performance information Proefschrift ter verkrijging van de graad van doctor aan de Radboud Universiteit Nijmegen op gezag van de rector magnificus volgens besluit van het college van decanen in het openbaar te verdedigen op woensdag 2 december 2015 om 10.30 uur precies door Nicole Antonia Berendina Maria Ketelaar geboren op 26 maart 1982 te Doetinchem Promotoren: Prof. dr. G.P. Westert Prof. dr. G. Elwyn (Dartmouth College, Verenigde Staten) Copromotoren: Dr. J.C.C. Braspenning Dr. M.J. Faber Manuscriptcommissie: prof. dr. J.W.A. Smit prof. dr. W.J.J. Assendelft prof. dr. D. Delnoij (Tilburg University) Contents Chapter Title page Chapter 1 Introduction Chapter 2 Exploring consumer values of comparative performance information for hospital choice Quality in Primary Care 2014; 22(2): 81-9. 19 Chapter 3 Patients’ expectations of variation in quality of care relates to their search for comparative performance information BMC Health Services Research 2014; 14: 617. 33 Chapter 4 Recognition of physiotherapists’ expertise in Parkinson’s disease BMC Health Services Research 2013; 13: 430. 47 Chapter 5 Public release of performance data in changing the behaviour of healthcare consumers, professionals or organizations (Review) Cochrane Database Systematic Review 2011; (11). 63 Chapter 6 Comparative performance information plays no role in the referral behaviour of GPs BMC Family Practice 2014; 15(1): 146. 119 Chapter 7 General discussion 135 Summary 153 Samenvatting 159 Dankwoord 165 Curriculum Vitae 169 Portfolio Radboud Institute for Health Sciences 173 7 Chapter 1 Introduction Chapter 1 INTRODUCTION Patients are increasingly referred to as ‘healthcare consumers’, which acknowledges their active position when they are making choices.1 This position has become much more prominent since the introduction of demand-driven care and market-based forces in the current healthcare system in the Netherlands. An emancipatory movement emphasizes an active position as well, which the increased focus on patients’ interests, needs, and attributes in the healthcare system reflects.2 Concepts in which the role of patients is empathetically defined and acknowledged are patient-centredness, shared decision-making and demand-driven care.3,4 In demand-driven healthcare systems consumers are expected to have an active role and to choose healthcare providers that offer high quality and ignore the ones that provide low quality.5,6 If they do so, consumers can influence the average levels of quality of care, provided that they act as rational consumers.7,8 However, as decisions become increasingly complicated and consequential, the assumption of a rational consumer becomes less realistic. Marshall argues that consumers take more time to get the best deal for a hotel stay or a car than for the best healthcare setting.9 Even so, encouraging consumers to actively choose a provider can be considered a goal in itself because it enhances patient empowerment, consumer activation, and selfdetermination. Active provider choice for consumers leads to an urgent demand for accessible, comprehensible information about differences in the performance of healthcare providers10-12, the comparative performance information (CPI). Other stakeholders need transparency of the quality of the hospital or the provider’s performance as well. Payers would like to use CPI for purchasing care based on quality instead of quantity; the inspectorate, for monitoring the minimum level of care delivered; and the healthcare services and providers, for improving quality of care processes. Whether all these goals can be reached simultaneously remains to be seen. However, in this thesis, the focus is on the 1 healthcare consumer’s use of CPI. The basic assumption is that healthcare consumers like to make their own choices in healthcare. The existing literature shows that they want more information about the quality of care.13,14 Nevertheless, the choice of providers does not seem to be a high priority for consumers.15 While a growing mass of CPI is freely available, consumers hardly use this information to choose their healthcare provider.11,16-21 At least three relevant topics have emerged from the healthcare reforms and the provision of CPI to healthcare consumers. First of all, why a majority of patients ignore CPI is unknown. Some research into the presentation of CPI and the patients’ 8 Introduction understanding of it has been carried out 17,22-24, but little is known about the search drivers and factors that affect it. Insight into the design and dissemination is needed to overcome these barriers to valuable usage. Second, CPI can empower patients in their choice of providers, but how much this affects actual behaviour is still unclear.25 Third, healthcare consumers probably need help in interpreting the CPI. CPI has been described as not easily accessible.10 Perhaps structural guidance, e.g. from healthcare professionals (GPs), can increase the use of CPI. This thesis studies healthcare consumers’ limited search for and use of CPI for the purpose of choosing a healthcare provider. The introduction briefly describes the changing position of healthcare consumers, provider choice in the healthcare system, the kind of the comparative performance information (CPI) available for consumers, and a further description of the three research themes. This chapter ends with the set of research questions that have been studied in the five consecutive studies. Provider choice in healthcare systems The way provider choice is structured in a healthcare system directly affects the consumer’s actual freedom of choice. There are several ways to organise provider choice. One option is to regulate provider choice either via a gatekeeper system or via health plans. Another option is free provider choice and patient access to specialised care without any interference. Countries differ in this organizational aspect of the healthcare system. Within Europe, the regulated form arranged by the gatekeeper system and free provider choice are common. The UK and the Netherlands have strong systems of primary-care gate-keeping, which means that the general practitioner (GP) has an important role in referral and the choice of a provider.26 As a result, consumers cannot easily consult a professional or a specialist, or visit a hospital, without seeing a GP first. In Sweden, provider choice is based on agreements between the government and the federation of county councils.27 In Germany, patients have free access to specialists and hospitals of choice. In France, patients are free to visit any GP or specialist practising privately or in a hospital.28 In the USA, the gatekeeper system also plays a role, but in a different way, as provider choice is mainly regulated by health plans.29 Healthcare consumers choose a health plan and pay for the care within the network. Plans that restrict provider choice cost less, while flexible plans with an extension of provider choice cost more.30 A second form of the American gatekeeper system demands that patients see their GP first to avoid excessive out-of-pocket payment charged by their health plan. Provider choice has changed in Western countries (including the USA, UK, Denmark, Norway, the Netherlands, and Sweden) in recent years, in favour of free provider choice. The wish to empower and activate consumers in healthcare goes hand in hand 9 1 Chapter 1 with these developments. The Dutch healthcare system has been actively encouraging consumers to choose a provider since 2006. The Health Insurance Act (HIA) made it possible for healthcare consumers to choose their preferred type of health insurance and their own healthcare providers. The GPs continued to act as the gatekeepers of specialised care. However, dental hygienists and physiotherapists have been freely accessible to consumers without an intervening consultation with a GP since the implementation of the HIA.31 Box 1 describes how provider choice has been expanded.32 Box 1. Important elements of the Dutch healthcare system after the introduction of the Health Insurance Act in 2006 Important elements of the Dutch healthcare system:  Healthcare insurance is now obligatory for every Dutch resident  The distinction between private and public healthcare insurance disappeared. All 18+ have to buy private insurance  Provider choice for healthcare consumers has expanded  Healthcare insurers are authorised to selectively contract providers  Healthcare organisations must inform the public about price, quality, and other characteristics of care  Both consumers’ choices and the insurers’ selective contracts are seen as main drivers in healthcare providers’ improvement of the quality of care at lower costs The wish to empower and activate patients in healthcare goes hand in hand with the societal demand for accountability in healthcare, transparency about the quality of hospital care, and provider performance.33,34 More information about the quality of care is becoming available in the public domain, not only for consumers, but also for other end-users: the health inspectorate, healthcare providers, and insurance companies. Each group has its own goal in using this type of information, and the question is whether these goals are compatible (Box 2).25 Box 2. Users and goals of different end-users of comparative performance information, based on Smith et al., 2009 Users of comparative performance information Government Health inspectorate Healthcare providers Health insurance Patients and healthcare consumers 10 Goals in using comparative performance information  Measuring the quality of healthcare  Monitoring regulatory effectiveness and efficiency  Ensuring the development of comparative performance information  Starting healthcare improvement projects  Selectively referring to high-quality providers  Selectively contracting the best health provider balanced against low cost  Addressing information asymmetry  Making an informed provider choice  Selecting high-quality providers Introduction Theoretically, there are a number of pathways through which the public release of CPI may have an impact on providers and the quality of care.35 Berwick and colleagues’ framework shows two pathways. The ‘pathway of selection’ implies that healthcare consumers drive healthcare quality improvement by using the release of comparative performance data to make choices between hospitals or providers. Consumers will identify the value of the outcomes, search for performance information about a healthcare provider, and then make an informed choice. The assumption is that they will try to obtain the best quality or avoid the worst, and select (or reward or recognise) the best hospital or provider (Figure 1). In the ‘pathway of change’, the extrinsic motivation is necessary to effect changes. Change may occur due to pressure to avoid being identified as a provider of poor-quality care or to improve a provider’s reputation in relation to other providers.36 Figure 1. Two pathways for improving performance by means of the public release of performance data Publicly reported performance data Knowledge Selection Pathway 1 Motivation Change Pathway 2 Performance The ‘pathway of change’ assumes that benchmarking healthcare organisations will encourage hospitals and healthcare providers to improve their performance. The pathways are strongly connected to one other, and for their effects, both mechanisms rely on the availability of reliable performance information that all users of the healthcare system can use and act upon. We include both pathways, with an emphasis on the pathway of selection. Comparative performance information Many different terms have been introduced to refer to information about the quality of the care delivered: comparative quality information, performance information, quality information, and healthcare information. This thesis uses the term ‘comparative performance information’ (CPI) within the following definition: “Systematically collected, publicly available information about the performance of health organizations or healthcare providers that can be compared objectively. The purpose of CPI is to inform and enable healthcare consumers to make their choices based on their own values” 11 1 Chapter 1 This kind of information has been developed in many countries, especially in the USA and the UK, where the production and dissemination of such information has been high on the agenda since the 1980s.37,38 One can differentiate three types of CPI: information about healthcare provider characteristics and services, experiences of healthcare consumers, and performance indicators (Table 1).39 Table 1. Types of comparative performance information Type 1 2 3 Description Information about healthcare provider characteristics and services Information from the healthcare consumer’s perspective and experience Performance indicators (process, structure, or outcome indicators) Examples Travelling distance, region, type of provider, provider specialties, treatment specialties, costs, waiting times, number of hospital beds, and personnel Experiences with treatment in a healthcare organisation (e.g. hospital and nursing home), satisfaction with food, privacy, communication, and the consumer quality index Hip-fracture patients having surgery 24 hours after admission (process), number of patients with pressure wounds, treatment volumes, methods of anaesthesia, and the hospital standardised mortality ratio These different types of information are often based on various sources and collected by a range of instruments. Data can be collected from healthcare providers, patients, and insurance companies. The instruments can be observation lists, medical records, interview protocols, or surveys. The Dutch government initiated placing CPI on websites, including www.kiesbeter.nl (make better choices) in 2005. This website includes quality information and assessments of individual hospital performance. It started with mainly a combination of types 1 and 2, but in recent years type 3 has also been presented on this website (Table 1). Other Dutch websites also offer comparisons of hospitals and other healthcare services (www.independer.nl, www.mediquest.nl, and www.zorgkaartnederland.nl). The Dutch daily newspaper Algemeen Dagblad produces a list of the 100 best Dutch hospitals every year, and the weekly magazine Elsevier publishes a list of best hospitals; their assessments are based on various indicators. Three emerging research themes of comparative performance information Search drivers and the possible determinants Discussions about selective provider choice usually start with the assumption that healthcare consumers are keen to choose their own healthcare providers. Researchers uncritically adopt this assumption, and then begin assessing the availability of the right information, the content of such information, and the influence of the way the CPI is presented.10,15,40 These elements are, without doubt, important. The debate focuses on 12 Introduction understanding how to present quality information in ways that are meaningful to consumers.38 However, the basic barriers to consumers using and trusting CPI are underexposed. The fundamental questions of how consumers perceive, value, and appraise CPI for the purpose of choosing healthcare providers are often neglected and poorly understood. An important factor that could influence the value of CPI, and therefore the search driver, is the patients’ recognition of quality-of-care differences. For this thesis, we studied whether expectations about the quality of care differences had an impact on the CPI search behaviour of patients. Since the context of the healthcare can affect the results, we did one study about elective surgery in a hospital and another about the chronic care setting in primary care. The impact of comparative performance information on behaviour From the time that CPI was first published, there have been doubts whether the consumers would use this information and how it would impact their choice behaviour.37 Research seems to confirm these doubts: it shows that few consumers actually use CPI. For example, the proportion of Americans using comparative data about physicians grew from 4% to 6% between 1996 and 2008.18 Research highlighted several barriers to the use of CPI. Examples are people’s distrust of hospital ranking and the reliability of health information on the internet, in the media, or from the government. Consequently, the relevance of CPI consistently ranks far below the information from family and friends.11,20,41 Experiences of family and friends reported by word of mouth have more impact than objective, measurable performance data.42,43 Nevertheless, there is still a firm belief that CPI will result in better informed decisionmaking. The literature does not support this hypothesis. Except for studies that used hypothetical scenarios to examine the effect of public reporting,23,44-48 when this research project began in 2009 there were still no systematic data available about the effects of public reporting of the actual behaviour regarding the selection of healthcare providers, quality improvement, and patient outcomes.49 Support of the consumer’s choice behaviour The literature teaches us that not all consumers have the same competencies to deal with complex choices50; consumers may need help to interpret such information. The availability of someone who provides additional choice expertise can help consumers choose.51 Because the primary-care system has a strong position in Dutch healthcare and Dutch GPs traditionally act as gatekeepers52, the GPs would be the logical partners to help Dutch consumers make choices. They can help consumers exercise their patient rights by offering a respectful relationship.2 Research from the UK indicates that GPs are sceptical about the use of objective information and prefer to rely on soft knowledge such as relationships with providers and the experiences of patients whom they had referred previously.53 However, it is not yet clear how GPs view their choice13 1 Chapter 1 supporting role when it comes to using CPI in consultation time, and in which ways they could help consumers. OBJECTIVES AND AIM OF THIS THESIS The aim of this thesis is to make a contribution to the empirical evidence about how consumers use CPI and how they choose a provider. The main focus is to make the perception of the consumer visible. The research questions for the three research themes that this thesis addresses are:  What are the barriers and facilitators for healthcare consumers to search and use CPI?  To what extent does publicly released performance information changes the choice behaviour of healthcare consumers, providers, and purchasers?  To what extent do GPs advocate and encourage patients to choose providers on the basis of CPI? The research focuses on the effects of public reporting, and the aim is to discover the conditions that either constrain or facilitate consumers’ searches and use of information that ultimately helps them choose a healthcare provider. In particular, we focus on:  Identifying the basic values, attitudes, and ideas regarding CPI, as well as choice behaviour among healthcare consumers;  Exploring whether expectations of quality of care affects consumers’ behaviour in their search for CPI;  Testing whether expectations of performance and recognition of healthcare providers’ expertise can foster the search for CPI and selective provider choice;  Assessing how the effects of public reporting changes the behaviour of consumers, professionals, healthcare organisations and purchasers of care (insurance companies);  Exploring GPs’ views of the use of CPI at the point of referral in primary care and their views of their own role of choice-supporting behaviour. OUTLINE OF THIS THESIS Chapter 2 reports the results of six focus groups with healthcare consumers. The main goal of this study is to clarify the value that CPI gives the consumers. Most research in this field is oriented towards the content and the presentation format of CPI, while little is known about how consumers value CPI and use this information. This chapter considers the awareness of consumers, current sources of decision-making, the value of CPI, and the effects of CPI. 14 Introduction Chapters 3 and 4 report how consumers’ awareness of the availability of variation in quality of care affects their use of CPI to find a hospital or healthcare provider for elective surgery or a progressive chronic-care condition. Chapter 3 deals with patient expectations about the variation in quality of care. Postal questionnaires were sent to 475 patients undergoing a total hip or knee replacement in three types of hospitals to obtain the necessary data. The objective of Chapter 4 is to explore the ability of people with Parkinson’s disease to recognise expertise among physiotherapists and to determine to what extent patients selectively choose expert physiotherapists. Chapter 5 describes a systematic review of the public release of performance data. The review concerns the content and effectiveness of determinants of behaviour change that prompt the actual behaviour in choosing. Using the suggestion to provide an advocate who can help consumers with their choice of hospital, Chapter 6 investigates how GPs view their role in using CPI during consultation time. The general discussion in Chapter 7 summarises the results in this thesis and discusses our findings in view of several methodological issues, implications for practice, and aims for future research. 15 1 Chapter 1 REFERENCES 1. 2. 3. 4. 5. 6. 7. 8. 9. 10. 11. 12. 13. 14. 15. 16. 17. 18. 19. 20. 21. 22. 23. 24. 16 Williamson C. The patient movement as an emancipation movement. Health Expect 2008;11(2):102-12. Dieterich A. The modern patient - threat or promise? Physicians' perspectives on patients’ changing attributes. Pat Educ Couns 2007;67(3):279-85. de Haes H. Dilemmas in patient centeredness and shared decision making: a case for vulnerability. Pat Educ Couns 2006;62(3):291-98. Elwyn G. Idealistic, impractical, impossible? Shared decision making in the real world. Br J Gen Pract 2006;56(527):403-4. Mukamel DB, Weimer DL, Zwanziger J, Gorthy SF, Mushlin AI. Quality report cards, selection of cardiac surgeons, and racial disparities: a study of the publication of the New York State Cardiac Surgery Reports. Inquiry 2004;41(4):435-46. Rothberg MB, Morsi E, Benjamin EM, Pekow PS, Lindenauer PK. Choosing the best hospital: the limitations of public quality reporting. Health Affairs 2008;27(6):1680-87. Botti S, Iyengar SS. The Dark Side of Choice: When Choice Impairs Social Welfare. J Public Policy Mark 2006;25(1):24-38. Glazer J, McGuire TG. Optimal quality reporting in markets for health plans. J Health Econ 2006;25(2):295-310. Marshall M, McLoughlin V. How do patients use information on health providers? BMJ 2010;341:c5272. Hibbard JH, Peters E. Supporting informed consumer health care decisions: data presentation approaches that facilitate the use of information in choice. Ann Rev Public Health 2003;24:413-33. Magee H, Davis LJ, Coulter A. Public views on healthcare performance indicators and patient choice. J Roy Soc Med 2003;96(7):338-42. Shaller D, Sofaer S, Findlay SD, Hibbard JH, Lansky D, Delbanco S. Consumers and quality-driven health care: a call to action. Health Affairs 2003;22(2):95-101. Fotaki M, Boyd A, Smits L, McDonald R, Sheaff R, Edwards A, et al. Patient Choice and the Organisation and Delivery of Health Services: Scoping review. University of Manchester: Report for the National Co-ordinating Centre for NHS Service Delivery and Organisation R&D (NCCSDO), 2005. Damman OC, Hendriks M, Rademakers J, Spreeuwenberg P, Delnoij DM, Groenewegen PP. Consumers’ interpretation and use of comparative information on the quality of health care: the effect of presentation approaches. Health Expect 2012;15(2):197-211. Fung CH, Lim YW, Mattke S, Damberg C, Shekelle PG. Systematic review: the evidence that publishing patient care performance data improves quality of care. Ann Intern Med 2008;148(2):111-23. Fotaki M, Roland M, Boyd A, McDonald R, Scheaff R, Smith L. What benefits will choice bring to patients? Literature review and assessment of implications. J Health Serv Res Policy 2008;13(3):178-84. Harris-Kojetin LD, Uhrig JD, Williams P, Bann C, Frentzel EM, McCormack L, et al. The “choose with care system” - development of education materials to support informed Medicare health plan choices. J Health Commun 2007;12(2):133-56. Kaiser Family Foundation. Update on consumers’ view of patient safety and quality information. Menlo Park, California: Agency for Healthcare Research and Quality, 2008. Schneider EC, Epstein AM. Use of public performance reports: a survey of patients undergoing cardiac surgery JAMA 1998;279(20):1638-42. Schwartz LM, Woloshin S, Birkmeyer JD. How do elderly patients decide where to go for major surgery? Telephone interview survey. BMJ 2005;331(7520):821-24. Werner RM, Asch DA. The unintended consequences of publicly reporting quality information. JAMA 2005;293(10):1239-44. Damman OC. Public reporting about healthcare users’ experiences. Utrecht: University of Tilburg, 2010. Peters E, Dieckmann N, Dixon A, Hibbard JH, Mertz CK. Less is more in presenting quality information to consumers. Med Care Res Rev 2007;64(2):169-90. Fasolo B, Reutskaja E, Dixon A, Boyce T. Helping patients choose: how to improve the design of comparative scorecards of hospital quality. Pat Educ Couns 2010;78(3):344-49. Introduction 25. 26. 27. 28. 29. 30. 31. 32. 33. 34. 35. 36. 37. 38. 39. 40. 41. 42. 43. 44. 45. 46. 47. 48. 49. 50. 51. Smith PC, Mossialos E, Papanicolas I, Leatherman S. Performance measurement for health system improvement. Experiences, challenges and prospects. Cambridge: University Press Cambridge, 2009. Dixon A, Robertson R, Bal R. The experience of implementing choice at point of referral: a comparison of the Netherlands and England. Health Econ Policy Law 2010;5(3):295-317. Thomson S, Dixon A. Choices in health care: the European experience. J Health Serv Res Policy 2006;11(3):167-71. Tuffs A. Germany plans to make GPs gatekeepers. BMJ 1999;318(7189):961. Harris-Kojetin LD, Stone RI. The role of consumer satisfaction in ensuring quality long-term care: opportunities and challenges. J Aging Soc Policy 2007;19(2):43-61. Medline Plus. Trusted Health Information for you http://www.nlm.nih.gov/medlineplus2014 [3003-2014]. Groenewoud AS. It’s your Choice! A study of search and selection processes, and the use of performance indicators in different patient groups. Veenendaal: Erasmus University Rotterdam, 2008. Westert GP, Burgers JS, Verkleij H. The Netherlands: regulated competition behind the dykes? BMJ 2009;339:b3397. Marshall MN, Shekelle PG, Leatherman S, Brook RH. Public disclosure of performance data: learning from the US experience. Qual Health Care 2000;9(1):53-57. Sinaiko AD, Eastman D, Rosenthal MB. How report cards on physicians, physician groups, and hospitals can have greater impact on consumer choices. Health Affairs 2012;31(3):602-11. Berwick DM, James B, Coye MJ. Connections between quality measurement and improvement. Med Care 2003;41(1 Suppl):I30-38. Shekelle PG, Lim YW, Soeren M, Damberg C. Does public release of performance results improve quality of care? Southern California: The Health Foundation, 2008. Marshall MN, Romano PS, Davies HT. How do we maximize the impact of the public reporting of quality of care? Int J Qual Health Care 2004;16(Suppl 1):i57-63. Marshall MN, Shekelle PG, Davies HT, Smith PC. Public reporting on quality in the United States and the United Kingdom. Health Affairs 2003;22(3):134-48. Van Loon AJM, Tolboom RAL. Design report “Choose Better” 2006-2007 [In Dutch: Ontwerprapport KiesBeter.nl 2006-2007]. Bilthoven: RIVM, 2005. Stein BD, Kogan JN, Essock S, Fudurich S. Views of mental health care consumers on public reporting of information on provider performance. Psychiatric Services 2009;60(5):689-92. RAND Health. Consumers and health care quality information: need, availability, utility. Oakland, CA: California Healthcare Foundation, 2001. Coulter A. Do patients want a choice and does it work? BMJ 2010;341:c4989. Harris KM, Beeuwkes Buntin M, The RAND Corporation. Choosing a health provider: the role of quality information. Princeton: Robert Wood Johnson Foundation, 2008. Hibbard JH, Slovic P, Peters E, Finucane ML. Strategies for reporting health plan performance information to consumers: evidence from controlled studies. Health Serv Res 2002;37(2):291-313. Schoenbaum M, Spranca M, Elliott M, Bhattacharya J, Short PF. Health plan choice and information about out-of-pocket costs: an experimental analysis. Inquiry 2001;38(1):35-48. Spranca M, Kanouse DE, Elliott M, Short PF, Farley DO, Hays RD. Do consumer reports of health plan quality affect health plan selection? Health Serv Res 2000;35(5 Pt 1):933-47. Uhrig JD, Harris-Kojetin L, Bann C, Kuo TM. Do content and format affect older consumers’ use of comparative information in a Medicare health plan choice? Results from a controlled experiment. Med Care Res Rev 2006;63(6):701-18. Uhrig JD, Short PF. Testing the effect of quality reports on the health plan choices of Medicare beneficiaries. Inquiry 2002;39(4):355-71. Shekelle PG. Public performance reporting on quality information. In: Smith PC, Mossialos E, Papanicolas I, Leatherman S (eds). Performance measurement for health system improvement: Experiences, challenges and prospects Cambridge: University Press Cambridge, 2009. Hibbard JH, Peters E, Dixon A, Tusler M. Consumer competencies and the use of comparative quality information: it isn’t just about literacy. Med Care Res Rev 2007;64(4):379-94. Schlesinger M. Choice cuts: parsing policymakers’ pursuit of patient empowerment from an individual perspective. Health Econ Policy Law 2010;5(3):365-87. 17 1 Chapter 1 52. 53. 18 Berendsen AJ, Kuiken A, Benneker WH, Meyboom-de Jong B, Voorn TB, Schuling J. How do general practitioners and specialists value their mutual communication? A survey. BMC Health Serv Res 2009;9:143. Rozen R, Florin D, Hutt R. An anatomy of GP referral decisions. A qualitative study of GP's view on their role in supporting patient choice. London: The King’s Fund, 2007. Chapter 2 Exploring consumer values of comparative performance information for hospital choice Nicole Ketelaar Marjan Faber Gert Westert Glyn Elwyn Jozé Braspenning Quality in Primary Care 2014;22(2):81–9. Chapter 2 ABSTRACT Background: In many countries, market orientation in healthcare has resulted in the publication of comparative performance information (CPI). Most of the research in this field is oriented towards the content and the presentation format of CPI while little is known about how consumers value CPI and the use of this information. Aim: The aim of this study was to clarify the perceived value that CPI brings for consumers of healthcare. Methods: Qualitative research using six focus group interviews. Twenty-seven healthcare consumers were recruited using a mailing list and by personal invitation. Data from focus group interviews were transcribed and thematic analysis undertaken. Results: Most participants were unaware of CPI, and valued alternative sources of information more than CPI. Through discussion with other consumers and by means of examples of CPI, respondents were able to express the values and perceived effects of CPI. Numerous underlying values hindered consumers’ use of CPI, and therefore clarification of consumer values gave insights into the current nonusage of CPI. Conclusions: CPI is marginally valued, partly because of conflicting values expressed by consumers and, as such, it does not yet provide a useful information source on hospital choice beyond consumers’ current selection routines in healthcare. Future research should be more focused on the values of consumers and their impact on the use of CPI. 20 Exploring consumer values of comparative performance information BACKGROUND Provider choice for healthcare consumers has emerged as a key policy focus in modern Western healthcare systems. The architects of the policy outlined several reasons to promote choice among healthcare consumers. First, to encourage competition between providers which was expected to improve the efficiency and quality of healthcare;1,2 and second, to increase patient empowerment and improve the position of consumers in healthcare.3,4 It is based on the expectation that patients will choose the best healthcare provider once they are informed. Systematically collected, publically available information about the performance of healthcare providers, called comparative performance information (CPI), can be used as a tool to inform healthcare consumers to enable them to make informed provider choices. The information consists of, for example, patient volumes, treatment methods, waiting lists and patient experiences, and can be found in leaflets, magazines, books and on the internet. Despite the efforts and resources that go into the collection, production and dissemination of information for the public,5 there is little evidence that CPI affects consumer choices. Many people do not understand the information, or do not view the information as useful.6 British and American studies have shown that as few as 5–7% of consumers use CPI to choose a hospital7,8 and only 12% use CPI to select a primary care physician,9 although awareness of the right to choose a provider has increased. The overall conclusion is that consumer usage of CPI to make an informed choice is still limited. Instead, consumers rely on information from family and friends or base their choice of hospital on recommendations from their referring primary care physician.10,11 The choice is largely driven by familiarity with a certain hospital12,13 or the distance between home and hospital.12,14 Using CPI is also a difficult and complex task for consumers, which limits its impact on consumer choice even more.14,15 Because provider choice by healthcare consumers is positively associated with autonomy and self-determination, both important principles of patient activation,16 the need to explain the limited exercise of choice by healthcare consumers might improve our understanding of active patient participation in healthcare. Previous studies have questioned whether this limited usage of CPI is caused by the absence of the right information content17 or whether CPI presentation formats do not support decision-making tasks.7 Evidence for both assumptions has been found. Damman and colleagues conclude that the presentation of CPI facilitates consumers’ correct interpretation as well as effective use, meaning the ability to choose the best performing provider.18 Although these are without doubt crucial elements in facilitating 21 2 Chapter 2 consumers in using such information, fundamental questions over how CPI is perceived, valued and appraised by healthcare consumers for their choice of hospital are overlooked. Some theory and evidence suggest that healthcare consumers are insufficiently informed about what is important for them.5,19 Current shared decisionmaking models for clinical treatment acknowledge the need to clarify patients’ values to promote active patient participation in decision-making.20,21 The aim of this study was to identify consumers’ values, appraisal, understanding, opinions and judgement of CPI. We conducted a series of focus group interviews. METHODS Approach The current study was a qualitative focus group study in 2009. The method allows researchers to capitalise on communication between participants to generate pertinent information.22 The focus group method was chosen because of its flexibility to explore unanticipated issues, and to make use of the interaction between group members. We encouraged the participants to discuss the subject of CPI, ask each other questions, exchange opinions and views, and share experiences.23 We presented the ‘values’ of CPI initially using terms such as ‘meaning’, ‘sense’, ‘ideals’ and ‘principles’ to explain their meaning for healthcare consumers. We introduced three real-world CPI examples to align the mindset of the participants in the discussion (Table 1). The example shown in Table 1 combined stars with percentages, the second used different kinds of bullets, and the third one showed the availability of a service using ‘yes’ or ‘no’. Table 1. Example of comparative performance information related to three quality indicators of hospital A, B and C Quality indicator Hospital A Hospital B Hospital C Cancelled surgeries (%)a 1.2  3.3  0.7  Pressure ulcers (%)b 8.1  2.5  1.5  Electronic personal health record (%)c 9.5  10.0  7.0  a Surgeries cancelled within 24 hours b New patients with pressure ulcers. All patients underwent total hip replacement c Availability of electronic personal health record including lab results, medication use, X-rays  = hospital performance was less than average;  = hospital performance was average;  = hospital performance was better than average Participants and sampling Our initial idea was to approach patients with a specific condition. However, a commonly cited concern is that patients predominantly focus on their personal context and return to the hospital where they were treated previously.24 Also, those who are most dependent on care services and who could benefit most from a ‘good choice’ are also those who tend to have more problems associated with the capacity to manage informed choices about care providers.25 We therefore decided to recruit a crosssection of healthcare consumers from the general population. By ‘healthcare consumer’ 22 Exploring consumer values of comparative performance information we mean anybody who is a user or potential user of healthcare. In order to increase the potential use of healthcare for themselves, family or friends, we selected respondents aged over 35 years. Our participants responded to an invitation letter sent to a random sample of 480 people listed at a large health insurance company. We used a stratification procedure to include a representative age distribution. We sent 160 letters to people aged 35–50 years and 320 to people aged 50–65 years (ratio: 1/3 to 2/3). Because the response rate was low, an additional approach was undertaken, namely a personal invitation, made by members of the research team, to healthcare consumers, including people sitting in a general practitioner’s (GP’s) waiting room. Participants received a small incentive in exchange for their efforts. No medical information was asked for, so approval by an ethics committee was not needed. Data collection First, an interview guide was developed by members of the research team (NK, MF, JB) and influenced by the available literature about this topic. The interview guide was used to guarantee consistency among groups. The interview guide is available in an online appendix. Before beginning each session, the aims and methods to be used were explained by the moderator (NK). Participants were asked to give their permission for audiotaping and transcribing. At the beginning of the focus group session, the moderator attempted to create a thoughtful, non-threatening atmosphere and set the tone for the discussion. Participants were encouraged to respond to all the issues raised by the moderator or other participants. Every effort was made to create an environment that encouraged individuals to participate fully in the discussion. After the introduction and the first items of the interview guide, real-world examples were explained by the moderator to encourage involvement and to support respondents in their discussion about big themes as values. Participants were asked to look at the examples and to answer the following questions:  Do you understand what the performance information is showing you?  Could this information benefit you in any way when you would be in a situation to make a comparison between different providers?  What benefit could you gain from this information? All focus group meetings lasted about 1.5 hours and were audiotaped. In addition, an assistant made notes during each session. Data saturation, the point at which the participants provided no new information to the researchers, was reached after focus group five. 23 2 Chapter 2 Analysis All audiotaped sessions were transcribed verbatim into a Microsoft Word file. Analysis of the transcripts was facilitated by the use of a qualitative software program Atlas.ti.5.2 to manage the data.25 Thematic analysis was undertaken to reveal core consistencies and meanings in the data by identifying and analysing themes, which are abstract categories of meaningful data fragments. In themes, several fragments, known as codes are connected to each other and recur in a patterned way.26 Data were collected and transcripts read thoroughly to estimate data saturation. The transcripts were read several times by two investigators (NK, MF) and the main themes were extracted. Participants’ statements referring to a particular theme were grouped by similar codes, and further explored. The analysis followed an inductive approach, which was emergent and strongly linked to the data because assumptions were datadriven. This means that the process of coding occurs without trying to fit the data into a pre-existing model or frame.27 Statements about how many people have said something can leave readers unsure how to interpret quantitative numbers in a qualitative study. However, a relatively high frequency may also signify the importance of a finding. In describing something in between, we avoid actual concrete numbers and used terms such as, for example, ‘many’, ‘most’ or ‘a minority’. We present the focus groups discussion in sufficient detail supported by quotes which can be read in the boxes after every theme, to allow readers check the interpretation made during the analysis.26 Every quotation used in the boxes is followed by the number of the focus group (FG), the gender and age of the participant. Trustworthiness We took several measures to ensure the trustworthiness of this study, including multiple methods of recruitment; multiple researches to reflect on the analysis process; multiple rounds in which data were read, analysed, compared and contrasted; project team meetings to review and explore scientific and organisational aspects of the project. RESULTS Thirty-seven consumers agreed to participate in the focus group interviews. Twenty people responded to the invitation letter. Seventeen consumers were recruited having been approached personally; two in the GP’s waiting room. Of the 37 participants, 27 finally participated in one of the six pre-planned focus group interviews (7 men and 20 women). The mean age of the participants was 59 years. We describe four themes that emerged from the analysis. 24 Exploring consumer values of comparative performance information Theme 1: awareness (Box 1) Most participants were not familiar with CPI as a tool to guide informed choice or to compare hospitals. Only a minority said they had seen CPI before. Although they believed themselves to be skilled internet users, most participants did not know where to find CPI. They said, ‘somewhere on the internet’, but they could not specify. Some were aware of the national website presenting CPI for hospitals in the Netherlands, a site the Dutch government initiated in 2005 (www.kiesbeter.nl). Despite a lack of awareness of CPI as a tool to compare hospitals, most participants reported that they knew about the possibility of comparing hospitals and were aware of their free choice of providers. Nevertheless, they mostly seemed to have a low level of interest in CPI. They said that, being in good health, they were not interested in hospital choice, and they did not feel a sense of urgency to look for CPI. The participants also agreed that if they became ill, they probably would not have the time and energy to look for CPI. During the focus group meetings, the participants realised that, because of a low level of awareness, there was a vicious circle in which they continued to be unaware of the potential value of CPI to help them. Box 1. Awareness  ‘I do not know where to find such information’ (FG6, female, 58). ‘No, I did not know either. And on which things do we need to compare? That is also a question for me.’ (FG6, female, age 60)  ‘If you feel healthy, you will not start looking for a hospital just in case you might need one.’ (FG1, female, age 53)  ‘Only, if it is really necessary then is the chance big enough that I would use this kind of information’ (FG2, female, 57) ‘For me as well, I have better things to do.’ (FG2, female, age 56) Theme 2: current sources for decision-making (Box 2) Most of the participants said that their hospital choice was fixed: essentially, they always went to the same hospital. This hospital was usually the closest one, as distance was an important choice attribute. They did not have any reason to change this routine, and questioned the added value of CPI. There were other significant sources they currently used for their decision making. Consumers said that their own previous experiences were important and outweighed the impact of CPI. They also highly valued the advice of family and friends. The consumers trusted their own GP as a source of information. There was some disagreement about whether to follow the GP’s advice and the GP’s role in providing information. In one session, some said they would not argue with their GP’s advice because they felt it could damage their relationship with their GP. 25 2 Chapter 2 Box 2. Current sources for decision making  ‘I think, well I would go to the hospital I always go to. I would not go searching. I am not sure if this wise, I have been in the [name] hospital a zillion times.’ (FG2, female, age 57)  ‘I just feel save in that hospital.’ (FG3,female, age 75)  ‘I have had personal experience with that hospital and I will never go back! And that is the way you choose. The list may point out that it is a great hospital, but if your personal experience was unpleasant, you simply do not believe the list.’ (FG1, female, age 63)  ‘It’s simple, you rely on the experiences of the people you know.’ (FG1, female, age 53a)  I don’t know how my PCP would react if I go to a different hospital than what he refers me to. I don’t know if it would affect the relationship. I still think you will be tend to listen to your PCP.’ (FG6, female, age 63)  ‘It would be nice, if my PCP would help me to remember that I have a choice, and that he would provide me with an overview or refer me to a website where I could look for it’ (FG4, female, age 54). ‘But, you can’t expect that the PCP know everything about this.’ (FG1, female, age 53b)  ‘If my PCP said, “I have faith in that specialist for these reasons” or “I would not rely on that specialist”, then I would indeed switch to another hospital.’ (FG1, female, age 53a). Theme 3: value of CPI (Box 3) Most participants had never seen CPI before attending the focus group interviews. They had some doubts and felt a little confused after being confronted with the CPI examples. The examples caused a variety of reactions among the participants. The inventory of these reactions, including start-up questions, was supportive for participants in the discussion that followed. At first, they gave a reaction to the examples and their usefulness in general. Some participants immediately tried to interpret what they saw, others were primarily looking to see if the examples confirmed their ideas or their own experiences. Some consumers questioned the added value of such information. During the discussion, reactions evolved to more specific goals and values by using the examples. Participants reported that having the ability to select a healthcare provider might involve a significant effort to find and compare information. They argued that choosing a hospital introduced a new responsibility for patients, including feelings of distrust and anxiety, since they could not foresee the consequences that they might be held responsible for. Not using CPI was sometimes explained as a strategy to prevent regret for a wrong decision. The participants pinpointed an important paradox: the more you know, the more uncertain you become, especially if information from different sources is inconsistent. The participants were afraid of losing their trust in certain aspects of healthcare that they had previously assumed to be good. An important theme was the importance consumers assigned to the reliability and trustworthiness of the information. Respondents complemented each other in listing 26 Exploring consumer values of comparative performance information conditions CPI must fulfill before it became of value for them and before they would start using this information. The discussion included the objectivity of the information, how the information was collected, which groups of patients were compared, and at what level the CPI was published. Most participants stressed the importance of the reliability of CPI, but at the same time they found it difficult to determine on what grounds CPI could be considered reliable and trustworthy. Our participants wished to see that sources presenting CPI would include disclaimers about reliability aspects and declare any conflicts of interest. Also, up-to-date information was of much value for consumers. The remarks regarding to this can be summarised with the question: ‘How do I know if what was good then is good for me right now?’ Some participants wanted a single composite indicator of overall quality to compare hospitals rapidly and easily. Other participants stated that the level of aggregation was too general in the realworld examples. The examples referred to an entire hospital, whereas participants wanted to have information at the level of departments, e.g. cardiology and orthopaedics, or at the level of individual doctors. Box 3. Values of comparative performance information  ‘I had to take a look at this for a second. What is this!? All these dots and circles. I'm just amazed by the way we are supposed to believe that this is useful for us as patients.’ (FG1, female, age 53a)  ‘What happens now is that someone tells me: “you are the patient, so make your own choice.” (..). The responsibility is passed on to me, the patient. This fits with the image of the consumer-driven health care system, but the question is whether you can deal with this and what happens if something goes wrong?’ (FG2 male, age 53)  ‘You read something and it stays in your head. But if you had not known it, you wouldn’t have that problem. Knowing everything is not that great either.’ (FG2, female, age 57) Theme 4: perceived effect of CPI (Box 4) The values identified stimulated the participants to mention possible effects of CPI: patient empowerment, waste, freedom of choice, benchmarking, changed perceptions, and a dichotomy in society. Patient empowerment was mentioned in several sessions, so was the counterpart that CPI is a waste. Some respondents found that using CPI to choose providers went beyond what is necessary in healthcare in terms of good care. Increasing freedom of choice was noted as a positive effect, as well as an attitude among some participants who felt their perceptions of quality of care had changed. They declared raising more awareness about the quality of care after the recent introduction of choice for consumers. Consumers supported the effect of benchmarking, so that professionals and organisations could compare each other in terms of quality of care. Finally, some participants had concerns about whether using this information might cause a further dichotomy in society by increasing inequity in healthcare. They foresaw that the ability to use information required skills of healthcare consumers for which some would be better than others. 27 2 Chapter 2 Box 4. Perceived effect of comparative performance information  ‘You now look differently. When I, recently, came to the Emergency Room with my mother, you start to look around, how is it here? How are the nurses doing things? How do they do that? What are the stories of my mother?’ Moderator: ‘Do you now look at things in health care from a different perspective?’ ‘Yes, more as a purchaser of care.’ (FG3, female, age 56)  [..] ‘So you go on this path, and you might get a dichotomy in society, that is a risk with this market in health care’ (female, age 57). ‘Yes’ [approvingly] (female, age 56). ‘You have people who can do this’ (female, age 57). ‘Yes, yes’ (female, age 56). ‘People who have the skills, but for others there will be less possibilities’ (female, age 57). ‘Yes, I am worry about that too.’ (FG3, female, age 56)  ‘I think patients become more assertive. Normally, you would take things for granted if they say ‘go to that hospital. While if you later find out you had a choice, you might say: if I could choose, I had perhaps done things differently and if I had known, I might have gone elsewhere; so I think this has an effect on empowerment of patients.’ (female, age 60) ‘Yes, I do think it is useful information that they receive.’ (FG6, male, age 63) DISCUSSION This study explored the values, thoughts, understanding and evaluations of CPI for hospital choice among healthcare consumers in the Netherlands. By means of realworld examples, healthcare consumers were able to express their views about CPI. The four themes (awareness, current sources of decision-making, value of CPI and perceived effect of CPI) from the data suggest that there are numerous underlying but conflicting values, which are important for healthcare consumers relating to their use of CPI. The CPI was only marginally valued, due to consumers’ values during their processing of CPI, and wider principles that limited consumers even considering the use of the information. Most participants were unaware of CPI, and did not use this kind of information. Therefore, participants could not give a direct answer when we asked them what kind of information they would like to have or what matters most to them at the beginning of the focus group sessions. It seemed that the participants had never reflected on the information they would like to have while comparing hospital care. This is a general finding in judgement and decision-making research because decision-makers often do not know their own values.28,29 The examples were needed to clarify information preferences and led to active debates during the focus group session and to a deeper understanding of the consumers’ values around CPI. In Moser and colleagues’ study, the use of concrete examples led to similar positive results.30 Hibbard et al,5 stressed the need for consumers to develop a better understanding of quality of care and current measures of quality. A recent study in clinical decision-making31 focuses on the social influence of interaction in decision-making. The concept of ‘shared mind’ as an underlying process for clarifying individual values between two or more people, in which new ideas and perspectives emerge, corresponds with our findings using focus 28 Exploring consumer values of comparative performance information groups. Epstein also underlines the power of multiple perspectives of patients, family, physicians or other members of the healthcare team.31 Our study showed that consumers expressed values on several levels and that these values sometimes conflicted with each other. Some values were in favour of the use of CPI, while other doubts, concerns and principles negatively affected consumers’ views of CPI. Participants relied on previous routines such as consulting GP, family, friends, and personal experiences as the basis for choosing a hospital.32,33 Nonetheless, they were also keen on having a choice. Respondents also appreciated the increased transparency in care, and the effect on patient empowerment.34,35 However, the responsibility was difficult, and participants felt that choosing a hospital was a bit of a burden. Fear of disrupting existing relationships was another consideration that prevented people from using CPI to choose a hospital. The relationship with their GP was of much more value to them, which corresponds with UK and Dutch studies.35–37 A new perspective that was raised in our study is that not using CPI was explained as a strategy to prevent regret for a wrong decision as participants could not foresee the consequences of using CPI. Trusted others can help them to clarify the possible consequences and compare this with personal values. Finally, as in other studies our participants placed much value on reliability and distrust the current CPI.24,30 The use of CPI by healthcare consumers is a complex process in which values, rather than rationality, play an important role. Our findings show that consumers need help from others to solve conflicting values, to develop a firmer understanding of the quality of care concept, and to move forward in making active and informed hospital choices. Limitations and strengths Our study adds to the understanding of the role of values, appraisal and judgement of CPI among Dutch consumers. However, the study has a number of limitations that limit the generalisability. First, the small sample size. We intended to include more participants, but recruitment was problematic, despite the use of various strategies. Owing to this difficult recruitment, we were not able to achieve our planned age distribution for the study. Moreover, 25% of the recruited participants did not attend the focus group session, underlining our finding that current CPI is valued marginally. Second, the data collection and analysis took place simultaneously, as the time frame of three months for focus group meetings was short. A strength of this study is the use of real-world examples, which was very helpful for the discussion. Implications Our results make clear that simply providing information is not enough and will not enhance the usage of CPI. CPI is valued only marginally, and as such it is not yet used by service users as an additional information source, nor does it challenge healthcare 29 2 Chapter 2 consumers’ current selection of hospitals. Our findings also show that more focus is needed on eliciting the underlying values of consumers. Several studies, including ours, have stressed the need for an agent that can support healthcare consumers in choosing and can coordinate on their behalf when bringing the choice into practice.38,39 Such an agent might be able to elicit consumers’ preferences, clarify the values of CPI and preferably integrate this into the referral processes of primary care providers. 30 Exploring consumer values of comparative performance information REFERENCES 1. 2. 3. 4. 5. 6. 7. 8. 9. 10. 11. 12. 13. 14. 15. 16. 17. 18. 19. 20. 21. 22. 23. 24. 25. 26. 27. Robertson R, Burge P. The impact of patient choice of provider on equity: analysis of a patient survey. J Health Serv Res Policy 2011;16(Suppl 1):22–8. Victoor A, Delnoij DM, Friele RD, Rademakers JJ. Determinants of patient choice of healthcare providers: a scoping review. BMC Health Serv Res 2012;12:272. Victoor A, Friele RD, Delnoij DM, Rademakers JJ. Free choice of healthcare providers in the Netherlands is both a goal in itself and a precondition: modelling the policy assumptions underlying the promotion of patient choice through documentary analysis and interviews. BMC Health Serv Res 2012;12(1):441. Fredriksson M. Is patient choice democratising Swedish primary care? Health Policy 2013;111(1):95– 98. Hibbard JH, Greene J, Daniel D. What is quality anyway? Performance reports that clearly communicate to consumers the meaning of quality of care. Med Care Res Rev 2010;67:275–93. Werner RM, Asch DA. The unintended consequences of publicly reporting quality information. JAMA 2005;293:1239–44. Fasolo B, Reutskaja E, Dixon A, Boyce T. Helping patients choose: how to improve the design of comparative scorecards of hospital quality. Pat Educ Couns 2010;78:344–49. Kaiser Family Foundation. Update on Consumers’ View of Patient Safety and Quality Information. Agency for Healthcare Research and Quality: Menlo Park, CA, 2008. Fanjiang G, von Glahn T, Chang H, Rogers WH, Safran DG. Providing patients web-based data to inform physician choice: if you build it, will they come? J Gen Intern Med 2007;22:1463–66. Harris KM, Buntin MB. Choosing a Health Care Provider: The role of quality information. Princeton, NJ: Robert Wood Johnson Foundation, 2008. Berendsen AJ, de Jong GM, Meyboom-de Jong B, Dekker JH, Schuling J. Transition of care: experiences and preferences of patients across the primary/secondary interface – a qualitative study. BMC Health Serv Res 2009;9:62. Groenewoud AS. It’s Your Choice! A study of search and selection processes, and the use of performance indicators in different patient groups. Veenendaal: Erasmus University Rotterdam, 2008. Spranca M, Kanouse DE, Elliott M, Short PF, Farley DO, Hays RD. Do consumer reports of health plan quality affect health plan selection? Health Serv Res 2000;35:933–47. Kolstad JT, Chernew ME. Quality and consumer decision making in the market for health insurance and health care services. Med Care Res Rev 2009;66(1 Suppl):28S–52S. Faber M, Bosch M, Wollersheim H, Leatherman S, Grol R. Public reporting in health care: how do consumers use quality-of-care information? A systematic review. Med Care 2009;47:1–8. Rademakers J, Nijman J, Brabers AE, de Jong JD, Hendriks M. The relative effect of health literacy and patient activation on provider choice in the Netherlands. Health Policy 2014;114: 200–6. Stein BD, Kogan JN, Essock S, Fudurich S. Views of mental health care consumers on public reporting of information on provider performance. Psychiatric Services 2009;60:689–92. Damman OC, Hendriks M, Rademakers J, Spreeuwenberg P, Delnoij DM, Groenewegen PP. Consumers’ interpretation and use of comparative information on the quality of health care: the effect of presentation approaches. Health Expect 2012;15: 197–211. Slovic P. The construction of preference. American Psychologist 1995;50:364–71. Epstein RM, Peters E. Beyond information: exploring patients’ preferences. JAMA 2009;302:195–97. Lee YK, Low WY, Ng CJ. Exploring patient values in medical decision making: a qualitative study. PloS One 2013;8(11): e80051. Krueger RA, Casey MA. Focus Groups: A practical guide for applied researchers (3e). Thousand Oaks, CA: SAGE, 2000. Barbour R. Doing Focus Groups. SAGE: London, 2007. Schwartz LM, Woloshin S, Birkmeyer JD. How do elderly patients decide where to go for major surgery? Telephone interview survey. BMJ 2005;331(7520):821. Sheon N (2007) Overview of Atlas.ti.5.2. San Francisco. www.palmpal.org/atlas.pdf (accessed 05/03/14). Buetow S. Thematic analysis and its reconceptualization as ‘saliency analysis’. J Health Serv Res Pol 2010;15:123–25. Jepson R, Harris FM, Bowes A, Robertson R, Avan G, Sheikh A. Physical activity in South Asians: an in-depth qualitative study to explore motivations and facilitators. PloS One 2012;7(10):e45333. 31 2 Chapter 2 28. 29. 30. 31. 32. 33. 34. 35. 36. 37. 38 . 39. 32 Lichtenstein S, Slovic P. The Construction of Preference. Cambridge: Cambridge University Press, 2006. Peters E, Dieckmann NF, Vastfjall D, Mertz CK, Slovic P, Hibbard JH. Bringing meaning to numbers: the impact of evaluative categories on decisions. J Exp Psychol Appl 2009;15:213–27. Moser A, Korstjens I, van der Weijden T, Tange H. Themes affecting health-care consumers’ choice of a hospital for elective surgery when receiving web-based comparative consumer information. Pat Educ Couns 2010;78:365–71. Epstein RM. Whole mind and shared mind in clinical decision-making. Pat Educ Couns 2013;90:200– 6. Kaiser Family Foundation. A National Survey on Consumers’ Experiences with Patient Safety and Quality Information. Menlo Park, CA: Kaiser Family Foundation, 2004. Sinaiko AD. How do quality information and cost affect patient choice of provider in a tiered network setting? Results from a survey. Health Serv Res 2011; 46:437–56. Coulter A. Engaging Patients in Healthcare. Buckingham: Open University Press, 2011. Dixon A, Robertson R, Appleby J, Burge P, Devlin N, Magee H. How Patients Choose and How Providers Respond. London: The King’s Fund, 2010. Groot de IB Otten W Smeets HJ et al. Is the impact of hospital performance data greater in patients who have compared hospitals? BMC Health Serv Res 2011;11:214. Lako CJ, Rosenau P. Demand-driven care and hospital choice. Dutch health policy toward demanddriven care: results from a survey into hospital choice. Health Care Analysis 2009;17:20–35. Meinow B, Parker MG, Thorslund M. Consumers of eldercare in Sweden: the semblance of choice. Soc Sci Med 2011;73:1285–89. Schlesinger M. Choice cuts: parsing policymakers’ pursuit of patient empowerment from an individual perspective. Health Econ Policy Law 2010;5:365–87. Chapter 3 Patients’ expectations of variation in quality of care relates to their search for comparative performance information Nicole Ketelaar Marjan Faber Jozé Braspenning Gert Westert BMC Health Services Research 2014; 14: 617. Chapter 3 ABSTRACT Background: Choice of hospital based on comparative performance information (CPI) was introduced for Dutch healthcare consumers at least 5 years ago, but CPI use has not yet become commonplace. Our aim was to assess the role of patients’ expectations regarding variation in the quality of hospital care in determining whether they search for CPI. Methods: A questionnaire (for a cross-sectional survey) was distributed to 475 orthopaedic patients in a consecutive sample, who underwent primary hip or knee replacement in a university, teaching, or community hospital between September 2009 and July 2010. Results: Of the 302 patients (63%) who responded, 13% reported searching for CPI to help them choose a hospital. People who expected quality differences between hospitals (67%) were more likely to search for CPI (OR =3.18 [95% CI: 1.02–9.89]; p <0.04) than those who did not. Quality differences were most often expected in hospital reputation, distance, and accessibility. Patients who did not search for CPI stated that they felt no need for this type of information. Conclusion: Patients’ expectations regarding variation in quality of care are positively related to their reported search for CPI. To increase the relevance of CPI for patients, future studies should explore the underlying reasoning of patients about meaningful quality-of-care variation between hospitals. 34 Patients’ expectations of variation in quality of care BACKGROUND The public release of comparative performance information (CPI) is common in many countries.1 The 2006 Healthcare Market Regulation Act led to better availability of CPI in the Netherlands. Choice for consumers and providers became a cornerstone of this new healthcare system based on market elements and competition. CPI can include information about service, patient experiences, and quality indicators for clinical care (structure, process, and patient outcomes). The CPI in the Netherlands includes items related to the hospital (e.g. ranking of 100 hospitals) and to condition-related factors (e.g. patient experiences, waiting lists, and annual patient volumes). The purpose of making this information publicly available is to enable healthcare consumers to choose high-quality healthcare2 and to empower them to make an informed choice about healthcare.3 There is no firm evidence that CPI influences patient choices.4 American and British studies have shown that the actual use of CPI for hospital care is restricted to 4–14% of the consumers5,6, while the idea of choice appeals to most consumers.7 Several Dutch studies of CPI for total hip or knee replacements have been performed.8-11 The patients in Moser and colleagues’ study considered CPI to be an additional source of information when they were preparing for a doctor’s appointment. They benefited from the information most when they had to undergo a total hip or knee replacement for the first time.9 American patients who report a lack of hospital choice for total hip or knee replacement are more likely to be dissatisfied with their surgery.8 This observation suggests that encouraging patients to engage in provider selection based on quality of care would improve their satisfaction. The low level of CPI search behaviour for selecting a hospital is partly due to the consumer’s previous experience with a specific hospital12-14 and the unfamiliarity of using CPI for hospital selection15,16, as well as the role of the referring physician. Dutch research shows that many patients prefer their primary care physician to be involved in the choice of hospital so they can either take the physician’s advice or delegate the decision.17-19 Furthermore, many consumers are unaware of the opportunity to consult CPI.6,20,21 To raise the level of CPI awareness (an important step in a causal chain towards using CPI) an attendant motive is required so that consumers feel a need for this information.21 One consumer choice model takes awareness as a starting point.18 Consumers must be aware that there is CPI, and that it is possible to make a choice. We presume that patients’ expectations are an important pre-step and have a subjective influence on awareness. Studies show that, when consumers start searching for information, the questions and expectations already in their minds will drive the 35 3 Chapter 3 direction of their search.22-24 Having expectations about the practice variation of hospitals and therefore perceiving a risk of receiving poor care might be a motive for using CPI. Despite on-going efforts and investments in the collection, production, and dissemination of CPI for the public15, there is no firm evidence that CPI influences patient choices.4 In an attempt to bridge this gap, we hypothesise that consumers who expect to find quality differences between hospitals are those who search for CPI. We tested whether patients’ expectations of variation in quality of care affect their reported search behaviour for hospital performance information, then we adjusted for potential confounders. Furthermore, respondents were asked in what ways they expected hospitals to differ. We also asked them about the most important reasons why they did not search for CPI. We obtained data from patients who had recently been admitted to hospital for an elective total hip or knee replacement. Admission to a hospital for elective surgery can be planned in advance which gives patients time to search for and look into CPI. These replacement procedures are provided at all 87 hospitals in the Netherlands. METHODS In a cross-sectional study of three types of hospitals, we used a consecutive sampling strategy to recruit 475 patients undergoing a total hip or knee replacement, and we invited them to participate in a paper-based survey. To make our sample representative, we included patients from a university hospital, a teaching hospital, and a community hospital. We included adult patients undergoing a primary hip or knee replacement because we expected that previous surgical experience (as in the case of a secondary replacement) would bias the selection. The annual patient volumes were 53 for primary hip replacements and 64 for knee replacements in the university hospital; 232 and 170, respectively, in the teaching hospital; and 236 and 153, respectively, in the community hospital. Data collection took from 5 to 9 months (September 2009 through July 2010). The nurse or anaesthetist who prepared the patient for surgery personally gave the survey questionnaire to the patient at the preoperative appointment 4 weeks prior to the operation. Reminders were sent 2–3 weeks later in the teaching and community hospitals, but no reminders were sent in the university hospital because permission for this was not granted. The institutional ethics committee reviewed the study protocol in accordance with local regulations in the Netherlands, and they concluded that the study was not subject to the Dutch Medical Research Involving Human Subjects Act. 36 Patients’ expectations of variation in quality of care The primary outcome measure was the self-reported search for CPI, i.e. ‘Did you search for additional CPI to compare hospitals after it became clear that you needed surgery?’ (answer: yes or no). Respondents did not search for CPI were asked to select their motive from a list of seven pre-listed reasons with an option to add one. The reasons were based on a literature search (unpublished search) and focus group interviews with consumers.25 Other measures concerned the previous treatment in the selected hospital: consumers’ perception of being well-informed to make a decision, the general practitioner’s (GP’s) role in advising a choice of hospital (all dichotomous variables), and expectations of variation in quality of care between hospitals (large, small, or no difference in quality of care). If respondents expected quality-of-care differences, they were asked to specify these differences for 14 pre-listed factors that were available from the Dutch internet sites www.kiesbeter.nl (make better choices) and www.independer.nl. Both sites based the factors they listed on a set of quality indicators that the Health Care Transparency Programme listed in 2009.26 Focus group interviews with consumers25 and the annual list of a Dutch magazine, Elsevier27, also presented factors to be considered. The factors included the available CPI for hip and knee replacements in the Netherlands, which gave the name of the organisation and the clinical performance of the care providers. Formally, ‘distance’ and ‘reputation’ are not CPI, but they were included because they are important to patients.9,11,28 In this sense, such information can be seen as part of the performance of the healthcare system. We used a broad definition of performance, as did Van Loon and Tolboom, who defined three information types: (1) factual information (names, addresses, and type of provider), (2) quality information based on performance, and (3) quality information based on consumer experience.29 The questionnaire contained items about demographic variables (age, education, and type of replacement [hip or knee]), and hospital characteristics (type and patient volumes for hip and knee replacements). We used descriptive statistics and frequency tables to describe our study population’s demographic variables (age, gender, and education), previous treatment in the current hospital, awareness that hospital performance information is available for comparison, receipt of choice options from GPs, the search for CPI, and expectations of quality differences in hospital care (Table 1). To address the issue of representativeness, we compared the characteristics of the participants in our study with the characteristics of a larger sample of 1508 Dutch patients who underwent a hip or knee replacement.30 We included data only for those respondents who provided valid answers for the core items of our study: search for CPI and expectations of quality differences in hospital care. The respondents were dichotomised into a group 65 years or younger, and a group older than 65 years. The education variable (the level of education) was measured on a 37 3 Chapter 3 four-point scale (none, low, middle, and high), and it was dichotomised for analytical purposes into low and high levels of education. We used logistic regression to analyse the relationship between expectations of quality differences (independent variable) and searching for CPI (dependent variable) for patients who underwent a hip or knee replacement (Table 2, model 1). In order to correct for possible confounders, we also performed this univariate analysis for the demographic variables (e.g. age, gender, education, and type of replacement), hospital type, GPs’ role in advising choice of hospital, awareness of available information for comparing hospital performance, and previous treatment at the current hospital. Potential confounders (p <0.2) in univariate analyses were added to the multivariate model, and we examined their effects on the beta coefficients. Any variables resulting in a change in the beta coefficient of more than 10% were included in the final model. We compared the univariate analysis (model 1) with the multivariate analysis (model 2) for odds ratios (ORs) with 95% confidence intervals (95% CIs). An association was considered statistically significant for p <0.05. We present descriptive statistics for the factors that we expected to differ between hospitals, as well as statistics for the reasons for not searching CPI. In describing these reasons, we distinguished between respondents who expected differences in the quality of hospitals and those who did not. We used SPSS 18.0 for all analyses. RESULTS Study population Of the 475 questionnaires sent out, 302 were returned completed (response rate 63%). 279 questionnaires had valid answers for the core items: search for CPI and expectations of quality differences in hospital care. Table 1 shows a comparison of consumer characteristics differentiated by the type of hospital. We compared our respondents with a sample of 1508 Dutch patients who underwent a hip or knee replacement.30 The age distribution of our respondents was 40% for those aged 65 years or less and 60% for those older than 65 years. For the sample of 1508 patients, the age distribution was 30% and 70%, respectively. While this was more or less similar, the samples showed greater differences for gender: 43% were male in our study versus 28% in the sample of 1508 patients; and for a low level education, 19% and 28%, respectively. 38 Patients’ expectations of variation in quality of care Table 1. Characteristics of the participants differentiated by type of hospitala Total University hospital n % 45 55 37 45 45 56 35 44 64 79 17 21 58 72 23 28 27 34 53 66 58 72 23 28 22 27 Teaching hospital N % 65 57 49 43 42 39 67 62 86 76 27 24 69 61 44 39 18 16 98 84 94 84 18 16 23 26 n % Women 158 57 Men 121 43 Age ≤ 65* 107 40 >65* 163 60 Low level of education 222 81 High level of education 52 19 Total primary hip replacementsb 179 65 Total primary knee replacements 97 35 Received no previous treatment in current hospital 59 21 Received previous treatment in current hospital 217 79 Did not receive hospital choice options from GPs 217 80 Received hospital choice options from GPs 55 20 24 Unaware that they could compare hospital 67 performance 77 87 Aware that they could compare hospital performance 210 76 60 73 Did not search for CPI 242 87 68 83 97 85 Searched for CPI 37 13 14 17 17 15 Expectations of quality differences in hospital care 32 36 No differences 91 33 17 21 52 59 Yes, small differences 137 49 37 45 17 19 Yes, large differences 51 18 28 34 a Data were based on answers from eligible respondents about the search for comparative information and expectations of quality differences in hospital care b Not every score accumulates to 279 because of missing characteristic data CPI: comparative performance information; GP: general practitioner Community hospital n % 48 58 35 42 20 25 61 75 72 90 8 10 52 63 30 37 14 17 68 83 65 82 14 18 23 19 69 77 6 77 93 7 38 46 41 49 4 5 performance Awareness of hospital comparison information and choice Most respondents (76%) reported that they were aware of the possibility of comparing hospital quality, and they (72%) were aware that they actually could choose a hospital for their surgery. Most respondents (73%) reported that they had a choice option. Most (89%) reported being well-informed about the choice of hospital. Of the total group of respondents, only 13% said they had searched for hospital CPI before choosing a hospital for the operation. Among those who did not have a choice option, a minority (14%) searched for information once they knew they needed surgery. It did not seem to matter whether patients had a choice option or not; a minority in each group searched for information. Expectations of differences in quality of hospital care Most participants (67%) expected to find differences in hospital quality of care. The variables gender, type of replacement, GP role in advising a choice of hospital, and being aware that they could compare hospital performance contributed to the 39 3 Chapter 3 explanation model with p <0.2. However, these variables did not effect a substantial change (>10%) in the beta coefficients, so were excluded from the final model. Table 2 shows the results of the univariate and multivariate logistic regression analyses for the relationship between the quality expectations of hospital care and the reported search for CPI. Table 2. The relationship between quality expectations and searching for comparative performance information by patients who underwent a hip or knee replacement (model 1), controlled for patient and hospital characteristics and previous treatment (model 2) Model 1a Univariate analysis OR [95% CI] p Model 2b Multivariate analysis OR [95% CI] Age <65 1.18 [0.55–2.53] ≥65 (reference) Hospital type 1.27 [0.54–2.95] Teaching Community 0.73 [0.24–2.22] University (reference) Previous treatment in current hospital 0.32 [0.15–0.70] Yes No (reference) Expectations regarding quality 0.00c differences in hospital care Yes, small differences 3.71 [1.22-11.27] 0.02c 3.18 [1.02–9.89] Yes, large differences 7.44 [2.28-24.30] 0.00c 5.05 [1.44–17.77] No, differences (reference) a Based on the answers of 279 respondents; b Based on the answers of 263 respondents c P < 0.05; OR [95% CI]: odds ratio [95% confidence interval] p 0.66 0.58 0.58 0.00c 0.04c 0.04c 0.01c Previous treatment in the current hospital also appeared to significantly influence the reported search for CPI. The univariate model shows that respondents who expected small differences in hospital quality were more likely to search for CPI than those who did not (OR = 3.71 [95% CI: 1.22–11.27]). For people who expected large quality-of-care differences, this effect was even greater (OR = 7.44 [95% CI: [2.28–24.3]). The adjusted ORs were 3.18 [95% CI: 1.02–9.89] for the group who expected small differences, and 5.05 [95% CI: 1.44–17.77] for the group who expected large differences. Respondents with previous treatment in the same hospital less often searched for CPI than those who had no previous treatment (OR =0.32 [95% CI: 0.15–0.70]). The participants’ rating of expected hospital quality differences in reputation, distance, and accessibility are ranked the highest (Table 3). 40 Patients’ expectations of variation in quality of care Table 3. Factors influencing choice of hospital for total hip and knee replacementsa Hospital-related factors 1. Reputation 2. Distance 3. Accessibility 4. Type of hospital (university, teaching, or community) 5. Ranking list of ‘100 hospitals’ 6. Hospital size 7. Number of cancelled operations Condition-related factors 8. Plan for pre-operative schedules on 1 day 9. Orthopaedic specialism 10. Patient experiences 11. PROMs 12. Waiting list 13. Annual patient volume 14. Infection rates a Data were based on 191 answers from eligible respondents PROMs: Patient-Reported Outcome Measures n 107 95 79 51 41 20 1 n 51 50 49 49 45 42 16 3 Reasons for not searching for comparative performance information Of the respondents who did not search for additional information, 179 said they felt no need for more information. Others gave far less common reasons: 19 had no internet access at home, 15 felt more information would create more doubts, 13 said searching for information was an extra burden, 7 thought choosing a hospital based on CPI was too much responsibility, 8 did not know where to look, 6 had no skills how to look, 8 had no hospital choice options, and 5 had no time. Feeling no need was by far the most important reason for not searching for hospital CPI. Though, this reason was not significant related to the expectations of quality-ofcare differences. DISCUSSION Thirteen per cent of our study population searched for CPI to compare hospitals. This is consistent with the results of American and Dutch studies for similar populations.14,31 Our hypothesis that patients who expect quality differences are those who search for CPI was confirmed. Previous experience with the hospital is another factor influencing the search for CPI. Expecting quality of care differences in hospital performance appears to be a stimulus for searching for CPI, although respondents who underwent previous treatment in the hospital tended to search less for this information. In our study, the impact of previous experience on hospital choice was consistent with Dixon and colleagues’ results.32 They compared the effect of consumer choice during the referral process in the Netherlands and England. For Dutch patients, ‘being in the neighbourhood’ or ‘having been there before’ were the most important reasons for 41 Chapter 3 choosing a hospital: patients usually returned to the same hospital.33 Interestingly, patients prefer being treated at their current hospital, even if they could choose a better alternative with higher-quality care.34,35 Choosing a familiar hospital instead of an unknown one suggests that personal experience is a value in itself. While CPI may indicate the best hospitals, patients may optimise the factors they value most rather than objectively maximize quality. Our respondents expected most differences to be in reputation, distance, and accessibility. As reported in other studies6,12,19, these are known choice factors in decision-making. Our study population expected differences mainly in the general performance of the healthcare system rather than in specific condition-related factors (total hip or knee replacement). The factors for which consumers expect quality differences may change in the future as consumers become more knowledgeable about CPI. Dutch studies among patients with a total hip or knee replacement found that both interpersonal aspects (conduct of doctors) and more technical ones (for example, the prevention of adverse effects of thrombosis and the specialist area of orthopaedists) are important to patients.11 Making the concept of quality more meaningful may also increase consumer interest and need for CPI.15 Bozic and colleagues have confirmed this statement: they have recently found that patients were very motivated to search for provider quality. In their study, physician manner and surgical outcomes appeared to be the most important considerations for selecting a provider for elective total joint arthroplasty.36 Some studies6,18 led us to expect that unawareness is an important reason why hospital performance information has little influence on decision-making. An information need and a sense of urgency are necessary ingredients for awareness and interest in this type of information. Although most of our respondents (76%) were aware that they could compare hospital performance and choose a hospital (72%), they still did not use CPI. This discrepancy might be due to their feeling no need for such information. Having alternative information sources9 or doubt about the trustworthiness of such information2 could also contribute to this feeling. This discrepancy also implies the need for a more outspoken reporting of hospital quality that emphasizes differences in quality. If rational patients assume that these differences are small, then they cannot be expected to look for and use such information. The ongoing efforts and investments that go into the collection, production, and dissemination of CPI would then be useless. Implications Our study shows that merely making CPI available in the public domain does not result in its use. This implies that further action, such as applying an implementation 42 Patients’ expectations of variation in quality of care strategy, is necessary. Other studies see the need for an infrastructure that provides patients with advice about their choices and helps them in actually choosing.37,38 Future research should explore the concept of patients’ expectations more comprehensively because the fact that patients’ expectations that quality differences exist affects their search for CPI, but does not affects their need for CPI. Research should use the resulting information to determine how fragile or robust these expectations are.39 Whereas CPI is based on measurable factors, consumer expectations are more diffuse and individually determined. More development of tailored methods to assess the understanding of variation in quality of care as a precondition for acquiring awareness, knowledge, and interest is necessary. Finally, further research should explore whether the sense of urgency for this information will increase if the concept of quality of care becomes more meaningful and patients start to realise that quality and outcomes of care do vary for both treatment and hospital factors. Limitations One strength of our study is the recruitment of respondents in three types of hospitals. Consumer characteristics differed somewhat among the three settings (Table 1), which confirmed the validity of our decision to include the three types. One limitation of our study is that only a minority reported searching for CPI, so we could not make precise estimations, as is reflected in the large confidence intervals in Table 2. We would have preferred a more balanced dataset for better data modelling. Another limitation is the self-reporting of our main outcome, which may have introduced a recall bias. However, we limited the time between selecting a hospital and completing the questionnaire by giving it to the participants at their appointments with a nurse or anaesthetist before surgery. This tactic minimized the time lag between choosing a hospital choice and the date of surgery. Conclusions CPI makes the variation in quality of care between hospitals transparent. This study shows that the number of people who report having searched for CPI is still limited, but may increase if patients become more aware of the quality-of-care variation of hospitals. However, this will be difficult to achieve because people who feel no need for more information – e.g. based on a lack of expected differences in quality of care – do not search for CPI. Awareness as a prerequisite for the use of CPI should not be limited to having knowledge about the existence of CPI and where to find it; awareness should also extend to the quality-of-care variation of hospitals. 43 3 Chapter 3 REFERENCES 1. 2. 3. 4. 5. 6. 7. 8. 9. 10. 11. 12. 13. 14. 15. 16. 17. 18. 19. 20. 21. 22. 23. 24. 25. 44 Victoor A, Delnoij DM, Friele RD, Rademakers JJ. Determinants of patient choice of healthcare providers: a scoping review. BMC Health Serv Res 2012; 12:272. Marshall M, McLoughlin V. How do patients use information on health providers? BMJ 2010; 341:c5272. Fasolo B, Reutskaja E, Dixon A, Boyce T. Helping patients choose: how to improve the design of comparative scorecards of hospital quality. Patient Educ Couns 2010; 78(3):344–349. Ketelaar NABM, Faber MJ, Flottorp S, Rygh LH, Deane KHO, Eccles MP. Public release of performance data in changing the behaviour of healthcare consumers, professionals or organisations. Cochrane Database Syst Rev 2011; 11:CD004538. Fung CH, Lim YW, Mattke S, Damberg C, Shekelle PG. Systematic review: the evidence that publishing patient care performance data improves quality of care. Ann Intern Med 2008;148(2):111–123. Harris KM, Buntin MB. Choosing a Health Care Provider: The Role of Quality Information. Princeton, NJ: Robert Wood Johnson Foundation, 2008. Coulter A. Engaging Patients in Healthcare. London: Open University Press, 2011. Losina E, Plerhoples T, Fossel AH, Mahomed NN, Barrett J, Creel AH, Wright EA, Katz JN. Offering patients the opportunity to choose their hospital for total knee replacement: impact on satisfaction with the surgery. Arthritis Rheum 2005;53(5):646–652. Moser A, Korstjens I, Van der Weijden T, Tange H. Patient’s decision making in selecting a hospital for elective orthopaedic surgery. J Eval Clin Pract 2010;16(6):1262–1268. Moser A, Korstjens I, van der Weijden T, Tange H. Themes affecting healthcare consumers’ choice of a hospital for elective surgery when receiving web-based comparative consumer information. Patient Educ Couns 2010;78(3):365–371. Zwijnenberg NC, Damman OC, Spreeuwenberg P, Hendriks M, Rademakers JJ. Different patient subgroup, different ranking? Which quality indicators do patients find important when choosing a hospital for hip- or knee arthroplasty? BMC Health Serv Res 2011;11:299. Kolstad JT, Chernew ME. Quality and consumer decision making in the market for health insurance and health care services. Med Care Res Rev 2009;66(1 Suppl):28S–52S. Marang van de Mheen PJ, Dijs-Elsinga J, Otten W, Versluijs M, Smeets HJ, Vree R, van der Made WJ, Kievit J. The relative importance of quality of care information when choosing a hospital for surgical treatment: a hospital choice experiment. Med Decis Making 2011;31(6):816–827. Schwartz LM, Woloshin S, Birkmeyer JD. How do elderly patients decide where to go for major surgery? Telephone interview survey. BMJ 2005;331(7520):821. Hibbard JH, Greene J, Daniel D. What is quality anyway? Performance reports that clearly communicate to consumers the meaning of quality of care. Med Care Res Rev 2010; 67(3):275–293. Trigg L. Patients’ opinions of health care providers for supporting choice and quality improvement. J Health Serv Res Policy 2011;16(2):102–107. Damman OC, Spreeuwenberg P, Rademakers J, Hendriks M. Creating compact comparative health care information: what are the key quality attributes to present for cataract and total hip or knee replacement surgery? Med Decis Making 2012;32(2):287–300. Faber M, Bosch M, Wollersheim H, Leatherman S, Grol R. Public reporting in health care: how do consumers use quality-of-care information? A systematic review. Med Care 2009;47(1):1–8. Lako CJ, Rosenau P. Demand-driven care and hospital choice: Dutch health policy toward demanddriven care: results from a survey into hospital choice. Health Care Anal 2009;17(1):20–35. Jha AK, Epstein AM. The predictive accuracy of the New York State coronary artery bypass surgery report-card system. Health Aff 2006;25(3):844–855. Wilson TD. Trends in…a critical review: information behaviour: an interdisciplinary perspective. Inform Process Manag 1997;33(4):551–572. Adam JA, Khaw FM, Thomson RG, Gregg PJ, Llewellyn-Thomas HA. Patient decision aids in joint replacement surgery: a literature review and an opinion survey of consultant orthopaedic surgeons. Ann R Coll Surg Engl 2008;90(3):198–207. Brashers DE. Communication and uncertainty management. J Commun 2001;51:477–497. Kivits J. Researching the ‘informed patient’. Information, Communication & Society 2004;7(4):510– 530. Ketelaar NABM, Faber MJ, Westert GP, Elwyn G, Braspenning JC. Exploring consumer values of comparative performance information for hospital choice. Qual Prim Care 2014; 22(2):81–89. Patients’ expectations of variation in quality of care 26. 27. 28. 29. 30. 31. 32. 33. 34. 35. 36. 37. 38. 39. Health Care Transparency Programme. Kwantitatieve Analyse Indicatoren Zichtbare Zorg. Ziekenhuizen - 2008 Data. Den Haag: Healthcare Inspectorate, 2009:1–443. Elsevier. The Best Hospitals 2008. In Elsevier, Volume 74. Amsterdam: Reed Elsevier, 2008. Epstein AJ. Effects of report cards on referral patterns to cardiac surgeons. J Health Econ 2010; 29(5):718–731. Van Loon AJM, Tolboom RAL. Design Report “Choose Better” 2006–2007 [In Dutch: Ontwerprapport KiesBeter.nl 2006–2007]. Bilthoven: RIVM, 2005. Stubbe JH, Gelsema T, Delnoij DM. The Consumer Quality Index Hip Knee Questionnaire measuring patients’ experiences with quality of care after a total hip or knee arthroplasty. BMC Health Serv Res 2007; 7:60. de Groot IB, Otten W, Smeets H, Marang-van de Mheen P, The Choice 2 Study Group. Is the impact of hospital performance data greater in patients who have compared hospitals? BMC Health Serv Res 2011; 11:214. Dixon A, Robertson R, Bal R. The experience of implementing choice at point of referral: a comparison of the Netherlands and England. Health Econ Policy Law 2010;5(3):295–317. Dijs-Elsinga J, Otten W, Versluijs MM, Smeets HJ, Kievit J, Vree R, van der Made WJ, Marang-van de Mheen PJ. Choosing a hospital for surgery: the importance of information on quality of care. Med Decis Making 2010;30(5):544–555. Boonen LH, Donkers B, Schut FT. Channeling consumers to preferred providers and the impact of status quo bias: does type of provider matter? Health Serv Res 2011;46(2):510–530. Laverty AA, Smith PC, Pape UJ, Mears A, Wachter RM, Millett C. High-profile investigations into hospital safety problems in England did not prompt patients to switch providers. Health Aff 2012;31(3):593–601. Bozic KJ, Kaufman D, Chan VC, Caminiti S, Lewis C. Factors that influence provider selection for elective total joint arthroplasty. Clin Orthop Relat Res 2013;471(6):1865–1872. Meinow B, Parker MG, Thorslund M. Consumers of eldercare in Sweden: the semblance of choice. Soc Sci Med 2011;73(9):1285–1289. Schlesinger M. Choice cuts: parsing policymakers’ pursuit of patient empowerment from an individual perspective. Health Econ Pol Law 2010; 5(3):365–387. Bowling A, Rowe G, Lambert N, Waddington M, Mahtani KR, Kenten C, Howe A, Francis SA. The measurement of patients’ expectations for health care: a review and psychometric testing of a measure of patients’ expectations. Health Technol Assess 2012;16(30):i–xii. 1–509. 45 3 Chapter 4 Recognition of physiotherapists’ expertise in Parkinson’s disease Nicole Ketelaar Marten Munneke Bas Bloem Gert Westert Marjan Faber BMC Health Services Research 2013; 13: 430. Chapter 4 ABSTRACT Background: Publicly available information comparing performance across quality and costs has proliferated in recent years, both about individual healthcare professionals and hospitals. This type of information is now becoming increasingly available for physiotherapists with expertise in Parkinson’s disease (PD). Our study aimed to explore the ability of people with Parkinson’s disease to recognise expertise, and to what extent respondents selectively choose such expert physiotherapists. Methods: We used claim data from the period 2009–2010 to select customers with PD who claimed physiotherapy. A random sample of 500 eligible respondents received a paper-based survey. We used descriptive statistics to compare the respondent characteristics, a qualitative programme to analyse the qualitative items, and univariate and multivariate regression. Results: Most respondents (89%) took their referring physician’s advice when selecting a physiotherapist, although this advice rarely was supported with arguments. The remaining respondents (11%) searched for comparative performance information about physiotherapists. Respondents who recognised the added value of PD expertise among physiotherapists were 3.28 times as likely to search for comparative performance information as those who did not understand. Respondents were willing to switch to an expert physiotherapist (68%), and this willingness increased if they recognised the value of PD expertise (p < .001). Conclusion: The participants were able to recognise certain aspects of expertise. Though they showed relatively few signs of selectively choice behaviour for expert physiotherapists. Both respondents and referring professionals need more understanding about the added value of an expert physiotherapist, to foster selective provider choice. 48 Recognition of physiotherapists’ expertise in Parkinson’s disease BACKGROUND The release of information in the public domain about the performance of healthcare providers has become a strategy for improving the quality of healthcare.1 Providing such comparative performance information (CPI) may empower and enable patients to identify and choose high-quality providers.2,3 While we have an increasing understanding of motivating factors for selective provider choice regarding elective conditions4-6, or primary care physicians7 there is no knowledge available about whether this evidence also applies to people with a chronic disorder such as Parkinson’s disease (PD). Our aim is to explore whether people are aware of, and able to recognise expertise, as an expression of quality of care, in PD among physiotherapists. There are an estimated 1.2 million people with Parkinson’s disease (PwP) in Europe.8 Physiotherapy is part of the treatment for many PwP.9 In the Netherlands and the United Kingdom, 54–60% of all PwP receive physiotherapy.10,11 In the former, physiotherapy is mainly provided in community-based settings, in provider-owned practices. The reimbursement of the first 21 physiotherapy sessions, on annual basis, depends on the consumers additional insurance package. Though, for the treatment of PD there is an unlimited reimbursement out of the basic insurance after these twentyone treatments. Dutch patients have free access to physiotherapy since 2006 so they can self-refer to a physiotherapist, which is considered to be a structure premise for selective provider choice.12 More than 75% of the Dutch allied health professionals report a lack of PD-specific expertise. More than half of them were unfamiliar with the treatment options of other professionals, and they had not participated in relevant educational programmes.9 In response to this gap of knowledge, a Dutch multidisciplinary network, ParkinsonNet, was developed and implemented to increase the PD expertise of physiotherapists.13 ParkinsonNet is a multifaceted intervention that includes several elements (Table 1). Table 1. Five core elements of ParkinsonNet14 1. Delivering care according to evidence-based guidelines 2. Continuous education and training of therapists (including physiotherapists, occupational therapists, and speech therapists) 3. Structured and preferred referral to ParkinsonNet therapists by neurologists, enabling each therapist to attract a large volume of patients to increase expertise 4. Encouraging communication and regional collaboration with referring physicians 5. Promoting visibility of the available expertise for both patients and professionals Launching of the sixty-fifth regional ParkinsonNetnetwork in 2010, national coverage was achieved. Currently, about 2700 professionals are connected throughout the 49 4 Chapter 4 Netherlands.14 Although, physiotherapists can be expert in PD without being connected to this multidisciplinary network, the term ‘expert physiotherapist’ is exclusively used throughout this paper for physiotherapists who are connected to ParkinsonNet. An expert physiotherapist received additional training in treating PwP. Evidence shows that specific treatment options provided by expert physiotherapists are more beneficial to patients.15,16 Also, expert physiotherapists show better guideline adherence scores compared to generically active physiotherapists and their PwP volume is larger.9 Awareness of referral options for other professional disciplines17 and knowledge and use of the Dutch PD guideline13 are necessary to attain expertise in PD. The implementation of ParkinsonNet increased the number of PwP treated by each physiotherapist.18 However, approximately 70% of the PwP are still treated by a general physiotherapist.18 This raises the questions whether PwP are aware of PD expertise in physiotherapy and whether they understand its value, i.e. these are, apart from free provider choice, two additional relevant conditions for selectively choosing a provider. This paper focuses on ability of PwP to recognise expertise in PD among physiotherapists, as a potential indicator of selective provider choice. We consider this in several ways: by seeking the perspective of PwP about the PD expertise of their physiotherapist, by ascertaining the descriptions of PD expertise from the perspective of PwP in a qualitative way, and whether physiotherapists’ PD-related expertise influences the search for CPI among PwP. In addition, we examine the role of referring physicians. We take into account factors that might affect provider selection such as consumer characteristics (e.g. age, education, and internet use), knowing where to search for CPI, the willingness to switch, and respondents’ expectations regarding variation in quality of care. METHODS Design and study population Our paper-based survey focused on the selective provider choice for expert physiotherapists in a cohort of people with PD. We selected PwP on the basis of claim data in the period 2009–2010 of a Dutch healthcare insurance company that accounts for 20% of the Dutch insurance market. We approached eligible candidates, that is, people who were registered as having PD in the diagnosis–treatment combination (Dutch version of diagnosis related groups) combined with an episode of physiotherapy in the year prior to our survey. Only consumers who received physiotherapy treatment for PD were included because we wanted to focus on peoples’ capacity to recognise physiotherapists’ PD expertise. People who had had physiotherapy for PD for several years were excluded; the decision-making process (deliberate or not) had to be recent to exclude recall bias. A total of 886 patients met these eligibility criteria. We sent 50 Recognition of physiotherapists’ expertise in Parkinson’s disease surveys to 500 participants randomly selected from this group. Due to privacy legalities, the insurance company drew the sample. According to local regulations in the Netherlands (Commission involving human subjects research) (CMO) region ArnhemNijmegen this study did not need approval of the ethical review board. Measures and data collection The survey consisted of 37 items. The first items ascertained the treatment with physiotherapy for PD in the past year to satisfy our inclusion criteria. We used five items to operationalise the recognition of physiotherapists’ PD expertise. Two items explored the level of awareness and the descriptions of PD expertise from the perspective of the participants: awareness of the existence of expert PD physiotherapists (dichotomous variable), and participants were asked to estimate whether their physiotherapist was an expert in PD. This latter item was answered on a three-point scale: yes, no, or do not know. We compared the participants’ views of physiotherapists’ expertise on the basis of claim data with the ParkinsonNet data to see if they correlated. The respondents answered an openended question that dealt with their assessment of physiotherapists’ expertise. We categorised the answers into eight core themes based on a framework for patient centeredness in PD19 and quality of care domains as formulated by the Institute of Medicine.20 We estimated the search for CPI by addressing the last two aspects used to operationalise the recognition of physiotherapists’ PD expertise: recognition of the added value of PD expertise, and whether participants paid attention to PD expertise among physiotherapists when they selected a provider (all dichotomous variables). Knowing where to look for CPI was a dichotomous variable. Expectations of variation in the quality of care between a generic physiotherapist and an expert physiotherapist formed a categorical variable, determined on a four-point scale: yes, large differences; yes, small differences; do not know; or no differences. The willingness to switch to an expert physiotherapist was determined on a five-point scale: most likely, likely, unlikely, most unlikely, and do not know. We dichotomised this into likely (most likely, likely) versus unlikely (most unlikely, unlikely, do not know) because the distribution of this variable was positively skewed. We assessed the role of referring physicians with the following questions: do referring physicians provide you multiple choice options for physiotherapists about where you could go to? Did your referring physicians give you an advice to which provider you should go for the best treatment? If so, is this advice accompanied by arguments and by CPI, and do you take this advice? (All of these are dichotomous variables). We asked the participants which attributes they searched for when choosing a physiotherapist, and for which of these attributes referring providers supplied them 51 4 Chapter 4 with information. The attributes contain items about services and quality of care. The first three items were available for PwP at the time the survey was send out, we added items 4 and 5 because we expected that these items soon become available as well and/or these items were common items with other (elective) conditions. 1. ParkinsonNet membership, e.g. practices connected to this network 2. Information regarding the added value of being treated by an expert PD physiotherapist 3. Distances to physiotherapy practices 4. Physiotherapists with PD expertise 5. Experiences of PwP who were treated by an expert physiotherapist. We also ascertained the demographics of the study participants. We treated age as a continuous variable. The variable of educational level was described as none/low, average, or high. We used the stages defined by Hoehn and Yahr21 to describe the selfreported disease characteristics. The Hoehn and Yahr (HY) stages range from no PD signs in stage 0 to needing a wheelchair or being bedridden in the most severe stage 5. Patients’ disease severity was classed as mild (HY stages 0–1), moderate (HY stages 2–3), or severe PD (HY stages 4–5). The survey was field tested and optimised by ten patients and four PD researchers. The final survey was sent by post, and a reminder was posted 2 weeks later. Data were collected in October and November 2010. Analysis The survey contained four qualitative questions. The filled-out paper-copy surveys were scanned and transcribed into an electronic format, by means of an automated process (Teleform). We did not use a separate programme for the qualitative analysis, such as Atlas.ti, as it did not bring much benefit for the rather small amount of qualitative data. Though we did apply the principles of thematic analysis (conducted by NK and MF), supported by the PD-specific framework for patient centeredness19 and a general framework for quality of care.20 Descriptive statistics were calculated for each survey item and compared with respondent characteristics. We explored the association between how patients value PD expertise and the search for CPI (treated as a dependent variable determined by ‘yes’ or ‘no’). We used univariate logistic regression analysis to separately examine the associations between the independent variables and the search for CPI. The independent variables were: age, education, internet use, awareness of the existence of expert PD physiotherapists, an understanding of the added value of an expert PD physiotherapist, prior attention to physiotherapists’ expertise, knowing where to search, expected quality differences between generic physiotherapists and expert physiotherapists, willingness to switch, and the provider options named by referring physicians. Statistically significant variables (p < 0.05) were 52 Recognition of physiotherapists’ expertise in Parkinson’s disease included in the stepwise forward multivariate logistic regression analysis. We calculated the outcomes separately for each independent variable while controlling for the other variables in the model. We presented these outcomes with odds ratios (ORs) and 95% confidence intervals (CIs). We used SPSS 18.0 to carry out the analyses. RESULTS Demographics In total, 380 respondents completed the survey (gross response rate=76%), and 320 surveys were analysed (net response rate=64%). The 60 participants whose surveys were excluded had not received physiotherapy for PD, but for another medical condition. Table 2 presents the background characteristics of the study population. Table 2. Background characteristics of study population General characteristics Gender Mean age in years ± SD Level of education Male n 147 72 % 56 ±9 None/low Average High 165 91 39 56 31 13 55 211 36 85 18 71 11 28 4 Residential status Alone Together Other situation Internet use Yes Specific Parkinson’s disease characteristics Diagnosis Parkinson’s disease Atypical parkinsonism Unknown Hoehn and Yahr stage (self-reported) Mild (0–1) Moderate (2–3) Severe (4–5) Yes Use of physiotherapy in the past year Because of missing data in the background characteristics, not every 320 292 10 4 95 3 2 83 30 127 41 87 29 320 100 score accumulates to the total of Quality of care for Parkinson’s disease Sixty percent of the study population expected quality differences in the care provided by generic physiotherapists and expert physiotherapists, and 34% of them expected these variations would be large. A minority of participants (5%) expected no quality differences, and the remaining respondents (35%) said they did not know. Of those who expected quality differences, 33% did not know what kind of value an expert physiotherapist could add. In total, about half the study population (51%) did not know 53 Chapter 4 the added value of an expert physiotherapist. Participants who expected to find quality differences were younger than participants who did not expect quality differences (71 ± 10 years versus 74 ± 8 years, p = .009). More than two-thirds of the respondents (68%) were willing to switch to an expert physiotherapist if it turned out that their current physiotherapist had no PD expertise. The distance the participants were willing to travel to see an expert physiotherapist was 5 km (interquartile range: 2–11 km). Respondents who had previously heard about expert physiotherapists were more willing to switch (82% versus 54%, p < .001) and respondents expecting differences in the quality of care were also more likely to switch (87% versus 31%; p < .001). Recognition of physiotherapist expertise in Parkinson disease Most participants (74%) had already heard about expert physiotherapists. Fewer participants (46%) said they had previously paid attention to whether the physiotherapist was an expert in PD before selecting a physiotherapist. Participants who had previously heard about physiotherapists with PD expertise had a higher educational level than those who did not know about expert physiotherapists (p = .001). Awareness of expert physiotherapists was also related to age. Participants who were aware of expert physiotherapists were younger than those who were unaware (71 ± 10 years versus 76 ± 8 years, p = .001). We asked those who had already heard about expert physiotherapists whether they were treated by an expert PD physiotherapist (n=229). More than 70% asserted they were treated by an expert, 12% stated that they were not, and 17% said they did not know. A comparison of the answers of our study population with ParkinsonNet data showed that 28% of our respondents were being treated by a ParkinsonNet-affiliated physiotherapist. Participants reported various themes describing what physiotherapists’ expertise and knowledge stands for. Table 3 presents the core themes and the underlying descriptions that the respondents gave. Thirty percent of the respondents said that physiotherapists’ treatment, exercises, and information express a degree of expertise. 54 Recognition of physiotherapists’ expertise in Parkinson’s disease Table 3. Themes related to the perceived expertise of physiotherapists from the patients’ perspective n 69 Treatment and exercises ‘Expertise is related to the good exercises I have to do at home’ (Woman, 82 years old) ‘Right exercises for posture and movement’ (Man, 76 years old) Tailored information and communication Information about the disease, (practical) ‘All arguments with the given instructions are correct’ advice, counselling, and communication (Man, 64 years old) with physiotherapist ‘He tries to find matching exercises for me in person’ (Man, 70 years old) ‘She knows a lot of Parkinson Disease, I receive good answers to my Parkinson Disease related questions’ (Woman, 74 years old) Knowledge of Parkinson disease Knowledge about and expertise in ‘He knows exactly what to do’ (Man, 78 years old) ‘He knows the Parkinson disease signs and tries the Parkinson disease symptoms. Noticing recommended treatment’ (Man, 57 years old) changes of condition ‘She knows the limitations Parkinson disease imposes on me, and gives useful suggestions’ (Woman, 82 years old) ‘Works with several patients who have Parkinson disease’ (Man, 60 years old) Effectiveness Improved condition, increased mobility, ‘I can to move better and my muscles are less stiff since the treatment’ (Woman, 82 years old) decreased freezing ‘My condition has improved since this treatment’ (Man, 75 years old) Cooperation with caregivers Need for interdisciplinary care. Working ‘He works with the neurologist’ (Man, 73 years old) ‘He is involved in the Park fit studies’ (Man, 71 years within a network with allied healthcare old) providers Emotional support, empathy, and respect ‘Shows great interest’ (Man, 74 years old) ‘Shows interest, takes me seriously’ (Woman, 77 years old) ‘Is very patient’ (Man, 88 years old) Unable to define the expertise of the physiotherapist Difficult to access the providers’ expertise ‘I do not know which exercises are the best for patients with Parkinson disease’ (Man, 61 years old) Treatment evaluation Providers ask questions about the ‘Takes the time to evaluate the treatment process’ (Man, 70 years old) treatment Accessibility of healthcare Treatment or physiotherapy at home ‘Treats me at home’ (Man, 70 years old) Total 66 34 25 19 16 4 2 2 103 The role of referring physicians The respondents were referred by: neurologists (49%), general practitioners (GPs; 18%), and specialist Parkinson’s nurses (18%). Another 15% of the respondents saw a physiotherapist on their own initiative. About half the participants who received a 55 4 Chapter 4 referral were provided with additional information regarding expert PD physiotherapists. Only a minority of the respondents (25%) received multiple choice options for a physiotherapist. Although 85% of the participants found it important to choose their own physiotherapist, most (89%) of those with a physician referral took their physician’s advice when selecting a physiotherapist. The search for comparative performance information Most respondents (89%) reported not having searched for CPI when it became clear they needed physiotherapy. Some respondents gave more than one reason for not searching. The most important reason was not perceiving any need for more information (60%). Other reasons were: no internet access at home (29%), not knowing where to search (15%), not knowing how to search (9%), lack of motivation (13%), not knowing how to look for information (12%). A smaller group found that more information led to more doubt (7%), some felt that it was too much responsibility (4%), and some had no time (3%). Participants who searched for information (11%) wanted to know about: practices connected to ParkinsonNet (n=13, 4%), expert PD physiotherapists (n=18, 6%), physiotherapist practices close to home (n=19, 6%), experiences of patients who received treatment from an expert physiotherapist (n=7, 2%), and what added value physical treatment from an expert physiotherapist can give a person with PD (n=11, 3%). Some declared they did not find the information they would have like to have. Others reported that the information was too general and not trustworthy. Univariate logistic regression analyses revealed that several variables are associated with the search for CPI. Respondents’ awareness of expert PD physiotherapists, an understanding of the added value of an expert physiotherapist, and the willingness to switch to an expert PD physiotherapist were statistically significant, as were consumer characteristics (age and internet use). These variables were included in the multivariate regression analysis. The stepwise multivariate logistic regression analysis revealed that recognising of added value of an expert PD physiotherapist was the most important predictor for search for physiotherapists’ CPI (Table 4). The likelihood of people who recognised added value searching for information was 3.28 times as great as the likelihood those who did not recognise (OR=3.28 [95% CI 1.42–7.58]). 56 the the the for Recognition of physiotherapists’ expertise in Parkinson’s disease Table 4. Univariate and forward stepwise multivariate regression relationship of searching for comparative performance information versus background characteristics, awareness, and understanding physiotherapists’ expertise in Parkinson’s disease Univariate analyses OR [95% CI] P 0.96 [0.93–0.99] 0.02* Multivariate analyses OR [95% CI] P Age Educational level Low (reference) Middle 1.49 [0.69–3.23] 0.32 High 0.73 [0.20–2.61] 0.62 No internet use 2.12 [1.02–4.39] 0.04* Not aware of expert PD physiotherapists 3.76 [1.11–12.70] 0.03* No recognition of added value of expert 3.29 [1.47–7.31] 0.01* 3.28 [1.42–7.58] 0.01* physiotherapist No prior attention to physiotherapist expertise 0.57 [0.18–1.82] 0.34 Not knowing where to look for CPI 1.13 [0.30–4.16] 0.86 Quality differences between generic and expert physiotherapists No differences (reference) Yes, large differences 1.41 [0.29–6.80] 0.67 Yes, small differences 0.91 [0.17–4.73] 0.92 Do not know 0.26 [0.04–1.56] 0.14 Not willing to switch 3.86 [0.89–16.78] 0.07 Received provider options from referring physician 1.79 [0.77–4.14] 0.17 The total number of respondents included in the analysis was 279. *p < 0.05 CI: Confidence interval; CPI: comparative performance information; OR: odds ratio; PD: Parkinson’s Disease DISCUSSION The ultimate goal of releasing CPI about the quality of expert physiotherapists is to improve quality of care for people with PD. Consumers of health care have the power to make a contribution to quality of care in competitive health care system by selective provider choice. This study shows that PwP identify aspects of expertise that appeared to align with the IOM-framework for quality of care, and mostly with patient centeredness. Moreover, the majority of participants (74%) were aware of expert and non-expert PD physiotherapists. Participants were able to describe what the PD-specific expertise and knowledge of their physiotherapist means to them. However, we found little evidence suggesting that the influence on how patients value expertise among physiotherapists influences the search for CPI and selective provider choice. Recognition of the additional value of a PD physiotherapist was a strong predictor of such a CPI search. Yet, about half the patients (51%) had this understanding; therefore, this situation can be improved. Our study shows that PwP hardly ever selectively chose a physiotherapist with PD expertise. Most took the physicians’ referral advice (89%), and the influence of CPI in the decision-making process was limited because only a minority searched for such information (11%). 57 4 Chapter 4 In terms of the way forward, we first discuss the selective referral behaviour of physicians. Currently, very few physicians’ selective referrals to expert physiotherapists occur. Only half the participants were given additional information regarding expert PD physiotherapists. A King’s Fund publication22 shows that most GPs in the United Kingdom did not give information about their referrals either. Knowing and recognising PD expertise are necessary conditions for providers’ selective referrals. Without these conditions, it is difficult to provide patients with information. It is also important that referring physicians (e.g. neurologists and GPs) proactively recall this knowledge when they advise and refer patients to a physiotherapist. Referring physicians should support consumers choose selectively by discussing their referral options, so that the choice becomes a matter of shared decision-making. Second, we focus on the consumers’ selective choice behaviour. Previous studies suggest that, although consumers value quality information23,24, the use of CPI is limited1,25,26 among different populations and for a diversity of conditions. Our study confirms this discrepancy. The respondents valued free provider choice as important, but usually followed the referring provider’s advice and did not use CPI to choose their physiotherapist. Previous research shows that once people understand the concept of quality of care, they give a higher value to the measures of quality performance.27 Further research should address in what way this discrepancy can be countered, for example by improving the circumstances so that consumers’ intentions and their behaviour coincide. Characteristics of expertise, as they were perceived and expressed by PwP in their own words (Table 3), are connected with the definition of patient-centred care for PD.19 Other quality criteria, like a large PwP volume, having followed specific training in treating PD and a connection to an expert network, which is important for the members of ParkinsonNet, were less frequently mentioned. We therefore conclude that the definition of expertise among respondents was rather narrow. The aspects that were less frequently mentioned by respondents, might need more attention since recent evidence shows that PwP allocated to multidisciplinary PD care have better quality of life, better motor scores, and less depression.28 Moreover, at a time when cost control dominates the health policy agenda, it is more urgent than ever to support consumers become a force for improving the quality of healthcare.29 Since the concept of ParkinsonNet is cost-effective guiding patients to physiotherapists that are connected to ParkinsonNet, might contribute to the containment of costs in our health care system. In regions where ParkinsonNet was active, PwP received more physical therapy, there were fewer admissions to nursing homes, fewer people needed revalidation treatment, and the reimbursed costs were 58 Recognition of physiotherapists’ expertise in Parkinson’s disease lower.30 These are all reasons for supporting initiatives that enhance knowledge among PwP about the added value of expert physiotherapists, by means of CPI that covers a broad definition of expertise and a range of quality of care criteria. Twenty-eight percent of our study population received treatment from ParkinsonNetaffiliated physiotherapists, while 70% claimed treatment from expert physiotherapists. This percentage of 28 is in line with previous evidence.18 Physiotherapists can be expert in PD without being connected to the multidisciplinary network, but the chance of being treated by an expert outside the network is smaller. Moreover, the multidisciplinary network has been implemented across the entire country. It is most likely that respondents overestimate their physiotherapists’ expertise in PD, otherwise they would have to admit receiving treatment from a physiotherapist who might not provide them with the best possible healthcare. Their overestimation makes switching to a physiotherapist with PD knowledge unlikely, as in their view, physiotherapists are already experts in PD. More consumer knowledge and recognition of expert physiotherapists might lead to an adjustment of consumers’ views towards the level of PD expertise among their physiotherapists and eventually to a search of CPI. Implications Demonstrating the value of expert PD physiotherapists to referring physicians is necessary. Further research should explore whether this will encourage selective referral behaviour. In terms of patient participation, more attention is required to clarify physicians’ decisions to refer to an expert physiotherapist. A practical implication is that CPI should be extended and made more accessible for PwP and their informal caregivers (family and friends). For example by spreading the information through the patient organisations for PwP on their website or magazine, or flyers in the waiting room of primary care practices. Further research is needed to explore how to encourage consumers to use the information. The CPI should also emphasise in more detail how and in what ways expert PD physiotherapists distinguish themselves from generic physiotherapists. Both the added value of expert physiotherapy and the multidisciplinary element should be emphasised. Limitations This study is not without shortcomings. First, as the understanding of the added value of an expert physiotherapist is related to the search for CPI, it would have been better to have more detailed questions about the perception of added value. For this purpose, future studies should focus on the perceived added value of PD expertise in a more extensive way. 59 4 Chapter 4 Second, we do not know whether the participants’ overestimation of the expertise of their physiotherapist correlates with their satisfaction with their treatment and physiotherapist in general. We did not take the element of consumer satisfaction into account. Future work should replicate these findings and control for consumer satisfaction when asking about physiotherapist expertise in PD. A recent paper shows that consumers are generally very satisfied with their physiotherapeutic care.31 Third, only a minority searched for CPI so that the influence of the variables could not always be estimated precisely, which the large confidence intervals reflect. Fourth, information about the number of physiotherapy episodes respondents had in the past year would brought us more insight whether the choice for a physiotherapist was temporally or more on and on. Future research should use items about the number of episodes, the length of the episodes and number of physiotherapists by whom they were treated during these episodes. A strength is that our sample is representative: the mean age of 72 ± 9 years is in line with the data based on the Dutch system of diagnosis-treatment combination (71 ± 10 years). The gender proportion (56% men) was also consistent with the national number.32 According to the Dutch Parkinson guideline, there are no data available regarding the severity of the disease. Conclusions This study shows that recognition of the added value of an expert physiotherapist was found to be the strongest predictor for the search for CPI. The definition of expertise expressed by PwP was in line with patientcenteredness, though in a rather narrow manner, as only certain characteristics of PD expertise were recognised. In order for PwP choosing high-quality care, improvements are needed. There is a lack of recognition of expertise caused by a mismatch in the current available CPI. As PwP showed relatively few signs of selectively choosing expert physiotherapists, CPI should include additional and crucial quality of care information that matters for PwP. After this, it is expected that expert physiotherapists become more easier recognisable for those who want to make a selective provider choice. Furthermore, PwP heavily rely on their referring providers, meaning that referring providers have a responsibility to act as a coach for their patients. This fact should be used advantageously by involving the professionals in a more active way. 60 Recognition of physiotherapists’ expertise in Parkinson’s disease REFERENCES 1. 2. 3. 4. 5. 6. 7. 8. 9. 10. 11. 12. 13. 14. 15. 16. 17. 18. 19. 20. 21. 22. 23. 24. Fung CH, Lim YW, Mattke S, Damberg C, Shekelle PG. Systematic review: the evidence that publishing patient care performance data improves quality of care. Ann Intern Med 2008;148(2): 111–23. Grabowski DC, Town RJ. Does information matter? Competition, quality, and the impact of nursing home report cards. Health Serv Res 2011;46(6pt1):1698–719. Werner RM, Norton EC, Konetzka RT, Polsky D. Do consumers respond to publicly reported quality information? Evidence from nursing homes. J Health Econ 2012;31(1):50–61. Lim JN, Edlin R. Preferences of older patients and choice of treatment location in the UK: a binary choice experiment. Health Policy 2009;91(3):252–57. Marang-van de Mheen PJ, Dijs-Elsinga J, Otten W, Versluijs M, Smeets HJ, Vree R, et al. The relative importance of quality of care information when choosing a hospital for surgical treatment: a hospital choice experiment. Med Decis Making 2011;31(6):816–27. Sivey P. The effect of waiting time and distance on hospital choice for english cataract patients. Health Econ 2012;21(4):444–56. Fanjiang G, von Glahn T, Chang H, Rogers WH, Safran DG. Providing patients web-based data to inform physician choice: if you build it, will they come? J Gen Intern Med 2007;22(10):1463–66. Andlin-Sobocki P, Jonsson B, Wittchen HU, Olesen J. Cost of disorders of the brain in Europe. Eur J Neurol 2005;12(1):1–27. Nijkrake MJ, Keus SH, Oostendorp RA, Overeem S, Mulleners W, Bloem BR, et al. Allied health care in Parkinson’s disease: referral, consultation, and professional expertise. Mov Disord 2009;24(2): 282–86. Keus SH, Bloem BR, Verbaan D, de Jonge PA, Hofman M, van Hilten BJ, et al. Physiotherapy in Parkinson’s disease: utilisation and patient satisfaction. J Neurol 2004;251(6):680–87. Chartered Society of Physiotherapy: Neurology.Parkinson’s disease, multiple sclerosis and severe traumatic brain injury. Physiotherapy Effectiveness Bulletin 2001;3:1–11. Victoor A, Delnoij DM, Friele RD, Rademakers JJ. Determinants of patient choice of healthcare providers: a scoping review. BMC Health Services Res 2012;12:272. Nijkrake MJ, Keus SH, Overeem S, Oostendorp RA, Vlieland TP, Mulleners W, et al. The ParkinsonNet concept: development, implementation and initial experience. Mov Disord Soc 2010; 25(7):823–29. Keus SH, Oude Nijhuis LB, Nijkrake MJ, Bloem BR, Munneke M. Improving community healthcare for patients with Parkinson’s disease: the dutch model. Parkinsons Dis 2012; 2012:543426. Keus SH, Bloem BR, Hendriks EJ, Bredero-Cohen AB, Munneke M. Evidence-based analysis of physical therapy in Parkinson’s disease with recommendations for practice and research. Movement Disord: Official J Movement Disord Soc 2007;22(4):451–60. Nijkrake MJ, Keus SH, Kalf JG, Sturkenboom IH, Munneke M, Kappelle AC, et al. Allied health care interventions and complementary therapies in Parkinson’s disease. Parkinsonism Relat Disord 2007; 13(3):S488–S494. Bloem BR, van Laar T, Keus SHJ, de Beer H, Poot E, Buskens E, et al. Multidisciplinary guideline in Parkinson disease [in dutch: multidisciplinaire richtlijn ziekte van Parkinson]. Alphen a/d Rijn: Van Zuiden Communication, 2010. Munneke M, Nijkrake MJ, Keus SH, Kwakkel G, Berendse HW, Roos RA, et al. Efficacy of community-based physiotherapy networks for patients with Parkinson’s disease: a clusterrandomised trial. Lancet Neurol 2010;9(1):46–54. van der Eijk M, Faber MJ, Al Shamma S, Munneke M, Bloem BR. Moving towards patient-centered healthcare for patients with Parkinson’s disease. Parkinsonism Relat Disord 2011;17(5):360–64. Institute of Medicine. Crossing the quality chasm: a new health system for the 21st century. Wasinghton: National Academy Press, 2001. Hoehn MM, Yahr MD. Parkinsonism: onset, progression and mortality. Neurology 1967;17(5):427–42. Dixon A, Robertson R, Appleby J, Burge P, Devlin N, Magee H. How patient choose and how providers respond. London: The King’s Fund, 2010. Coulter A. Engaging patients in healthcare. London: Open University Press, 2011. Losina E, Plerhoples T, Fossel AH, Mahomed NN, Barrett J, Creel AH, et al. Offering patients the opportunity to choose their hospital for total knee replacement: impact on satisfaction with the surgery. Arthritis Rheum 2005; 53(5):646–52. 61 4 Chapter 4 25. 26. 27. 28. 29. 30. 31. 32. 62 Harris KM, Buntin MB. Choosing a health care provider: the role of quality information. Princeton: Robert Wood Johnson Foundation, 2008. Kaiser Family Foundation. Update on consumers’ view of patient safety and quality information. California: Agency for Healthcare Research and Quality, 2008. Hibbard JH, Greene J, Daniel D. What is quality anyway? Performance reports that clearly communicate to consumers the meaning of quality of care. Med Care Res Rev 2010; 67(3):275–293. van der Marck MA, Bloem BR, Borm GF, Overeem S, Munneke M, Guttman M. Effectiveness of multidisciplinary care for Parkinson’s disease: a randomized, controlled trial. Movement Disord: Official J Movement Disord Soc 2013; 28(5):605–11. Sinaiko AD, Eastman D, Rosenthal MB. How report cards on physicians, physician groups, and hospitals can have greater impact on consumer choices. Health Aff (Millwood) 2012; 31(3):602–611. Beersen N, Berg M, van Galen M, Huijsmans K, Hoeksema N. Study to the value of ParkinsonNet [in dutch: onderzoek naar de meerwaarde van ParkinsonNet]. Zeist: Vektis and Plexus, 2011. Calsbeek H, van Dulmen SA, Nijhuis-van der Sanden MWG, Braspenning JCC. Kwaliefy indicators: results measures 2011 [in Dutch: kwaliefyindicatorenset: resultaten meting 2011]. FysioPraxis 2012. In Press:3. Berg van D. Parksinon disease and parkisonism in The Netherlands. [in dutch: de ziekte van parkinson en parkinsonisme in nederland: een schatting van de prevalentie en incidentie op basis van data uit het DBC informatie systeem]. Amsterdam: Parkinson Vereniging, 2010:1–25. Chapter 5 Public release of performance data in changing the behaviour of healthcare consumers, professionals or organizations (Review) Nicole Ketelaar Marjan Faber Signe Flottorp Liv Rygh Katherine Deane Martin Eccles Cochrane Database Systematic Review 2011; 11. Chapter 5 ABSTRACT Background: It is becoming increasingly common to release information about the performance of hospitals, health professionals or providers, and healthcare organisations into the public domain. However, we do not know how this information is used and to what extent such reporting leads to quality improvement by changing the behaviour of healthcare consumers, providers and purchasers, or to what extent the performance of professionals and providers can be affected. Objectives: To determine the effectiveness of the public release of performance data in changing the behaviour of healthcare consumers, professionals and organisations. Search strategy: We searched the Cochrane Central Register of Controlled Trials (CENTRAL), Cochrane Effective Practice and Organisation of Care (EPOC) Trials Register, MEDLINE Ovid (from 1966), EMBASE Ovid (from 1979), CINAHL, PsycINFO Ovid (from 1806) and DARE up to 2011. Selection criteria: We searched for randomised or quasi-randomised trials, interrupted time series and controlled before-after studies of the effects of publicly releasing data regarding any aspect of the performance of healthcare organisations or individuals. The papers had to report at least one main outcome related to selecting or changing care. Other outcome measures were awareness, attitude, views and knowledge of performance data and costs. Data collection and analysis: Two review authors independently screened studies for eligibility and extracted data. For each study, we extracted data about the target groups (healthcare consumers, healthcare providers and healthcare purchasers), performance data, main outcomes (choice of healthcare provider and improvement by means of changes in care) and other outcomes (awareness, attitude, views, knowledge of performance data and costs). Main results: We included four studies containing more than 35,000 consumers, and 1560 hospitals. Three studies were conducted in the USA and examined consumer behaviour after the public release of performance data. Two studies found no effect of Consumer Assessment of Healthcare Providers and Systems information on health plan choice in a Medicaid population. One interrupted time series study found a small positive effect of the publishing of data on patient volumes for coronary bypass surgery and low-complication outliers for lumbar discectomy, but these effects did not persist longer than two months after each public release. No effects on patient volumes for acute myocardial infarction were found. One cluster-randomised controlled trial, conducted in Canada, studied improvement changes in care after the public release of performance data for patients with acute myocardial infarction and congestive heart failure. No effects for the composite process-of-care indicators for either condition were found, but there were some improvements in the individual process-of-care indicators. There was an effect on the mortality rates for acute myocardial infarction. More quality improvement activities 64 Public release of performance data in changing behaviour were initiated in response to the publicly-released report cards. No secondary outcomes were reported. Authors’ conclusions: The small body of evidence available provides no consistent evidence that the public release of performance data changes consumer behaviour or improves care. Evidence that the public release of performance data may have an impact on the behaviour of healthcare professionals or organisations is lacking. 5 65 Chapter 5 BACKGROUND It is becoming increasingly common to release information about the performance of healthcare systems into the public domain. In the present era of accountability, costeffectiveness, quality improvement and demand-driven healthcare systems, policy and decision-makers such as governments, regulators, purchaser and provider organisations, health professionals and consumers of health care are becoming more interested in measuring performance.1 The measurements may appear in consumer reports, provider profiles or report cards. It is not always clear who the information users are or what the release of data is expected to achieve. However, it is often assumed that the information will affect and facilitate the decisions and behaviours of various stakeholders and ultimately result in health system improvements.1-3 The stakeholders in this review include healthcare consumers, professionals, providers and purchaser organisations. Accountability relationships connect all the stakeholders. These relationships have two prominent elements, namely the ‘provision of information’ about performance and the ‘sanctions or rewards for the accountable party’.1 The main role of performance measurement is to keep the various agents accountable by enabling stakeholders to make informed decisions.1 Various suggested uses of performance measurements are linked to the accountability perspective. Examples of underlying objectives are (1) the use of performance measurements to promote more efficient and demand-driven healthcare4,5 and (2) applying the results as a marketing tool.6 One user goal is to use the public disclosure of performance measurements to encourage providers to focus on quality problems and to stimulate performance improvement.5,7,8 From a healthcare consumer’s perspective, the data can encourage patients to preferentially choose high-quality health care, i.e. the best health plan or provider9-11 or to assess the performance of individual professionals.12,13 Other proposed user goals for performance measurements have been linked to controlling costs,14,15 regulating the healthcare system16,17 and influencing the decisions of healthcare purchasers.18-20 Consumers must overcome barriers to the use of performance data. Examples of such barriers are the complexity of the performance data,21 lack of skills to comprehend and use performance data22-25 and the way data are presented.8,26-28 A negative consequence of such barriers might be related to the impact of choice on equity in healthcare. Consumers from poorer backgrounds and with lower educational levels will be less likely to be given a choice, less able to choose and less able to afford travel to a better performing, but more distant, provider.29 Professionals focus on the barriers to accessibility,30 the validity of the performance measures themselves,31,32 and the validity of implicit or explicit comparisons of 66 Public release of performance data in changing behaviour performance.33,34 There are concerns that failure to adequately adjust for the case mix in the data sets may lead to hospitals or clinicians who treat higher-risk patients being labelled as poor performers, or to providers preferentially selecting lower-risk patients.35-37 In healthcare systems where providers charge for their services, the ‘better’ performing providers may charge more,20 thereby restricting access to better care. Publicly releasing performance data may have other unintended consequences as well. There is a risk that the release may lead to improved reporting without necessarily improving performance. It has been said that the care tasks that are easiest to measure are often those least important in a quality improvement context and that other task measurements will be neglected.38 Thus, the impact of public release of performance data may have various mechanisms. Most commentators seem to consider the most important goal of publishing performance data to be to cause providers to improve their performance. This goal can be achieved in a selection pathway or a change pathway.3 Consumers, patients and purchaser organisations that are in a position to do so can select the best healthcare professionals and organisations. This type of selection will not change the quality of the delivered care by itself, but it can be a stimulus for quality improvement. In a change pathway, healthcare professionals and organisations can improve performance by changing their work procedures or professional culture, and organisations can make structural changes. Description of the intervention Public release of performance data is the release of information about the quality of care so that patients and consumers can better decide what health care they wish to select and healthcare professionals and organisations can better decide what to provide, to improve or to purchase. This mechanism excludes the use of auditing and feedback as a tool for improving professional practice and healthcare outcomes. This subject has been reviewed elsewhere.39 Why it is important to do this review Some systematic reviews2,8,40,41 have suggested positive effects of publicly releasing performance data. However, none of them focuses on identifying and synthesising only the most robust evidence available; this systematic review will do so. Objectives To estimate the effects of publicly releasing performance data on changing the behaviour of three target groups: healthcare consumers (patients), providers of healthcare (health professionals) and purchasers of healthcare. 67 5 Chapter 5 METHODS Criteria for considering studies for this review Types of studies • Randomised controlled trials (RCT), including cluster-randomised controlled trials (ClRCTs) • Quasi randomised trials (QRT), including cluster quasi randomised trials (ClQ-RCTs) using methods of allocation such as alternation or allocation by case note number. • Interrupted time series (ITS) studies with at least three data points before and three data points after the intervention. • Controlled before-after (CBA) studies, with at least two intervention sites and two control sites that are chosen for similarity of main outcome measures at baseline. Types of participants Patients or other healthcare consumers and healthcare providers, including organisations (e.g. hospitals, practices and individual healthcare professionals) without any restriction by type of healthcare professional, provider, setting or purchaser. Types of interventions We included interventions that contained the following elements. • Performance data about any aspect of the healthcare organisations or individuals, including process measures (e.g. waiting times), healthcare outcomes (e.g. mortality), structure measures (e.g. presence of waiting rooms), consumer or patient experiences (e.g. Consumer Assessment of Healthcare Providers and System (CAHPS) data) and/or expert or peer-assessed measures (e.g. certification, accreditation and quality ratings given by colleagues).42 The data presented may or may not provide comparisons with similar providers or quality standards and may or may not be adjusted for case mix. Performance data may be prepared and released by any organisation, such as the government, insurers or consumer organisations. • The release of performance data into the public domain in written or electronic form, with varying degrees of accessibility, such as a report in a publicly accessible library or more active dissemination directly to consumers in newspapers, leaflets, personal mailings, broadcasting media, etc. The data may be presented numerically, graphically or pictorially. Comparators The following comparisons were planned. 1. Public release of performance data compared to control (the control intervention should consist of the usual practice in that setting, which may include other interventions aimed at quality improvement, such as the internal use of the same performance data). 68 Public release of performance data in changing behaviour 2. Different types of public release of performance data compared to each other. We excluded studies that did not expose participants to performance data concerning process measures, healthcare outcomes, structure measure, consumer/patient experiences or expert or peer assessed measures. We also excluded studies that reported only hypothetical choices. Types of outcome measures Main outcome measures We planned to the primary outcome measures according to two important aims of those publicly releasing performance data. 1. Improvement by selection • Changes in the healthcare utilisation decisions of consumers (public and patients) • Changes in the healthcare utilisation decisions of healthcare providers (professionals and organisations) • Changes in the healthcare utilisation decisions of purchasers 2. Improvement by changes in care • Objective measures of provider performance, including those that were made public and others that were not • Valid measures of staff morale or behaviour (‘valid’ defined as having the development of the assessment tool reported in a peer-reviewed journal) Other outcome measures If a study reported at least one main outcome measure we also collected those concerning awareness, attitude, views, knowledge of performance data in all target groups and cost data. Where possible, we planned to collect data about the extent to which outcome measures varied with participant characteristics. We excluded studies that reported awareness, knowledge, attitude or costs in the absence of objective measures of provider performance or decision behaviour of healthcare consumers, providers or purchasers. Where possible, we planned to collect data about the extent to which outcome measures varied with participant characteristics. Search methods for identification of studies We searched the Cochrane Central Register of Controlled Trials (CENTRAL), MEDLINE Ovid (from 1966), EMBASE Ovid (from1979), CINAHL, PsycINFO Ovid (from1806) and DARE up to 2011. For MEDLINE, we used subject headings and the relevant quality of healthcare MeSH terms, such as ‘process assessment’, ‘outcome assessment’, ‘quality indicators’, ‘quality assurance’ and ‘benchmarking’. We also used text words and phrases such as ‘performance outcome‘, ‘report card’, ‘criteria’, ‘standard’, 69 5 Chapter 5 ‘disclosure’, ‘quality information’ and ‘public information’. We combined these terms with forms of decision-making such as ‘choice behaviour’, ‘patient preferences’, ‘patient acceptance’ and ‘consumer satisfaction’. We searched the other databases using the appropriate controlled vocabulary. In addition, we identified potentially relevant studies in the reference lists of key articles. Appendix 1 to Appendix 5 give full details of the search terms. Data collection and analysis Selection of studies We downloaded all titles and abstracts (N=6839) retrieved in the electronic search to a reference management database. We removed the duplicates, then two review authors independently examined the remaining references. All review authors recorded their assessments of abstracts with points: ‘0’ for exclusion, ‘1’ for doubtful and ‘2’ for inclusion. Two review authors independently rated each abstract, therefore a minimum score of ‘0’ and a maximum score of ‘4’ was possible. Abstracts with a combined score of 0 or 1 were excluded. Studies with a combined score of 3 or 4 were included. Two review authors resolved the fate of studies with a combined score of 2 by discussion. A third review author (ME) decided any disagreements that remained unresolved. We documented the reasons for exclusion. We obtained full-text copies of papers taken from references for inclusion. Two authors of our review independently assessed the eligibility of these papers. Data extraction and management After the first selection round, relevant studies were retrieved for full-text reading (Figure 1). We distributed these studies to our authors in such way that they did not receive studies for text reading that they already evaluated in the first round. We extracted the data about the study design, patient and provider characteristics, interventions, outcome measures, and healthcare choices to a form specially designed for our review (Appendix 6). We used another form (Appendix 7) for the studies we retrieved for a more detailed evaluation. The two review authors resolved disagreements by discussion if possible. A third review author (ME or MF) dealt with disagreements that the two review authors could not resolve. Assessment of risk of bias in the included studies We assessed the risk of bias on the basis of the Cochrane Collaboration criteria43: (i) adequate sequence generation, (ii) concealment of allocation, (iii) blinding, (iv) incomplete outcome data, (v) selective reporting and (vi) no risk of bias from other sources. We used three additional criteria that the Cochrane Effective Practice and Organisation of Care (EPOC) Group specifies44: (vii) baseline characteristic similarity, 70 Public release of performance data in changing behaviour (viii) reliable primary outcome measures and (ix) adequate protection against contamination. We used these nine standard criteria for ClRCTs, ClQ-RCTs and CBA studies. We used seven criteria for ITS studies: (i) the intervention is independent of other changes, (ii) the shape of the intervention effect is pre-specified, (iii) the intervention is unlikely to affect data collection, (iv) knowledge of the allocated interventions is adequately prevented during the study, (v) the outcome data are incomplete, (vi) reporting is not selective and (vii) there is no risk of bias from other sources. Two review authors independently examined the risk of bias assessment and resolved disagreements by discussion. There were some disagreements about the rating of the criterion as ‘yes’ instead of ‘unclear’ or vice versa. Sometimes the rating was based on a different phrase in the text. A third review author (MPE or MF) dealt with any disagreements that the two review authors could not resolve. Unit of analysis issues We noted whether studies randomised patients or healthcare providers. If analysis did not allow for clustering of patients within healthcare providers, we recorded a unit of analysis error, because such analyses tend to overestimate the precision of the treatment effect. Data synthesis We report the effect sizes for each outcome for each study. Quantitative synthesis was not possible. RESULTS Description of studies See: Characteristics of included studies; Characteristics of excluded studies. See table 1 ‘Characteristics of included studies’ and table 3 ‘Characteristics of excluded studies’. Results of the search The searches found 6839 references; we excluded 6786 references because the titles and abstracts did not meet our inclusion criteria. We retrieved the full texts of publications listed in 53 references. Four citations were irretrievable and could not be considered for inclusion in the review. An additional search brought two more references to light. Altogether, we retrieved full text versions of 51 papers. Forty-two of these papers did not fulfill the inclusion criteria. We evaluated nine papers in more detail, and subsequently we excluded another five papers. We reported the reasons for exclusion of these 51 studies (including the four irretrievable citations) in the ‘Characteristics of excluded studies’ table. Four papers met the inclusion criteria of the review. Figure 1 presents the study flow chart.45 71 5 Chapter 5 Figure 1. Flowchart for Study Selection References identified by database searching – records screened N = 6839 Potentially relevant studies identified by screening of abstracts N = 53 Additional searches in key articles, other potentially relevant reviews N=2 (49 + 2 = 51) References excluded N = 6786 References not available N=4 (53 – 4 = 49) Studies excluded after full-text reading N = 42 Studies identified for full-text reading N = 51 Studies retrieved for more detailed evaluation Studies excluded based on design N=9 N=5 Studies included N=4 Included studies Characteristics of setting and patients/consumers See tables 1 and 2. 72 Table 1. Characteristics of included studies Farley 2002a Methods Participants Interventions Outcomes Notes Design: ClRCT Unit of allocation: New cases (household units) were randomly assigned to either an experimental or control group. This random assignment was independent of case size, county of residence and initial plan assignment. The Iowa Medicaid office supplied the authors with data files for the full sample of new beneficiaries Unit of analysis: Medicaid beneficiaries. All analyses were corrected for clustering of beneficiaries within cases using Huber/White corrections Sample size calculation: Not done. Statistical significance was assessed at the 0.05 level The number randomised into the trial: 13,077 new beneficiaries in 7016 cases; 6515 beneficiaries in the control group, 6562 beneficiaries in the intervention group Characteristics of participating beneficiaries: 72% of ‘cases’ (family or household units) had 2 or more members with an average of 2.8 beneficiaries per case Age of beneficiaries: Unclear Gender of beneficiaries: Unclear Ethnicity Unclear Characteristics of participating health plans: Two Health Maintenance Organisations (HMOs) under contract with the Medicaid programme and 1 primary care case management plan (MediPass). One was the lower-scoring HMO and the other was the higher-scoring HMO Setting: Iowa, USA. Within the state, counties were divided into 3 groups: those that only contained Medicaid plans and other low-rated plans; those that only contained Medicaid plans and other high-rated plans; and those that contained Medicaid plans and both lower and higher-rated plans Intervention group: Standard enrolment materials and Consumer Assessment of Healthcare Providers and Systems (CAHPS) report delivered by personal mailing to beneficiaries Duration of intervention: February 2000 to May 2000 Intervention deliverer: The Iowa Medicaid office mailed beneficiaries a packet health plan enrolment materials that include items such as a plan enrolment form Control Group: standard enrolment materials delivered by personal mailing to beneficiaries Source of funding for study: Co-operative agreement 5U18HS09204-05; the Agency for Healthcare Research and Quality and the Center for Medicare and Medicare Services Main outcome: For each of the other 2 counties the authors performed a multinomial logistic regression in which for each beneficiary the outcome took on the values 1 = stay in assigned HMO; 2 = switch to other HMO; 3 = switch to MediPass The star charts in the CAHPS report were based on a HMO’s performance. The bar charts included 3 charts with ratings of the health plan, healthcare and personal doctor. Five charts were included of service by the providers or health plan 5 Risk of bias Bias Random sequence generation (selection bias) Authors’ judgement Support for judgement Unclear risk The new cases enrolled during the study period were randomly assigned to an experimental or control group. This random assignment was independent of case size, county of residence and initial plan assignment (p. 326) Allocation concealment (selection Unclear risk The new cases enrolled during the study period were randomly assigned to an experimental or control group (p. 326) bias) Low risk Incomplete outcome data “Medicaid office supplied us with data files for the full sample of new beneficiaries (p. 328)” (attrition bias) All outcomes The full sample consisted of 13,077 new beneficiaries in 7016 cases. Results from all are presented in Table 1 (p. 330) Selective reporting (reporting bias) Low risk Only one outcome which was measured in 100% of all cases/new beneficiaries (p. 330) Other bias Low risk Only one outcome which was measured in 100% of all cases/new beneficiaries (p. 330) Adequate blinding of participants, Unclear risk “The Iowa Medicaid office supplied us with data files for the full sample of new beneficiaries. personnel and outcome assessors? The data identified the initially assigned (default) plan, the final plan (different from the default plan only if an active choice was made), whether CAHPS reports were mailed, the county of residence, and the number of beneficiaries in each case (case size)” (p. 328) Protection against contamination Unclear risk The authors claim that “the mail-based administration of the process, including the distribution of CAHPS reports, effectively isolated beneficiaries in the control group (those not receiving CAHPS reports) from exposure to CAHPS information, be it from other beneficiaries outside their household, benefits counsellors, or media information” (p. 327). This statement has not been verified Baseline characteristics similar? Unclear risk “Demographic variables could not be measured directly (p. 329) and as such, were not reported for the intervention and control groups” Reliable outcome measurements Low risk The primary outcome measure, i.e. plan choice, was extracted from an automated system (P328). No reliability measure for the procedure was reported Farley 2002b Methods Design: ClQ-RCT Unit of allocation: New “cases” (family or household units) were assigned to either an experimental or control group, based on whether the last digit of the case was odd or even. The New Jersey Medicaid office supplied the authors with data files for the full sample of all new 5217 enrollees Unit of analysis: New Medicaid cases; cases are the family units that qualify for Medicaid coverage. Medicaid-eligible family units that include an adult are referred to as adult cases, and those in which only children are Medicaid-eligible are child cases Sample size calculation: No justification for the size of the overall sample Participants Interventions Outcomes Notes Risk of bias Bias Random sequence generation (selection bias) Allocation concealment (selection bias) The number randomised into the trial: 5217 cases. Control: 2,568 cases. Intervention: 2,649 cases Characteristics of participating cases: Cases are the family units that qualify for Medicaid coverage. Medicaid-eligible family units that include an adult are referred to as adult cases, and those in which only children are Medicaid-eligible are child cases. New Jersey requires all members of each Medicaid case to enrol in the same HMO Age: Unclear Gender: Men and women Ethnicity: Whole population Self reported health status: All health statuses Characteristics of participating health plans: Medicaid health plans in the state of New Jersey. The Medicaid program has a form of mandatory (auto-assignment) or voluntary managed care programme, which includes one or more HMOs or (sometimes) a primary care case management plan Setting of care: Medicaid health plans choice between one or more health maintenance organizations (HMOs) and sometimes a primary care case management plan Country: USA, state of New Jersey Intervention group: Standard enrolment materials and Consumer Assessment of Health Plans Study (CAHPS) report delivered by personal mailing Duration of intervention: A 3 week period from 25 March to 15 April 1998 Intervention deliverer: The New Jersey Medicaid office published a 7-page brochure “choosing an HMO” that compared the Medicaid HMO with respect to the consumer ratings and experiences reported in the CAHPS survey Control group: Standard enrolment materials delivered by personal mailing Source of funding: Co-operative agreements No. 5U18HS09204-05;Consumer Assessment ofHealth Plans Study (CAHPS) from the Agency for Healthcare Research and Quality (AHRQ) Main outcome: Auto assignment rates Plan choices The star chart in CAHPS report were based on a HMO’s performance compared to the average in every county of residence. The counts ranged from 20 to 29 stars. The resulting standardized CAHPS ratings ranged from -8.40 (well below the average) to 6.26 (well above the county average) Authors’ judgement Support for judgement High risk Quote: "Based on whether the last digit of the case ID was odd or even, half the cases were randomly assigned to an experimental group and half were assigned to a control group” (p. 989) High risk Allocation concealment was based on case ID number, therefore research investigators enrolling participants could possibly foresee assignment 5 Incomplete outcome data (attrition bias) Selective reporting (reporting bias) Other bias Adequate blinding of participants, personnel and outcome assessors? Low risk Low risk Low risk Unclear risk Protection against contamination Low risk Baseline characteristics similar? Reliable outcome measurements Unclear risk Low risk Romano 2004 Methods Participants Quote: “The analysis of the overall effects of CAHPS included the entire April 1998 sample of enrollees, and is therefore not subject to non-response bias” (p.1002) Results from all hypotheses listed in methods section are reported No details were provided about blinding: Quote: “The New Jersey Medicaid office supplied us with a data file that identified plan choices, auto-assignment and demographics for the full sample” (p. 990) No information was reported. Quote: “about half the cases mailed a CAHPS report said they received and read the report” (p.996) Demographic characteristics were not reported Quote: “The primary outcome measure, i.e. plan choice, was extracted from an automated system”(p. 994). No reliability measures for the procedure were reported Design: Interrupted time series (ITS) Data analysis: The data were analysed using 2 different approaches: 1) Ordinary least squares (OLS). This method does not take into account the interdependency of subsequent measurements. It is, however, adjusted for several independent variables: state-wise hospital volume for same condition or procedure, hospital, monthly volume before publication of report card for each hospital, interaction between state volume and hospital, unrelated volume in each hospital. The method is used as follows: based on model predict volume for hospital for each of the 12 months after publication of report card; model volume based on data preceding publication of report card; aggregate predicted volumes from all hospitals in same performance category; estimate 95% confidence intervals (CI) for predictions; if actual volume falls outside CI, then significant effect of report; 2) Autoregressive (ARIMA) methods with P = 1, q = 0 and the same main effects. As a result of software limitations, the authors were unable to include 2-way interactions. Second-order autoregressive models were also tested but generated very similar results. Because the residual autocorrelations were not significant (P > 0.10) for all but 1 model, the authors did not estimate moving average models. The autoregressive and OLS results differed, especially for AMI volume, so both sets of numbers were reported to demonstrate the sensitivity of our results to different statistical methods. Autoregressive models adjust for observed correlations, in some analyses, between current and previous volume changes; however only the OLS models adjust for hospital-level interactions Patients admitted to hospitals designated as outliers in reports in New York and California Characteristics of participating patients: The total number of patients with a topic condition or procedure, or related condition or procedure, who were admitted to a specific hospital in a specific calendar month California - clinical problems:  Acute myocardial infarction (AMI)  Coronary artery bypass grafting (CABG) (AMI-related)  Percutaneous coronary angioplasty (AMI-related)  Congestive heart failure (AMI-related)  Cervical discectomy (target)  Lumbar discectomy (target)  Back or neck procedures (discectomy-related)  Medical back problems (discectomy-related)  Knee arthroplasty (discectomy-related)  Hip arthroplasty (discectomy-related) New York - clinical problems:  AMI  CABG  Percutaneous coronary angioplasty (AMI-related)  Congestive heart failure (AMI-related) Age: The authors excluded children less than 18 years of age, different groups: < 55 years, 55 to 64 years, 65 to 74 years, > 74 years Gender: No restriction Ethnicity: Black, Hispanic, White Other characteristics: Patients admitted for psychiatric conditions, injury or poisoning, or rehabilitation were excluded. Also patients transferred from other acute care hospitals were excluded, because transfers generally reflect the capabilities of different facilities, or insurance arrangement, rather than consumer’s choice Characteristics of participating hospitals: Acute care, non-federal hospitals. All analyses were limited to hospitals that were included in the report card Setting of care: Hospital/inpatient Country: USA, in states of California and New York Intervention: Annual reports on risk-adjusted outcomes; focused on specific conditions or procedures; the reports incorporate clinical expertise and address regional concerns Intervention duration: Using hospital months. California: study period 24 months before publication of first report in 1993 and 12 months after. In 1996, 24 months before second report and 7 months after New York: hospital ratings released every 12 to 24 months since December 1990 Intervention deliverer: Report cards were published by agencies in California and New York Source of funding: US Agency for Healthcare Research and Quality Interventions 5 Outcomes Notes Main outcome: Change in the utilisation decisions of consumer, healthcare professional or purchasers Study period: California: First report released in December 1993 classified hospital mortality for AMI and complication rated for cervical and lumbar discectomy as either ‘better’ or ‘not better’ than expected. The second report, released in May 1996 rated as either ’better’, ‘worse’, or ‘neither better nor worse’ than expected. New York: Hospital-specific, risk-adjusted mortality rates and 3-category ratings have been released every 12 to 24 months since December 1990 Risk of bias Bias Authors’ judgement Support for judgement Incomplete outcome data (attrition Low risk Aggregate data from administrative databases. California analysis was based on the bias) California Patient Discharge Data Set. The New York analysis was based on the Statewide Planning and Research Co-operative System (SPARCS). The number of drop-outs was not applicable here, since the databases and several independent variables were used to predict each hospitals' patient volume after publication of a report card. Selective reporting (reporting bias) Low risk All outcomes and results outlined in the Methods section are reported in tables and/or text. Results for all primary and secondary objectives are reported. Other bias High risk Main analysis based on the assumption of same trend before and after intervention. Difference from predicted values was reported, rather than change in trend and level. Shape of intervention effect preLow risk Quote: “We predicted what each outlier hospital’s volume should have been in each of the 12 months after publication of a report card. These predicted volumes were aggregated for specified? all hospitals assigned to the same risk-adjusted performance category (e.g. higher-thanexpected AMI mortality) in that report card”. (p.371) Intervention is independent of Unclear risk It is not sure that the intervention occurred independently of other changes over time or other changes? that the outcome was not influence by other confounding variables events during study period Intervention unlikely to affect / Low risk Sources and methods of data collection were the same before and after the intervention bias data collection? Knowledge of the interventions Low risk Data collection based on administrative database and performed retrospectively adequately prevented during the study? Reliable outcome measurements Unclear risk The outcome measures were based on a conceptual framework and hypotheses. Hypotheses were based on a validated assumption (p.368). No clear assessment about the reliability of outcomes measures were reported in the Method section. Tu 2009 Methods Design: Cluster-randomised trial Unit of allocation: Hospitals were randomised to receive either early or delayed feedback of a publicly released report card (p. 2331). The randomisation was stratified by type of hospitals (p. 2332). Types of hospitals were classified as teaching hospitals, large community hospitals, small hospitals (p. 2332). Unit of analysis: Patients treated for acute myocardial infarction (AMI) and congestive heart failure (CHF), taking hospital clustering into account in the analysis Sample size calculation: The study had 84% power to detect 5% absolute difference on the composite quality indicators. The power calculation assumed a baseline performance rate on each composite indicator of 70% (standard deviation 10%) in each study group, and that there would be a secular improvement of 75% (SD 7.5%) in the composite indicator, independent of the study intervention (p. 2332). 86 hospital corporations were randomised: 44 hospitals in the early feedback report card release and 42 hospitals in the delayed feedback report card release (Figure 1, p. 2331) Characteristics of hospitals in the early feedback report card release: At baseline, 5676 patients were admitted with AMI and 5073 patients were admitted with CHF Characteristics of hospitals in the delayed feedback report card release: At baseline, 5070 patients were admitted with AMI and 4220 patients were admitted with CHF Setting: The study was conducted in Ontario, Canada. All 130 acute hospitals were assessed for eligibility and 86 hospitals were included. Baseline assessment: At each participating hospital a target sample of 125 charts (or all patients if < 125 patients were treated) for patients receiving care for AMI and/or CHF between 1 April 1999 and 31 March 2001 was abstracted. The baseline performance was based on a set of 12 process of care indicators for AMI and 6 indicators for CHF. Early feedback report card release: The hospitals received their baseline performance data in October 2003 for internal validation and the results were publicly released at a press conference and on the Web in January 2004 (p. 2332) Duration of intervention: January 2004 to 1 April 1 2004 (inclusive the follow-up period: January 2004 to 31 March 2005) Delayed feedback report card release: The hospitals received their baseline performance data in September 2005 for internal validation and the results were also publicly released on the internet in September 2005 (p. 2332). No extensive media or associated press was covered. Duration of intervention: Not applicable, as the delayed feedback group received the intervention after follow-up data were collected Participants Interventions 5 Outcomes Notes Risk of bias Bias Random sequence generation (selection bias) Allocation concealment (selection bias) Incomplete outcome data (attrition bias) Intervention deliverer: The Canadian Cardiovascular Outcomes Research Team, which is a national team of cardiovascular outcomes researchers from across Canada. The team also was involved, together with the Canadian Cardiovascular Society, in the development and endorsement of the set of quality of care indicators, as used in this study. Source of funding for study: The EFFECT study was supported by a Canadian Institutes of Health Research team grant in cardiovascular outcomes research to the Canadian Cardiovascular Outcomes research Team Main outcome: There was no significant improvement in the composite AMI or CHF processof-care indicator. One out of 12 individual process of care AMI indicators improved significant more in the early feedback group compared to the delayed feedback group. One out of 6 of the individual process of CHF indicators improved significantly more in the early feedback group. Regarding mortality rates, as an outcome indicator, 30-day mortality significantly decreased in the early feedback group for AMI, while 3 other mortality-related measures for AMI and CHF did not change. The survey showed that the early feedback group reported significantly more often the start of one or more quality improvement initiatives for AMI care and for CHF care Authors’ judgement Support for judgement Low risk The hospitals were randomly assigned to the early feedback group or the delayed feedback group Low risk Quote: “This random assignment was stratified by type of hospital and performed by a study statistician” (p. 2332) Unclear risk One hospital withdrew from the baseline phase, after randomisation and 4 withdrew from the follow-up phase, all due to resource constraints (p. 2331). No intention-to-treat analysis was performed. Additional exclusions of patients were not reported, but cannot be verified. Selective reporting (reporting bias) Low risk Results from all indicators, individual and composite, are reported as well as the hospital outcome indicators Other bias Low risk Adequate blinding of participants, High risk Quote: “It was not possible to blind the hospitals to their status” (p. 2332). Quote: “We could not blind the delayed feedback group to the media coverage and personnel and outcome assessors? associated publicity surrounding the study results” (p. 2336). Quote: “Patient charts were abstracted by an experienced research nurse” (p.2332), but it is unclear whether or not she was blinded for allocation Protection against contamination High risk Quote: “There was extensive media coverage following the release of the baseline performance for the early feedback hospitals”(p. 2332). The authors mention that “one unanticipated observation" was that several hospitals in the delayed feedback group reported that they also initiated some quality improvement activities after becoming aware of the publicly released early feedback report card” (p. 2336) Baseline characteristics similar? Table 1 (p. 2331) shows the baseline characteristics of the hospitals and patients. Quote: “The hospitals were well balanced across the 2 groups in terms of clinical characteristics of patients” (p. 2333) Reliable outcome measurements Quote: “The primary outcome measures were a set of national process-of-care quality indicators for AMI and CHF care which were developed and endorsed by the Canadian Cardiovascular Outcomes Research Team” (p. 2332). It is likely that the face validity of the indicators is guaranteed, but no field test was performed. The last step in the validation process was not undertaken and no verdict about the content validity is possible. Therefore, we scores this item as high-risk CIRT: Cluster-randomised controlled trial; ClQ-RCT: cluster quasi-randomised trial; ITS: interrupted time series; HMOs: health maintenance organisations; CBAG: coronary artery bypass grafting; AMI: acute myocardial infarction; CHF: congestive heart failure; SD: standard deviation; CI: confidence interval; CAHPS: Consumer Assessment of Healthcare Providers and System; ARIMA: autoregressive integrated moving average 5 Chapter 5 Table 2. Characteristics of settings and consumers (recorded as patients and Medicaid enrolees) Study Methods 46 Farley 2002a Design: ClRCT Unit of allocation: new cases (household units) Power calculation: not done 47 Design: ClQ-RCT Farley 2002b Unit of allocation: new cases (household units) Power calculation: unclear Romano 200448 Design: ITS Unit of allocation: not applicable Power calculation: unclear Setting of Patients/consumers care Medicaid beneficiaries: 13077 Health plans: Age: unclear HMOs Gender: unclear Medicaid beneficiaries: 5878 Age: Unclear Gender: men and women Health plans: HMOs Patients: Given CABG in New Hospitals: nonYork, and treated for AMI or federal given post discectomy surgery hospitals in California Number of patients: unclear Age: children younger than 18 years were excluded Gender: men and women 49 Tu 2009 Design: ClRCT Patients: 15997 patients Hospitals: Unit of allocation: hospitals treated for AMI or CHF teaching, Power calculation: the study had 84% Age: no restriction community or power to detect 5% absolute difference in Gender: men and women small the composite quality indicators. The assumptions were a baseline performance rate of 70% (SD 10%) for each composite indicator in each study group, and a secular improvement of 75% (SD 7.5%) in the composite indicator, independent of the study intervention. CIRT: cluster-randomised controlled trial; ClQ-RCT: cluster quasi-randomised trial; ITS: interrupted time series; HMOs: health maintenance organisations; CBAG: coronary artery bypass grafting; AMI: acute myocardial infarction; CHF: congestive heart failure; SD: standard deviation We included four studies46-49 comprising more than 35,000 consumers (recorded as patients, and Medicaid enrollees), and 1560 hospitals. Three studies were conducted in the USA and one study was conducted in Canada. Farley46 took place in Iowa and Farley47 in New Jersey; both studies were set in health plans. Romano48 was set in hospitals in California and New York. Tu49 was set in hospitals in Canada. Farley46 conducted their study in 35 of the 99 Iowa counties. These counties represented 60% of the total Iowa Medicaid population. The study included MediPass and two types of health maintenance organisations (HMOs) that differed in their performance as assessed with CAHPS surveys scores: one high and one low-rated. The counties were subdivided into three health plan options: type I (MediPass and two HMOs), type II (MediPass and one HMO with a high rating) and type III (MediPass and one HMO with a low rating). The CAHPS survey measures several dimensions of health plan performance including ratings of health plans, primary doctors and reports of experiences with using a health plan. The ratings are for individual items using response scales ranging from 0 to 10. The 82 Public release of performance data in changing behaviour reports of experiences are composite scores that are averages of responses to sets of individual items with four-category response options. Farley47 was based on the New Jersey Medicaid programme. There was a mandatory HMO enrolment period for Aid for Dependent Children and other welfare-related beneficiaries in 17 of its 21 counties. In February 1998, 91% of these beneficiaries were enrolled in Medicaid HMOs. Romano48 was based on the California Hospital Outcomes Project (CHOP). In California and the Cardiac Surgery Reporting System (CSRS) in New York. Trends in hospital volumes for certain diagnoses after publication of report cards were evaluated. In California the CHOP report published in 1993 evaluated acute myocardial infarction (AMI) mortality at 394 hospitals, complications after lumbar discectomy at 344 hospitals, and complications after cervical discectomy at 277 hospitals. In New York, the CSRS report evaluated 30 hospitals in December 1992 and 31 hospitals in December 1993 and June 1995. In Canada, Tu 200949 evaluated the public release of performance data of 12 processof-care indicators for AMI and six indicators for congestive heart failure (CHF) in 86 hospitals. The hospitals were categorised by either early (2004) or delayed (2005) feedback of a publicly released report card about their baseline performance. The Canadian Cardiovascular Outcomes Research Team and the Canadian Cardiovascular Society developed the indicators. Excluded studies In total, we excluded 47 studies after assessing full copies of the papers. The main reasons for exclusion were: design (study was not a ClRCT, ClQ-RCT, CBA or ITS (34)), interventions did not contain process measures, health care outcomes, structure measures, consumer or patient experiences, expert- or peer-assessed measures (18), no objective outcome data were recorded or available for one or both arms (seven), and/or the study was about hypothetical choices (six). We excluded four studies because we were unable to obtain the full-text articles (Table 3). Table 3. Characteristics of excluded studies Study Alteras 2000 50 Beaulieu 2002 51 Beaulieu 2002 52 Bundorf 2009 53 Dawson 2007 54 Dranove 2008 36 Ettinger 2008 55 Reason for exclusion Unable to retrieve Studydesign Studydesign Studydesign Studydesign Studydesign Studydesign/outcome measure 83 5 Chapter 5 Study Fanjiang2007 56 Fine 1998 57 Fong 2008 58 Fotaki 2008 13 Fox 2001 59 Goldstein 2001 60 Goss 2006 61 Hannan 2003 62 Harris 2002 42 Harris-Kojetin 2007 26 63 Hibbard 1996 Hibbard 2000 64 Hibbard 2001 22 Hibbard 2002 65 Hibbard 2002 66 Hibbard 2003 67 Hibbard 2005 68 Hibbard 2005 69 Hollenbeak 2008 Jensen 2004 71 Jha 2006 72 Jian 2009 73 Knutson 1998 74 Krupat 2004 70 75 Lindenauer 2007 7 Mannion 2003 76 McCormack 2001 77 McCormack 2001 78 Moscucci 2005 79 Norem 2004 80 O’Connor 1991 81 Peters 2007 27 Peters 2009 82 Schoenbaum 2001 83 Scott 2006 84 Spranca 2000 85 Spranca 2007 86 Swaminathan 2008 87 Tai-Seale 2004 88 Uhrig2002 89 Uhrig2006 90 Wedig2002 91 Werner 2005 35 Werner 2005 92 Reason for exclusion Studydesign Unable to retrieve Studydesign Studydesign Studydesign Studydesign Studydesign Studydesign, controlled before-after, 2 intervention sites, only1 control site Studydesign Outcome measure/types of intervention Outcome measure Outcome measure Unable to retrieve Studydesign Outcome measures/hypotheticaldata Outcome measures Studydesign Studydesign;2 intervention and 1 control group/raw data was not reported Studydesign Studydesign Studydesign Studydesign Studydesign Types of intervention/outcome measures, design;2 intervention groups, 1 control group Studydesign Studydesign Outcome measures Outcome measures Study design, controlled before-after design; no information reported from the 2 included registries. Not enough information was reported regarding the baseline data Studydesign/outcome measures Unable to retrieve Types of intervention/ outcome measures/hypotheticaldata Studydesign Outcome measures/hypothetical data Studydesign Outcome measures/hypothetical data Studydesign/outcome measures Studydesign Studydesign/interventions/outcome measures Types of intervention/hypothetical data Outcome measures/hypotheticaldata Studydesign, not enough data point for interrupted time series criteria Studydesign Studydesign, a single control and single intervention before and after comparison Risk of bias in included studies We included three study designs (ClRCT, ClQ-RCT and ITS) which we rated on different risk of bias items, we applied items as appropriate for the relevant study design. 84 Public release of performance data in changing behaviour Allocation concealment (selection bias) One study46 provided insufficient information about allocation of concealment to allow judgement of the degree of the risk of bias. One study47 described a non-random method of concealing allocation: research investigators enrolling participants could possibly foresee assignment, therefore there is a high risk of bias. In Tu49 a statistician randomised participating hospitals stratified by type of hospital, we rated this as a low risk of bias. Adequate sequence generation (selection bias) One study46 provided insufficient information about the sequence generation for judging the degree of the risk of bias. One study47 described a non-random method of sequence generation (sequence determined by odd or even case record numbers), so it is possible that selection bias occurred. A third study49 used a random method of sequence generation to assign the hospitals to the early feedback group or the delayed feedback group. Blinding Blinding of the participants was impossible because they had to see what they received.46,47 Analysis was based on computerised discharged abstracts, for which participants could not be blinded.48 It was also impossible to blind hospitals to their randomisation status.49 Incomplete outcome data Three studies46-48 had complete outcome data for the primary outcomes. The results for the entire sample are presented. In Tu49 one of 86 hospitals withdrew from the baseline phase after randomisation, and four withdrew from the follow-up phase, all due to resource constraints, although they did not report a reason for the drop-out. We rated this item as having an ‘unclear risk of bias’: five hospitals dropped out, and this affected both intervention and control groups. Selective reporting We have checked two study protocols, the published reports include the expected outcomes.46,47 As far as the other two studies were concerned,48,49 we were not able to check whether the publications included the expected outcomes. Other potential sources of bias Three studies46,47,49 were free of other bias. The ITS study48 had a potential bias since the collection periods were temporally moved about dependent upon when the hospital became an outlier. 85 5 Chapter 5 Baseline characteristics In two studies46,47 the risk of bias regarding the baseline characteristics is unclear, since they did not report demographic variables for the intervention and control groups. One study49 reported the baseline characteristics across their two groups of hospitals. Reliable outcome measures Two studies46,47 achieved appropriate methods for the outcome measurements, and one study49 did not. The primary outcomes measures were developed in a national team of experts, but the measures were not field-tested. The last step in the validation process was not undertaken, thus the reliability of the measures was impossible to determine. Protection against contamination The risk of contamination in one study was unclear.46 In another study, the risk of contamination was low because the enrolling participants received the enrolment materials in their homes.47 It is likely that a few respondents would discuss the CAHPS material with others, but the reality is that the risk of contamination cannot be managed in such cases, simply because of the nature of public reporting. The third study did not provide an explicit statement regarding the methods used to prevent against contamination.49 There was extensive media coverage following the release of the baseline performance data for the intervention group. The control group also initiated some quality improvement activities after becoming aware of the release of the performance data, which could indicate that the control group had been affected. As in Farley,47 this is difficult to prevent because of the nature of public reporting. However, Tu49 might have seen that the extended media coverage would affect the hospitals in the control group. Intervention independent of other changes In the ITS study48 it is unclear whether the intervention occurred independently of other changes over time or whether that the outcome was influenced by other confounding variables and events during the study period. Shape of intervention effect pre-specified The Romano 2004 study adequately pre-specified the shape of the intervention effect. Knowledge of the allocated interventions adequately prevented during the study The Romano 2004 study dealt with the knowledge of the allocated interventions suitably. 86 Public release of performance data in changing behaviour Intervention unlikely to affect data collection The Romano 2004 study appropriately managed the risk of affecting the data collection. Effects of interventions See: Summary of findings for the main comparison. Characteristics of interventions In Farley46 conducted between February and May 2000, the control group received standard enrolment materials by personal mailing post, including items such as Medicaid benefits, instructions about the enrolment process, and available information sources. The experimental group received this standard enrolment material plus the Consumer Assessment of Healthcare Providers and System (CAHPS) report. Health plans were categorised on the basis of their CAHPS performance, defined as high and low performance plans. The CAHPS measures and report template used bar charts rating the overall health plan, overall healthcare and the personal doctor. Additional charts reported respondents’ views on five aspects of service: ‘getting needed care’, ‘getting care without long waits’, ‘how well doctors communicate’, ‘courtesy, respect and helpfulness’, and ‘health plan customer service’. A three-point scale (sometimes/ never, usually and always) was used. The Iowa Medicaid programme did not offer additional proactive support to the intervention group participants for making health plan choices. In Farley47 conducted in March and April 1998, the control group beneficiaries received the standard mailing of Medicaid enrolment materials. The experimental group received the standard enrolment material plus the CAHPS report. Following the CAHPS convention for comparative rating, a three-star rating was used with one star for plans with survey results that scored significantly lower than average, two stars for those that were not significantly different from the average for all other Medicaid plans in New Jersey, and three stars for plans that scored significantly better than average. The participants were asked to choose one HMO and sometimes a primary care casemanagement plan. The state contracted a private firm to manage the enrolment process and assist participants in choosing their plans. They were able to call a free phone number and ask questions about plans. The contractor also sent ‘health benefit co-ordinators’ into county welfare offices and the community to assist participants in choosing. The Medicaid office automatically assigned participants who did not make to a health plan by the Medicaid office. In Romano 2004,48 report cards were published by agencies in California and New York, reporting on patient outcomes for coronary artery bypass grafting (CABG), acute 87 5 Chapter 5 myocardial infarction (AMI) or postdiscectomy complications. The California data began in 1991 with the California Hospital Outcomes Project. The first report, released in California, December 1993, used a two-category rating and classified hospital mortality for AMI and complication rates for cervical and lumbar discectomy as either ‘better’ and ‘not better’ than expected. The second report, released in May 1996, classified hospital mortality for AMI into three categories as ‘better’, ‘worse’ or ‘neither better nor worse’ than expected. The analysis for California was based on the California Patient Discharge Data Set. The New York Cardiac Surgery Reporting System(CSRS) began in 1989 with the creation of a special data system for cardiac surgery. In New York hospital-specific, riskadjusted mortality rates using a three-category classification have been released every 12 to 24 months since December 1990. The analysis for New York is based on the Statewide Planning and Research Co-operative System. In Tu 2009,49 conducted between April 1999 and April 2005, the early feedback group (42 hospitals) received their baseline performance data of 12 process-of-care indicators for AMI and six indicators for congestive heart failure (CHF) for internal validation checks. The results were publicly released at a press conference and on the internet in January 2004. The early feedback hospitals were encouraged to develop standardised admitting orders and discharge plans, based on the baseline performance. Baseline performance results of the delayed feedback group (N=41 hospitals) were publicly released on the internet in September 2005 after internal validation. To determine the effect of the public release and feedback, clinical information was collected from chart reviews during the follow-up (1 April 2004 to 31 March 2005 inclusive 15,997 patients) and compared with the baseline performance data (1 April 1999 to 31 March 2001 inclusive 20,039 patients). The primary outcome measures were defined as being the difference in the mean hospital-specific performance between the two study groups on two composite indicators, i.e. one for AMI and one for CHF. Main outcome measures Interventions targeting improvement through selection: changes in healthcare utilisation decisions of consumers or healthcare providers In Farley 200246 22.6% of the participants switched from the default health plan to another health plan. Participants in the type I counties with three plan choices were less likely to switch (19.9%) than those in the type II or type III counties with only two choices (25.4% overall). Availability of CAHPS data had no effect on the switching rate; individuals not receiving information moved from lower to higher quality plans as often as those who did receive CAHPS data (Table 4). 88 Public release of performance data in changing behaviour Table 4. Estimated effects of CAHPS information on enrolment choices by new beneficiaries enrolled in the Iowa Medicaid programme Percentages (unadjusted frequenties) Control Intervention (No CAHPS) (CAHPS) Odds ratio (95% CI) Lower CI Upper CI for the CAHPS group Type I counties Assigned to high-rated HMO N=1717 N=1693 Stayed in HMO 84.0% 85.7% 0.80 0.58 1.09 Switched to MediPass 13.2% 10.6% Switched to low-rated HMO 2.7 % 3.8 % 1.36 0.75 2.45 Assigned to low-rated HMO N=1614 N=1679 Stayed in HMO 76.0% 74.7% 1.03 0.75 1.39 Switched to MediPass 14.1% 14.4% Switched to high-rated HMO 9.9% 11.0% 1.13 0.79 1.60 Type II counties N=1087 N=1037 Assigned to high-rated HMO Stayed in HMO 70.5% 71.8% 0.92 0.68 1.24 Switched to MediPass 29.5% 28.2% Type III counties N=2097 N=2153 Assigned to low-rated HMO Stayed in HMO 76.3% 76.4% 0.99 0.79 1.23 Switched to MediPass 23.7% 23.6% CAHPS: Consumer Assessment of Health Plans Study; CI: Confidence Intervals; HMOs: health maintenance organisations; MediPass: Medicaid primary care case management programme High, low-rated: the reports or experience are composite scores that are averages of response to sets of individual items using four-category response options. Farley 200246 Farley 200247 did not find any significant differences between the plan choices of the enrollees in the intervention and control groups. Sixty-eight percent of the intervention group and 69% of the control group chose a plan. The standardised CAHPS rating for those who chose a plan were -0.03 for the intervention and 0.03 for the control groups; 28% and 27% respectively chose the dominant HMO. For those not selecting the dominant HMO, the standardised CAHPS ratings of the selected plan were 1.80 and 1.73 respectively (Table 5). Table 5. Plan choices for April enrollees Proportion choosing a plan Farley 200247 Mean or proportion Reports (n=2649) 0.68 Control (n=2568) 0.69 Romano48 estimated time series models using ordinary least squares (OLS) data from the states of New York and California. They re-analysed the California data with autoregressive integrated moving average (ARIMA) methods. In autoregressive models there were no clear patterns of effect developed between AMI report cards and subsequent hospital volume for either AMI or related AMI conditions. There was a small and temporary increase in volume in low complication rate hospitals for lumbar 89 5 Chapter 5 discectomy. Romano48 only report OLS results from New York because autocorrelation was minimal in that state. The study also found a significant increase in CABG volume for low-mortality hospitals in New York within the first month after publication and a significant decrease in volume for high-mortality outliers in the second month after release of the information (Table 6 and Table 7). Table 6. Mean differences between actual and predicted monthly patient volume for the average outlier hospital, over 4 consecutive months in New York. After publication of a riskadjusted outcome study, using ordinary least-squares regressiona Actual minus predicted monthly patient volume (95% confidence interval) Outlier Month 2 (NY) Month 3 (NY) Month 4 (NY) groupb Month 1 (NY) Better 13.4d (4.3 to 22.6) 5.5 (-3.5 to –14.7) 6.7 (-1.5 to –15.0) 3.0 (-5.0 to 11.0) (D=1.92) Worse -4.0 (-9.0 to –1.0) -7.1d (-12.3 to -1.9) -2.7 (-8.0 to –2.7) -0.9 (-5.9 to 4.1) (D=1.91) NY CABGBetter -4.9 (-12.3 to –2.4) -1.4 (-8.7 to –5.9) -1.9 (-8.7 to –4.8) 0.5 (-6.1 to 7.2) related (AMI) (D=1.96) Worse -4.5c (-8.5 to -0.6) -1.2 (-5.2 to –2.8) -1.6 (-5.4 to –2.2) -6.0d (-9.8 to -2.2) (D=1.38) NY CABG related Better 3.7 (-3.2 to –10.8) 1.1 (-6.0 to –8.3) 0.6 (-6.1 to –7.4) -1.2 (-7.8 to 5.5) (PTCA) (D=2.14) Worse -2.6 (-7.0 to –1.8) -1.4 (-6.0 to –3.1) 0.4 (-4.2 to –4.9) -2.1 (-6.6 to 2.5) (D=1.34) NY CABGBetter -2.8 (-8.7 to –3.1) -4.0 (-9.9 to –2.0) -0.5 (-6.0 to –5.0) -1.7 (-7.1 to 3.7) related (D=1.74) (CHF) Worse -1.0 (-5.8 to –3.9) -2.0 (-7.1 to –3.1) -1.7 (-6.6 to –3.2) -0.1 (-4.8 to 4.7) (D=2.14) a Positive numbers indicate that hospitals in that category had more admissions than predicted; negative numbers indicate that hospitals in that category had fewer admissions than predicted b The Durbin-Watson statistics in this column represent the magnitude of autocorrelation affecting OLS models. Values close to 2 indicates the absence of autocorrelation c Two-tailed p < 0.005; d Two-tailed p < 0.01 AMI: Acute myocardial infarction; CABG: coronary artery bypass grafting; PTCA: percutaneous transluminal coronary angioplasty; CHF: congestive heart failure. Romano48 State Condition or procedure NY CABG (target) 90 Public release of performance data in changing behaviour Table 7. Mean differences between actual and predicted monthly patient volume for the average outlier hospital in California, over 4 consecutive quarters after publication of a riskadjusted outcome study, using autoregressive modelsa (ARIMA) Actual minus predicted monthly patient volume (95% confidence interval) State Condition or Outlier Quarter 2 (CA) Quarter 3 (CA) Quarter 4 (CA) procedure groupb Quarter 1 (CA) CA AMI (target) Betterb 1.9 (-0.1 to -3.9) -1.1 (-3.2 to –0.9) -0.6 (- 2.7 to -1.6) 1.1 (-1.3 to 3.6) Worseb 0.7 (-1.6 to –3.0) 1.0 (-1.4 to –3.5) 0.0 (-2.3 to –2.4) 0.6 (-2.0 to 3.3) CA AMI-related Better -1.1 (-4.9 to –2.7) 4.2 (-0.1 to –8.5) -3.8 (-0.8 to –8.3) -0.1 (-4.6 to 4.5) Worse 1.0 (-1.5 to –3.6) 0.4 (-2.5 to –3.2) 0.4 (-3.2 to –2.5) -1.0 (-4.1 to 2.2) CA Cervical Better 0.2 (-1.1 to –1.5) -0.3 (-1.8 to –1.3) -1.6c (0.0 to –3.2) -0.6 (-2.2 to 1.0) c discectomy Worse -1.1 (-2.0 to -0.0) 0.3 (-0.9 to –1.6) 1.1 (-0.1 to –2.3) 0.9 (-0.4 to 2.1) (target) CA Lumbar Betterb 0.6 c(0.0 to –1.1) 0.3 (-0.3 to –0.9) 0.5 (-0.2 to –1.2) 0.8c (-0.1 to 1.5) discectomy Worse -0.1 (-0.8 to –0.6) -0.1 (-0.9 to –0.7) -0.3 (-1.2 to –0.6) -0.5 (-1.4 to 0.3) (target) CA Discectomy- Betterb 0.4 (-0.1 to –1.9) -0.9 (-2.4 to –0.7) -1.1 (-2.7 to -0.4) 0.4 (-1.4 to 2.1) related Worse -1.4c (-2.4 to -0.3) 0.2 (1.1 to –1.4) 0.0 (-1.2 to –1.2) 0.2 (1.0 to 1.5) a Positive numbers indicate that hospitals in that category hadmore admissions than predicted; negative numbers indicate that hospitals in that category had fewer admissions than predicted. To estimate the total difference in patient volume for the average California hospital in each quarter, the numbers shown should be multiplied by 3 b The Durbin-Watson statistics in this column represent the magnitude of autocorrelation affecting OLS models. Values close to 2 indicates the absence of autocorrelation c Two-tailed P < 0.005 ARIMA: autoregressive integrated moving average; AMI: Acutemyocardial infarction; CABG: coronary artery bypass grafting; PTCA: percutaneous transluminal coronary angioplasty; CHF: congestive heart failure. Romano48 Interventions targeting improvement through changes in care: objectives measures of provider performance Tu 2009 did not find significant differences in either the composite AMI indicator (absolute change 1.5%; 95% CI -2.2% to 5.1%; P=0.43) (Table 6) or composite CHF indicator (absolute change 0.6%; 95% CI -4.5% to 5.7%; P=0.81) (Table 7) in the early feedback group compared with the delayed feedback group. Regarding individual process-of-care indicators, one of the 12 for AMI and one of the six for CHF improved significantly in the early feedback group (Table 8 and Table 9). The AMI 30-days mortality rate was significantly lower in the early feedback group than in the delayed feedback group (absolute change -2.5%; 95% CI -0.1% to -4.9%; P=0.045), while the oneyear mortality rates of the early, and delayed feedback groups were comparable. The 30-days and one-year CHF mortality rates did not differ significantly. In addition to the release of a public report card, there was a hospital survey. The early feedback group initiated more quality improvement activities in response to the publicly released report card (for AMI 73.2% versus 46.7%; P=0.003 and for CHF 61% versus 50%; P=0.04). 91 5 Table 8. Mean changes in acute myocardial infarction (AMI) quality indicators in hospitals after publication of report cards for the early feedback group Early feedback (n=42) Delayed feedback (n=39) Absolute difference for Baseline Follow-up Absolute change % Baseline Follow-up Absolute change % early versus delayed % (95% CI) % % (95% CI) feedback % (CI)* P value % All 12 AMI process-of-care 57.4 65.5 8.2 (5.8 to 10.7) 56.5 63.6 7.1 (4.3 to 10.0) 1.5 (-2.2 to 5.1) 0.43 indicators process-of-care quality indicators: 72.5 Left of standard admission 73.3 -0.8 (-5.9 to 4.3) 72.6 66.4 -6.2 (-13.7 to 1.2) 5.8 (-2.6 to 14.2) 0.17 orders Left ventricular function 45.6 49.8 4.2 (-0.9 to 9.4) 39.3 46.9 7.6 (3.1 to 12.2) -2.0 (-8.7 to 4.7) 0.56 assessment Lipid test ≤ 24 h of arrival 34.1 51.1 17.0 (10.7 to 23.3) 35.7 54.9 19.2 (12.8 to 25.8) -2.9 (-10.7 to 4.9) 0.46 Fibrinolytics ≤ 30 min or 39.0 45.7 6.7 (-0.8 to 14.2) 35.9 43.1 7.2 (-0.5 to 15.1) 3.3 (-5.7 to 12.4) 0.47 primary PCI ≤ 90 min Fibronolytic 19.9 (10.7 to 29.1) 68.8 86.3 17.5 (9.2 to 25.9) -1.6 (-9.5 to 6.4) 0.70 64.4 84.3 administration decided by emergency department physician Fibrinolytics giver prior to 80.4 95.7 16.3 (7.1 to 23.7) 85.5 91.9 6.4 (0.1 to 12.7) 5.8 (1.1 to 10.5) 0.02 transfer to CCU or ICU Aspirin ≤ 6 h of arrival 75.9 82.6 6.7 (3.7 to 9.6) 72.8 77.1 4.3 (0.2 to 8.3) 4.3 (-0.1 to 8.8) 0.06 ß-blockers ≤ 12 h of 28.3 73.7 45.4 (38.8 to 51.9) 32.2 71.3 39.1 (31.3 to 46.8) 3.1 (-5.8 to 12.1) 0.049 arrival Aspirin at discharge 84.6 84.0 -0.6 (-4.2 to 2.7) 84.6 83.1 -1.5 (-6.5 to 3.4) 0.9 (-3.2 to 4.3) 0.75 ß-blockers at discharge 77.4 85.6 8.2 (5.4 to 11.1) 77.4 85.0 7.6 (4.1 to 11.2) 0.6 (-3.2 to 4.3) 0.75 ACE inhibitors or ARB for 75 81.7 6.7 (1.0 to 12.4) 71.6 77.0 5.4 (-0.8 to 11.5) 2.8 (-5.2 to 10.8) 0.48 left ventricular dysfunction Statin at discharge 57.6 85.5 27.9 (20.0 to 35.8) 57.8 85.8 28.0 (19.7 to 36.3) -0.3 (-9.0 to 8.5) 0.95 ACE: angiotensin-converting enzyme; ARB: angiotensin II receptor blocker; CCU: coronary care unit; CI: confidence interval; ICU: intensive care unit; PCI: percutaneous coronary intervention. Positive values indicate better performance in the early feedback hospitals. Tu 200949 Table 9. Mean changes in congestive heart failure (CHF) quality indicators among hospitals after publication of report cards for the early feedback group Delayed feedback (n=39) Early feedback (n=42) Baseline % Follow-up % Absolute change % (95% CI) Baseline % Follow-up % Absolute difference for Absolute change % early versus delayed P value feedback % (CI)a (95% CI) All 6 CHF process-of54.8 54.6 -0.2 (-5.0 to 4.6) 51.8 53.6 1.8 (-2.7 to 6.1) 0.6 (-4.5 to 5.7) 0.81 care indicators Individual CHF process-of-care quality indicators Left ventricular 47.9 55.2 -7.3 (1.5 to 13.0) 43.4 52.5 9.1 (3.5 to 14.6) 1.2 (-5.3 to 7.7) 0.72 function assessment Daily weights recorded 14.8 24 9.2 (4.3 to 14.0) 15.1 22.7 7.6 (2.4 to 12.8) 1.8 (-5.2 to 8.8) 0.60 Counselling on ≥ 1 56.2 -10.5 (-18.2 to 2.7) -0.4 (-8.4 to 7.6) 0.92 68.4 55.3 -13.0 (-21.8 to -4.5) 66.7 aspect of CHF ACE inhibitor or ARB for left ventricular 88.2 92.4 4.2 (0.7 to 7.8) 86.5 86.1 -0.4 (-7.4 to 6.5) 5.9 (1.0 to 10.7) 0.02 dysfunction ß-blocker for left 40 71.7 31.7 (22.6 to 40.9) 38.3 67.7 29.4 (18.9 to 39.8) 3.5 (-6.1 to 13.1) 0.47 ventricular dysfunctionb Warfarin for artrial 52.4 64.2 11.8 (4.3 to 19.2) 49.3 63.6 14.3 (6.8 to 22.0) -0.2 (-6.5 to 6.2) 0.96 fibrillation ACE: angiotensin-converting enzyme; ARB: angiotensin II receptor blocker; CCU: coronary care unit; CI: confidence interval; ICU: intensive care unit; PCI: percutaneous coronary intervention. a Represents mean relative improvement in each indicator in the early feedback hospitals with the delayed feedback hospitals in the follow-up patient cohort after adjusting for indicator performance in the baseline patient cohort and type of hospital. Positive values indicate better performance in the early feedback hospitals. b Patients with documented ejection fraction of 40% or less during the index admission or within the previous 6 months were considered to have left ventricular dysfunction. 5 Chapter 5 Other outcome measures Awareness, attitude, views and knowledge of performance data and cost data were not reported in three of the included studies. Farley47 reported secondary outcomes as a result of a survey. Farley47 used a ratio of 3:1 (report versus no report) to send the survey. They had problems with differential sampling and response rates: therefore the data become difficult to interpret. We decided to exclude these results and so do not report these outcomes. Data syntheses We have summarised the outcome data extracted from papers in a narrative format in the section ‘Characteristics of interventions’. We did not synthesise any quantitative data. DISCUSSION Summary of main results In four studies interventions consisting of either direct (mailed) or indirect (internet) release of performance data were focused on changes in selection or changes in care. For changes in selection, from two studies we found no impact on choice of health plan in Medicaid populations. From one study there was a small effect of the public release of mortality and complication data on patient volumes for CABG surgery, and lumbar discectomy; however, these effects did not persist for more than two months after each release. There were no effects of releasing mortality data on patient volumes for AMI outcomes. The changes of care were evaluated for the indirect release of performance data for patients with AMI and CHF in one study. There were no effects on composite process-of-care indicators for either condition, but there were some improvements in individual process-of-care indicators for both AMI and CHF. There was also an effect on 30 day AMI mortality rates, and more quality improvement activities were initiated in response to the publicly-released report cards. Overall completeness and applicability of evidence The three studies that took place in the USA involved only a small proportion of the numerous major reporting systems available. We included one new study from Canada49 that was published after the last systematic reviews by Shekelle,40 Fung8 and Faber.41 We excluded many of the more recent studies because they did not have a rigorous study design or did not report the defined primary outcome measures. Regarding overall completeness, we conclude that evaluations of public reporting system are scarce. Only a few current reporting systems have been subjected to scientific evaluation to determine the effects of public disclosure of quality information 94 Public release of performance data in changing behaviour in various stakeholders.40,93 Studies that compare different reporting systems are lacking, as are studies of purchaser behaviour. Despite evidence that secondary outcome measures (e.g. awareness, attitude, knowledge of performance data) are crucial since public reporting can only change behaviour if the target population (healthcare consumers, providers or purchasers of care) understand the provided information,21 these measures are lacking in the included studies. Because of that it is difficult to explain the lack of effect. Faber41 demonstrated that effect of performance data was higher for those who understand the information. Damman28 showed that comparative performance information is complex, and consumers had difficulties in interpreting and using performance data. One type of performance information included in our studies was about patients experiences (CAHPS), items e.g. regarding doctor-patient communication, long waits, respect. Other included types of performance information were mortality, and complication data. Patient-Reported outcome measures (PROMs) were not included, nor was performance information about services. Mortality and complication data were included, but only for two conditions (AMI, CHF) and two surgical procedures (CABG, discectomy). Quality of the evidence The quality of the evidence in this review appears to be low based on the analysis with the Grades of Recommendations Assessment, Development and Evaluation (GRADE) system. We downgraded the quality of the evidence for the outcomes due to some concerns with risk of bias in the studies, loss to follow-up, and very sparse data. There is one more source of concern regarding the quality of the included studies: we did not have access to the complete study protocols for two of the studies, so we could not judge the risk of selective reporting definitively. The issue of contamination is difficult to tackle for a public reporting intervention because it is often impossible to prevent control groups from seeing information that is publicly available on websites and in the media. Control of exposure can be gained if the information is only posted to consumers personally or if the control and intervention groups are geographically separated. In one study,49 there was extensive media coverage when hospitals in the early feedback group received their baseline performance data. A survey among hospitals in the delayed feedback group confirmed that these control hospitals were affected by the release of performance data in the early feedback group. 95 5 Chapter 5 Potential biases in the review process Although our search was comprehensive, we cannot exclude the possibility of having missed relevant studies. We were unable to retrieve and assess four possibly relevant studies in full text. Two review authors independently examined all the references we found in our search. Two review authors independently extracted detailed data and assessed the risk of bias and a third review author settled any disagreements. We did this to exclude bias in the review process. Agreements and disagreements with other studies or reviews There are three relevant publications: an article by Kolstad10 and two systematic reviews by Faber41 and Fung 2008.8 Our conclusion agrees with those of Kolstad10 and Faber;41 we do not know the extent to which quality reporting leads to improvement of health care quality. We also agree with the conclusion of Fung 2008;8 despite the existence of major public reporting systems, we lack rigorous evaluations of the effects of these systems. AUTHORS’ CONCLUSIONS Implications for practice The results of this review do not enable us to make any strong recommendations for practice. Whilst performance data may be publicly released for many reasons, we cannot conclude from the limited evidence whether disclosure of performance information can reliably change the behaviour of consumers, providers, purchasers or professionals. Implications for research In order to understand the effectiveness of the public release of performance data, we need more longitudinal studies with robust evaluation designs and, in particular, studies that test for delayed or cumulative effects with continuing measurements. To improve our insight into the current and potential impacts of public reporting, we need to evaluate a variety of reporting systems in the USA and other countries. As the lack of effect might be due to a missing of actual exposure to performance data, a specific implication for future studies targeting the consumer’s choice behaviour is that the intervention group (i.e. those provided with performance data) should actually read and understand the performance data. Additional interventions might enhance the impact on consumers with limited health literacy in the intervention group.25 Studies targeting improvements effected by changes in care might benefit from baseline performance data for the intervention group that is released repeatedly instead of only once. 96 Public release of performance data in changing behaviour Berwick’s model suggests that public release of performance data may improve quality of care by means of a pathway of change or selection.3 The studies we included focused on either one or the other of these pathways exclusively. We suggest a study design that combines the two pathways to assess the relationship between them. A basic assumption underlying the provision of report cards is that provider choice is a rational decision. In other words, consumers prefer the healthcare provider or health plan rated as the best. Evidence that confirms this assumption is limited.10,41 However, several factors that influence the choice of consumers are known, such as established relationships with local physicians, health plans, 9,94 hospitals, distance, and opinions of friends, and family.29,30 Future studies should address the range, and relative impact of factors such as these. 5 97 Chapter 5 REFERENCES 1. 2. 3. 4. 5. 6. 7. 8. 9. 10. 11. 12. 13. 14. 15. 16. 17. 18. 19. 20. 21. 22. 23. 24. 98 Smith PC, Mossialos E, Papanicolas I, Leatherman S. Performance measurement for health system improvement. Experiences, Challenges and Prospects. Cambridge: Cambridge University Press, 2009. Marshall MN, Shekelle PG, Leatherman S, Brook RH. The public release of performance data: what do we expect to gain? A review of the evidence. JAMA 2000;283(14):1866-74. Berwick DM, James B, Coye MJ. Connections between quality measurement and improvement. Med Care 2003;41(1 Suppl):I30-38. Bentley JM, Nash DB. How Pennsylvania hospitals have responded to publicly released reports on coronary artery bypass graft surgery. Jt Comm J Qual Improv 1998;24(1):40-49. Hendriks M, Spreeuwenberg P, Rademakers J, Delnoij DM. Dutch healthcare reform: did it result in performance improvement of health plans? A comparison of consumer experiences over time. BMC Health Serv Res 2009;9167. Longo DR, Land G, Schramm W, Fraas J, Hoskins B, Howell V. Consumer reports in health care. Do they make a difference in patient care? JAMA 1997;278(19):1579-84. Lindenauer PK, Remus D, Roman S, Rothberg MB, Benjamin EM, Ma A, et al. Public reporting and pay for performance in hospital quality improvement. N Engl J Med 2007;356(5):486-96. Fung CH, Lim YW, Mattke S, Damberg C, Shekelle PG. Systematic review: the evidence that publishing patient care performance data improves quality of care. Ann Intern Med 2008;148(2): 111-23. Hibbard JH. Using systematic measurement to target consumer activation strategies. Med Care Res Rev 2009;66(1 Suppl):9S-27S. Kolstad JT, Chernew ME. Quality and consumer decision making in the market for health insurance and health care services. Med Care Res Rev 2009;66(1 Suppl):28S-52S. Werner RM, Konetzka RT, Kruse GB. Impact of public reporting on unreported quality of care. Health Serv Res 2009;44(2 Pt 1):379-98. Marshall MN, Romano PS, Davies HT. How do we maximize the impact of the public reporting of quality of care? Int J Qual Health Care 2004;16 (Suppl 1):i57-63. Fotaki M, Roland M, Boyd A, McDonald R, Scheaff R, Smith L. What benefits will choice bring to patients? Literature review and assessment of implications. J Health Serv Res Policy 2008;13(3): 178-84. Berwick DM, Wald DL. Hospital leaders’ opinions of the HCFA mortality data. JAMA 1990;263(2): 247-49. Sirio CA, McGee JL. Public reporting of clinical outcomes--the data needs of health care stakeholders. Am J Med Qual 1996;11(1): S78-81. Rosenthal GE, Hammar PJ, Way LE, Shipley SA, Doner D, Wojtala B, et al. Using hospital performance data in quality improvement: the Cleveland Health Quality Choice experience. Jt Comm J Qual Improv 1998;24(7):47-60. Schut FT, Van de Ven WP. Rationing and competition in the Dutch health-care system. Health Econ 2005;14(Suppl 1):S59-74. Brook RH. Health care reform is on the way: do we want to compete on quality? Ann Intern Med 1994;120(1):84-86. Hibbard JH, Jewett JJ, Legnini MW, Tusler M. Choosing a health plan: do large employers use the data? Health Aff (Millwood) 1997;16(6):172-80. Mukamel DB, Mushlin AI. Quality of care information makes a difference: an analysis of market share and price changes after publication of the New York State Cardiac Surgery Mortality Reports. Med Care 1998;36(7):945-54. Hibbard JH, Greene J, Daniel D. What is quality anyway? Performance reports that clearly communicate to consumers the meaning of quality of care. Med Care Res Rev 2010;67(3):275-93. Hibbard JH, Peters E, Slovic P, Finucane ML, Tusler M. Making health care quality reports easier to use. Jt Comm J Qual Improv 2001;27(11):591-604. Magee H, Davis LJ, Coulter A. Public views on healthcare performance indicators and patient choice. J R Soc Med 2003;96(7):338-42. O’Meara J, Kitchener M, Collier E, Lyons M, de Billwiller-Kiss A, Simon LP, et al. Case study: development of and stakeholder responses to a nursing home consumer information system. Am J Med Qual 2005;20(1):40-50. Public release of performance data in changing behaviour 25. 26. 27. 28. 29. 30. 31. 32. 33. 34. 35. 36. 37. 38. 39. 40. 41. 42. 43. 44. 45. 46. 47. 48. 49. Hibbard JH, Peters E, Dixon A, Tusler M. Consumer competencies and the use of comparative quality information: it isn’t just about literacy. Med Care Res Rev 2007;64(4):379-94. Harris-Kojetin LD, Uhrig JD, Williams P, Bann C, Frentzel EM, McCormack L, et al. The “choose with care system” - development of education materials to support informed Medicare health plan choices. J Health Commun 2007;12(2):133-56. Peters E, Dieckmann N, Dixon A, Hibbard JH, Mertz CK. Less is more in presenting quality information to consumers. Med Care Res Rev 2007;64(2):169-90. Damman OC, van den Hengel YK, van Loon AJ, Rademakers J. An international comparison of webbased reporting about health care quality: content analysis. J Med Internet Res 2010;12(2):e8. Dixon A, Robertson R, Appleby J, Burge P, Devlin N, Magee H. Patient choice. How patients choose and providers respond. In, London: The King’s Fund, 2010. Harris KM, Beeuwkes Buntin M, Cooperation. TR. Research Synthesis Report. Choosing a healthcare provider: the role of quality information. Princeton: Robert Wood Johnson Foundation, 2008. Giuffrida A, Gravelle H, Roland M. Measuring quality of care with routine data: avoiding confusion between performance indicators and health outcomes. BMJ 1999;319(7202):94-98. Kerr EA, Hofer TP, Hayward RA, Adams JL, Hogan MM, McGlynn EA, et al. Quality by any other name?: a comparison of three profiling systems for assessing health care quality. Health Serv Res 2007;42(5):2070-87. Parry GJ, Gould CR, McCabe CJ, Tarnow-Mordi WO. Annual league tables of mortality in neonatal intensive care units: longitudinal study. International Neonatal Network and the Scottish Neonatal Consultants and Nurses Collaborative Study Group. BMJ 1998;316(7149):1931-35. Rixom A. Performance league tables. BMJ 2002;325(7357):177-78. Werner RM, Asch DA. The unintended consequences of publicly reporting quality information. JAMA 2005;293(10):1239-44. Dranove D, Sfekas A. Start spreading the news: a structural estimate of the effects of New York hospital report cards. J Health Econ 2008;27(5):1201-7. Bardach NS, Cabana MD. The unintended consequences of quality improvement. Curr Opin Pediatr 2009;21(6):777-82. Loeb JM. The current state of performance measurement in health care. Int J Qual Health Care 2004;16(Suppl 1)i5-9. Jamtvedt G, Young JM, Kristoffersen DT, O’Brien MA, Oxman AD. Audit and feedback: effects on professional practice and health care outcomes. Cochrane Database Syst Rev 2006(2):CD000259. Shekelle PG, Lim Y-W, Mattke S, Damberg C, Southern California Evidence-based Practice Centre, Corporation. R. Does public release of performance results improve quality of care? A systematic review. London: The Health Foundation, 2008. Faber M, Bosch M, Wollersheim H, Leatherman S, Grol R. Public reporting in health care: how do consumers use quality-of-care information? A systematic review. Med Care 2009;47(1):1-8. Harris KM. Can high quality overcome consumer resistance to restricted provider access? Evidence from a health plan choice experiment. Health Serv Res 2002;37(3):551-71. Higgins JPT, Green S. Cochrane Handbook for Systematic Reviews of Interventions Version 5.1.0 The Cochrane Collaboration, 2011. www.cochrane-handbook.org, 2011. EPOC. EPOC risk of bias guideline. http://epoc.cochrane.org/epoc-resources-review-authors. 2009. Moher D, Cook DJ, Eastwood S, Olkin I, Rennie D, Stroup DF. Improving the quality of reports of meta-analyses of randomised controlled trials: the QUOROM statement. Quality of Reporting of Meta-analyses. Lancet 1999;354(9193):1896-900. Farley DO, Elliott MN, Short PF, Damiano P, Kanouse DE, Hays RD. Effect of CAHPS performance information on health plan choices by Iowa Medicaid beneficiaries. Med Care Res Rev 2002;59(3): 319-36. Farley DO, Short PF, Elliott MN, Kanouse DE, Brown JA, Hays RD. Effects of CAHPS health plan performance information on plan choices by New Jersey Medicaid beneficiaries. Health Serv Res 2002;37(4):985-1007. Romano PS, Zhou H. Do well-publicized risk-adjusted outcomes reports affect hospital volume? Med Care 2004;42(4):367-77. Tu JV, Donovan LR, Lee DS, Wang JT, Austin PC, Alter DA, et al. Effectiveness of public report cards for improving the quality of cardiac care: the EFFECT study: a randomized trial. JAMA 2009; 302(21):2330-37. 99 5 Chapter 5 50. 51. 52. 53. 54. 55. 56. 57. 58. 59. 60. 61. 62. 63. 64. 65. 66. 67. 68. 69. 70. 71. 72. 73. 74. 100 Alteras TT. Health plan report cards may influence insurers more than consumers: their effect on insurer behavior in Minnesota. Find Brief 2000;3(3):1-2. Dean Beaulieu N, Epstein AM. National Committee on Quality Assurance health-plan accreditation: predictors, correlates of performance, and market impact. Med Care 2002;40(4):325-37. Beaulieu ND. Quality information and consumer health plan choices. J Health Econ 2002;21(1):4363. Bundorf MK, Chun N, Goda GS, Kessler DP. Do markets respond to quality information? The case of fertility clinics. J Health Econ 2009;28(3):718-27. Dawson D, Gravelle H, Jacobs R, Martin S, Smith PC. The effects of expanding patient choice of provider on waiting times: evidence from a policy experiment. Health Econ 2007;16(2):113-28. Ettinger WH, Hylka SM, Phillips RA, Harrison LH, Jr., Cyr JA, Sussman AJ. When things go wrong: the impact of being a statistical outlier in publicly reported coronary artery bypass graft surgery mortality data. Am J Med Qual 2008;23(2):90-95. Fanjiang G, von Glahn T, Chang H, Rogers WH, Safran DG. Providing patients web-based data to inform physician choice: if you build it, will they come? J Gen Intern Med 2007;22(10):1463-66. Fine A. The effect of grading and publicizing the physician groups' performance. Exec Solut Healthc Manag 1998;1(9):2-3. Fong J, Marsh GM, Stokan LA, Weilian S, Vinson C, Ruhl L. Hospital quality performance report: an application of composite scoring. Am J Med Qual 2008;23(4):287-95. Fox MH, Moore J, Zimmerman M, Hill S, Foster CH. The effectiveness of CAHPS among women enrolling in Medicaid managed care. J Ambul Care Manage 2001;24(4):76-91. Goldstein E, Fyock J. Reporting of CAHPS quality information to medicare beneficiaries. Health Serv Res 2001;36(3):477-88. Goss JR, Maynard C, Aldea GS, Marcus-Smith M, Whitten RW, Johnston G, et al. Effects of a statewide physician-led quality-improvement program on the quality of cardiac care. Am Heart J 2006;151(5):1033-42. Hannan EL, Sarrazin MS, Doran DR, Rosenthal GE. Provider profiling and quality improvement efforts in coronary artery bypass graft surgery: the effect on short-term mortality among Medicare beneficiaries. Med Care 2003;41(10):1164-72. Hibbard JH, Sofaer S, Jewett JJ. Condition-specific performance information: assessing salience, comprehension, and approaches for communicating quality. Health Care Financ Rev 1996;18(1): 95-109. Hibbard JH, Harris-Kojetin L, Mullin P, Lubalin J, Garfinkel S. Increasing the impact of health plan report cards by addressing consumers' concerns. Health Aff (Millwood) 2000;19(5):138-43. Hibbard JH, Berkman N, McCormack LA, Jael E. The impact of a CAHPS report on employee knowledge, beliefs, and decisions. Med Care Res Rev 2002;59(1):104-16. Hibbard JH, Slovic P, Peters E, Finucane ML. Strategies for reporting health plan performance information to consumers: evidence from controlled studies. Health Serv Res 2002;37(2):291-313. Hibbard JH, Stockard J, Tusler M. Does publicizing hospital performance stimulate quality improvement efforts? Health Aff (Millwood) 2003;22(2):84-94. Hibbard JH, Stockard J, Tusler M. It isn’t just about choice: the potential of a public performance report to affect the public image of hospitals. Med Care Res Rev 2005;62(3):358-71. Hibbard JH, Stockard J, Tusler M. Hospital performance reports: impact on quality, market share, and reputation. Health Aff (Millwood) 2005;24(4):1150-60. Hollenbeak CS, Gorton CP, Tabak YP, Jones JL, Milstein A, Johannes RS. Reductions in mortality associated with intensive public reporting of hospital outcomes. Am J Med Qual 2008;23(4):279-86. Howgill M, Blaza J, Cunningham L, Foster KL. The ratings game. How important are hospital rankings to consumers? Interview by Joyce Jensen. Mark Health Serv 2004;24(1):40-45. Jha AK, Epstein AM. The predictive accuracy of the New York State coronary artery bypass surgery report-card system. Health Aff (Millwood) 2006;25(3):844-55. Jian W, Huang Y, Hu M, Zhang X. Performance evaluation of inpatient service in Beijing: a horizontal comparison with risk adjustment based on Diagnosis Related Groups. BMC Health Serv Res 2009;972. Knutson DJ, Kind EA, Fowles JB, Adlis S. Impact of report cards on employees: a natural experiment. Health Care Financ Rev 1998;20(1):5-27. Public release of performance data in changing behaviour 75. 76. 77. 78. 79. 80. 81. 82. 83. 84. 85. 86. 87. 88. 89. 90. 91. 92. 93. 94. Krupat E, Hsu J, Irish J, Schmittdiel JA, Selby J. Matching patients and practitioners based on beliefs about care: results of a randomized controlled trial. Am J Manag Care 2004;10(11 Pt 1): 814-22. Mannion R, Goddard M. Public disclosure of comparative clinical performance data: lessons from the Scottish experience. J Eval Clin Pract 2003;9(2):277-86. McCormack LA, Garfinkel SA, Hibbard JH, Norton EC, Bayen UJ. Health plan decision making with new medicare information materials. Health Serv Res 2001;36(3):531-54. McCormack LA, Anderson WL, Uhrig JD, Garfinkel SA, Sofaer S, Terrell SA. Health plan decision making in the Medicare population: results from a national randomized experiment. Health Serv Res 2001;36(6 Pt 2):133-49. Moscucci M, Eagle KA, Share D, Smith D, De Franco AC, O’Donnell M, et al. Public reporting and case selection for percutaneous coronary interventions: an analysis from two large multicenter percutaneous coronary intervention databases. J Am Coll Cardiol 2005;45(11):1759-65. Norem J, Moen MA. The Websites of Norwegian hospitals: do they meet national guidelines and patient’s expectations? J Telemed Telecare 2004;10(5):272-76. O’Connor SJ, Shewchuk RM, Bowers MR. A model of service quality perceptions and health care consumer behavior. J Hosp Mark 1991;6(1):69-92. Petersen LA, Woodard LD, Henderson LM, Urech TH, Pietz K. Will hypertension performance measures used for pay-for-performance programs penalize those who care for medically complex patients? Circulation 2009;119(23):2978-85. Schoenbaum M, Spranca M, Elliott M, Bhattacharya J, Short PF. Health plan choice and information about out-of-pocket costs: an experimental analysis. Inquiry 2001;38(1):35-48. Scott IA, Ward M. Public reporting of hospital outcomes based on administrative data: risks and opportunities. Med J Aust 2006;184(11):571-75. Spranca M, Kanouse DE, Elliott M, Short PF, Farley DO, Hays RD. Do consumer reports of health plan quality affect health plan selection? Health Serv Res 2000;35(5 Pt 1):933-47. Spranca MD, Elliott MN, Shaw R, Kanouse DE. Disenrollment information and Medicare plan choice: is more information better? Health Care Financ Rev 2007;28(3):47-59. Swaminathan S, Chernew M, Scanlon DP. Persistence of HMO performance measures. Health Serv Res 2008;43(6):2033-49. Tai-Seale M. Does consumer satisfaction information matter? Evidence on member retention in FEHBP plans. Med Care Res Rev 2004;61(2):171-86. Uhrig JD, Short PF. Testing the effect of quality reports on the health plan choices of Medicare beneficiaries. Inquiry 2002;39(4):355-71. Uhrig JD, Harris-Kojetin L, Bann C, Kuo TM. Do content and format affect older consumers’ use of comparative information in a Medicare health plan choice? Results from a controlled experiment. Med Care Res Rev 2006;63(6):701-18. Wedig GJ, Tai-Seale M. The effect of report cards on consumer choice in the health insurance market. J Health Econ 2002;21(6):1031-48. Werner RM, Asch DA, Polsky D. Racial profiling: the unintended consequences of coronary artery bypass graft report cards. Circulation 2005;111(10):1257-63. Smith PC, Mossialos E, Papanicolas I. Performance measurement for health system improvement: experiences, challenges and prospects. World Health Organization, 2008. Schwartz LM, Woloshin S, Birkmeyer JD. How do elderly patients decide where to go for major surgery? Telephone interview survey. BMJ 2005;331(7520):821. 101 5 Chapter 5 APPENDICES Appendix 1. MEDLINE search strategy MEDLINE (OVID) Syntax guide / - index term (MeSH heading) exp - explode: includes narrower terms to the index term being exploded .tw. - text word in title or abstract fields $ - truncation/wild card: adds no or more characters ? - truncation/wild card: adds no or one character # - truncation/wild card: retrieves alternative single character adjx - adjacency: required words are adjacent to each other, or within x words of each other .pt. - publication type Description of search strategy 1. random$.tw. 2. multicenter study.pt. 3. randomized controlled trial.pt. 4. randomized controlled trial.pt. 5. clinical trial.pt. 6. intervention studies/ 7. experiment$.tw. 8. (time adj series).tw. 9. (pre test or pretest or (posttest or post test)).tw. 10. random allocation/ 11. impact.tw. 12. intervention?.tw. 13. chang$.tw. 14. evaluation studies/ 15. evaluat$.tw. 16. effect?.tw. 17. comparative studies/ 18. compar$.tw. 19. 1 or 2 or 3 or 4 or 5 or 6 or 7 or 8 or 9 or 10 or 11 or 12 or 13 or 14 or 15 or 16 or 17 or 18 20. editorial.pt. 21. letter.pt. 22. comment.pt. 23. 20 or 21 or 22 24. animals/ 25. humans/ 26. 24 not 25 27. 23 or 26 28. 19 not 27 29. (public release of performance data and healthcare providers).mp. [mp=title, original title, abstract, name of substance word, subject heading word] 30. exp Primary Health Care/ 31. exp Hospitals/ 32. physicians/ 33. health professionals.ab,ti. 34. health personnel/ 35. health plans.ab,ti. 36. health plan.ab,ti. 37. insurance.ab,ti. 38. *Physician’s practice patterns/ 102 Public release of performance data in changing behaviour 39. *Group Practice/ 40. *Institutional Practice/ 41. *Private Practice/ 42. *Family Practice/ 43. *Physicians/ 44. *Physicians, Family/ 45. *Professional Practice/ 46. *Nurses/ 47. *Nurse Clinicians/ 48. *Nurse practitioners/ 49. *Pharmacists/ 50. *Pharmacies/ 51. *Pharmacy/ 52. *Hospitals/ 53. (physician$ or GP? or doctor? or general pract$ or prescriber? or group pract$ or institutional pract$ or partnership pract$ or family pract$ or general pract$ or office pract$ or private pract$ or primary pract$ or nurse or nurses).tw. 54. (pharmacist? or pharmacies or pharmacy).tw. 55. hospital?.tw. 56. physiotherapist.mp. 57. midwife.mp. 58. health care centre.mp. 59. dietician.mp. 60. health care provider.mp. 61. *Allied Health Personnel/ 62. *Dental Clinics/ 63. *Dentists/ 64. *Outpatient Clinics, Hospital/ 65. general pract$.tw. 66. psychologist.mp. 67. psychiatrist.mp. 68. 35 or 36 or 37 69. or/38-67 70. or/30-35 71. 69 or 70 72. 68 or 71 73. quality assurance, health care/ 74. *benchmarking/ 75. *“process assessment (health care)”/ 76. *“outcome assessment (health care)”/ 77. exp Quality Indicators, Health Care/ 78. performance outcome.ab,ti. 79. (quality adj2 indicator?).tw. 80. (quality adj (criteria or criterion or standard? or norm)).tw. 81. (performance adj (indicator? or measure? or data or rating)).tw. 82. disclosure/ 83. Information Services/ 84. report card.ab,ti. 85. quality information.ab,ti. 86. public information.ab,ti. 87. consumer information.ab,ti. 88. patient information.ab,ti. 89. 73 or 74 or 75 or 76 or 77 or 78 or 79 or 80 or 81 or 82 or 83 or 84 or 85 or 86 or 87 or 88 90. exp Consumer Satisfaction/ 91. patient preferences.ab,ti. 92. public reporting.tw. 93. consumer reports.ab,ti. 103 5 Chapter 5 94. decision making.ab,ti. 95. choice behaviour.ab,ti. 96. choice behaviour.ab,ti. 97. exp “Patient Acceptance of Health Care”/ 98. ’provider profiling’.ab,ti. 99. 90 or 91 or 92 or 93 or 94 or 95 or 96 or 97 or 98 100. 28 and 72 and 89 and 99 104 Public release of performance data in changing behaviour Appendix 2. EMBASE search strategy EMBASE (OVID) Syntax guide / - index term (EMTREE heading) exp - explode: includes narrower terms to the index term being exploded .tw. - text word In title or abstract fields $ - truncation/wild card: adds no or more characters ? - truncation/wild card: adds no or one character # - truncation/wild card: retrieves alternative single character adjx - adjacency: required words are adjacent to each other, or within x words of each other .pt. - publication type Description of search strategy 1. exp consumer/ or *consumer health information/ 2. patient preferences.ab,ti. 3. *patient attitude/ 4. *patient participation/ 5. *decision making/ 6. *patient decision making/ 7. 6 or 4 or 1 or 5 or 3 or 2 8. *total quality management/ 9. *performance measurement system/ 10. public reporting.mp. 11. *decision making/ 12. *outcome assessment/ 13. *interpersonal communication/ 14. *health care quality/ or exp clinical indicator/ or exp “quality of nursing care”/ 15. *quality control/ 16. report card.ab,ti. 17. public information.mp. 18. consumer information.mp. 19. 13 or 18 or 10 or 14 or 16 or 9 or 15 or 8 or 11 or 17 or 12 20. random$.tw. 21. multicenter study.mp. 22. randomized controlled trial.mp. 23. randomized controlled trial.mp. 24. clinical trial.mp. 25. intervention studies/ 26. experiment$.tw. 27. (time adj series).tw. 28. (pre test or pretest or (posttest or post test)).tw. 29. random allocation/ 30. impact.tw. 31. intervention?.tw. 32. chang$.tw. 33. evaluation studies/ 34. evaluat$.tw. 35. effect?.tw. 36. comparative studies/ 37. compar$.tw. 38. 20 or 21 or 22 or 23 or 24 or 25 or 26 or 27 or 28 or 29 or 30 or 31 or 32 or 33 or 34 or 35 or 36 or 37 39. editorial.pt. 40. letter.pt. 41. comment.pt. 42. 39 or 40 or 41 105 5 Chapter 5 43. animals/ 44. humans/ 45. 43 not 44 46. 42 or 45 47. 38 not 46 48. exp Primary Health Care/ 49. exp Hospitals/ 50. physicians/ 51. health professionals.ab,ti. 52. health personnel/ 53. health plans.ab,ti. 54. health plan.ab,ti. 55. insurance.ab,ti. 56. (physician$ or GP? or doctor? or general pract$ or prescriber? or group pract$ or institutional pract$ or partnership pract$ or family pract$ or general pract$ or office pract$ or private pract$ or primary pract$ or nurse or nurses).tw. 57. (pharmacist? or pharmacies or pharmacy).tw. 58. hospital?.tw. 59. physiotherapist.mp. 60. midwife.mp. 61. health care centre.mp. 62. dietician.mp. 63. health care provider.mp. 64. general pract$.tw. 65. psychologist.mp. 66. psychiatrist.mp. 67. exp Group practice/ 68. exp benchmarking/ 69. exp Institutional practice/ 70. exp Physician’s Practice Patterns/ 71. exp private practice/ 72. exp family practice/ 73. exp physicians/ 74. exp Physicians, family/ 75. exp professional practice/ 76. exp nurses/ 77. exp nurse clinicians/ 78. 67 or 63 or 53 or 71 or 70 or 68 or 48 or 77 or 72 or 65 or 55 or 74 or 50 or 75 or 64 or 57 or 61 or 51 or 58 or 69 or 52 or 59 or 60 or 49 or 56 or 73 or 66 or 76 or 62 or 54 79. 7 and 19 and 78 and 47 106 Public release of performance data in changing behaviour Appendix 3. PsycINFO search strategy (OVID) Syntax guide / - index term (APA thesaurus) exp - explode: includes narrower terms to the index term being exploded .tw. - text word in title or abstract fields $ - truncation/wild card: adds no or more characters ? - truncation/wild card: adds no or one character # - truncation/wild card: retrieves alternative single character adjx - adjacency: required words are adjacent to each other, or within x words of each other .pt. - publication type Description of search strategy 1. random$.tw. 2. multicenter study.tw. 3. randomized controlled trial.tw. 4. clinical trial.tw. 5. intervention studies.mp. 6. experiment$.tw. 7. (time adj series).tw. 8. (pre test or pretest or (posttest or post test)).tw. 9. random allocation.mp. 10. impact.tw. 11. intervention?.tw. 12. chang$.tw. 13. evaluation studies.mp. 14. evaluat$.tw. 15. effect?.tw. 16. compar$.tw. 17. comparative studies.mp. 18. 1 or 2 or 3 or 4 or 5 or 6 or 7 or 8 or 9 or 10 or 11 or 12 or 13 or 14 or 15 or 16 or 17 19. editorial.tw. 20. letter.tw. 21. comment.tw. 22. 21 or 19 or 20 23. animals.mp. 24. humans.mp. 25. 23 not 24 26. 22 or 25 27. 18 not 25 28. Consumer Satisfaction.mp. 29. patient preferences.ab,ti. 30. public reporting.tw. 31. consumer reports.ab,ti. 32. decision making.ab,ti. 33. choice behaviour.ab,ti. 34. 33 or 32 or 28 or 30 or 31 or 29 35. quality assurance, health care.mp. 36. benchmarking.mp. 37. process assessment.mp. 38. outcome assessment.mp. 39. Quality Indicators.mp. 40. performance outcome.ab,ti. 41. (quality adj2 indicator?).tw. 42. (quality adj (criteria or criterion or standard? or norm)).tw. 5 107 Chapter 5 43. (performance adj (indicator? or measure? or data or rating)).tw. 44. disclosure.mp. 45. Information Services.mp. 46. report card.ab,ti. 47. quality information.ab,ti. 48. public information.ab,ti. 49. consumer information.ab,ti. 50. patient information.ab,ti. 51. 35 or 50 or 39 or 40 or 36 or 41 or 48 or 47 or 38 or 42 or 49 or 46 or 45 or 37 or 43 or 44 52. 27 and 34 and 51 108 Public release of performance data in changing behaviour Appendix 4. CINAHL search strategy EBSCO Syntax guide MH - CINAHL subject heading MM - CINAHL major subject heading + - explode: includes narrower terms to the index term being exploded TI - word in the title field AB - word in the abstract field * - truncation/wild card: adds no or more characters Nx - adjacency: required words are adjacent to each other, or within x words of each other PT - publication type Description of search strategy 1. MM “Clinical Trials” 2. TI control* or AB control* 3. TI random* or AB random* 4. MM “Comparative Studies” 5. TI experiment* OR AB experiment 6. TI time N2 series or AB time N2 series 7. TI impact OR AB impact 8. TI intervention* OR AB intervention* 9. Ti evaluat* OR AB evaluat* 10. TI effect? OR AB effect?* 11. “Pretest-Posttest Design+” 12. “quasi-experimental studies+” 13. or/1-12 14. SO cochrane database of systematic reviews 15. 13 not 14 16. MM “Quality of Health Care” or MM “Quality of Nursing Care” 17. MM “Benchmarking” 18. MM “Process Assessment (Health Care)” 19. MM “Outcome Assessment” 20. TI performance outcome OR AB performance outcome 21. TI quality information OR AB quality information 22. TI patient information OR AB patient information 23. TI consumer information OR AB consumer information 24. TI public information OR AB public information 25. TI public reporting OR AB public reporting 26. TI disclosure OR AB disclosure 27. MM “Quality of Health Care” 28. MM “Consumer Satisfaction” 29. TI patient preferences OR AB patient preferences 30. TI consumer reports OR AB consumer reports 31. TI decision making OR AB decision making 32. TI choice behaviour OR AB choice behaviour 33. Ti ’provider profiling’ or AB ‘provider profiling’ 34. TI report card or AB report card 35. S16 or S17 or S18 or S19 or S20 or S21 or S22 or S23 or S24 or S25 or S26 or S34 36. S27 or S28 or S29 or S30 or S31 or S32 or S33 37. S15 and S35 and S36 (exclude Medline records) 5 109 Chapter 5 Appendix 5. DARE & CENTRAL search strategy COCHRANE (PrimaryHealthCare)OR (Hospitals)OR (Health Personnel)OR (physicians)OR (Nurses)OR (Professional Practice/)OR (Physician‘s Practice Patterns) OR (Institution Practice) OR (Nurse Clinicians)OR (Pharmacists) OR (Pharmacy) OR (Physiotherapist) OR (Midwife):ti,ab,kw, in Clinical Trials AND (Consumer Satisfaction/ OR patient preferencesOR public reporting OR consumer reportsOR decision making OR choice behaviour OR choice behaviour OR exp “Patient Acceptance of Health Care”/ OR ’provider profiling’), in Clinical Trials AND (quality assurance, health care/)OR(benchmarking)OR(Quality Indicators,HealthCare/) OR(disclosure/)OR(Information Services/) OR (report card.ab,ti.) OR (performance outcome.ab,ti.) OR (“Outcome Assessment (Health Care)”/) in Clinical Trials, in Clinical Trials 110 Public release of performance data in changing behaviour Appendix 6. Criteria for full-text screening Nr: Review author: Yes Design: randomised controlled trials, quasi-randomised trials, controlled before-after studies, interrupted time series Types of participants: health care providers and professionals, including organisations e.g. hospitals, practice, patients, health care insurance companies, health plans Types of intervention: participant is exposed to performance information (see*)  Process measures (e.g. waiting times)  Healthcare outcomes (e.g. mortality)  Structure measure (e.g. presence of waiting rooms)  Patient experiences (like CAHPS)  Expert of peer-assessed measures Types of outcome measures: Primary outcomes:  Consumer choice of healthcare provider (public and patients)  Healthcare professional choice of healthcare provider  Purchasers choice of healthcare provider  Objective measures of provider performance, including those that were made public and others that were not  Valid measures of staff morale or behaviour (“valid” defined as having the development of the assessment tool reported in a peer reviewed journal) Secondary outcomes:  Awareness, attitude, views, knowledge in all target groups  Costs TEMPORARILY INCLUSION No Doubt Comments Crucial; score: ‘no’ or ‘doubt’ = exclusion All participants are important for us, at least 1 type should be central in the study There should be a description of at least one primary outcome. Only one secondary outcome measure is insufficient. Hypothetical behaviour = exclusion 111 5 Chapter 5 Appendix 7. Data collection form Systematic review: the effectiveness of the public release of performance data in changing consumer, healthcare professional or organisational behaviour (Comments can be made either at the question itself or on a separate sheet (please specify question number) Name review author: Date: Article: ID article: Title: Authors: Source + year: Article found in: MEDLINE EMBASE CINAHL PsycINFO Cochrane Other (please specify): Unclear Study period: Study design: The design of the study is (state which): Controlled experimental: randomised controlled trial (RCT) If the author(s) state explicitly (usually by some variant of the term ‘random’ to describe the allocation procedure used) that the groups compared in the trial were established by random allocation, then the trial is classified as a ‘RCT’ (randomised controlled trial). Controlled experimental: controlled clinical trial (CCT) or quasi-randomised studies If the author(s) do not state explicitly that the trial was randomised, but randomisation cannot be ruled out, the report is classified as a ‘CCT’ (controlled clinical trial). The classification ‘CCT’ is where the method of allocation is known but is not considered strictly random, and possibly quasi-randomised trials. Examples of quasi-random methods of assignment include alternation, date of birth and medical record number. Controlled experimental: controlled before-after study (CBA) Involvement of intervention and control groups other than by random process, and inclusion of baseline period of assessment of main outcomes. There are two minimum criteria for inclusion of CBAs in EPOC reviews: at least two intervention sites and two control groups are chosen to be similar in respect of the main outcome measures at baseline. Study and control sites are comparable with respect to dominant reimbursement system, level of care, setting of care and/or academic status. Uncontrolled observational: interrupted time series (ITS) A change in trend attributable to the intervention. There are two minimum criteria for inclusion of ITS designs in EPOC reviews: clearly defined point in time when the intervention occurred. At least three data points before and three after the intervention. Classification of study quality: For all study designs Quality criteria  N/A a) The objective measurement of performance/provider behaviour or patient (health) outcomes  Done  Not clear  Not done Page: b) Relevant and interpretable data presented or obtained  Done  Not clear  Not done Page: 112 Public release of performance data in changing behaviour For RCT and CCT Quality criteria  N/A 1. Concealment of allocation (protection against selection bias) (the unit of allocation was by institution, team or professional and any random process is described explicitly, e.g. the use of random number tables, OR the unit of allocation was by patient or episode of care and there was some form of centralised randomisation scheme, an on-site computer or sealed opaque envelopes were used)  Done  Not clear  Not done 2. Follow-up of professionals (if outcome measures obtained for 80% to 100% of health professionals randomised in the study, do not assume 100% follow-up unless stated explicitly)  Done  Not clear  Not done 3. Follow-up of patients (if outcome measures obtained for 80% to 100% of subjects randomised or for patients who entered the trial, do not assume 100% follow-up unless stated explicitly)  Done  Not clear  Not done Page: 4. Blinded assessment of primary outcome(s)∗ (if the authors state explicitly that the primary outcome variables were assessed blindly OR the outcome variables are objective, e.g. length of hospital stay, drug levels assessed by a standardised test, medical records used)  Done  Not clear  Not done Page: 5. Baseline measurement (if performance or patient outcomes were measured prior to the intervention, and no substantial differences were present across study groups)  Done  Not clear  Not done  N.A. 6. Reliable primary outcome measures (if two or more raters with at least 90% agreement or kappa $ 0.8 OR the outcome is obtained form some automated system, e.g. length of hospital stay, drug levels assessed by a standardised test)  Done  Not clear  Not done Page: 7. Protection against contamination (if allocation was by community, institution or practice and it is unlikely that the control group received the intervention)  Done  Not clear  Not done Page: ∗ If it is a self administered questionnaire: than it is not blinded. Not clear: contact authors For CBA Quality criteria  N/A 1. Contemporaneous data collection (if data collection was conducted at the same time as pre and postintervention periods for study and control activities or sites)  Done  Not clear  Not done Page: 2. Baseline measurement (if performance or patient outcomes were measured prior to the intervention, and no substantial differences were present across study groups)  Done  Not clear  Not done Page: 3. Baseline characteristics are similar for two intervention sites &two control groups in respect of the main outcome measures  Done  Not clear  Not done Page: i. Characteristics for studies using second site (if characteristics of study and control providers (or patients) are reported and similar) Page:  Done  Not clear  Not done  N.A. ii. Characteristics for studies using untargeted activities as controls (if study and control activities are comparable with respect to characteristics of targeted behaviour) Page:  Done  Not clear  Not done  N.A. iii. Characteristics for studies using patients as control (if characteristics of study and control providers (or patients) are reported and similar)  Done  Not clear  Not done Page:  N.A. 113 5 Chapter 5 Quality criteria  N/A 4. Reliable primary outcome measures (if two or more raters with at least 90% agreement or kappa $ 0.8 OR the outcome is obtained form some automated system, e.g. length of hospital stay, drug levels assessed by a standardised test)  Done  Not clear  Not done Page: 5. Follow-up of professionals (if outcome measures obtained for 80% to 100% of health professionals randomised in the study, do not assume 100% follow-up unless stated explicitly)  Done  Not clear  Not done Page: 6. Follow-up of patients (if outcome measures obtained for 80% to 100% of subjects randomised or for patients who entered the trial, do not assume 100% follow-up unless stated explicitly)  Done  Not clear  Not done Page: 7. Protection against contamination (if allocation was by community, institution or practice and it is unlikely that the control group received the intervention)  Done  Not clear  Not done Page: i. Characteristics for studies using second site (if characteristics of study and control providers (or patients) are reported and similar)  Done  Not clear  Not done Page: ii. Characteristics for studies using untargeted activities as controls (if study and control activities are comparable with respect to characteristics of targeted behaviour)  Done  Not clear  Not done Page: iii. Characteristics for studies using patients as control (if characteristics of study and control providers (or patients) are reported and similar)  Done  Not clear  Not done Page: For ITS Quality criteria  N/A 1. Clearly defined point in time when the intervention occurred. Page:  Done  Not clear  Not done  N.A. 2. Protection against secular changes: i. Intervention is independent of other changes (if the intervention occurred independent of other changes in time)  Not done Page:  Done  Not clear ii. Sufficient data points to enable reliable statistical inference (if at least 3 data points are recorded before and 3 data points recorded after the intervention)  Done  Not clear  Not done Page: iii. Formal test for trend (if formal test for change in trend using appropriate method is reported (e.g. Cook & Campbell ’79)  Done  Not clear  Not done Page: 3. Protection against detection bias: iv. Data collection is identical before and after intervention (if reported that sources and methods of data collection identical before and after intervention)  Not done Page:  Done  Not clear v. Intervention unlikely to affect/bias data collection ( if reported that the intervention unlikely to affect data collection directly)  Done  Not clear  Not done Page: vi. Blinded assessment of primary outcome(s) (if the authors state explicitly that the primary outcome variables were assessed blindly OR the outcome variables are objective, e.g. length of hospital stay, drug levels assessed by standardised test)  Done  Not clear  Not done Page: 4. Completeness of data set (if data set covers 80% to 100% of total providers and episodes of care in study area)  Done  Not clear  Not done Page: 5. Reliable primary outcome measure(s) (if two or more raters with at least 90% agreement or kappa $ 0.8 OR the outcome is obtained from some automated system, e.g. length of hospital stay, drug levels assessed by standardised test)  Done  Not clear  Not done Page: 114 Public release of performance data in changing behaviour Risk of bias tables of studies with a separate control group (RCTs, CCTs, CBAs) Risk of bias Item Judgement Description Sequence generation Y/N/? Allocation concealment Y/N/? Baseline outcome measurements similar Y/N/? Baseline characteristics similar Y/N/? Incomplete outcome data Y/N/? Knowledge of the allocated interventions Y/N/? adequately prevented during the study Protection against contamination Y/N/? Selective outcome reporting Y/N/? Free of other bias Y/N/? Risk of bias tables of interrupted time series studies Risk of bias Item Judgement Intervention independent of other changes Y/N/? Shape of the intervention effect pre-specified Y/N/? Intervention unlikely to affect data collection Y/N/? Knowledge of the allocated interventions Y/N/? adequately prevented during the study Incomplete outcome data adequately addressed Y/N/? Selective outcome reporting Y/N/? Free of other bias Y/N/? Description Characteristics of study Type of participants  Patients/consumers/providers/purchasers (Medicaid enrollees) Page Number Patient/consumer/client Gender Age Clinical problem Other characteristics  patient  Male  Mixed  Client/consumer  Female 5 1.1.1.1  Hospitals page Number Size Other characteristics 1.1.1.2  Referring physicians page Number Gender Age Clinical specialty Referring to Other characteristics  Primary care  Secondary and/or tertiary care 115 Chapter 5  Purchasers of health care: insurance companies page Private or state? HMO Collective Company that buys care from employees Other         private state yes nol yes no yes no Setting of care  General practice  Outpatient clinic  Community care  Hospital/inpatient  Disabled/inpatient  Elderly  Any care setting  Other (please specify): Medicaid health plans Country  North America, including USA and Canada  South America  Europe  Australia or New Zealand  Asia  Africa  Unclear/not specified Type of control intervention  Usual setting  Other efforts on quality improvement Type of intervention: participant is exposed to performance information All based on actual data Content of performance information  Process measures (e.g. waiting times)  Patient outcomes (e.g. mortality)  Structure measure (e.g. presence of waiting rooms)  Patient experiences (like CAHPS )  Expert or peer-assessed measures  Other, specify…(‘report cards’) Description of intervention for both intervention and control groups Intervention group: Control group: Duration of the intervention: Duration of the intervention: Intensity: Intensity: Duration of follow-up: Duration of follow-up:  Not clear  Not clear 116 Public release of performance data in changing behaviour Information dissemination Intervention Page Control Page Way of data presentation (numerical, graphical, pictorial summary, star ratings...) How is information made available to the participants (personal mailing, journal article, active or passive dissemination) Who disseminated the data? Outcome measures Selection:  Changes in the healthcare utilisation decisions of consumer (public and patients)  Changes in the healthcare utilisation decisions of healthcare professional  Changes in the healthcare utilisation decisions of purchasers Specify:  Choosing the best health plans  Choosing the best healthcare provider (individual physician)  Choosing the best healthcare provider (organisation)  Referring to high quality care provider (individual physician)  Referring to high quality care provider (organisation)  Other….. Changes in care:  Objective measures of provider performance, including those that were made public and others that were not  Valid measures of staff morale or behaviour (’valid’ defined as having the development of the assessment tool reported in a peer  reviewed journal)  Other (for example: number of quality improvement efforts): ......................................................... Attitudes/knowledge/views/understanding/beliefs etc.:  Awareness of information (recall receiving & seen information)  Comprehension of quality of care information (do they understand the information?)  Knowledge about quality of care (’know who is best’)  Believes regarding quality of care information (e.g. trust, usefulness, appreciation)  Costs versus quality consideration  Other………………………………….. Data analyses and results How is outcome data collected? Number of observations for primary outcome measure Proportion of subjects of study who 100%  Not clear (information is not available participate out of the total number in the sampling frame (response rate) Number of drop-outs  Not clear  Not applicable Reason for drop-out mentioned?  yes  no  Not clear  N.A. Confounder or case-mix correction  yes  no  Not clear  partially applied? If Yes: for which variables was corrected? (for example: hospital size) Data analysis technique (s): State the main results of the main outcome measure(s), for each group (pre- and post values; intervention and control groups), in natural units (mean, SD, n) State the corrected intervention effects (mean, 95% confidence interval, P values) Describe the main study conclusion 117 5 Chapter 5 1.Were the conclusions made by the author(s) supported by the data and/or the analysis reported in the article? 2.What of the following is applicable to this study? a) Conclusions inconsistent with results b) Conclusions go beyond the data c) No evidence interpreted as no effect d) Implications for research inconsistent with identified shortcoming 3.Overall (1-2), how would you rate the methods used to analyse the findings relative to the primary question addressed in the study? Comments 118 Yes  Yes     No  No     Can’t tell/partially  Can’t tell/partially      Major limitations  Moderate limitations  Minor limitations Chapter 6 Comparative performance information plays no role in the referral behaviour of GPs Nicole Ketelaar Marjan Faber Glyn Elwyn Gert Westert Jozé Braspenning BMC Family Practice 2014; 15(1): 146. Chapter 6 ABSTRACT Background: Comparative performance information (CPI) about the quality of hospital care is information used to identify high-quality hospitals and providers. As the gatekeeper to secondary care, the general practitioner (GP) can use CPI to reflect on the pros and cons of the available options with the patient and choose a provider best fitted to the patient’s needs. We investigated how GPs view their role in using CPI to choose providers and support patients. Method: We used a mixed-method, sequential, exploratory design to conduct explorative interviews with 15 GPs about their referral routines, methods of referral consideration, patient involvement, and the role of CPI. Then we quantified the qualitative results by sending a survey questionnaire to 81 GPs affiliated with a representative national research network. Results: Seventy GPs (86% response rate) filled out the questionnaire. Most GPs did not know where to find CPI (87%) and had never searched for it (94%). The GPs reported that they were not motivated to use CPI due to doubts about its role as support information, uncertainty about the effect of using CPI, lack of faith in better outcomes, and uncertainty about CPI content and validity. Nonetheless, most GPs believed that patients would like to be informed about quality-ofcare differences (62%), and about half the GPs discussed quality-of-care differences with their patients (46%), though these discussions were not based on CPI. Conclusion: Decisions about referrals to hospital care are not based on CPI exchanges during GP consultations. As a gatekeeper, the GP is in a good position to guide patients through the enormous amount of quality information that is available. Nevertheless, it is unclear how and whether the GP’s role in using information about quality of care in the referral process can grow, as patients hardly ever initiate a discussion based on CPI, though they seem to be increasingly more critical about differences in quality of care. Future research should address the conditions needed to support GPs’ ability and willingness to use CPI to guide their patients in the referral process. 120 Comparative performance information plays no role in the referral behaviour of GPs BACKGROUND As comparative performance information (CPI) about healthcare service, patient experiences, and quality of clinical care becomes increasingly available, questions about its use arise, as do questions about general practitioner (GP) views of CPI at the time of referral. In healthcare systems where the GPs are the gatekeepers of secondary care, which they are in the Netherlands and the UK, GPs refer their patients to specialists for further examination, diagnosis, or treatment. In doing so, they play an important intermediary role between patient and hospital.1-3 International studies have shown that the referral is traditionally affected by previous experiences with specialists, perceptions of specialists’ interactions with patients, office location, specialists’ medical skills, and patient preferences.4-7 These traditional considerations are all understandable. The current focus on the patient’s choice of healthcare provider, with CPI for identifying the quality of provider performance8,9, calls for GPs to reflect anew on the current referral process. The patient’s involvement in choosing a healthcare provider in the Netherlands has been encouraged since regulated competition was introduced during the 2006 healthcare system reform. Publicly available CPI introduced to encourage this competition, contains information about the performance, quality of care and is available for various providers10, also patient experiences plays an increasingly important role. The information covers items at the hospital level (patient volumes, inspection scores defined by the Dutch Health Care Inspectorate, which includes specific conditions such as waiting lists, treatment volumes, treatment methods, methods of anaesthesia, number of specialists treating a given condition, and patient experiences11. The CPI can make an impact when patients select providers of high-quality care on the basis of this kind of information. However, patients hardly use such information for selectively choosing a provider.12,13 Bringing such a choice into practice is a difficult and complex task for patients; e.g. they do not know how to set their own values.14,15 The CPI can be difficult to interpret, especially when it contains conflicting criteria, shows multiplicity formats, or the presentation makes it difficult to understand.14,16,17 Given the lack of CPI usage among patients for selectively choosing a provider, we are looking for ways to provide additional support for the patients. Schlesinger and colleagues advise providing advocates who can help patients with their choices of hospital and who can act on their behalf if they have difficulties putting their choices into practice.18 From the patient’s perspective, this advocate could be the GP. Patients do not seem to search for CPI themselves, but they do ask for advice when choosing a healthcare provider. The GP is an important advisor for about half the Dutch patients.19 because patients consider their GP to be a reliable source of quality information.20 121 6 Chapter 6 Dutch research confirms that GPs have significant influence in directing patients: 68% of the patients who searched for information to select a hospital noted that they based their final decision on GP advice.21 Several studies have revealed how providers respond to performance information.22-25 A 1996 study among cardiologists and cardiac surgeons shows that the publication of report cards for grafts bypassing the coronary artery has little credibility and therefore little influence on referral recommendations.25 A mixed group of physicians described several issues that made them sceptical of the data and concerned about using the information with patients.23 Further, it appears that quality-of-care data have little impact on referral decisions.22 This paper addresses the following research questions:  Can the GP be a choice-supporting advocate for helping patients use comparative performance information?  What are the current referral considerations?  What is the GP’s perception of patient involvement in referral decisions and the use of comparative performance information?  What factors constrain GPs in using comparative performance information in the referral process? We conducted explorative interviews to review their referral routines in which we included the current considerations, patient involvement, and the role of CPI in referral decisions. Using the results of these interviews, we designed and conducted a quantitative survey with a representative sample of Dutch GPs. METHODS Design We used a mixed-method, sequential, exploratory design.26 In this design, the qualitative element is considered first for exploring the research area, then the quantitative element is used to extend and quantify the qualitative results.26 The methods are integrated in three ways. First we focused on building, while the interview results are used in the data collection to build the survey.27 The second way was merging: we used both databases for analysis and comparison. Thirdly, we transformed qualitative data to quantitative data, then integrated the results with illustrative quotes28. A small part of the qualitative data was not transformed in the survey, though it will be used in the results. Participants For the explorative interviews, we recruited GPs from the Nijmegen University Network of General Practitioners and from a network of innovative primary care projects financed by a Dutch healthcare insurance company. The resulting survey questionnaire, 122 Comparative performance information plays no role in the referral behaviour of GPs designed to quantify the issues raised in these interviews, was distributed among a sample of 81 primary care practices affiliated with a representative national network of general practices. Participation was voluntary, and no incentives were offered. The study has been carried out in the Netherlands in accordance with the applicable rules concerning the review of research ethics committees and informed consent. Explorative interviews We conducted explorative interviews with 15 GPs that focussed on referral routines related to three main issues, namely (1) referral considerations (which and why), (2) patient involvement in the referral process in general, and (3) the role of comparative performance information during referral in terms of knowledge about CPI, attitudes towards it, and actual usage behaviour. The first author (NK) interviewed all the GPs. The interviews lasted from 30 to 45 minutes. The interviews were audio-recorded and transcribed verbatim. Two researchers (NK and MF) analysed the transcripts. First, they read the interviews to obtain a comprehensive impression of the material. Second, the data were extensively and inductively coded. Indexing the data created a large number of codes that were repeatedly refined and reduced in several rounds.29 Third, with regard to the role of comparative performance information, a framework analysis was used to approach the data deductively. Cabana and colleagues developed the general guidelines for this framework for improvement.30 The use of CPI and the use of professional guidelines differ, though there is a resemblance in the way they both could be implemented. The successive steps helped us analyse the interviews, and we used them as a guide to present the results in the section ‘Use of comparative performance information’. Figure 1 shows the findings for this part of the interviews. We used the analysis software Atlas.ti.5.2 to facilitate the coding process.31 Survey We built on the issues that arose in the interviews and transformed them into a survey questionnaire. The CPIrelated questions were added to the annual survey of the Dutch National Information Network of General Practice (LINH). The LINH has a nationally representative database maintaining longitudinal data derived from patients’ electronic medical records about consultations, morbidity, drug prescriptions, and referrals.32 The LINH consists of 81 general practices with approximately 335,000 patients. The data were collected between September and December 2012. The survey focused on GP considerations in the current referral process, GPs’ views towards patient involvement, GPs’ current experiences with patient involvement, and the role of comparative performance information in referral decisions. The seven items in GP considerations in current referrals were patient preferences, experiences of other patients, waiting lists, quality of care, specific treatment or techniques, patient travelling distance, and personal contact with a specialist. The question ‘to what extent did the GP consider 123 6 Chapter 6 these items in the decision to refer a patient’ was to be answered on a five-point Likert scale ranging from 1 (never) to 5 (always). Five points for patient involvement were formulated: (1) to what extent did GPs agree with patients’ needs for information about quality differences, (2) how often did patients refer to comparative performance information during consultations, (3) what were GPs’ evaluations of patients’ use of quality information about hospitals, (4) how did GPs perceive the ability of patients to decide on a hospital themselves, and (5) what about the GPs’ view that the use of CPI is the patient’s own responsibility? A fivepoint rating scale ranging from 1 (totally disagree) to 5 (totally agree) was used. The main part was about the role of comparative performance information in referral decisions (21 items). The findings of Figure 1 about knowledge, attitude, and behaviour were listed in the items. The GPs were asked if they knew where to find information about quality of care, and whether they searched for CPI (both dichotomous variables). Ten statements about comparative performance information, containing the elements of attitude and behaviour, were developed. They included elements of quality-of-care differences between hospitals, GP use of comparative performance information to select a hospital, the GPs’ role and responsibility regarding the use of comparative performance information, effects of CPI on the continuity of care, time management regarding comparative performance information, and GP views of the use of CPI in the next 5 years. For the descriptive analyses, the ratings of these items were transformed into the percentages of GPs who agreed (rating of 4 or 5 on the Likert scale). Finally, GPs were asked to express their opinions about the currently available CPI with respect to credibility, transparency about how information was gained, contradictory sources, user friendliness, is it up to date, comprehensibility, the ability of CPI to show quality of care differences, connection to patients’ wishes, and the information content. A five-point scale ranging from 1 (totally disagree) to 5 (totally agree) was used; ‘Do not know’ was a separate option in a separate column. We gathered general practice characteristics about the locality of the practice (urbanization), number of patients in the practice, and the practice type (single or group practice). Background information about the GP included gender and the number of days a week the GP was available in the practice (part time or full time). The data are presented in terms of means (s.d.) or percentages (%). RESULTS The response rate of the survey was 86%. Table 1 presents the characteristics. 124 Comparative performance information plays no role in the referral behaviour of GPs Table 1. Characteristics of the 70 participating general practitioners and their practices GP characteristics Male Full-time GP Practice characteristics Single-handed practice Duo practice Group practice Healthcare centre Urbanization level Very high High Moderate Low Rural Number % 55 66 80 52 38 12 12 8 54 17 17 12 20 11 14 14 11 29 16 20 20 16 Current referral considerations In the interviews, GPs spoke in great detail about their preference for using their own prior experiences and personal contacts with specialists or hospitals when considering their referral. Personal contacts were important to the GPs because they provide an opportunity to ask medical questions and to estimate colleagues’ interactions with patients. In contrast, the surveyed GPs stated that, in deciding about a referral, they primarily considered patients preferences for a hospital or a provider, then the quality of care, and then the distance from the patient’s home (Table 2). ‘I mainly refer on personal grounds and experiences. This might be a really bad thing to do. Still, I think it works this way.’ (N 1) Table 2. The importance of factors in the referral process for selecting a hospital or a specialist Mean (SD) of the 70 responses Patients’ preferences for a hospital or provider 4.3 (0.7) Quality of care 3.8 (1.0) Patient’s travel distance to a hospital or provider 3.8 (0.9) GP’s personal contact with a specialist 3.6 (0.9) Waiting list 3.5 (0.8) Specific treatment or techniques 3.5 (0.8) Experiences of other patients 3.4 (1.0) The responses were given on a five-point Likert scale, with ‘5’ representing ‘always taken into consideration’, and ‘1’ as ‘never taken into consideration’. Sd: Standard deviation Patient involvement During the interviews, the GPs said that they always started by asking what the patient wanted, and they repeatedly highlighted the fact that when selecting a healthcare provider, patients valued other choice attributes than those reflected in CPI. The GPs 125 6 Chapter 6 also noted that comparing providers and making a rational trade-off based on CPI are difficult tasks for patients. ‘Familiarity with a hospital, distance, and knowing where to go: these are much stronger arguments for the patient than quality of care.’ (N 10) ‘The mortality rates in hospital A are better than in hospital B, but A is a generic hospital while B is a top clinical one with an intensive care unit. For that reason, there is a greater chance that people will die in hospital B. The mortality rates show you the data, but you need to interpret them with the background information in your head. As a doctor I can do that, but patients?’ (N 6) There was a high level of agreement between the surveyed GPs about the importance of patients making their own hospital choice and needing to be informed about differences in quality of care (for both items, M=4.0; SD=0.8). There was less agreement about the statements that quality information about hospitals has added value for patients (M=3.0; SD=0.8) and that the use of CPI is a patient’s own responsibility (M=3.0; SD=0.7). There was little GP agreement about how often patients refer to CPI during consultations (M=2.3; SD=0.9). Approximately half the GPs (47%) agreed that they were ‘sceptical about patient use of CPI’ (M=3.0; SD=0.9). Use of CPI Knowledge Most GPs (83%) reported that they did not know where to find performance information to compare their regional hospitals. Attitude: motives The GPs varied widely in their attitude towards CPI. We distinguished four motives that shaped GPs’ negative attitudes towards the use of CPI. First, the GPs want the best care for their patients, and they doubt the role that CPI plays in supporting referral choices. They suggested that they would like to have a ‘tailored’ referral process because the extent of patient involvement in choosing a hospital varies. About half of the surveyed GPs (47%) agreed that they have a task in supporting patients’ hospital choice based on CPI. They recognized CPI as a type of information that can facilitate patient involvement in choosing a hospital. ‘Patients want to steer their decisions.... When they are old and weak they say,‘Put me in the back seat and drive me to the nearest hospital.’ Other patients say,‘Sit next to me and tell me how, but I am the one who’s 126 Comparative performance information plays no role in the referral behaviour of GPs driving.’ And there is a group of patients who ask,‘Where is the highway to the best specialist in this area?’ (N 14) A second motive reported in the interviews was that the use of CPI interferes with the GP’s professional role. The GPs felt responsible for an optimal referral and emphasized their role as coordinators. They expected patients to rely on their referral advice. Therefore, they needed to adjust to the idea that patients can now propose alternatives. They were afraid that patients could decide to go to hospitals outside their professional network as a result of CPI. This could increase the number of medical specialists GPs have to deal with and thereby impede communication since it is easier to talk to someone you know. The GPs also greatly valued patients’ anecdotal reports, so if the size of the professional network were to increase, these patient reports would become more difficult to interpret. ‘When people came up with propositions to go elsewhere, I was unprepared. The way I was educated to deal with my profession as a GP - it was all about personal contact with specialists and providers, not about the arguments of performance information gathered elsewhere or waiting lists.’ (N 13) As a third motive in the interviews, the GPs expressed concerns regarding unintended consequences of using CPI during the referral process. This might limit the accessibility of certain types of care for those patients who could not easily choose to go elsewhere. Almost half the GPs surveyed (46%) agreed that referral based on CPI could increase fragmentation of care and threaten the continuity of care. ‘You can be treated in many hospitals for all kinds of things, but you need to have some sort of continuity in your treatment, which often means you end up in the same hospital.’ (N 15) Fourth, during the interviews several GPs said that it would take too much time to remain up to date about the CPI for multiple conditions and for a range of patient groups, even though only a minority (23%) of surveyed GPs agreed that ‘the use of CPI takes too much time’. Attitude: outcome expectancy The GPs had doubts about the outcome expectancy for CPI. In the interviews, some questioned whether using CPI would lead to an improvement in quality of care. The surveyed GPs were also divided in their opinions about the differences in quality of care in hospitals: 66% disagreed with the statement that the quality of care varies greatly between hospitals. 127 6 Chapter 6 Attitude: self-efficacy During the interviews, some GPs said they were unsure whether they could interpret CPI information and make a trade-off based on all the available information. The results in Table 3 show that it is difficult for GPs to interpret CPI information. ‘It is really difficult to assess the quality of care provided by my colleagues, and they are in the same building! Not to mention colleagues elsewhere or specialists in the hospital. It is almost impossible to make good judgements about that.’ (N 8) Lack of agreement with content and validity elements The GPs had their doubts regarding both the content itself and the validity of CPI (Table 3). A fairly large proportion of the GPs said ‘I do not know’ when they were asked about various CPI elements. In the interviews, some GPs mentioned conflicts of interest. They felt that the sources on which CPI is based should include disclaimers about various aspects of data collection and validity, and that the sources should declare any conflicts of interest. ‘I do not know about using CPI for referral decisions, but I get the feeling that I’m promoting a particular hospital.’ (N 9) Table 3. General practitioners’ level of agreement about statements concerning outcome expectancy, content, and validity of comparative performance information Statements Totally agree/ disagree Do not know Mean (SD) % CPI is not transparent about how information is determined 3.6 (0.7) 18 CPI is not clear because of contradictory sources 3.6 (0.7) 20 20 CPI is not credible 3.2 (0.7) 27 CPI is not in line with patients’ wishes 3.2 (0.6) 21 CPI is difficult for patients to understand 3.3 (0.7) 21 CPI is not specific enough 3.3 (0.7) 3.3 (0.7) 23 CPI is not user friendly CPI gives the wrong choice attributes 3.2 (0.6) 20 28 CPI is not up to date 3.1 (0.6) 3.0 (0.6) 20 CPI has no ability to show differences in quality of care The 68 responses were given on a five-point Likert scale, with ‘5’ representing ‘Totally agree’ and ‘1’ representing ‘Totally disagree’. ‘Do not know’, was a separate sixth answer possibility. SD: Standard deviation Behaviour Most of the GPs (94%) declared that they had never searched for hospital performance information in their region; however, 12% reported that they had used CPI for selecting 128 Comparative performance information plays no role in the referral behaviour of GPs a hospital. Further, approximately half the GPs agreed that they had discussed qualityof-care differences between hospitals with their patients (M=3.2; SD=0.8). The GPs were undecided regarding the expectation that CPI will becomes a part of their referral advice within 5 years – only a minority agreed with this statement (M=3.0; SD=0.8). Interviewer: ‘Do you believe that patients will make more informed choices in the future?’ GP: ‘Frankly? No, I do not think so.’ (N 4) Environmental factors A lack of reimbursement, time to search for CPI, the small number of hospitals to choose from, and not having an electronic referral system containing CPI were mentioned in the interviews. About half the surveyed GPs (48%) noted that they lacked an electronic system to help them use CPI in the referral process. DISCUSSION Can the GP be a choice-supportive advocate for patients to overcome patients’ lack of CPI usage? Our study shows that we cannot expect that a GP can play an advocate’s role in the use of CPI. Currently, GP considerations at the point of referral are patient preferences, quality of care, and travel distance, and there is no role for CPI as an additional source. The GPs feel that patients should become more aware of quality differences in general. They do not believe that current CPI has any added value for patients. The GPs rarely see patients initiating a discussion about CPI during consultations, and most are sceptical about the ability of patients to use CPI. The GPs’ own use of CPI is hindered by several barriers, including indecisiveness about their role in supporting patients’ choices and their task in addressing CPI during consultations. Comparison with existing literature The healthcare reforms in north-western European countries have been designed to encourage a greater role for patients in choosing a provider and to spur providers on to support this choice. The purpose of this design is to increase the competition between providers for the benefit of the patients.33,34 Our results show that current practice does not yet support the concept of GPs acting as agents of patient choice and users of CPI. The GPs in our study rarely had patients who mentioned CPI during a consultation. To decide on a referral, the GPs focus on patient preferences informal sources (e.g. connections with specialists), their own previous experiences, and hospital distance from the patient’s home. Our study confirms various findings from the UK, Denmark, and the Netherlands.24,35-37 The GPs feel responsible for coordinating care for their patients, but see no need for using CPI during the referral process. This is partly due to not knowing where to find 129 6 Chapter 6 CPI, but there is also some ignorance regarding the content and outcome expectancy for CPI. This ignorance may influence the ratings they gave. In another study, GPs did not view CPI as a source of information.37 A precondition for this kind of CPI usage is the reliability of the information and its sources. As in other studies, our GPs reported distrust of the content and validity of CPI.16,23,25 As long as GPs do not trust CPI, other sources of information will remain more important in their referral considerations. Because CPI makes clear statements on quality differences and providers of highquality care, it can be seen as a powerful source of information for GPs in selectively choosing a provider as well as in being the patient’s advocate while interpreting and discussing the available data. Regarding selective choice of a provider, a recent Dutch study showed that GP referral patterns were unaffected by report cards, with the exception of outcome indicators for breast cancer.38 Thus, even if CPI highlights differences in the quality of care, GP referral decisions are not, or are hardly, affected. Consequently, the intended impact of CPI in enabling a selective choice of a provider is not achieved. Regarding the GP’s role as an advocate, it seems that patients hardly ever introduced CPI. On the basis of our results, we can question whether the GP would use CPI if the patient suggested it. A lack of knowledge and a certain unwillingness both seem to contribute to the GP’s not using this kind of information during the referral process. A recent study37 suggests an interaction between the GP’s use of CPI and patients’ use of publicly available CPI in the decision-making discussion about referral with their GPs. Hence, if patients were to approach their GPs with publicly available information about quality more often, their GPs would be more likely to have consulted CPI themselves. However, because the patients hardly use CPI, and GPs do not either, the status quo continues. A UK study has shown that none of their participating GPs initiated a discussion of differences between services with patients.18 Approximately half the GPs in our study said that they discussed quality of care differences with their patients. Given their responses, we see that these discussions are not based on publicity available CPI. It may be that patients, despite their not using CPI, may become increasingly critical about differences in quality of care. Ensuring that care quality becomes an issue in the patient consultation can be considered a ‘tipping point’ in the path towards the use of CPI in the referral process. Despite the GPs’ restraint towards CPI, leaving the choice of provider in the hands of the patient alone worried some GPs as well. In relation to the coordinator role, the GPs in our study feared a further fragmentation of care, as patients might, as a result of CPI, choose providers outside the reach of their professional network. This reasoning has been described in another study as well39, and it makes sense because it is difficult 130 Comparative performance information plays no role in the referral behaviour of GPs to predict how and in which cases the benefits of using CPI and the choice of highquality providers outweigh the threats to continuity of care. New in our study is that GPs link this concern to their own professional role and to the potential weakening of their professional network – they like to keep an overview in their role as the coordinators in the Dutch ‘gatekeeper’ system. Implications Our study has various implications:  GPs should discuss whether and how to act as supportive agents for their patients using CPI in a way that does justice to their feelings of responsibility, concerns, and practical conditions  Education of GPs about CPI, its measures, the methodology on which information is based, and the possible better outcomes, as well as teaching them how to discuss CPI with patients  CPI should be publicized and made available to GPs so that they become aware of the information, can access it easily, and recognize practice variation between hospitals  Time is required to improve patient engagement in referral discussions (e.g. longer consultations). Strengths and limitations One of this study’s strengths is the use of both qualitative and quantitative data. The GPs interviewed came from innovative and frontrunner general practices. Even though this might have affected the interview results, the participants were drawn from a representative sample of Dutch general practices.40 We therefore used the survey results to draw a picture of how Dutch GPs use CPI, while the interview results were used mainly to illustrate the quantitative findings. A limitation was the number of CPIrelated questions that could be added to the annual LINH survey. Therefore, not every item highlighted in the interviews could be added to the survey in order to quantify our findings. We focussed on the barriers that the GPs encountered without explicitly discussing facilitating factors. The GPs noted their intention to act in the patient’s best interests in the referral considerations, the importance of the free choice of provider for patients, and the discussion of quality of care with patients, though none of these factors included facilitators for the use of CPI. Conclusions General practitioners play a key role in referring patients to hospital care. Their decisions about referrals to hospital care are not based on systematically collected CPI because other referral considerations are more important. CPI is assumed to be an important factor in selective-referral behaviour, as is supporting the patient’s ability to 131 6 Chapter 6 choose a provider of high-quality care by offering more transparency. Despite policy measures that encourage selectively choosing a provider and the expectations that both patients and GPs will make an active and informed choice based on the increasing availability of CPI, both are in a preliminary phase of using this data. Whether and how the GP’s roles in CPI use and patient support should be actively stimulated and supported is still to be determined. 132 Comparative performance information plays no role in the referral behaviour of GPs REFERENCES 1. 2. 3. 4. 5. 6. 7. 8. 9. 10. 11. 12. 13. 14. 15. 16. 17. 18. 19. 20. 21. 22. 23. Akbari A, Mayhew A, Al-Alawi MA, Grimshaw J, Winkens R, Glidewell E, et al. Interventions to improve outpatient referrals from primary care to secondary care. Cochrane Database Syst Rev 2008;(4):CD005471. Berendsen AJ, de Jong GM, Meyboom-de Jong B, Dekker JH, Schuling J. Transition of care: experiences and preferences of patients across the primary/secondary interface - a qualitative study. BMC Health Serv Res 2009;9:62. Dixon A, Robertson R, Bal R. The experience of implementing choice at point of referral: a comparison of the Netherlands and England. Health Econ Policy Law 2010;5(3):295–317. Barnett ML, Keating NL, Christakis NA, O’Malley AJ, Landon BE. Reasons for choice of referral physician among primary care and specialist physicians. J Gen Intern Med 2012;27(5):506–12. Forrest CB, Nutting PA, von Schrader S, Rohde C, Starfield B. Primary care physician specialty referral decision making: patient, physician, and health care system determinants. Med Decis Making 2006;26(1):76–85. Kinchen KS, Cooper LA, Levine D, Wang NY, Powe NR. Referral of patients to specialists: factors affecting choice of specialist by primary care physicians. Ann Fam Med 2004;2(3):245–52. Piterman L, Koritsas S. Part II. General practitioner-specialist referral process. Intern Med J 2005; 35(8):491–496. Fung CH, Lim YW, Mattke S, Damberg C, Shekelle PG. Systematic review: the evidence that publishing patient care performance data improves quality of care. Ann Intern Med 2008; 148(2):111–123. Wang J, Hockenberry J, Chou SY, Yang M. Do bad report cards have consequences? Impacts of publicly reported provider quality information on the CABG market in Pennsylvania. J Health Econ 2011;30(2):392–407. Marshall MN, Romano PS, Davies HT. How do we maximize the impact of the public reporting of quality of care? Int J Qual Health Care 2004;16(Suppl 1):i57–i63. Zwijnenberg NC, Damman OC, Spreeuwenberg P, Hendriks M, Rademakers JJ. Different patient subgroup, different ranking? Which quality indicators do patients find important when choosing a hospital for hip- or knee arthroplasty? BMC Health Serv Res 2011;11:299. de Groot IB, Otten W, Smeets HJ, Mharang-van de Mheen PJ, The Choice-2 study group. Is the impact of hospital performance data greater in patients who have compared hospitals? BMC Health Serv Res 2011;11:214. Rothberg MB, Benjamin EM, Lindenauer PK. Public reporting of hospital quality: recommendations to benefit patients and hospitals. J Hosp Med 2009;4(9):541–45. Ketelaar NABM, Faber MJ, Westert GP, Elwyn G, Braspenning JC. Exploring consumer values of comparative performance information for hospital choice. Quality in Primary Care 2014;22(2):81–9. Lichtenstein S, Slovic P: The Construction of Preference. 1st edition. Cambridge: Cambridge University Press, 2006. Fasolo B, Reutskaja E, Dixon A, Boyce T. Helping patients choose: how to improve the design of comparative scorecards of hospital quality. Patient Educ Couns 2010;78(3):344–49. Hibbard JH, Peters E. Supporting informed consumer health care decisions: data presentation approaches that facilitate the use of information in choice. Annu Rev Public Health 2003;24:413– 33. Schlesinger M. Choice cuts: parsing policymakers’ pursuit of patient empowerment from an individual perspective. Health Econ Policy Law 2010;5(3):365–87. Van der Geest SA, Varkevisser M. Zorgconsumenten en kwaliteitsinformatie [In Dutch]. Economisch Statistische Berichten (ESB) 2012;97(4631):174–75. Alexander JA, Hearld LR, Hasnain-Wynia R, Christianson JB, Martsolf GR. Consumer trust in sources of physician quality information. Med Care Res Rev 2011;68(4):421–40. Reitsma M, Brabers A, Masman W, de Jong J. The Choosing Citizen [In Dutch: De kiezende burger]. Nivel: Utrecht; 2012. Audet AM, Doty MM, Shamasdin J, Schoenbaum SC. Physicians’ Views on Quality of Care: Findings from the Commonwealth Fund National Survey of Physicians and Quality of Care. New York: The Commonwealth Fund, 2005. Barr JK, Bernard SL, Sofaer S, Giannotti TE, Lenfestey NF, Miranda DJ. Physicians’ views on public reporting of hospital quality data. Med Care Res Rev 2008;65(6):655–73. 133 6 Chapter 6 24. 25. 26. 27. 28. 29. 30. 31. 32. 33. 34. 35. 36. 37. 38. 39. 40. 134 Rozen R, Florin D, Hutt R. An Anatomy of GP Referral Decisions. A Qualitative Study of GP’s View on their Role in Supporting Patient Choice. London: The King’s Fund, 2007. Schneider EC, Epstein AM. Influence of cardiac-surgery performance reports on referral practices and access to care. A survey of cardiovascular specialists. N Engl J Med 1996;335(4):251–56. Bishop FL, Holmes MM. Mixed methods in CAM research: a systematic review of studies published in 2012. Evid Based Complement Alternat Med 2013;2013:187365. Creswell JW, Klassen AC, Plano Clark VL, Smith KC, Assistance WG. Best Practices for Mixed Methods Research in the Health Sciences. Washington, DC: Office of Behavioral and Social Sciences Research (OBSSR), National Institutes ofHealth (NIH), 2011. Fetters MD, Curry LA, Creswell JW. Achieving integration in mixed methods designs-principles and practices. Health Serv Res 2013;48(6 Pt 2):2134–56. Pope C, Ziebland S, Mays N. Qualitative research in health care. Analysing qualitative data. BMJ 2000; 320(7227):114–16. Cabana MD, Rand CS, Powe NR, Wu AW, Wilson MH, Abboud PA, et al. Why don’t physicians follow clinical practice guidelines? A framework for improvement. JAMA 1999; 282(15):1458–65. Sheon N. Overview of Atlas.ti.5.2. San Franciso: University of California, 2007. Dutch Information Network of General Practice: Dutch Information Network of General Practice (LINH). www.nivel.nl [20-7-2014]. Botje D, Klazinga NS, Wagner C. To what degree is the governance of Dutch hospitals orientated towards quality in care? Does this really affect performance? Health Policy 2013;113(1–2):134–41. Victoor A, Delnoij DM, Friele RD, Rademakers JJ. Determinants of patient choice of healthcare providers: a scoping review. BMC Health Serv Res 2012;12:272. Birk HO, Henriksen LO. Which factors decided general practitioners’ choice of hospital on behalf of their patients in an area with free choice of public hospital? A questionnaire study. BMC Health Serv Res 2012;12:126. Victoor A, Noordman J, Sonderkamp JA, Delnoij DM, Friele RD, van Dulmen S, et al. Are patients’ preferences regarding the place of treatment heard and addressed at the point of referral: an exploratory study based on observations of GP-patient consultations. BMC Fam Pract 2013;14:189. Doering N, Maarse H. The use of publicly available quality information when choosing a hospital or health-care provider: the role of the GP. Health Expect 2014; DOI:10.1111/hex.12187. Ikkersheim D, Koolman X. The use of quality information by general practitioners: does it alter choices? A randomized clustered study. BMC Fam Pract 2013;14:95. Coulter A. Do patients want a choice and does it work? BMJ 2010;341:c4989. Braspenning JC, Wichers L, Faber MJ. “The running of LINH practices is typical of Dutch practices in general”. [In Dutch: Praktijkvoering LINH-praktijken representatief]. Huisarts Wet [In Dutch] 2007;50(4):230-31. Chapter 7 General discussion Chapter 7 This thesis focuses on the conditions that affect healthcare consumers’ search for and use of comparative performance information (CPI). Healthcare consumers that are often imperfectly informed and make choices that are inconsistent with their preferences. CPI has been developed and introduced to help consumers choose a healthcare provider that matches their needs and preferences. Such information can be simultaneously used to drive healthcare providers to improve the quality of care. We address three themes in this thesis: search triggers, choice behaviour, and professional guidance. Figure 1. Three themes in this thesis search triggers choice behaviour professional guidance The research questions are:  What are the barriers and facilitators for healthcare consumers to search for and use CPI?  To what extent does publicly released performance information change the choice behaviour of healthcare consumers, providers, and purchasers?  To what extent do GPs advocate choice and encourage patients to choose providers on the basis of CPI? SEARCH TRIGGERS Main findings Consumers and their choice of healthcare providers (Chapter 2)  Most consumers are unfamiliar with CPI.  After CPI has been introduced, consumers indicate that they value other sources more than CPI, e.g. personal experience. 136 General discussion  Consumers doubt the reliability of CPI because the information from different sources is often inconsistent. Patients choosing a hospital for a primary hip or knee surgery (Chapter 3)  A minority of patients search for CPI.  Patients who expect differences in quality of care are significantly more likely to search for CPI than those who do not expect such differences.  Patients expect quality differences (mainly in general hospital characteristics such as reputation, distance, and accessibility) rather than treatment differences (in hip and knee surgery, for example). Referral of patients with Parkinson disease to a physiotherapist (Chapter 4)  Most participants with Parkinson disease take the referring provider’s advice, though this advice is rarely accompanied by supporting arguments or data.  The willingness to switch to an expert physiotherapist increases significantly when patients recognize the added value of the specific expertise.  About half of the patients consider their own physiotherapist to be an expert, which probably reduces their motivation to search for CPI. Only a minority of the healthcare consumers search for CPI about hospitals or healthcare providers.1-6 The literature reports some factors that do not stimulate a search for CPI; some of these factors can be addressed with an adequate intervention, while others are more or less fixed. Modifiable factors include the lack of awareness that CPI is available7-13 and the availability of an acceptable alternative provider in the proximity of one’s community. Fixed factors include the characteristic that women are more prone to search than men and the fact that a long-term health condition correlates with more consumer searching.6 The results of this thesis highlight three triggers that affect searching for CPI: the value of CPI versus the importance of other sources, consumers’ expectations of variation of care quality, and consumers’ recognition of expertise differences among providers. These three search triggers can be helpful in understanding why CPI is underused. Other studies that sought reasons for the lack of CPI usage focus on how CPI is presented or what the content of CPI should be from the consumer’s point of view. In this thesis, the focus is on an earlier phase, namely the search triggers. Knowing that consumers have an interest in choosing the best doctor makes one wonder why CPI is used so little. The value of CPI versus the importance of other sources As consumers hardly value CPI, they likely feel no need to search for it (Chapter 2). People are inclined to ignore less meaningful information.14 In choosing healthcare, 137 7 Chapter 7 consumers seem to value other sources more than CPI. Existing research already states that the impact of informal views of quality of care views of family members and friends as well as familiar authorities such as the GP, and local reputation counts more for most people than CPI does.15-21 The consumers use often their own personal experience to choose a healthcare provider, and also habit plays an obvious role in this.13,22-24 A new information source such as CPI should be perceived as a source with added value. New in our focus group study is the extent of and insight into many of the consumers’ underlying values and principles. Our findings suggest that the consumers were not really interested in CPI, and the currently formulated CPI was too abstract and not very relevant, so it seemed invaluable for them. It is debatable whether and how CPI can be become a source alongside other sources that is meaningful. Some studies illustrate the power of anecdotal narratives and suggest that the CPI should be given a more story-telling form25,26, in which case the information would reflect more of the information from the informal network. Then it would be richer in meaning and emotional nuance.27 There is a growing interest in testing how to collect and insert these narratives into healthcare providers’ CPI as well as how to display this material on websites.13 Consumers are apparently only minimally aware that CPI exists, and CPI as currently formulated seems to be irrelevant to them. Consumers’ expectations about differences in care quality Healthcare consumers search more for CPI when they expect to find variations in care quality (Chapter 3). Our study is one of the first that shows how patients’ expectations of quality of care affect the search for CPI. The finding is consistent with two other studies that suggest that a key component in public reporting is showing consumers that variation in quality is meaningful.28,29 So far, patients’ expectations of healthcare and variation in quality of care seem to be underexposed in public reporting. We found that the quality differences patients expected were oriented less toward the treatmentrelated performance indicators and more toward general issues such as reputation, distance, and accessibility. Thus, the trigger to search could become stronger if the patient were more aware of differences in treatment quality. The previous examination of consumers’ interest in different types of indicators produced mixed results. Some report that consumers are far more likely to focus on technical aspects of care than on interpersonal skills and services30, while others suggest the opposite pattern31, or no pattern at all; in short, all aspects seem equally important.7 Perhaps the clinical setting or the specific condition as suggested in a narrative review of the concept and measurement of patient expectations of quality32 could explain some of these differences. The authors also question the robustness of the measurement and plead for more research into the measurement itself. Although we do need more research into the measurement and the determinants of consumers expectations about 138 General discussion differences in quality of care, awareness of quality differences is a trigger for the search for quality information. Consumers’ recognition of expertise among providers An interesting finding of our study of patients with Parkinson’s disease is that the recognition of the value of an expert physiotherapist motivated some patients to search for CPI (Chapter 4). As described in Chapters 2 and 3, most patients did not use CPI to choose a healthcare provider, or in this study, specifically a physiotherapist. Chapter 3 reveals that expert awareness, just as knowledge, of quality differences motivates consumers to search for quality information. This awareness also seems to lead to actual behaviour. Although recognizing the value of an expert physiotherapist increased their willingness to switch physiotherapists, half the patients still wanted to continue the visits to their own current physiotherapist. Moreover, they characterized their own physiotherapist as an expert. The phenomenon is linked to the consideration presented elsewhere that your ‘own’ hospital or provider is the best.5,33 It is unlikely and unrealistic that patients think critically about possible mistakes in ‘their’ hospital.17 Even when they are confronted with quality differences in healthcare services, being aware of that motivated only half the patients with Parkinson disease to search for CPI. The studies of search triggers show that awareness of quality differences and availability of expertise can be a trigger for consumers to search for CPI, but when they were confronted with the currently available CPI, they were sceptical. The consumers see current CPI as rather abstract and irrelevant (low value), so they prefer more local patterns and use reputation to choose between healthcare services. To increase its value, CPI should be adjusted more towards the needs of the consumers. In exploring this direction, the following finding should be taken into account. The study in Chapter 2 reveals that becoming doubtful about the quality of certain aspects of care that consumers originally thought was good causes anxiety and distrust. This issue has been pointed out.34 It stirs up the discussion about trust versus control in healthcare. Transparency gives a certain kind of control, but it can also harm certain aspects of trust.35 This is not a plea to abolish public availability of quality information, but to balance the kind of quality information published publicly against the trust of healthcare consumers in the competencies of the healthcare provider. CHOICE BEHAVIOUR Main findings (Chapter 5)  The review of the literature shows almost no effect of CPI on behaviour neither via selection nor via improving the quality of care. The effects shown lasted no longer than two months. 139 7 Chapter 7  The level of evidence from the prospective studies was low: only four of 6839 studies were eligible for inclusion. CPI and behavioural changes in healthcare performance: the underlying mechanism Much has been invested in the development and provision of CPI since the early 1990s. However, a systematic review as presented in Chapter 5 about the effects of the public release of CPI on changing behaviour of healthcare consumers, providers, and purchasers was lacking. It is often assumed that the information will affect and facilitate decisions and behaviours of various parties and will ultimately improve the healthcare system. Berwick and colleagues describe two pathways of how public reporting can lead to performance change, improvement of quality of care.36 The first pathway, ‘selection’, represents the mechanism in which consumers can vote with their feet. Since consumers choose healthcare services on the basis of quality, healthcare providers are tempted to improve their performance to avoid losing customers. The second pathway, ‘change’, represents the mechanism of benchmarking and mutual comparison. Knowing that their group is lagging behind motivates healthcare providers to improve work procedures and professional culture. New material Since the review was published in 2011 and most studies were conducted in a rather dynamic era of early development, we wondered whether more recent studies were available able to produce additional insights. In the context of the general discussion of this thesis, we started a narrative search of the MEDLINE (Ovid) database (15 January 2015) with the same search strategy as used earlier in Chapter 5. Only one database was searched instead of five in the original review process. The search identified 996 relevant papers in the period of April 2011 to January 2015. Eleven studies qualified for a full text analysis, and five of them will most likely fit in an update of the Cochrane review. Thus, the narrative search shows that, despite a growing number of papers published in recent years, the number of new studies that bring us additional evidencebased information is still limited. The two main reasons for exclusion were once again a lack of description of the primary outcome measure and deficiencies in the study design. Four studies we found in this narrative search focused on only one of the pathways. Only one paper focussed on a combination of both pathways. The five studies included are listed in Table 1. Two papers studied the pathway of selection. One study of the impact of public reporting of surgery performance for coronary artery bypass shows a decrease in the mortality rate after the release of public performance data. The same study also states that public reporting may have caused that high-risk patients avoid high-mortality hospitals either by provider choice or selective referral behaviour (risk avoidance). As well that public reporting might cause an increase of patients who went more often to low-mortality hospitals.37 140 Table 1. Overview of studies with the effectiveness of releasing public performance information about behavioural changes Year of publication Design Author Romano et al. 2011 37 Martino et al. 2012 39 2013 38 Chen et al. Renzi et al. Ryan et al. 40 41 Improvement by Pathway I: Selection Pathway II: Improvement Interrupted time series Provider choice by (three before-and-after patients/selective data points) referral behaviour by primary care physicians or cardiologists An experimental design: Choice of new healthrandomized plan members for encouragement design primary care physicians based on performance data Use and analysis of Better performance of national cardiac arrest hospitals registry 2014 Pre- and post-evaluation and comparative evaluation 2012 Interrupted time series using three models to estimate the reduction of mortality: 1) beforeand-after Hospital Compare, 2) publicly reported diagnoses, 3) trends in non-reported diagnoses Impact of public reporting on quality indicators for acute myocardial infarction Patients changing from Hospital’s lower- to higher-quality improvements of the hospitals publicly reported measures (patient mortality) Outcomes The release of public performance reports was associated with increased patient volume at low mortality in the first 6 months No evidence of effectiveness of physician performance report on consumer choice Hospitals performed better with regard to publicly reported outcomes for common medical conditions, though the outcomes for survival rate of in-hospital cardiac arrest patients did not improve Public reporting has is associated with an increasing proportion of patients treated in time with percutaneous coronary interventions The 30-day mortality rate after acute myocardial infarction did not improve Improvements accelerated immediately after reporting the results, though factors other than public reporting can plausibly explain the improvements Patients and physicians do not change to highquality hospitals. Mortality rates of heart attacks and pneumonia do not change 7 Chapter 7 The other study of cardiac arrest shows no effect of physician-quality data on consumers’ behaviour in choosing their primary-care physicians.38 Two studies present findings about the second pathway of change.39,40 One study shows the impact of public reporting on hospital care, though hospitals that performed better with regard to publicly reported outcomes for common medical conditions did not necessarily have better survival rates for cardiac arrest.39 Another study looks at the impact of public reporting on an increased proportion of patients with myocardial infarction who received timely treatment with a percutaneous coronary intervention. The number of patients receiving this treatment increased significantly in the regions with public reporting compared to other regions without public reporting.40 One study examines the effect of the public reporting of both pathways either on the hospital’s improving the publicly reported measures (patient mortality), or on patients changing from lower-quality to higher-quality hospitals. They show no evidence that patients or physicians changed to high-quality hospitals nor showed they any evidence that public reporting reduced mortality due to heart attack or pneumonia.41 Overall, the findings hint at an effect on behaviour change, although we should probably conclude that more sound studies are needed in this area. Many studies in this field have been conducted in the last few years, though only a few of them appear to meet the strict criteria of the Cochrane Reviews. For the purpose of this general discussion, we have been quite compliant about the inclusion of studies that we found in the narrative search, but we expect that there might be one or two studies that will not be included when the review is updated. The authors of the six recent studies emphasize the facts that the evidence for the effectiveness of public reporting is limited and the outcomes should be interpreted with care. Several authors of these studies point out the difficulties of well-designed studies on public reporting, and their attempts to design the studies as well as possible. All in all, this update and scan of the literature is in line with the results of our review published in 2011. We conclude that so far no firm conclusions can be drawn about the effect of CPI. PROFESSIONAL GUIDANCE Main findings GPs experience of choice-supporting behaviour in the referral process (Chapter 6). Most GPs do not know where to find CPI and have never used it before. The decisions GPs make in the current process for referring patients to hospital care are not, or hardly, based on CPI exchanges during consultations. Other referral considerations and sources (e.g. personal contact with specialists, their own and patients’ experiences with providers, and other patient experiences) are more important. 142 General discussion Most GPs are sceptical about the ability of patients to use CPI. Several barriers also hinder their own use of CPI. These barriers include indecisiveness about their role in supporting patient choices, doubts about responsibility, feelings of uncertainty about the interpretation of the information, doubts about the content of the information and its reliability, and fear of weakening existing networks. Our findings make two things clear. First, the pathway of selection that Berwick and colleagues advocate as the mechanism to improve quality of care via public reporting36 does not seem to work effectively for either healthcare consumers or referring providers. Like consumers, GPs find other sources, such as patient preferences, personal contacts, pervious experiences with a hospital, and mouth-to-mouth information, all of which are more important to them than CPI.42-47 Second, there is a gap between assumptions and practice regarding professional guidance for consumers. Ideally, GPs discuss the information and elucidate patient values before referring a patient to a hospital.5 However, the results do not confirm the assumption that even though patients hardly use CPI, they will benefit from performance information via their GPs, who will bring the options for hospital choice to their attention. Other studies find GPs similarly reluctant to make patients aware of performance information.11,44 In general, the discussion of CPI between patients and referring healthcare providers – researched via GPs and cardiologists – is rare.11,34,48,49 Given the increased emphasis on a healthcare system that encourages active patient involvement in their care process, it is remarkable that even the discussion of these quality data with patients is so uncommon. Discussion of GPs’ limited guidance Consumer engagement does not come easily. There are several explanations for the GPs limited engagement in helping patients make choices and for their lack of selective referral behaviour. One explanation is the lack of patient pressure on the GP.44 We found that GPs rarely have patients initiating discussions about CPI. Others suggest that if more patients approached their GPs with questions about this kind of information, GPs might pay more attention to CPI themselves.44,50 The second explanation for the limited impact on referral patterns is the lack of awareness of CPI45,51, though recent findings from the USA43 show that even if almost every provider is aware of CPI, use does not necessarily increase. For example, wider exposure did not convince referring cardiologists to trust these data and use the information for referrals. The third explanation is that GPs are reluctant to share information with which they are not familiar themselves.12 This clashes with the claim that GPs should be capable of understanding and interpreting data such as CPI, given their training and experience.46 This thesis reveals that GPs 143 7 Chapter 7 criticize the content of the current information for both their patients and themselves. A recent study of Dutch GPs’ referral behaviour finds that referral patterns overall remain unaffected by the available CPI about medical effectiveness and patient experiences for the conditions of cataract surgery, breast cancer, and hip and knee replacement. Nonetheless, referral patterns for the condition of breast cancer were affected because GPs make more referrals to hospitals with a higher rating for medical effectiveness. In line with qualitative statements, CPI about breast cancer appeared to contain the most valid, reliable, and differentiating data.46 This calls for CPI based on outcome indicators that shows relevant differences. Furthermore, referring providers generally consider such data untrustworthy and not up to date; they judge the sources as unreliable or the data as inaccurately assessing the quality of providers.43,44,52 The findings in this thesis stress the importance of reliable performance information for GPs. A final explanation for the limited guidance is that the findings reveal the GPs’ indecisiveness about their role in assisting patients in making choices. Further, these explanations lead to the question of whether there are any people or professionals other than the GPs to guide patients in using CPI. Patients might benefit from having an advocate who makes sense of patients’ circumstances, helps them use CPI to make their choices, and advises them about their future choices.34,53 The benefits of support have been demonstrated for other choices, such as healthcare insurance plans and treatment.54-56 Helping consumers understand insurance coverage conditions and supporting appeals for coverage denials are called ‘consumer assistance programmes’. In the USA, the assistance initiative has altered the way consumers engage with healthcare insurers. It enhances their capacity for selfadvocacy and involvement, though many disempowered consumers still fail to seek assistance.54 The effectiveness of decision coaching for patients who face a decision between at least two treatment options has been reviewed. The findings, based on several studies, show that decision coaching improves patient’s knowledge; such knowledge is an important element in decision-making. However, decision coaching in clinical care is still a novelty.55,56 As in other settings, the use of a ‘coach’, a ‘consumer assistant’, or an ‘advocate’ in helping consumers make decisions based on CPI remains open to debate. METHODOLOGICAL CONSIDERATIONS One strength of this thesis is the variety of research methods that we have used to examine the research questions, namely, a systematic literature review, a qualitative focus group, quantitative cross-sectional questionnaires, and a mixed-method study with interviews and a questionnaire. Here we present our general reflections on the methodological limitations and strengths of our work in the context of our research methods. 144 General discussion The first limitation is the number of healthcare consumers who participated in the different studies. The small sample sizes threaten the validity of the outcomes. The recruitment problem was not limited to healthcare consumers (Chapter 2); it was also difficult to attract GPs (Chapter 6). From this we conclude that CPI is indeed not high on the agenda of either healthcare consumers or providers. The second limitation concerns the relatively high age (> 65 years) of the participating consumers. The rationale for focusing on rather older participants was that these people were more likely to make actual choices in healthcare (hip and knee replacements for those confronted with Parkinson’s disease), and we wanted to find out why this group of consumers had trouble seeking and using CPI. Still, to use the words of Meinow and colleagues, ‘results suggest that those elderly people who are most dependent on care and who could benefit most from a “good choice” are also the ones who have the highest prevalence rates of cognitive, physical, and sensory limitations’ p.1289. The fact that this study used real and openly available CPI can be seen as a third limitation. It limited us in terms of content, volume, distinctiveness, and reliability of CPI. Though some of the attributes were not yet public, we added them because we knew they would soon become available and we wanted to know whether people would search for this information. At the same time, this limitation is a strength of the study: the information was closely connected to the current practice of patient choice and the use of CPI. The fourth and last limitation of this study is the strict inclusion criteria we used in conducting our systematic review (Chapter 5). We prefer to use these criteria to measure the impact on real behaviour rather than hypothetical behaviour, and to make a judgement based on the highest level of evidence. However, we are aware of the large number of studies that are being published and the limited number of studies that were suitable for inclusion in the systematic review and in the narrative search. These elements force us to rethink our strict design criteria and consider other designs or combinations of design for this type of research. The design must do justice to the complex reality as well. IMPLICATIONS Taken together, this study and past work suggest that the provision of CPI to engage patients and consumers in healthcare and to direct provider choice while simultaneously driving the improvement of quality of healthcare has not shown to be effective thus far. Nevertheless, the area of public reporting and the use of CPI do not seem to be at a dead end. On the contrary, researchers, policy makers, insurers, and 145 7 Chapter 7 consumers maintain a continuous desire to keep moving in the direction of CPI and help consumers improve the quality of the healthcare system by their choices of high-quality providers despite the disappointing results.58 By far the most important reason why CPI remains attractive is that we will not go back to the time when patients and consumers had minor roles in the healthcare system. The source of CPI lies in accountability, the transparency of quality performance of healthcare services, which has been positioned as the consumer’s fundamental right to know one’s care, care options, and quality of care.59 Even though three decades have passed since the beginning of public reporting, we are just getting started. At the same time, there are signs that freedom of choice for consumers might be under pressure and may become limited as the geographical concentration of care increases. We see that the healthcare insurers are already offering contracts that limit the number of available healthcare providers on beforehand. Based on findings in this thesis, we formulate several implications for those undertaking future research, policy-makers, and consumer organizations. Finding-based implications for future research This thesis reveals that consumer expectation and recognition of quality differences between hospitals and providers play an important role in the search for performance information. More research is needed to get a better picture of how robust or fragile healthcare consumer expectation and recognition of quality differences are, as well as to determine what causes or influences expectation and recognition. Moreover, our findings imply that performance information fails in the expectation of triggering consumer interest. Studies that focus on how to strengthen the demand for CPI as well as on finding an effective way to increase healthcare consumers’ knowledge about the underlying concepts of quality of care and the purpose of performance information are needed. Our findings further reveal that performance information remains difficult to understand. It may be that data currently presented in CPI fail to resonate with the values and feelings that matter to consumers. It has been recently suggested that anecdotal information from consumers13,60 or online reviews on social media sites and commercial rating sites may be useful.61 These types of information may be better understood than the current CPI, so they would be more valuable to consumers and more likely to gain interest and satisfy a need. Giving consumers the ability to share their experiences and views directly other than in patient experience surveys can be an important aim in itself. Future research that focuses on how consumers can effectively balance and integrate these personal commentaries would therefore be a necessary addition to the literature of public reporting.13 146 General discussion This study shows that most consumers still have difficulties using performance information to choose a provider. Help and support from a professional nearby, such as the GP, seem lacking. The GPs reported various barriers to their use of CPI in the process of referral and to their taking a choice-supporting role. Future research into the effectiveness of consumer support by a trusted agent is needed. Furthermore studies that focus on the interaction between consumers and GPs or another trusted agent while using CPI would fill a gap in the literature. Research into the public reporting published over the last three decades is mainly about the impact on improving the quality of care or selection of good practices in care rather than about comparing and evaluating specific report designs to sort out the effective and ineffective ones. A classical RCT design is not useful for capturing the impact of public reporting. Given the public character of performance information, it is complex and too difficult to determine whether control groups get access to CPI or have ideas about it. Because of this, interrupted time series design and controlled before-and-after studies are more suitable, though a lot of studies with either of these designs do not satisfy the requirement of the multiple time points. At least three time points are required. One task for future research is to gather more evidence about which designs, or combinations of designs, can provide reliable information about the impact of CPI on choice behaviour. Finding-based implications for policy-makers and consumer organizations This thesis contributes to the developing literature about the complex reasons why most consumers are not prone or not eager to use CPI. The current results show that the lack of demand for public reporting depends on the availability of other important and more valued sources, lack of routine in using the information, outcome expectancy of using CPI, and barriers to using the information (e.g. too much responsibility, choosing is a burden, and questioning trustworthiness). The outcome expectancy may be the biggest reason for some healthcare consumers’ lack of perceived need, whereas the personal restraints of others may be their most prominent reason. Knowledge and recognition of quality differences can probably motivate some consumers to search for CPI. The consumers perceived professional guidance as helpful, but not necessarily based on CPI. The GPs in our study questioned the current CPI, and they preferred other sources to help the consumer select a healthcare provider. A series of initiatives should counteract the concerns about the lack of information, insight, and understanding for good quality of care. Starting to increase the urgency for consumers and GPs, CPI need to address variation in quality of care, and visualize clearly these differences. An issue that concerns not only for consumers and GPs, but for all stakeholders is the design of publicly reported data. Much more effort is needed to get more sophisticated information products. 147 7 Chapter 7 Further, providing more time in the referral process for both GPs and consumers would be very helpful. More time might make informed choice possible for consumers and improve the ability of GPs to help them. Currently, consumers receive referrals from their GPs, and within minutes the choice for a hospital has been made. The moment of actual choice has passed before the consumer has had a chance to catch up with it, let alone actually use performance data or discuss their informed choice with their GP. We invite policy-makers, GPs, and insurance companies to find the financial support for extending the 10-minute standard GP consultation to allow a valuable discussion of choice. FINAL REMARK CPI has been developed to help healthcare consumers choose a healthcare provider that matches their needs and preferences and to drive healthcare providers to improve the quality of their care. However, the evidence that CPI has an actual impact on behaviour is still scant. Consumers say that the current CPI is novel, not well designed, too abstract, too vague, meaningless, and too difficult to use. From this perspective, the consumers’ non-use of the information might be reasonable, as might be their sticking to their current selection routines. Help and support from a trusted agent can overcome some of these issues. However, the GPs pointed out that the current state of the CPI, fear of discontinuity and fragmentation in healthcare are important things that stops them from using it. The promise of CPI remains attractive because of the underlying principles of more transparency and increasing consumer involvement in healthcare. The current state of affairs can be seen as an ongoing train, but the current design of CPI seems to be more appropriate for policy reasons than for the individual selection process of a healthcare provider. CPI seems inadequate for consumers. Perhaps this is partly attributable to the fact that consumers hardly play a role in the development of CPI. The production and dissemination of CPI is also meant to encourage a more central position for consumers in healthcare. Of course consumers do not suddenly achieve that position in healthcare without effort from anyone. We call for policy-makers, CPI developers, and consumer organizations to address the need for changing the format to help develop more consumer-friendly and -oriented information. The value of such data might increase once consumers become more involved for a longer time in developing performance information. 148 General discussion REFERENCES 1. 2. 3. 4. 5. 6. 7. 8. 9. 10. 11. 12. 13. 14. 15. 16. 17. 18. 19. 20. 21. 22. 23. de Groot IB, Otten W, Smeets HJ, Marang-van de Mheen PJ, the CHOICE-2 study group. Is the impact of hospital performance data greater in patients who have compared hospitals? BMC Health Serv Res 2011;11:214. Harris KM. How do patients choose physicians? Evidence from a national survey of enrollees in employment-related health plans. Health Serv Res 2003;38(2): 711-32. Rademakers J, Nijman J, Brabers AE, de Jong JD, Hendriks M. The relative effect of health literacy and patient activation on provider choice in the Netherlands. Health Policy 2014;114(2-3): 200-6. Robertson R, Dixon A. Choice at the point of referral. London: The King’s Fund, 2009. Schwartz LM, Woloshin S, Birkmeyer JD. How do elderly patients decide where to go for major surgery? Telephone interview survey. BMJ 2005;331(7520):821. Victoor A, Rademakers J, Reitsma-van Rooijen M, de Jong J, Delnoij D, Friele R. The effect of the proximity of patients’ nearest alternative hospital on their intention to search for information on hospital quality. J Health Serv Res Policy 2014;19(1):4-11. Zwijnenberg NC, Damman OC, Spreeuwenberg P, Hendriks M, Rademakers JJ. Different patient subgroup, different ranking? Which quality indicators do patients find important when choosing a hospital for hip- or knee arthroplasty? BMC Health Serv Res 2011;11:299. Faber M, Bosch M, Wollersheim H, Leatherman S, Grol R. Public reporting in health care: how do consumers use quality-of-care information? A systematic review. Med Care 2009;47(1):1-8. Fotaki M, Roland M, Boyd A, McDonald R, Scheaff R, Smith L. What benefits will choice bring to patients? Literature review and assessment of implications. J Health Serv Res Policy 2008;13(3): 178-84. Damman OC, Hendriks M, Rademakers J, Delnoij DM, Groenewegen PP. How do healthcare consumers process and evaluate comparative healthcare information? A qualitative study using cognitive interviews. BMC Public Health 2009;9:423. Coulter A. Do patients want a choice and does it work? BMJ 2010;341:c4989. Dixon A, Robertson R, Appleby J, Burge P, Devlin N, Magee H. Patient choice. How patients choose and providers respond. London: The King’s Fund, 2010. Greaves F, Millett C, Nuki P. England’s Experience incorporating “anecdotal” reports from consumers into their national reporting system: lessons for the United States of what to do or not to do? Med Care Res Rev 2014;71(5 Suppl):65S-80S. Vaiana ME, McGlynn EA. What cognitive science tells us about the design of reports for consumers. Med Care Res Rev 2002;59(1):3-35. Harris KM, Melinda Beeuwkes Buntin M, The RAND Corporation. Choosing a health care provider: The role of quality information. Princeton: The Robert Wood Johnson Foundation, 2008. Alexander JA, Hearld LR, Hasnain-Wynia R, Christianson JB, Martsolf GR. Consumer trust in sources of physician quality information. Med Care Res Rev 2011;68(4):421-40. Victoor A, Delnoij D, Friele R, Rademakers J. Why patients may not exercise their choice when referred for hospital care. An exploratory study based on interviews with patients. Health Expect 2014. Dijs-Elsinga J, Otten W, Versluijs MM, Smeets HJ, Kievit J, Vree R, et al. Choosing a hospital for surgery: the importance of information on quality of care. Med Decis Making 2010;30(5):544-55. Fung CH, Lim YW, Mattke S, Damberg C, Shekelle PG. Systematic review: the evidence that publishing patient care performance data improves quality of care. Ann Intern Med 2008;148(2): 111-23. Mittler JN, Martsolf GR, Telenko SJ, Scanlon DP. Making sense of “consumer engagement” initiatives to improve health and health care: a conceptual framework to guide policy and practice. Milbank Q 2013;91(1):37-77. Moser A, Korstjens I, van der Weijden T, Tange H. Patient’s decision making in selecting a hospital for elective orthopaedic surgery. J Eval Clin Pract 2010;16(6):1262-68. Vonberg RP, Sander C, Gastmeier P. Consumer attitudes about health care acquired infections: a German survey on factors considered important in the choice of a hospital. Am J Med Qual 2008; 23(1):56-59. Trigg L. Patients’ opinions of health care providers for supporting choice and quality improvement. J Health Serv Res Policy 2011;16(2):102-7. 149 7 Chapter 7 24. 25. 26. 27. 28. 29. 30. 31. 32. 33. 34. 35. 36. 37. 38. 39. 40. 41. 42. 43. 44. 45. 150 Fotaki M, Boyd A, Smith L, McDonald R, Roland M, Sheaff R, et al. Patient Choice and the Organisation and Delivery of Health Services: Scoping review In, Manchester: Report for the National Co-ordinating Centre for NHS Service Delivery and Organisation R&D (NCCSDO) 2005. Marshall MN, Romano PS, Davies HT. How do we maximize the impact of the public reporting of quality of care? Int J Qual Health Care 2004;16(Suppl 1):i57-63. Huppertz JW, Carlson JP. Consumers' use of HCAHPS ratings and word-of-mouth in hospital choice. Health Serv Res 2010;45(6 Pt 1):1602-13. Lagu T, Lindenauer PK. Putting the public back in public reporting of health care quality. JAMA 2010;304(15):1711-12. Stein BD, Kogan JN, Essock S, Fudurich S. Views of mental health care consumers on public reporting of information on provider performance. Psychiatr Serv 2009;60(5):689-92. de Groot IB, Otten W, Dijs-Elsinga J, Smeets HJ, Kievit J, Marang-van de Mheen PJ, et al. Choosing between hospitals: the influence of the experiences of other patients. Med Decis Making 2012;32(6):764-78. Fung CH, Elliott MN, Hays RD, Kahn KL, Kanouse DE, McGlynn EA, et al. Patients’ preferences for technical versus interpersonal quality when selecting a primary care physician. Health Serv Res 2005;40(4):957-77. Werner RM, Norton EC, Konetzka RT, Polsky D. Do consumers respond to publicly reported quality information? Evidence from nursing homes. J Health Econ 2012;31(1):50-61. Bowling A, Rowe G, Lambert N, Waddington M, Mahtani KR, Kenten C, et al. The measurement of patients’ expectations for health care: a review and psychometric testing of a measure of patients’ expectations. Health Technol Assess 2012;16(30): i-xii, 1-509. Joseph-Williams N, Elwyn G, Edwards A. Knowledge is not power for patients: a systematic review and thematic synthesis of patient-reported barriers and facilitators to shared decision making. Patient Educ Couns 2014;94(3):291-309. Schlesinger M. Choice cuts: parsing policymakers’ pursuit of patient empowerment from an individual perspective. Health Econ Policy Law 2010;5(3): 365-87. Fotaki M. Can consumer choice replace trust in the National Health Service in England? Towards developing an affective psychosocial conception of trust in health care. Sociol Health Illn 2014; 36(8): 1276-94. Berwick DM, James B, Coye MJ. Connections between quality measurement and improvement. Med Care 2003;41(1 Suppl):I30-8. Romano PS, Marcin JP, Dai JJ, Yang XD, Kravitz RL, Rocke DM, et al. Impact of public reporting of coronary artery bypass graft surgery performance data on market share, mortality, and patient selection. Med Care 2011;49(12):1118-25. Martino SC, Kanouse DE, Elliott MN, Teleki SS, Hays RD. A field experiment on the impact of physician-level performance data on consumers' choice of physician. Med Care 2012;50(Suppl):S6573. Chen LM, Nallamothu BK, Krumholz HM, Spertus JA, Tang F, Chan PS, et al. Association between a hospital’s quality performance for in-hospital cardiac arrest and common medical conditions. Circ Cardiovasc Qual Outcomes 2013;6(6):700-7. Renzi C, Asta F, Fusco D, Agabiti N, Davoli M, Perucci CA. Does public reporting improve the quality of hospital care for acute myocardial infarction? Results from a regional outcome evaluation program in Italy. Int J Qual Health Care 2014;26(3):223-30. Ryan AM, Nallamothu BK, Dimick JB. Medicare'’ public reporting initiative on hospital quality had modest or no impact on mortality from three key conditions. Health Aff (Millwood) 2012;31(3): 585-92. Birk HO, Henriksen LO. Which factors decided general practitioners’ choice of hospital on behalf of their patients in an area with free choice of public hospital? A questionnaire study. BMC Health Serv Res 2012;12:126. Brown DL, Epstein AM, Schneider EC. Influence of cardiac surgeon report cards on patient referral by cardiologists in New York state after 20 years of public reporting. Circ Cardiovasc Qual Outcomes 2013;6(6):643-48. Doering N, Maarse H. The use of publicly available quality information when choosing a hospital or health-care provider: the role of the GP. Health Expect 2014. Epstein AJ. Effects of report cards on referral patterns to cardiac surgeons. J Health Econ 2010; 29(5):718-31. General discussion 46. 47. 48. 49. 50. 51. 52. 53. 54. 55. 56. 57. 58. 59. 60. 61. Ikkersheim D, Koolman X. The use of quality information by general practitioners: does it alter choices? A randomized clustered study. BMC Fam Pract 2013;1495. Audet AM, Doty MM, Shamasdin J, Schoenbaum SC. Physicians’ views on quality of care: Findings from the Commonwealth Fund National Survey of Physicians and Quality of Care. New York The Commonwealth Fund, 2005. Hannan EL, Stone CC, Biddle TL, DeBuono BA. Public release of cardiac surgery outcomes data in New York: what do New York state cardiologists think of it? Am Heart J 1997;134(6):1120-28. Schneider EC, Epstein AM. Influence of cardiac-surgery performance reports on referral practices and access to care. A survey of cardiovascular specialists. N Engl J Med 1996;335(4):251-56. Barr JK, Bernard SL, Sofaer S, Giannotti TE, Lenfestey NF, Miranda DJ. Physicians’ views on public reporting of hospital quality data. Med Care Res Rev 2008;65(6):655-73. Morsi E, Lindenauer PK, Rothberg MB. Primary care physicians’ use of publicly reported quality data in hospital referral decisions. J Hosp Med 2012;7(5):370-75. Rozen R, Florin D, Hutt R. An anatomy of GP referral decisions. A qualitative study of GP's view on their role in supporting patient choice. London: The King's Fund, 2007. Shaller D, Kanouse DE, Schlesinger M. Context-based strategies for engaging consumers with public reports about health care providers. Med Care Res Rev 2014;71(5 Suppl):17S-37S. Grob R, Schlesinger M, Davis S, Cohen D, Lapps J. The Affordable Care Act's plan for consumer assistance with insurance moves states forward but remains a work in progress. Health Aff (Millwood) 2013;32(2):347-56. Stacey D, Kryworuchko J, Belkora J, Davison BJ, Durand MA, Eden KB, et al. Coaching and guidance with patient decision aids: A review of theoretical and empirical evidence. BMC Med Inform Decis Making 2013;13(Suppl 2):S11. Stacey D, Kryworuchko J, Bennett C, Murray MA, Mullan S, Legare F. Decision coaching to prepare patients for making health decisions: a systematic review of decision coaching in trials of patient decision AIDS. Med Decis Making 2012;32(3): E22-33. Meinow B, Parker MG, Thorslund M. Consumers of eldercare in Sweden: the semblance of choice. Soc Sci Med 2011; 73(9): 1285-9. Damberg CL, McNamara P. Postscript: research agenda to guide the next generation of public reports for consumers. Med Care Res Rev 2014;71(5 Suppl): 97S-107S. Hussey PS, Luft HS, McNamara P. Public reporting of provider performance at a crossroads in the United States: summary of current barriers and recommendations on how to move forward. Med Care Res Rev 2014;71(5 Suppl): 5S-16S. Egmond van S, Heerings M. In Dutch: Sterke verhalen uit het ziekenhuis. Leren van patientervaringen voor goede zorg. In, Den Haag.: Rathenau Instituut, 2014. Lagu T, Greaves F. From Public to Social Reporting of Hospital Quality. J Gen Intern Med 2015. 7 151 Summary Summary Chapter 1 presents an overview of this thesis, which describes the barriers and facilitating factors of search behaviour for comparative performance information (CPI) and healthcare consumers’ selective behaviour in choosing healthcare providers. Providing CPI for healthcare services and healthcare providers is one way of giving healthcare consumers insight into the quality of care. Then they can use this information to make an informed choice of healthcare provider. The underlying idea is that consumers who have good and reliable performance information will choose the best healthcare provider, which will pressure healthcare providers to improve the quality of care. Whether consumers are rational decision-makers who base their choices of healthcare providers on cognitive considerations is the question. However, it is much more important to provide consumers with good information so that they have more control, influence, and power to make their own decisions. Although consumers strive for more involvement and wish to have such information, they hardly use it. This thesis explores three research themes: search triggers that affect the search behaviour, choice behaviour, and professional guidance to support choice behaviour. First, we explore why healthcare consumers do not yet use this type of information, why they neglect it, misunderstand it, or have trouble searching for it. Second, we examine the effect of CPI on the behaviour of various parties. Third, we examine the effect of performance information on the behaviour of these parties. Fourth, we focus on the GPs and their use of CPI in the current referral process, as well as their role in helping patients make an informed choice. Chapter 2 presents the findings of a qualitative focus group study among healthcare consumers. Our aim was to get insight into consumers’ attitudes, opinions, value received, and appreciation concerning CPI. Six focus groups were conducted with 27 healthcare consumers. The results showed that most consumers were unfamiliar with the information and had not used it before. We used clear examples of CPI to help consumers understand it. The consumers very ably put into words why they did not search for this information. They valued other sources, including personal experience, more than CPI. Information from different sources was often inconsistent, which cast doubt on its reliability. This study led to new and extended insight into consumers’ distrust. There was a fear of disrupting existing relationships, loss of faith in certain aspects of care that consumers had thought were good, and the pressure of insecurity due to new responsibilities. This study also reveals that consumers noted positive effects of CPI such as: more freedom of choice, more involvement for healthcare consumers, the potential effect of benchmarking, and a new-found awareness of quality of care in their own perception. A possible negative side-effect of CPI is that it might cause a dichotomy in society between those who search for CPI and use it and those who do not. Nonetheless, we conclude that the consumers valued the CPI only 154 Summary marginally. Different values and principles limit the search for CPI, which often precludes its use. Chapter 3 examines the patients’ expectations concerning differences in care quality and their search for CPI. Questionnaires were distributed among 475 patients who underwent primary hip or knee surgery in a university, teaching, or community hospital. Of the 302 patients who responded (response rate 63%), only a small minority (13%) searched for CPI after they had found out they needed surgery. Most of those who did not search for CPI felt no need of more information. Most patients (67%) expected to find differences in the quality of care. These were the ones who were significantly more likely to search for CPI (OR=3.18 [95% CI: 1.02–9.89]; p<0.04) than those who did not expect such differences. The search trigger was even greater for patients expecting large differences (OR=5.05 [95% CI: 1.44-17.77]; p<0.01). The patients expected differences especially in general qualities such as reputation, distance, and accessibility. They expected more differences in general performance than in specific factors related to the patient’s disorder. Chapter 4 reports the findings of a quantitative study about the ability of people with Parkinson’s disease to recognize expertise, and it considers to what extent their recognition and evaluation of this expertise triggers their search and selective choice of experts. This survey study in a chronic-care setting focuses on physiotherapists’ expertise. Five hundred eligible respondents were invited to participate. This sample was based on claim data of the Dutch insurance company CZ. We analysed 320 surveys (response rate 64%). Healthcare providers’ expertise is based on a quality accreditation for special training to work according to evidence-based recommendations as well as structured referrals to increase the volume of patients with Parkinson’s disease whom they treat. Most participants (89%) with Parkinson’s disease took the referring provider’s advice, though this advice was rarely accompanied by supporting arguments. Not many looked for additional information (11%), but the participants did recognize some qualities of expertise. The respondents who did recognize the added value of specific expertise were significantly more likely to search (3.28 times more often) for performance information than those who did not. The willingness to switch to an expert physiotherapist increased significantly (p<0.01) once patients recognized the added value of specific expertise. About half the people with Parkinson’s disease did not recognize the added value, and they overestimated the expertise of their own healthcare providers. Based on the claim data, approximately 28% of the respondents received treatment from a physiotherapist with specific expertise in Parkinson’s disease, while 70% of the respondents claimed they received treatment from specialized physiotherapists. Considering their own physiotherapists (without special 155 Summary training) to be experts in Parkinson’s disease might reduce their motivation to search for CPI or to search for another physiotherapist. Chapter 5 presents a systematic review that estimates the effects of the public release of CPI about changing the behaviour of healthcare consumers, providers, and purchasers. Much has been invested in the development and provision of performance information since the early 1990s in the USA. However, England and, in more recent years, the Netherlands, along with many other countries, have produced no systematic reviews about the effectiveness of such information. Because of the lack of evidence, we analysed studies of the effects of public performance information on changing behaviour. It is often assumed that the information will affect and facilitate decisions and behaviours of various parties and will ultimately improve the healthcare system. Various mechanisms in the public release of performance information may increase the quality of care. Berwick and colleagues describe mechanisms that work via two pathways: 1) the behaviour in choosing the best healthcare provider or organization and 2) improving performance by changing work procedures or professional culture. We used these two pathways as our primary outcome measures. We found 6839 relevant studies by using an extended search strategy to search five databases for studies published up to early 2011. Ultimately, after a strict selection procedure, as applied to the Cochrane review (Chapter 5), we included four studies. Two studies focused on the first pathway (selection by choice behaviour) and two studies focused on the second pathway (improvements in care). The findings show almost no effect of CPI on behavioural changes in either pathway. Selection and improving quality of care had virtually no effect. Any impact we found did not persist longer than 2 months. The level of evidence, however, was low, given the inclusion of only four studies and the analysis based on the GRADE1 system. Our mixed-method study with 15 interviews with GPs and a subsequent survey in 81 general practices (Chapter 6) aimed to examine the position of the GP in terms of guidance and support of patients making choices. We explored the barriers to GPs’ search for and use of CPI. Most GPs did not know where to find CPI (87%) and had never used it before (94%). The findings show that the decisions GPs make in the current process for referring patients to hospital care are not, or hardly, based on CPI exchanges during consultations. Other referral considerations and sources (e.g. personal contact with specialists, their own and patients’ experiences with providers, and other patient experiences) are still more important than patient choice, and there is no role for CPI as an additional source. According to the GPs, patients hardly ever 1 Grading of Recommendations, Assessment, Development and Evaluations http://clinicalevidence.bmj.com/x/set/static/ebm/learn/665072.html 156 Summary initiate a discussion of CPI during consultations, though they seem to be increasingly critical about differences in care quality. Most GPs are sceptical about the ability of patients to use CPI; they find the current information too difficult for most patients. Asking GPs about this matter taught us that several barriers also hinder their own use of CPI. These barriers include indecisiveness about their role in supporting patient choices, their task in addressing CPI during consultations, doubts about whose responsibility this is, feelings of uncertainty about the interpretation of the information, doubts and distrust of the content of the information and its reliability, and fear of weakening their networks. Whether GPs can fulfil a role in providing patients with guidance in the use of CPI is still undecided. First there must be more clarity about the conditions. In Chapter 7, the final chapter of this thesis, the most important findings and conclusions of the studies in Chapters 2–6 are discussed. The results are placed in wider perspectives and compared with other studies. The most relevant methodological limitations are considered, practical implications are presented, and recommendations are put forward for further research. The purpose of CPI is 1) to facilitate the consumer’s choice of a healthcare provider based on their requirements and personal preferences and 2) to force providers to improve the care. However, we see little proof that CPI achieves this purpose. Both consumers and GPs criticize the current CPI and therefore do not use it. Policy reasons seem to be the foundation of the current design of CPI, but the promise of CPI that provides more transparency and enables better choices can still be fulfilled and has therefore value in itself. 157 Samenvatting Samenvatting Hoofdstuk 1 presenteert een overzicht van de inhoud van dit proefschrift dat de belemmerende en bevorderende factoren van zoekgedrag naar kwaliteitsinformatie en het keuzegedrag naar zorgaanbieders door consumenten beschrijft. Het beschikbaar stellen van kwaliteitsinformatie over zorgaanbieders is een manier om zorgconsumenten inzicht te geven in de kwaliteit van zorg, zodat zij een geïnformeerde keuze kunnen maken voor een zorgaanbieder. Het onderliggende idee is dat consumenten met goede informatie kunnen kiezen voor de beste zorgaanbieder, waarmee zij druk uitoefenen op zorgaanbieders om de kwaliteit van zorg te verbeteren. Of consumenten zich zo rationeel gedragen om de keuze voor een zorgaanbieder te maken op basis van cognitieve overwegingen is de vraag. Los daarvan is het een veel belangrijker doel consumenten van goede informatie te voorzien en hen hierdoor meer controle, invloed en kracht te geven om hun eigen keuzes te maken. Hoewel zorgconsumenten meer eigen regie nastreven en ook aangeven deze informatie te wensen, gebruiken zij de informatie die hen daartoe in de gelegenheid stelt niet of in zeer beperkte mate. In dit proefschrift exploreren we drie onderzoeksthema’s, namelijk: prikkels die zoekgedrag beïnvloeden, keuzegedrag en ondersteuning bij het maken van keuzes. Allereerst kijken we naar de onderliggende redenen waarom consumenten deze informatie (nog) niet gebruiken, negeren, of moeite hebben om deze informatie te begrijpen en om op zoek te gaan naar deze informatie. Vervolgens kijken we naar het effect van kwaliteitsinformatie op gedrag bij diverse partijen. En tot slot focussen we op de rol van de huisarts om consumenten te helpen bij het maken van een geïnformeerde keuze en het gebruik van kwaliteitsinformatie tijdens het verwijsproces. In het tweede hoofdstuk presenteren we de bevindingen van een focusgroepstudie onder consumenten. Het doel van deze studie was om inzicht te krijgen in waardering, begrip, opinie en beoordeling van kwaliteitsinformatie. Zes focusgroepen vonden plaats met consumenten (n=27). De meeste deelnemers waren niet bekend met dergelijke informatie en hadden het niet eerder gebruikt bij het maken van een keuze binnen de zorg. De consumenten gaven aan niet de noodzaak te voelen om te zoeken naar kwaliteitsinformatie. Tijdens de focusgroepen gebruikten we diverse voorbeelden van kwaliteitsinformatie om de discussie verder op gang te helpen. Als reactie daarop konden consumenten sneller hun twijfels, terughoudendheid en kritische vragen onder woorden brengen waarom ze niet op zoek gingen naar deze informatie. Zo gaven zij aan andere bronnen (familie/vrienden, eigen ervaring en de huisarts) belangrijker te vinden dan kwaliteitsinformatie. Barrières ten aanzien van de waarde van kwaliteitsinformatie betroffen: gebrek aan toepasbaarheid omdat kwaliteitsinformatie vanuit diverse bronnen elkaar soms tegenspreken, onzekerheid omdat dit type informatie een beroep doet op een nieuw soort verantwoordelijkheidsgevoel waarvan ze de gevolgen (nog) niet overzien en angst om het vertrouwen te verliezen in bepaalde 160 Samenvatting aspecten van zorg waarvan ze dachten dat die goed zijn. Een ander belangrijk punt was de betrouwbaarheid van de informatie. Ook liet deze studie zien dat consumenten positieve effecten ervaren van kwaliteitsinformatie, zoals: meer keuzevrijheid, meer betrokkenheid van en regie voor consumenten, een toename in bewustzijn over kwaliteit, als ook (mogelijk) nadelige effecten, zoals tweedeling tussen mensen die wel in staat zijn om de informatie te gebruiken en zij die dit niet kunnen. Op basis hiervan kunnen we concluderen dat consumenten kwaliteitsinformatie slechts marginaal waarderen. Verschillende waarden en principes beperken het zoeken naar kwaliteitsinformatie, en dus ook het gebruik ervan. In het derde hoofdstuk is onderzocht of de verwachtingen die patiënten hebben over verschillen in kwaliteit van zorg van invloed kunnen zijn op het zoekgedrag naar kwaliteitsinformatie. Vragenlijsten werden uitgezet onder 475 patiënten die een electieve heup- of knieoperatie ondergingen in een academisch, topklinisch of streekziekenhuis. Van de 302 (respons 63%) patiënten die de vragenlijst invulden, liet een kleine minderheid weten (13%) op zoek te zijn gegaan naar kwaliteitsinformatie nadat zij hadden gehoord een vervangende heup-of knieoperatie te moeten ondergaan. Voor patiënten die niet op zoek gingen naar kwaliteitsinformatie was de belangrijkste reden dat ze simpelweg geen behoefte hadden aan meer informatie. Een meerderheid van de patiënten (67%) gaf aan verschillen te verwachten in de kwaliteit van zorg, zij gingen vaker op zoek naar kwaliteitsinformatie (OR=3.18 [95% BI: 1.02-9.89]; p <0.04) dan patiënten die geen verschillen in kwaliteit verwachtten. Voor patiënten die grote verschillen verwachtten was deze relatie nog sterker ten opzichte van patiënten die geen verschillen verwachtten (OR=5.05 [95% BI: 1.44-17.77]; p<0.01). Patiënten verwachtten voornamelijk verschillen in reputatie, afstand en toegankelijkheid. Deze aspecten hebben meer betrekking op algemene kenmerken dan specifieke, aandoeninggerelateerde aspecten. Wanneer de kennis over het concept kwaliteit van zorg toeneemt, kunnen ook de aspecten waarin patiënten verschillen in verwachtingen mogelijk veranderen. In hoofdstuk 4 rapporteren we de bevindingen van een kwantitatieve studie waarin we onderzocht hebben of patiënten in staat zijn om specifieke expertise bij hun zorgverleners te herkennen en of deze herkenning een stimulerende rol speelt bij zoekgedrag naar informatie over een zorgverlener. Om dit doel te bereiken, hebben we een studie opgezet binnen de chronische zorg. Rondom de Ziekte van Parkinson is de afgelopen jaren een multidisciplinair netwerk opgezet waarin zorgverleners (neurologen, verpleegkundigen, ergotherapeuten, fysiotherapeuten, huisartsen) intensief met elkaar samenwerken. Zorgverleners aangesloten bij dit netwerk worden extra getraind in het behandelen van mensen met de ziekte van Parkinson. Door het volgen van specifieke richtlijnen en het behandelen van een hoger patiëntenvolume van 161 Samenvatting patiënten met de ziekte van Parkinson krijgen zorgverleners meer specifieke expertise. In deze studie hebben we ons gericht op de expertise van fysiotherapeuten. De vraag is of mensen met de ziekte van Parkinson deze expertise herkennen en waarderen en ook of zij deze expertise meenemen in het zoeken naar kwaliteitsinformatie om een keuze te maken voor een fysiotherapeut. Op basis van declaratiegegevens van zorgverzekeraar CZ nodigden we 500 mensen uit om een vragenlijst in te vullen, waarvan we 320 ingevulde vragenlijsten (response 64%) retour kregen. Veruit de meeste mensen met Ziekte van Parkinson in onze studie (89%) volgden het advies op van hun verwijzer, dit advies was zelden onderbouwd met argumenten. Een kleine groep mensen (11%) ging op zoek naar extra informatie. Respondenten bleken verschillende aspecten van expertise te herkennen. De groep die specifieke Parkinson expertise herkende, ging 3.28 keer vaker op zoek naar informatie dan zij die deze toegevoegde expertise niet herkenden. Ook de bereidheid om te wisselen van fysiotherapeut naar een in Parkinson gespecialiseerde fysiotherapeut bleek groter (p < .001) te zijn bij respondenten die de expertise onderschreven. Op basis van de declareergegevens zou ongeveer 28% van de respondenten onder behandeling zijn van een gespecialiseerde fysiotherapeut, terwijl 70% van de respondenten aangaf onder behandeling te zijn van een gespecialiseerde fysiotherapeut. Deze (mogelijke) overschatting van de specifieke expertise in Parkinson bij de eigen fysiotherapeut kan ook een rol spelen in het op zoek gaan naar kwaliteitsinformatie. Mensen gaan er al vanuit dat zij bij de beste zitten en zien daarom geen noodzaak om op zoek te gaan naar kwaliteitsinformatie en met die informatie op te zoek te gaan naar een andere fysiotherapeut. In het vijfde hoofdstuk presenteren we een systematische literatuurstudie naar de effecten van openbare kwaliteitsinformatie op gedrag van zorgconsumenten, zorgaanbieders en zorginkopers. Hoewel er sinds begin jaren ’90 in de Verenigde Staten en Engeland - en in recente jaren in veel andere landen, waaronder Nederland - veel geïnvesteerd is in de ontwikkeling van kwaliteitsinformatie en het beschikbaar maken van die informatie, waren er tot dusverre geen reviews uitgevoerd naar de effectiviteit. Daarom analyseerden we studies naar de effecten van openbaar beschikbare kwaliteitsinformatie op gedrag. Het achterliggende idee van het openbaar maken van kwaliteitsinformatie is dat patiënten deze informatie inzetten om de kwaliteit van te zorg te verbeteren. Uit de literatuur gebruikten we een model met twee paden dat laat zien hoe kwaliteitsinformatie een effect zou kunnen uitoefenen om de zorg te verbeteren namelijk door: 1) keuzegedrag voor de beste zorgaanbieder en 2) verbeterinitiatieven vanuit zorgaanbieders om de kwaliteit van zorg te verbeteren. Wij gebruikten deze twee paden als gedragsuitkomstmaten. Met een uitgebreide zoekstrategie, waarmee vijf databanken werden doorzocht tot 2011, werd een groot aantal studies (n=6839) gevonden. Uiteindelijk konden na een strenge 162 Samenvatting selectieprocedure vier studies geïncludeerd worden. Twee studies richtten zich op het eerste pad (selectie door middel van keuzegedrag) en twee studies focusten zich op het tweede pad (verbeteringen in de zorg). De impact van kwaliteitsinformatie op gedrag was niet of nauwelijks aanwezig en die impact die wél gevonden werd, was van korte duur. Concluderend kunnen we stellen dat kwaliteitsinformatie niet of nauwelijks een effect heeft op gedrag. Maar, het bewijs waarop we deze conclusie baseren is beperkt met slechts vier studies, en de analyse van de studies met het GRADE1 systeem laat zien dat de kwaliteit van de studies laag is. Daarbij zijn alle vier studies uitgevoerd in een relatief nieuw onderzoeksveld en in een sterk veranderende context. Concluderend kunnen we stellen dat het effect nog weinig op een goede wijze is onderzocht en dat er dus nog geen harde conclusie getrokken kan worden over het effect van kwaliteitsinformatie op gedrag. In het zesde hoofdstuk beschrijven we een mixed-method studie met als doel te onderzoeken welke rol kwaliteitsinformatie speelt in het huidige verwijsbeleid van huisartsen en welke rol de huisarts speelt bij het stimuleren van keuzegedrag, gebaseerd op kwaliteitsinformatie door patiënten. We hebben gekeken welke barrières huisartsen ondervinden bij het zoeken en gebruiken van kwaliteitsinformatie. Vijftien diepte-interviews vonden plaats met huisartsen over verwijsroutines, patiëntbetrokkenheid en de rol van kwaliteitsinformatie tijdens het verwijsproces. De interviews werd letterlijk uitgewerkt, geanalyseerd en de resultaten werden vervolgens verwerkt in een vragenlijst die werd uitgezet onder 81 huisartsen die verbonden waren aan het Landelijke Informatie Netwerk Huisartsenzorg 70 huisartsen (86% respons) vulden de vragenlijst in. De meeste huisartsen wisten niet waar ze kwaliteitsinformatie zouden moeten zoeken (87%) en hadden het nooit eerder gebruikt (94%). Kwaliteitsinformatie speelt geen tot een minimale rol bij het verwijzen van patiënten naar een ziekenhuis of andere professionals, terwijl andere bronnen zoals het persoonlijk contact met artsen, eigen ervaringen, patiëntervaringen en patiëntvoorkeuren voor huisartsen belangrijke bronnen zijn in het verwijsproces. Huisartsen noemden veel verschillende barrières die ervoor zorgden dat ze niet gemotiveerd worden om kwaliteitsinformatie te gebruiken. Ze twijfelden aan hun rol om patiënten te stimuleren in keuzegedrag, en vroegen zich af of het gebruik van kwaliteitsinformatie niet behoort tot de eigen verantwoordelijkheid van de patiënt, terwijl ze zich tegelijk zeer verantwoordelijk voelden voor een goed verwijsproces. Daarnaast twijfelden ze aan de toegevoegde waarde van kwaliteitsinformatie en aan de mogelijk positieve uitkomsten voor patiënten. Ten aanzien van het eigen gebruik van kwaliteitsinformatie voelden huisartsen zich onzeker of zij in staat zijn om de 1 Grades of Recommendations Assessment Development and Evaluation (GRADE) http://clinicalevidence.bmj.com/x/set/static/ebm/learn/665072.html 163 Samenvatting informatie op een goede manier te interpreteren. Zij bekritiseerden ook de inhoud van de informatie, omdat het volgens hen inhoudelijk gezien niet de belangrijkste informatie bevat waaraan patiënten behoefte hebben en zij waren kritisch over de validiteit van de informatie, waardoor ze de betrouwbaarheid ervan in twijfel trokken. Daarnaast gaven ze aan weinig noodzaak te voelen om het te gebruiken, aangezien zij nauwelijks geconfronteerd worden met vragen vanuit patiënten over kwaliteitsinformatie. Volgens huisartsen is een keuze maken op basis van kwaliteitsinformatie een erg moeilijke opgave voor patiënten. Tot slot noemden zij dat het gebruik van kwaliteitsinformatie tot een (te) grote verwijsregio (meer ziekenhuizen en zorgverleners) zou kunnen leiden, waarmee ze aangaven daarmee ook overzicht en belangrijke informatiebronnen te verliezen. Of huisartsen een stimulerende rol kunnen vervullen in het gebruik van kwaliteitsinformatie door patiënten staat nog niet vast, eerst moet meer duidelijk worden welke voorwaarden voor huisartsen gerealiseerd moeten worden om deze rol vorm te geven. In hoofdstuk 7, het laatste hoofdstuk van dit proefschrift, worden de belangrijkste bevindingen van dit proefschrift uit de voorgaande hoofdstukken samengevat en bediscussieerd. De resultaten worden in een groter perspectief geplaatst en vergeleken met andere studies. De meest relevante methodologische overwegingen en beperkingen worden besproken, praktische implicaties voor beleidsmakers, ontwikkelaars van kwaliteitsinformatie worden gepresenteerd en er worden aanbevelingen gedaan voor vervolgonderzoek. Vergelijkende kwaliteitsinformatie is ontwikkeld om 1) consumenten en patiënten te helpen bij het kiezen van een zorgverlener die past bij hun behoeften en voorkeuren 2) en indirect zorgverleners hiermee te stimuleren om de kwaliteit van zorg (verder) te verbeteren. Echter, er is weinig bewijs dat de presentatie van kwaliteitsinformatie aan consumenten en patiënten een effect heeft op gedrag. Zowel consumenten als huisartsen bekritiseren huidige kwaliteitsinformatie en gebruiken het mede daarom niet of nauwelijks. Toch kan de belofte dat kwaliteitsinformatie meer transparantie biedt en mensen in staat stelt betere keuzes te maken nog steeds vervuld worden en heeft in dat opzicht waarde in zichzelf. 164 Dankwoord Dankwoord Een ware uitdaging om het grote aantal mensen dat mij geholpen heeft hier kort te bedanken. In de eerste plaats wil ik alle respondenten (zorgconsumenten, patiënten en huisartsen) bedanken die een bijdrage hebben geleverd. Veel dank ben ik verschuldigd aan mijn promotoren prof. dr. Gert Westert, prof. dr. Glyn Elwyn en copromotoren dr. Marjan Faber, dr. Jozé Braspenning. Beste Marjan, het duurde even voordat wij met elkaar de juiste modus hadden gevonden en goed op elkaar ingespeeld raakten. Ik in mijn rol als promovendus, jij in je rol als copromotor voor wie ik de eerste promovendus was. Jouw inhoudelijke expertise, schrijfkunsten, parate kennis van de literatuur en werklust zijn een bron van inspiratie. Veel dank voor je oneindige precisie en het immer secuur lezen van de zoveelste versie van mijn artikelen. Beste Jozé, op heldere wijze wist je onvolkomenheden in de structuur onder de aandacht te brengen en mij te voorzien van scherpe analyses, zodat ik nieuwe stappen kon zetten in het schrijfproces. Heel veel dank voor je inzet de afgelopen jaren. Beste Gert, je volgde prof. Richard Grol op als directeur van IQ en als promotor, hierdoor stapte je iets later in het traject. Je feedback was altijd constructief en dit zorgde voor betere artikelen. Veel dank voor je adviezen en je tijd om dit tot een succesvol einde te brengen. Dear Glyn, many thanks for your efforts and time spend traveling to Nijmegen. It was a privilege to work with you. Beste Richard, kort na de start van mijn promotietraject besloot je dat ik niet bij de club met promovendi zat die je nog ging begeleiden ná je emeritaat. Helaas, maar het doet me genoegen dat ik met je heb mogen werken. Ik dank je voor de start van het traject en het gestelde vertrouwen in mij. Dit proefschrift had niet tot stand kunnen komen zonder de medewerking en financiële steun van zorgverzekeraar CZ, dank voor de samenwerking en support. Speciale dank gaat uit naar Wiro Gruisen, Jolyn van Vuuren, Marjolein Morres en Linda van Mierlo. Een woord van dank gaat uit naar de poli’s Orthopedie van de deelnemende ziekenhuizen Radboudumc, Slingeland Ziekenhuis en Elizabeth Ziekenhuis Eindhoven. Externe co-auteurs: Martin Eccles, Katherine Deane, Signe Flottorp, Liv Rygh, Marten Munneke en Bas Bloem, bedankt voor jullie bijdragen en correcties bij de betreffende artikelen. 166 Dankwoord Collega’s van IQ healthcare Anita Huis, Hilly Calsbeek en Simone van Dulmen, mijn drie geweldige kamergenoten, wil ik heel hartelijk bedanken voor het meeleven van de hele rit van begin tot eind. Goede adviezen en praktische tips bij het organiseren en plannen van de onderzoeksactiviteiten. Hilly en Simone, wat was het bijzonder om drie gelijktijdige zwangerschappen met elkaar te delen, gevolgd door goede pedagogische adviezen bij het jonge moederschap. Anita, jouw ‘24 uur-crisisdienst verhalen’ en bizarre voorvallen bezorgden mij regelmatig de slappe lach. En wat ben ik blij dat onze goede traditie van de ‘R&A etentjes’ blijven doorgaan. Maud Heinen en Betsie van Gaal, wat ontzettend fijn dat jullie zijn aangeschoven, input leveren voor de ‘R&A’ en meedelen in deze traditie. Hopelijk houden we die nog heel lang vast. Onderzoeksondersteuning en praktische hulp kreeg ik van Marc Padros, Ellen Keizer en Reinier Akkermans, veel dank voor jullie hulp! Alice Tillema, mijn dank is groot voor je hulp om alle databases te doorzoeken en bij het opstellen van de juiste zoektermen. Sylvia van Roosmalen, veel dank gaat uit naar je werk van het redigeren van mijn artikelen en je inzet bij de totstandkoming van dit manuscript. Jolanda van Haren, dankjewel voor je hulp, het meekijken en je adviezen in de totstandkoming van het manuscript en uiteindelijke boekje. Speciale dank gaat uit naar mijn intervisie collega’s van IQ; Anke Oerlemans, Karin Neeleman-van der Steen, Geertje van de Ven, Gijs Hesselink en Irene van de Glind. Irene, dank ook voor je tijd dat we samen medevoorzitter waren van Schil. Ik kijk uit naar jouw promotie! Renate Jansink en Kirsten Kirschner, jullie waren al bezig met jullie promoties toen ik bij IQ kwam. We hadden meteen een ‘klik’ en ik blij dat we die ondanks de drukke levens proberen vast te houden. Dank voor jullie meeleven en spreekwoordelijke dansjes bij grote en kleine lichtpuntjes. Arna van Doorn–Klomberg wil ik bedanken voor het prettig sparren en meelezen in een fase waarin het schrijven even op slot zat. Renske Keizer, oud studiegenote van ASW, al lang gepromoveerd en al ruim een jaar hoogleraar. Geweldig! Hartelijk dank voor je hulp en adviezen bij het schrijven van de discussie. Collega’s Windesheim Mijn aanstelling bij Windesheim combineerde ik met de afronding van mijn proefschrift. Heel fijn om herkenbare hobbels en heuglijke feiten met jullie te kunnen delen. Speciale dank gaat uit naar Leontine Groen–van de Ven, Marijke Span, Monique Mensen, Jan Jukema en Carolien Smits: jullie wijsheden en aanmoedigingen ‘Trust the 167 Dankwoord process’ en ‘Shut up and write’ zorgden ervoor dat ik ondanks de drukke werkzaamheden bij Windesheim moedig voortwaarts bleef gaan. Collega’s Saxion Geen bijdrage in het traject, maar wél de vreugde van de afronding vieren. Ik ben blij dat ik met zo’n geweldige club mensen mag werken! Paranimfen Mirjam, het is niet meer haalbaar om te carpoolen nu we in Nijmegen en Enschede werken, maar wat kijk ik goed terug op onze dagelijkse evaluatiemomenten in de auto. Jouw nuchtere blik en pragmatische houding, gecombineerd met een wetenschappelijke analyse is onovertroffen. Veel dank dat je vandaag naast mij wilt staan. Lieve Anne, onze levens zijn innig met elkaar verbonden. Mijn dierbare vriendin en schoonzus waarmee ik keihard kan lachen, kan relaxen, kan mopperen over de dagelijkse beslommeringen en met wie ik de meest ambitieuze plannen kan ontvouwen. Wat een eer dat je naast mij staat! Familie Het afronden van een promotie naast een andere baan en een gezin heeft ook één en ander gevraagd van de mensen die dicht bij mij staan. Lieve schoonfamilie, Anne & Roy, Els & Joop, heel erg bedankt voor het intensief meeleven, het blijven vragen en al die extra oppasmomenten van de afgelopen tijd! Lieve zusjes en zwagers, Marloes & Frank, Janneke & Rick, we beleefden met elkaar de afgelopen twee jaar op z’n zachtst gezegd ‘roerige tijden’. Dank voor alle gezellige momenten en steun. Lieve papa en mama, wat de toekomst brengen zal is nog ongewis, maar de basis die jullie mij hebben meegegeven is van jullie samen. Veel dank voor jullie liefde en steun! Mama, dank voor al die onzichtbare en ontelbare dingen die er in de afgelopen tijd voor zorgden dat ik rustig heb kunnen schrijven. Mijn stoere ‘mannen’, Jens en Lars, ik geniet volop van jullie en ben zó blij dat ik vanaf nu meer tijd voor jullie heb. Lieve Martijn, jouw steun is ontzettend belangrijk geweest om dit tot einde te kunnen brengen. Dat alles hoeft wat jou betreft niet breed uitgesponnen te worden, dat weten is al genoeg. Het levensmotto ‘het zijn de kleine dingen…’ past je als geen ander. DANK! 168 Curriculum Vitae 169 Curriculum Vitae 170 Curriculum Vitae Nicole Ketelaar werd geboren op 26 maart 1982 in Doetinchem. Na haar VWO-diploma (2001) startte zij aan de Universiteit Utrecht met de studie Algemene Sociale Wetenschappen. Haar masterthesis werd uitgevoerd binnen het Slingeland Ziekenhuis in Doetinchem en richtte zich op de transmurale zorg, in het bijzonder op de overdracht van farmaceutische zorg van intra- naar extramurale setting. Zij behaalde haar Master binnen de afstudeerrichting Cultuur, Zorg en Welzijn in 2005. Na haar studie werkte zij korte tijd (2006-2007) als beleidsmedewerker in het Facilitair Bedrijf van het UMC Utrecht. Zij combineerde dit met een aanstelling als interviewer bij de stichting ‘Cliënt en Kwaliteit’. In 2007 maakte zij de overstap naar het Radboudumc, bij het Scientific Institute for Quality of Healthcare (IQ healthcare). Halverwege 2008 startte haar promotieonderzoek, ‘De Kiezende Zorggebruiker’, waarvan de resultaten in dit proefschrift beschreven zijn. Dit project werd gesubsidieerd door het CZ Fonds. Naast haar promotieonderzoek was zij betrokken bij verschillende onderzoeksprojecten en schreef zij mee aan een handboek ‘Doen bij Depressie’. In het najaar van 2012 werd Nijmegen verruild voor Zwolle en werkte zij als onderzoeker/docent aan de Hogeschool Windesheim, Lectoraat Innoveren in de Ouderenzorg. In samenwerking met de Hogeschool Arnhem en Nijmegen werden er praktische tools ontwikkeld voor casemanagers dementie binnen drie dementienetwerken. Momenteel is zij als onderzoeker verbonden aan de Hogeschool Saxion, Academie Mens en Maatschappij, waar zij een onderzoek uitvoert naar de transitie van het sociale domein en sociale wijkteams binnen drie Twentse gemeenten. Naast dit onderzoeksproject neemt zij als onderzoeker deel aan het Expertise Centrum Jeugdzorg Twente (EJT) en is zij als hoofddocent betrokken bij de Master Health Care and Social Work. Nicole woont samen met Martijn Kraan. Zij zijn de trotse ouders van Jens (5) en Lars (2). 171 Portfolio Radboud Institute for Health Sciences 174 Name PhD student: N.A.B.M. Ketelaar Department: Scientific Institute for Quality of Health Graduate School: Radboud Institute for Health Sciences PhD period: 01-06-2008 – 02-12-2015 Promotor(s): Prof. G.P. Westert, Prof. G. Elwyn Co-promotor(s): Dr M.J. Faber, Dr J.C. Braspenning TRAINING ACTIVITIES a) Courses & Workshops ‐ Herregistratie Basiscursus Regelgeving en Organisatie voor Klinisch onderzoekers (BROK) ‐ CaRe annual day ‘International collaboration in Primary Health Care Research’ ‐ Solliciteren en netwerken, Radboud University ‐ IQ healthcare congress, Radboudumc ‐ Masterclass Lecture: ‘Planning for future career’ ‐ Academic writing (one-one-coaching), Radboud in’to languages ‐ Cochrane review / review manager, EMGO ‐ Hulp bij subsidieaanvragen, ZonMw ‐ Instruction about literature searches ‐ Advanced conversation, Radboud in’to languages ‐ IQ healthcare congress, Radboudumc ‐ Presenteren eigen onderzoek, Radboud University ‐ Basiscursus Regelgeving en Organisatie voor Klinisch onderzoekers (BROK) ‐ Cursus SPSS, PAO Heyendael ‐ Academic writing, Radboud in’to languages ‐ Introductie cursus CaRe ‐ PubMed introduction, Medical Library ‐ Posterpresentatie / opzetten van wetenschappelijke posters, NCEBP ‐ Implementatie. Effectieve strategieën, IQ healthcare ‐ Wetenschapsjournalistiek, Radboud University b) Seminars & lectures^ ‐ Netwerkbijeenkomst Bruikbaar Onderzoek, ZonMw ‐ Patient Preferences in quality improvement research (seminar) UMC St Radboud. ^ presentation ‐ Onderzoek naar keuze van patiënten met de ziekte van Parkinson voor een gespecialiseerde fysiotherapeut. UMC St Radboud, afdeling neurologie. ^ presentatie ‐ Zijn er effecten door het gebruik van kwaliteitsinformatie? Zorgverzekeraar CZ, Tilburg. ‐ Wat is er al bekend over de kiezende zorggebruikers? Onderzoeksopzet. IQ healthcare. ^ presentatie Year(s) ECTS 2013 0.1 2012 0.25 2012 2012 2011 2011 1.75 0.25 0.1 2 2010 2010 2010 2010 0.2 0.1 0.1 2 0.25 1.5 1.75 2010 2009 2009 2009 2009 2009 2008 2008 0.2 3 0.5 0.1 0.1 2007 - 2008 2007 2 2 2013 2011 0.1 0.25 2011 0.25 2011 0.1 2008 0.1 175 Year(s) ECTS 2015 0.25 2011 0.5 2009 0.5 2008 - 2010 0.2 2008 1.5 2014 2011 - 2012 2012 - 2015 0.1 1 0.7 2011 2009 2008 2007 1 1 2 0.5 TRAINING ACTIVITIES c) Symposia & congresses^ ‐ Vascular Symposium, ‘Innovation in time of scarcity. Think ‘evidence’ and ‘quality’, University Hospitals Leuven, Belgium ^ presentation ‐ International Shared Decision Making Conference (ISDM), University Maastricht, Maastricht ^ presentation ‐ International Shared Decision Making Conference (ISDM), Harvard Medical School, Boston ^ poster ‐ Visiting several congress regarding the subject ‘Choosing healthcare providers’ (e.e.g. Ede, Erasmus University) ^ visiting ‐ Co-organisation symposia: 5 jaar samenwerking van IQ healthcare en Zorgverzekeraar CZ d) Other ‐ Te gast bij journal club van NIVEL ‐ Intervisie met promovendi ‐ Review scientific publication, journals: Health Expectations, Health Services Research, Plos One, International Journal of Quality in Healthcare Care, BMC Medical Informatics and Decision Making, Evaluation of Clinical Practice ‐ Lid van Kennisgroep ‘Patient Empowerment’ ‐ Lid van journal club ‐ Voorziter van Schil ‐ Lid van Schil. Overleg orgaan voor junior onderzoekers en promovendi binnen IQ healthcare TEACHING ACTIVITIES e) Lecturing ‐ Kiezen in Zorg. Transparantie en kwaliteitsinformatie. UMC St Radboud. Onderwijs voor geneeskunde studenten. f) Supervision of internships / other ‐ TOTAL ^Indicate oral or poster presentation 176 2011 - 2012 2011 2010 2012 0.4 28.4