Evaluating patient and public involvement in research

BMJ 2018;363:k5147 doi: 10.1136/bmj.k5147 (Published 6 December 2018) Page 1 of 2 Editorials Evaluating patient and public involvement in research If we are serious about involvement, we need to be equally serious about evaluation 1 2 Antoine Boivin co-director , Tessa Richards senior editor/patient partnership , Laura Forsythe 3 1 director of evaluation and analysis , Alexandre Grégoire patient partner , Audrey L’Espérance 1 4 3 senior adviser , Julia Abelson professor , Kristin L Carman director of public and patient engagement 1 Center of Excellence for Partnership with Patients and the Public, University of Montreal Hospital Research Center and Faculty of Medicine, Montreal, Canada; 2BMJ, London, UK; 3Patient-Centered Outcomes Research Institute, Washington, DC, USA; 4Department of Health Research Methods, Evidence, and Impact, Centre for Health Economics and Policy Analysis, McMaster University, Hamilton, ON, Canada Research funders increasingly recommend and require patient and public involvement (PPI) in the design, conduct, and dissemination of health and social care research.1-3 In the literature and policy discourse, PPI is justified by two lines of argument: one on the basis of ethical principles, the other on the assumption that it may improve the quality, relevance, and uptake of research.4 The scientific community holds polarised views on involvement, but all are calling for stronger evidence.5 Those critical of PPI want more evidence on the costs, benefits, and risks before they undertake anything more than a tick box approach to obtaining grants. For advocates already engaging with patients and the public, evaluation is necessary to understand how best to do PPI and fully reap the benefits of working together. Over the past 10 years, the international literature evaluating PPI has more than tripled.6-9 The recent systematic review by Cocker and colleagues focusing on PPI’s effect on enrolment and retention in clinical trials (doi:10.1136/bmj.k4738)10 is an important addition. Their review suggests that PPI can improve rates of enrolment in clinical trials, thus bringing the most robust evidence to an association that has long been hypothesised.8 The finding is important because recruitment difficulties can reduce trial validity, add costs, and increase the risk of studies being abandoned and not reported.11 Although PPI significantly improved recruitment, the effect size was modest. This might reflect the fact that all forms of PPI were pooled, from simply having a patient on an advisory committee to full patient partnership in research governance, design, and peer recruitment. It also suggests that PPI alone cannot be expected to solve all recruitment problems. A second important finding is that the effectiveness of PPI is strongest when people with lived experience of the condition being studied are involved as research partners. This supports the view of patients and the public as experience based experts who contribute knowledge that is complementary to that of scientists and professionals.12 Moving forward, PPI evaluation should be embedded in the broader evaluation of research quality, relevance, and effect. PPI evaluation should be grounded in key principles (box 1). Box 1: Key principles for PPI evaluation • Clarity • Reflexivity • Methodological rigour • Transparency • Pragmatism • Reciprocity The language, definitions, and goals of PPI vary between stakeholders, cultures, and countries. Clarity of goals and definitions is a precondition for evaluation. This starts with characterisation of PPI—its core features, activities, and mechanisms—and being explicit about which goals are being pursued by each person involved.13 We may not reach perfect consensus, but if we hope to support those who view involvement as an ethical imperative and those who want proof of effectiveness to proceed, we need to attend to the information needs of both. Evaluators need to be reflective about unspoken values and power imbalance underpinning patient and public involvement and its evaluation.14 15 For example, calls for evaluating the effect of PPI raise further questions about “effect on what and for whom”? While some researchers may involve patients and the public to increase study recruitment rates, patients and citizens may be more interested in ensuring that research answers questions and includes outcomes that matter to them and their communities. Strengthening the science underpinning PPI is key to ensuring that it becomes an integral, robustly conducted, and well resourced component of research, not a last minute add on.16 Progress has been made in defining and measuring high quality Correspondence to: A Boivin antoine.boivin@umontreal.ca For personal use only: See rights and reprints http://www.bmj.com/permissions Subscribe: http://www.bmj.com/subscribe BMJ: first published as 10.1136/bmj.k5147 on 6 December 2018. Downloaded from http://www.bmj.com/ on 24 April 2020 by guest. Protected by copyright. EDITORIALS BMJ 2018;363:k5147 doi: 10.1136/bmj.k5147 (Published 6 December 2018) Page 2 of 2 EDITORIALS Greater transparency, including proper scrutiny of the involvement process and support structures, is essential to interpreting whether an “absence of effect” is because the PPI is ineffective (theory failure) or because of inappropriate resources and suboptimal implementation (practice failure).24 Funders and publishers can help open the black box by requiring more detailed information about PPI in grants and publications. We must move forward pragmatically, to ensure that evaluation efforts are not paralysed by the misguided perception that PPI is too controversial or complex to be studied. Evaluation need not be complicated to be useful. Simple feedback between patients and researchers can improve the involvement process, spur mutual learning, and change researchers’ mindsets and future practice.25 26 Finally, as we move toward co-production models of research (focusing on co-leadership, mutual learning, and shared benefits between science and society), reciprocity becomes a core principle of PPI and its evaluation. Involvement has to become a truly joint enterprise where patients and citizens are empowered to act as full partners in research and its evaluation, on an equal footing with other stakeholders.27 A vast amount of public money and human capital is invested in health research. Since PPI is increasingly seen as pivotal to improving the value and relevance of research, we need to get serious about how it is done and equally serious about how it is evaluated. 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 We thank Susan Hrisos (international conference on evaluation of patient and public involvement in research, Newcastle University), Sophie Staniszewska 20 (University of Warwick) and Valerie Lahaie (Centre hospitalier de l’Universitéde Montréal) for their contribution. 21 Competing interests: We have read and understood BMJ policy on declaration of 22 interests and declare AB holds the Canada research chair on patient and public partnership and is assistant professor at the University of Montreal’s department 23 of family medicine and department of management, evaluation, and health policy. AG is funded by the Quebec SPOR-SUPPORT Unit patient and public partnership 24 strategy. Provenance and peer review: Commissioned; not externally peer reviewed. 25 1 2 3 National Institute for Health Research. Patient and public involvement in health and social care research: a handbook for researchers. National Institute for Health Research London, 2010. Fleurence R, Selby JV, Odom-Walker K, etal . How the Patient-Centered Outcomes Research Institute is engaging patients and others in shaping its research agenda. Health Aff (Millwood) 2013;32:393-400. 10.1377/hlthaff.2012.1176 23381533 Canadian Institutes of Health Research. Strategy for patient-oriented research—patient engagement framework. 2014; http://www.cihr-irsc.gc.ca/e/48413.html 26 27 Wicks P, Richards T, Denegri S, Godlee F. Patients’ roles and rights in research. BMJ 2018;362:k3193. 10.1136/bmj.k3193 30045909 Becker S, Sempik J, Bryman A. Advocates, agnostics and adversaries: researchers’ perceptions of service user involvement in social policy research. Soc Policy Soc 2010;9:355-66.10.1017/S1474746410000072 Boote J, Wong R, Booth A. “Talking the talk or walking the walk?” A bibliometric review of the literature on public involvement in health research published between 1995 and 2009. Health Expect 2015;18:44-57. 10.1111/hex.12007 23033933 Domecq JP, Prutsky G, Elraiyah T, etal . Patient engagement in research: a systematic review. BMC Health Serv Res 2014;14:89. 10.1186/1472-6963-14-89 24568690 Brett J, Staniszewska S, Mockford C, etal . Mapping the impact of patient and public involvement on health and social care research: a systematic review. Health Expect 2014;17:637-50. 10.1111/j.1369-7625.2012.00795.x 22809132 Jagosh J, Macaulay AC, Pluye P, etal . Uncovering the benefits of participatory research: implications of a realist review for health research and practice. Milbank Q 2012;90:311-46. 10.1111/j.1468-0009.2012.00665.x 22709390 Crocker JC, Ricci-Cabello I, Parker A, etal . Impact of patient and public involvement on enrolment and retention in clinical trials: systematic review and meta-analysis. BMJ 2018;363:k4738. 10.1136/bmj.k4738 30487232 Duley L, Gillman A, Duggan M, etal . What are the main inefficiencies in trial conduct: a survey of UKCRC registered clinical trials units in the UK. Trials 2018;19:15. 10.1186/s13063-017-2378-5 29310685 Karazivan P, Dumez V, Flora L, etal . The patient-as-partner approach in health care: a conceptual framework for a necessary transition. Acad Med 2015;90:437-41. 10.1097/ACM.0000000000000603 25607943 Abelson J, Humphrey A, Syrowatka A, Bidonde J, Judd M. Evaluating patient, family and public engagement in health services improvement and system redesign. Healthc Q 2018;21:61-7. 10.12927/hcq.2018.25636 Boivin A, Lehoux P, Burgers J, Grol R. What are the key ingredients for effective public involvement in health care improvement and policy decisions? A randomized trial process evaluation. Milbank Q 2014;92:319-50. 10.1111/1468-0009.12060 24890250 Madden M, Speed E. Beware zombies and unicorns: toward critical patient and public involvement in health research in a neoliberal context. Frontiers in Sociology 2017;2:7.10.3389/fsoc.2017.00007 Sheridan S, Schrandt S, Forsythe L, Hilliard TS, Paez KAAdvisory Panel on Patient Engagement (2013 inaugural panel). The PCORI engagement rubric: promising practices for partnering in research. Ann Fam Med 2017;15:165-70. 10.1370/afm.2042 28289118 Abelson J, Gauvin F-P. Assessing the impacts of public participation: concepts, evidence and policy implications. Canadian Policy Research Networks Ottawa, 2006. Gibson A, Welsman J, Britten N. Evaluating patient and public involvement in health research: from theoretical model to practical workshop. Health Expect 2017;20:826-35. 10.1111/hex.12486 28664563 Carman KL, Dardess P, Maurer M, etal . Patient and family engagement: a framework for understanding the elements and developing interventions and policies. Health Aff (Millwood) 2013;32:223-31. 10.1377/hlthaff.2012.1133 23381514 Boivin A, L’Espérance A, Gauvin FP, etal . Patient and public engagement in research and health system decision making: a systematic review of evaluation tools. Health Expect 2018;21:1075-84. 10.1111/hex.12804 30062858 Staniszewska S, Brett J, Simera I, etal . GRIPP2 reporting checklists: tools to improve reporting of patient and public involvement in research. Res Involv Engagem 2017;3:13. 10.1186/s40900-017-0062-2 29062538 Staniszewska S, Adebajo A, Barber R, etal . Developing the evidence base of patient and public involvement in health and social care research: the case for measuring impact. Int J Consum Stud 2011;35:628-3210.1111/j.1470-6431.2011.01020.x Moving forward with global patient and public involvement in research. 2018; https:// community.cochrane.org/news/moving-forward-global-patient-and-public-involvementresearch Sofaer S. Using the taxonomy and the metrics: what to study when and why. Comment on “Metrics and evaluation tools for patient engagement in healthcare organization-and system-level decision-making: a systematic review.”Int J Health Policy Manag 2018;8:51-4.10.15171/ijhpm.2018.99 Mathie E, Wythe H, Munday D, etal . Reciprocal relationships and the importance of feedback in patient and public involvement: a mixed methods study. Health Expect 2018;21:899-908. 10.1111/hex.12684 29654644 Staley K. “Is it worth doing?” Measuring the impact of patient and public involvement in research. Res Involv Engagem 2015;1:6. 10.1186/s40900-015-0008-5 29062495 Durose C, Richardson L, Perry B. Craft metrics to value co-production. Nature 2018;562:32-3. 10.1038/d41586-018-06860-w 30283121 Published by the BMJ Publishing Group Limited. For permission to use (where not already granted under a licence) please go to http://group.bmj.com/group/rights-licensing/ permissions For personal use only: See rights and reprints http://www.bmj.com/permissions Subscribe: http://www.bmj.com/subscribe BMJ: first published as 10.1136/bmj.k5147 on 6 December 2018. Downloaded from http://www.bmj.com/ on 24 April 2020 by guest. Protected by copyright. PPI in different contexts,17-20 how to report it,21 and how to assess its effect.16 22 International collaborations are also helping to foster involvement science and methodological rigour.23