The Participatory Processes in Public Policy-Making: A Scoping Review

Abstract

The topic of participatory processes in public policy-making is becoming increasingly common, yet their quality remains disputed. There is as yet no reproducible quantitative assessment method that can be generalized to all such processes. This article proposes a scoping review, guided by the PRISMA research protocol, to identify key concepts linked with the democratic quality of participatory processes. The key concepts which appear are accountability, comprehensiveness, influence, results, deliberation, inclusiveness, representativeness, transparency and context. We conclude that this scoping review provides a rigorous basis upon which a generalizable quantitative evaluation framework can be developed.

1. Introduction

Citizen participation is an increasingly common topic worldwide. Citizens are demanding greater opportunities to participate in the development of public policies, and the authorities concerned are multiplying the number of mechanisms in this direction. However, the participatory processes proposed by local or national government authorities are not universally supported. The real influence of participants and their usefulness in producing effective public policies are called into question [1,2]. Processes are often evaluated in isolation by the same agencies that designed them, by independent firms or sometimes by researchers. Depending on the actors involved, the evaluation methods and vocabulary used vary, resulting in a large number of potentially confounding variables [3,4]. Evaluation frameworks have been devised by various researchers over the last few decades, and case studies have identified important factors for the democratic quality of participatory processes [5,6,7]. As each process has its own specific objectives, responding to the needs of its stakeholders and evolving in a different context, it is important to take these parameters into account when carrying out a relevant evaluation.

In this context, we define participatory processes in public policy-making as mechanisms through which citizen participation, in various forms, aims to influence the direction of public policies. This influence may occur directly, by enabling citizens to interact with policymakers to co-create policies, or indirectly, by helping them make informed choices in direct democracy processes. Ultimately, these processes—whether direct or indirect—seek to meaningfully alter the content or direction of public action by empowering citizens to influence policy choices.

Democratic quality in participatory processes refers to their ability to ensure inclusive representation, active and informed participation, and transparency throughout decision-making, as highlighted by Arnstein [8]. Beyond mere consultation, democratic quality requires providing citizens with real influence over political decisions, thereby fostering perceived legitimacy in public action. This includes input, throughput and output legitimacy, as well as integration within the public sphere [9], ensuring that participatory processes genuinely alter the initial political action line.

Defining each stakeholder’s expectations precisely when designing the process itself in a participatory way is essential [3,4,5,6,7,8,9,10]. This should focus not just on expected results but also on the entire procedure [11]. The meta-criteria present in the scientific literature enable an assessment of whether a process upholds the principles of democratic quality. Fairness [12] concerns whether each stakeholder can play a meaningful role in the participation process. The four keys to fairness are the opportunity to be present, initiate discussion, participate in discussion and participate in decision-making. Competence [12] concerns whether a process can effectively lead participants to the best solution with minimum resources. Key factors are access to and interpretation of information, and use of the best procedures for knowledge selection. Social learning [13] concerns whether stakeholders can learn from each other and build relationships. It enables better conflict management, better relations between groups, a sense of belonging and awareness of participation costs. It also considers the participants’ ability to improve their participation practices and meet their future needs.

Despite the importance of the issue in current decision-making processes, we have not found any systematic review on this topic. The aim of this paper is to identify key concepts for assessing the democratic quality of participatory processes in the context of public policy decision-making.

2. Materials and Methods

The scoping review was carried out by two researchers, following the steps in accordance with the PRISMA research protocol [14]. Five searches were carried out on the Scopus and Web of Sciences databases (SSCI and SCI-EXPANDED) over a search period from December 2022 to March 2023. Article languages were limited to English, Spanish and French. We focused our search on publications from the 2000s onwards, a period characterized by an intensification of citizen participation practices in public policy development.

Although this is a scoping review, we followed the PRISMA guidelines to maintain a high level of scientific rigor. The PRISMA protocol is a reference guide for conducting high-quality systematic reviews. By precisely structuring the steps of identification, screening, eligibility and inclusion of studies, following the PRISMA protocol makes it possible to ensure comprehensiveness in identifying relevant key concepts while minimizing the risk of bias. The PRISMA 2020 checklist is available for consultation in Appendix A. The protocol for this scoping review was developed before the research began. However, this review was not registered in any specific database or repository.

The first search consisted of targeting any existing systematic reviews similar to the object of study. The second search was about existing frameworks for evaluating participatory processes. Finally, the last three searches consisted of identifying the various criteria and indicators defined as important for the democratic quality of participatory processes.

In order to filter out only the most relevant data and to limit bias regarding the object of study (participatory processes), we drew up a list of exclusion criteria, shown in Figure 1. During the selection stage by title and abstract, we excluded those that did not correspond to the objective of the review and those whose results did not concern a public policy process, excluding, for example, the many results concerning participatory processes with a purely research focus. During the eligibility stage based on full-text reading, in addition to the above criteria, we excluded articles that did not concern public participation with lay citizens and those that focused only on the impacts of participation and not on the quality of the process, as well as case studies that did not contain the population sample, the location of the case study, the names of the actors who initiated the process studied, or the method used to validate the tools. To avoid sampling bias, we included participatory processes of widely differing types, such as citizen juries, citizen reviews and participatory budgets. These filters enabled us to select only case studies with robust scientific methodology. The data extraction method was carried out freely by reading all the remaining articles.

We chose to include varied studies (single case studies, comparisons and international comparisons) to identify a wide range of democratic quality indicators. This methodological diversity allows us to explore different scales of analysis while highlighting differences and similarities between various contexts. It thus reflects the complexity of participatory processes and supports our inductive approach, aiming to comprehensively catalog the criteria of democratic quality.

Below are two examples of search strategies, each based on a different database. The first involves searching Web of Sciences’ SCI and SCII-Expanded to target the main evaluation indicators for participatory processes used in public policy cases. The keywords used in the search were (TITLE-ABS-KEY (participatory AND process) AND TITLE-ABS-KEY (indicator) AND TITLE-ABS-KEY (public AND policies)). The second example concerns a search on Scopus, carried out to identify evaluation frameworks. The keywords used were (TIILE-ABS-KEY (framework AND evaluation AND participatory AND democracy)).

Dimensions, criteria, indicators, key areas and explanatory factors are key concepts for evaluating citizen participation processes. Within our research and based on the extracted data, we define dimensions as the aspects to be evaluated. Criteria are the standards for judging the effectiveness of participation. Indicators allow measurement of the criteria or dimensions. Key areas are the essential components for the functioning of the process. Explanatory factors influence the quality of participation. We will seek to determine which ones appear the most and how they fit together.

3. Results

After filtering the 650 results obtained, according to the established criteria, we arrived at a total of 12 case studies, 0 evaluation frameworks and 0 systematic reviews (Figure 1).

The selected case studies were published between 2007 and 2021 and carried out in 13 countries on 4 continents: Canada, the USA, Australia, the UK, Finland, Italy, Spain, Malta, France, Germany, Belgium, Brazil and Japan. All the characteristics of the studies included are summarized in

Table 1. Studies and their characteristics.

Researchers	Setting	Scope of Application	Sample P = People	Methodology	Case Study
(Conrad & Cassar & Christie & Fazey, 2011) [15]	Malta, National	Governmental authorities, Malta Environment and Planning Authority, Territory and environment management	n = 45 p	Qualitative. Interviews with two groups: 1. public; 2. professional planners and/or policymakers	Workshop to assess, for the island of Malta, (i) expectations of participation processes; (ii) the extent to which practices meet expectations; (iii) ways in which participation practices could be rendered more effective
(Galais & Navarro & Fontcuberta, 2013) [16]	Andalusia municipalities—Spain	Local administrations, 120 Andalusian municipalities’ local participatory processes	n = 156 participatory processes; people without data: 3%; <10: 10%; 10–24: 24%; 25–49: 24%; 50–99: 12%; 100–299: 10%; 300–499: 5%; 500–1000: 5%; 1000>: 7%	Quantitative. Survey carried out during the autumn of 2009 among technicians from some 120 Andalusian municipalities	Made the link between the quality of participation and the context in which the process takes place
(Garcia, 2017) [17]	Sevilla, Ferrol, Novelda and Torreperogil, Spain; France; the UK; Germany; Italy; Brazil /!\ Each city is not listed	Municipal authorities of each country, participatory budget. IESA-CSIC in the project: Democracia, participación y espacio político: Un estudio comparado	n = 1881 p 17 experiences	Quantitative. Subjective survey	Understood the important points that satisfy citizens for a participatory budget
(Holden, 2011) [18]	Vancouver, Canada	Regional administration, Regional Vancouver Urban Observatory, Common Agenda in Urban Governance	n = ~150 p	Qualitative. Study-circle method	Questioning a Common Agenda in Urban Governance and the capacity of public participation to produce local sustainability indicators
(Horlick- Jones & Rowe & Walls, 2007) [19]	United Kingdom—Britain, National	Governmental authorities, GM (genetically modified) Britain debate	n = 752 p	Quantitative: survey; qualitative: focus groups	New criterion for quality of participatory democracy: Translation quality
(Knobloch & Gastil & Feller & Richards, 2014) [20]	Oregon State, United States of America	Government authorities, Citizens’ Initiative Review Commission from the governor of Oregon Education and Casino	n = 2 × 24 p for panel and broad ballot for voting	Qualitative: direct observation and panelist interviews; quantitative: panelists were addressed each day using a Lickert scale survey + large- sample, statewide telephone survey of 800 likely Oregon voters	Oregon Citizens’ Initiative Review (CIR), a mix between Citizen’s jury and deliberative poll
(Mah N-Y & Cheung M-W & Wai Yin Lam & Siu & Sone & Li, 2021) [21]	National Process—Japan	Governmental authorities, Government of Japan, A national deliberative pool on energy theme	n = 285 p deliberation = 6849 p poll	A “trust-based systems” framework of deliberative policy-making has been designed to examine and conceptualize the quality of such policy-making processes. Triangulation of methods. Qualitative: observation data and focus groups; quantitative: surveys	Deliberative pool on energy held in Japan in 2012
(Michels & Binnema, 2018) [22]	Belgium: 1. Leuven; 2. Brussels; 3. Arlon; 4. Brussels; 5. Namur; 6. Ghent; 7. Brussels	Process held on the platform G1000, citizen platform inspired by researchers, among others David van Reyburk: 1. King Baudouin Foundation -National Institute for Health and Disability Insurance—National 2. Particitiz—Brussels Government—Regional 3. University of Liège—Province of Luxembourg—Local 4. Particitiz—Brussels Parliament—Regional 5. Destrée Institute—Walloon Parliament—Regional 6. City of Ghent—City of Ghent—Local 7. Particitiz—Municipality of Molenbeek-Saint-Jean—Local	n = 1. 32 p; 2. 55 p; 3. 33 p; 4. 38 p; 5. 30 p; 6. 150 p; 7. 112 p	Qualitative: Analysis of 7 mini-publics of G1000 based on reports from the local G1000 organizers; observations and semi-structured interviews with participants; interviews with G1000 organizers, municipal councilors, aldermen and civil servants; content analysis of the political agendas of the council, newspaper articles, newsletters, websites and documents published by the local G1000 organizers; quantitative: a digital survey among participants	Fundamental needs and tensions between high-quality deliberation and voting during a participatory process (G1000 mini-publics)
(Molster & Maxwell & Youngs & Kyne & Hope & Dawkins & O’ Leary, 2011) [23]	Perth, Western Australia	Governmental authorities, The Office of Population Health Genomics (OPHG), Biobanking	n = 15–20 p	Designed a framework using the theoretical literature and analyzed it with a practical case study. Quantitative: survey	Deliberative forum for 4 full days
(Parés & Brugué & Espluga & Miralles & Ballester, 2015) [24]	Catalonia, Spain	Regional authorities, Catalan Water Agency (from the Catalan government)	n = +1600 p	Qualitative: textual analysis of the materials produced during the process; quantitative: interviews conducted with social and institutional stakeholders who actively participated	Implementation of the WFD
(Rondinella, Segre, Zola, 2015) [25]	National: Canada, Australia, the USA, the UK and Italy	Five governmental authorities: 1. Canada, Atkinson Charitable Foundation; 2. Australia, Australian Bureau of Statistics; 3. the USA, Government Accountability Office (GAO) in partnership with the National Academy of Sciences (NAS); 4. the UK, the British Prime Minister and the National Statistician; 5. Italy, Italian National Institute of Statistics (Istat) and the National Council for Economics and Labour Process initiated by a private agency: Italy, Sbilanciamoci!	n = 1. 800 p; 2. online national survey (https://base.socioeco.org/docs/measures_of_australia_s_progress_consultation_report.pdf “URL (accessed on 16 March 2023)”); 3. idem; 4. 34.000 p; 5. +2.500 p; 6. +40 p	Analyses and comparison of six similar processes using Archon Fung’s analytical framework; quantitative: submitting questionnaires to involved practitioners	Analyses, through the lens of Archon Fung’ s analytical framework, on how the conditions for granting legitimacy have been addressed
(Rosenström & Kyllönen, 2007) [26]	Helsinki, Finland	Governmental authorities, Finnish sustainable Development indicator (SDI)	n = 49 p	Qualitative. Based on written records, written comments and a study on indicator use	Analysis of the participatory process used to create a sustainable development index (SDI) to assess participation and to assess the quality of the criteria

.

Through this scoping review, we identified a total of 12 dimensions, 30 criteria, 70 indicators, 17 key areas and 18 explanatory factors, all presented in Figure 2.

One complexity of this field is that, according to the authors, the same concept can be named in several ways. As such, a key concept described as a dimension in one article may appear as a criterion in another or as an indicator in the next. To provide more clarity in describing the results, we designed a concept matrix (

Table 2. Concept matrix.

Authors/Concept (by Occurrence)	Conrad et al., 2011 [15]	Galais et al., 2013 [16]	Garcia, 2017 [17]	Horlick- Jones et al., 2007 [19]	Holden (2011) [18]	Knobloch et al., 2014 [20]	Mah et al., 2021 [21]	Michels & Binnema, 2018 [22]	Molster et al., 2011 [23]	Parés et al., 2015 [24]	Rondinella et al., 2015 [25]	Rosenström & Kyllönen, 2007 [26]
Accountability/Commitment			Indicator				Indicator	Criterion	Criterion	Explanatory factor	Explanatory factor
Comprehensiveness			Indicator	Criterion	Key Area	Indicator						Criterion
Influence	Key Area	Dimension	Dimension					Key Area				Criterion
Results/Impacts		Dimension				Criterion	Indicator			Criterion		Criterion
Deliberation		Dimension						Dimension	Indicator	Criterion	Explanatory factor
Inclusiveness		Dimension				Indicator	Indicator	Criterion	Indicator
Representativeness							Indicator	Criterion	Criterion			Criterion
Context/Culture	Key Area		Dimension				Explanatory factor		Explanatory factor	Explanatory factor
Transparency	Key Area		Indicator				Indicator					Criterion

) where each one appears clearly according to the authors’ terminology. The differences will be explained concept by concept below.

As shown in the following figure (Figure 3), some key concepts stand out more than others across the articles.

To organize the description of the key concepts, we opted to sort them by denomination and by similar definition. In doing so, when two terms are not equal (e.g., accountability and commitment) but share similar usage across the review, we group and explain them in detail in order to enable a smooth summarization of the results without interpretation. Since the key concepts share intertwined variables, certain notions that may be used several times are explained only once in this section. Each of the nine key concepts listed in Table 2 is explained below. The concepts “accountability/commitment”, “comprehensiveness” and “influence of stakeholders” appear in 6 of the 12 articles included in the review; the concepts “results/impacts”, “deliberation”, “inclusiveness of perspectives”, “representativeness of participants” and “context/culture” appear in 5 of the articles; and the concept “transparency” is present in 4 of them.

3.1. Accountability/Commitment

The concept “accountability” is described by Michels and Binnema [22] as one of the determining criteria for the political impact of a process and is primarily dependent on the number of citizens who have access to deliberation. Indeed, depending on the influence that citizens have in policy-making, responsibilities should be shared between politicians and citizens, ensuring greater decision-making legitimacy. The number of participants reinforces the justification for sharing responsibilities. According to Molster et al. [23], “accountability” is a criterion of democratic quality that depends on real-time sharing of information related to the process, both on deliberation and on the results and their impact on political decisions, including the reasons why policymakers have included or excluded the deliberants’ recommendations in policy. It also includes the possibility of revisiting contested political issues. According to Rondinella et al. [25], it is a key area. Processes should have tools that enhance the accountability of sponsors engaging in participatory processes in order to not disengage citizens in the future when they perceive a lack of legitimacy in the decisions made. “Commitment” is a concept closely related to “accountability”, which is why we have included these two concepts in the description. Mah et al. [21] describe “credible political commitment” as an indicator necessary for the “trust in the motives” people attribute to the sponsors. These people should honor their agreements and be consistent in their political decisions. Parés et al. [24] emphasize commitment as an explanatory factor for the impacts of the process, noting its importance because politicians support the process externally, which allows for the inclusion of more contentious issues on the agenda through the involvement of key actors, thereby facilitating more meaningful deliberation. In this regard, Holden [18] presents similar findings concerning the openness to contentious issues. Conversely, Garcia [17] identifies the “perception of the degree of political commitment” as an explanatory factor for citizen influence. Lastly, the commitment of sponsors through the assurance of decision implementation appears crucial for the effective functioning of a participatory process.

3.2. Comprehensiveness

Mah et al. [21] identify “comprehensiveness” as an indicator measuring trust in information. They emphasize that the government should provide all relevant information pertaining to the subject matter. Garcia [17] describes “ease of understanding” as an indicator measuring the dimension of “horizontality” in a process. To that end, the availability of information has to be valued. Knobloch et al. [20] view it as an indicator measuring the criterion of “facilitation”, since it involved the role of facilitators to rephrase the transmission of information to participants. Holden [18] focuses on mutual understanding of interactions between participants, by balancing rational communication and acceptance of contentious subjects. Rosenström and Kyllönen [26] discuss the “degree of awareness and knowledge achieved”. They describe this quality criterion as the level of awareness about the issues and perspectives of different stakeholders generated through the process and debate, including experts and interest groups. They add that optimizing consensus requires participants to be in an equal knowledge position to reach a conclusion. Horlick-Jones et al. [19] devoted their investigation to the concept of “information translation.” This encompasses the effectiveness of gathering and transmitting information to participants and, thus, its general comprehensibility. The distinction of information “translation” lies in its concrete formatting, how it is interpreted, processed and then applied through implemented public policies.

3.3. Influence of Stakeholders

Rosenström and Kyllönen [26] consider influence as a quality participation criterion, describing it as the extent to which the program and mandate for participation supported the objectives of the participants. This relates to the fairness and credibility of the process, ensuring substantive issues are not omitted from discussion. Galais et al. [16] view it as a dimension evaluable through three indicators: “phases of participation”, denoting when citizens have input; “degree of decision”; and “adoption feedback process”. Conrad et al. [15] note that planners tend to believe that public influence can be limited by ignorance, attitudes and motivations, while public faith in science declines in “risk” and “knowledge” societies, creating demand for public engagement in political decision-making. Their findings suggest that a balance may be struck when citizen participation is appropriately integrated into participation process phases. To ensure proper functioning, participation goals and scope should be clearly defined at the outset. As noted earlier, Michels and Binnema [22] consider that participants’ influence over policy-making depends on sponsors’ commitment, concurring with Garcia’s study [17] assessing influence via the number of approved proposals executed. Mah et al. [22] propose evaluating the participants’ “trust in motives” of the government’s “perceived policy outcomes,” i.e., whether participants feel their considered views could impact policy, mirroring other authors’ framing of influence.

3.4. Impacts

Parés et al. [24] evaluated “tangible results”, which include the conclusions and direct effects of the process. That is, to what extent are the proposals made accepted and, in that case, are they innovative or projects already planned by the organizers. They also evaluated “intangible results”, which relate to policy legitimacy, achieving a position of public interest, mutual recognition between stakeholders and social learning produced by the process. Michels and Binnema [22] evaluated political impact via the actual influence of deliberations on enacted public policies. This involves sharing accountability, as described above, as well as the decision-making procedure. They emphasized that the outcomes of deliberation should imperatively be debated and voted on in a public referendum or parliament to obtain real citizen influence. Galais et al. [16], evaluating participatory budgeting results, distinguished between “political results”, which include incorporating new perspectives emerging from citizen participation and improving municipal services, and “civil society relations results”, including enhancing relations with associations and the general public. They also stated that it is impossible to meet all the quality participation criteria in an optimal way and that it is necessary to choose beforehand which ones to prioritize. Rosenström and Kyllönen [26] stressed “legitimacy of the product” as a participation quality criterion, measured by the possible benefit to the decision process from participation, and whether that can be demonstrated (subsequent complaints and potential consensus). Their study results showed that intensive participation by experts and civil servants enhanced outcome competence but resulted in a democratic involvement deficit, lack of social learning and very low product legitimacy. Knobloch et al. [20], in their case study, assessed the impact of the process by measuring the awareness and usefulness for the broader voting population reached through the report prepared in advance during the small-scale participatory deliberation.

3.5. Deliberation

In the study by Parés et al. [24], the communication strategy used by the consulting firm, as well as the rules and protocols, helped shift the stakeholders from distrust and the participants from suspicion towards an atmosphere of mutual trust. However, this was only possible because the participants exhibited behavior conducive to deliberation and civic virtues. In addition to a climate of trust, effective deliberation requires a public capable of producing constructive, reasoned and cooperative discussions. One explanatory factor is the level of deliberative culture shared among all participants. Molster et al. [23] describe the “deliberation mechanism” as an indicator measuring the quality criterion of “structure of process and procedures”. This includes small group deliberations followed by plenary sessions where collective decision-making occurs. The aim is to identify any persistent disagreements and ratify consensus in the absence of objections to an apparent collective position reformulated by the facilitator. The rules are developed by the deliberators themselves. Reports are drafted by the small deliberating groups and communicated by a self-designated member. The independence of external facilitators is verified at the start of the process. Internal facilitators receive guidance explaining their role and how to direct deliberation to create opportunities for participation from everyone and ensure the discussion stays on the main subject. Task definition, a criterion also similarly addressed by Rosenström and Kyllönen [26] helps orient and structure the deliberation process by specifying the goals and responsibilities of participants, as well as expectations for deliberators’ contribution to decision-making. Michels and Binnema [22] share similar findings on the dimension of “deliberation quality”, citing the importance of small groups, mutual respect, independent facilitators and consideration of diverse perspectives. A key area is that experts should play a role in providing and challenging balanced and factual information. Galais et al. [16] also emphasize the indicators “information quality” and “deliberative methods used” to measure deliberation quality. Rondinella et al. [25] consider deliberation as an explanatory factor enabling the shift from a communicative to a participatory process.

3.6. Inclusiveness of Perspectives

Michels and Binnema [22] consider “diverse perspectives” as a criterion for evaluating the quality of deliberation. In order to optimize the tension between process efficiency and inclusiveness, they propose opening the agenda to issues raised by participants only if they relate to the main topic being discussed. This allows all dimensions of the topic to be addressed without losing the deliberation in the meanders of vague and superficially addressed subjects. Knobloch et al. [20] see the “consideration of different views” as an indicator measuring the facilitation of the democratic process. In this case study, participants assumed that sponsors were defending a legitimate collective interest, so they invested the time of deliberation in effectiveness and promoting analytical rigor, which includes examining a range of alternatives, weighing their pros and cons, reflecting on their underlying values and ensuring a non-coercive process. Molster et al. [23] developed the aspect of “diversity of perspectives” by describing it as an equitable approach allowing citizens to debate together and make collective decisions. The aim is to gather information on citizens’ considered opinions, common values and acceptable compromises in public interest, as well as to influence policymakers to take into account the outcomes of deliberation. Mah et al. [21] stress it as an indicator of “trust in motives”, focusing on the fact that the government should listen to opposing arguments to earn more legitimacy. While most authors refer to inclusion with respect to participant arguments, Galais et al. [16] determine its measurement indicators according to the number of participants (open process) and the diversity of stakeholders (group process).

3.7. Representativeness of Participants

Rosenström and Kyllönen [26] define “representativity of the participants” as a quality criterion of the process, referring to the extent to which participants were representative of all stakeholders with a potential interest in the policy. Michels and Binnema [22] discuss the criterion of “diversity of the population” as a component of deliberation quality. They explain that reimbursing voluntary participants is as important as paying politician and bureaucrat participants to ensure equality among them at all levels and sustain their engagement throughout the process. This principle is also supported by Molster et al. [23] in the name of access equity, which would additionally promote access to “hard to reach” populations. They advocate for discursive representation rather than demographic representation. It is determined through a selection process that emphasizes diversity of perspectives among deliberants. Rondinella et al. [25] consider it a key area to ensure the legitimacy of the process. The effectiveness of public representation seems to be more successfully executed by organized civil society rather than institutional representatives. Mah et al. [21] use it as an indicator measuring the “trust in motives” of the sponsors. Indeed, representativeness should enable deliberation to take into account the entire range of the population’s needs and demands.

3.8. Context/Culture

Galais et al. [16] establish that socio-economic and political characteristics, such as population size, economic level, election turnout and ruling party, do not influence the quality of participation. The existence of a participation council also has no effect. However, institutional resources and prior experience greatly impact the quality of the process. Indeed, the availability of external funding provides access to more human and material resources. It is also noted that processes inspired by previous examples tend to demonstrate good practices, such as implementing an institutional coordination plan, which enhances the administrative adaptability of the process within the institution’s operations. On the other hand, benefiting from an external consulting firm offers advantages, such as increased stakeholder mobilization, but does not improve the relationships between participants and the organizing institution.

In the case study by Mah et al. [21], key contextual factors were identified. In a context of distrust, general events play a major role in deliberation. This was evident in the role assigned to nuclear energy in Japan shortly after the Fukushima accident. With regard to contextual political conditions, the links between participants’ trust and the legitimacy attributed to policymakers can be influenced by the decision-making system in place. Another example is the pro-nuclear coalitions between bureaucrats and nuclear industrialists, which illustrate how strong lobbying power reinforces public distrust. The case study highlights the main points of attention to be measured in terms of the trust gap and their indicators. In the scoping review, we have already mentioned the requirements for transparency, comprehensiveness of information, political engagement, inclusivity of perspectives and representativeness. In a general context of distrust, “trust in information” also depends on perceived objectivity and reliability, when the government provides stakeholders with balanced information that includes a variety of perspectives rather than biased or partial information, or misinterpretation of information. Furthermore, “trust in motives” of participation depends on the openness of the process as well as perceived integrity, when the government is seen as honest, not hiding information about the risks of the presented public policy and not manipulating a participatory event for its own purposes. Finally, “trust in competence” depends on two indicators. The first is the ability to operate effectively, when the government is perceived as acquiring the necessary knowledge and skills for effective operations. The second is risk management capability, when the government is perceived as capable of handling risks related to the policy.

Conrad et al. [15] stress the importance of local participatory culture and social capital, highlighting the public’s capacity to act not merely as passive recipients of information but to engage actively as actors who prioritize collective interests over personal ones. The global contextual conditions in Malta benefit from a small territorial area. However, the high population density is prone to exacerbate land management issues. Moreover, the number of people involved in public participation is generally low and dominated by specific interest groups, leading to the marginalization of the general public. Finally, public participation takes place within a largely centralized administrative framework where local councils have little effective decision-making power.

Regarding the study by Molster et al. [23], significant investment in human and financial resources was made possible by the prevailing economic climate in the country at the time the process was designed. Additionally, the internal culture favorable to innovation, as well as the trusting relationship between external facilitators and policymakers, played a role in overcoming institutional obstacles related to designing an innovative process despite the lack of empirical studies. The deliberative culture was explained to all internal stakeholders to help them understand the principles of deliberative engagement with the public. However, the ongoing political mandate constrained the openness of the deliberation. Finally, the influence of the sponsoring institution was balanced by framing the information to ensure good-quality deliberation and by external checks on the independence of the internal facilitators.

Lastly, it is deemed essential that a global cultural shift be achieved to learn and internalize a deliberative approach to governance [19]. Deliberative processes will only succeed when politicians, public administration, stakeholders and the entire society adopt a genuine deliberative culture.

3.9. Transparency

Rosenström and Kyllönen [26] establish “transparency” as a criterion for assessing the quality of participation. They define it as the openness of the process, the accessibility of all background materials and objectives to participants, and an equitable starting point shared with the organizers. Mah et al. [21] emphasize it as an indicator for measuring the “trust in information” that participants may place in sponsors. The government should proactively provide information in a meaningful, accessible format, free of charge or at a reasonable cost. Conrad et al. [15] underscore the importance of transparency within the framework of any form of public engagement, explicitly delineating the reasons and methods for involving the public. Garcia [17] regards it as an indicator of the “symmetry” dimension, which pertains to the notion of information equity between participants and organizers.

4. Discussion

Through this review, we were able to highlight nine key concepts and analyze their characteristics. These nine key concepts are interdependent. Representativeness enables better inclusiveness, which allows for more in-depth deliberation. Transparency and facilitation of deliberation enable comprehensiveness and understanding of information. The commitment and accountability of sponsors enables greater credibility of the exercise and influence of participants, resulting in more meaningful direct and social results. Thus, the criteria mutually reinforce each other and must be considered together to ensure the overall quality of the participatory process. We can draw parallels between our findings and those of several authors who have previously developed evaluation frameworks. For example, in Rowe and Frever’s [5] framework, we find the criteria of “representativeness”, “influence” and “transparency”, which are identical to the concepts that we identified. Moreover, some of the criteria they identify demonstrate similarities with certain important variables that we did. This is the case with “early involvement” and “task definition” taken up by Rosenström and Kyllönen [26] and “structured decision-making” by Michels and Binnema [22], underlining the importance of the decision-making procedure. Almost all of the variables mentioned by Abelson et al. [4] are part of our results. This is the case with “degree of citizen control”, which aligns with the “influence” concept; “selection process of participant”, corresponding to “representativeness”; “legitimacy and accountability”, corresponding to “accountancy and commitment”; “deliberation”, sharing the same variables of “mutual respect” and “challenging expert information”; and “input into agenda setting”, being aligned with the concept of “inclusiveness”. The “information” indicator, which they related to “accessibility”, “readability”, “digestibility” and “selection and presentation”, correspond with the variables that we identified as part of the concepts of “transparency” and “comprehensiveness”.

The results could lead us to think that the importance of the concepts and their content could be weighted according to the number of times they appear. In this way, a hierarchy could be established, valuing the accountability/commitment, the comprehensiveness and the influence of the participants, and devaluing the concept of transparency. However, we have just seen that for each process, it is impossible to complete all criteria and that it is up to the participant to prioritize those that suit them best. So, their relative importance may depend more on the contextual position in which they are situated and on the needs of the participants as a whole. So, the weighting process is not so simple.

Indeed, more than identifying contexts, Webler and Juler [27] identified four perspectives due to the citizen’s needs: the perspective of the science-centered stakeholder consultation, which consists of consulting citizens to obtain information efficiently; the egalitarian deliberation, which emphasizes legitimacy and egalitarian links in relation to the sharing of power; efficient cooperation, which considers the authority as fair and making decisions in the common interest, where the participants work for it; and finally, informed collaboration, where the authority and the participants work together to evaluate which solution is the best. They demonstrate that the design quality of a process depends on universal variables, but that the weight of these variables, and even the mobilization of other variables, also depend on the needs and preferences of the participants. This is consistent with the findings of this research that, for example, a population that trusts its authorities will be more likely to emphasize an efficient cooperation perspective [20], where it will be assumed that sponsors are defending a legitimate collective interest and so the focus will be on the effectiveness of decision-making, whereas a population that places little trust in sponsors will prefer an egalitarian deliberation and informed collaborative perspective, where the effort will be focused on interactions and the structure, favoring the development of the legitimacy of decision-making that establishes the horizontality of decision-making as an indicator of quality [17]. The majority of participants tend to prefer a mix of these perspectives.

Therefore, before starting an evaluation, the contextual conditions in which the process takes place should be taken into account in order to, first, establish the limits imposed and to know how to adjust to them correctly; second, identify and collectively validate the needs and preferences of the stakeholders in order to calibrate the evaluation; and then third, take into account the key concepts in order to design the process according to the results expected in a shared manner and with previsualized indicators. In addition, participants’ preferences are continually evolving through social learning [13], so it is essential to take into account the need to readapt them to each process design [3] and not simply to the territory in which it is designed.

There are, however, some limitations to this review. Firstly, the review only included studies published in English, French and Spanish, so some relevant studies may have been excluded. Secondly, we did not include the grey literature, which contains numerous evaluation frameworks, as we chose to focus on data from rigorous academic studies. Thirdly, although the research is comprehensive within the selected databases, it is possible that some key studies (e.g., works by Bächtiger, Goldberg and Setälä) were not included in the analysis due to not meeting the research selection criteria or because of the specific indexing characteristics of the databases used.

This review is not intended to establish definitive conclusions about all the variables of the participatory processes assessed as it is a scoping review and the intention of the study has been to make a first approach to them. We sought to map the key concepts detected in the literature with the aim of being able to deepen them in future studies. However, we consider there to be sufficient accuracy and consistency of the results analyzed in this scoping review for it to be useful to guide future research.

5. Conclusions

In conclusion, we have conducted this scoping review in order to identify the key concepts for assessing the democratic quality of participatory processes and sought to determine how they are connected together. In doing so, we have detected, firstly, that in the current scientific literature, which includes our research criteria, there is no attempt at a quantitative weighting of democratic quality criteria. Secondly, we have identified and analyzed nine key concepts whose characteristics repeatedly emerge as important, or even essential. Thirdly, since the democratic quality of participatory processes depends not only on adjustable criteria but also on the context in which they evolve, we have identified the main contextual characteristics to be taken into account. Identifying these variables makes it possible to prevent their possible effects on the overall quality of the process by making the necessary adjustments from the outset. Finally, this scoping review provides a rigorous basis, which could be the starting point for developing a generalizable quantitative evaluation framework for democratic quality of the participatory processes.

Future research will use questionnaires and stakeholder interviews to identify, within the key concepts identified, which variables are needed to assess them quantitatively and to detect other important concepts that have not been reported in this review.