Rural areas in Europe represent outstanding examples of Cultural and Natural Heritage (CNH) that could be used as a valuable asset for social and economic development. This article describes the process for developing a monitoring platform based on Key Performance Indicators (KPI) and implemented in six rural areas around Europe. The goal of this monitoring system is to provide evidence of the role of CNH in rural areas as a driver for sustainable growth. Several data collection procedures are described, including regular, non-regular, and co-monitoring. To combine the selected cross-thematic and multi-scale KPIs, weights have been assigned to indicators, according to the knowledge provided by domain experts and using group decision-making techniques. A detailed description of the dashboards developed for the monitoring platform, and all the information gathered is included. Several dashboards have been designed focusing on KPI values and their evolution.

1 Introduction

Rural areas have been defined traditionally by what they lack, not by what they have, although it could be more empowering to capitalise the resources owned by a community rather than identifying all the resources the community does not have. According to Eurostat, 27.8% of the EU population lives in rural areas and 32% in urban-rural intermediate areas, representing over 80% of its territory. Around 46.5% of EU Gross Value Added is created in intermediate and predominantly rural areas. The Community Capital Framework (CCF) [5] offers a structure to consider and valorise the natural and cultural richness of rural areas as a first step to transforming these values into other capitals (human, social, built, and financial), since the accumulation of different forms of capital within a community is mutually self-reinforcing [7]. It also offers the possibility to capitalise intangible heritage and traditions, especially rich in rural areas. The richness of cognitive elements, or the way individuals think and behave, could be as important for the success of a territorial system as the material resources [3]. The rural identities shape the rural character of the intangible networks, norms, and behaviours, and these intangible resources tend to be more localized and immobile [24] and therefore better preserved in rural areas than in globalised urban environments.

This article describes a robust monitoring system developed to assess the effectiveness of an innovative rural regeneration paradigm based on Cultural and Natural Heritage (CNH), consolidating the role of culture and nature as the fourth pillar of sustainable development and contributing to economic growth, social inclusion, and environmental sustainability in rural areas. A new understanding of CNH as a peculiarity of European rural areas, turning a range of various cultural elements and relationships into a combination of factors driving the development and regeneration of rural areas is described in the RURITAGE paradigm [19]. In this line, the CCF considers that the growth of some forms of capital in a community is ready to create virtuous spirals of development [7]. This monitoring platform considers cultural (including intangible heritage), natural, built (mainly built cultural heritage), social (including political), human (people value and engagement), and financial capitals to measure the effectiveness of the actions and practices developed in a territory, acting as levers for change from the initial stock of capitals to other kinds of capital.

The OECD approach to the “social capital” [15] proposes four interpretations based on (i) personal relationships, (ii) social network support, (iii) civic engagement, and (iv) trust and cooperative norms and is one of the most empirically sound ways to estimate social capital. The OECD working paper argues that there is not one single interpretation of social capital but rather several different approaches, so the authors of this article decided to stick to the above-mentioned CCF, according to the research developed in the RURITAGE project.

The literature already considers nature capital and social capital as important competitive forces for rural areas [23], being key assets of rural areas [17]. In Reference [5], authors add to these two capitals the cultural capital as a key asset for rural areas, especially in the form of intangible cultural heritage, and aims to use the built cultural heritage as an asset within the infrastructure capital. Interest in cultural heritage and rural areas is growing and significant room still exists for the development of computational methods applied to solving real-world CNH problems in rural areas [1] and empirically based predictive models [18, 21].

There are several examples of Key Performance Indicators (KPI) monitoring and data-driven decision making, e.g., in Business Intelligence or in monitoring technical performance, such as in computer networks, but none on rural development apart from rural finance investment, such as the International Fund for Agricultural Development, although it is not related to heritage-led development.

The EU communication “A Long-Term Vision for the EU’s Rural Areas” [9] mentions the EU Rural Observatory, whose main objective is to further improve data collection and analysis on rural areas, although first results are not expected until the end of 2022. This observatory is intended to increase the quantity and quality of available data, as this is essential to understanding rural conditions and thus act on them in an adequate way.

The most similar examples found while searching the literature are the “Smart Rural 21” project [22], the “TExTOUR” project [25], and the “Cultural and Creative Cities Monitor” [8]. The first deals with the monitoring of development and the implementation of smart village approaches and strategies across Europe, but it is still under development, and no results have been published to date. The second aims to develop a novel approach to understanding and addressing cultural tourism and to promote the development of disadvantaged areas, identifying different layers of data and capitalising on existing practices but strongly oriented to tourism and Europeanisation. The third is designed to help national, regional, and municipal policy makers identify local strengths and opportunities and benchmark their cities against similar urban centres, using both quantitative and qualitative data. The Cultural and Creative Cities Monitor is thus an instrument to promote mutual exchange and learning between cities. For researchers, the pool of comparable data is expected to generate new questions and insights into the role of culture and creativity in cities’ social and economic well-being. To the authors’ best knowledge, there is no similar initiative for rural areas. In this article, the authors propose a way to build such a robust monitoring platform following the methodology developed in the frame of the RURITAGE project [19] and Figure 1. The novelty of the proposed methodology is the use of a computer-aided monitoring platform for assessing the impact of heritage-led rural regeneration actions.

Fig. 1.

This article is organised as follows: Section 2 describes the methodology used for the identification and selection of KPI. Section 3 explains how different indicators can be combined to obtain a global value, and Section 4 discusses the results. Finally, the main conclusions and further research are outlined in Section 5.

2 Key Performance Indicators and Monitoring

A monitoring system is a method to keep track of relevant parameters of the object under study. It is called robust if it is able of coping with errors during execution and end-users’ erroneous input. Within the scope of heritage in rural areas, let us consider the Monitoring Platform as a tool to measure some specific indicators related to competitiveness, growth, and sustainable and inclusive development driven by CNH. This wide range of Key Performance Indicators is cross-thematic and multi-scale, being related to environmental, social, cultural, and economic impact categories. Associated technical methodology and tools for KPI measuring have also been defined [4, 16].

2.1 Identification and Selection of Cultural and Natural Heritage Indicators for Rural Areas

KPI identification is the basis for developing an integrated evaluation procedure to measure the performance and impacts achieved through the implementation of the heritage-led regeneration plans. Identified KPIs are related to cultural, social, environmental, and economic impact categories, in direct relation to the six capitals from the CCF considered in this research.

The selection of the most suitable KPIs has been done following the Relevant, Accepted, Credible, Easy, and Robust (RACER) evaluation framework [6], developed for assessing the value of scientific tools for use in policy making. In this case, the RACER framework and its sub-criteria (see Table 1) have been adapted and simplified, leaving two sub-criteria within each RACER category, making a total of 10, as the basis for the evaluation framework. More than 200 indicators have been identified and evaluated, and 60 have finally been selected and grouped in six Community Capitals, as shown in Table 2.

Table 1.

RACER Criteria	Sub-Criteria	Description	Levels
Relevance	Meaningful	Is the indicator meaningful for the objectives?	high/mid/low
Relevance	Comparable	Is the indicator comparable across different cases?	yes/no
Accepted	Previously Used	Has the indicator been previously used?	yes/no
Accepted	Standard	Is it a “standard” indicator?	yes/no
Credible	Unambiguous	Are the results unambiguous?	yes/no
Credible	Clear Methodology	Has the indicator a clear methodology?	yes/no
Easy	Availability	Are the data easily available?	high/mid/low
Easy	Easy to Calculate	Is the indicator easy to calculate?	high/mid/low
Robust	Real Data	Does the indicator use real data or robust estimations?	real/estimations
Robust	Applicable to Similar Cases	Is it possible to apply the indicator in numerous (similar but different) cases? Has it been used in different circumstances and delivered reasonable results?	yes/no

Table 1. RACER Sub-criteria for KPI Evaluation

Table 2.

CCF	Code	KPI Description	Unit/Scale
Cultural	CC-01	No. of enterprises in the cultural sector	Integer
Cultural	CC-02	Increment in number of mentions of CNH in social media, media, press, etc.	Percent
Cultural	CC-03	Users registered in the digital hub or following the social networks (Facebook, Twitter...)	Integer
Cultural	CC-04	Posts in the RURITAGE digital hub	Integer
Cultural	CC-05	Posts mentioning RURITAGE at local level	Integer
Cultural	CC-06a	Actions and cultural events produced by citizens at local level	Integer
Cultural	CC-06b	People reached by actions and cultural events produced by citizens at local level	Integer
Cultural	CC-07	Crowdfunding campaigns launched	Integer
Cultural	CC-08	People trained (traditional skills, etc.)	Integer
Cultural	CC-09	Places involved in the tourism offer	Integer
Cultural	CC-10	Total no. of arrivals of tourist in the last year	Integer
Natural	NC-01	No. of ecosystem services	Integer
Natural	NC-02	No. of designations	Integer
Natural	NC-03	Area of designations	sqkm
Natural	NC-04	Emission of greenhouse gases	kg \(CO_2\) eq.
Natural	NC-05	Share of renewable energy in gross final energy consumption	Percent
Natural	NC-06	Companies and organizations with sustainability certifications and labelling	Integer
Natural	NC-07	Shops, restaurants and tourism facilities selling local products (km0)	Integer
Natural	NC-08	No. of ‘green tourism packages’	Integer
Built	BC-01	No. of hotspots provided	Integer
Built	BC-02	People reached through RURITAGE digital tools	Integer
Built	BC-03	No. of CNH objects mapped trough RURITAGE ATLAS	Integer
Built	BC-04	No. of beds	Integer
Built	BC-05	No. of restaurants	Integer
Built	BC-06	Cycle paths	km
Built	BC-07	Pedestrian/hiking paths	km
Built	BC-08	Share of people served by public transport services	Percent
Built	BC-09	Shared transport services (bike sharing, car sharing, etc.)	Integer
Built	BC-10	Sites accessible by people with disabilities	Integer
Built	BC-11	Buildings restored/retrofitted	Integer
Built	BC-12	Reused buildings	Integer
Built	BC-13	Brands and labels granted for local products and services	Integer
Built	BC-14	Fairs and tourism events per year related to the promotion of the area and related products	Integer
Built	BC-15a	Sites provided with signals and explanation panels to help describing the sites and orienteering visitors	Integer
Built	BC-15b	Routes provided with signals and exp. panels to help describing the sites and orienteering visitors	km
Social	SC-01a	No. of citizens engagement activities	Integer
Social	SC-01b	Participants in citizens engagement activities	Integer
Social	SC-02	No. per type of stakeholder involved	Integer
Social	SC-03	No. of local associations involved	Integer
Social	SC-04	Participants in formal or informal voluntary activities or active citizenship	Integer
Social	SC-05a	Projects addressing migrants	Integer
Social	SC-05b	People involved in projects addressing migrants	Integer
Social	SC-06a	Projects addressing people with disabilities	Integer
Social	SC-06b	People involved in projects addressing people with disabilities	Integer
Social	SC-07	No. of disadvantaged people engaged (elderly, migrants, unemployed)	Integer
Human	HC-01	Level of education	Percent
Human	HC-02	Recreational facilities/events	Integer
Human	HC-03	Migrants involved in educational-training programs	Integer
Human	HC-04	Internship for migrants activated	Integer
Human	HC-05	No. of self-employees	Integer
Human	HC-06	Internship for students	Integer
Human	HC-07	People trained in IT and tourism	Integer
Human	HC-08	People involved in professional management training course (summer school, master)	Integer
Human	HC-09	Publication as recommendation and guidelines provided	Integer
Financial	FC-01	Nights spent at tourist accommodation establishments	Integer
Financial	FC-02	Year revenues per sector/municipality	Integer
Financial	FC-03	No. of PPPs set and signed	Integer
Financial	FC-04	Unemployment rate	Percent
Financial	FC-05	Start-ups and spin-off created/Birth of enterprises	Integer
Financial	FC-06	No. of companies supported in defining new business models and innovative processes of production	Integer

Table 2. Final List of Selected KPIs and Unit/Scale of Values, Grouped by CCF

The answers provided by domain experts to the criteria in Table 1 give a global RACER score. Every sub-criteria is assigned a value in the range \([0, 10]\) according to the different levels, e.g., Meaningful value could be \(\lbrace 10, 5, 0\rbrace\) depending on the {high, mid, low} level value. To select the final set of KPIs, the following criteria has been followed: (i) Only indicators with a high meaningful score have been selected; (ii) only comparable indicators have been selected; (iii) indicators with low data availability have been discarded; (iv) indicators not applicable to similar cases have been discarded; and (v) indicators with a RACER score lower than 50 (of 100) have been discarded.

3 Group Decision Making

Decision making is the cognitive process of selecting the best alternative (or alternatives) among multiple different ones. Decision making not only occurs for isolated individuals. Some have to be solved by a group of persons (usually experts). Then it is known as Group Decision Making (GDM), i.e., selecting the best alternative, or alternatives, from a finite set of feasible alternatives, considering the preferences of a group of experts (see Figure 2).

Fig. 2.

3.1 Global Performance Index

In a world of big data, where even rural environments are generating vast amounts of data, once the way to tap into the various data sources has been figured out, and the method to collect, process, and store them through the KPIs has been previously defined, the next step is data analysis. Monitoring and visualisation of data is considered a key practice to detect patterns and take action when identifying anomalous behaviour. This can provide the visibility required for understanding what is happening at a given point in time. A common procedure is to calculate a global value that summarises the data of the individual indicators. This is what we have called the Global Performance Index (GPI). Although the monitoring platform allows us to set specific weights for every indicator on a case-by-case basis, this article describes the general methodology for estimating the base values for the weights when no other specific criteria or constraints are available.

Problems arise when handling data from different sources, because there might be some undesirable effects, such as different units for the same measure or different ranges. To avoid these effects, it is necessary to employ such methods as data normalisation or standardisation to convert all data into a common format to allow proper comparison. Normalisation, for instance, is used to scale numeric values to a particular range, usually to the interval \([0, 1]\), also known as the “z-score” normalisation. Data harmonisation is based on a detailed description of the individual elements in the data coming from diverse sources.

When calculating the capital values, i.e., the global value for each Community Capital, and the GPI for each rural area, not every KPI has the same impact. The proposed way to obtain the weight of KPIs is by GDM. Opinions from six domain experts in the RURITAGE consortium, with different backgrounds and expertise, ranging from university professors to technologists, from Italy (University of Bologna), UK (University of Plymouth), Germany (ICLEI), and Spain (Tecnalia and CARTIF), have been collected and analysed. The experts have been weighted equally in this investigation, although different weights could be also agreed upon. Other methods could be used to cope with differences of opinion between experts, such as experts’ panels or other types of discussions, but the reason for using the method here described is to automate the process, independently of the number of participating experts.

The proposed way to estimate the KPI weights (see Figure 2) is to generate a model by applying a method based on the Analytic Hierarchy Process (AHP) [20] to the knowledge provided by domain experts. The objective is to shed light on what degree of importance each KPI has in its specific Community Capital. To do this, the weight that can be attributed to each KPI is estimated, based on the opinion or criteria of the group of experts. The evaluation method consists in making a ranking with the KPIs according to their importance for the Community Capital and assigning a score (see Table 3) according to its relative relevance in comparison to the next KPI in the ranking.

Table 3.

Score	Relevance
3	Much more important than...
2	More important than...
1	Slightly more important than...
0	Same importance as...

Table 3. KPI Relative Relevance by Successive Comparison

The first step is to assign a ranking, i.e., order of importance, to every indicator. Let n be the number of experts and m be the number of indicators in the set. This intermediate result is a permutation (\(A_i\)), or arrangement (order matters), of the initial indicators set, defined by an expert (\(E_j\)) according to the relevance (or criteria \(C_k\)) as in Table 3. No repetition is allowed at this point, but the relative relevance of an indicator in comparison to the next one in the list should be stated through the scores defined in Table 3. The next step is to sort the indicators according to the ranking previously stated. For every row, the sum of the cumulative scores (\(v_p\)) among the current indicator and the previous indicators is calculated according to Equation (1), as shown in Table 4,

\begin{equation} v_p = 1 + \sum _{k=p}^{m}{C_k}, \quad \forall p \in [1, m]. \end{equation}

(1)

Table 4.

\(A_i\)	Code	\(C_k\)	—	0	0	1	0	1	2	0	0	1	3	\(v_p\)	\(I_q\)
			Score
			CC-10	CC-09	CC-06b	CC-06a	CC-07	CC-01	CC-04	CC-02	CC-08	CC-03	CC-05
1	CC-10	—	0	0	0	1	1	2	4	4	4	5	8	9	0.13
2	CC-09	0		0	0	1	1	2	4	4	4	5	8	9	0.13
3	CC-06b	0			0	1	1	2	4	4	4	5	8	9	0.13
4	CC-06a	1				0	0	1	3	3	3	4	7	8	0.11
5	CC-07	0					0	1	3	3	3	4	7	8	0.11
6	CC-01	1						0	2	2	2	3	6	7	0.10
7	CC-04	2							0	0	0	1	4	5	0.07
8	CC-02	0								0	0	1	4	5	0.07
9	CC-08	0									0	1	4	5	0.07
10	CC-03	1										0	3	4	0.06
11	CC-05	3											0	1	0.01
														70	1.00

Table 4. AHP Results for Cultural Capitals Indicators According to Expert No 1 (\(E_1\))

The last step is to estimate the relative relevance, or Influence (\(I_q\)), of the indicators collecting the individual values assigned by every domain expert according to Equation (2) (see Table 5 for the case of Cultural Capital indicators). As a result, the influence of every indicator is obtained, expressed as a percentage, e.g., column E1 in Table 5 correspond to the results of “Expert No 1.” It is necessary to repeat this process with every expert’s scoring,

\begin{equation} I_q = \frac{v_p}{\sum _{p=1}^{m}{v_p}}, \quad \forall p \in [1, m]. \end{equation}

(2)

Table 5.

Code	Description	Influence (I)						Average
Code	Description	E1	E2	E3	E4	E5	E6	Average
CC-01	No. of enterprises in the cultural sector	0.10	0.16	0.15	0.16	0.17	0.16	0.15
CC-02	Increment in no. of mentions of CNH in social media, media, press, etc.	0.07	0.13	0.10	0.03	0.07	0.05	0.08
CC-03	Users registered in the digital hub or following the social networks	0.06	0.06	0.09	0.03	0.02	0.01	0.05
CC-04	Posts in the RURITAGE digital hub	0.07	0.02	0.01	0.03	0.02	0.01	0.03
CC-05	Posts mentioning RURITAGE at local level	0.01	0.01	0.05	0.03	0.02	0.01	0.02
CC-06a	Actions and cultural events produced by citizens at local level	0.11	0.13	0.16	0.23	0.17	0.13	0.15
CC-06b	People reached by actions and cultural events (CC-06a)	0.13	0.12	0.16	0.19	0.15	0.14	0.15
CC-07	Crowdfunding campaigns launched	0.11	0.04	0.03	0.13	0.07	0.08	0.08
CC-08	People trained (traditional skills, etc.)	0.07	0.14	0.13	0.06	0.10	0.16	0.11
CC-09	Places involved in the tourism offer	0.13	0.08	0.08	0.06	0.10	0.11	0.09
CC-10	Total no. of arrivals of tourist in the last year	0.13	0.10	0.04	0.03	0.12	0.13	0.09
								1.00

Table 5. Indicators Influence According to Domain Experts’ Scores, e.g., on Cultural Capital Indicators

3.2 Balancing Differences among Expert Opinions with OWA

In group decision-making processes, and usually in the presence of conflicting goals, the idea of tradeoffs corresponds to viewing the global evaluation of an action as lying within the worst and best ratings. Ordered Weighted Averaging (OWA) operators [26] can realize tradeoffs between objectives by allowing a positive compensation between ratings, i.e., a higher degree of satisfaction of one of the criteria can compensate for a lower degree of satisfaction of another criterion to a certain extent [11].

An OWA operator of dimension n is a mapping function F that has an associated vector \(w=(w_1, w_2, \ldots , w_n)^T\) such as \(w_i \in [0, 1], 1 \le i \le n\), and

\begin{equation} \sum _{i=1}^{n}{w_i} = w_1 + \cdots + w_n = 1. \end{equation}

(3)

Furthermore

\begin{equation} F(I_1, \ldots , I_n) = \sum _{j=1}^{n}{w_jb_j} = {w_1}{b_1} + \cdots + {w_n}{b_n}, \end{equation}

(4)

where \(b_j\) is the jth largest element of the bag \(\langle I_1, \ldots , I_n\rangle\). It should be noted that different OWA operators are distinguished by their weighting function. Then the weights can compensate for the best and worst scores of an alternative. Oring the criteria means full compensation, while anding the criteria means no compensation. The measure of orness associated with any vector w is used to classify OWA operators with regard to their location between and and or, see Equation (5) and Equation (6),

\begin{equation} orness(w) = \frac{1}{n-1}\sum _{i=1}^{n}{(n-i)w_i}, \end{equation}

(5)

\begin{equation} andness(w) := 1 - orness(w). \end{equation}

(6)

Another OWA feature is the measure of “dispersion” of a weighting vector w, which defines how uniformly the \(w_i\) are used. An important application of the OWA operators is in the area of quantifier guided aggregations \(Q()\). The weights associated with this quantified guided aggregation are obtained as follows:

\begin{equation} w_i = Q\left(\frac{i}{n}\right) - Q\left(\frac{i-1}{n}\right), \quad i=1, \ldots , n. \end{equation}

(7)

In this study, every expert has provided an influence I value for the indicators (see Table 5). In some cases, e.g., CC-01, all the experts more or less agree about the I value assigned to the indicator. In other cases, e.g., CC-02, there are significant differences among the experts’ evaluations, for instance between \(E_2\) and \(E_4\). An OWA operator is used to aggregate all the answers. Thus, an OWA-modified weight is assigned to each indicator, where higher values means that most of the experts consider the KPI is among the most important, while lower values means that experts consider the KPI is not so important.

Table 6 shows the relevance of each KPI, taking into account the different criteria expressed by the experts. Specifically, in this analysis, the OWA operator uses the RIM (Regular Increasing Monotone) quantifier, shown in Equation (9), with the weights associated with this quantified guided aggregation obtained from Equation (10), which also defines the dispersion. The highest result indicates the most important KPI, taking into account the specific criteria. The results are also illustrated by Figure 3. It is thus possible to see at a glance which are the most and also the least relevant indicators,

\begin{equation} \alpha = \frac{1 - orness}{orness}, \end{equation}

(8)

\begin{equation} Q_\alpha (r) = r^\alpha , \end{equation}

(9)

\begin{equation} w_i = Q\left(\frac{i}{n}\right) - Q\left(\frac{i-1}{n}\right) = \left(\frac{i}{n} \right)^{\alpha } - \left(\frac{i-1}{n}\right)^\alpha . \end{equation}

(10)

Fig. 3.

Table 6.

Code	\(Orness=0.1\)		\(Orness=0.4\)		\(Orness=0.5\)		\(Orness=0.6\)		\(Orness=0.9\)		Relevance
Code	Weight	Ranking	Weight	Ranking	Weight	Ranking	Weight	Ranking	Weight	Ranking	Relevance
CC-01	17.44%	3	16.03%	1	15.01%	2	14.05%	3	11.89%	3	high
CC-02	5.79%	6	7.26%	8	7.65%	8	8.00%	8	8.74%	7	med
CC-03	2.23%	9	4.03%	9	4.58%	9	5.05%	9	6.06%	9	low
CC-04	1.75%	10	2.25%	10	2.69%	10	3.17%	10	4.46%	10	low
CC-05	1.54%	11	1.99%	11	2.28%	11	2.58%	11	3.31%	11	low
CC-06a	18.66%	2	15.79%	2	15.42%	1	15.22%	1	15.13%	1	high
CC-06b	19.98%	1	15.74%	3	14.90%	3	14.28%	2	13.20%	2	high
CC-07	5.49%	8	7.27%	7	7.71%	7	8.04%	7	8.57%	8	med
CC-08	10.65%	5	11.06%	4	11.15%	4	11.17%	4	11.09%	4	med
CC-09	10.79%	4	9.79%	5	9.53%	5	9.29%	5	8.76%	6	med
CC-10	5.68%	7	8.79%	6	9.08%	6	9.14%	6	8.79%	5	med

Table 6. Sensitivity Analysis of the Orness Effect on Cultural Capital Indicators’ Weights and Rankings

The sensitivity analysis illustrated in Table 6 shows the effect of the orness parameter in the weights and rankings, i.e., setting the relevance of the indicators. Higher orness values give more importance to the highest weights in a more conservative approach, while lower values promote the lowest weights, trying to soften the discrepancy among experts. According to this, \(orness = 0.5\) gives the same importance to all the values, so it produces an arithmetic mean, as shown in the last column in Table 5. In this case, the sensitivity analysis also shows that some indicators always have either high or low ranking positions, despite the orness value; so it is possible to group the KPIs in three sets {high, medium, low} according to their relevance (last column in Table 6). See the full table description for every KPI in Appendix A (Tables A.1–A.5).

4 Results

The monitoring platform described in this article has been implemented in six rural areas (Rs) around Europe (Figure 4) and provides from global performance values, i.e., GPI, to detailed KPI data. The chosen tools were MongoDB [14] as NoSQL database software and Grafana [12] to build the dashboards, since it is an open source metric analytic and visualisation suite most commonly used for visualising time-series data and able to work with multiple data stores. It supports many different storage backends for time-series data (data source). Each data source has a specific query editor customised for the features and capabilities that the particular data source sets out. Grafana also allows data from multiple data sources to be combined on a single dashboard. Some additional functionalities have been developed using Flask [10], python, javascript, and Bootstrap [2].

Fig. 4.

4.1 Setting the Baseline

A “baseline” is an established state by which something is measured or compared. Therefore, in any project oriented toward evaluating the impact of some actions or interventions, it is necessary to know the starting situation against which to monitor the results obtained.

The baseline of the rural areas taking part in this study establishes the starting point for monitoring on the diagnosis of their current situation. It is the first measurement of all the key performance indicators, both letting the values of these indicators be known before the execution of any heritage-led regeneration actions and easing the comparison between the said indicators after the execution of these regeneration actions.

4.2 Data Collection

The main data sources were the local authorities or other stakeholders in the rural areas through surveys and questionnaires, complemented with alternative data sources such as official statistics. Google Trends have been used to analyse the popularity of top search queries in Google across various topics on the pilots. Data collection and KPI calculation lasted 2 years, from December 2019 to December 2021. Throughout this time, a full set of data was collected through data collection campaigns every 6 months, so as to ensure a proper supervision and analysis. Once validated, the data were included in the database.

4.3 The Dashboards

A dashboard is a common type of data visualisation that provides at-a-glance views of KPIs relevant to a particular objective. A set of detailed dashboards has been developed for the monitoring platform to show all the gathered information. Two of them focus on KPI values and their evolution, while the other two focus on Community Capitals and their evolution over time. Figure 5 shows the landing page, i.e., the welcome page when the user gets into the platform. It contains some basic instructions and links to the main functionalities of the tool.

Fig. 5.

The global performance index is represented by a gauge chart, as shown in Figure 6. Small-gauge figures represent the global values for every single Community Capital. The combination of these individual values produces the GPI value, as explained in Section 3.1. Going deeper into the details, Figure 7 illustrates the same values in the form of a radar chart (corresponding to rural area No.2 in Norway, R2). It is thus also possible to represent the values of other rural areas under study.

Fig. 6.

Fig. 7.

From the radar charts in Figures 7 and 8, it is possible to get an idea, at a glance, of how analysed rural regions are performing on each Capital according to the values of the KPIs. On the one hand, the graphs show that some regions are already performing well in some of the indicators; but, on the other hand, there is still room for improvement. The interesting part for these KPIs is to see their evolution over time. The figures also show that selected regions are well balanced, because where one has a high score, others do not. This means that different starting points have been taken into account and a monitoring system could help regions to learn from each other to improve their results and overall situation, helping them to readjust activities and take decisions accordingly.

Fig. 8.

The Progress Report dashboard (see Figure 8) is composed by several sections. The first section shows the general information related to the Community Capitals and the Global Performance Indicator. The Detailed Information section shows KPI data by Community Capital. In every section, you can choose either to show or hide the table with the data displayed in the charts. Additional and complementary information is shown when the mouse pointer hovers over the charts and tables. Functionalities include checking and unchecking the “Show?” Checkbox column by KPI to set those specific indicators to be shown in the chart. By default, the platform shows the values for last available Activity/Event, but the user can change this and choose a specific event to show. There is also a checkbox “Show all” to see all the available events or the Monitoring Periods only.

The Action Plan is the collection of activities that the participating rural areas have developed aligned with the heritage-led regeneration strategy. This dashboard (see Figure 9) summarises only those KPIs related to a specific action. Additional and complementary information is shown when the mouse pointer hovers over the charts and tables.

Fig. 9.

The data management options (see Figure 10) allow the users to set the baseline, define the activities within the Action Plan, create the necessary data gathering campaigns for monitoring, and include any other activities or events related to the heritage-led regeneration plan.

Fig. 10.

4.4 Discussion

The KPI monitoring and assessment process leads to an objective evaluation of concrete heritage-led actions/policies in rural areas. Therefore, it is the basis upon which to build up scalable and replicable models of those areas with similar characteristics and common problems throughout Europe and beyond. On the one hand, they can be the typical cause–effect models on the frequent occasions when reality faces a number of limited and quantifiable indicators. On the other hand, rural areas can be considered as complex systems featured by a holistic approach; so a less formal type of model, e.g., using system dynamics, will allow a more structured view of the problem to be obtained, monitoring the most critical aspects, where charts and diagrams allow feedback loops and time delays that affect their behaviour over time to be determined. Currently, authors are working on a System Dynamics model, based on the data and findings of this research, and intended to provide the users with a tool to simulate possible what-if scenarios.

The methodology developed by the authors allows an initial set of indicators, as large as necessary, to be analysed. Then, via an objective framework such as RACER, it can be reduced to a manageable number, lowering the dimension of the problem. In this article, the selection criteria has been based on those KPIs that score higher than a threshold; but in other cases, the selection criteria could be according to a certain number of indicators, e.g., 20 or 30, with the highest scores. Nevertheless, the resulting set of KPIs could be extremely diverse and difficult to combine and compare, so group decision-making techniques have been introduced to reach a tradeoff among the experts in how to combine the data from the indicators and get meaningful KPIs. The monitoring platform shows the results obtained in an easy-to-use web application available for end-users as a Software as a Service, so no management is needed.

The feedback from the six rural areas that have been testing the functioning of the platform shows that a small set of indicators are not so informative as expected, e.g., “NC-04: Emissions of greenhouse gases” and “HC-01: Level of education”; while collecting the data for other indicators has been harder than expected, e.g., “CC-01: Number of enterprises in the cultural sector” and “FC-02: Year revenues per sector.” These insights will be used to make the selection of KPIs more flexible for new users of the monitoring platform, while keeping the ratio among the weights.

5 Conclusions

This article describes a CNH monitoring platform and evaluation scheme based on cross-thematic and multiscale KPIs. More than 200 indicators were initially identified and evaluated, and, finally, 60 were selected and grouped into six Community Capitals, providing quantifiable evidence of the potential role of CNH as a driver for sustainable growth on the basis of concrete actions.

Key performance indicators can be a valuable tool for establishing future rural strategies, as well as for evaluating development action plan impacts. However, nowadays, no standard has been developed for evaluating heritage-led practices in rural areas, and there is no broadly accepted indicator system that integrates the systemic innovation areas of RURITAGE as a framework to identify unique CNH potential within rural communities: pilgrimage, resilience, sustainable local food production, integrated landscape management, migration, and art and festivals. As a result, a tailored procedure has been established to define the KPIs that would take part in a reliable evaluation plan.

The monitoring platform shows from GPI to detailed KPI data through spider or radar charts and data tables. This would help local stakeholders and public authorities to make better informed decisions to definitively boost rural areas over CNH as primary resources.

The methodology here described can be further improved by relaxing the rules while selecting the KPIs in the monitoring platform but keeping the weights’ ratio. This functionality has been already included in the monitoring platform, but only for new users. Further research can be also performed regarding the development of advanced analysis of monitoring data. For instance, authors have started to draft a System Dynamics model allowing a more structured view of the most critical aspects of the problem and studying the relationship among the actions, their expected impacts, and the budget estimation.

Appendix

A OWA Detailed Results

Table A.1.

Code	\(Orness=0.1\)		\(Orness=0.4\)		\(Orness=0.5\)		\(Orness=0.6\)		\(Orness=0.9\)		Relevance
Code	Weight	Ranking	Weight	Ranking	Weight	Ranking	Weight	Ranking	Weight	Ranking	Relevance
NC-01	45.29%	1	26.36%	1	23.59%	1	21.49%	1	17.55%	1	high
NC-02	5.91%	6	9.40%	6	10.21%	6	10.80%	6	11.76%	5	low
NC-03	8.79%	5	13.54%	3	13.54%	3	13.39%	3	12.74%	3	med
NC-04	4.47%	8	7.67%	7	8.64%	7	9.33%	7	10.41%	7	low
NC-05	10.12%	3	11.83%	4	11.68%	4	11.46%	4	10.78%	6	med
NC-06	4.48%	7	5.46%	8	6.39%	8	7.40%	8	10.08%	8	low
NC-07	11.89%	2	15.52%	2	15.25%	2	14.99%	2	14.49%	2	high
NC-08	9.06%	4	10.21%	5	10.70%	5	11.15%	5	12.20%	4	med

Table A.1. Sensitivity Analysis of the Orness Effect on Natural Capital Indicators’ Weights and Rankings

Table A.2.

Code	\(Orness=0.1\)		\(Orness=0.4\)		\(Orness=0.5\)		\(Orness=0.6\)		\(Orness=0.9\)		Relevance
Code	Weight	Ranking	Weight	Ranking	Weight	Ranking	Weight	Ranking	Weight	Ranking	Relevance
BC-01	1.37%	15	2.88%	14	3.42%	14	3.87%	14	4.77%	14	low
BC-02	0.96%	16	1.23%	16	1.49%	16	1.78%	16	2.58%	15	low
BC-03	1.56%	14	1.90%	15	1.90%	15	1.89%	15	1.86%	16	low
BC-04	6.59%	6	5.64%	12	5.66%	12	5.75%	11	6.18%	11	low
BC-05	5.32%	11	5.00%	13	5.13%	13	5.29%	13	5.74%	12	low
BC-06	13.62%	2	9.56%	2	8.96%	2	8.50%	2	7.61%	5	high
BC-07	13.79%	1	10.02%	1	9.32%	1	8.77%	1	7.66%	4	high
BC-08	8.43%	4	8.07%	5	7.91%	5	7.74%	5	7.31%	7	high
BC-09	6.28%	7	5.83%	10	5.75%	11	5.69%	12	5.61%	13	med
BC-10	5.52%	10	6.58%	8	6.59%	8	6.52%	9	6.18%	10	med
BC-11	6.27%	8	7.00%	7	7.29%	7	7.46%	6	7.55%	6	med
BC-12	9.00%	3	8.67%	3	8.50%	3	8.29%	3	7.69%	3	high
BC-13	6.20%	9	6.02%	9	6.14%	10	6.28%	10	6.66%	8	med
BC-14	3.28%	13	5.76%	11	6.37%	9	6.92%	8	8.24%	1	med
BC-15a	8.05%	5	8.48%	4	8.28%	4	8.10%	4	7.74%	2	high
BC-15b	3.75%	12	7.37%	6	7.31%	6	7.15%	7	6.63%	9	med

Table A.2. Sensitivity Analysis of the Orness Effect on Built Capital Indicators’ Weights and Rankings

Table A.3.

Code	\(Orness=0.1\)		\(Orness=0.4\)		\(Orness=0.5\)		\(Orness=0.6\)		\(Orness=0.9\)		Relevance
Code	Weight	Ranking	Weight	Ranking	Weight	Ranking	Weight	Ranking	Weight	Ranking	Relevance
SC-01a	23.14%	2	17.25%	2	16.33%	2	15.62%	2	14.30%	2	high
SC-01b	23.18%	1	19.47%	1	19.15%	1	18.98%	1	18.80%	1	high
SC-02	6.48%	5	10.51%	3	10.99%	3	11.24%	3	11.46%	3	high
SC-03	11.70%	3	10.38%	4	10.09%	4	9.87%	4	9.44%	5	med
SC-04	5.27%	8	8.35%	6	8.70%	6	8.91%	6	9.12%	6	med
SC-05a	3.52%	10	4.95%	9	5.28%	9	5.52%	9	5.92%	9	low
SC-05b	5.79%	6	7.55%	7	7.46%	7	7.29%	8	6.79%	8	med
SC-06a	4.11%	9	4.80%	10	5.02%	10	5.25%	10	5.83%	10	low
SC-06b	5.48%	7	7.14%	8	7.38%	8	7.56%	7	7.91%	7	low
SC-07	11.33%	4	9.60%	5	9.61%	5	9.76%	5	10.43%	4	med

Table A.3. Sensitivity Analysis of the Orness Effect on Social Capital Indicators’ Weights and Rankings

Table A.4.

Code	\(Orness=0.1\)		\(Orness=0.4\)		\(Orness=0.5\)		\(Orness=0.6\)		\(Orness=0.9\)		Relevance
Code	Weight	Ranking	Weight	Ranking	Weight	Ranking	Weight	Ranking	Weight	Ranking	Relevance
HC-01	21.89%	1	19.52%	1	18.63%	1	18.02%	1	17.18%	1	high
HC-02	6.51%	8	7.70%	8	8.39%	8	8.95%	8	9.98%	7	low
HC-03	7.94%	6	9.52%	6	10.36%	5	11.05%	5	12.33%	3	med
HC-04	12.42%	4	11.75%	4	11.65%	4	11.59%	4	11.48%	4	med
HC-05	17.53%	2	14.24%	2	13.70%	2	13.28%	2	12.48%	2	high
HC-06	7.55%	7	10.37%	5	10.03%	6	9.72%	7	9.08%	8	low
HC-07	14.41%	3	13.57%	3	12.81%	3	12.12%	3	10.60%	6	high
HC-08	8.51%	5	9.09%	7	9.62%	7	10.02%	6	10.70%	5	med
HC-09	3.23%	9	4.24%	9	4.80%	9	5.26%	9	6.17%	9	low

Table A.4. Sensitivity Analysis of the Orness Effect on Human Capital Indicators’ Weights and Rankings

Table A.5.

Code	\(Orness=0.1\)		\(Orness=0.4\)		\(Orness=0.5\)		\(Orness=0.6\)		\(Orness=0.9\)		Relevance
Code	Weight	Ranking	Weight	Ranking	Weight	Ranking	Weight	Ranking	Weight	Ranking	Relevance
FC-01	7.66%	6	11.68%	5	12.63%	5	13.55%	5	15.93%	4	low
FC-02	17.75%	3	18.82%	3	18.19%	3	17.64%	3	16.44%	3	med
FC-03	10.26%	5	9.67%	6	10.27%	6	10.79%	6	11.79%	6	low
FC-04	26.07%	2	24.50%	1	24.23%	1	24.03%	1	23.68%	1	high
FC-05	26.12%	1	20.30%	2	19.27%	2	18.46%	2	16.90%	2	high
FC-06	12.14%	4	15.03%	4	15.41%	4	15.53%	4	15.26%	5	med

Table A.5. Sensitivity Analysis of the Orness Effect on Financial Capital Indicators’ Weights and Rankings

References

[1]

Francisco Barrientos, John Martin, Claudia De Luca, Simona Tondelli, Jaime Gómez-García-Bermejo, and Eduardo Zalama Casanova. 2021. Computational methods and rural Cultural & Natural Heritage: A review. J. Cult. Herit. (2021). DOI:

Abstract

1 Introduction

2 Key Performance Indicators and Monitoring

2.1 Identification and Selection of Cultural and Natural Heritage Indicators for Rural Areas

3 Group Decision Making

3.1 Global Performance Index

3.2 Balancing Differences among Expert Opinions with OWA

4 Results

4.1 Setting the Baseline

4.2 Data Collection

4.3 The Dashboards

4.4 Discussion

5 Conclusions

Appendix

A OWA Detailed Results

References

Cited By

Index Terms

Recommendations

Constructing a semantic wiki for living cultural heritage: case study of Korean tea culture

Exploring the Cultural Heritage of a Territory through the Cinematography Production

3D Models in Cultural Heritage: Approaches for Their Creation and Use

Comments

Information

Published In

Publisher

Publication History

Check for updates

Author Tags

Qualifiers

Funding Sources

Contributors

Other Metrics

Bibliometrics

Article Metrics

Other Metrics

Citations

Cited By

View options

PDF

eReader

HTML Format

Login options

Full Access

Figures

Other

Share

Share this Publication link

Share on social media

Affiliations