Utilities Policy xxx (xxxx) xxx

Contents lists available at ScienceDirect

Utilities Policy
journal homepage: www.elsevier.com/locate/jup

A composite indicator for measuring the environmental performance of

water, wastewater, and solid waste utilities
Anna Mergoni a, *, Giovanna D’Inverno a, Laura Carosi b
a
KU Leuven, Faculty of Business and Economics (FEB), Leuven, Belgium
b
University of Pisa, Department of Economics and Management, Pisa, Italy

A R T I C L E I N F O A B S T R A C T

Keywords: The aim of this paper is to measure and benchmark the environmental performance of Portuguese utilities jointly
Environmental sustainability active in the three sectors of water supply, wastewater collection and solid waste management. To do so, we
Environmental pressure indicator suggest the use of a traditional (optimistic) directional distance Benefit of the Doubt index. We complement the
Benefit of the doubt
analysis by considering also the pessimistic version of the proposed BoD and by implementing a robust and
Composite indicator
Robust and conditional analysis
conditional approach. The obtained results show that there is space for improvement in the pressure balance of
these utilities, especially for small and very large units, mostly operating in urban areas.

1. Introduction finding sufficient water sources and sustainable solutions to waste

disposal) by involving high level of energy usage, pollution emission,
Environmental sustainability is defined as the set of rules for the physical infrastructure and financial inputs. For this reason the litera­
“maintenance of the natural capital” (Goodland, 1995, p.10) or as “the ture about water and waste services sustainability is wide (see Walter
ability to maintain the qualities that are valued in the physical envi­ et al., 2009; Juwana et al., 2012; Simões and Marques, 2012; Wor­
ronment” (Sutton, 2004, p.1). Within this framework the environmental thington, 2014; Allesch and Brunner, 2014; Vilanova et al., 2015;
pressure indicators, i.e. the indicators which focus on the exchanges Margallo et al., 2015; da Silva et al., 2020; Zeller et al., 2020, and the
between the human activities and the environment, play a fundamental references therein).
role. The idea of controlling for the release of substances and for the use Nevertheless, it seems that there is a lack of consensus on a widely
of resources is intrinsic in the definition itself of environmental sus­ accepted method to assess the environmental sustainability of water
tainability and has been widely used in the literature (see for example: supply, collection and solid waste management services (Simões and
Moldan et al., 2012; Dahl, 2012; D’Amato et al., 2017; Purvis et al., Marques, 2012; Marques et al., 2015; Sala et al., 2015; Pèrez et al.,
2019). However, the term ‘environmental pressure indicator’, intro­ 2018). The major challenges can be attributed to the multidimension­
duced by Smeets and Weterings (1999), has not received equal attention ality of the phenomenon and regard, on the one hand, the choice of the
and there are only a few scholars who adopted explicitly this termi­ relevant indicators, on the other, the choice of a fair aggregating
nology: Nikolaou (2001); Munksgaard et al. (2005); Giannouli et al. method.
(2006); Geelen et al. (2009); Gonzàlez-Benito and Gonzàlez-Benito As for the former challenge, several authors suggest different
(2010); Liang et al. (2014). possible sets of indexes for mea-suring the environmental sustainability
The idea of a pressure indicator directly relates with the necessity of of the urban solid waste and water services. Despite the relevant dif­
measuring the pressures on the environment exerted by the activities ferences among the suggested approaches, all the proposed criteria can
involved in the economic and the social development. Nowadays, as the be interpreted as sub-indicators of ‘good’ or ‘bad’ pressures (Marques
population grows, climate change threats and economic activity spreads et al., 2015; Molinos-Senante et al., 2016; Pinto et al., 2017; Pèrez et al.,
irresponsibly, this necessity is even more compelling. To this purpose, 2018).
the monitoring of water and waste sector is fundamental (Lombardi As for the second challenge, i.e. the choice of a fair aggregating
et al., 2019; Degli Antoni and Marzetti, 2019; Das et al., 2019). The method, a wide variety of methodologies is available. Data Envelopment
water and waste sector responds to important social necessities (such as Analysis (DEA) and DEA-like approaches are among the most used ones

* Corresponding author.
E-mail addresses: anna.mergoni@kuleuven.be (A. Mergoni), giovanna.dinverno@kuleuven.be (G. D’Inverno), laura.carosi@unipi.it (L. Carosi).

https://doi.org/10.1016/j.jup.2021.101285
Received 18 March 2021; Received in revised form 11 August 2021; Accepted 11 August 2021
0957-1787/© 2021 Elsevier Ltd. All rights reserved.

Please cite this article as: Anna Mergoni, Utilities Policy, https://doi.org/10.1016/j.jup.2021.101285
A. Mergoni et al. Utilities Policy xxx (xxxx) xxx

(see Romano and Guerrini, 2011; Molinos-Senante et al., 2017; Marques interdependence among the three main areas. From an environmental
et al., 2018; Caldas et al., 2019, among others). perspective, the peculiarities of these sectors make Portugal an inter­
The Composite Indicator (hereafter CI) implemented in this paper esting laboratory for testing the suggested composite indicator.
belongs to this family and is based on the model by Zanella et al. (2015), First, these sectors are vulnerable. Portugal is prone to seasonality,
which Rogge et al. (2017) defined as a directional distance version of the with abundance of water in winter and scarcity in summer, especially in
Benefit of the Doubt (BoD) model. Such approach allows us to select the the south and it is suffering climate change, which is impacting the
benchmarking units in a completely data driven way, to evaluate the quality and the availability of surface and underground drinking water
utilities along desirable and undesirable dimensions and to ensure the sources, with serious consequences for the water provision (Serra et al.,
best possible rank to each unit. To obtain information about the weakest 2021; EurEau Association, 2021). Besides, the economic growth has
environmental areas and their potentially harmful impact, we comple­ increased in absolute and in relative terms the waste production (Kaza
ment this traditional (optimistic) approach with a pessimistic version of et al., 2018).
the BoD model. Besides, in its robust and conditional form, the direc­ Second, these sectors are dynamic and constantly evolving. During
tional distance BoD model allows to account for the possible presence of the last decades Portugal has committed considerable resources in the
outliers and to ensure a context-unbiased evaluation. mentioned sectors yielding an increasing attention of the public debate
We add to the previous literature with a number of contributions. and a positive thrust to the quality and the coverage of the offered
First, we develop a pressure indicator to evaluate the utilities jointly and services. While in 1994 the coverage for the services of water supply,
simultaneously active in the areas of water supply, wastewater collec­ wastewater collection and solid waste management was, respectively,
tion and solid waste management. In particular, our indicator evalu-ates 81.5%, 60.7% and 98%, nowadays it increased up to the 96%, the 85%
the utilities according to their ability of reducing the resource usage, the and the 100% (for the Portuguese mainland), corresponding to 9.6, 8.6
release of noxious substances in the environment, in line with the and 10 million of inhabitants.
indication provided by OECD Environment Directorate (2008); OECD Third, in Portugal the water and waste sectors are deeply integrated.
(2020) and Dong and Hauschild (2017). Second, from a theoretical Though they involve three distinct macro-areas - water supply, easte­
perspective, we complement the CI proposed by Zanella et al. (2015) in water collection and solid waste management - they are regulated and
two ways. On the one hand, we introduce the formulations of its pessi­ supervised by the same authority, and often they are managed by the
mistic version, following insights from Zhou et al. (2007) and Rogge same entities. Specifically, 48% of the utilities are jointly active in the
(2012). On the other hand, we use the robust and conditional analysis three macro-areas.
introduced by Cazals et al. (2002), following the path of Rogge et al. Fourth, the Entidade Reguladora dos Serviços de Agua e Resìduos
(2017), Lavigne et al. (2019), Fusco et al. (2020) and D’Inverno et al. (ERSAR - Regulatory Entity for Water and Waste), created in 1997 under
(2020). To do so, a revision of the definition of the CI has been neces­ the name of IRAR, is the fundamental body for the strategic decision-
sary. Third, we implement the suggested approach to the Portuguese making planning and the management of water supply, collection and
case. By evaluating the entities that are active both in the water and waste management. ERSAR acquired its regulatory power in 2009 and
waste sectors, we account for the possible interactions and synergies that become an independent administrative entity in 2014, however since
may occur in the joint management of these sectors. 2004 it is responsible for the performance assess-ment and bench­
To the best of authors knowledge, there is no previous study which marking of the utilities active in the sector. This responsibility has two
accounts for these two sectors together, in the framework of environ­ direct consequences: first, by evaluating the quality of the utilities,
mental performance. There are only a few studies that treat the water ERSAR implicitly decides which are the important criteria to be assessed
and waste sector jointly, specifically Allesch and Brunner (2014), Bel and the target to be reached (Gonçalves et al., 2014); second, ERSAR
and Warner (2008) and Caldas et al. (2019). However these papers focus collects the necessary data to analyze the performance of the utilities.
on the economic aspect, respectively the presence of privatization and of Fifth, these sectors are increasingly involved in the environmental
scale economies, and the environmental perspective is not considered. cause, by addressing cir-cular economy strategies and including waste
Finally, from a policy perspective, by benchmarking in comparative recycling in agriculture (Serra et al., 2019). At the beginning of the new
terms the utilities, we promote information exchange and encourage the millennium, Portugal faced the challenge of increasing the coverage and
imitation of the best performing practices. improving the performance of these services (Correia and Marques,
The rest of the paper is organized as follows. In section 2 we briefly 2010; Marques et al., 2018). Today, the challenge is to protect their
justify the choice of Portugal and we present the data. In section 3 we sustainability by providing and implementing solutions to minimize the
present the methodology and the path that brought us to the choice of negative impact on the environment and to ensure the continuous sup­
the directional distance BoD CI, both in its optimistic and pessimistic ply of high quality water, the collection and treatment of wastewater
formulation, and to its implementation in a robust and conditional and to reduce the amount of waste, for present and future generations
framework. In section 4 we report and comment the obtained results. (UN General Assembly, 2015).
Section 5 concludes the paper with some final remarks on the policy
relevance of the proposed tool. 2.2. The data

2. Empirical framework and data The database at our disposal contains information about the whole
population of water supply, collection and waste management utilities
2.1. Water and waste in Portugal in Portugal mainland in 2018 (ERSAR, 2018).
We restrict our focus on the retail utilities simultaneously active in
The idea of measuring the environmental pressure of waste and the three macro-areas, i.e., on the utilities providing jointly the three
water utilities is implemented by looking at the Portuguese case. In this services of water supply, wastewater collection and solid waste man­
country the system for water supply, wastewater collection and solid agement for the households. This allows us to construct a comprehensive
waste management shows a number of relevant characteristics that have indicator which fulfills the homogeneity assumption (see Dyson et al.,
drawn the attention of many scholars, generating a flourishing scientific 2001, p. 247), since all the units in our sample have similar productive
debate (see, among others, the recent papers by Martins et al., 2020; processes. In Portugal there are 180 utilities active in the three sectors,
Henriques et al., 2020; Marques and Simões, 2020; Silva and Rosa however it was possible to include in the analysis only the 149 units
2020). From a juridic perspective, private, state and municipal owned which provided sufficient information along the dimensions of interest.
utilities coexist. These utilities operate in the water supply, wastewater The units employed for our analysis provide more than 223 billion m3
collection and solid waste management and present a strong water per year, collect almost 279 billion m3 wastewater per year and

2
A. Mergoni et al. Utilities Policy xxx (xxxx) xxx

collect more than 2 million tons of urban solid waste per year, providing Table 1
the services of water supply and waste management, respectively, to 2, Definition of the environmental pressure sub-indicators chosen to construct the
207,000 and 2,240,000 of households. Among the sub-indicators composite indicator.
collected by ERSAR, four have been selected to measure the pressure Sub-indicator Pressure Definition
on the environment by water and waste utilities: 1) water losses 2) Real water Bad The volume of actual losses per unit length of conduit
structural collapses 3) gas emission and 4) recycled waste. The choice of losses in a day, measured in volume of losses/connections in
these indicators has been driven by the idea of accounting for the a day.
pressure (in terms of release of substances) that the water and waste ERSAR database code: AA12b.
Structural Bad The number of structural collapses in 100 km of
utilities exert on the environment (Marques et al., 2015; Molinos-Se­
collapse collectors in a year. ERSAR database code: AR08b.
nante et al., 2016; Pinto et al., 2017; Pèrez et al., 2018). This leads us to Gas emissions Bad Total amount of CO2 emissions from undifferentiated
the choice of our four sub-indicators. These indicators comprehensively collection vehicles per ton of waste collected in the
represent the multidimensional environmental pressure framework. manage-ment area of the management body. ERSAR
Moreover, we remark that the inclusion of less informative database code:RU17b.
Recycled Good Ratio among the ton of waste recycled and the target
sub-indicators would be paid by the exclusion of several units due to waste ton of waste recycled in the year. ERSAR database
missing values, without changing drastically the main findings (see also code: RU07b.
Henriques et al., 2020).
The first and the second criteria, water losses and structural col­
lapses, are indicators of bad pressure. Uncontrolled water release is bad Table 2
for the environment on different levels. First, it promotes soil erosion, Control variables used in the conditional analysis to account for possible het­
which is one of the greatest environmental threats to sustainability (Zhu erogeneity of the context where the utilities operate.
et al., 2019). Second, it is associated with leaching and nutrient loss,
Control variable Definition
leading to groundwater contaminations with nitrate and other soluble
compounds (Serra et al., 2019). Then, water quality also has an effect on Geographical Portugal is divided into five macro regions: the region of the
position North, the region of the Centre, the region of Lisbon, Alentejo
soil quality, modifying soil conditions and altering mineral nutrition and Algarve. In line with ERSAR reports, we consider just three
(Garcìa et al., 2008). The third indicator, gas emission, is also an indi­ macro domains: the North, equivalent to the region of the
cator of the bad pressure exerted by the utilities on the environment in North, the Centre, equivalent to the region of the Centre plus
the form of release of greenhouse gas (ERSAR, 2020). It refers to the the Lisbon district, and the South, composed by Alentejo and
Algarve.
total amount of CO2 emissions from undifferentiated collection vehicles
Intervention area We consider the typology of areas according to the definition
per ton of waste collected in the management area. The last criteria, i.e. of the Deliberations n. 488/98 and n. 2717/2009, followed
recycled waste - criterion 4, instead, is a measure of positive pressure also by the Portuguese national institute of Statistics. Three
exerted by the utilities on the environment, if properly managed. By intervention areas are identified: predominantly rural areas,
recycling the waste collected from the households, the utilities control medium urban areas and predominantly urban areas.
Volume of activity Volume of water (in m3) supplied in a year. The ‘Drinking
and prevent an otherwise inevitable release of polluting substances, as Water Directive’ (Council Directive 98/83/EC) distinguishes
long as duly managed. between large and small water utilities: ‘large water supplies
As Serra et al. (2021) report, Portugal mainland is characterized by provide either more than 1000 m3 drinking water per day as an
considerable heterogeneity in terms of climate, orography and land use. average or serve more than 5000 persons’a. For the present
application we refer to the volume of water.
To account for the possible impact of these external factors on the
behaviour of the utilities, we implemented a conditional analysis. Spe­ a
See https://eur-lex.europa.eu/legal-content/EN/TXT/HTML/?uri=CELEX:5
cifically, three control variables have been selected as possibly influ­ 2016DC0666&from=EN.
ential external variables: 1) geographical position 2) intervention area
and 3) volume of water supplied. These variables do not directly enter in
the construction of the composite indicator, but they might still affect Table 3
Mean distribution of the sub-indicators in 2018.
the assessed environmental pressure of the utilities. Specifically, their
location is directly related to their service provision since water utilities N Coverage Water Structural Gas Recycled
loss l/ collapse emission waste (%)
operate as natural monopoly. Volume of water supplied is used as a
day (n/km. kg (CO2/
proxy for the size (note that volume of water supplied is highly corre­ year) t)
lated with the volume of water and waste collected). Similarly to what
Overall 149 83% 146.1 3.82 20.15 86.58
happens for the economic assessment, the size might influence also the
min 1.8 0 6 28
environmental pressure. The urban areas reveal different needs and max 502 173 52 281
challenges with respect to the rural or the semi-urban ones, especially Geographical location
from an environmental perspective. North 39 81% 158.8 4.308 20.87 81.49
Centre 60 90% 141.51 5.480 20.90 81.63
For more details on the definitions of the sub-indicators and the
South 50 78% 141.84 1.438 18.7 96.5
control variables see Tables 1 and 2. As it can be noticed, the variables Intervention area
are measured in different scale, but this is not an issue as the imple­ Rural 112 80% 135.42 2.97 21.98 * 82.6
mented methodology does account for this. Semi- 29 97% 181.6 * 5.76 15.28 *** 103.3*
Table 3 shows that there is heterogeneity among the units located in urban
Urban 8 89% 167.88 8.575 12.25 *** 81.88
different intervention area and with different volumes of activity,
Volume of activity
especially for the indicators gas emission and recycled waste. Instead, Small 53 75% 125.8 5.113 25.08 *** 77.25**
the differences along the geographical position are not significant. Medium 48 87% 162.4 0.51** 19.9 83.77
Specifically, it emerges that, considering the variable intervention area, Large 48 97% 152.4 5.69 14.98 *** 99.71**
the units located in urban and semi-urban emit, on average, less gas, Note: The significance of the difference between the overall distribution and the
while the units located in rural area produce, on average, more gas. This distribution per groups has been computed through the t -test. *p<0.1;
pattern can be explained by the fact that in rural areas the households **p<0.05; ***p<0.01.
are located further one to the other, so that the companies are more Source: Authors’ own elaboration based on data from ERSAR.
prone to cover longer distances to deliver the services, and therefore, to
emit more gas. According to the volume of activity, small units do

3
A. Mergoni et al. Utilities Policy xxx (xxxx) xxx

significantly worse than the average in the emission of gas and in the where l, s, n respectively represent the number of undesirable sub-
recycling waste, while the large units do significantly better than the indicators, the number of desirable ones and the number of DMUs,
average according with these indicators. respectively; j0 is the evaluated DMU, bkj is the value of the undesirable
Fig. 1 complements Table 3 by showing the geographical variability sub-indicator k of the unit j, while yrj is the value of the desirable sub-
of the four environmental pressure sub-indicators over the Portuguese indicator r of the unit j. The vector g = (− gb, gy) represents the direc­
territory. Inspired by the ERSAR reports, we display in red the utilities tion along which the simultaneous contraction of the undesirable in­
that are exerting a high (negative) and so an unacceptable level of dicators and expansion of the desirable ones is possible. The current
pressure, in yellow a medium level and in green a low and so a less literature proposes alternative directions values: for example, directions
urgent level. From a policy-making perspective, choosing to address one values equal to one, g = (1, 1); equal to the current indicator values of
issue, e.g. the water losses, might lead to overlook utilities unsatisfac­ the unit under evaluation, g = (− bkj0, yrj0); or equal to the average values
torily performing in other domains.
across all the units under analysis, g = (− bk , yr ) (for a further discussion,
we refer to Rogge et al., 2017). Different directions give rise to different
3. Methodology
interpretations. For our model we choose g = (− gb, gy) = (− bkj0, yrj0), so
that each utility follows its own improvement path and a great level of
A crucial issue in the construction of Composite Indicators (CIs) is the
flexibility and proportional interpretation of the results are granted. β is
aggregating method as, in most cases, there is only disparate expert
the value of the directional distance function for the evaluated DMU and
opinion available about the appropriate weights to be used in the
it measures the room of possible improvements along the direction g; the
aggregator function. The Benefit of the Doubt (BoD) approach, pre­
optimal value of the problem, β*, belongs to (0, +∞) and, accordingly,
sented by Melyn and Moesen (1991) and then popularized by Cherchye 1
the associated CI is defined as 1+β * This formulation allows to ‘control’
et al. (2007), allows to overcome this problem. It endogenously assigns
weights so that the overall score depicts each analyzed decision making the value of the β*, so the value of the CI belongs to (0, 1]. The higher the
unit (DMU) in the best possible light relatively to the other observations. value of the CI, the closer the DMU is to the best-practice frontier. DMUs
So every DMU is granted with the ‘Benefit of the Doubt’ and the approach on the frontier assume a CI = 1 (see also Zanella et al., 2015; Rogge et al.,
is strongly data oriented. These two qualities explain a major part of the 2017; Lavigne et al., 2019).
appeal of the BoD-based CIs in real settings.
3.2. A complement to the traditional BoD: a pessimistic approach
3.1. The traditional BoD model: an optimistic approach
By construction, the weights assigned by the traditional Benefit of
The BoD approach has its root in the Data Envelopment Analysis the Doubt allow to evaluate each utility under the best possible light.
(DEA) model of Charnes et al. (1978); it actually can be seen as an This measure is obtained by overemphasizing the dimensions where the
input-oriented DEA model with unitary input and the sub-indicators as units perform the best and mostly neglecting where they perform the
outputs. Therefore, we can translate also the interpretation of the score: worst. This endogenous weighting mechanism grants a fair evaluation
a good relative performance of a DMU, in one particular sub-indicator, and mostly avoids complaints among the evaluated units. In spite of the
indicates that the evaluated unit considers that specific dimension as fairness granted to the utilities under evaluation, the BoD analysis might
relatively important. overlook very poor performances along some dimensions, thus cannot
The value of the performance is obtained by aggregating all the sub- be completely informative from an environmental footprint perspective
indicators values, weighting them in the most convenient way for the and might suggest inappropriate policy measures. To avoid this issue, we
unit under analysis, subject to two constraints: 1) the weights have to be complement the traditional (optimistic) evaluation with the so-called
positive and, 2) the value of the CI, for no unit in the sample can exceed a “pessimistic” version of the BoD model (Dardha and Rogge, 2020).
given threshold (usually fixed at 1). From an intuitive point of view, the pessimistic approach evaluates how
The BoD model has been designed to deal with ‘desirable’ sub- close is each utility to the worst performing utilities in the sample under
indicators (meaning the higher the better). Nevertheless, it may occur the least favorable evaluation conditions, that is, assigning high weights
that some relevant dimensions of the analyzed units are described by on areas where the utility exerts a relatively high environmental pres­
means of ‘undesirable’ sub-indicators (meaning the lower the better). sure level and low weights where it exerts relatively low environmental
Whenever both ‘desirable’ and ‘undesirable’ sub-indicators are consid­ pressure level (Rogge, 2012). To the best of the authors’ knowledge, this
ered, the standard BoD model cannot be applied. To overcome this is the first application of the pessimistic scenario adapted to the main
drawback, Zanella et al. (2015) propose an alternative formulation on model proposed by Zanella et al. (2015) following insights from Zhou
the basis of the directional distance function approach of Chung et al. et al. (2007) and Rogge (2012). The problem (1) adjusted for its pessi­
(1997). Like the BoD model, a dummy input is fixed at a unitary level mistic counterpart then becomes the following (in Appendix A we pro­
and like the directional distance function models, a suitable directional vide also the multiplier formulation for both the optimistic and the
vector g is considered to allow the simultaneous contraction of the un­ pessimistic directional distance BoD model):
desirable indicators and expansion of the desirable ones. According to ⎧
⎪
Zanella et al. (2015)(p.523), in this paper, CIs are computed by solving ⎪
⎪
⎪
⎪
the following maximization problem: ⎪
⎪
⎪ min βP
⎪
⎪
⎪ ∑n
⎧ ⎪
⎪
⎪ ⎪ s.t
⎪ bkj λj ≥ bkjo − βP gb , k = 1, …, l
⎪
⎪ ⎪
⎪ j=1
⎪
⎪ ⎨ ∑n
⎪
⎪
⎪
⎪ max β yrj λj ≤ yrjo + βP gy , r = 1, …, s (2)
⎪
⎪ ∑n ⎪
⎪
j=1
⎪
⎪ ⎪ ∑n
⎪
⎪ s.t
⎪ bkj λj ≤ bkjo − βgb , k = 1, …, l ⎪
⎪ λj = 1
⎪
⎪ j=1 ⎪
⎪ j=1
⎨ ∑n ⎪
⎪
⎪
j=1
yrj λj ≥ yrjo + βgy , r = 1, …, s (1) ⎪
⎪
⎪ λj ≥ 0, j = 1, …, n
⎪
⎪ ⎪
⎪
⎪ ∑n
⎪ ⎪
⎩
⎪
⎪ λj = 1
⎪
⎪ j=1
⎪
⎪
⎪
⎪
⎪ λj ≥ 0, j = 1, …, n
⎪
⎪
⎪ where l, s, n respectively represent the number of undesirable sub-
⎪
⎩ indicators, the number of desirable ones and the number of DMUs,
respectively; j0 is the evaluated DMU, bkj is the value of the undesirable

4
A. Mergoni et al. Utilities Policy xxx (xxxx) xxx

Fig. 1. Geographical distribution of the environmental pressure sub-indicators.

Source: Authors’ own elaboration based on data from ERSAR relative to 2018.

sub-indicator k of the unit j, while yrj is the value of the desirable sub- 0, while βP* tends to 1 for the least worst performing units. Similarly to
indicator r of the unit j. 1
the optimistic case, the associated CIP is defined as 1+β * Accordingly, the

Coherently with the optimistic version, we set g = (− gb, gy) = (− bkj0,

P

value of the CIP belongs to [1, +∞). The lower the value of the CIP, the
yrj0). Intuitively, this means moving along the direction opposite to the
closer the DMU is to the worst-case scenario. The worst-performing
optimistic one or, in another way, the direction along which the
DMUs assume CIP = 1 (see also Zhou et al., 2007; Rogge, 2012).
simultaneous expansion of the undesirable indicators and contraction of
the desirable ones is possible, so to reach the worst-case scenario. βP is
3.3. Beyond the deterministic nature of BoD: a robust and conditional
the value of the directional distance function for the evaluated DMU. In
approach
the pessimistic case, the optimal value of the problem is non-positive, as
βP* belongs to (− 1, 0]. Hence, the worst performing units assume βP* =
Previous literature (see e.g. Nardo et al., 2005, or Daraio and Simar,

5
A. Mergoni et al. Utilities Policy xxx (xxxx) xxx

2007) highlighted some typical limitations related to the use of a considering only the DMUs belonging to Γb,m,z. Then, the mean of the
non-parametric approach. In particular, the deterministic nature of the obtained B values of βb,m,z is calculated, and the corresponding Com­
CI leads to three issues: 1. statistical inference is difficult, 2. the scores posite Indicators CIm,z is computed by using function (3).
are sensitive to outliers and 3. to the sample size. To face these problems, The interpretation of the conditional Composite Indicator CIm,z has to
we complement the model with a robust and a conditional analysis, by go arm in arm with the comparison between this indicator and the
applying the methodology proposed by Cazals et al. (2002) and by robust one, namely CIm. To investigate the source of the difference be­
Daraio and Simar (2005, 2007) (see also Rogge et al., 2017; Fusco et al., tween them, the ratio CIm/CIm,z is considered, as suggested by Daraio
2020; Lavigne et al., 2019; D’Inverno et al., 2020). and Simar (2007).2 If the ratio is increasing along the environmental
The robust evaluation of Cazals et al. (2002), also called ‘order-m’, variable, it means that this variable has a positive influence on the
consists of a Monte Carlo simulation. Each DMU is evaluated B times performance of the utilities we are measuring. Vice versa a decreasing
with respect to m units randomly drawn with replacement from the ratio shows an unfavorable environment. We regress the ratio of the
original sample Γ (with n > m). This allows to control for extremes and robust over the conditional on the environmental variables using a
outliers. non-parametric regression (as suggested by Daraio and Simar, 2007
B sub-samples Γb,m are generated for each DMUj under analysis and B page 113):
scores are calculated. 0βb,m is the directional distance BoD score
CI m
computed for the DMUj, using the bth sub-sample of dimension m. = g(Zi ) + εi , i = 1, ...., n. (4)
CI m,z
Therefore, in the robust version of Model 1, a given DMU j appears in the
0 constraints only if j ∈ Γb,m. Once obtained the B βb,m coefficients, we
∑ 4. Results and discussion
define βm = Bb=1 B1βb,m .
It is important to notice that the DMUj0 under analysis may not be The environmental pressure index was computed for 149 utilities
drawn in the sub-sample used as reference set. For this reason each βb,m that provide both waste and water services in Portugal. For the esti­
belongs to ℝ (and so does βm). The more negative the βm the further the mation we followed the methodology described in the previous section,
DMUj over the frontier, in the sense that it is performing better. If βm < so to get an aggregate indicator that measures how well the operators
0 the DMU is referred to as super-performing.1 Since βm ∈ ℝ, the previous are coping with the environmental pressure they exert on the environ­
formulation of the CI loses its explanatory power. This is due to two ment. First, we explored the obtained findings for the deterministic case.
main reasons: first, the function CI(βm) = 1+β 1
m is defined over (− ∞, − 1)∪ Second, we explored the results considering the optimistic and the
(− 1, +∞) and not over ℝ (note that in the deterministic model this was pessimistic environmental scenario. Third, we gave insights on the
not a problem as βm belonged to [0, +∞). Second, interpretation prob­ robust and the conditional analyses. Finally, we investigated the influ­
lems arise for those DMUs having a value of βm lower than − 1; although ence of the operating context through statistical inference.
they are super-performing, their corresponding CI is negative and,
accordingly, they are judged worse than the bad performing ones, i.e. 4.1. Results from the traditional BoD model
those with a high and positive value of βm. To avoid these problems, we
propose the following construction of the robust Composite Indicator: Table 4 shows the descriptive statistics of the environmental pressure
⎧ Composite Indicator (CI) scores for the deterministic case. The mean
⎨ 1
⎪
if βm ≥ 0 value of 0.7398 suggests that there is room for improvements in envi­
(3)
m
CI m (βm ) = 1 + β ronmental pressure reduction if all the entities would perform on the
⎪
⎩
log(1 − βm ) + 1 if βm < 0 four sub-indicators as well as the best performing entities. The minimum
value of 0.5819 together with the first quartile of 0.6614 denotes the
The performance score CI(βm) is now defined over R and it is
widespread presence of poorly performing operators, i.e., operators
continuous and differentiable. As in the deterministic case, it is
which are outperformed despite being evaluated in the most favorable
decreasing with respect to β and preserves the interpretation proposed
way along different measures of environmental pressure. Previous
by Daraio and Simar (2007): a value of CI(βm) greater than one indicates
literature had already detected the need for a performance enhancement
that the unit j0 is better performing than the average of m peers
of the Portuguese water and waste sectors (see among others Ferreira da
randomly drawn from the population (p.71).
Cruz et al., 2012; Marques et al., 2015; Molinos-Senante et al., 2016;
To properly account for the influence of the exogenous characteris­
Pèrez et al., 2019). Our findings complemented this evidence by giving
tics and therefore to ensure a fairer evaluation, we allow the bench­
specific emphasis on the environmental sustainability issue and partic­
marking frontier to ‘adapt’ according to the exogenous characteristics of
ularly from an environmental pressure perspective.
the unit under analysis, i.e., we adopt the conditional analysis (see
We identified 11 best performing operators (CI = 1) out of the 149 in
developed in Cazals et al., 2002; Daraio and Simar, 2005, 2007; De Witte
the sample. This means that a relatively small percentage of our sample
and Rogge, 2011). The basic idea is to condition the choice of the
(7.38%) can be considered as best practice for the others that report CI
reference set for the DMU j0 under evaluation according to its own
scores lower than one. We also explored the characteristics of these units
exogenous characteristics. While in the robust scenario the units of the
by looking at the distribution of the CI along the operating context
reference group Γb,m
j0 are drawn with replacement from a uniform dis­ variables introduced in section 2, namely the geographical location, the
tribution, in the conditional case they are included in the reference area of intervention and size. At first sight, the utilities that report the
group Γ b,m,z
j0 according to the probability of being similar to the obser­ highest mean and median values are located, more likely, in the South of
vation j0 (with Γb,m,z = Γb,m|Z). Similarity is measured by means of the Portugal, or in areas predominantly urban, or they are large.
probability distribution for the joint Z variables, estimated by a kernel
function (see also De Witte et al., 2013; Li and Racine, 2003). Using a
Monte Carlo simulation procedure, B sub-samples Γb,m,z are generated
for each DMUj and B βb,m,z are obtained by running model (1)

1 2
The terminology used in the literature is super-efficiency; since we refer to a Daraio and Simar (2007) use the inverse of this ratio. The reason of our
composite indicator, we prefer to talk of ‘performance’ instead of ‘efficiency, in choice is that it simplifies the interpretation of the estimated relationships
line with Rogge et al. (2017) and Lavigne et al. (2019). (Rogge et al., 2017).

6
A. Mergoni et al. Utilities Policy xxx (xxxx) xxx

Table 4
Descriptive statistics of the environmental pressure composite indicator scores (both overall and grouped by operating context variables). The scores are obtained
implementing the deterministic and unconditional analysis.
N Mean SD Min. Q1 Median Q3 Max.

Deterministic unconditional 149 0.7398 0.1156 0.5819 0.6614 0.6993 0.7914 1.0000
Geographical location
North 39 0.7281 0.1192 0.5819 0.6500 0.6914 0.7600 1.0000
Centre 60 0.7288 0.1098 0.5897 0.6495 0.6956 0.7758 1.0000
South 50 0.7622 0.1186 0.5915 0.6790 0.7030 0.8652 1.0000
Intervention area
Rural 112 0.7281 0.1133 0.5819 0.6563 0.6921 0.7587 1.0000
Semiurban 29 0.7680 0.1158 0.6223 0.6652 0.7594 0.8156 1.0000
Urban 8 0.8018 0.1235 0.6364 0.7416 0.7864 0.8626 1.0000
Volume of activity
Small 53 0.6996 0.0954 0.5819 0.6463 0.6786 0.7022 1.0000
Medium 48 0.7272 0.1094 0.5915 0.6604 0.6942 0.7540 1.0000
Large 48 0.7969 0.1214 0.6223 0.6890 0.7719 0.8986 1.0000

4.2. The environmental pressure index in an optimistic and pessimistic sectors, we complement the results obtained using the traditional/
scenario comparison optimistic BoD approach with a pessimistic one. In the former we give
more emphasis on the areas where utilities are exerting a relatively low
To provide a more comprehensive picture of the environmental pressure level compared to the other utilities, highlighting the best
pressure exerted by the Portuguese utilities jointly operating in the three scenario. In the latter we obtain information on how well they are

Fig. 2. Comparison of environmental pressure performance in an optimistic and pessimistic scenario (Note: Overall good performance for CI = 1 and CIP > 1; Overall
mediocre performance for CI < 1 and CIP > 1; Overall poor performance for CI < 1 and CIP = 1).
Source: Authors’ own elaboration.

7
A. Mergoni et al. Utilities Policy xxx (xxxx) xxx

performing despite the least favorable evaluation, outlying the worst their environmental footprint.
scenario.
Fig. 2a shows in a synthetic way the results obtained in these two 4.3. The environmental pressure index accounting for outliers and
opposite scenarios (we report in Appendix B the descriptive statistics of exogenous characteristics
the pessimistic environmental pressure composite indicator scores).
Utilities with a CI lower than 1 are the ones displaying a low perfor­ To account for the possible presence of atypical observations and to
mance, despite being evaluated in their most favorable scenario. Utili­ properly detect the influence of the exogenous characteristics, we esti­
ties with a CIP equal to one are the ones performing weakly in the mated the traditional model in its robust unconditional and conditional
majority or even all the dimensions. version, so that unit’s performance was assessed in a fairer way. Table 5
Following Rogge (2012), we can distinguish three groups of utilities shows the descriptive statistics of the environmental pressure Composite
based on their CI and CIP. The first group is characterized by an overall Indicator (CI) scores for these two cases together with the deterministic
good environmental pressure level, with CI = 1 and CIP > 1. The utilities case, for comparison purposes. To implement the order-m directional
in this group perform well both under the optimistic and the pessimistic distance BoD model, a value for m must be chosen. A recipe to choose the
scenario. Thus, they don’t show a peculiar specialization on a particular suitable value of m does not exists, however we followed the procedure
area, but they perform relatively strongly compared to the other utili­ suggested by Daraio and Simar (2007), p. 78–81. Accordingly, a value of
ties, in all, or almost all, the environmental pressure sub-indicators m equal to 65 seemed the most appropriate choice. Both the robust
considered. From a policy-making perspective, these operators (CM de unconditional and conditional estimates display higher CI scores with
Ansião, CM de Èvora, CM de Ferreira do Zézere, CM de Melgaço, CM de respect to the deterministic case in all the summary statistics.
Òbidos, CM de Ponte de Lima, CM de Pòvoa de Varzim, CM de Santiago These estimations yielded CI scores greater than one in the upper
do Cacèm, INFRAQUINTA, INFRATROÌA, SM de Castelo Branco) are the part of the score distribution. This denotes the presence of super-
best practices that the other utilities should look at to reduce their performing units, i.e., units performing better than the average units
environmental pressure or that show extremely outstanding perfor­ they are compared with. Moreover, from the comparison of the median
mance. This is for example the case of the operator INFRAQUINTA, that and the mean values, we also notice that the distribution of the condi­
reports one of the highest CIP scores, CIP = 5.3368, and CI = 1. This tional scores has a fatter right tale than the robust unconditional one,
operator has been already identified in other studies as one of the Por­ suggesting that the majority of the units are working in an unfavorable
tuguese top performing utilities (see Molinos- Senante et al., 2016 and context. Nevertheless, there are units that are still very poorly per­
Henriques et al., 2020). This can be seen as an example of utility that forming as pointed out by the minimum value of 0.5930 for the robust
promotes environmental sustainability and tackles environmental unconditional case and 0.6034 for the conditional one.
pressure in water supply, wastewater sanitation and urban solid waste The three estimated environmental pressure indexes are quite highly
management sectors as a public commitment (see https://www.infr correlated (0.9688, 0.8259, 0.8623) and the distribution of the units
aquinta.pt/en/empresa/activities-plan). The exceptional good perfor­ among the observed background characteristics display a pattern similar
mance of this unit can be partly explained by its recent re-organisation to the one described for the deterministic unconditional case.
and the relatively modern infrastructures, which create also the expec­ Fig. 3 shows the geographical distribution of the estimated efficiency
tation of future investment return (Henriques et al., 2020). scores. The three CI scores display a similar pattern, confirming that the
The second group is characterized by an overall mediocre perfor­ potential presence of outliers or different operating contexts do not
mance, with CI < 1 and CIP > 1. The utilities in this group do not significantly affect the outlined trend. An interesting feature of the
perform as good as the best practices, but also not as bad as to be environmental pressure Composite Indicator suggested in this paper is
considered the worst performing units. In this sense, they might have that it allows to aggregate different dimensions in a fully data driven
focused their effort on a specific sector or only on a few ones to deal with way. Therefore, with a single glance we are able to identify the most
the environmental pressure they exert. Regarding this group, policy­ critical areas, beyond the partial view offered separately by each sub-
makers should pay attention to the dimensions mostly left behind and indicator as presented in section 2.
provide incentives for their improvements. The third group is charac­ The comparison between the robust unconditional and conditional
terized by an overall poor performance, with CI < 1 and CIP = 1. In an CI scores allowed us to detect the influence of the background charac­
environmental perspective, these utilities should be the first ones to be teristics on the estimated level of environmental pressure. Preliminarily,
looked at, since they exert the highest level of environmental pressure. the Kolmogorov - Smirnov test was implemented to test if the difference
From a policy perspective, the goal is not to ‘name and shame’ these among the conditional and the robust CI scores is statistically signifi­
utilities (Cabus and De Witte, 2012), but rather to identify them and cant. The obtained p-value (0.0003338) provided a strong evidence in
support them, as they are the detected most harmful ones for the envi­ favor of this hypothesis. We focused on the partial regression plots re­
ronment. As a last remark we point out that there are no utilities with CI ported in Fig. 4 to investigate the source of this difference. The back­
= 1 and CIP = 1, ruling out the presence of extreme scenarios with ground variables were regressed on the ratio between the robust
excellent performance on one dimension and very poor performance on unconditional and conditional, following the insights provided by Dar­
another one at the same time. aio and Simar (2007). If the ratio is increasing along the background
Fig. 2b–d shows the distribution of the utilities by the operating variable, it means that this variable has a positive influence on the
context variables and by their performance level. Most of the utilities performance of the utilities we are measuring and the opposite holds
belong to the overall mediocre performance group. From the analysis of otherwise.
the weights and the contribution of each element to the composite in­ We observed that the size and the area of intervention display a
dicator, we can observe that in the optimistic case relatively good per­ statistically significant relationship with the score ratio. Specifically, we
formance is related to the water loss and gas emission indicators, while observed a reversed U-shaped relation between the size and the esti­
in the pessimistic case the most critical component is the level of recy­ mated environmental pressure composite indicator, suggesting the po­
cled waste, confirming the intuition we get from Fig. 1. While for the tential presence of an optimal size. Previous literature on economies of
geographical location and the intervention area there is no clear evi­ scale and scope investigated the possible existence of an optimal size
dence of best practices, the volume of activity suggests that the small both for the water and waste sectors, concluding that a wide range of
utilities are overall the worst performing ones. These utilities face huge optimal scales can be detected and diseconomies in larger utilities can be
costs to reduce their environmental impact in the three sectors and found (see for example Simões et al., 2013; Carvalho and Marques,
diseconomies of scale and scope arise. Policy makers should monitor 2014, 2016; Caldas et al., 2019, and the references therein). Evidence
more closely their activity and generate an incentive scheme to reduce from our empirical analysis suggests that whenever the utility is either

8
A. Mergoni et al. Utilities Policy xxx (xxxx) xxx

Table 5
Descriptive Statistics of the environmental pressure Composite Indicators scores for different model specifications.
N Mean SD Min. Q1 Median Q3 Max.

Deterministic unconditional 149 0.7398 0.1156 0.5819 0.6614 0.6993 0.7914 1.0000
Robust unconditional 149 0.8319 0.2272 0.5930 0.7029 0.7445 0.8749 1.8229
Robust conditional 149 0.8486 0.1604 0.6034 0.7371 0.8098 0.9867 1.7203

Fig. 3. Geographical distribution of the environmental pressure composite indicator.

Source: Authors’ own elaboration.

too small or too large, it becomes difficult to contain the release of activity and to take action toward a more environmentally sustainable
pollutants and high investments should be done to ameliorate the development (Molinos-Senante et al., 2016).
existing infrastructures or to increase their production capacity.
About the intervention area, the predominantly rural areas have a 5. Conclusions
positive relationship with the environmental pressure management,
while the opposite holds for medium urban areas and predominantly urban The urgent need for an environmentally sustainable development
areas. To reconcile this evidence with the one stemming from the calls for practical actions by managers and policy makers. However,
descriptive statistics, we consider that the higher scores of the urban finding good practices and selecting intervention areas are hard tasks for
utilities might be mostly driven by the size (as the urban utilities are also utilities that jointly cover services in different sectors, such as the water
the larger). Besides, it could be noticed from Fig. 3 that units located in supply, the wastewater collection and the solid waste management. This
the mountainous areas (therefore in the north or in the Serra da Estrela), is because, for each service, they exert different levels of pressure on the
on average, performed worse. A possible mechanism to explain this is environment, either in terms of substance release or in terms of resource
that a steep terrain causes higher maintenance costs, therefore higher use. As they all are potentially harmful for the environment, assigning an
water losses, and higher transportation costs, therefore higher gas order of importance in an objective way becomes a tricky challenge.
emissions and less solid waste recycling (see also Gaeta et al. (2017); We contribute to the literature by proposing a novel pressure com­
Sarra et al. (2017)). posite indicator to measure and benchmark utilities active in different
Finally, the geographical location did not display any particularly sectors and exerting different forms of environmental pressure. Specif­
statistically significant association, even if it is still worth to be ically, we complement the use of a traditional directional distance
accounted for in the conditional estimation. Two factors mostly offset Benefit of the Doubt composite indicator with its pessimistic version so
this evidence. From a territorial perspective, northern regions are mostly to take into account the most harmful impact in the worst environmental
characterised by mountainous areas, while southern regions suffer scenario. In addition, we integrate the composite indicator with a robust
particularly of seasonal imbalances and drought (Ferreira da Cruz et al., and conditional approach so to account for the potential presence of
2012; European Commission, 2014), causing again higher costs and atypical observations and the influence of contextual variables.
difficulties in the process of collecting waste and treating water. From a We test the proposed evaluation framework by evaluating 149 Por­
regulatory perspective, recent changes of the social tariff regime also tuguese utilities jointly active in the water supply, wastewater collection
played a role in jeopardizing the equity, sustainability and territorial and solid waste management sectors. In the annual reports, the Portu­
cohesion of this regulated sector (Martins et al., 2020). guese regulator (ERSAR) identifies room for improvement in any of the
The evidence stemming from the statistical inference offers an sub-indicators accounted in the proposed environmental pressure com­
informative picture about the environmental pressure management in posite indicator, even suggesting potential ways to pursue it. With this
the Portuguese waste and water sectors. This can be considered as a respect, the beneficial feature of the proposed composite indicator is
starting point for further discussion to raise awareness among all the twofold. First, it detects the operators that exert the highest negative
involved stakeholders on the environmental impact of these services’ pressure on the environment encompassing the three services as a

9
A. Mergoni et al. Utilities Policy xxx (xxxx) xxx

Fig. 4. Visualization of the partial regression plots with confidence intervals for the operating context variables. A positive slope denotes a favorable influence on the
environmental pressure composite indicator level, while the opposite holds for a negative slope.
Source: Authors’ own elaboration.

whole, granting them the most favorable assessment. Second, it suggests the short run, but empirical findings can direct the effort at national
possible role models by looking at the best practices that emerge from level and within the region. To this extent, we remark that the volume of
the benchmarking exercise. On average, we find that there is room to activity plays a very significant role with respect to the environmental
alleviate the exerted environmental pressure and 11 utilities are detec­ pressure. Specifically, small utilities are
ted as the best practises under both the optimistic and the pessimistic the most critical ones and the regulator should encourage shared
scenario. Most importantly, we are able to identify the utilities that are service arrangement to seek increasing returns to scale and to invest
poorly performing in all the environmental dimensions. Ideally, all the more on their environmental sustainability. Furthermore, environ­
utilities exerting a sizable pressure on the environment should be pushed mental best practices can be stimulated emphasizing the good perfor­
to improve their pressure. However, in a context where there are limited mance signalling role of certifications, that currently are still not widely
resources and the measures to be taken are on a large (national) scale, acknowledged (Molinos-Senante et al., 2016), as well as promoting
the policy makers should start intervening in the areas where the envi­ more public commitment and transparency (Henriques et al., 2020).
ronmental pressure is very critical, suggesting how to alleviate it by The present analysis focuses on a cross-sectional dataset. Further
looking at the best practices observed from the analysis. Accordingly, we research might explore the time component to check whether poor
draw the national regulator’s attention on the utilities with lower scores performing utilities are catching up with the best ones narrowing
in both scenarios and whose background characteristics represent the the gap and alleviating the overall environmental impact
most unfavorable environment. Certainly the background characteris­ (Horta and Camanho, 2015; Henriques et al., 2020). Moreover, the main
tics are variables that cannot be changed by the managers, especially in results point at the most critical areas, but additional analysis might

10
A. Mergoni et al. Utilities Policy xxx (xxxx) xxx

follow to further explain the mechanisms and the hidden synergies Declaration of competing interest
behind the joint management of the three sectors (Caldas et al., 2019).
In this paper, the case of the Portugal has been presented to measure The authors declare that they have no known competing financial
the environmental pressure of water supply, wastewater collection and interests or personal relationships that could have appeared to influence
urban solid waste management sectors. Given the worldwide relevance the work reported in this paper.
of the environmental sustainability and pressure, the proposed approach
can be interestingly used for other countries and/or for other indicators Acknowledgments
to get useful insights where to intervene first. This will raise awareness
of critical areas among the involved stakeholders and promote greater The authors gratefully acknowledge Ana Camanho, Kristof De Witte,
transparency in the environmental impact of the activities under scru­ Maria Molinos-Senante, Giulia Romano and Fritz Schiltz for their useful
tiny, to grant a sustainable development not only for the present gen­ comments and suggestions on a previous version of this paper. Giovanna
erations but also for the future ones. D’Inverno acknowledges financial support from FWO Postdoctoral
Fellowship (Grant 12U0219N). Laura Carosi acknowledges financial
support from University of Pisa (PRA 2018 Project).

Appendix A. ‘Optimistic’ VS ‘Pessimistic’ version of the BoD model

In the following table we present the primal and multiplier formulation of the model introduced by Zanella et al., (2015), along with its pessimistic
counterpart.

Zanella et al. (2015, mod.7-8 p.523) Based on Zanella et al. (2015)

OPTIMISTIC VERSION PESSIMISTIC VERSION
Primal formulation Primal formulation

maxβ minβP

n
∑ n
∑
s.t. bkj λj ≤ bkj0 − βgb s.t. bkj λj ≥ bkj0 − βgb
j=1 j=1
for k = 1, …, l for k = 1, …, l

n
∑ n
∑
yrj λj ≥ yrj0 + βgy yrj λj ≤ yrj0 + βgy
j=1 j=1
for r = 1, …, s for r = 1, …, s

n
∑ n
∑
λj = 1 λj = 1
j=1 j=1

λj ≥ 0 for j = 1, …, j0 , …, n λj ≥ 0 for j = 1, …, j0 , …, n

Multiplier formulation Multiplier formulation

s l
∑ s l
∑
∑ ∑
βj0 = min − yrj0 urj0 + bkj0 pkj0 + vj0 βPj0 = max − yrj0 urj0 + bkj0 pkj0 + vj0
r=1 k=1 r=1 k=1

s l
∑
∑ l
s.t. gy urj0 + gb pkj0 = 1 s
∑ ∑
r=1 s.t. gy urj0 + gb pkj0 = 1
k=1
r=1 k=1
s l
∑
∑
− yrj urj0 + bkj pkj0 + vj0 ≥ 0 s
∑ l
∑
r=1 k=1 − yrj urj0 + bkj pkj0 + vj0 ≤ 0
for j = 1, …, j0 , …, n r=1 k=1

for j = 1, …, j0 , …, n
urj0 ≥ 0 for r = 1, …, s

urj0 ≥ 0 for r = 1, …, s
pkj0 ≥ 0 for k = 1, …, l

vj 0 ∈ ℜ pkj0 ≥ 0 for k = 1, …, l

where CIj0 = 1/(1 + βj0 ) ∈ (0, 1] vj0 ∈ ℜ

where CIjP0 = 1/(1 + βPj0 ) ∈ [1, +∞)

yrj0 and bkj0 respectively refer to the observed r desirable and k undesirable indicator of the evaluated DMU j0. urj0 and pkj0 are the BoD weights
corresponding to the r desirable and k undesirable indicator for the evaluated DMU j0. In the optimistic model they represent the most favorable
weights for the unit under evaluation, in the pessimistic model the least favorable. yrj and bkj respectively refer to the r desirable and k undesirable
indicator of every DMU j in the dataset; n is the number of DMU under analysis; s and l respectively denote the number of desirable and undesirable
indicators considered in the application.

11
A. Mergoni et al. Utilities Policy xxx (xxxx) xxx

Appendix B. Descriptive statistics of the pessimistic BoD

In the following we present the descriptive statistics for the pessimistic version of the proposed environmental pressure index, both overall and
grouped by operating context variables.

Table B.1
Descriptive statistics of the pessimistic environmental pressure composite indicator scores (both overall and grouped by operating context variables). The scores are
obtained implementing the deterministic and unconditional analysis.

N Mean SD Min. Q1 Median Q3 Max.

Deterministic unconditional 149 2.213 0.9602 1.000 1.481 2.122 2.679 5.776
Geographical location
North 39 2.104 0.9171 1.000 1.317 2.131 2.642 4.429
Centre 60 2.169 0.9861 1.000 1.478 2.059 2.547 5.776
South 50 2.352 0.9644 1.000 1.731 2.204 2.695 5.404
Intervention area
Rural 112 2.145 0.8928 1.000 1.490 2.058 2.577 5.776
Semi-urban 29 2.465 1.1425 1.000 1.481 2.194 3.017 5.404
Urban 8 2.254 1.1368 1.000 1.558 1.940 2.610 4.429
Volume of activity
Small 53 1.944 0.7049 1.000 1.382 2.053 2.378 4.057
Medium 48 2.079 0.8729 1.000 1.442 1.874 2.531 5.776
Large 48 2.646 1.1403 1.000 1.756 2.377 3.234 5.492

References D’Amato, D., Droste, N., Allen, B., Kettunen, M., Lähtinen, K., Korhonen, J., Leskinen, P.,
Matthies, B.D., Toppinen, A., 2017. Green, circular, bio economy: a comparative
analysis of sustainability avenues. J. Clean. Prod. 168, 716–734.
Allesch, A., Brunner, P.H., 2014. Assessment methods for solid waste management: a
D’Inverno, G., Carosi, L., Romano, G., 2020. Environmental Sustainability and Service
literature review. Waste Manag. Res. 32, 461–473.
Quality beyond Economic and Financial Indicators: A Performance Evaluation of
Bel, G., Warner, M., 2008. Does privatization of solid waste and water services reduce
Italian Water Utilities. Socio-Economic Planning Sciences, p. 100852.
costs? a review of empirical studies. Resour. Conserv. Recycl. 52, 1337–1348.
ERSAR, 2018. Dados de base. http://www.ersar.pt/pt/setor/factos-e-numeros/dado
Cabus, S.J., De Witte, K., 2012. Naming and shaming in a ‘fair’way. on disentangling the
s-de-base. (Accessed 12 March 2021).
influence of policy in observed outcomes. J. Pol. Model. 34, 767–787.
ERSAR, 2020. Rasarp, vol. 1. http://www.ersar.pt/pt/site-publicacoes/Paginas/edicoes-
Caldas, P., Ferreira, D., Dollery, B., Marques, R., 2019. Are there scale economies in
anuais-do-RASARP.aspx. (Accessed 11 May 2021).
urban waste and wastewater municipal services? a non-radial input-oriented model
EurEau Association, 2021. Water Matters. http://www.eureau.org/. (Accessed 12 March
applied to the Portuguese local government. J. Clean. Prod. 219, 531–539.
2021).
Carvalho, P., Marques, R.C., 2014. Economies of size and density in municipal solid
European Commission, 2014. Potential for Growth & Job Creation through the
waste recycling in Portugal. Waste Manag. 34, 12–20.
Protection of Water Resources, with a Special Focus on the Further Implementation
Carvalho, P., Marques, R.C., 2016. Computing economies of scope using robust partial
of the WFD and FD, Issued under the EC Freshwater Policy Framework ENV/D.1/
frontier nonparametric methods. Water 8 (3), 1–23, 82.
FRA/2012/0014. MS fiche, Portugal.
Cazals, C., Florens, J., Simar, L., 2002. Nonparametric frontier estimation: a robust
Ferreira da Cruz, N., Marques, R.C., Romano, G., Guerrini, A., 2012. Measuring the
approach. J. Econom. 106, 1–25.
efficiency of water utilities: a cross-national comparison between Portugal and Italy.
Charnes, A., Cooper, W., Rhodes, E., 1978. Measuring the efficiency of decision making
Water Pol. 14, 841–853.
units. Eur. J. Oper. Res. 2, 429–444.
Fusco, E., Vidoli, F., Rogge, N., 2020. Spatial directional robust benefit of the doubt
Cherchye, L., Moesen, W., Rogge, N., Puyenbroeck, T., 2007. An introduction to ‘benefit
approach in presence of undesirable output: an application to Italian waste sector.
of the doubt’ composite indicators. Soc. Indicat. Res.: An International and
Omega 94, 102053.
Interdisciplinary Journal for Quality-of-Life Measurement 82, 111–145.
Gaeta, G.L., Ghinoi, S., Silvestri, F., 2017. Municipal performance in waste recycling: an
Chung, Y., Färe, R., Grosskopf, S., 1997. Productivity and undesirable outputs: a
empirical analysis based on data from the lombardy region (Italy). Letters in Spatial
directional distance function approach. J. Environ. Manag. 51, 229–240.
and Resource Sciences 10, 337–352.
Correia, T., Marques, R.C., 2010. Performance of Portuguese water utilities: how do
Garcìa, I., Mendoza, R., Pomar, M.C., 2008. Deficit and excess of soil water impact on
ownership, size, diversification and vertical integration relate to efficiency? Water
plant growth of lotus tenuis by affecting nutrient uptake and arbuscular mycorrhizal
Pol. 13, 343–361.
symbiosis. Plant Soil 304, 117–131.
da Silva, J., Fernandes, V., Limont, M., Dziedzic, M., Andreoli, C., Rauen, W., 2020.
Geelen, L.M., Huijbregts, M.A., den Hollander, H., Ragas, A.M., van Jaarsveld, H.A., de
Water sustainability assessment from the perspective of sustainable development
Zwart, D., 2009. Confronting environmental pressure, environmental quality and
capitals: concep- tual model and index based on literature review. J. Environ.
human health impact indicators of priority air emissions. Atmos. Environ. 43,
Manag. 254, 109750.
1613–1621.
Dahl, A.L., 2012. Achievements and gaps in indicators for sustainability. Ecol. Indicat.
Giannouli, M., Samaras, Z., Keller, M., DeHaan, P., Kallivoda, M., Sorenson, S.,
17, 14–19.
Georgakaki, A., 2006. Development of a database system for the calculation of
Daraio, C., Simar, L., 2005. Introducing environmental variables in nonparametric
indicators of environmental pressure caused by transport. Sci. Total Environ. 357,
frontier models: a probabilistic approach. J. Prod. Anal. 24, 93–121.
247–270.
Daraio, C., Simar, L., 2007. Advanced Robust and Nonparametric Methods in Efficiency
Gonçalves, I., Alves, D., Robalo, G., 2014. Social tariffs for water and waste services in
Analysis: Methodology and Applications. Springer Science & Business Media.
mainland Portugal: an impact analysis. Water Sci. Technol. Water Supply 14,
Dardha, E., Rogge, N., 2020. How’s life in your region? measuring regional material
513–521.
living conditions, quality of life and subjective well-being in oecd countries using a
Gonzàlez-Benito, J., Gonzàlez-Benito, Ó., 2010. A study of determinant factors of
robust, conditional benefit-of-the-doubt model. Soc. Indicat. Res. 151, 1015–1073.
stakeholder environmental pressure perceived by industrial companies. Bus. Strat.
Das, S., Lee, S.H., Kumar, P., Kim, K.H., Lee, S.S., Bhattacharya, S.S., 2019. Solid waste
Environ. 19, 164–181.
management: scope and the challenge of sustainability. J. Clean. Prod. 228,
Goodland, R., 1995. The concept of environmental sustainability. Annu. Rev. Ecol.
658–678.
Systemat. 26, 1–24.
De Witte, K., Rogge, N., 2011. Accounting for exogenous influences in performance
Henriques, A.A., Camanho, A.S., Amorim, P., Silva, J.G., 2020. Performance
evaluations of teachers. Econ. Educ. Rev. 30, 641–653.
benchmarking using composite indicators to support regulation of the Portuguese
De Witte, K., Cabus, S., Thyssen, G., Groot, W., Maassen van den Brink, H., 2013.
wastewater sector. Util. Pol. 66, 101082.
A critical review of the literature on school dropout. Educ. Res. Rev. 10, 13–28.
Horta, I.M., Camanho, A.S., 2015. A nonparametric methodology for evaluating
Degli Antoni, G., Marzetti, G.V., 2019. Recycling and waste generation: an estimate of
convergence in a multi-input multi-output setting. Eur. J. Oper. Res. 246, 554–561.
the source reduction effect of recycling programs. Ecol. Econ. 161, 321–329.
Juwana, I., Muttil, N., Perera, B., 2012. Indicator-based water sustainability
Dong, Y., Hauschild, M.Z., 2017. Indicators for environmental sustainability. Procedia
assessment—a review. Sci. Total Environ. 438, 357–371.
CIRP 61, 697–702.
Kaza, S., Yao, L., Bhada-Tata, P., Van Woerden, F., 2018. What a Waste 2.0: a Global
Dyson, R., Allen, R., Camanho, A., Podinovski, V., Sarrico, C., Shale, E., 2001. Pitfalls and
Snapshot of Solid Waste Management to 2050. World Bank Publications.
protocols in DEA. Eur. J. Oper. Res. 132, 245–259.

12
A. Mergoni et al. Utilities Policy xxx (xxxx) xxx

Lavigne, C., Jaeger, S.D., Rogge, N., 2019. Identifying the most relevant peers for Purvis, B., Mao, Y., Robinson, D., 2019. Three pillars of sustainability: in search of
benchmark- ing waste management performance: a conditional directional distance conceptual origins. Sustainability science 14, 681–695.
Benefit-of-the-Doubt approach. Waste Manag. 89, 418–429. Rogge, N., 2012. Undesirable specialization in the construction of composite policy
Li, Q., Racine, J., 2003. Non-parametric estimation of distributions with categorical and indicators: the environmental performance index. Ecol. Indicat. 23, 143–154.
contin- uous data. J. Multivariate Anal. 86, 266–292. Rogge, N., De Jaeger, S., Lavigne, C., 2017. Waste performance of NUTS 2-regions in the
Liang, S., Liu, Z., Crawford-Brown, D., Wang, Y., Xu, M., 2014. Decoupling analysis and eu: a conditional directional distance benefit-of-the-doubt model. Ecol. Econ. 139,
socioeconomic drivers of environmental pressure in China. Environ. Sci. Technol. 48, 19–32.
1103–1113. Romano, G., Guerrini, A., 2011. Measuring and comparing the efficiency of water utility
Lombardi, G., Stefani, G., Paci, A., Becagli, C., Miliacca, M., Gastaldi, M., Giannetti, B., com- panies: a data envelopment analysis approach. Util. Pol. 19, 202–209.
Almeida, C., 2019. The sustainability of the Italian water sector: an empirical Sala, S., Ciuffo, B., Nijkamp, P., 2015. A systemic framework for sustainability
analysis by DEA. J. Clean. Prod. 227, 1035–1043. assessment. Ecol. Econ. 119, 314–325.
Margallo, M., Taddei, M.B.M., Hernández-Pellón, A., Aldaco, R., Irabien, A., 2015. Sarra, A., Mazzocchitti, M., Rapposelli, A., 2017. Evaluating joint environmental and cost
Environmen- tal sustainability assessment of the management of municipal solid performance in municipal waste management systems through data envelopment
waste incineration residues: a review of the current situation. Clean Technol. analysis: scale effects and policy implications. Ecol. Indicat. 73, 756–771.
Environ. Policy 17, 1333–1353. Serra, J., Cordovil, C.M., Cruz, S., Cameira, M.R., Hutchings, N.J., 2019. Challenges and
Marques, R.C., da Cruz, N.F., Pires, J., 2015. Measuring the sustainability of urban water solu- tions in identifying agricultural pollution hotspots using gross nitrogen
services. Environ. Sci. Pol. 54, 142–151. balances. Agric. Ecosyst. Environ. 283, 106568.
Marques, R.C., Simões, P., Pinto, F.S., 2018. Tariff regulation in the waste sector: an Serra, J., do Rosàrio Cameira, M., Cordovil, C.M., Hutchings, N.J., 2021. Development of
unavoid- able future. Waste Manag. 78, 292–300. a groundwater contamination index based on the agricultural hazard and aquifer
Marques, R.C., Simões, P., 2020. Revisiting the comparison of public and private water vulnerability: application to Portugal. Sci. Total Environ. 772, 145032.
service provision: an empirical study in Portugal. Water 12, 1477. Silva, C., Rosa, M.J., 2020. Performance assessment of 23 wastewater treatment plants-a
Martins, R., Antunes, M., Fortunato, A., 2020. Regulatory changes to Portugal’s social case study. Urban Water J. 17, 78–85.
tariffs: carrying water in a sieve? Util. Pol. 64, 101038. Simões, P., Cavalho, P., Marques, R.C., 2013. The market structure of urban solid waste
Melyn, W., Moesen, W., 1991. Towards a synthetic indicator of macroeconomic services: how different models lead to different results. Local Govern. Stud. 39,
performance: unequal weighting when limited information is available. Public 396–413.
economics research papers 1–24. Simões, P., Marques, R.C., 2012. On the economic performance of the waste sector. A
Moldan, B., Janoušková, S., Hák, T., 2012. How to understand and measure literature review. J. Environ. Manag. 106, 40–47.
environmental sustainability: indicators and targets. Ecol. Indicat. 17, 4–13. Smeets, E., Weterings, R., 1999. Environmental Indicators: Typology and Overviewe, vol.
Molinos-Senante, M., Marques, R.C., Pèrez, F., Gòmez, T., Sala-Garrido, R., Caballero, R., 25. Technical report, pp. 411–432.
2016. Assessing the sustainability of water companies: a synthetic indicator Sutton, P., 2004. A perspective on environmental sustainability. Paper on the Victorian
approach. Ecol. Indicat. 61, 577–587. Com- missioner for Environmental Sustainability 1–32.
Molinos-Senante, M., Gòmez, T., Caballero, R., Sala-Garrido, R., 2017. Assessing the UN General Assembly, 2015. Transforming Our World: the 2030 Agenda for Sustainable
quality of service to customers provided by water utilities: a synthetic index Devel- Opment (New York: United Nations).
approach. Ecol. Indicat. 78, 214–220. Vilanova, M.R.N., Magalh~aes Filho, P., Balestieri, J.A.P., 2015. Performance
Munksgaard, J., Wier, M., Lenzen, M., Dey, C., 2005. Using input-output analysis to measurement and indicators for water supply management: review and international
measure the environmental pressure of consumption at different spatial levels. cases. Renew. Sustain. Energy Rev. 43, 1–12.
J. Ind. Ecol. 9, 169–185. Walter, M., Cullmann, A., von Hirschhausen, C., Wand, R., Zschille, M., 2009. Quo vadis
Nardo, M., Saisana, M., Saltelli, A., Tarantola, S., Hoffman, A., Giovannini, E., 2005. efficiency analysis of water distribution? a comparative literature review. Util. Pol.
Handbook on Constructing Composite Indicators, vol. 3. OECD Statistics Working 17, 225–232.
Papers. Worthington, A.C., 2014. A review of frontier approaches to efficiency and productivity
Nikolaou, K., 2001. Planning of an environmental pressure indicators system in a mea- surement in urban water utilities. Urban Water J. 11, 55–73.
metropolitan area. Journal of Environmental Protection and Ecology 2, 595–602. Zanella, A., Camanho, A.S., Dias, T.G., 2015. Undesirable outputs and weighting schemes
OECD, 2020. Environment at a glance indicators. https://www.oecd-ilibrary.org/content in composite indicators based on data envelopment analysis. Eur. J. Oper. Res. 245,
/publication/ac4b8b89-en. https://doi.org/10.1787/ac4b8b89-en. 517–530.
OECD Environment Directorate, 2008. OECD Key Environmental Indicators. Zeller, V., Lavigne, C., D’Ans, P., Towa, E., Achten, W., 2020. Assessing the
Organisation for Economic Development and Cooperation, Paris, France. environmental performance for more local and more circular biowaste management
Pèrez, F., Delgado-Antequera, L., Gòmez, T., 2019. A two-phase method to assess the sus- options at city-region level. Sci. Total Environ. 745, 140690.
tainability of water companies. Energies 12, 2638. Zhou, P., Ang, B., Poh, K., 2007. A mathematical programming approach to constructing
Pèrez, F., Molinos-Senante, M., Gòmez, T., Caballero, R., Sala-Garrido, R., 2018. composite indicators. Ecol. Econ. 62, 291–297.
Dynamic goal programming synthetic indicator: an application for water companies Zhu, G., Tang, Z., Shangguan, Z., Peng, C., Deng, L., 2019. Factors affecting the spatial
sustainability as- sessment. Urban Water J. 15, 592–600. and temporal variations in soil erodibility of China. J. Geophys. Res.: Earth Surface
Pinto, F.S., Costa, A.S., Figueira, J., Marques, R.C., 2017. The quality of service: an 124, 737–749.
overall performance assessment for water utilities. Omega 69, 115–125.

13