International Journal of

Environmental Research
and Public Health

Article
System Approaches to Water, Sanitation, and Hygiene:
A Systematic Literature Review
Nicholas Valcourt 1,2, * , Amy Javernick-Will 1,2 , Jeffrey Walters 2,3 and Karl Linden 1,2
1 Department of Civil, Environmental, and Architectural Engineering, University of Colorado Boulder,
Boulder, CO 80309, USA; amy.javernick@colorado.edu (A.J.-W.); Karl.Linden@colorado.edu (K.L.)
2 USAID Sustainable WASH Systems Learning Partnership, United State Agency for International
Development, Washington, DC 20004, USA; jwalters@georgefox.edu
3 College of Engineering, George Fox University, Newberg, OR 97132, USA
* Correspondence: nicholas.valcourt@colorado.edu




Received: 4 December 2019; Accepted: 14 January 2020; Published: 21 January 2020


Abstract: Endemic issues of sustainability in the water, sanitation, and hygiene (WASH) sector have
led to the rapid expansion of ‘system approaches’ for assessing the multitude of interconnected factors
that affect WASH outcomes. However, the sector lacks a systematic analysis and characterization of
the knowledge base for systems approaches, in particular how and where they are being implemented
and what outcomes have resulted from their application. To address this need, we conducted a
wide-ranging systematic literature review of systems approaches for WASH across peer-reviewed,
grey, and organizational literature. Our results show a myriad of methods, scopes, and applications
within the sector, but an inadequate level of information in the literature to evaluate the utility
and efficacy of systems approaches for improving WASH service sustainability. Based on this
analysis, we propose four recommendations for improving the evidence base including: diversifying
methods that explicitly evaluate interconnections between factors within WASH systems; expanding
geopolitical applications; improving reporting on resources required to implement given approaches;
and enhancing documentation of effects of systems approaches on WASH services. Overall, these
findings provide a robust survey of the existing landscape of systems approaches for WASH and
propose a path for future research in this emerging field.

Keywords: WASH; systems approaches; systematic literature review; grey literature

1. Introduction
Despite substantial gains in access to water, sanitation, and hygiene (WASH) services over the
past thirty years, it is estimated that 2.3 billion people worldwide still lack access to these basic
human necessities [1]. For those who have gained access, monitoring trends show that over time,
WASH services consistently fail to function as intended [2,3]. This service sustainability issue persists
despite a highly uniform and proven set of technologies and approaches that have been developed
over the decades within the WASH sector [4]. In Sub-Saharan African countries alone, up to 70%
of rural water schemes are estimated to be non-functional or intermittently functional at any given
time [5]. Worldwide, low service sustainability contributes to nearly two million preventable deaths
and 82 million disability-adjusted life years (DALYs) each year attributed to WASH [6]. These impacts
disproportionately affect communities that are rural, poor, and resource-limited [7]. Accordingly,
service sustainability is identified as a key challenge to be addressed in order for investments in WASH
hardware and software to deliver their intended public health impacts [8,9].
Within the WASH sector there is a growing acknowledgement that existing approaches, which
have traditionally focused on the installation and maintenance of hardware (e.g., hand pumps, latrines,

Int. J. Environ. Res. Public Health 2020, 17, 702; doi:10.3390/ijerph17030702 www.mdpi.com/journal/ijerph
Int. J. Environ. Res. Public Health 2020, 17, 702 2 of 18

hand-washing stations) and community-based management models [10–12], will not be sufficient on
their own to meet the universal access and service targets of Sustainable Development Goal (SDG) #6:
ensuring availability and sustainable management of water and sanitation for all [13–18]. Given the
interconnected nature of financial, institutional, environmental, technological, and social factors that
influence sustained service delivery [19], the sector has started to advocate for a ‘systems’ approach
to sustain WASH service delivery [20–22]. Central to this approach is the idea that challenges to the
sustainability of WASH services are not due to the weakness of any single factor (e.g., cost recovery),
but rather the collective effect of a wide range of interacting factors. Thus, improving the sustainability
of WASH services requires an enhanced understanding of the combined effect that a multitude of
factors exert service delivery outcomes [23–25].
This systems perspective of WASH is embraced by many in the sector and has led to a rapid
expansion in the number of tools, frameworks, and approaches for understanding the interconnected
factors of WASH services [26] including collective action coalitions [27], multi-criteria decision
analysis [28], market-based approaches [29], soft systems methodologies [30], composite scoring [31],
and system dynamics [32], among others. Indeed a recently published book focused solely on
presenting systems approaches to WASH for practitioners [33]. However, to better understand the
systems approaches being applied in WASH, there is a need to characterize and synthesize the literature,
including the methods, approaches, and applications used. While others have conducted reviews of
systems approaches for international development at large [34–36], and within the WASH sector in
particular [21,37], these studies were not intended to be either comprehensive or systematic in nature,
and instead focused on a particular method, approach or application. This lack of understanding of
the existing landscape of different methods for systems approaches for WASH may ultimately restrict
their use and uptake across the sector, especially for local practitioners who have limited resources to
implement complex approaches.

Study Objectives
To address this need, we seek to characterize the breadth of methods employed for WASH systems
approaches, the use of these methods in WASH projects, the impacts these approaches have on service
outcomes, as well as identify key knowledge gaps in the existing knowledge base. For the purposes of
this study, we define a WASH system as a collection of all of the factors and their interactions which
influence WASH service delivery within a given contextual, institutional or geopolitical boundary. We
conceptualize factors as any tangible or abstract element, aspect or component thought to directly
or indirectly influence the WASH system. Examples include finances, hardware, actors, gender, and
socio-economic conditions, among others.
To address these objectives, this study proposes four research questions (RQs): (RQ 1) What
methods (i.e., analyses) are being employed for understanding and engaging with WASH systems?;
(RQ 2) In what contexts (i.e., geographic and programmatic) are these approaches being applied?;
(RQ 3) What evidence exists that these systems approaches improve the sustainability of WASH
services?; (RQ 4) What gaps exist in the current knowledge base for systems approaches to WASH?

2. Materials and Methods

To address the study objectives, we conducted a systematic literature review based on best
practices for research synthesis by Cooper et al. [38] and the Preferred Reporting Items for Systematic
Literature Reviews and Meta-Analyses (PRISMA) statement to structure the review [39,40]. The
PRISMA checklist used for this review is included in Table S1. Due to the nature of the study, we did
not seek to include an assessment of effects and effect sizes or the quality of evidence.

2.1. Literature Identification

We identified a diverse set of 19 databases containing peer-reviewed, grey, and organizational
literature on WASH systems approaches. We use the term organizational to refer to literature written
Int. J. Environ. Res. Public Health 2020, 17, 702 3 of 18

and published directly by an organization (e.g., United Nations International Children’s Emergency
Fund (UNICEF)); in contrast, we use grey to refer to any literature that is not peer-reviewed and does
not represent the work of an individual organization (e.g., books, compendiums, reports). A description
of the databases and rationale for inclusion is presented in (Table S2-1). A preliminary review of WASH
systems literature and past assessments of systems tools within WASH [41,42] revealed that much of the
information about systems approaches to WASH existed in grey literature sources and organizations’
repositories, outside of a peer-reviewed journal database. To ensure that applicable literature from
grey and organizational sources were included, additional records were obtained from bibliographic
hand searches and expert consultations. This included reviewing works cited in applicable literature,
conference proceedings, newsletters, and blogs related to systems approaches for WASH.
Based on pilot search results, we noted ambiguity around what constitutes a ‘systems approach’
for WASH; thus, searching for records based on this term alone, or using it as a screening criterion,
would not have represented the full breath of literature relevant to our review. To address this, we
developed a more inclusive search strategy that contained two primary elements: keywords for WASH
and systems. We reviewed indices and key terms from seminal texts on systems-thinking topics to
identify keywords. This produced a list of over 400 keywords which were affinity grouped into a
representative list of 18 systems keywords (Table S2-2) based on comparable terms and frequency of
terms. Combining these with select keywords for WASH and select wildcard operators to allow for
similar versions of the term, generated the search string formula used in the literature search. Search
queries were time-limited to only return documents published after 2000 (the year the Millennium
Development Goals (MDGs) were adopted) and before April 2019. Database searches were conducted
in August 2018 and repeated in March 2019.

2.2. Selection of Articles and Data Extraction

To characterize the breadth of methods and approaches for evaluating WASH systems, a two-step
screening process was developed. In the initial screening, papers were vetted for their applicability
to WASH and WASH systems. In the secondary screening, studies were reviewed for three
attributes—method, scope and application—that were developed to directly address the review’s
objectives (RQ1–3, respectively).

2.2.1. Initial Screening

For consistency, the search strategy was implemented across all databases and extracted titles,
abstracts, and identifier (Digital Object Identifier (DOI) or other) for each study. All studies were then
given a new unique identifier based on the databases from which they were retrieved, and the results
were compiled into a single excel spreadsheet. Records obtained through hand searches [38], following
best practices and examples for the use of grey literature in WASH meta-analyses [43,44], were also
added to this database with a unique numeric identifier.
After removing duplicate titles, each study was screened for its applicability to WASH and WASH
systems based on the title and content of the abstract. For a study to be included in the full text review,
it had to have an explicit focus on a water, sanitation or hygiene topic or the WASH sector overall.
Examples included rural water delivery, fecal sludge management or sector financing. To assist with
this screening process, multiple exclusion terms were developed to remove studies that focused on
water or wastewater treatment, hydrology, water resource management, and topics bounded more
specifically to public health. Of these exclusion criteria, water resource management topics, especially
integrated water resource management (IWRM) approaches, had many of the same qualities as systems
approaches to WASH. However, after further review, it was determined that the IWRM literature was
distinct enough from approaches applied to WASH services that it was prudent to exclude it from
the review. Additionally, WRM and IWRM systems approaches have previously been addressed in
literature reviews by others [45–47].
Int. J. Environ. Res. Public Health 2020, 17, 702 4 of 18

To ensure that studies were applicable to systems approaches, full text records were also screened
to confirm that they referenced two or more factors of the WASH system (e.g., finances, governance,
hardware, etc.). While meta-analyses or other reviews were excluded, we retained full texts of these
studies to identify other studies cited in the literature to determine if they met the eligibility criteria. If
applicable, these studies were included in the ‘hand search’ category. Exclusion terms and criteria for
both applicability to WASH and WASH systems are presented in (Table S2-3).

2.2.2. Secondary Screening

To accurately characterize the existing landscape of systems approaches to WASH services, the
review examined three key attributes of each study that met the primary screening criteria above:
method (RQ1), scope (RQ2), and application (RQ3). RQ4, or gaps in knowledge, was addressed
through a review of results from all attributes. To assess the qualities of each study in regard to the
research questions, a set of study descriptors or important dimensions of the attribute, were developed.
Each descriptor represents an important dimension of the attribute and was developed based on
comparable literature reviews of WASH and systems-thinking for development topics [35,48] as well
as systematic coding procedures outlined by Cooper et al. [38]. Each attribute is described below with
descriptors and evaluation criteria presented in a coding form in Table S3-1. All secondary screening
was conducted by the first author.

Method Attribute
The review deductively coded information relevant to the methodology attribute into the following
descriptors for each study: methods, data sources, factors, interactions, analytical complexity, and how
the method was applied in the study (Table 1). Within each descriptor, we used both inductive and
deductive coding techniques to characterize evaluation criteria.

Table 1. Descriptors and criteria for assessing the method attribute.

Method Descriptors Criteria

Stated name of method of analysis or approach employed by the study (open
Method 2
response) and open coding of methodologies.
Open coding of source of data for the study (e.g., surveys, interviews, focus
Data Sources 2
groups, observations, etc.).
Open coding of factors included in the study and count (e.g., finance,
Factors 2
hardware, regulations).
Interactions 1 Evaluates if the study considers interactions among factors (coded as yes/no).
Low: Non-computational tasks can be completed without specialized
knowledge or training.
Medium: Some specific knowledge or training is required to complete
Analytical Complexity 1
computational tasks.
High: Process requires a high level of specialized knowledge or training to
conduct the analysis.
Analysis: Application of an established analytical method, presented without
a broader theory on how it should be applied, or steps for applying it (e.g.,
statistical regression of survey data).
Tool: A discrete, standalone activity or analysis presented with sufficient
detail to be readily replicated, or an analytical program or software (e.g.,
Method Application 1
checklist for assessing service sustainability).
Framework: A guiding outline of activities, analyses or procedures for
applying the method (e.g., list of principles for engaging in local systems [49]).
Approach: A theoretical or conceptual construct, without discrete steps for
implementation of the method (e.g., collective impact/action).
1 a priori, deductive criteria; 2 open or emergent criteria.
Int. J. Environ. Res. Public Health 2020, 17, 702 5 of 18

(1) Methods Coding

To capture the range of methods being employed for systems approaches to WASH, we identified
the methods for each study by recording both the proper name(s) of the method used by the authors
in the text, and inductively coding methods into categories of methodologies. For example, a study
that employed a linear regression model would be coded into a ‘statistics’ category while a study that
used stock-and-flow models would be coded into a ‘system dynamics’ category. Codes for specific
methods were developed for each set of methods that was represented by at least two or more studies.
A codebook for the resulting descriptor categories is presented in Table S3.

(2) Data Sources

To classify the sources of data that the included studies draw on, we coded sources, primarily from
data collection descriptions, into common categories including primary and secondary data, interviews,
and surveys, among others. Sources of data were coded to best represent relevant categories for each
study; for example, one study that described data collection as ‘surveys administered through face to
face interviews’ was coded under ‘Surveys’ to represent the primary nature of the data collection.

(3) Factors

To identify the scope of the different elements, aspects, and components of the WASH systems that
the included studies evaluated, we openly coded factor categories based on descriptions of variables
presented in each study. For example, studies that examined the relationships between willingness to
pay and gender of the head of household would be coded as financial and gender, respectively, within
the factors descriptor.

(4) Interactions

As factor interaction is a fundamental concept of systems thinking approaches [50–53], we also

evaluated whether or not the study explicitly analyzed interactions or relationships between the factors
included in the study. Interaction treatment was evaluated as a yes/no descriptor based directly on the
analytical components of the method as described by the authors.

(5) Analytical Complexity

To characterize the complexity of these approaches within the WASH sector, we sought to
assess the analytical complexity of each method. As a direct comparison of the multitude and wide
variety of anticipated methods did not appear feasible, analytical complexity was evaluated using a
low-medium-high rubric based on the degree of specialized knowledge or training required to conduct
the analysis as described in Table 1.

(6) Method Application

To evaluate how different methods are applied with each study, we categorized method application
into four pre-determined categories (Table 1) ranging from a direct application of an established method
(analysis) to a theoretical or conceptual framing of how the WASH topic could be evaluated (approach).
A preliminary review of studies prior to the structured literature search revealed that studies could
contain both multiple methods and multiple applications of those methods, and thus neither of these
criteria were assumed to be mutually exclusive. This resulted in some studies being classified under
multiple method applications.
Int. J. Environ. Res. Public Health 2020, 17, 702 6 of 18

Scope Attribute
For RQ2, we evaluated and coded the scope of each study into three study descriptors: aspects of
WASH addressed, the focal outcomes of the study, and the contexts in which the approach was applied,
if any (Table 2). Taken together, these three coding categories will help to provide more clarity on what
aspects of WASH, and in what contexts, systems approaches are being prioritized in the emerging
systems literature.

Table 2. Descriptors criteria for assessing the scope attribute.

Scope Descriptors Criteria

Aspects of WASH 1 Water, sanitation, and/or hygiene
Emergent coding of study outcomes or dependent variable (e.g.,
Outcome Scope 2
behavior change, access to services, service sustainability, etc.)
Rural/urban
Context 1 Local (including community), city, regional (including district), national
sector

1 a priori, deductive criteria; 2 open or emergent criteria.

(1) Aspects of WASH

The application of the approach to water, sanitation, and/or hygiene was assessed directly based
on the described scope of the study. Categories for this descriptor were not deemed to be mutually
exclusive as the scope of the study could include more than one aspect of WASH (i.e., water and
sanitation, sanitation and hygiene).

(2) Outcome Scope

A direct comparison of study outcomes and dependent variables was not feasible; thus, we
emergently coded outcomes into descriptive categories. For example, a study that focused on promoting
good governance and finance for water system operations would be coded as ‘Sustainability’ as it
addresses factors related to the ongoing operation of an existing water delivery scheme. Conversely,
studies which focused on the sale of new latrines would be coded as expanding ‘Access’ to services.

(3) Context

Context was coded using two a priori, deductive coding categories that assessed context based
on both a rural/urban classification and by administrative designations of community, city, regional
(including district), and national. A ‘Sector’ category was also included for studies which focused on
sector-wide issues across multiple geographies.

Application Attribute
The third attribute examines the application of the study to a WASH program or project, including
the phase of project implementation, the geographic location, as well as any reported impacts that
occurred as a result of the study (Table 3).
Int. J. Environ. Res. Public Health 2020, 17, 702 7 of 18

Table 3. Descriptors and criteria for assessing the application attribute.

Application Descriptors Criteria

Assessment of what stage, if any, the study was implemented in relation
to a WASH infrastructure or services project:
Planning—Used to plan future WASH service delivery project;
Implementation—Method used to guide the implementation of a WASH
Project Application 1 project;
Evaluation—Post-project assessment of outcomes;
Case Study—Standalone analysis of a specific case(s), not related to a
WASH project;
None—No project application conducted.
Name and count of geographic locations where study was applied
Location 1
(Based on UN Geoscheme [54].
Stated outcomes or effects on WASH services that occurred as a result of
Reporting of Impacts 2
the application of the study (if any).
1 a priori, deductive criteria; 2 open or emergent criteria.

(1) Project Application

The project application descriptor was developed to assess what aspects of WASH projects systems
approaches are being used to, or proposed to, investigate. We evaluated project application based on a
priori codes of project phases including planning (before), implementation (during), and evaluation
(after). Studies that evaluated WASH services or factors to support services but were not specific to any
aspect of an implementation project were classified as case studies. Any studies that were not related
to a specific project were labeled as none. Similar to the method application, the project application
descriptor was not assumed to be mutually exclusive. For example, a project that evaluated factors
that were critical to project implementation, and drew on multiple cases, would be categorized as both
implementation and case studies.

(2) Location

To assess where systems approaches are being implemented, we recorded the specific location(s)
of each study, where applicable, to assess the geographic distribution of study locations. Studies did
not need to be integral to project implementation in order to be assessed for the location descriptor. If a
study specified that work was conducted in multiple districts or regions within a country, the count
and names of the locations were recorded as well. In classifying locations, we used the United Nations
geoscheme for regions and sub-regions globally [54].

(3) Reporting of Impacts

To address RQ3 (What evidence exists for systems approaches?) we sought to capture all reporting
of impacts on services or other factors of the WASH system (e.g., policy, behavior change) that
occurred due to the use of the analysis, tool, framework or approach implemented in the study. In
collecting information on study impacts, we did not seek to assess the quality of evidence presented in
reporting those impacts. Instead, we recorded whether or not impacts were reported, and captured the
descriptions of the impacts as described by the authors.
In addition to the three study attributes and descriptors defined above, the review also captured
report characteristics for each study, including author(s), journal, year published, whether the study
represented a peer-reviewed, grey or organizational literature source, and if the study was available
as open-access.
Int. J. Environ. Res. Public Health 2020, 17, 702 8 of 18

3. Results

3.1. Literature Search Results

Through our search, we obtained 8139 articles from databases searches and 100 from hand searches
that were potentially eligible for the review. Of these articles, 472 were eliminated because they were
duplicates, resulting in 7767 articles eligible for primary screening (Figure 1). After reviewing articles
for their applicability to WASH and treatment of systems as described in Section 2.1, 584 full text
articles were obtained for secondary screening to determine if enough information was available on
our study’s descriptors. Of these articles, 451 were excluded because they either did not have sufficient
information to address the study descriptors (e.g., research briefs from grey literature) or because the
full text review indicated they did not meet the initial screening criteria. This resulted in 133 studies
included in the final review. A majority of the studies (75) were classified as peer-reviewed literature,
35 were classified as grey literature, and 23 as organizational literature. Full results of the review,
including a breakdown of descriptors by literature type, is presented in Table S4. Unless otherwise
specified, all percentages presented in the following sections are expressed in regard to the total count
of
Int.studies = 133).
(nRes.
J. Environ. Public Health 2020, 17, x FOR PEER REVIEW 8 of 18

Figure 1. Preferred Reporting Items for Systematic Literature Reviews and Meta-Analyses (PRISMA)
flowchart
flowchart with
with selection
selection of
of articles
articles included
included in
in this
this review.
review.

3.2. Method Assessment (RQ1)

3.2. Method Assessment (RQ1)
Overall, we identified 41 unique methods in the peer-reviewed, grey, and organizational studies.
Overall, we identified 41 unique methods in the peer-reviewed, grey, and organizational studies.
These methods could be employed non-exclusively (i.e., papers could be categorized to more than one
These methods could be employed non-exclusively (i.e., papers could be categorized to more than
method). Of these methods, unique frameworks (e.g., sanitation cityscape conceptual framework [55]),
one method). Of these methods, unique frameworks (e.g., sanitation cityscape conceptual framework
composite scoring, qualitative data analysis, and statistical methods were employed most frequently
[55]), composite scoring, qualitative data analysis, and statistical methods were employed most
(Figure 2). A breakdown of methods by literature source indicates that each of the top three methods
frequently (Figure 2). A breakdown of methods by literature source indicates that each of the top
(frameworks, composite scoring, and qualitative data analysis) were most prevalent in the grey
three methods (frameworks, composite scoring, and qualitative data analysis) were most prevalent
and organizational literature; whereas a majority of the statistical, social network analysis, system
in the grey and organizational literature; whereas a majority of the statistical, social network analysis,
dynamics, and causal loop diagraming methods were the most prevalent methods applied in the
system dynamics, and causal loop diagraming methods were the most prevalent methods applied in
peer-reviewed literature.
the peer-reviewed literature.
When the studies were assessed for analytical complexity, results indicated a relatively even
distribution of high complexity (38%) (e.g., statistics, network analysis, system dynamics), medium
complexity (37%) (e.g., qualitative data analysis, composite scoring), and low complexity (25%) (e.g.,
frameworks, checklists). Surprisingly for a review of systems approaches, only 35 studies (26%)
included a method that explicitly analyzed interactions between factors that were considered in the
analysis (e.g., causal loop diagraming, network analysis). Of these, 26 (81%) were from the peer-
reviewed literature.
Int. J. Environ. Res. Public Health 2020, 17, 702 9 of 18
Int. J. Environ. Res. Public Health 2020, 17, x FOR PEER REVIEW 9 of 18

Figure 2. Relative
Figure 2. Relative frequency
frequency ofof the
the 15
15 most
most frequently
frequently used
used methods
methods byby literature
literature source,
source, listed
listed in
in
descending
descending order
orderofoffrequency
frequencyofof
total references
total from
references all literature
from sources
all literature (peer-reviewed
sources = 75; =
(peer-reviewed grey
75;
35, =
=grey organizational = 23).= 23).
(org) (org)
35, organizational

Our
When review of sources
the studies wereofassessed
data, another non-exclusive
for analytical descriptor,
complexity, indicated
results indicatedthata secondary
relatively evendata
(53%) (e.g., government
distribution of high complexity reports,(38%)
national
(e.g.,surveys,
statistics,etc.) and interviews
network (47%) dynamics),
analysis, system were the two most
medium
common
complexity sources. Thesequalitative
(37%) (e.g., were followed by primary
data analysis, data (28%),
composite observations
scoring), and low (276%),
complexity surveys
(25%) (24%),
(e.g.,
focus groups checklists).
frameworks, (23%), expert opinions (21%),
Surprisingly and workshops
for a review of systems (5%). Notably, only
approaches, secondary data (26%)
35 studies (e.g.,
national
includedsurveys,
a method document review)analyzed
that explicitly were used in a majority
interactions (83%) factors
between of studies
thatinwere
the organizational
considered in
literature,
the analysis the(e.g.,
mostcausal
of any loop
data source by literature
diagraming, network type.
analysis). Of these, 26 (81%) were from the
Forty unique
peer-reviewed factors were referenced by at least two or more studies from our factor coding
literature.
(Figure
Our3). Of these,
review Financial
of sources (74%)
of data, was overwhelmingly
another the most indicated
non-exclusive descriptor, common thatfactor, followeddata
secondary by
Technical
(53%) (53%),
(e.g., Institutional
government (43%),
reports, Social (41%),
national surveys,and Environmental
etc.) and interviews (40%)(47%)
factors.
were However,
the two these
most
factors
common were not equally
sources. These werestudied across by
followed theprimary
different literature
data types; Financial
(28%), observations factors
(276%), (e.g., tariffs)
surveys (24%),
were
focus nearly
groups ubiquitous
(23%), expert inopinions
the organizational
(21%), and (91%) and grey
workshops (5%).(89%) literature
Notably, thandata
secondary they(e.g.,
were in the
national
peer-reviewed
surveys, document literature
review)(61%).
were Conversely, Economic(83%)
used in a majority factorsof (e.g.,
studiesmarkets) were twice as frequently
in the organizational literature,
in
thethe peer-reviewed
most of any data sourceliterature
by (40%) thantype.
literature the grey or organizational literature (both 17%). Across all
literature
Fortysources, the average
unique factors number of by
were referenced factors each
at least twostudy
or more examined
studieswasfrom7.9,ourwith
factorthecoding
most
common3).count
(Figure of factors
Of these, being
Financial five was
(74%) (17%), six (15%), eight
overwhelmingly the(14%), and seven
most common (11%)followed
factor, factors, byin
descending
Technical orderInstitutional
(53%), of frequency. (43%), Social (41%), and Environmental (40%) factors. However, these
factors were not equally studied across the different literature types; Financial factors (e.g., tariffs)
were nearly ubiquitous in the organizational (91%) and grey (89%) literature than they were in the
peer-reviewed literature (61%). Conversely, Economic factors (e.g., markets) were twice as frequently in
the peer-reviewed literature (40%) than the grey or organizational literature (both 17%). Across all
literature sources, the average number of factors each study examined was 7.9, with the most common
count of factors being five (17%), six (15%), eight (14%), and seven (11%) factors, in descending order
of frequency.
were nearly ubiquitous in the organizational (91%) and grey (89%) literature than they were in the
peer-reviewed literature (61%). Conversely, Economic factors (e.g., markets) were twice as frequently
in the peer-reviewed literature (40%) than the grey or organizational literature (both 17%). Across all
literature sources, the average number of factors each study examined was 7.9, with the most
common
Int. count
J. Environ. of factors
Res. Public being
Health 2020, five (17%), six (15%), eight (14%), and seven (11%) factors,
17, 702 10 of in
18
descending order of frequency.

Figure 3. Relative frequency of factors referenced in at least 20% of all studies listed in descending order
of frequency of total references from all literature sources (peer-reviewed = 75; grey = 35; organizational
(org) = 23).

Our assessment of method application, a non-exclusive category (multiple descriptors could

apply to a study), indicated that methods were used nearly equally as tools (35%), frameworks (34%),
analyses (32%), and approaches (26%). However, as with methods and factors, notable differences exist
across the literature sources, with nearly half of all studies from the organizational literature proposing
a novel tool (52%), and a smaller proportion of grey and organizational studies presenting a novel
framework of 40% and 43%, respectively. In the peer-reviewed literature, we observed that a large
proportion of studies (39%) represented methods used for one-off case study analyses.

3.3. Scope Assessment (RQ2)

Results of coding the scope of the studies showed that the most focused on or were intended for,
were eater services (41%) followed by sanitation (22%), and then eater and sanitation together (15%).
Few studies explored issues around hygiene (2%) or sanitation and hygiene (3%), while a notably
larger proportion examined WASH-sector topics at large (12%). Despite open-coding outcomes in the
review, our assessment identified only two main themes in the literature: service sustainability (78%)
and access to services (22%). Other focal areas such as improving health outcomes, promoting behavior
change, fostering coordination, improving functionality, and advocating for policy change, were each
mentioned in less than 3% of the studies. These results highlight the prominence of sustainability as a
key focal outcome, with 104 of the 133 studies addressing a service sustainability issue.
Another non-exclusive descriptor of the scope attribute, context, showed a high propensity for
rural (74%) and community-focused (59%) study scopes. Urban contexts were studied notably less
(43%), but 23% of studies purported to be applicable to both rural and urban contexts. However, only
11% of the studies focused solely on city-level scope, highlighting the traditional lack of attention paid
to peri-urban contexts. Many studies were focused at a regional level (32%), reflecting current trends
of district-wide approaches in the WASH sector, while fewer were focused on national-level systems
(20%). Only 8% of the studies focused purely on sector-level issues.

3.4. Application Assessment (RQ3)

Results for project application showed that a majority of studies included in the review were
focused on evaluating project outcomes (42%), while 30% focused on planning, and only 11% on
implementation. While an overwhelming majority of studies had some form of project application
(116 of 133), results indicated that 35% of all studies represented case studies that focused on only
Int. J. Environ. Res. Public Health 2020, 17, 702 11 of 18

one geographic context, which may limit the generalizability of these studies. Studies that included
a project application were mostly equally distributed between countries in East Africa (40%), South
Asia (32%), and West Africa (24%). The review also showed the disproportionate focus of studies that
assessed projects in India (15%), Ghana (14%), and Uganda (13%). Overall, applications in 60 countries
were represented in the literature.
Our evaluation of study impacts also showed that 32 of the 133 studies (24%) reported some form
of impacts that resulted because of the use of the analysis, tool, framework or approach. Non-exclusive
open coding of the study impacts identified eight types of reported impacts, including: uptake of the
tool, framework or approach (12%); effects on services (7%); policy changes (6%); improvements in
coordination (4%); behavior change (3%); financial impacts (2%;, impacts on users (2%); health impacts
(>1%); and changes in levels of access to services (>1%).

4. Discussion
Results of the review paint a complex scene of the existing landscape of analyses, tools, frameworks,
and approaches that are incorporated into systems approaches for WASH. In this section, we reflect on
some of the most prominent themes that emerged from the analysis as they relate to methods employed
(RQ1), context and application (RQ2), and evidence of impact (RQ3). In addition, we discuss gaps
in the existing knowledge base for systems approaches to WASH (RQ4). In particular we highlight
findings salient to WASH sector practitioners, including analytical complexity (RQ1–RQ4) and focus
and scope (RQ1–RQ4). In so doing, we offer insights and suggestions for future research and practice
in systems approaches to WASH.

4.1. Many Methods, More Factors, and a Myriad of Frameworks

While we identified a wide range of methods in the systems approaches literature, a majority
of these methods appear to require a modicum of specific analytical capacity in order to implement.
Overall, 75% of studies across all literature sources were coded as having a high or medium analytical
complexity to conduct. Further, nearly a quarter of the ‘high’ and ‘medium’ complexity tools were
applied at the mostly rural household or community level. This finding, in part, is due to the
large proportion of peer-reviewed studies in the review (75 of 133), however it highlights a growing
disconnect between the increasing complexity of managing WASH services in urban environments
and the focus of the literature on contexts with lower population densities. The studies where more
accessible approaches were proposed (e.g., frameworks, approaches) were generally vague on how
these approaches can be put into practice at the project or program level.
The review also highlighted a wide diversity of factors used across the literature to examine WASH
systems. In addition to traditional technical, financial, and environmental aspects of WASH service
sustainability, we observed many references to factors focused on social, institutional, community, and
user aspects of WASH systems. However, despite the diversity of methods and factors, few studies
explicitly considered the interactions between these factors. Studies which did explore interactions
(n = 35) were almost exclusively from peer-reviewed literature (n = 26), suggesting that there is
less focus on interactions in the practitioner-focused grey or organizational literature. Moreover, of
the studies that did investigate interactions, only four reported any impacts that occurred because
of the use of the tool, framework or approach. Considering that interactions are a fundamental
component of systems thinking [50,56,57], their omission from many of the analytical methods, and
the lack of reported impacts in studies where they were included, suggests that current work may
be neglecting a key dimension of systems approaches for WASH. This finding tracked closely with
analytical complexity where 20% of methods with high analytical complexity included some analysis
of interactions, whereas medium and low complexity methods only accounted for interactions in
5% and 2% of studies, respectively. Additionally, we found little mention in the literature regarding
descriptions of direct versus indirect effects, feedback mechanisms or dynamic behavior, key concepts
of complex systems thinking [52].
Int. J. Environ. Res. Public Health 2020, 17, 702 12 of 18

Additionally, within the WASH systems literature, one-third of all studies (n = 41) represented
an original/novel framework. These frameworks were proportionally more common in the grey and
organizational literature. We find this notable as others reported that 80% of WASH sector professionals
have indicated in previous studies that they did not use a formal planning tool in the implementation
of water and sanitation projects because “the context-specific nature of project planning decreases
the applicability of a planning framework” [58] (p. 79). On reflection, this raises questions about the
demand for, development of, and application of, new frameworks for evaluating WASH systems, as
well as the ability to compare findings across frameworks and contexts to learn and advance the sector.
Contrary to how many of the studies present these tools, frameworks, and approaches, the review
indicates that there is a lack of evidence for their broad application across diverse settings. For more
information on individual frameworks presented by various studies see Table S3-2 for source material.

4.2. Geographic Dispersion of Applications

Overall, 116 of the 133 studies referenced at least one application of the tool, however, applications
were not equally dispersed across geographies. In particular, applications in either India, Ghana
or Uganda, were referenced in 41% of the studies reviewed. Our findings align with findings from
previous studies on the disproportionate focus of the international development sector in African [59]
and Southeast Asian [60] countries in general, and may limit the generalizability of systems approaches
to WASH, where many tools, frameworks, and approaches appear to be built specifically to conditions
in these contexts. There are some noteworthy exceptions to this trend where systems approaches have
been informed by findings from globally dispersed contexts. For example, one study representing
a “Suite of Tools” for a systems-based approach to water service delivery [61] draws on the authors’
experience working in 30 districts across nine countries and offers recommendations on how the
tools can be used to assess different aspects of factors supporting service delivery. Similarly, another
study [62] presents a novel framework for promoting market-based approaches to sanitation based
on experiences from 22 World Bank projects across 13 countries. Considering the important role that
local context plays in influencing WASH service delivery models and sustainability [63], the limited
geographic focus within and across studies included in this review raises questions about the external
validity of findings and insights within the existing body of knowledge.

4.3. Reporting on Service Sustainability

As described above, one motivation for conducting this review is the hypothesis that an increase
in systems approaches to WASH predicated their intention to address pervasive issues of service
sustainability. Our findings for outcome scope showed that while sustainability is indeed the main focal
area of many systems approaches, only 24 of the 104 studies (23%) that focused on service sustainability
included any mention of study impacts. While assessments of service sustainability are relatively new
to the sector and require some form of longitudinal data collection, there is no shortage of tools for
measuring service delivery outcomes in WASH that would hinder reporting on these dimensions in
contemporary literature [64]. Further, while reporting impacts that result from a study (as opposed to
findings) is not common practice in academic journals, half of the studies (16 of 32) that cited some
impact were from peer-reviewed literature.
Of the reported impacts, some identified null or undesired outcomes such as a lack of change in
hygiene practices [65,66] or unmaterialized changes in national policy [67]. These studies represent
an encouraging and necessary trend that has been embraced by the sector [68], yet more reporting
and dissemination of programmatic outcomes, both positive and negative, is clearly needed. Overall,
the low incidence of studies reporting on impacts on WASH services (7%) presents a challenge for
developing the evidence base for the effect that systems approaches have directly on the sustainability
of WASH services. Thus, in order to determine the effectiveness of these approaches, more information
is required that connects the implementation of these approaches to tangible service delivery outcomes,
and ultimately, improvements in public health.
Int. J. Environ. Res. Public Health 2020, 17, 702 13 of 18

4.4. Reporting on Resources and Replicability

The review also highlights the limited potential for the replicability, or scalability, of many of
the approaches proposed in the literature. For example, one study that was classified as both a
tool and framework [69] consisted of data collection through focus groups and interviews—used to
score multi-criteria decision analysis matrices using an analytical hierarchy process—followed by
a multi-step statistical analysis and social network analysis. While applying this framework may
produce rich insights into a particular context, the approach overall may be difficult to replicate due
to resource needs that include external, highly skilled WASH professionals. Although we did not
conduct a direct assessment of the replicability or scalability of each approach, we likely would have
been hindered in doing so, since few studies, with some noted exceptions [37,70,71], reported on
the resources that are required to implement the proposed approach. In order to better evaluate the
practicality of these approaches, more reporting is needed on the resources required for studies which
are applied across multiple contexts.

4.5. Diversifying Methods, Analytical Complexity and WASH Scope

Int. J. Environ. Res. Public Health 2020, 17, x FOR PEER REVIEW 13 of 18
While the review highlighted the wide variety of methods being employed in systems approaches
to WASH,While the review
a majority of thesehighlighted
methods were the wide variety or
of a medium of high
methods being complexity
analytical employed in systems
applied in only
approaches to WASH, a majority of these methods were of a medium or high analytical complexity
one, usually local, context or none at all. The result of this combination is that the largest proportion
applied in only one, usually local, context or none at all. The result of this combination is that the
of studies included in the review appears to use complex analytical methods to assess an issue at
largest proportion of studies included in the review appears to use complex analytical methods to
the local-services
assess an issuelevel
at the(e.g., household,
local-services levelcommunity) (Figure
(e.g., household, 4). Examples
community) (Figureof4).this genre of
Examples systems
of this
approaches
genre ofinclude evaluatinginclude
systems approaches contextual, technical,
evaluating social,
contextual, and financial
technical, detriments
social, and of handpump
financial detriments
functionality through
of handpump Bayesianthrough
functionality networkBayesian
analysisnetwork
[72], using agent-based
analysis [72], usingmodeling to assess
agent-based household
modeling to
waterassess
quality [73], and
household waterthequality
application
[73], andofthefuzzy set qualitative
application of fuzzy set comparative analysis for
qualitative comparative assessing
analysis
forinassessing
failure community failure in community
sanitation projectssanitation
[74]. Thisprojects [74]. This
trend suggests trend
that fewsuggests that fewintended
of the methods of the for
addressing local service sustainability issues are likely accessible to those who are tasked withwho
methods intended for addressing local service sustainability issues are likely accessible to those making
aredecisions
critical tasked with
at making
the localcritical
level decisions
around WASHat the local levelprovision.
service around WASH service provision.

Figure 4. Analytical
Figure complexity
4. Analytical complexityversus scopeof
versus scope ofapplication
applicationbyby literature
literature source.
source.

The strong preference

The strong of of
preference systems
systemsapproaches
approaches to focuson
to focus onrural
rural water
water services
services (48%(48%
of allofstudies)
all studies)
also points to another
also points significant
to another knowledge
significant knowledge gapgapin in
the existing
the existingliterature.
literature. Although waterservices
Although water services are
an essential part of WASH
are an essential that need
part of WASH thattoneed
be sustained in order
to be sustained to deliver
in order public
to deliver health
public impacts,
health impacts, recent
recent
studies studies of environmental
of environmental enteropathy enteropathy effectshave
effects [75–77] [75–77] havethat
shown shown that
there is there is little impact
little impact from water
from
services water
alone if services alone
sanitation andif hygiene
sanitationpractices
and hygiene practices
are not are not also
also improved andimproved
sustained andsimultaneously.
sustained
simultaneously. Furthermore, as a majority of the world’s population
Furthermore, as a majority of the world’s population now resides in urban environments, now resides in urban
and these
environments, and these areas will continue to grow exponentially in population and density [78],
areas will continue to grow exponentially in population and density [78], systems approaches need
systems approaches need to begin shifting their focus to these more complex contexts in order to
to begin shifting their focus to these more complex contexts in order to meet the needs of future
meet the needs of future populations. As such, it would be in the WASH sector’s interest to begin
expanding systems approaches beyond a traditional focus on rural water service delivery to include
a focused analysis of the how the multitude of factors in peri-urban or informal settlements interact
with one another to inhibit or promote WASH services.
Overall, while our review identified a large number of studies with a wide variety of methods,
scopes, and project applications, we found that studies generally tended to represent one-off case
Int. J. Environ. Res. Public Health 2020, 17, 702 14 of 18

populations. As such, it would be in the WASH sector’s interest to begin expanding systems approaches
beyond a traditional focus on rural water service delivery to include a focused analysis of the how
the multitude of factors in peri-urban or informal settlements interact with one another to inhibit or
promote WASH services.
Overall, while our review identified a large number of studies with a wide variety of methods,
scopes, and project applications, we found that studies generally tended to represent one-off case
studies (41%) that employed a method requiring medium-to-high analytical complexity (75%), most
commonly applied to the analysis of a rural water context (48%), that generally focused on financial
(75%) or technical factors (54%), and did not explicitly consider interactions between factors (75%).

4.6. Limitations
By the nature of such a large secondary data review, there are inherent limitations to the findings
presented in this study. For example, while we sought to gather as much applicable literature as
possible, it is possible that some grey or organizational literature may not have been captured in the
review and thus may be underrepresented in the results. As with any literature review, our choice
of screening criteria and principal categories focused our analysis; different screening criteria and
categories would alter the results.

5. Conclusions
We conducted a systematic review of peer-reviewed, grey, and organizational literature of systems
approaches to WASH. Our objectives were to assess the methods and factors being included in systems
approaches, the contextual scope of these approaches, their geographic and programmatic applications,
evidence of their impact on WASH services, and gaps in the existing knowledge base. From 7764
unique search results we identified 130 studies that met our two-phase screening criteria and evaluated
these articles for 12 study descriptors across three categories of method, scope, and application. We
then evaluated needs and gaps based upon this review.
The results of the review indicate a propensity in the systems approaches literature towards the
application of complex analytical methods to singular case studies focused on financial and technical
factors and are most commonly applied to rural water service delivery contexts. Despite finding a
large diversity of factors in the literature, few studies explicitly evaluated interactions or relationships
between factors, a fundamental concept of systems thinking. The review also found inadequate and
inconclusive information to assess the impacts—positive, null or negative—that systems approaches
have demonstrated on WASH service sustainability. While this could be related to the time needed
to foster systems change and impacts, increased reporting and a plan for monitoring and reporting
impacts over time are needed. Thus, our key recommendations call for: (i) a diversification of the
methods, scopes, and applications of systems approaches for WASH; (ii) further investigation and
application of system approaches that explicitly consider factor interactions; (iii) increased reporting
of resources required to implement the approaches; and (iv) more documentation of the impacts to
WASH services that result from the application of a systems analysis, tool, framework or approach.
Overall, these findings provide a robust survey of the existing landscape of systems approaches for
WASH and illuminate a path for future research in this emerging field.

Supplementary Materials: The following are available online at http://www.mdpi.com/1660-4601/17/3/702/s1,

Table S1: PRISMA checklist; Table S2-1: Search strategies; Table S2-2: Keywords and Search String Formula;
Table S2-3: Initial Screening Exclusion Terms and Criteria; Table S3-1: Coding Form; Table S3-2: Literature Search;
Table S4: Results summary.
Author Contributions: Conceptualization, N.V., A.J.-W., J.W., and K.L.; data curation, N.V.; formal analysis,
N.V.; funding acquisition, A.J.-W. and K.L.; investigation, K.L.; methodology, N.V., A.J.-W., and J.W.; project
administration, N.V., A.J.-W., and K.L.; supervision, A.J.-W., J.W., and K.L.; validation, N.V.; visualization, N.V.;
writing—original draft, N.V.; writing—review and editing, N.V., A.J.-W., J.W., and K.L. All authors have read and
agreed to the published version of the manuscript.
Int. J. Environ. Res. Public Health 2020, 17, 702 15 of 18

Funding: This work was completed with financial support from the Sustainable WASH Systems Learning
Partnership through USAID under the terms of the Cooperative Agreement AID-OAA-A-16-00075 to the
University of Colorado Boulder. The contents are the responsibility of the University of Colorado Boulder
Sustainable WASH Systems Learning Partnership and do not necessarily reflect the views of USAID or the United
States Government. For more information, visit www.globalwaters.org/SWS. Publication of this article was
funded by the University of Colorado Boulder Libraries Open Access Fund.
Conflicts of Interest: The authors declare no conflicts of interest.

References
1. JMP. Progress on Drinking Water, Sanitation and Hygiene: 2017 Update and SDG Baselines; World Health
Organization (WHO) and the United Nations Children’s Fund (UNICEF): Geneva, Switzerland, 2017.
2. Foster, T.; Willetts, J.; Lane, M.; Thomson, P.; Katuva, J.; Hope, R. Risk factors associated with rural water
supply failure: A 30-year retrospective study of handpumps on the south coast of Kenya. Sci. Total Environ.
2018, 626, 156–164. [CrossRef]
3. Improve International: Statistics on Water Point Failures. Available online: http://www.improveinternational.
org/2012/10/25/sad-stats/ (accessed on 22 September 2018).
4. Lockwood, H.; Smits, S. Supporting Rural Water Supply: Moving Towards a Service Delivery Approach; Practical
Action Publishing: Warwickshire, UK, 2011; ISBN 978-1-85339-729-5.
5. Foster, T. Predictors of sustainability for community-managed handpumps in sub-Saharan Africa: Evidence
from Liberia, Sierra Leone, and Uganda. Environ. Sci. Technol. 2013, 47, 12037–12046. [CrossRef] [PubMed]
6. Prüss-Ustün, A.; Bartram, J.; Clasen, T.; Colford, J.M.; Cumming, O.; Curtis, V.; Bonjour, S.; Dangour, A.D.;
De France, J.; Fewtrell, L.; et al. Burden of disease from inadequate water, sanitation and hygiene in low- and
middle-income settings: A retrospective analysis of data from 145 countries. Trop. Med. Int. Health 2014, 19,
894–905. [CrossRef] [PubMed]
7. de Oliveira, A.F.; da Leite, I.C.; Valente, J.G. Global burden of diarrheal disease attributable to the water
supply and sanitation system in the State of Minas Gerais, Brazil: 2005. Ciência Saúde Coletiva 2015, 20,
1027–1036. [CrossRef] [PubMed]
8. Banks, B.; Furey, S. What’s Working, Where, and for How Long: A 2016 Water Point Update; RWSN: Abidjan,
Cote d’Ivoire, 2016.
9. Carter, R.C. Editorial: Researching ‘how’ rather than ‘why’. Waterlines 2013, 32, 3–4. [CrossRef]
10. Dube, T. Emerging Issues on the Sustainability of the Community Based Rural Water Resources Management
Approach in Zimbabwe: A Case Study of Gwanda District. SSRN Electron. J. 2013, 1, 644–655. [CrossRef]
11. Hope, R. Is community water management the community’s choice? Implications for water and development
policy in Africa. Water Policy 2015, 17, 664–678. [CrossRef]
12. Water, P.M.I.; Centre, S.; Haag, D.; Netherlands, T. Trends in Rural Water Supply: Towards a Service Delivery
Approach. Water Altern. 2013, 6, 21.
13. van Ginneken, M.; Netterstrom, U.; Bennett, A. Trends in Public Expenditure on Water and Sanitation in
Sub-Saharan Africa; Water Papers; The World Bank: Washington, DC, USA, 2011; p. 120.
14. Khatri, K.; Vairavamoorthy, K.; Porto, M. Challenges for urban water supply and sanitation in developing
countries. In Water for a Changing World Developing Local Knowledge and Capacity, Proceedings of the International
Symposium Water for a Changing World—Developing Local Knowledge and Capacity, Delft, the Netherlands,
June 13–15, 2007; Alaerts, G., Dickinson, N., Eds.; CRC Press: Delft, The Netherlands, 2009; pp. 81–99,
ISBN 978-0-415-47757-4.
15. Parris, T.M.; Kates, R.W. Characterizing a sustainability transition: Goals, targets, trends, and driving forces.
Proc. Natl. Acad. Sci. USA 2003, 100, 8068–8073. [CrossRef]
16. Schultz, B.; Uhlenbrook, S. (Eds.) Water Security: What Does It Mean, What May It Imply? In Proceedings of
the Water for a Changing World-Developing Local Knowledge and Capacity; CRC Press: Boca Raton, FL, USA, 2007.
17. Sparkman, D.; Sturzenegger, G. Why Business as Usual Will Not Achieve SDG6 in LAC: The Promise of
Wastewater Reuse, Green Infrastructure and Small Business around WASH: Conclusions from World Water Week
2016; Inter-American Development Bank: Washington, DC, USA, 2017.
18. Thomas, E.A. Broken Pumps and Promises: Incentivizing Impact in Environmental Health; Springer: Cham
Switzerland, 2016; ISBN 3-319-28643-9.
Int. J. Environ. Res. Public Health 2020, 17, 702 16 of 18

19. Sustainability—WASH Alliance International. Available online: https://wash-alliance.org/our-approach/

sustainability/ (accessed on 3 November 2019).
20. Casella, D.; Van Tongeren, S.; Nikolic, I. Change in Complex Adaptive Systems: A review of Concepts, Theory and
Approaches for Tackling ‘Wicked’ Problems in Achieving Sustainable Rural Water Services; IRC WASH: The Hauge,
The Netherlands, 2015; p. 37.
21. Huston, A.; Moriarty, P. Understanding the WASH system and its Building Blocks: Building Strong WASH Systems
for the SDGs; IRC WASH: Delft, The Netherlands, 2018; p. 40.
22. Lockwood, H.; Moriarty, P.; Ingeborg, K.; Skilling, H. Kampala WASH Symposium_Background Note. In
Proceedings of the Kampala WASH Symposium; Aguaconsult: Kampala, Uganda, 20–23 June 2016.
23. Lockwood, H.; Smits, S.; Schouten, T.; Moriarty, P. Providing sustainable water services at scale. In
Proceedings of the International Symposium on Rural Water Services, Kampala, Uganda, 13–15 April 2010;
pp. 13–15.
24. Schouten, T.; Moriarty, P. The Triple-S Theory of Change; Triple-S Working Papers; IRC International Water and
Sanitation Centre: The Hague, The Netherlands, 2013.
25. World Bank Group. Sustainability Assessment of Rural Water Service Delivery Models: Findings of a Multi-Country
Review; World Bank: Washington, DC, USA, 2017.
26. IRC. All Systems Go! Symposium Proceedings; International Water and Sanitation Centre: The Hague,
The Netherlands, 2019.
27. Kobayashi, K. Community Based Water Management and Social Capital; IWA Publishing: London, UK, 2014;
Volume 13.
28. Bhattarai, S.; Starkl, M. Rural Water Supply and Sanitation in Developing Countries. In Proceedings of the
ISAHP, Honolulu, HI, USA, 8–10 July 2005; Volume 9.
29. Roberts, M.; Dauget, M. Synchronizing Complex Systems in Real Time to Accelerate Latrine Sales; Tactic Report;
iDE: Phnom Penh, Cambodia, 2017.
30. Kayaga, S. Soft systems methodology for performance measurement in the Uganda water sector. Water
Policy 2008, 10, 273–284. [CrossRef]
31. Malik, M.; Karangwa, L.; Muzola, A.; Sano, J.; Vianney, J.M.; Musabyimana, G. Assessment of Sustainability
of Rural Water, Sanitation and Hygiene Interventions in Rwanda. In Proceedings of the 7th RWSN Forum
“Water for Everyone”, Abidjan, Cote d’Ivoire, 29 November–2 December 2016; Volume 8.
32. Walters, J.P.; Javernick-Will, A.N. Long-Term Functionality of Rural Water Services in Developing Countries:
A System Dynamics Approach to Understanding the Dynamic Interaction of Factors. Environ. Sci. Technol.
2015, 49, 5035–5043. [CrossRef] [PubMed]
33. Neely, K. Systems Thinking and WASH: Tools and Case Studies for a Sustainable Water Supply; Practical Action
Publishing Ltd.: London, UK, 2019; ISBN 978-1-78853-026-2.
34. Amadei, B. A Systems Approach to Modeling Community Development Projects; Momentum Press: New York,
NY, USA, 2015; ISBN 1-60650-519-X.
35. Lee, B.; Sommerville, P.; Farley, S.; Mayega, S. SPACES MERL: Systems and Complexity White Paper; U.S.
Agency for International Development: Washington, DC, USA, 2016.
36. Ramalingam, B.; Laric, M.; Primrose, J. From Best Practice to Best Fit: Understanding and Navigating Wicked
Problems in International Development; Overseas Development Institute: London, UK, 2014; p. 55.
37. Lockwood, H.; Duti, V. Whole System Change: Capturing the Change Process in the Ghana Rural Water Sub-Sector;
IRC WASH: Accra, Ghana, 2015; p. 39.
38. Cooper, H.; Hedges, L.V.; Valentine, J.C. The Handbook of Research Synthesis and Meta-Analysis; Russell Sage
Foundation: New York, NY, USA, 2009; ISBN 1-61044-138-9.
39. Liberati, A.; Altman, D.G.; Tetzlaff, J.; Mulrow, C.; Gøtzsche, P.C.; Ioannidis, J.P.A.; Clarke, M.; Devereaux, P.J.;
Kleijnen, J.; Moher, D. The PRISMA Statement for Reporting Systematic Reviews and Meta-Analyses of
Studies That Evaluate Health Care Interventions: Explanation and Elaboration. PLoS Med. 2009, 6, e1000100.
[CrossRef] [PubMed]
40. Moher, D.; Liberati, A.; Tetzlaff, J.; Altman, D.G.; Group, P. Preferred reporting items for systematic reviews
and meta-analyses: The PRISMA statement. PLoS Med. 2009, 6, e1000097. [CrossRef]
41. Liddle, E.S.; Fenner, R. Water point failure in sub-Saharan Africa: The value of a systems thinking approach.
Waterlines 2017, 36, 140–166. [CrossRef]
42. Triple-S. Available online: http://www.ircwash.org/projects/triple-s (accessed on 28 April 2017).
Int. J. Environ. Res. Public Health 2020, 17, 702 17 of 18

43. Conn, V.S.; Valentine, J.C.; Cooper, H.M.; Rantz, M.J. Grey Literature in Meta-Analyses. Nurs. Res. 2003, 52,
256–261. [CrossRef] [PubMed]
44. Waddington, H.; Snilstveit, B. Effectiveness and sustainability of water, sanitation, and hygiene interventions
in combating diarrhoea. J. Dev. Eff. 2009, 1, 295–335. [CrossRef]
45. Butterworth, J.; Warner, J.F.; Moriarty, P.; Smits, S.; Batchelor, C. Finding practical approaches to integrated
water resources management. Water Altern. 2010, 3, 68–81.
46. Gallego-Ayala, J. Trends in integrated water resources management research: A literature review. Water
Policy 2013, 15, 628–647. [CrossRef]
47. Saravanan, V.S.; McDonald, G.T.; Mollinga, P.P. Critical review of Integrated Water Resources Management:
Moving beyond polarised discourse. Nat. Resour. Forum 2009, 33, 76–86. [CrossRef]
48. Venkataramanan, V. Testing CLTS Approaches for Scalability: Systematic Literature Review; The Water Institute:
Chapel Hill, NC, USA, 2012.
49. Walker, T. Local Systems: A Framework for Supporting Sustained Development; Bureau for Policy, Planning and
Learning; USAID: Washington, DC, USA, 2014.
50. Meadows, D.H.; Wright, D. Thinking in Systems: A Primer; Chelsea Green Pub: White River Junction, VT,
USA, 2008; ISBN 978-1-60358-055-7.
51. Sterman, J.D. Learning from evidence in a complex world. Am. J. Public Health 2006, 96, 505–514. [CrossRef]
52. Sterman, J.D. Business Dynamics: Systems Thinking and Modeling for a Complex World; Nachdr;
Irwin/McGraw-Hill: Boston, MA, USA, 2009; ISBN 978-0-07-238915-9.
53. Williams, B.; Hummelbrunner, R. Systems Concepts in Action: A Practitioner’s Toolkit; Stanford Business Books:
Stanford, CA, USA, 2011; ISBN 978-0-8047-7062-0.
54. United Nations (Ed.) Standard Country or Area Codes for Statistical Use: Current Information as at 31 August
1999 = Codes Standard des Pays et des Zones à Usage Statistique; United Nations: New York, NY, USA, 1999;
ISBN 978-92-1-061175-6.
55. Scott, P. The Sanitation Cityscape Conceptual Framework—Understanding urban sanitation systems. In
Proceedings of the IRC All Systems Go Conference Proceedings, The Hauge, The Netherlands, 12–14 March
2019; p. 10.
56. Ackoff, R.L. Systems thinking and thinking systems. Syst. Dyn. Rev. 1994, 10, 175–188. [CrossRef]
57. Bean, W.B. Cybernetics: Circular Causal and Feedback Mechanisms in Biological and Social Systems;
Transactions of the Tenth Conference, 22–24 April 1953. AMA Arch. Intern. Med. 1956, 98, 127–128. [CrossRef]
58. Barnes, R.; Ashbolt, N.; Roser, D.; Brown, P. Implementing sustainable water and sanitation projects in rural,
developing communities. Waterlines 2014, 33, 71–88. [CrossRef]
59. Kwemo, A.B. Making Africa Great Again: Reducing aid dependency. Retrieved August 2017, 6, 2018.
60. Htoon, H.Y.Z. ASEAN: Shaping the Future of Regional Development in Southeast Asia; The Asia Foundation:
San Francisco, CA, USA, 2018.
61. Lemme, K.; Latham, K.; Kugler, K. A Suite of Tools to Support a Systems-Based Approach to Sustainable Management
of Water Service Delivery; Rural Water Supply Network: Abidjan, Cote d’Ivoire, 2016.
62. Dumpert, J.; Perez, E. Going beyond mason training: Enabling, facilitating, and engaging rural sanitation
markets for the base of the pyramid. Waterlines 2015, 34, 210–226. [CrossRef]
63. Fisher, M.B.; Shields, K.F.; Chan, T.U.; Christenson, E.; Cronk, R.D.; Leker, H.; Samani, D.; Apoya, P.; Lutz, A.;
Bartram, J. Understanding handpump sustainability: Determinants of rural water source functionality in the
Greater Afram Plains region of Ghana: Determinants of handpump sustainability in rural Ghana. Water
Resour. Res. 2015, 51, 8431–8449. [CrossRef]
64. Boulenouar, J.; Schweitzer, R.; Lockwood, H. Mapping Sustainability Assessment Tools to Support Sustainable
Water and Sanitation Service Delivery; Triple-S Working Papers; IRC International Water and Sanitation Centre:
Den Haag, The Netherlands, 2013; p. 26.
65. Mosler, H.; Sonego, I.L. Improved latrine cleanliness through behaviour change and changes in quality of
latrine construction: A longitudinal intervention study in rural Burundi. Int. J. Environ. Health Res. 2017, 27,
355–367. [CrossRef]
66. Hetherington, E.; Eggers, M.; Wamoyi, J.; Hatfield, J.; Manyama, M.; Kutz, S.; Bastien, S. Participatory science
and innovation for improved sanitation and hygiene: Process and outcome evaluation of project SHINE, a
school-based intervention in Rural Tanzania. BMC Public Health 2017, 17, 172. [CrossRef]
Int. J. Environ. Res. Public Health 2020, 17, 702 18 of 18

67. Rosenweig, F. Synthesis of Four Country Enabling Environment Assessments for Scaling Up Handwashing Programs;
Global Scaling Up Handwashing Project; The Water and Sanitation Program, The World Bank: Washington,
DC, USA, 2008.
68. Water, Sanitation and Health at University of Leeds the Nakuru Accord. Available online: https://wash.leeds.
ac.uk/failing-better-in-the-wash-sector/ (accessed on 1 March 2019).
69. Starkl, M.; Brunner, N.; López, E.; Martínez-Ruiz, J.L. A planning-oriented sustainability assessment
framework for peri-urban water management in developing countries. Water Res. 2013, 47, 7175–7183.
[CrossRef]
70. Prokopy, L.S. The relationship between participation and project outcomes: Evidence from rural water
supply projects in India. World Dev. 2005, 33, 1801–1819. [CrossRef]
71. Davis, A.; Javernick-Will, A.; Cook, S.M. Priority Addressment Protocol: Understanding the Ability and
Potential of Sanitation Systems to Address Priorities. Environ. Sci. Technol. 2019, 53, 401–411. [CrossRef]
[PubMed]
72. Cronk, R.; Bartram, J. Factors Influencing Water System Functionality in Nigeria and Tanzania: A Regression
and Bayesian Network Analysis. Environ. Sci. Technol. 2017, 51, 11336–11345. [CrossRef] [PubMed]
73. Mellor, J.E.; Smith, J.A.; Learmonth, G.P.; Netshandama, V.O.; Dillingham, R.A. Modeling the Complexities of
Water, Hygiene, and Health in Limpopo Province, South Africa. Environ. Sci. Technol. 2012, 46, 13512–13520.
[CrossRef] [PubMed]
74. Davis, A.; Javernick-Will, A.; Cook, S.M. The use of qualitative comparative analysis to identify pathways to
successful and failed sanitation systems. Sci. Total Environ. 2019, 663, 507–517. [CrossRef]
75. Luby, S.P.; Rahman, M.; Arnold, B.F.; Unicomb, L.; Ashraf, S.; Winch, P.J.; Stewart, C.P.; Begum, F.; Hussain, F.;
Benjamin-Chung, J.; et al. Effects of water quality, sanitation, handwashing, and nutritional interventions on
diarrhoea and child growth in rural Bangladesh: A cluster randomised controlled trial. Lancet Glob. Health
2018, 6, e302–e315. [CrossRef]
76. Null, C.; Stewart, C.P.; Pickering, A.J.; Dentz, H.N.; Arnold, B.F.; Arnold, C.D.; Benjamin-Chung, J.; Clasen, T.;
Dewey, K.G.; Fernald, L.C.H.; et al. Effects of water quality, sanitation, handwashing, and nutritional
interventions on diarrhoea and child growth in rural Kenya: A cluster-randomised controlled trial. Lancet
Glob. Health 2018, 6, e316–e329. [CrossRef]
77. Arnold, B.F.; Null, C.; Luby, S.P.; Colford, J.M. Implications of WASH Benefits trials for water and
sanitation—Authors’ reply. Lancet Glob. Health 2018, 6, e616–e617. [CrossRef]
78. Ritchie, H.; Roser, M. Urbanization. Our World Data. Available online: https://ourworldindata.org/
urbanization (accessed on 2 December 2018).

© 2020 by the authors. Licensee MDPI, Basel, Switzerland. This article is an open access
article distributed under the terms and conditions of the Creative Commons Attribution
(CC BY) license (http://creativecommons.org/licenses/by/4.0/).