Ecological Engineering 127 (2019) 103–113

Contents lists available at ScienceDirect

Ecological Engineering
journal homepage: www.elsevier.com/locate/ecoleng

Two calibration methods for modeling streamflow and suspended sediment T

with the swat model
⁎
Tássia Mattos Brighentia, , Nadia Bernardi Bonumáb, Fernando Grisonc, Aline de Almeida Motac,
Masato Kobiyamad, Pedro Luiz Borges Chaffeb
a
Graduate Program in Environmental Engineering, Federal University of Santa Catarina, Florianopolis, Brazil
b
Department of Sanitary and Environmental Engineering, Federal University of Santa Catarina, Florianopolis, Brazil
c
Federal University of Fronteira Sul, Chapeco, Brazil
d
Hydraulic Research Institute, Federal University of Rio Grande do Sul, Porto Alegre, Brazil

A R T I C LE I N FO A B S T R A C T

Keywords: The proper estimation of streamflow (Q) and suspended sediment (SS) have important implications for sus-
Sequential calibration tainable water management. The Soil and Water Assessment Tool (SWAT) is a distributed, physically-based and
Simultaneous calibration dynamic model that integrates water quantity and quality routines. SWAT is considered an effective tool for
Uncertainty analysis assessing water and soil resources problems in a worldwide range of environmental conditions; however every
SUFI-2
hydrological model application is limited by the choices made in the calibration process. The aim of this paper
was to assess the differences between the sequential and simultaneous calibration methods when simulating
streamflow and suspended sediment with SWAT. We used the Sequential Uncertainty Fitting (SUFI-2) combined
with SWAT for: (i) sensitivity analysis; (ii) parameter calibration using Q and SS data with a combined multi-site
and multi-objective approaches; and (iii) evaluation of the differences in the estimated uncertainty of both
calibration methods. Our results suggest that the simultaneous calibration was more accurate, especially for SS
values with King-Gupta Efficiency (KGE) objective function. The simultaneous calibration was less time con-
suming and the sensitivity analysis indicated that it needed less parameters in the calibration process. The multi-
site calibration method improved the model results in 3 out of 4 simulations of suspended sediment in the basin
outlet.

1. Introduction issues including streamflow generation, sediment transport and climate

and land cover change impacts (Abeyou et al., 2018; Wu and Chen,
Proper estimation of streamflow and sediment generation is essen- 2012). Regardless of the application, every hydrological model needs to
tial for water resources management. Hydrological models are the basic be calibrated and several algorithms have been developed to address
tools for simulating those processes and have been widely used for both the calibration problem (Surfleet and Tullos, 2013; Wu and Chen,
change detection and attribution in catchment systems (Folton et al., 2015). The Sequential Uncertainty Fitting – SUFI-2 algorithm has been
2015; Hassan et al., 2010; Vandenberghe et al., 2006). If used in- developed to attempt parameters optimization. In this algorithm all
appropriately, model simulations may lead to unrealistic understanding uncertainties (parameter, conceptual model, input, etc.) are mapped
and poor policy recommendation. According to Daggupati et al. (2015) onto the parameter ranges as the procedure tries to capture most of the
the required modeling accuracy may change for different applications, measured data within the 95% prediction uncertainty (Abbaspour et al.,
based on the risk associated with actions that follow model use (e.g., 2009, 2004). The SUFI-2 algorithm has been extensively applied to
explanatory, planning and/or regulatory). analyze sensitivity and critical sources of uncertainty in SWAT (Wu and
The Soil and Water Assessment Tool – SWAT (Arnold et al., 2012) is Chen, 2015).
a well-established physically based, semi-distributed hydrological The importance of the calibration process has received a lot of at-
model for river basin scale application. The model can capture the tention through the years (Andreassian et al., 2009; Arnold et al., 2012;
watershed spatial heterogeneity and has been used for a variety of Beven et al., 2011; Gupta et al., 1998; Kirchner, 2006; Thirel et al.,

⁎
Corresponding author.
E-mail addresses: tassia.brighenti@posgrad.ufsc.br (T.M. Brighenti), nadia.bonuma@ufsc.br (N.B. Bonumá), fernando.grison@uffs.edu.br (F. Grison),
aline.mota@uffs.edu.br (A.d.A. Mota), masato.kobiyama@ufrgs.br (M. Kobiyama), pedro.chaffe@ufsc.br (P.L.B. Chaffe).

https://doi.org/10.1016/j.ecoleng.2018.11.007
Received 15 May 2018; Received in revised form 5 November 2018; Accepted 8 November 2018
0925-8574/ © 2018 Elsevier B.V. All rights reserved.
T.M. Brighenti et al. Ecological Engineering 127 (2019) 103–113

2015). However, this topic can be approached in very distinct ways and location of the monitoring data (rainfall, streamflow and suspended
there are no universally accepted procedures for calibration in the lit- sediment). The native forest is 84% of the total area, characterized by
erature (Arnold et al., 2015; Moriasi and Wilson, 2012). There are mixed ombrophilous forest or Brazilian pine forests, which has been
suggestions for the use of: optimization algorithms (Abbaspour et al., transformed into pine reforestation, 8% of the total area. Besides re-
2004; Vrugt and Beven, 2016); multi-objective approaches (Fenicia forestation, there is agriculture (mainly maize and soybeans) and pas-
et al., 2006; Rodrigues et al., 2015); data series split (Klemeš, 1986; tures, remaining the 8% of the total area. The soil type is predominantly
Zheng et al., 2018); hydrograph separation methods (Nicolle et al., Cambisols. According to Thornthwaite classification, the climate in the
2014; Zhang et al., 2011) crash test applications (Andreassian et al., studied region is wet, mesothermal, with little or no water deficit. The
2009); the use of hard and soft data (Seibert and Mcdonnell, 2003); and mean annual temperature is between 17 °C and the average precipita-
sequential or simultaneous methods (Daggupati et al., 2015; Wellen tion is 1500 mm/year.
et al., 2014). The water level and turbidity were automatically measured every
Two main approaches are usually used when there is more than one 10 min using pressure transducers and turbidity sensors. Through the
output and observed variable of interest (e.g., streamflow and sedi- stage-discharge curve, level data were converted in discharge (Q) and
ment). The first one suggests that we should calibrate the parameters suspended sediments (SS) (Cardoso, 2013; Grison, 2013). These data
related to streamflow generation before any other processes that are were integrated to a daily time step for SWAT model simulation and
dependent on runoff (Arnold et al., 2015; Daggupati et al., 2015; Santhi calibration process.
et al., 2002), this is the sequential approach advocated for SWAT and The monitoring period ranges from 2011 to 2013 for streamflow; a
other distributed models (Santhi et al., 2001). The second calibration total of 322 data was selected for RB10 and 435 for RB11. The mon-
approach proposes a simultaneous calibration, where the output vari- itoring period for suspended sediment was 2011–2013; a total of 458
ables are calibrated together; the parameters related to two or more data was selected for RB10 and 617 for RB11. The values that extra-
variables of interest (e.g., streamflow and sediment) are modified at the polated the rating-curves were removed from the sample. Rainfall was
same time (Wellen et al., 2014). also measured automatically every 10 min for the same period. The
Despite the method of choice, there are intrinsic uncertainties in any climate data was obtained from the EPAGRI (Extension Company of the
type of modeling and calibration exercise. Those uncertainties are due State of Santa Catarina) station, located 18 km from the basin outlet.
to model parameters, measured data, model structure, and epistemic We had daily data for a ten-years period.
errors (Abbaspour et al., 2004; Beven and Binley, 1992; Gupta et al.,
1998; Vrugt and Beven, 2016). It is also this context that we cannot 2.2. SWAT model
disregard equifinality, and there is not a single optimum parameter set
for a modeling case but actually a range of them (Beven, 2006). The SWAT model is a continuous and physically-base hydrological
Therefore, uncertainty analysis is necessary and essential in any rig- model developed to exploring the effects of climate and land manage-
orous hydrological modeling exercise (Wu and Chen, 2015). ment practices on water, sediment, and agricultural chemical yields
Although SWAT has been extensively evaluated in respect to (Arnold et al., 2012). The hydrological part of the model is based on the
streamflow, there is a dearth of literature which validates the accuracy water balance equation in the soil prolife with process including pre-
of sediment simulations (Zeiger and Hubbart, 2016). Wellen et al. cipitation, surface runoff, infiltration, evapotranspiration, lateral flow,
(2014) described the need to determine whether a simultaneous or a percolation and groundwater flow. The simulation unit of the model is a
sequential calibration would yield better results. The authors found that Hydrological Response Unity (HRU) that is defined as an area com-
when the model was updated with the sediment data; the streamflow prised of a unique land cover and soil type (Neitsch et al., 2011). In this
parameters exhibited significant changes of shape, with credible in- study the runoff is calculated with the Curve Number (CN) method and
tervals larger than 95% (after the update of the sediment data), con- channel routing is calculated using the Muskingum method. The time of
cluding that optimal parameter space for streamflow simulation may concentration is estimated using Manning’s formula, considering both
not be optimal for sediment data. In this sense, we intend to understand overland and channel flow. The Modified Universal Soil Loss Equation
how and why different calibration methods might influence uncertainty (MUSLE) is used to estimate the sediment yield and the channel
in streamflow and sediment estimations. The aim of this paper was to transport is made by Bagnold’s equation. The Penman-Monteith equa-
assess the differences between the sequential and simultaneous cali- tion is used to calculate the potential evapotranspiration.
bration methods when simulating streamflow and suspended sediment. The model requires information about the topography (digital ele-
We used the SWAT model coupled with the SUFI-2 algorithm, which vation model – DEM), soil, land used, and climate data; the climate data
was developed by (Abbaspour et al., 2004) as it was shown to be an is composed by precipitation, atmospheric temperature, solar radiation,
effective tool for assessing hydrological process in a watershed scale wind speed, and humid data. The DEM map was obtained through the
(Gassman et al., 2014; Krysanova and Srinivasan, 2015; Van Griensven Secretary of Sustainable Development of Santa Catarina (SDS) at a
et al., 2012). Three main steps were followed in the present study: (i) 1:10.000 scale; the land use map was made through LANDSAT images;
model sensitivity analysis; (ii) calibration of the SWAT model para- and the soil classification map was provided by EPAGRI at a 1:50,000
meters for streamflow and suspended sediment using data from two scale. The warm-up period was one year.
nested outlets; and (iii) evaluation of the differences in the sequential
and simultaneous calibration methods using uncertainty analysis and 2.3. Sufi-2
different objective functions.
The Sequential Uncertainty Fitting – SUFI-2 (Abbaspour et al.,
2. Materials and methods 2004) was used in calibration and uncertainty analysis. The method has
a Bayesian framework and determines the uncertainties through the
2.1. Study area sequential and fitting process. In this method, uncertainties account for
different possible sources, including model input, model structure,
The study area is located in Santa Catarina state, southern Brazil parameters, and observed data for calibration and validation purposes.
between latitudes 26°26′24.6′' S and 26°15′0.3′' S and longitude An objective function needs to be defined before uncertainty analysis
49°34′40.8′' W and 49°34′3.1′' W. The present study was carried out in and assigned with a required stopping rule (Wu and Chen, 2015). The
the called RB10 (11 km2) basin and in it’s a nested sub-basin, the RB11 method is an inverse optimization approach that uses the Latin Hy-
(7 km2). The basins altitudes range between 818 and 982 m. Fig. 1 percube sampling procedure along with a global search algorithm to
shows the study area location, the Digital Elevation Model (DEM), and examine the behavior of objective functions by analyzing the Jacobian

104
T.M. Brighenti et al. Ecological Engineering 127 (2019) 103–113

Fig. 1. Study area location in Santa Catarina state, southern Brazil.

and Hessian matrices. The initial parameters ranges are updated at each response variable. With a p-value of 0.05, there is only a 5% chance that
iteration, and a narrower parameter uncertainty is obtained. The SUFI-2 results would have come up in a random distribution, with a 95%
is iterated until the uncertainty criteria are satisfied. At the end of the probability of being correct that the variable is having some effect in
simulation an uncertainty envelope is obtained, which corresponds to the output (Abbaspour et al., 2004; Me et al., 2015).
95% of the forecast uncertainty (95PPU), the 95PPU is calculated be- Table 1 describes all parameters obtained in SA for the two cali-
tween 2.5% and 97.5% of the cumulative distribution of the output bration methods. The check mark indicates that model was sensitive to
variable (Abbaspour et al., 2015, 2004). the parameter for that calibration method and that the parameter was
used in the calibration process. The selected parameters belong to the
model routines of groundwater, management, main channel, HRU’s,
2.4. Sensitivity analysis
basin physical process, soil, and plant growth. The range column in-
dicates the initial parameter distribution during the sensitive analysis
Sensitivity Analysis (SA) is a fundamental step in understanding the
and the calibration process, the values are based on the available in-
parameters influence on model behavior, and has role to decrease the
formation and physical meaning., The methods used to adjust the
number of parameters when calibration process is required (Van
parameters values in SUFI-2 are represented as the first letter before the
Griensven and Bauwens, 2003). In hydrological modeling the SA is
parameters name (R, V, and A); the letter V represents replacing the
defined as a function that investigates the change in model outputs
existing parameter value, A is adding a given value to the existing
related to the changes in their parameters. The Global Sensitive Ana-
parameter value, and R is multiplying (1 + a given value) to the ex-
lysis has the capacity to evaluate the influence of the parameters
isting parameter value. The last column indicates if the main parameter
throughout its range of variation, incorporating the uncertainty factor
influence is related to streamflow or suspended sediment processes.
under this variation (Song et al., 2015; Wang et al., 2013). The sensi-
The SA was done for the two different calibration methods, se-
tivity may vary according to the approach used, thus, it is needed to use
quential and simultaneous, and for the two objective functions, NS and
different objective functions to activate an efficient model calibration.
KGE. In sequential calibration, the sensitivity was made first for the
In this study the Global Sensitivity Analysis was apply to the SWAT
streamflow parameters, and after for the suspended sediment para-
model, and helps to identify the most dominant parameters that should
meters. In simultaneous calibration, all parameters were modified to-
be used for calibration (Table 1).
gether in the analysis. In relation to the objective functions, two para-
The sensitivity ranking is defined by the evaluation of two coeffi-
meters sets were obtained, one for NS and one for KGE. The Global
cients: the t-stat index, obtained dividing the parameter coefficient
Sensitivity Analysis was carried out using SUFI-2 and 1 iteration with
found in the multiple regression analysis by the standard error; and the
2000 runs was made to activate the sensitive parameters. The initial
p-value hypothesis test (Student’s test). The t-stat index is a better choice
parameters were all the listed in the model manual, a total of 25 were
for the modeler to understand the magnitude of the sensibility, while
selected, where 20 were used in the sequential calibration and 17 in the
the p-value is better to define which parameters are more sensitive; we
simultaneous one. Related to the process influence, 17 are for stream-
use the p-value to define the most sensitive parameters. A low p-value
flow influence, and 8 are for suspended sediment
(≤0.05) indicates that the null hypothesis can be rejected, the predictor
that has low value is likely to be a meaningful addition to the model,
because changes in the predictor’s value are related to changes in the

105
T.M. Brighenti et al. Ecological Engineering 127 (2019) 103–113

Table 1
Parameter range and calibration used in the Sensitivity Analysis. The first letter before the parameters name indicates the adjusted method in SUFI-2. The check mark
indicates that the model is sensitive to the parameter for each calibration method. The process column indicates if the parameter is mainly related to streamflow or
suspended sediment.
Parameters Range Calibration Method Process

Max. Min. Simultaneous Sequential

A__GWQMN.gw −1000.00 1000.00 ✓ ✓ streamflow

V__GW_REVAP.gw 0.020 0.20 ✓ ✓ streamflow
A__RCHRG_DP.gw −0.05 0.05 ✓ streamflow
A__GW_DELAY.gw −56.33 −8.45 ✓ streamflow
R__CN2.mgt −0.10 0.10 ✓ ✓ streamflow
R__CH_W2.rte −0.15 0.15 ✓ streamflow
R__CH_S2.rte −0.15 0.15 ✓ ✓ streamflow
V__CH_N2.rte 0.010 0.20 ✓ ✓ streamflow
V__CH_K2.rte 0 10 ✓ ✓ streamflow
R__CH_L2.rte −0.09 0.012 ✓ streamflow
V__CH_COV1.rte 0 1 ✓ s. sediment
V__CH_COV2.rte 0 1 ✓ ✓ s. sediment
V__CH_BED_D50.rte 100 6000 ✓ ✓ s. sediment
V__LAT_TTIME.hru 0 100 ✓ streamflow
V__ESCO.hru 0.7 0.95 ✓ streamflow
R__SLSUBBSN.hru −0.2 0.2 ✓ ✓ s. sediment
R__OV_N.hru −0.29 −0.078 ✓ streamflow
V__LAT_SED.hru 0 5000 ✓ ✓ s. sediment
V__SPCON.bsn 0.0001 0.01 ✓ ✓ s. sediment
V__SPEXP.bsn 1 1.5 ✓ ✓ s. sediment
V__SURLAG.,bsn 5.54 14 ✓ streamflow
R__MSK_CO1.bsn −0.004 0.46 ✓ streamflow
R__MSK_CO2.bsn −0.63 0.01 ✓ streamflow
R__USLE_C{6}.plant.dat −0.2 0.2 ✓ s. sediment
R__USLE_C{7}.plant.dat −0.2 0.2 ✓ s. sediment
R__SOL_BD.sol −0.23 −0.07 ✓ streamflow

2.5. Calibration process variables (e.g., streamflow and suspended sediment); (iii) multi-site, the
parameters are calibrated against different sites (e.g., RB10 and RB11);
Considering the objective of this study, the SWAT model was only and (iv) multi-objective, more than one objective function is used in
calibrated, not validated and the SUFI-2 algorithm was used. The pro- calibration (Nash-Sutcliffe and King-Gupta Efficiency). All these pro-
cedure requires objective functions, which will optimize the search in cedures are mixed with the two calibration methods.
the parameter space and find the best combinations that reflect the The final model parameter ranges are always conditioned on the
watershed and data characteristics (Sun et al., 2017). The uncertainty form of the objective function (Shrestha et al., 2016). Actually, there
analysis is a consequence of calibration process and will deal with the are several objective functions in the literature, including statistically
equifinality phenomena. Fig. 2 illustrates the calibration process, the based and maximum likelihood (Oudin et al., 2006). In this study we
follow approaches are: (i) the model run 2000 times, for 4 iterations; focus our choice in two purposes, a simple easy interpretation and
(ii) multi-variable, the parameters are calibrated on different observed highly literature recommendation. Then, to activate the multi-objective

Fig. 2. Diagram of the calibration methods.

106
T.M. Brighenti et al. Ecological Engineering 127 (2019) 103–113

calibration method two objective function were chosen, the Nash-Sut- nXin
p - factor =
cliffe (NS) (Nash and Sutcliffe, 1970) and King-Gupta Efficiency (KGE) n
(Gupta et al., 2009). These two formulations were widely used and 1 n
∑ti= 1 (XtM − XtM )
suggested in SWAT and hydrological model application (e.g., Schaefli r - factor = n i,97.5% i,2.5%

and Gupta, 2007; Asadzadeh et al., 2016; Guse et al., 2017; Kayastha σobs
et al., 2013; Nicolle et al., 2014; Magand et al., 2015; Osuch et al., where XtM and XtM represent the upper and lower simulated
i,97.5% i,2.5%
2015). When a single indicator is used may lead to incorrect verification boundaries at the time ti of the 95PPU; n is the number of observed data
of the model. Instead, a combination of graphical results, absolute value points; M refers to modeled; ti is the simulation time step; σobs stands for
error, and normalized goodness-of-fit statistics is currently re- the standard deviation of the measured data; and nXin is the number of
commended (Ritter and Muñoz-Carpena, 2013). Therefore, together observed data in the 95PPU interval.
with the statistical indexes (NS, KGE, Pbias, p-factor, r-factor, r2), a
graphical analysis (hydrographs, duration curves, histograms and 2.6. Uncertainty analysis of observed data
scatter plots) were used to compare the simulated and observed data.
The NS is the most used objective function in hydrological modeling To incorporate the measured data errors in uncertainty analysis, the
and in SWAT model application. It is a normalized statistic that de- Topping (1972)formulation was used. The formulation is widely ac-
termines the relative magnitude of the residual variance compared to cepted (Bonumá et al., 2013; Harmel et al., 2006; Rodrigues et al.,
the measured data variance, its priority to high values of sampling is a 2015). The error value for each measured data is provided by Harmel
common point discussed by various authors (Asadzadeh et al., 2016; et al. (2006) and the best scenarios were chosen. Uncertainty infers in
Guse et al., 2017; Moriasi et al., 2007; Schaefli and Gupta, 2007). The streamflow and suspended sediment measured values was based on the
NS values range between -∞ to 1, where 1 is a perfect simulation, zero following methods: velocity-area method, stage-discharge relationship,
representing balance accuracy, and observations below zero represent and automated sampling. These values are incorporated in SUFI-2 si-
unacceptable model performance. According to Moriasi et al. (2007) mulations and used to calculate the p-factor.
the NS values ≥ 0.5 were considered satisfactory model performance 1
for flow and sediment. ⎡1
n
⎤
2
Ep = ⎢ ∑ (ds2 ) ⎥
nobs sim 2 n
⎡ ∑ (Xi − Xi ) ⎤ ⎣ s=1 ⎦
NS = 1 − ⎢ ni = 1 obs
⎣ ∑i = 1 (Xi − X
mean )2 ⎥
⎦ where Ep is the probable error range ( ± %); n is the number of po-
tential errors sources; and ds2 are the uncertainties associated with each
where Xisim is simulated values; Xiobs is observed values; X mean is the potential error source.
mean of n observed values.
The KGE is the decomposition of NS in three components (alpha, 3. Results and discussion
beta and r), which can be separately considered at each iteration, if
necessary. The formulation also allows for unequal weighting of the Fig. 3 introduces the observed data of the study area; rainfall,
three components if one wishes to emphasize certain areas of the ag- streamflow and suspended sediment. The maximum amount of rainfall
gregate function tradeoff space. The KGE ranges from −∞ to 1, a in a day was 115 mm, which led to the higher streamflow discharge for
performance above 0.75 and 0.5 is considered to be as good and in- both basins, RB10 and RB11. Instead, the largest suspended sediment
termediate, respectively. The ideal coefficient value is one. discharge occurred during the second highest daily rainfall (92 mm).
The shape of the streamflow and suspended sediment response in RB10
KGE = 1 − (r − 1)2 + (α − 1)2 + (β − 1)2
and RB11 basins is almost the same, the difference is in the magnitude
where r is the correlation coefficient between observed and simulated of the values in each output. The shaded area of the plot (RB10) is the
values; α is the measure of relative variability in the simulated and period that is showed in the results simulation.
observed values; and β is the bias normalized by the standard deviation The differences between the two calibration methods are identified
in the observed values. through the evaluation of plot illustrations to visual inspections and
The model performance was evaluated using four well-known sta- statistical coefficients. The results and discussions presented in this
tistical criteria: NS; KGE; Percent bias (Pbias) (Gupta et al., 1999); p- section are based on the simulations values from RB10, the basin outlet.
factor and r-factor (Abbaspour et al., 2004). The NS and KGE coeffi- For general aspects, the fit between streamflow and suspended sedi-
cients were used in the SUFI-2 algorithm in calibration process; the ments predictions through observed data showed good agreement as
Pbias, r-factor and p-factor statistical indexes were used only for model indicated the evaluation of KGE (Table 2) and NS (Table 3), for both
evaluation after calibration. They were chosen because assess different sequential and simultaneous methods. The KGE for simultaneous and
modeling skills. The Pbias evaluates the trend that the average of the sequential streamflow calibration are 0.75 and 0.63; and for suspended
simulated values has in relation to the observed ones. The ideal value of sediment are 0.67 and 0.78, respectively. The NS for simultaneous and
Pbias is zero (%); a good model performance could be ± 25% for sequential streamflow calibration are 0.60 and 0.67; and for suspended
streamflow and ± 55% for suspended sediment, positive values in- sediment are 0.62 and 0.57, respectively. These values were within the
dicate a model underestimation and negative values overestimation. acceptable level reported by Moriasi et al. (2007), showing that the
quality of streamflow simulations was maintained with the suspended
n obs sim 2
⎡ ∑ (Xi − Xi ) ⎤ sediment calibration. The Pbias was calculated through the best results
Pbias = ⎢ i = 1 n ⎥ ∗ 100
obs
∑i = 1 (Xi ) of each objective function, the values range from 5.9 to 18 for
⎣ ⎦
streamflow and −25.6 to 28.4 for suspended sediment, all values are
where Xisim is simulated values; and Xiobs is observed values. consistent with Moriasi et al. (2007), that are ± 25% for streamflow
The p-factor and r-factor are the two uncertainty measured offered and ± 55% for suspended sediment. To measure the uncertainty in si-
by SUFI-2. The p-factor is the percentage of observed data bracketed by mulation the p-factor and r-factor are the metrics used; the values are
the 95PPU. The r-factor is equal to the average thickness of 95PPU band calculated based on the 95PPU. The streamflow p-factor values range
divided by the standard deviation of the observed data. A p-factor of one from 0.49 to 0.83, while those for suspended sediment range from 0.81
and r-factor of zero is a simulation that exactly matches the observed to 0.98. The r-factor values range from 0.42 to 0.66 for streamflow, and
data. However, p-factor above of 0,70 and a r-factor under of 1,5 can be 1.22 to 5.82 for suspended sediment. All the metrics results are pre-
considered satisfactory simulations (Abbaspour et al., 2015). sented in Table 2.

107
T.M. Brighenti et al. Ecological Engineering 127 (2019) 103–113

Fig. 3. Observed rainfall (mm/dia), streamflow (m3/s) and suspended sediment (ton/day) for the RB10 and RB11 basin. The shaded area of the plot is the period that
is showed in results for hydrographs representation.

The bars next to the values in the Table 2 are representation of Table 3
which is the best simulation when compared between them. For ex- Performance of the SWAT Model for KGE and NS objective functions in RB10
ample, the best value for KGE is 0.78 for suspended sediment in se- basin, without RB11 station. The values of KGE, NS and Pbias are for the best
quential calibration method; the worse value for KGE/p-factor is 0.49 simulation, the p-factor and r-factor are for the 95PPU. The bold values means
for streamflow in simultaneous calibration. the best simulation, between simultaneous and sequential method.
Analyzing the objective functions, KGE had more acceptable values Calibration KGE/NS Pbias p-factor r-factor
with the simultaneous calibration, 5 out of 8; using the NS as the ob- Method
Q SS Q SS Q SS Q SS
jective function, the opposite happened, where 5 out of every 8 times
the best values were found for the sequential calibration. When we look Sequential/KGE 0.38 −5.52 −18.20 100.00 0.71 0.80 0.63 21.07
for the process, streamflow and suspended sediment, the proportion is Simultaneous/ 0.77 0.73 0.10 0.60 0.50 0.97 0.44 2.35
the same, 50% for sequential and 50% for simultaneous calibration KGE
method. Therefore, with this simple results comparison, no much dif- Sequential/NS 0.66 0.56 −1.50 50.00 0.86 0.76 0.79 0.73
Simultaneous/ 0.62 0.59 0.50 29.00 0.52 0.93 0.43 1.23
ference between two simulation methods is noted, leading to the con- NS
clusion that either can be used without any big differences in the re-
sults. However, looking in more detail, mainly for r-factor values, the
uncertainty envelope of simultaneous calibration method is always In Fig. 4 are represented the hydrographs resulting from calibrations
smaller and has more consistent values. In suspended sediments, the r- processes (the shaded area of Fig. 3). The graphs show the uncertainty
factor range in sequential calibration is not acceptable, much larger envelope, blue for KGE simulations and red for NS; the red dotted line is
than the recommended in Abbaspour et al. (2015), making the p-factor the best simulation, the observed data are represented by black points.
values a false positive. The error bar is the uncertainty calculated by Topping equation, 20%

Table 2
Performance of the SWAT Model for the KGE and NS objective functions in RB10 basin. The values of KGE, NS and Pbias are for the best simulation, the p-factor and r-
factor are for the 95PPU. The blue bars indicate how good the result is, comparing to the study itself.

108
T.M. Brighenti et al. Ecological Engineering 127 (2019) 103–113

Fig. 4. Hydrographs results representation for calibrations processes (the shaded area of Fig. 3). The graphs show the uncertainty envelope (95PPU), the blue is for
KGE objective function simulations and the red is for NS; the red dotted line is the best simulation result, the observed data is represented by the black points. The
error bar is the uncertainty calculated by Topping equation, 20% for streamflow and 33% for suspended sediment. The duration curves are shown at the top right
corner of the hydrographs and were calculated with the entire time series data used in the calibration process. (For interpretation of the references to colour in this
figure legend, the reader is referred to the web version of this article.)

for streamflow and 33% for suspended sediment. The duration curves basis, which can be insufficient to capture the basin behavior during
calculated with the entire time series data used in the process are shown low suspended sediment discharge. The histograms and the scatter plots
at the top right corner of the hydrographs. The comparisons for the of observed data and the best simulation are shown in Fig. 5.
observed and model simulated streamflow show an overall good The frequency distribution that best fits with the streamflow ob-
agreement in basin patterns with some discrepancies in low values, for served data is in the KGE simultaneous calibration results, for sus-
both sequential and simultaneous calibration. The best duration curve pended sediments the NS simultaneous distribution show more con-
fit is in KGE simultaneous calibration, but the smaller uncertainty range sistency. The highest histograms heights are always corresponding,
is in NS sequential calibration. In suspended sediment the best curve showing a positive association between the two variables. In the scatter
duration fit and the smaller uncertainty range are in simultaneous ca- plot analysis the strongest relationship for streamflow is in NS se-
libration, even with the NS calibration having more than 50% of the quential calibration, for suspended sediments in NS simultaneous cali-
simulation values equal to zero. None of the model configurations was bration. An improvement by simultaneous calibration is also noted in
able to capture the suspended sediment low values, which might be the values magnitude of the simulated peaks (see Fig. 4). The sequential
attributed to the very small values of observed data. We raised two calibration, for example, simulates a suspended sediment yield of ap-
hypotheses, which should be investigated in future, for this fact. One, is proximately 80 ton/day, for KGE objective function, when the max-
the use of Bagnold equation. According to Wu and Chen (2012), Bag- imum observed value on this day was approximately 6 tons/day. Si-
nold method behaves better for simulating high sediment concentration multaneous calibration also presents high simulated values for this
in high flows. The other is regarding the simulation time step, daily event, but the values do not exceed 15 ton/day.

109
T.M. Brighenti et al. Ecological Engineering 127 (2019) 103–113

Fig. 5. Relationship between the observed and simulated data for streamflow and suspend sediment. On the right, the histograms with the frequency distributions; on
the left, the respective scatter plot with the reference lines.

Fig. 6 demonstrates the objective functions values for the 2000 si- line are behavioral solutions with the best parameters estimation. For
mulations of the last model iteration. The red line is the threshold for the two methods, the majority of streamflow simulation are above limit.
the behavioral solutions, we choose the limit of 0.5 for streamflow and For suspended sediment, the numbers of behavioral solutions are dif-
suspended sediment, for both objective functions. The dots above the ferent from one method to another, the method that presents the

110
T.M. Brighenti et al. Ecological Engineering 127 (2019) 103–113

Fig. 6. The behavioral simulation representation n. The black dots are the objective functions values for the 2000 simulations of the last model iteration. The red line
is the threshold for the behavioral solution. The limit is: 0.5 for streamflow and suspended sediment, for both objective functions. (For interpretation of the references
to colour in this figure legend, the reader is referred to the web version of this article.)

greatest amount of acceptable results is the KGE simultaneous method In the hydrographs and the flow duration curve analyses we noticed
with, 38% of behavioral solutions. that while the model simulations for baseflow conditions were less sa-
In an attempt to validate the conclusions, we made a small mod- tisfactory, there was a general good fit for medium and high values. The
ification in the RB10 basin calibration process. The modification con- total uncertainty range was smaller in simultaneous calibration, for
sists in a calibration process simplification, where the RB11 station was both streamflow (Q) and suspended sediments (SS). In the frequency
removed of the process, meaning that multi-site calibration no longer distribution, the highest histograms heights are always corresponding,
exists. Summarizing, the patterns are same, and the simultaneous ca- showing a positive association between the simulated and observed
libration has more satisfactory results. 7 of 8 for NS objective function values. Regarding to the behavioral solution in the calibration methods
and 5 of 8 for KGE (Table 3). We also found that multi-site calibration the majority of streamflow simulations are above limit. For suspended
technique improves the results in 3 out of every 4 simulations for sus- sediment, the numbers of behavioral solutions are different from one
pended sediments, indicating the importance of including multiple method to another; the method that presents the greatest amount of
monitoring points in SS model calibration. For streamflow, the im- acceptable results is the KGE simultaneous method, with 38% of be-
provement was not significant, while both the sequential and the si- havioral solutions. Therefore, the simultaneous calibration presented
multaneous simulations using NS were similar; one had better results advantages in relation to the sequential.
using the KGE sequential rather that with the KGE simultaneous si- Additionally, we also found that multi-site calibration method im-
mulation (Tables 2 and 3). proves the results in 3 of 4 simulations for suspended sediments, in-
Furthermore, the simultaneous calibration is less time consuming; dicating the importance of including multiple monitoring points in SS
in sequential we have a total of 7 iterations (3 for streamflow and 4 for model calibration.
suspended sediment), counts 4 iterations of the simultaneous calibra- Since our analysis has been made for a small basin with two mon-
tion. The number of parameters required in the simultaneous calibra- itoring points, it is important that the study be extended to larger basins
tion process is also smaller, which according to Daggupati et al. (2015) with more monitoring points. Also, the use of a multi-objective ap-
and Finger et al. (2012) reduces the equifinality problems, since it is proach (e.g., Oudin et al., 2006) an alternative to hydrological model
expected that fewer model parameters sets can satisfy the calibration calibration. Finally, the use of sub-daily data with a different SS
criteria. That might indicate which field data are more important to be equation is also recommended, in order to correct the low values si-
monitored in order to improve calibration and the understanding of the mulation problems.
dominant streamflow generation processes in the basin.
Acknowledgements
4. Conclusions
The authors would like to thank the Associate Editor and the two
This study investigated the impact of sequential and simultaneous anonymous reviewers whose suggestions greatly improved the manu-
calibration methods for streamflow (Q) and suspended sediment (SS) script; the CNPq (National Council for Scientific and Technological
simulations using the SWAT. We have analyzed the differences between Development) and PPGEA-UFSC (Environmental Engineering Pos-
the two methods through statistical indexes and graphical representa- Graduation Program of Federal University of Santa Catarina) for a PhD
tion. financial support; and to Hydrology Laboratory team-UFSC for the
We found that it is important to analyze a combination of statistical support during the research time.
indexes and graphical solutions, to reduce the subjectivity commonly
introduced by only one evaluation approach. More specifically, when References
KGE was used as objective function the results were more acceptable in
simultaneous calibration, for NS the best values were in sequential Abbaspour, K.C., Faramarzi, M., Ghasemi, S.S., Yang, H., 2009. Assessing the impact of
climate change on water resources in Iran. Water Resour. Res. 45, 1–16. https://doi.
calibration. The quality of streamflow simulation was maintained
org/10.1029/2008WR007615.
during the suspended sediment calibration.

111
T.M. Brighenti et al. Ecological Engineering 127 (2019) 103–113

Abbaspour, K.C., Johnson, C.A., van Genuchten, M.T., 2004. Estimating uncertain flow watershed, France. Hydrol. Sci. J. 60, 1408–1423. https://doi.org/10.1080/
and transport parameters using a sequential uncertainty fitting procedure. Vadose Zo. 02626667.2014.993643.
J. 3, 1340. https://doi.org/10.2136/vzj2004.1340. Me, W., Abell, J.M., Hamilton, D.P., 2015. Effects of hydrologic conditions on SWAT
Abbaspour, K.C., Rouholahnejad, E., Vaghefi, S., Srinivasan, R., Yang, H., Kløve, B., 2015. model performance and parameter sensitivity for a small, mixed land use catchment
A continental-scale hydrology and water quality model for Europe: calibration and in New Zealand. Hydrol. Earth Syst. Sci. 19, 4127–4147. https://doi.org/10.5194/
uncertainty of a high-resolution large-scale SWAT model. J. Hydrol. 524, 733–752. hess-19-4127-2015.
https://doi.org/10.1016/j.jhydrol.2015.03.027. Moriasi, D., Wilson, B., 2012. Hydrologic and water quality models: use, calibration, and
Abeyou, W.W., Ayana, E.K., Yen, H., Jeong, J., MacAlister, C., Taylor, R., Gerik, T.J., validation. Am. Soc. Agric. Biol. Eng. 55, 1241–1247. https://doi.org/10.13031/
Steenhuis, T.S., 2018. Evaluating hydrologic responses to soil characteristics using 2013.42265.
SWAT model in a paired-watersheds in the Upper Blue Nile Basin. Catena 163, Moriasi, D.N., Arnold, J.G., Van Liew, M.W., Binger, R.L., Harmel, R.D., Veith, T.L., 2007.
332–341. https://doi.org/10.1016/j.catena.2017.12.040. Model evaluation guidelines for systematic quantification of accuracy in watershed
Andreassian, V., Perrin, C., Berthet, L., Moine, N Le, Lerat, J., Loumagne, C., Oudin, L., simulations. Trans. ASABE 50, 885–900. https://doi.org/10.13031/2013.23153.
Mathevet, T., Ramos, M.-H., Valery, A., 2009. HESS Opinions “Crash tests for a Nash, J.E., Sutcliffe, J.V., 1970. River flow forecasting through conceptual models part I –
standardized evaluation of hydrological models”. Earth 1757–1764. a discussion of principles. J. Hydrol. 10, 282–290. https://doi.org/10.1016/0022-
Arnold, J.G., Moriasi, D.N., Gassman, P.W., Abbaspour, K.C., White, M.J., Srinivasan, R., 1694(70)90255-6.
Santhi, C., Harmel, R.D., Griensven, A Van, VanLiew, M.W., Kannan, N., Jha, M.K., Neitsch, S., Arnold, J., Kiniry, J., Williams, J., 2011. Soil & water assessment tool theo-
2012. Swat: model use, calibration, and validation. ASABE 55, 1491–1508. retical documentation version 2009. Texas Water Resour. Inst. 1–647. https://doi.
Arnold, J.G., Youssef, M.A., Yen, H., White, M.J., Sheshukov, A.Y., Sadeghi, A.M., org/10.1016/j.scitotenv.2015.11.063.
Moriasi, D.N., Steiner, J.L., Amatya, D.M., Skaggs, R.W., Haney, E.B., Jeong, J., Nicolle, P., Pushpalatha, R., Perrin, C., François, D., Thiéry, D., Mathevet, T., Le Lay, M.,
Arabi, M., Gowda, P.H., 2015. Hydrological processes and model representation: Besson, F., Soubeyroux, J.M., Viel, C., Regimbeau, F., Andréassian, V., Maugis, P.,
impact of soft data on calibration. Trans. ASABE 58, 1637–1660. https://doi.org/10. Augeard, B., Morice, E., 2014. Benchmarking hydrological models for low-flow si-
13031/trans.58.10726. mulation and forecasting on French catchments. Hydrol. Earth Syst. Sci. 18,
Asadzadeh, M., Leon, L., Yang, W., Bosch, D., 2016. One-day offset in daily hydrologic 2829–2857. https://doi.org/10.5194/hess-18-2829-2014.
modeling: an exploration of the issue in automatic model calibration. J. Hydrol. 534, Osuch, M., Romanowicz, R.J., Booij, M.J., 2015. The influence of parametric uncertainty
164–177. https://doi.org/10.1016/j.jhydrol.2015.12.056. on the relationships between HBV model parameters and climatic characteristics.
Beven, K., 2006. A manifesto for the equifinality thesis. J. Hydrol. 320, 18–36. https:// Hydrol. Sci. J. 60, 1299–1316. https://doi.org/10.1080/02626667.2014.967694.
doi.org/10.1016/j.jhydrol.2005.07.007. Oudin, L., Andréassian, V., Mathevet, T., Perrin, C., Michel, C., 2006. Dynamic averaging
Beven, K., Binley, A., 1992. The future of distributed models: model calibration and of rainfall-runoff model simulations from complementary model parameterizations.
uncertainty preditiction. Hydrol. Process. 6, 279–298. Water Resour. Res. 42, 1–10. https://doi.org/10.1029/2005WR004636.
Beven, K., Smith, P.J., Wood, A., 2011. On the colour and spin of epistemic error (and Ritter, A., Muñoz-Carpena, R., 2013. Performance evaluation of hydrological models:
what we might do about it). Hydrol. Earth Syst. Sci. 15, 3123–3133. https://doi.org/ Statistical significance for reducing subjectivity in goodness-of-fit assessments. J.
10.5194/hess-15-3123-2011. Hydrol. 480, 33–45. https://doi.org/10.1016/j.jhydrol.2012.12.004.
Bonumá, N.B., Rossi, C.G., Arnold, J.G., Reichert, J.M., Paiva, E.M.C.D., 2013. Hydrology Rodrigues, D.B.B., Gupta, H.V., Mendiondo, E.M., Oliveira, P.T.S., 2015. Assessing un-
evaluation of the soil and water assessment tool considering measurement un- certainties in surface water security: an empirical multimodel approach. Water
certainty for a small watershed in Southern Brazil. Appl. Eng. Agric. 29, 189–200. Resour. Res. Res. 9013–9028. https://doi.org/10.1002/2014WR015432.Received.
https://doi.org/10.13031/2013.42651. Santhi, C., Arnold, J.G., Williams, J.R., Dugas, W.A., Srinivasan, R., Hauck, L.M., 2001.
Cardoso, A.T., 2013. Estudo Hidrossedimentológico em Três Bacias Embutidas no Validation of the SWAT model on a large river basin with point and nonpoint sources.
Município de Rio Negrinho – SC. J. Am. Water Resour. Assoc. 37, 1169–1188. https://doi.org/10.1111/j.1752-1688.
Daggupati, P., Pai, N., Ale, S., Douglas-Mankin, K.R., Zeckoski, R.W., Jeong, J., Parajuli, 2001.tb03630.x.
P.B., Saraswat, D., Youssef, M.A., 2015. A recommended calibration and validation Santhi, C., Arnold, J.G., Williams, J.R., Dugas, W.A., Srinivasan, R., Hauck, L.M., 2002.
strategy for hydrologic and water quality models. Trans. ASABE 58, 1705–1719. Validation of the SWAT model on a large river basin with point and nonpoint sources.
https://doi.org/10.13031/trans.58.10712. J. Am. Water Resour. Assoc. 37, 1169–1188. https://doi.org/10.1111/j.1752-1688.
Fenicia, F., Savenije, H.H.G., Matgen, P., Pfister, L., 2006. Is the groundwater reservoir 2001.tb03630.x.
linear? Learning from data in hydrological modelling. Hydrol. Earth Syst. Sci. Schaefli, B., Gupta, H.V., 2007. Do Nash values have value? Bettina. Hydrol. Process. 21,
Discuss. 10, 139–150. https://doi.org/10.5194/hessd-2-1717-2005. 2075–2080. https://doi.org/10.1002/hyp.
Finger, D., Heinrich, G., Gobiet, A., Bauder, A., 2012. Projections of future water re- Seibert, J., Mcdonnell, J.J., 2003. Calibration of Watershed models, sixth ed.
sources and their uncertainty in a glacierized catchment in the Swiss Alps and the Shrestha, B., Cochrane, T.A., Caruso, B.S., Arias, M.E., Piman, T., 2016. Uncertainty in
subsequent effects on hydropower production during the 21st century. Water Resour. flow and sediment projections due to future climate scenarios for the 3S Rivers in the
Res. 48, 1–20. https://doi.org/10.1029/2011WR010733. Mekong Basin. J. Hydrol. 540, 1088–1104. https://doi.org/10.1016/j.jhydrol.2016.
Folton, N., Andréassian, V., Duperray, R., 2015. Hydrological impact of forest-fire from 07.019.
paired-catchment and rainfall–runoff modelling perspectives. Hydrol. Sci. J. 60, Song, X., Zhang, J., Zhan, C., Xuan, Y., Ye, M., Xu, C., 2015. Global sensitivity analysis in
1213–1224. https://doi.org/10.1080/02626667.2015.1035274. hydrological modeling: review of concepts, methods, theoretical framework, and
Gassman, P.W., Sadeghi, A.M., Srinivasan, R., 2014. Applications of the SWAT model applications. J. Hydrol. 523, 739–757. https://doi.org/10.1016/j.jhydrol.2015.02.
special section: overview and insights. J. Environ. Qual. 43, 1. https://doi.org/10. 013.
2134/jeq2013.11.0466. Sun, W., Wang, Y., Wang, G., Cui, X., Yu, J., Zuo, D., Xu, Z., 2017. Physically based
Grison, F., 2013. Estudo da Geometria Hidráulica do Rio dos Bugres, do Município de Rio distributed hydrological model calibration based on a short period of streamflow
Negrinho – SC. data: Case studies in four Chinese basins. Hydrol. Earth Syst. Sci. 21, 251–265.
Gupta, H.V., Sorooshian, S., Yapo, P.O., 1999. Status of automatic calibration for hy- https://doi.org/10.5194/hess-21-251-2017.
drologic models: comparison with multilevel expert calibration. J. Hydrol. Eng. Surfleet, C.G., Tullos, D., 2013. Uncertainty in hydrologic modelling for estimating hy-
135–143. https://doi.org/10.1002/fut.20174. drologic response due to climate change (Santiam River, Oregon). Hydrol. Process.
Gupta, H.V., Sorooshian, S., Yapo, P.O., 1998. Toward improved calibration of hydrologic 27, 3560–3576. https://doi.org/10.1002/hyp.9485.
models: multiple and noncommensurable measures of information. Water Resour. Thirel, G., Andréassian, V., Perrin, C., 2015. On the need to test hydrological models
Res. 34, 751–763. https://doi.org/10.1029/97WR03495. under changing conditions. Hydrol. Sci. J. 60, 37–41. https://doi.org/10.1080/
Gupta, H.V., Kling, H., Yilmaz, K.K., Martinez, G.F., 2009. Decomposition of the mean 02626667.2015.1050027.
squared error and NSE performance criteria: implications for improving hydrological Topping, J., 1972. Errors of Observation and Their Treatment, fourth ed. Chapman and
modelling. J. Hydrol. 377, 80–91. https://doi.org/10.1016/j.jhydrol.2009.08.003. Hall, London.
Guse, B., Pfannerstill, M., Gafurov, A., Kiesel, J., Lehr, C., Fohrer, N., 2017. Identifying Van Griensven, A., Bauwens, W., 2003. Multiobjective autocalibration for semidistributed
the connective strength between model parameters and performance criteria. Hydrol. water quality models. Water Resour. Res. 39, 1–9. https://doi.org/10.1029/
Earth Syst. Sci. Discuss. 1–30. https://doi.org/10.5194/hess-2017-28. 2003WR002284.
Harmel, R.D., Cooper, R.J., Slade, R.M., Haney, R.L., Arnold, J.G., 2006. Cumulative Van Griensven, A., Ndomba, P., Yalew, S., Kilonzo, F., 2012. Critical review of SWAT
uncertainty in measured streamflow and water quality data for small watersheds. applications in the upper Nile basin countries. Hydrol. Earth Syst. Sci. 16,
ASABE 49, 689–701. 3371–3381. https://doi.org/10.5194/hess-16-3371-2012.
Hassan, M.A., Church, M., Yan, Y., Slaymaker, O., 2010. Spatial and temporal variation of Vandenberghe, V., Griensven, A. Van, Bauwens, W., Vanrolleghem, P.A., 2006. Effect of
in – reach suspended sediment dynamics along the mainstem of Changjiang (Yangtze different river water quality model concepts used for river basin management deci-
River), China. Water Resour. Res. 46, 1–14. https://doi.org/10.1029/ sions 277–284. https://doi.org/10.2166/wst.2006.322.
2010WR009228. Vrugt, J., Beven, K., 2016. Embracing equifinality with efficiency: limits of acceptability
Kirchner, J.W., 2006. Getting the right answers for the right reasons: linking measure- sampling using the DREAM_ABC. J. Hydrol.
ments, analyses, and models to advance the science of hydrology. Water Resour. Res. Wang, J., Li, X., Lu, L., Fang, F., 2013. Parameter sensitivity analysis of crop growth
42, 1–5. https://doi.org/10.1029/2005WR004362. models based on the extended Fourier Amplitude Sensitivity Test method. Environ.
Klemeš, V., 1986. Operational testing of hydrological simulation models. Hydrol. Sci. J. Model. Softw. 48, 171–182. https://doi.org/10.1016/j.envsoft.2013.06.007.
31, 13–24. https://doi.org/10.1080/02626668609491024. Wellen, C., Arhonditsis, G.B., Long, T., Boyd, D., 2014. Quantifying the uncertainty of
Krysanova, V., Srinivasan, R., 2015. Assessment of climate and land use change impacts nonpoint source attribution in distributed water quality models: a Bayesian assess-
with SWAT. Reg Environ. Chang. 15, 431–434. https://doi.org/10.1007/s10113- ment of SWAT’s sediment export predictions. J. Hydrol. 519, 3353–3368. https://doi.
014-0742-5. org/10.1016/j.jhydrol.2014.10.007.
Magand, C., Ducharne, A., Le Moine, N., Brigode, P., 2015. Parameter transferability Wu, H., Chen, B., 2015. Evaluating uncertainty estimates in distributed hydrological
under changing climate: case study with a land surface model in the Durance modeling for the Wenjing River watershed in China by GLUE, SUFI-2, and ParaSol

112
T.M. Brighenti et al. Ecological Engineering 127 (2019) 103–113

methods. Ecol. Eng. 76, 110–121. https://doi.org/10.1016/j.ecoleng., 2014.05.014. Zhang, X., Srinivasan, R., Arnold, J., Izaurralde, R.C., Bosch, D., 2011. Simultaneous
Wu, Y., Chen, J., 2012. Science of the Total Environment Modeling of soil erosion and calibration of surface flow and baseflow simulations: a revisit of the SWAT model
sediment transport in the East River Basin in southern China. Sci. Total Environ. 441, calibration framework. Hydrol. Process. 25, 2313–2320. https://doi.org/10.1002/
159–168. https://doi.org/10.1016/j.scitotenv.2012.09.057. hyp.8058.
Zeiger, S.J., Hubbart, J.A., 2016. Science of the Total Environment A SWAT model va- Zheng, F., Maier, H.R., Wu, W., Dandy, G.C., Gupta, H.V., Zhang, T., 2018. On lack of
lidation of nested-scale contemporaneous stream flow, suspended sediment and nu- robustness in hydrological model development due to absence of guidelines for se-
trients from a multiple-land-use watershed of the central USA. Sci. Total Environ. lecting calibration and evaluation data: demonstration for data-driven models. Water
572, 232–243. https://doi.org/10.1016/j.scitotenv.2016.07.178. Resour. Res. 54, 1013–1030. https://doi.org/10.1002/2017WR021470.

113