DOI: 10.1111/j.1539-6924.2012.01829.x
Risk Analysis, Vol. 33, No. 1, 2013
Capturing and Integrating Knowledge for Managing Risks in
Tunnel Works
Ibsen Chivatá Cárdenas,1,∗ Saad S. H. Al-jibouri,1 Johannes I. M. Halman,1
and Frits A. van Tol2
Risk-related knowledge gained from past construction projects is regarded as potentially extremely useful in risk management. This article describes a proposed approach to capture
and integrate risk-related knowledge to support decision making in construction projects. To
ameliorate the problem related to the scarcity of risks information often encountered in construction projects, Bayesian Belief Networks are used and expert judgment is elicited to augment available information. Particularly, the article provides an overview of judgment-based
biases that can appear in the elicitation of judgments for constructing Bayesian Networks and
the provisos that can be made in this respect to minimize these types of bias. The proposed
approach is successfully applied to develop six models for top risks in tunnel works. More
than 30 tunneling experts in the Netherlands and Germany were involved in the investigation
to provide information on identifying relevant scenarios than can lead to failure events associated with tunneling risks. The article has provided an illustration of the applicability of the
developed approach for the case of “face instability in soft soils using slurry shields.”
KEY WORDS: Bayesian Belief Networks; epistemic uncertainty; expert judgment elicitation; reliability
modeling; risk-related knowledge integration; risk modeling
project operations caused significant damage or injury to others.
Muir Wood,(2,3) Sowers,(4) Bea,(5) van Tol,(6)
Wearne,(7) and Mitchell(8) have all highlighted that
failures in ground-related construction projects are
mainly due to shortcomings in the use of available
and relevant knowledge rather than the result of uncertainty due to unknown factors. These shortcomings reflected situations in which tacit or explicit information, such as technological knowledge, design
assumptions, monitoring records, thresholds and tolerances, and risks, was ignored, improperly used, rejected, or not passed on by someone in the project.
The objective of traditional approaches to
project risk management is to identify those risks
that can lead to project failure and to implement effective strategies to manage them. Information on
the relevant causes and conditions in which failures arise is usually required as necessary inputs for
1. INTRODUCTION
According to the International Association of
Engineering Insurers (IAEI), more than €570 million of economic losses were incurred in 18 tunneling
projects worldwide between 1994 and 2005 because
of failures in underground construction projects.(1)
Those failures also led to an average delay in completion of 19 months. These specific failures were
events in which a collapse or excessive deformation
of significant parts of the works occurred, or where
1 Department of Construction Management and Engineering, Uni-
versity of Twente, Enschede, The Netherlands
Geoengineering, Delft University of Technology, 2600
GA, Delft, The Netherlands
∗ Address correspondence to Ibsen Chivatá Cárdenas, Department
of Construction Management and Engineering, University of
Twente, P.O. Box 217, 7500 AE, Enschede, The Netherlands; tel:
+31-53-895988, mobile: +31-6-57777783; fax: +31-53-489-2511;
c.cardenas@utwente.nl.
2 Section
92
C 2012 Society for Risk Analysis
0272-4332/13/0100-0092$22.00/1
Capturing and Integrating Risk-Related Knowledge
93
determining possible risk strategies. In the particular
case of tunneling, the Code of Practice for Risk Management of Tunnel Works(9) stipulates that the use
of a formalized risk management procedure to document the identification, evaluation and allocation of
risks is compulsory. Similarly, the Guidelines for tunneling risk management from the International Tunnelling Association(10) recommended the use of risk
analysis to identify, quantify risks and to visualize
their causes and effects as well as the course (chain)
of events. To evaluate the risks associated with road
tunnels in operation, risk analysis also became one
of the explicit requirements under the European
Union (EU) Directive (2004/54/EC).(11−13) Performing risk analysis for underground construction
projects, however, is not straightforward in practice. Explicit and integrated knowledge about the
relevant causes and conditions that lead to major
risks is often absent. Therefore, there is a need
to make this type of integrated knowledge available in order to facilitate risk analysis and management. In doing so it is very valuable to explore
the feasibility of developing models that integrate
knowledge about the risk factors in construction
projects.
The remainder of this article is divided into four
sections. Section 2 provides a review of the background literature. Section 3 describes the successive
process steps that have been followed in developing models that integrate knowledge on six important
risks in tunneling projects. In Section 4, the applicability of the developed approach for risk elicitation
and representation is illustrated for the case of “face
instability in soft soils using slurry shields.” In the final section, a number of important contributions and
limitations of this study are discussed, followed by
some conclusions.
ally recorded. As a consequence, the interrelationships between risk factors and project outcomes are
difficult to understand.(17) To overcome, to some extent, the lack of historical data, expert judgments
could be used as an alternative source to develop
construction risk-related knowledge. Examples of
this type of approach can be found in Bles et al.(20)
and in Choi et al.(21)
Expert judgment is often employed to bridge the
gap between hard evidence and unknown characteristics of a system. This “knowledge” is not certain, and entails an implicit level of assumed confidence.(22) It is nevertheless widely acknowledged that
experts, as compared with novice professionals, have
greater abilities to identify relevant domain-related
information.(23) Thus, the use of expert knowledge
increases the likelihood of delivering relevant information to decision-makers. Nevertheless, expert
judgments are prone to be affected by various factors leading to unreliable data. To increase the reliability in expert judgments, the literature provides
criteria and procedures for eliciting this information
while reducing the risk of obtaining biased judgments
(e.g., Cooke and Goossens,(24) Ayyub,(25) Garthwaite
et al.,(26) Goossens et al.,(22) Kynn,(27) and Hallowell
and Gambatese(28) ).
Further, integrating expert judgments is not
straightforward. Under conditions of limited information, a diversity in judgments will usually emerge.
Discrepancies in data are a sign that there may be
epistemic uncertainties allowing different experts’
views to differ.(29) One reason for a diversity in judgments arises from the differences in the experts’
experiences regarding the events under consideration.(19) Literature offers various aggregation methods for discrepant judgments, however, capturing,
processing, and using information conveying epistemic uncertainties is a subject that has yet to be satisfactorily resolved.(30)
An initial investigation, based on specialized literature and explorative interviews, to determine the
information required for this study allowed additional elements to be considered. Some of the mechanisms leading to failure events remain little known
and might well consist of a large number of interacting factors. Hence, the integration of risk-related
knowledge demands appropriate methods that represent uncertain knowledge and capture strongly interactive risks factors. There are many tools, with different purposes, for representing risks. Failure mode
and effects analysis, hazard analysis, top-level event
tree, and fault tree analysis are all methods regularly
2. BACKGROUND LITERATURE
Although advances have been made in developing knowledge-based risk systems based on experience gained from past projects (e.g., Tah and Carr(14) ;
De Zoysa and Russell(15) ; Choi and Mahadevan(16) ;
Dikmen et al.(17) ), work remains to be done in capturing, representing, and using risk-related knowledge.(15,18) Capturing relevant knowledge on risks is
constrained by the fact that historical data on failures are usually scarce, often confidential, and not
available until several years after the event.(7,19) Similarly, information on the circumstances under which
project outcomes were not accomplished is not usu-
94
used to represent risks.(31) However, these standard
approaches would struggle to deal with the integration and analysis of uncertain information regarding
construction risks.
Recent research has proposed the Bayesian
Belief Networks (BBNs) as a more suitable alternative for representing risks.(32,33) BBNs are especially suited for representing complex and uncertain relationships among many factors that contribute to the realization of risk.(34) Besides BNNs,
there are other approaches to representing risks
such as Markov Chains, Petri Nets, Artificial Neural Networks (ANNs), Analytical Hierarchy Process
(AHP), Systems Dynamics, and Fuzzy Systems.(35)
Some of these (Markov Chains and Petri Nets) are
regarded as being too complicated for use by practitioners(36) and others, such as ANN, AHP, and Systems Dynamics, entail considerable modeling effort
and require abundant data which are usually unavailable in the case of construction risks. Fuzzy Systems
modeling would be an alternative to the BBNs approach, however, the BBN approach is based on a
probabilistic background and hence has a more developed mathematical axiomatization in comparison
to Fuzzy Systems approach. Dubois(37) and Novák(38)
have identified remaining questions to be addressed
in the field of Fuzzy Systems, among others, developing a general axiomatic framework for ranking purposes and the refinement of aggregation functions
which are important tasks in risk analysis. In addition, BBNs approach is simpler to use than Fuzzy
Systems. Its inference mechanism is based on the
Bayes theorem, which makes it possible to compute the probability of an effect on any variable in
the model from the probability of a given cause.
In comparison, Fuzzy System modeling relies on
fuzzy inference as described in Mamdani and Assilian(39) and Sugeno(40) . Accordingly, the process of
fuzzy inference usually requires additional computational effort that might exasperate the computation
costs. For instance, a number of membership functions, logical operations or if-then rules, fuzzyfication and defuzzification tasks are needed to be developed and computed. Furthermore, additional problems concerning judgment-based biases are raised
from the need to elicit the fuzzy structures, which
are more complex that those ones used in BBNs.
Some researchers applied other approaches to deal
with uncertainty due to lack of information. Ferson and Ginzburg(41) proposed using interval analysis. Baraldi and Zio(42) presented a method based on
a combination of Monte Carlo and Possibilistic ap-
Cárdenas et al.
proaches. Baraldi and Zio(43) used Dempster–Shafer
structures.
For the latter authors, issues such as modeling
uncertainty of unknown variables and the evaluation
of Bayesian methods to model uncertainty when new
information becomes available; are subjects the deserve further study.(43) Recent improvements of the
BBNs approach seem to provide options to address
the abovementioned issues which might be favored
by the mechanism of inference employed, its level of
mathematical axiomatization allowing essential tasks
in risk analysis to be performed in a consistent manner and the use of approximated models to represent causal relationships and unknown factors, which
requires fewer estimates and accordingly much less
elicitation effort.
BBNs have been used in the construction industry for various purposes. For instance, Nasir
et al.,(44) Marzouk and Eldieb,(45) Edieb,(46) and Luu
et al.(47) have developed models for construction
schedule and cost overrun risks on a project basis, whereas Bayraktar and Hastak(48) introduced a
decision-support system to study the performance of
highway work projects. In terms of underground construction projects, Bles et al.(20) have applied BNNs
to model construction risks. Surprisingly, given the
importance of using reliable information, little is reported in the noted publications on the provisions
that have to be made to capture expert knowledge
to develop BBNs in such a way that reliability is ensured. In line with this, Bielza et al.(49) observed that
the details of how to elicit knowledge from experts
to construct Bayesian Network models are not well
documented in the literature.
3. MODEL DEVELOPMENT PROCESS
This section describes the successive process
steps that have been followed to develop models
that integrate information on six risks in tunneling
projects. The research process can be divided into
five steps: (1) selection of risks, (2) elicitation of experts’ judgments on the selected risks, (3) discrepancy analysis, (4) developing a BBN-based model for
each risk, and (5) model evaluation.
3.1 Selection of Six Main Underground
Construction Risks
On the basis of the literature studied including
risk databases, reports on failure events, and treatises
on tunnel works, a detailed inventory of risks and
Capturing and Integrating Risk-Related Knowledge
95
events was made that included more than 500 risk
issues related to different stages in tunnel projects.
Evidence reported by the International Association
of Engineering Insurers(1) suggested focusing on critical risks in the construction phase because the IAEI
report shows that most failures occur during the construction stage.
To determine the risks to focus upon, issues related to the construction stage in tunnel works were
classified into three main categories: failures, causes,
and impacts. Typically, issues related to injuries or
loss of life, damage to third parties, additional costs,
delays in completion of the project, or failure to meet
quality requirements were regarded as impacts. The
immediately preceding events that led to the abovementioned impacts were identified as failures. The
events causing and preceding the failures were identified as causes.
A number of exploratory interviews with specialists were conducted to elicit their perceptions of the
most important risks in tunneling works during the
construction phase. Details on the experts consulted
are shown in Table I. During the exploratory interviews, the experts were asked to review and classify the risks included in the developed inventory
into the predefined categories of: failures, causes, and
impacts. The experts were also encouraged to raise
other relevant issues that they regarded as being absent from the inventory. Similarly, some issues were
removed because they were considered as irrelevant.
The round of exploratory interviews failed to achieve
full agreement among the experts on what should be
considered as the main risks. Nevertheless, the following issues do represent the ones that were most
frequently selected by experts as being major risks
during the construction phase:
these risks. Consequently, these major risks were selected as the further focus of the research and model
development.
• Face instability in soft soils when using Slurry
Shields;
• Face instability in soft soils when using Earth
Pressure Balance Shields;
• Excessive volume loss leading to settlement in
tunnels bored in soft soils;
• Excessive deformation, damage, or leakage in
concrete linings;
• Collapse and large deformations of excavated
shafts in soft soils; and
• Collapse and large deformations of excavated
cross passages in soft soils.
The experts’ reasons for selecting the above risks
were their high impact in terms of costs, delays, and
damage; and the particular difficulty in controlling
3.2 Data Requirements and Scope
Usually, risks are characterized merely in terms
of their probability and impacts.(35) In this research,
the risk characterization is extended to include the:
• Relevant causes and conditions (risk factors)
associated with the risks under study;
• Plausible relationships among the risk factors;
• Likelihood of occurrence of the risk factors;
• Strength of the relationships among the risk
factors.
The relevant risk factors are assumed to be those
identified as such by experts and are defined in
terms of deviations from project requirements (assumptions, expectations, specifications, tolerances,
and thresholds) that could lead to the failure being
considered.
Information on the likelihood of the relevant
causes depends on the particular setting of each
project. However, for test purposes, experts were
asked to give their best estimation of the likelihood
of the risk factors occurring. Similarly, because construction risk impacts are very project-specific, information on risk impacts was deliberately excluded
from the characterization of the risks. This means
that information on risk impacts has to be incorporated into the characterizations on a case-by-case
basis.
3.3 Expert Elicitation Process
3.3.1 Identification and Selection of Experts for the
Elicitation Process
In selecting experts, names of professionals were
initially identified from the specialized literature on
tunneling in the Netherlands. In this way, 126 people were identified as professionals involved in ongoing or past underground construction projects such
as tunnel boring and deep shaft excavations. These
professionals were then contacted and also asked
to suggest other reputable specialists. By studying
the topics of the papers written by the 126 identified specialists, the list was shortened to 59 who
were invited to provide information for this research
project. A total of 31 professionals agreed to be interviewed. The participating experts all originated from
96
Cárdenas et al.
Table I. Characteristics of the Experts Involved
Number of experts interviewed
Tunneling experience
0–5 years
6–10 years
11–15 years
>15 years
Number of experts with publications
None
1 publication
2 to 5 publications
More than 5 publications
Current position
Researcher
Design consultant
Construction supervisor
Constructor
Project manager
Tunneling method experience
Slurry shield
Earth pressure balance shield
Cut and cover
Others
Number of projects
Involved in 1–2 projects
Involved in 3–5 projects
Involved in 6–10 projects
Involved in >10 projects
Length of tunnels built
0–5 km
5.1–10 km
10.1–15 km
15.1–20 km
>20 km
Location of last project
Urban settings
River/sea crossing
1st Round of Interviews
2nd Round of Interviews
3rd Round of Interviews
7
11
1
1
4
1
24
Number of experts
3
3
11
7
0
4
2
1
1
22
0
1
0
8
1
2
0
3
1
1
2
2
10
4
3
5
1
2
3
3
2
5
2
3
4
23
7
19
18
11
1
7
7
3
1
1
2
4
10
7
3
4
6
1
0
1
2
1
0
3
8
6
5
2
3
2
4
3
2
0
6
1
22
2
8
3
the Netherlands or Germany. The specialists work or
had worked for organizations such as research institutes, governmental agencies, or companies providing tunnel design, supervision, and construction services. Table I shows the characteristics of the specialists consulted in more detail.
3.3.2 Elicitation Process Design
Experts’ judgment is prone to be highly biased
and this influences the reliability of the data collected. To reduce the likelihood of obtaining biased
data, the elicitation procedure used in this investigation largely followed the criteria provided by Cooke
and Goossens,(24) Renooij,(50) Ayyub,(25) Garthwaite
et al.,(26) Goossens et al.,(22) Kynn,(27) and Hallowell
0
0
9
2
and Gambatese.(28) The specific criteria that were
adopted are referred to where appropriate throughout this section. Fig. 1 is an adaption of the elicitation
processes described by Goossens et al.(22) and Hallowell and Gambatese,(28) and shows the successive
steps that were taken during the elicitation process.
Table II provides an overview of the potential biases
that were considered during the elicitation process
and how these were avoided or at least minimized.
Direct one-to-one interviews were the main instrument used to elicit the experts’ judgments. This
choice was based on the fact that gathering the required input data, which is specified in Section 3.2,
demands detailed interaction with the specialists to
provide the opportunity to discuss possible emerging
divergences in data to be analyzed further.
Capturing and Integrating Risk-Related Knowledge
Objectives:
- identification of top risks in tunnel
works,
- identification and selection of experts.
Literature and documents review
Identification of experts
97
Objectives:
- identification of the relevant risk factors
associated with the top risks,
- elicitation of chance of risk factors.
Literature and documents review
Pilot interviews (semi-structured)
Selection of experts
Pilot interviews (open-ended
interviews)
Objectives:
- elicitation of strength of influence of
relationships amongst risk factors,
- internal validation of relationships.
Pilot interviews (semi-structured)
Second round of interviews (semistructured)
First round of interviews (open-ended
interviews)
Third round of interviews (semistructured)
Develop feedback
on divergences
Develop feedback
on divergences
Evaluation of data,
-discrepancy analysis,
-sensitivity analysis
Model prototype
Fig. 1. Expert elicitation process (based on Goossens et al.(22) and Hallowell and Gambatese).(28)
Even though literature indicates that there is no
significant difference in terms of reliability between
individual interviews and group approaches such as
a workshop sessions(24,51) ; an important reason for
choosing individual interviews is that they help avoid
the obtained data being unreliable due to biases in
judgment caused by phenomena such as “collective
unconsciousness” and “dominance,” which can occur
in group sessions as described in Table II.
The above arguments led to the conclusion that
individual interviews would be an efficient and reliable option. In this work, an interview protocol, de-
scribed later in this section, was designed to address
further considerations reported by Clemen and Winkler,(51) who stated that direct interaction among experts in a group session can be valuable in “ironing
out” differences in definitions and assumptions, clarifying what is to be assessed, and exchanging information before the elicitation process. The interview
protocol designed in this work meets these requirements without the need for group sessions.
Additional provisos were included to ensure that
the information provided by an expert would be usable. Adams(19) commented that engineering experts
98
Cárdenas et al.
Table II. Judgment-Based Bias in Risk Studies and Provisions Made in This Particular Study to Minimize Them
Bias
Definition
Representativenessa,b
Subjects do not base their estimates on
frequency of experienced occurrence data
Collective unconsciousc
Individuals tend to follow a popular trend
Contrast effectc
Perception of a given subject is enhanced or
diminished by the value of the immediately
preceding subject
Individuals underestimate the role of
probability/uncertainty in the subjective
quantification of risk
Neglect of probability/poor
interpretation of uncertainty
(variance)b,c
Overestimation of probabilitya
People overestimate values when the events
they are asked about are linked to high
magnitude impacts
Myside biasc
With controversial issues, individuals tend to
provide arguments from just one perspective,
i.e. ignore contrary evidence
Recency effect (availability)a,b,c
Subjects are more likely to artificially inflate risk
ratings because similar incidents have recently
occurred, and correspondingly underestimate
those that are in the remote past
Subjects generally overestimate a priori
probabilities for events they thought had
occurred and underestimate for events they
thought had not occurred
People expect a sample from a population to
represent all the essential characteristics of a
population
Hindsight biasb
Law of small numbersb
Primacy effectc
Dominancec
Misunderstanding of events
Overconfidencea,b,d
a Ayyub.(25)
b Garthwaite
c Hallowell
et al.(26)
and Gambatese.(28)
d Adams.(19)
An individual is more likely to assign
importance to the issues at the beginning of
an interview than those at the end of it
A very vocal or intimidating group member
controls the ratings of the other members
Vague identification of the variables and events
on which probability estimates are assigned
Subjects tend to provide narrower confidence
intervals compared to real intervals
Provision made to minimize bias
Using diagrams showing relationships with
other variables with unconditional probability
assignments so that the estimates provided
were reviewed and made coherent;
unconditional and conditional probabilities
were elicited and processed in Bayesian
networks
No interaction among specialists, individual
interviews
Randomization of questions during the
interviews
Using intervals of probability, rejection of
outlying data according to discrepancy
analysis and Bayesian aggregation methods of
probabilities
Requesting estimates of probabilities
independently of the impacts, using intervals
of chance, requesting degrees of confidence
on the assessments, rejecting outlying data
according to discrepancy analysis and
Bayesian aggregation methods of
probabilities
Revisiting conflicting issues in a third round of
interviews and retaining the discrepant
evidence until performing discrepancy
analysis at the end of elicitation process
Requesting estimates of probabilities
independently of the impacts, rejection of
outlying data according to discrepancy
analysis and Bayesian aggregation methods
Multiple expert elicitation process, rejection of
outlying data according to discrepancy
analysis and Bayesian aggregation methods of
probabilities
Multiple expert elicitation process, rejection of
outlying data according to discrepancy
analysis and Bayesian aggregation methods of
probabilities
Specialists were given the chance to select the
risks to assess
No interaction among specialists, individual
interviews, equal weighting of responses, data
shared anonymously with other specialists
Review by external professional in construction
and revisiting the specialists over vague issues
Subjects are required to give probability
assignments in terms of probability intervals,
requesting degree of confidence on the
assignments
Capturing and Integrating Risk-Related Knowledge
99
reach the height of their expertise after about ten
years of relevant experience. Accordingly, information used in this study is only drawn from experts
with a minimum of ten years of tunneling experience.
Further, the minimum number of experts was considered. In this research, probabilistic input information used to characterize each risk studied had to be
provided by at least three experts. The use of this
small number of experts is based on the work described by Clemen and Winkler(51) and Adams.(19)
Clemen and Winkler(51) indicated that with three to
five experts much of the total knowledge is usually
achieved. Nevertheless, in this work the size of an expert group was increased if and when significant discrepancies amongst the data became apparent during
the elicitation process.
As shown in Fig. 1, the elicitation process was
carried out in three rounds of interviews. First round
of interviews were the initial exploratory interviews
described in Section 3.1. The second round of interviews was carried out to gather data on potential
causes of the selected tunneling risks. At the beginning of this round of interviews, the experts were presented with a preprepared list of potential risk factors
as derived from the literature study and were asked
to add possible additional factors to this list and to
suggest plausible relationships. After collecting information from various experts, the inventory was
translated into node-and-arc diagrams showing the
risk factors and their relationships. A node-and-arc
diagram was constructed for each of the risks studied. Once these had been drawn up, the diagrams
were used during all further elicitation sessions. In
this way, information anonymously provided by one
expert was successively reviewed by other experts
later in the process and iteratively refined to ensure
that dependencies and directions in the relationships
arcs reflected the experts indications.
As a precaution to ensure comprehensiveness in
the data, the experts were explicitly asked to give
their judgments based on the real situation found
in practice and not on ideal situations. Goossens
et al.(22) have pointed out that, if this proviso is ignored, experts might implicitly assume that the works
to be considered are those designed and built in satisfactory but exceptional conditions, and hence they
might disregard a large proportion of the relevant
risk factors.
Experts were finally asked to give their estimates
on the likelihood of the risk factors occurring regardless of the extent of their possible impacts. This procedure was adopted to reduce the effect of proba-
bility overestimation that can occur when individuals are asked about events linked to high magnitude
impacts. This is a recognized judgmental bias, known
as the “contrast effect,” and is described in Table II.
Special effort was also made to establish unambiguous identifications of the risk factors. As such, the
risk factors were described using a format to ensure
clear interpretation and hence reliable judgments.
The descriptions of the factors were also revisited if
there were wide discrepancies among the probability
estimates provided by the different experts.
The second round of interviews was ended once
three successive sessions failed to add any additional
information on the risk factors obtained. This “data
saturation point” was initially identified in the 21st
interview and confirmed in later elicitation sessions.
Thus, in total, 24 experts participated in the second
round of this study. On average, each specialist provided information on two failure types related to
their own fields of expertise. There were some instances where some experts provided only partial information on the potential failures under analysis,
arguing they were not able to provide the required
information. On average, each failure type was analyzed by five experts. Each failure is described using
more than forty factors. An example of the information obtained is presented in Section 4.
Following the second round of interviews, a third
and final round of interviews was carried out to collect information on the strength of the influences of
the earlier identified relationships. This round was
also used to internally validate the collected data
with the support of the experts involved. Because the
objective of this round was to measure the strength of
the influence of each of the identified relationships, a
validation of the structure as represented in the diagrams was performed indirectly. The individual estimates of the strengths of the influences of the relationships were aggregated mathematically into probabilistic distributions.
3.3.3 Interview Protocol
The interviews were guided by a protocol using
the elicitation criteria referred to above. Each elicitation session started with an explanation of the interview structure. Following this, each interview was
conducted in three phases.
The first phase was intended to collect information about the specialist’s background
by completing a specifically designed form. The
expert’s background data were collected to assess
100
the professional’s ability to provide information
relevant to the study and to decide how to use the
information provided. In this phase, the interviewees were further informed about the confidential
treatment of their personal details and that the data
provided would only be shared anonymously with
other experts.
The second phase of the elicitation sessions was
primarily intended to familiarize the experts with the
major research issues so that they were able to develop a common understanding.
The core part of the interviews, the third phase,
concerned the elicitation of risk information. Initially
in this third phase, the experts were presented with
the set of major risks and given the option to choose
which ones to assess during the interview. This particular action was intended to obtain data that was
closely related to the particular expert’s experience
and thus avoid the “primacy effect” judgment bias as
described in Table II.
3.4 Discrepancy Analysis
Once information describing the risks was collected, a discrepancy analysis was performed. Discrepancy analysis aims to identify data pieces where
the experts’ assessments differ most. These data
should be reviewed to ascertain any avoidable discrepancy causes(24) or so that one can adopt values with associated confidence bounds.(25) Discrepancy analysis is intended to eliminate outlying data
originating from potential judgment biases, as described in Table II. Discrepancy analysis was performed throughout the entire elicitation process although a definitive assessment was made at the very
end of the process to avoid any unjustified elimination of information. This approach is intended to reduce the effect of another judgmental bias known as
“myside bias,” which in this particular study would
be associated with the arbitrary neglect of contrary
evidence. Discrepant information can arise in the description of risk factors, in the existence and direction
of relationships among the factors, in the probability assignments, and in the estimates of conditional
probabilities.
In performing the discrepancy analysis, two assessments were made following the criteria described
by Woodberry et al.(52) These assessments consisted
of comparing values across different experts and reexamining those instances where experts indicated
low confidence. Experts were encouraged to indicate
their confidence in their estimates using a numerical
Cárdenas et al.
scale. Values of variables with a low degree of confidence were flagged for later attention.
Further checks were undertaken. Discrepant
data are also a sign that there may be epistemic uncertainties that allow expert views to differ due to incomplete evidence.(29) The occurrence of this particular form of discrepant data is observed when two or
more significant and equivalently sized sets of estimates point toward conflicting assessments. To illustrate this, let us assume that two experts, A and B,
are asked to provide estimates of the probability of a
given event. There is a strong disagreement between
the two experts: one expert confidently estimates that
the event has a 1 in 200 chance of occurring; the other
expert confidently estimates the probability as much
higher at 1 in 50. This is identified as an epistemic
uncertainty if the disagreement is maintained with a
larger number of experts.
After the discrepancy analysis was performed,
divergent data were revisited, sometimes modified by
their providers, or disregarded when either a high
discrepancy remained. In case of emerging epistemic
uncertainties, these were further analyzed to observe
their impact in the model performance. Note that discrepancy analysis was only performed to trace divergent information in the likelihood and conditional
probability assessments.
3.5 The BBN Risk Models
A Bayesian Belief Network, or BBN, essentially
provides a framework for representing the causal relationships between variables and capturing the uncertainties in the dependencies between these variables using conditional probabilities.(53) In a BBN
interrelationships between variables are expressed
graphically. Variables that have interdependencies
are connected, whereas independent variables are
not. The direction of the arrows reflects the direction of causal influence, as perceived by the expert
or confirmed experimentally. A probability distribution for each variable and the relationship in the network is assigned. Figs. 2 and 4 provide an example
of Bayesian Belief Networks. Some relevant details
about the construction of the developed risk models
using the BBN framework are described in this section.
As discussed earlier, some variables in the risk
models are not known with certainty. To reflect this
uncertainty, the variables are represented using “evidence probability distributions.” An evidence distribution is a representation that provides information
Capturing and Integrating Risk-Related Knowledge
101
Bentonite leakage into the ground
Excessive loss of support
Frequent
8%
Frequent
25%
Probable
8%
Probable
38%
32%
Occasional
42%
Occasional
Remote
33%
Remote
4%
Improbable
9%
Improbable
1%
Absent
0%
Absent
0%
Excessive loss of air pressure
Fig. 2. Example of distributions
representing evidence supporting the
variables in the models.
Frequent
0%
Probable
0%
Occasional
0%
Remote
0%
Improbable
0%
Absent
100%
Underesmaon support pressure
Label
Frequent
Probable
0%
Occasional
0%
Remote
0%
Improbable 0%
Absent
Chance
Descripon
0%
100%
on values of the variables and available supporting
evidence based on assessments by experts. In this
study, the variables are represented so as to reflect
the evidence that supports the values attached to
the variables. This is achieved by using a graphical representation of the uncertainty associated with
each variable to provide an instant overview of the
strength of the evidence supporting each model. Information on the available evidence is regarded as
of paramount importance to the end-users in this
study. Fig. 2 shows a set of variables and their evidence probability distributions. This particular example shows, for the variable “Excessive loss of support,” that the “Probable” likelihood category has
more supportive evidence than the other states of
this variable.
Each variable in the six developed risk models is
an event or condition that represents a fault event,
state of failure, or an unfavorable condition. Fault
events and states of failure of a variable can be events
in which a risk factor exceeds a predefined threshold. In this research only discrete variables are used
and, initially, most of the variables were allowed two
possible states: absent or present. A variable is regarded as being “Absent” when it is not active based
on the particular conditions included in the analysis. As an example, in the situation under analysis in
Frequent
>30%
Probable
10-30%
To occur several mes during construcon of tunnels
Occasional
5-10%
To occur once during construcon of a tunnel
Remote
1-5%
To occur from project to project
Improbable
0-1%
Extremely rare to occur
To occur repeatedly during construcon of tunnels
Fig. 2, only one variable is being taken into account
whereas two are absent. The “Present” state is further discretized into five levels of probability. Some
variables can have more than two possible states, but
few of the variables in the models needed to be expressed in such a way.
To obtain trial information for the risk models, experts were requested to provide the chance
of the risk factors in terms of qualitative probability estimates using a five-point scale. Terés
Flores,(54) Visée,(55) and Bielza et al.(49) reported
positive results using this approach and emphasized that most people find it easier to express probabilities qualitatively rather than quantitatively. This was supported by the experts that
were consulted in this study. Using qualitative
probabilities is also beneficial in reducing bias in
data, as shown in Table II.
In accordance with Dı́ez and Druzdzel,(56) specific questions were asked to each expert to obtain
conditional probability estimates for each kind of relationship. Conditional probabilities on exclusive relationships were obtained by addressing the question What is the chance that a cause Xi (conditioning
variable) results in an event Y (conditioned variable)
when no other cause is present? For the inclusive interactions, the question addressed was What is the
102
chance that Xi (conditioning variable) lowers Y (conditioned variable) when no other factor has lowered
it? Inclusive interactions are those in which, given a
number of known causes of a variable occurring, a
change in the conditional variable is necessarily due
to a combination of changes in every cause occurring at the same time. In inclusive interactions, the
conditioning variables might be mutually supportive.
Conversely, in exclusive interactions, the causes are
each assumed to be able to produce a change in the
conditional variable in the absence of other causes,
and their ability to cause a change in the conditional
variable is assumed to be independent of the presence of other causes.(57) The explanations as to why
these questions were worded in such a way are described in Onisko et al.(57) and Dı́ez and Druzdzel.(56)
The experts were requested to provide estimates of
the conditional probabilities in terms of qualitative
probabilities using the same scale of five categories
deployed for measuring the chance of variables described earlier.
3.6 Risk Models Evaluation
To obtain the desired reliability in the models
that integrate knowledge of the risks under study, a
set of provisos was made. Probability estimates and
conditional probabilities were used to define the occurrence of the variables and their relationships in
the models. As discussed earlier, this input information was subjected to a discrepancy analysis. Discrepancy analysis provided information on which information pieces were appropriate to incorporate in the
models and which should be rejected or further analyzed to assess the effect of epistemic uncertainties.
In addition, a review of the models’ structures
was carried out by various experts during the elicitation sessions. By using the networks that depicted the
risks under consideration each expert had the chance
to successively review the relationships among the
variables in the models and estimate the strength of
the influence of these relationships. This intensive review of the relationships can be seen as providing internal model validation. In a few instances, the experts had diverging views on the relationships, and
the impact of these divergences on model performance was investigated using different relationship
variants.
The research steps thus ended with the evaluation of each risk network model representing each of
the six identified major tunnel risks.
Cárdenas et al.
4. EXAMPLE: TUNNEL FACE INSTABILITY
RISK MODEL
This section provides details of the developed
tunnel face instability risk model for slurry shields in
soft soils.
When boring a tunnel using tunnel boring machines (TBMs), the soil is removed by a cutting
head and transported back from the excavation face.
To temporarily support the excavated ground, some
types of TBMs have pressurized compartments at
their front end to balance the ground and water pressure.(58) Face instability in bored tunnels occurs when
the ground and water pressure, in conjunction with
the shear strength available in the ground, are not in
equilibrium with the pressure provided by the boring
machine. This situation can potentially lead to a face
collapse.(59)
The integrated knowledge on the tunnel face
instability risk covers issues related to the Dutch
ground conditions and construction practices. Soft
soils with low stiffness and a high groundwater table
are the dominant ground conditions in the Netherlands, and, particularly, saturated medium-fine sandy
soils are the usual ground conditions encountered
when tunneling.(60,61) Tunnel boring is typically carried out in the Netherlands using closed shields such
as slurry and earth pressure balance shields. These
closed shields provide continuous support to the face
during excavation. Slurry shields use bentonite suspensions as a supporting fluid. Earth pressure shields
use the excavated soil as support, often conditioned
with bentonite slurry and other additives.(62)
Fig. 3 displays the components of the tunnel face
instability risk model (variable states are not shown
for reasons of clarity). Three major event-scenarios
(the ovals in Fig. 3) can lead to face instability when
using slurry shields: excessive loss of support, excessive support pressure, and loss of air pressure. These
three scenarios share common causes that can be
clustered into components such as faults in the excavation process, in design, or in monitoring, and mistakes by on-site operators, and are further affected
by variables related to the ground conditions (the
boxes in Fig. 3).
Fig. 4 shows a fragment of the model that represents those variables associated with the eventscenario “excessive loss of support.” The submodel
consists of subsets of variables that can be classified into causes, conditions, and inhibitors. Causes
are those variables that can lead to the failure occurring. Conditions are the variables necessary to
Capturing and Integrating Risk-Related Knowledge
103
Excessive loss of
air pressure
Excessive loss of
support
Ground
condions
Face instability
Excessive support
pressure
Construcon
process
Design
Monitoring
Machine
operator acons
Response
acons
Fig. 3. Face instability risk: model prototype.
Excessive pore
pressure
Unexpected
variability of soil
permeability
Causes
Incorrect properes
of bentonite
Ground
condions
Incorrect TBM
penetraon rate
Deficient control
bentonite
properes
Incomplete ground
model
Underesmaon of
support pressure
Open wells/pipes
going to surface
Bentonite leakage
into ground
Insufficient
penetraon of
bentonite
Excessive loss of
support
Face instability
Local face instability
Excessive loss air
pressure
(maintenance)
Insufficient air
supply
Insufficient
bentonite pressure
Miscommunicaon
operator monitoring
Reacon to loss of
support event
Incomplete
countermeasures
Machine operator
mistake
Fig. 4. Excessive loss of support risk submodel.
Inhibitors
104
produce the failure but that depend on the causes in
the model to effectively drive the occurrence of failure. Inhibitors are those variables with the capability of reducing the chance of failure. An example inhibitor variable in the model is “reaction to excessive
loss of support.” If an excessive loss of support occurs, the reaction determines whether the event can
result in face instability. Ground conditions typically
function as conditioning variables. For instance, to
create an excessive loss of support event, “open wells
or pipes extending to the surface” is a necessary condition for rising a bentonite leak into a ground event.
The fragment of the face instability risk model
shown in Fig. 4 takes into account other factors than
those currently used in analytical methods to estimate tunnel-face instability. In these other methods, tunnel-face stability is estimated from factors
such as overburden stresses and excess pore pressure
in the ground, effective weight, strength, soil particle size, yield strength of the bentonite. As such,
the presented model extends information on the
risk of face instability by involving other relevant
variables.
The model was populated with the estimates of
experts after performing the discrepancy analysis described earlier. As a result of the final discrepancy
assessment, three pieces of evidence out of 136 estimates of conditional probabilities measuring the
strength of the interactions among factors and 13 of
the initial set of the (170) unconditional probability
estimates associated with the likelihood of the factors occurring were eliminated. With this refinement
in the data, most of the unconditional evidence probability distributions (25) and the conditional probabilities distributions (16) are spread over two to three
categories of chance.
To illustrate the functionality of the developed
model, experimental data on the chance of risk factors is incorporated and propagated into the model.
Unconditional distributions representing the likelihood of the risk factors occurring are attached to
each variable in the model. The risk factors associated with ground conditions were modeled using uniform distributions. The model propagates this information and particular analysis can be performed, for
instance, the identification of the most sensitive factors given certain conditions in a project. Details of
this analysis will be presented in a separate paper.
In this article an example of the analysis of a fragment of the developed model is described. The fragment under analysis corresponds to the set of risk
factors directly associated with the event “excessive
Cárdenas et al.
loss of support” and is shown in Fig. 4. According to
the sensitivity analysis carried out using experimental
unconditional probabilities, it appears that the event
“Underestimation of support pressure” is more critical to the occurrence of an excessive loss of support
than the other influencing risk factors. The results
of this analysis are represented by means of a tornado graph as shown in Fig. 4. The numbers indicate
the relative importance of the risk factors based on
the estimated values of the sensitivity indicators. This
provides an indication of how resources can be apportioned to control the occurrence of excessive loss
of support event. Such analysis can be repeated for
any set of variables in the model.
5. CONTRIBUTIONS, LIMITATIONS, AND
CONCLUSIONS
The main objective of this study has been to develop and to demonstrate the application of an approach for collecting and integrating risk-related information. Six risk models were developed using this
approach. In the remainder of this section, a number of important contributions and limitations of this
study are discussed and the section ends by drawing
some conclusions.
5.1 Contributions
A recent review by Bielza et al.(49) draws attention to the fact that details on how to elicit
expert judgments to construct Bayesian Network
models have not been well documented. Renooij(50)
reported that building Bayesian Belief Networks
usually entails an elicitation of a large number of
probabilities which is often referred to as a major obstacle to further demonstrate the feasibility
of modeling complex real-life problem domains.
This problem has been ameliorated with the
development, for instance, of approximations of
Conditional Probability Tables such as Noisy-MAX
and Noisy-AND gates which require fewer estimates
to model causal relationships. A comprehensive
framework on the use of this approximations were
provided by Dı́ez and Druzdzel.(56) This particular
development and others have encouraged expert
judgment to be encoded to develop Bayesian Belief
Network models. Accordingly and as described by
Renooij,(50) it is important to consider that issues
related to human capabilities with respect to making
judgments come into play when relying on experts
for probability elicitation. Extensive psychological
Capturing and Integrating Risk-Related Knowledge
105
research has shown that people, including experts,
can deliver biased probability assessments. Decision
analysis literature have provided a number of methods for the elicitation of judgment. These methods
needed to be particularly customized for constructing Belief Networks to enable epistemic uncertainty
to be captured while providing models building efficiency. An important first contribution of this study
is therefore the detailed description and justification
of the successive interactive steps needed to elicit
expert knowledge for constructing risk models based
on Bayesian Networks. To this end, criteria for
identifying and selecting experts, the risk elicitation
process itself, and data evaluation were reviewed
and integrated. Further, a number of judgmental
biases were identified as factors that potentially
affect the development process of Bayesian Network
risk models. To minimize the effect of these biases,
a set of provisos were specified. These provisos include issues concerning the conditions of elicitation
sessions, the application of discrepancy analysis, the
operationalization of variables in the risk models,
the process of review and feedback to experts, and
the form of the questions addressed to the experts.
Traditional elicitation procedures strive to reach
consensus among participants, and hence eliminating
information that might be associated with epistemic
uncertainties which reflects a judgmental bias.(26,28)
Unlike these traditional procedures, the proposed
approach uses criteria that avoid epistemic uncertainties being arbitrarily eliminated. The assessment
of the effect of epistemic uncertainties is important
because it provides information on the uncertainties involved in risk management decisions.(29,63−66)
In line with the above, the concept of an evidence
probability distribution is used in this study to reflect
the epistemic uncertainties in variables and relationships. Information on variables and relationships in
the models was probabilistically “weighted” in such
evidence probability distributions and then propagated into the Bayesian Networks.
A second contribution of this research concerns the use of Bayesian Networks, in conjunction
with the proposed approach, to capture risk-related
knowledge. This combination is beneficial in avoiding or mitigating undesirable biases that are known
to be common problems in expert judgment elicitation processes, as noted by Paté-Cornell,(29) Tah and
Carr,(14) Garthwaite et al.,(26) Dikmen et al.,(17) and
Taroun et al.(35)
To summarize, in this process, the following benefits were achieved:
• Evidence probability distributions: Existing
methods to aggregate or average the judgments of multiple experts are questioned as
they can lead to misrepresentation of uncertainty in models. The proposed evidence
probability distributions, updated with the
Bayesian scheme, however provide an alternative that does not truncate or mask divergent
evidence.
• Probability intervals: Arriving at probability
distributions is often a difficult task. The way
variables are represented here, using probability distributions derived from probability intervals, leads to a reduction in biases in the probability assessment and to a rapid elicitation process.
• Bayesian Networks for multivariate elicitation:
To elicit a joint probability distribution for
two or more quantities, experts are required to
somehow provide joint probability estimates,
a process which is difficult and unreliable.
Bayesian Networks enable risks to be broken
down into simple tasks, allowing probability
assessments and joint probability distributions
to be obtained without “mental acrobatics.”
• Risk diagrams: In any elicitation process, the
raw data provided by experts are commonly
expected to be inconsistent with the laws of
probability. Bayesian Networks assist in checking and adjusting internal consistency. Similarly, the use of diagrams representing risks
proved to be very beneficial in the elicitation
process. Each expert could trace paths for different chains of factors allowing them to verify their quantitative judgments by comparing
their probability estimate with others in the diagram and to add further information such as
previously disregarded risk factors.
Another contribution of this study concerns the
treatment of epistemic uncertainties and the time
involved in developing risk models. If the developed approach is compared to existing methods
such as the Mixture approach,(67) Bayesian updating,(68) the NUREG-1150 approach,(69) the PACUA
approach,(70) the Technical Facilitator-Integrator approach,(71) Risk Diagnosis Methodology(72) ; the developed approach uses Bayesian Networks to assemble evidence which has the effect of reducing the
elicitation effort and avoiding epistemic uncertainty
being misrepresented. In the developed approach, a
106
limited set of questions is addressed by experts leading to a less time-consuming elicitation process.
5.2 Limitations
This study has concentrated on the development of models for six tunneling risks through eliciting risk-related knowledge from experts. Future research could extend the findings from this research
by addressing its current limitations.
A first limitation concerns the incompleteness of
the developed tunnel risk models. Although the developed models do provide comprehensive information on risk factor interactions including those rarely
measured during tunnel construction, these models
remain inherently incomplete because they are unlikely to fully encompass the range of possible risk
factors. Such incompleteness might be due to the experts having a limited understanding of the risks, or
to undetected flaws in the elicitation procedure. As
new information becomes available from further research, the models could be updated to reflect this
enhanced information.
A second reservation concerns the added value
of the models when used in practice. The models will
have to be evaluated on their performance in providing useful and consistent information for deriving
risk measures while affording an assessment of the
effect of discrepant data caused by epistemic uncertainties. In addition, one needs to assess the model
performance when faced with different degrees of
confidence in the information pieces provided by the
experts. In these models, the same degree of confidence was assigned to each piece of evidence.
The integrated knowledge used to characterize
tunnel risks included issues limited to soft soils similar to Dutch ground conditions. Dutch ground conditions are characterized by saturated, low stiffness
sandy soils with medium-fine size particles and a high
groundwater table. The developed tunnel risk models refer only to tunnel boring with closed shields,
such as slurry or earth pressure balance shields, and
using concrete linings as the definitive support. Two
of the developed models contain issues related to excavating shafts and cross passages, but these are also
limited to the same ground conditions. Future studies could extend the research focus to include other
ground conditions and also develop additional models for other tunnel works risks.
A final remark on the limitations of this study
concerns the question as to what extent the approach
advocated in this article could be applicable for other
Cárdenas et al.
types of civil engineering projects and indeed other
projects in general. We believe the methodology outlined in this article has the potential to assist in providing clear information on relevant scenarios that
could lead to project failure elsewhere, but this remains to be proven.
5.3 Conclusions
The aims of this study were twofold. First, the
article describes the development of an approach to
collect and integrate risk-related information that resulted in six models for major tunneling risks. The
feasibility, in terms of cost-effectiveness and reliability, of developing such models for construction risks
has been shown. The second aim was to demonstrate
the applicability of the proposed approach. In this
article, the approach was successfully applied to the
risk of “face instability in soft soils when using slurry
shields.”
ACKNOWLEDGMENTS
The authors are very grateful to the anonymous
reviewers who provided some valuable and useful
suggestions. This research has been supported by
DELTARES, whose financial support is gratefully
acknowledged. The authors also thank the experts
who agreed to be involved in this research, without
whose insights and support the work could not have
been achieved.
REFERENCES
1. Landrin H, Blückert C, Perrin JP, Stacey S, Stolfa A.
ALOP/DSU coverage for tunnelling risks? Boston: International Association of Engineering Insurers, 2006. 19 p. Report
No.: IMIA WGP 48 (06).
2. Muir Wood AM. Tunnel hazards: UK experience. Pp. 149–164
in Davis JS (ed.) Conference of Hazards in Tunnelling and on
Falsework. London: Institution of Civil Engineers, 1975.
3. Muir Wood AM. Tunnelling: Management by Design. London
and New York: Taylor & Francis Group, 2000. 256 p.
4. Sowers GF. Human factors in civil and geotechnical engineering failures. J Geotech Eng, 1993; 119(2):238–256.
5. Bea R. Reliability and human factors in geotechnical engineering. J Geotech and Geoenvir Engrg, 2006; 132(5):631–642.
6. van Tol AF. Schadegevallen bij bouwputten. [Damage caused
by Deep Excavations]. Cement, 2007; 59(6):6–13. Dutch.
7. Wearne S. Organisational lessons from failures. Proc Instit
Civil Eng-Civil Engrg, 2008; 161(6):4–7.
8. Mitchell JK. Geotechnical surprises-or are they? The 2004 H.
Bolton Seed lecture. J Geotech and Geoenvir Engrg, 2009;
135(8):998–1010.
9. Bravery P, Cross S, Gallagher R, Hautefeuille O, Reiner H,
Spencer M, Smith P, Stolfa A, Wannick H. A Code of Practice
for Risk Management of Tunnel Works. Munich: The International Tunnelling Insurance Group, 2006. 28 p.
Capturing and Integrating Risk-Related Knowledge
10. Eskesen SD, Tengborg P, Kampmann J, Holst Veicherts,
T. Guidelines for tunneling risk management: International Tunneling Association, Working Group No. 2.
Tunnelling Underground Space Technol, 2004; 19:217–
237.
11. European Parliament and the Council of the European Union.
Directive 2004/54/EC of the European parliament and of the
council of 29 April 2004 on minimum safety requirements for
tunnels in the Trans-European Road Network. Official J Eur
Union, 2004; L 167:39–91.
12. Meng Q, Qu X, Wang X, Yuanita V, Wong SC. Quantitative
risk assessment modeling for nonhomogeneous urban road
tunnels. Risk Analysis, 2011; 31(3):382–403.
13. Qu X, Meng Q, Yuanita V, Wong YH. Design and implementation of a quantitative risk assessment software tool for
Singapore road tunnels. Expert Syst Appl, 2011; 38:13827–
13834.
14. Tah JHM, Carr V. Knowledge-based approach to construction project risk management. J Comput Civil Eng. 2001;
15(3):170–177.
15. De Zoysa S, Russell AD. Structuring of risk information to
assist in knowledge based identification of the life cycle risks
of civil engineering projects. Pp. 1–10 in Proceedings of the
5th Construction Specialty Conference of the Canadian Society for Civil Engineering. Moncton, Nouveau-Brunswick,
Canada: Canadian Society for Civil Engineering, 2003. Report
No.: COF-470.
16. Choi HH, Mahadevan S. Construction project risk assessment using existing database and project-specific information.
J Constr Eng Manage, 2008; 134(11):894–903.
17. Dikmen I, Birgonul MT, Anac C, Tah JHM, Aouad G. Learning from risks: A tool for post-project risk assessment. Automat Constr, 2008; 18(1):42–50.
18. Tserng HP, Yin SYL, Dzeng RJ, Wou B, Tsai MD, Chen WY.
A study of ontology-based risk management framework of
construction projects through project life cycle. Automat Constr, 2009; 18(7): 994–1008.
19. Adams FK. Expert elicitation and Bayesian analysis of construction contract risks: An investigation. Constr Manage
Econom, 2006; 24(1):81–96.
20. Bles T, Al-Jibouri S, van de Adel J. A risk model for pile foundations. Pp. 421–426 in Maas G, van Gassel F (eds.). Proceedings of the 20th International Symposium on Automation and
Robotics in Construction ISARC2003 The Future Site. Eindhoven: IAARC, 2003.
21. Choi HH, Cho HN, Seo JW. Risk assessment methodology
for underground construction projects. J Constr Eng Manage,
2004; 130(2):258–272.
22. Goossens LHJ, Cooke RM, Hale AR, Rodic-Wiersma LJ. Fifteen years of expert judgement at TUDelft. Safety Sci, 2008;
46(2):234–244.
23. Anandarajan A, Kleinman G, Palmon D. Novice and expert
judgment in the presence of going concern uncertainty. The
influence of heuristic biases and other relevant factors. Managerial Audit J, 2008; 23(4):345–366.
24. Cooke RM, Goossens LHJ. Procedures guide for structural
expert judgement in accident consequence modelling. Radiat
Prot Dosimetry. 2000; 90(3):303–309.
25. Ayyub BM. Elicitation of Expert Opinions for Uncertainty
and Risks. 1st. ed. Boca Raton, London, New York, Washington, DC: CRC Press; 2001. 328 p.
26. Garthwaite PH, Kadane JB, O’Hagan A. Statistical methods
for eliciting probability distributions. J Am Stat Assoc, 2005;
100(470):680–700.
27. Kynn M. The ‘heuristics and biases’ bias in expert elicitation.
J R Stat Soc A 2008; 171(1):239–264.
28. Hallowell MR, Gambatese JA. Qualitative research: Application of the Delphi method to CEM research. J Constr Eng
Manage. 2010; 136(1):99–107.
107
29. Pate-Cornell ME. Uncertainties in risk analysis: six levels of
treatment. Reliability Engrg Syst Safety, 1996; 54(2–3):95–
111.
30. Zio E. Reliability engineering: Old problems and new challenges. Reliability Engrg Syst Safety, 2009;94(2):125–141.
31. Bedford T, Quigley J, Walls L. Expert elicitation for reliable
system design. Stat Sci, 2006; 21(4):428–450.
32. Sigurdsson JH, Walls LA, Quigley JL. Bayesian belief nets for
managing expert judgement and modelling reliability. Qual
Reliab Engrg Intl, 2001; 17:181–190.
33. Weber P, Medina-Oliva G, Simon C, Iung B. Overview on
Bayesian networks applications for dependability, risk analysis and maintenance areas. Engrg Appl Artif Intell, 2010.
DOI:10.1016/j.engappai.2010.06.002.
34. Daniels DC, Hudson LD, Mahoney SM, Ware BS, Wright EJ.
Terrorism risk management. Pp. 428 in Puorret O, Naim P,
Marcot B (eds). Bayesian Networks: A Practical Guide to Applications. 1st ed. Padstow: John Wiley & Sons Ltd, 2008.
35. Taroun A, Yang JB, Lowe D. Construction risk modelling and
assessment: Insights from a literature review. The Built Human Environ Rev, 2011; 4(1):87–97.
36. Simon Ch, Weber Ph, Levrat E. Bayesian networks and evidence theory to model complex systems reliability. J Comput,
2007; 2(1):33–43.
37. Dubois D. The role of fuzzy sets in decision sciences: Old techniques and new directions. Fuzzy Sets Syst, 2011; 184:3–28.
38. Novák V. Reasoning about mathematical fuzzy logic and its
future. Fuzzy Sets Syst, 2012; 192:25–44.
39. Mamdani EH, Assilian S. An experiment in linguistic synthesis with a fuzzy logic controller. Intl J Man-Machine Stud,
1975; 7(1):1–13.
40. Sugeno M. Industrial Applications of Fuzzy Control. New
York: Elsevier Science Inc., 1985. 269 p.
41. Ferson S, Ginzburg L. Different methods are needed to propagate ignorance and variability. Reliability Engrg Syst Safety,
1996; 54:133–144.
42. Baraldi P, Zio E. A combined Monte Carlo and possibilistic
approach to uncertainty propagation in event tree analysis.
Risk Analysis, 2008; 28(5):1309–1325.
43. Baraldi P, Zio E. A comparison between probabilistic and
Dempster-Shafertheory approaches to model uncertainty
analysis in the performance assessment of radioactive waste
repositories. Risk Analysis, 2010; 30(7):1139–1155.
44. Nasir D, McCabe B, Hartono L. Evaluating risk in
construction-schedule model (ERIC-S): Construction schedule risk model. J Constr Eng Manage. 2003; 129(5):518–527.
45. Marzouk M, Eldieb A. Risk assessment in pipeline projects using belief networks and computer simulation. Pp. 3384 in Proceedings of 33rd Annual Conference of the Canadian Society
for Civil Engineering 2005: Gateway To Excellence. Toronto,
Ontario: Curran Associates Inc., 2007.
46. Edieb A. DSS-PL: Decision support system for risk assessment of pipeline projects. Cost Engrg, 2007; 49(11):20–29.
47. Luu VT, Kim SY, Van Tuan N, Ogunlana O. Quantifying
schedule risk in construction projects using Bayesian belief
networks. Int J Project Manage, 2009; 27(1):39–50.
48. Bayraktar ME, Hastak M. A decision support system for selecting the optimal contracting strategy in highway work zone
projects. Automat Constr, 2009; 18(6):834–843.
49. Bielza C, Gómez M, Shenoy PP. Modeling challenges with influence diagrams: Constructing probability and utility models.
Decision Support Syst, 2010; 49(4):354–364.
50. Renooij S. Probability elicitation for belief networks: Issues to
consider. Knowledge Engrg Rev, 2001; 16(3):255–269.
51. Clemen RT, Winkler RL. Combining probability distributions
from experts in risk analysis. Risk Analysis, 1999; 19(2):187–
203.
52. Woodberry O, Nicholson AN, Korb KB, Pollino C. Parameterising Bayesian networks. Pp. 711–745 in Webb G, Yu X
108
53.
54.
55.
56.
57.
58.
59.
60.
61.
62.
(eds). Proceedings of 17th Australian Joint Conference on Artificial Intelligence, 2004, Cairns (Australia). Berlin, Heidelberg: Springer-Verlag, 2005.
van der Gaag LC. Bayesian belief networks: Odds and ends.
Comput J, 1996; 39(2):97–113.
Terés Flores JR. Risk management in Canadian Tunnelling
Projects [Master’s thesis]. [Edmonton]: University of Alberta,
2006. 158 p.
Visée JV. An adequate method to quantify probabilities:
Case studies in the context of early reliability prediction using Bayesian networks restricted to subjective data [master’s
thesis]. [Eindhoven]: Techische Universiteit Eindhoven, 2009.
65 p.
Dı́ez FJ, Druzdzel MJ. Canonical Probabilistic Models for
Knowledge Engineering. Version 0.9 (April 28, 2007), 2007.
59 p. Report No.: CISIAD-06-01.
Onisko A, Druzdzel MJ, Wasyluk H. Learning Bayesian network parameters from small DataSets: Application of noisyOR gates. Intl J Approx Reason, 2001; 27(2):165–182.
Anagnostou G, Kovári K. Face stability in slurry and EPB
shield tunneling. Pp. 453–458 in Mair RJ, Taylor RN (eds).
Proceedings of the International Symposium on Geotechnical
Aspects of Underground Construction in Soft Ground. London, UK, Rotterdam: Balkema, Apr 15–17, 1996.
Guglielmetti V, Grasso P, Mahtab A, Xu S. Mechanized Tunnelling in Urban Areas—Design Methodology and Construction Control. 1st ed. Leiden, The Netherlands: Taylor and
Francis/Balkema, 2008. 507 p.
Bakker KJ, Bezuijen A. Ten years of bored tunnels in the
Netherlands: Part I: Geotechnical issues. Pp. 243–248 in Ng
CWW, Huang HW, Liu GB (eds). Proceedings of the 6th International Symposium on Geotechnical Aspects of Underground Construction in Soft Ground (IS-Shanghai 2008); 2008
Apr 10–12; Shanghai (China). Leiden, The Netherlands: CRC
Press/Balkema, 2009.
Bezuijen A, Talmon AM. Processes around a TBM. Pp. 3–14
in Ng CWW, Huang HW, Liu GB (eds). Proceedings of the
6th International Symposium on Geotechnical Aspects of Underground Construction in Soft Ground (IS-Shanghai 2008);
2008 Apr 10–12; Shanghai (China). Leiden, The Netherlands:
CRC Press/Balkema, 2009.
Broere, W. Influence of excess pore pressures on the stability
of the tunnel face. Pp. 759–765 in Saveur J (ed). (Re)Claiming
Cárdenas et al.
63.
64.
65.
66.
67.
68.
69.
70.
71.
72.
the underground space, Amsterdam (The Netherlands). Lisse,
The Netherlands: Balkema, 2003.
Aven T. Perspectives on risk in a decision-making context—
review and discussion. Safety Sci, 2009; 47(10): 1355–
1364.
Abt E, Rodricks JV, Levy JI, Zeise L, Burke TA. Science
and decisions: Advancing risk assessment. Risk Analysis, 2010;
30(7):1028–1036.
Dubois D, Guyonnet D. Risk-informed decision-making in
the presence of epistemic uncertainty. Intl J Gen Syst, 2011;
40(2):145–167.
Aven T, Zio E. Some considerations on the treatment of uncertainties in risk assessment for practical decision making.
Reliab Engrg Syst Safety, 2011; 96(1):64–74.
Apostolakis G. A commentary on model uncertainty. In:
Mosleh A, Siu N, Smidts C, Lui C (eds), Proceedings of
Workshop on Model Uncertainty: Its Characterization and
Quantification, 1993 Oct 20–22; Annapolis (MD). Washington, DC: US Nuclear Regulatory Commission, 1993. Report
No.: NUREG/CP-0138.
Winkler R. Model uncertainty: Probabilities for models? In:
Mosleh A, Siu N, Smidts C, Lui C (eds), Proceedings of
Workshop on Model Uncertainty: Its Characterization and
Quantification, 1993 Oct 20–22, Annapolis (MD). Washington, DC: US Nuclear Regulatory Commission, 1993. Report
No.: NUREG/CP-0138.
Keeney R, von Winterfeldt D. Eliciting probabilities from experts in complex technical problems. IEEE Trans Eng Manage, 1991; 38(3):191–201.
Little MP, Muirhead CR, Goossens LHJ, Kraan BCP, Cooke
RM, Harper FT, Hora SC. Probabilistic accident consequence
uncertainty analysis. Washington, DC: US Nuclear Regulatory
Commission and the Commission of European Communities,
1997. 62 p. Report No.: NUREG/CR-6555 and NUREG/CR6523.
Budnitz R, Apostolakis G, Boore D, Cluff L, Coppersmith K,
Cornell C, Morris P. Recommendations for probabilistic seismic hazard analysis: Guidance on uncertainty and the use of
experts. Washington, DC: US Nuclear Regulatory Commission, 1997. 888 p. Report No.: NUREG/CR-6372.
Halman JIM, Keizer JA. Diagnosing risks in productinnovation projects. Intl J Project Manage, 1994;12(2):75–
80.