DOI: 10.1111/j.1539-6924.2012.01829.x Risk Analysis, Vol. 33, No. 1, 2013 Capturing and Integrating Knowledge for Managing Risks in Tunnel Works Ibsen Chivatá Cárdenas,1,∗ Saad S. H. Al-jibouri,1 Johannes I. M. Halman,1 and Frits A. van Tol2 Risk-related knowledge gained from past construction projects is regarded as potentially extremely useful in risk management. This article describes a proposed approach to capture and integrate risk-related knowledge to support decision making in construction projects. To ameliorate the problem related to the scarcity of risks information often encountered in construction projects, Bayesian Belief Networks are used and expert judgment is elicited to augment available information. Particularly, the article provides an overview of judgment-based biases that can appear in the elicitation of judgments for constructing Bayesian Networks and the provisos that can be made in this respect to minimize these types of bias. The proposed approach is successfully applied to develop six models for top risks in tunnel works. More than 30 tunneling experts in the Netherlands and Germany were involved in the investigation to provide information on identifying relevant scenarios than can lead to failure events associated with tunneling risks. The article has provided an illustration of the applicability of the developed approach for the case of “face instability in soft soils using slurry shields.” KEY WORDS: Bayesian Belief Networks; epistemic uncertainty; expert judgment elicitation; reliability modeling; risk-related knowledge integration; risk modeling project operations caused significant damage or injury to others. Muir Wood,(2,3) Sowers,(4) Bea,(5) van Tol,(6) Wearne,(7) and Mitchell(8) have all highlighted that failures in ground-related construction projects are mainly due to shortcomings in the use of available and relevant knowledge rather than the result of uncertainty due to unknown factors. These shortcomings reflected situations in which tacit or explicit information, such as technological knowledge, design assumptions, monitoring records, thresholds and tolerances, and risks, was ignored, improperly used, rejected, or not passed on by someone in the project. The objective of traditional approaches to project risk management is to identify those risks that can lead to project failure and to implement effective strategies to manage them. Information on the relevant causes and conditions in which failures arise is usually required as necessary inputs for 1. INTRODUCTION According to the International Association of Engineering Insurers (IAEI), more than €570 million of economic losses were incurred in 18 tunneling projects worldwide between 1994 and 2005 because of failures in underground construction projects.(1) Those failures also led to an average delay in completion of 19 months. These specific failures were events in which a collapse or excessive deformation of significant parts of the works occurred, or where 1 Department of Construction Management and Engineering, Uni- versity of Twente, Enschede, The Netherlands Geoengineering, Delft University of Technology, 2600 GA, Delft, The Netherlands ∗ Address correspondence to Ibsen Chivatá Cárdenas, Department of Construction Management and Engineering, University of Twente, P.O. Box 217, 7500 AE, Enschede, The Netherlands; tel: +31-53-895988, mobile: +31-6-57777783; fax: +31-53-489-2511; c.cardenas@utwente.nl. 2 Section 92 C 2012 Society for Risk Analysis 0272-4332/13/0100-0092$22.00/1  Capturing and Integrating Risk-Related Knowledge 93 determining possible risk strategies. In the particular case of tunneling, the Code of Practice for Risk Management of Tunnel Works(9) stipulates that the use of a formalized risk management procedure to document the identification, evaluation and allocation of risks is compulsory. Similarly, the Guidelines for tunneling risk management from the International Tunnelling Association(10) recommended the use of risk analysis to identify, quantify risks and to visualize their causes and effects as well as the course (chain) of events. To evaluate the risks associated with road tunnels in operation, risk analysis also became one of the explicit requirements under the European Union (EU) Directive (2004/54/EC).(11−13) Performing risk analysis for underground construction projects, however, is not straightforward in practice. Explicit and integrated knowledge about the relevant causes and conditions that lead to major risks is often absent. Therefore, there is a need to make this type of integrated knowledge available in order to facilitate risk analysis and management. In doing so it is very valuable to explore the feasibility of developing models that integrate knowledge about the risk factors in construction projects. The remainder of this article is divided into four sections. Section 2 provides a review of the background literature. Section 3 describes the successive process steps that have been followed in developing models that integrate knowledge on six important risks in tunneling projects. In Section 4, the applicability of the developed approach for risk elicitation and representation is illustrated for the case of “face instability in soft soils using slurry shields.” In the final section, a number of important contributions and limitations of this study are discussed, followed by some conclusions. ally recorded. As a consequence, the interrelationships between risk factors and project outcomes are difficult to understand.(17) To overcome, to some extent, the lack of historical data, expert judgments could be used as an alternative source to develop construction risk-related knowledge. Examples of this type of approach can be found in Bles et al.(20) and in Choi et al.(21) Expert judgment is often employed to bridge the gap between hard evidence and unknown characteristics of a system. This “knowledge” is not certain, and entails an implicit level of assumed confidence.(22) It is nevertheless widely acknowledged that experts, as compared with novice professionals, have greater abilities to identify relevant domain-related information.(23) Thus, the use of expert knowledge increases the likelihood of delivering relevant information to decision-makers. Nevertheless, expert judgments are prone to be affected by various factors leading to unreliable data. To increase the reliability in expert judgments, the literature provides criteria and procedures for eliciting this information while reducing the risk of obtaining biased judgments (e.g., Cooke and Goossens,(24) Ayyub,(25) Garthwaite et al.,(26) Goossens et al.,(22) Kynn,(27) and Hallowell and Gambatese(28) ). Further, integrating expert judgments is not straightforward. Under conditions of limited information, a diversity in judgments will usually emerge. Discrepancies in data are a sign that there may be epistemic uncertainties allowing different experts’ views to differ.(29) One reason for a diversity in judgments arises from the differences in the experts’ experiences regarding the events under consideration.(19) Literature offers various aggregation methods for discrepant judgments, however, capturing, processing, and using information conveying epistemic uncertainties is a subject that has yet to be satisfactorily resolved.(30) An initial investigation, based on specialized literature and explorative interviews, to determine the information required for this study allowed additional elements to be considered. Some of the mechanisms leading to failure events remain little known and might well consist of a large number of interacting factors. Hence, the integration of risk-related knowledge demands appropriate methods that represent uncertain knowledge and capture strongly interactive risks factors. There are many tools, with different purposes, for representing risks. Failure mode and effects analysis, hazard analysis, top-level event tree, and fault tree analysis are all methods regularly 2. BACKGROUND LITERATURE Although advances have been made in developing knowledge-based risk systems based on experience gained from past projects (e.g., Tah and Carr(14) ; De Zoysa and Russell(15) ; Choi and Mahadevan(16) ; Dikmen et al.(17) ), work remains to be done in capturing, representing, and using risk-related knowledge.(15,18) Capturing relevant knowledge on risks is constrained by the fact that historical data on failures are usually scarce, often confidential, and not available until several years after the event.(7,19) Similarly, information on the circumstances under which project outcomes were not accomplished is not usu- 94 used to represent risks.(31) However, these standard approaches would struggle to deal with the integration and analysis of uncertain information regarding construction risks. Recent research has proposed the Bayesian Belief Networks (BBNs) as a more suitable alternative for representing risks.(32,33) BBNs are especially suited for representing complex and uncertain relationships among many factors that contribute to the realization of risk.(34) Besides BNNs, there are other approaches to representing risks such as Markov Chains, Petri Nets, Artificial Neural Networks (ANNs), Analytical Hierarchy Process (AHP), Systems Dynamics, and Fuzzy Systems.(35) Some of these (Markov Chains and Petri Nets) are regarded as being too complicated for use by practitioners(36) and others, such as ANN, AHP, and Systems Dynamics, entail considerable modeling effort and require abundant data which are usually unavailable in the case of construction risks. Fuzzy Systems modeling would be an alternative to the BBNs approach, however, the BBN approach is based on a probabilistic background and hence has a more developed mathematical axiomatization in comparison to Fuzzy Systems approach. Dubois(37) and Novák(38) have identified remaining questions to be addressed in the field of Fuzzy Systems, among others, developing a general axiomatic framework for ranking purposes and the refinement of aggregation functions which are important tasks in risk analysis. In addition, BBNs approach is simpler to use than Fuzzy Systems. Its inference mechanism is based on the Bayes theorem, which makes it possible to compute the probability of an effect on any variable in the model from the probability of a given cause. In comparison, Fuzzy System modeling relies on fuzzy inference as described in Mamdani and Assilian(39) and Sugeno(40) . Accordingly, the process of fuzzy inference usually requires additional computational effort that might exasperate the computation costs. For instance, a number of membership functions, logical operations or if-then rules, fuzzyfication and defuzzification tasks are needed to be developed and computed. Furthermore, additional problems concerning judgment-based biases are raised from the need to elicit the fuzzy structures, which are more complex that those ones used in BBNs. Some researchers applied other approaches to deal with uncertainty due to lack of information. Ferson and Ginzburg(41) proposed using interval analysis. Baraldi and Zio(42) presented a method based on a combination of Monte Carlo and Possibilistic ap- Cárdenas et al. proaches. Baraldi and Zio(43) used Dempster–Shafer structures. For the latter authors, issues such as modeling uncertainty of unknown variables and the evaluation of Bayesian methods to model uncertainty when new information becomes available; are subjects the deserve further study.(43) Recent improvements of the BBNs approach seem to provide options to address the abovementioned issues which might be favored by the mechanism of inference employed, its level of mathematical axiomatization allowing essential tasks in risk analysis to be performed in a consistent manner and the use of approximated models to represent causal relationships and unknown factors, which requires fewer estimates and accordingly much less elicitation effort. BBNs have been used in the construction industry for various purposes. For instance, Nasir et al.,(44) Marzouk and Eldieb,(45) Edieb,(46) and Luu et al.(47) have developed models for construction schedule and cost overrun risks on a project basis, whereas Bayraktar and Hastak(48) introduced a decision-support system to study the performance of highway work projects. In terms of underground construction projects, Bles et al.(20) have applied BNNs to model construction risks. Surprisingly, given the importance of using reliable information, little is reported in the noted publications on the provisions that have to be made to capture expert knowledge to develop BBNs in such a way that reliability is ensured. In line with this, Bielza et al.(49) observed that the details of how to elicit knowledge from experts to construct Bayesian Network models are not well documented in the literature. 3. MODEL DEVELOPMENT PROCESS This section describes the successive process steps that have been followed to develop models that integrate information on six risks in tunneling projects. The research process can be divided into five steps: (1) selection of risks, (2) elicitation of experts’ judgments on the selected risks, (3) discrepancy analysis, (4) developing a BBN-based model for each risk, and (5) model evaluation. 3.1 Selection of Six Main Underground Construction Risks On the basis of the literature studied including risk databases, reports on failure events, and treatises on tunnel works, a detailed inventory of risks and Capturing and Integrating Risk-Related Knowledge 95 events was made that included more than 500 risk issues related to different stages in tunnel projects. Evidence reported by the International Association of Engineering Insurers(1) suggested focusing on critical risks in the construction phase because the IAEI report shows that most failures occur during the construction stage. To determine the risks to focus upon, issues related to the construction stage in tunnel works were classified into three main categories: failures, causes, and impacts. Typically, issues related to injuries or loss of life, damage to third parties, additional costs, delays in completion of the project, or failure to meet quality requirements were regarded as impacts. The immediately preceding events that led to the abovementioned impacts were identified as failures. The events causing and preceding the failures were identified as causes. A number of exploratory interviews with specialists were conducted to elicit their perceptions of the most important risks in tunneling works during the construction phase. Details on the experts consulted are shown in Table I. During the exploratory interviews, the experts were asked to review and classify the risks included in the developed inventory into the predefined categories of: failures, causes, and impacts. The experts were also encouraged to raise other relevant issues that they regarded as being absent from the inventory. Similarly, some issues were removed because they were considered as irrelevant. The round of exploratory interviews failed to achieve full agreement among the experts on what should be considered as the main risks. Nevertheless, the following issues do represent the ones that were most frequently selected by experts as being major risks during the construction phase: these risks. Consequently, these major risks were selected as the further focus of the research and model development. • Face instability in soft soils when using Slurry Shields; • Face instability in soft soils when using Earth Pressure Balance Shields; • Excessive volume loss leading to settlement in tunnels bored in soft soils; • Excessive deformation, damage, or leakage in concrete linings; • Collapse and large deformations of excavated shafts in soft soils; and • Collapse and large deformations of excavated cross passages in soft soils. The experts’ reasons for selecting the above risks were their high impact in terms of costs, delays, and damage; and the particular difficulty in controlling 3.2 Data Requirements and Scope Usually, risks are characterized merely in terms of their probability and impacts.(35) In this research, the risk characterization is extended to include the: • Relevant causes and conditions (risk factors) associated with the risks under study; • Plausible relationships among the risk factors; • Likelihood of occurrence of the risk factors; • Strength of the relationships among the risk factors. The relevant risk factors are assumed to be those identified as such by experts and are defined in terms of deviations from project requirements (assumptions, expectations, specifications, tolerances, and thresholds) that could lead to the failure being considered. Information on the likelihood of the relevant causes depends on the particular setting of each project. However, for test purposes, experts were asked to give their best estimation of the likelihood of the risk factors occurring. Similarly, because construction risk impacts are very project-specific, information on risk impacts was deliberately excluded from the characterization of the risks. This means that information on risk impacts has to be incorporated into the characterizations on a case-by-case basis. 3.3 Expert Elicitation Process 3.3.1 Identification and Selection of Experts for the Elicitation Process In selecting experts, names of professionals were initially identified from the specialized literature on tunneling in the Netherlands. In this way, 126 people were identified as professionals involved in ongoing or past underground construction projects such as tunnel boring and deep shaft excavations. These professionals were then contacted and also asked to suggest other reputable specialists. By studying the topics of the papers written by the 126 identified specialists, the list was shortened to 59 who were invited to provide information for this research project. A total of 31 professionals agreed to be interviewed. The participating experts all originated from 96 Cárdenas et al. Table I. Characteristics of the Experts Involved Number of experts interviewed Tunneling experience 0–5 years 6–10 years 11–15 years >15 years Number of experts with publications None 1 publication 2 to 5 publications More than 5 publications Current position Researcher Design consultant Construction supervisor Constructor Project manager Tunneling method experience Slurry shield Earth pressure balance shield Cut and cover Others Number of projects Involved in 1–2 projects Involved in 3–5 projects Involved in 6–10 projects Involved in >10 projects Length of tunnels built 0–5 km 5.1–10 km 10.1–15 km 15.1–20 km >20 km Location of last project Urban settings River/sea crossing 1st Round of Interviews 2nd Round of Interviews 3rd Round of Interviews 7 11 1 1 4 1 24 Number of experts 3 3 11 7 0 4 2 1 1 22 0 1 0 8 1 2 0 3 1 1 2 2 10 4 3 5 1 2 3 3 2 5 2 3 4 23 7 19 18 11 1 7 7 3 1 1 2 4 10 7 3 4 6 1 0 1 2 1 0 3 8 6 5 2 3 2 4 3 2 0 6 1 22 2 8 3 the Netherlands or Germany. The specialists work or had worked for organizations such as research institutes, governmental agencies, or companies providing tunnel design, supervision, and construction services. Table I shows the characteristics of the specialists consulted in more detail. 3.3.2 Elicitation Process Design Experts’ judgment is prone to be highly biased and this influences the reliability of the data collected. To reduce the likelihood of obtaining biased data, the elicitation procedure used in this investigation largely followed the criteria provided by Cooke and Goossens,(24) Renooij,(50) Ayyub,(25) Garthwaite et al.,(26) Goossens et al.,(22) Kynn,(27) and Hallowell 0 0 9 2 and Gambatese.(28) The specific criteria that were adopted are referred to where appropriate throughout this section. Fig. 1 is an adaption of the elicitation processes described by Goossens et al.(22) and Hallowell and Gambatese,(28) and shows the successive steps that were taken during the elicitation process. Table II provides an overview of the potential biases that were considered during the elicitation process and how these were avoided or at least minimized. Direct one-to-one interviews were the main instrument used to elicit the experts’ judgments. This choice was based on the fact that gathering the required input data, which is specified in Section 3.2, demands detailed interaction with the specialists to provide the opportunity to discuss possible emerging divergences in data to be analyzed further. Capturing and Integrating Risk-Related Knowledge Objectives: - identification of top risks in tunnel works, - identification and selection of experts. Literature and documents review Identification of experts 97 Objectives: - identification of the relevant risk factors associated with the top risks, - elicitation of chance of risk factors. Literature and documents review Pilot interviews (semi-structured) Selection of experts Pilot interviews (open-ended interviews) Objectives: - elicitation of strength of influence of relationships amongst risk factors, - internal validation of relationships. Pilot interviews (semi-structured) Second round of interviews (semistructured) First round of interviews (open-ended interviews) Third round of interviews (semistructured) Develop feedback on divergences Develop feedback on divergences Evaluation of data, -discrepancy analysis, -sensitivity analysis Model prototype Fig. 1. Expert elicitation process (based on Goossens et al.(22) and Hallowell and Gambatese).(28) Even though literature indicates that there is no significant difference in terms of reliability between individual interviews and group approaches such as a workshop sessions(24,51) ; an important reason for choosing individual interviews is that they help avoid the obtained data being unreliable due to biases in judgment caused by phenomena such as “collective unconsciousness” and “dominance,” which can occur in group sessions as described in Table II. The above arguments led to the conclusion that individual interviews would be an efficient and reliable option. In this work, an interview protocol, de- scribed later in this section, was designed to address further considerations reported by Clemen and Winkler,(51) who stated that direct interaction among experts in a group session can be valuable in “ironing out” differences in definitions and assumptions, clarifying what is to be assessed, and exchanging information before the elicitation process. The interview protocol designed in this work meets these requirements without the need for group sessions. Additional provisos were included to ensure that the information provided by an expert would be usable. Adams(19) commented that engineering experts 98 Cárdenas et al. Table II. Judgment-Based Bias in Risk Studies and Provisions Made in This Particular Study to Minimize Them Bias Definition Representativenessa,b Subjects do not base their estimates on frequency of experienced occurrence data Collective unconsciousc Individuals tend to follow a popular trend Contrast effectc Perception of a given subject is enhanced or diminished by the value of the immediately preceding subject Individuals underestimate the role of probability/uncertainty in the subjective quantification of risk Neglect of probability/poor interpretation of uncertainty (variance)b,c Overestimation of probabilitya People overestimate values when the events they are asked about are linked to high magnitude impacts Myside biasc With controversial issues, individuals tend to provide arguments from just one perspective, i.e. ignore contrary evidence Recency effect (availability)a,b,c Subjects are more likely to artificially inflate risk ratings because similar incidents have recently occurred, and correspondingly underestimate those that are in the remote past Subjects generally overestimate a priori probabilities for events they thought had occurred and underestimate for events they thought had not occurred People expect a sample from a population to represent all the essential characteristics of a population Hindsight biasb Law of small numbersb Primacy effectc Dominancec Misunderstanding of events Overconfidencea,b,d a Ayyub.(25) b Garthwaite c Hallowell et al.(26) and Gambatese.(28) d Adams.(19) An individual is more likely to assign importance to the issues at the beginning of an interview than those at the end of it A very vocal or intimidating group member controls the ratings of the other members Vague identification of the variables and events on which probability estimates are assigned Subjects tend to provide narrower confidence intervals compared to real intervals Provision made to minimize bias Using diagrams showing relationships with other variables with unconditional probability assignments so that the estimates provided were reviewed and made coherent; unconditional and conditional probabilities were elicited and processed in Bayesian networks No interaction among specialists, individual interviews Randomization of questions during the interviews Using intervals of probability, rejection of outlying data according to discrepancy analysis and Bayesian aggregation methods of probabilities Requesting estimates of probabilities independently of the impacts, using intervals of chance, requesting degrees of confidence on the assessments, rejecting outlying data according to discrepancy analysis and Bayesian aggregation methods of probabilities Revisiting conflicting issues in a third round of interviews and retaining the discrepant evidence until performing discrepancy analysis at the end of elicitation process Requesting estimates of probabilities independently of the impacts, rejection of outlying data according to discrepancy analysis and Bayesian aggregation methods Multiple expert elicitation process, rejection of outlying data according to discrepancy analysis and Bayesian aggregation methods of probabilities Multiple expert elicitation process, rejection of outlying data according to discrepancy analysis and Bayesian aggregation methods of probabilities Specialists were given the chance to select the risks to assess No interaction among specialists, individual interviews, equal weighting of responses, data shared anonymously with other specialists Review by external professional in construction and revisiting the specialists over vague issues Subjects are required to give probability assignments in terms of probability intervals, requesting degree of confidence on the assignments Capturing and Integrating Risk-Related Knowledge 99 reach the height of their expertise after about ten years of relevant experience. Accordingly, information used in this study is only drawn from experts with a minimum of ten years of tunneling experience. Further, the minimum number of experts was considered. In this research, probabilistic input information used to characterize each risk studied had to be provided by at least three experts. The use of this small number of experts is based on the work described by Clemen and Winkler(51) and Adams.(19) Clemen and Winkler(51) indicated that with three to five experts much of the total knowledge is usually achieved. Nevertheless, in this work the size of an expert group was increased if and when significant discrepancies amongst the data became apparent during the elicitation process. As shown in Fig. 1, the elicitation process was carried out in three rounds of interviews. First round of interviews were the initial exploratory interviews described in Section 3.1. The second round of interviews was carried out to gather data on potential causes of the selected tunneling risks. At the beginning of this round of interviews, the experts were presented with a preprepared list of potential risk factors as derived from the literature study and were asked to add possible additional factors to this list and to suggest plausible relationships. After collecting information from various experts, the inventory was translated into node-and-arc diagrams showing the risk factors and their relationships. A node-and-arc diagram was constructed for each of the risks studied. Once these had been drawn up, the diagrams were used during all further elicitation sessions. In this way, information anonymously provided by one expert was successively reviewed by other experts later in the process and iteratively refined to ensure that dependencies and directions in the relationships arcs reflected the experts indications. As a precaution to ensure comprehensiveness in the data, the experts were explicitly asked to give their judgments based on the real situation found in practice and not on ideal situations. Goossens et al.(22) have pointed out that, if this proviso is ignored, experts might implicitly assume that the works to be considered are those designed and built in satisfactory but exceptional conditions, and hence they might disregard a large proportion of the relevant risk factors. Experts were finally asked to give their estimates on the likelihood of the risk factors occurring regardless of the extent of their possible impacts. This procedure was adopted to reduce the effect of proba- bility overestimation that can occur when individuals are asked about events linked to high magnitude impacts. This is a recognized judgmental bias, known as the “contrast effect,” and is described in Table II. Special effort was also made to establish unambiguous identifications of the risk factors. As such, the risk factors were described using a format to ensure clear interpretation and hence reliable judgments. The descriptions of the factors were also revisited if there were wide discrepancies among the probability estimates provided by the different experts. The second round of interviews was ended once three successive sessions failed to add any additional information on the risk factors obtained. This “data saturation point” was initially identified in the 21st interview and confirmed in later elicitation sessions. Thus, in total, 24 experts participated in the second round of this study. On average, each specialist provided information on two failure types related to their own fields of expertise. There were some instances where some experts provided only partial information on the potential failures under analysis, arguing they were not able to provide the required information. On average, each failure type was analyzed by five experts. Each failure is described using more than forty factors. An example of the information obtained is presented in Section 4. Following the second round of interviews, a third and final round of interviews was carried out to collect information on the strength of the influences of the earlier identified relationships. This round was also used to internally validate the collected data with the support of the experts involved. Because the objective of this round was to measure the strength of the influence of each of the identified relationships, a validation of the structure as represented in the diagrams was performed indirectly. The individual estimates of the strengths of the influences of the relationships were aggregated mathematically into probabilistic distributions. 3.3.3 Interview Protocol The interviews were guided by a protocol using the elicitation criteria referred to above. Each elicitation session started with an explanation of the interview structure. Following this, each interview was conducted in three phases. The first phase was intended to collect information about the specialist’s background by completing a specifically designed form. The expert’s background data were collected to assess 100 the professional’s ability to provide information relevant to the study and to decide how to use the information provided. In this phase, the interviewees were further informed about the confidential treatment of their personal details and that the data provided would only be shared anonymously with other experts. The second phase of the elicitation sessions was primarily intended to familiarize the experts with the major research issues so that they were able to develop a common understanding. The core part of the interviews, the third phase, concerned the elicitation of risk information. Initially in this third phase, the experts were presented with the set of major risks and given the option to choose which ones to assess during the interview. This particular action was intended to obtain data that was closely related to the particular expert’s experience and thus avoid the “primacy effect” judgment bias as described in Table II. 3.4 Discrepancy Analysis Once information describing the risks was collected, a discrepancy analysis was performed. Discrepancy analysis aims to identify data pieces where the experts’ assessments differ most. These data should be reviewed to ascertain any avoidable discrepancy causes(24) or so that one can adopt values with associated confidence bounds.(25) Discrepancy analysis is intended to eliminate outlying data originating from potential judgment biases, as described in Table II. Discrepancy analysis was performed throughout the entire elicitation process although a definitive assessment was made at the very end of the process to avoid any unjustified elimination of information. This approach is intended to reduce the effect of another judgmental bias known as “myside bias,” which in this particular study would be associated with the arbitrary neglect of contrary evidence. Discrepant information can arise in the description of risk factors, in the existence and direction of relationships among the factors, in the probability assignments, and in the estimates of conditional probabilities. In performing the discrepancy analysis, two assessments were made following the criteria described by Woodberry et al.(52) These assessments consisted of comparing values across different experts and reexamining those instances where experts indicated low confidence. Experts were encouraged to indicate their confidence in their estimates using a numerical Cárdenas et al. scale. Values of variables with a low degree of confidence were flagged for later attention. Further checks were undertaken. Discrepant data are also a sign that there may be epistemic uncertainties that allow expert views to differ due to incomplete evidence.(29) The occurrence of this particular form of discrepant data is observed when two or more significant and equivalently sized sets of estimates point toward conflicting assessments. To illustrate this, let us assume that two experts, A and B, are asked to provide estimates of the probability of a given event. There is a strong disagreement between the two experts: one expert confidently estimates that the event has a 1 in 200 chance of occurring; the other expert confidently estimates the probability as much higher at 1 in 50. This is identified as an epistemic uncertainty if the disagreement is maintained with a larger number of experts. After the discrepancy analysis was performed, divergent data were revisited, sometimes modified by their providers, or disregarded when either a high discrepancy remained. In case of emerging epistemic uncertainties, these were further analyzed to observe their impact in the model performance. Note that discrepancy analysis was only performed to trace divergent information in the likelihood and conditional probability assessments. 3.5 The BBN Risk Models A Bayesian Belief Network, or BBN, essentially provides a framework for representing the causal relationships between variables and capturing the uncertainties in the dependencies between these variables using conditional probabilities.(53) In a BBN interrelationships between variables are expressed graphically. Variables that have interdependencies are connected, whereas independent variables are not. The direction of the arrows reflects the direction of causal influence, as perceived by the expert or confirmed experimentally. A probability distribution for each variable and the relationship in the network is assigned. Figs. 2 and 4 provide an example of Bayesian Belief Networks. Some relevant details about the construction of the developed risk models using the BBN framework are described in this section. As discussed earlier, some variables in the risk models are not known with certainty. To reflect this uncertainty, the variables are represented using “evidence probability distributions.” An evidence distribution is a representation that provides information Capturing and Integrating Risk-Related Knowledge 101 Bentonite leakage into the ground Excessive loss of support Frequent 8% Frequent 25% Probable 8% Probable 38% 32% Occasional 42% Occasional Remote 33% Remote 4% Improbable 9% Improbable 1% Absent 0% Absent 0% Excessive loss of air pressure Fig. 2. Example of distributions representing evidence supporting the variables in the models. Frequent 0% Probable 0% Occasional 0% Remote 0% Improbable 0% Absent 100% Underesmaon support pressure Label Frequent Probable 0% Occasional 0% Remote 0% Improbable 0% Absent Chance Descripon 0% 100% on values of the variables and available supporting evidence based on assessments by experts. In this study, the variables are represented so as to reflect the evidence that supports the values attached to the variables. This is achieved by using a graphical representation of the uncertainty associated with each variable to provide an instant overview of the strength of the evidence supporting each model. Information on the available evidence is regarded as of paramount importance to the end-users in this study. Fig. 2 shows a set of variables and their evidence probability distributions. This particular example shows, for the variable “Excessive loss of support,” that the “Probable” likelihood category has more supportive evidence than the other states of this variable. Each variable in the six developed risk models is an event or condition that represents a fault event, state of failure, or an unfavorable condition. Fault events and states of failure of a variable can be events in which a risk factor exceeds a predefined threshold. In this research only discrete variables are used and, initially, most of the variables were allowed two possible states: absent or present. A variable is regarded as being “Absent” when it is not active based on the particular conditions included in the analysis. As an example, in the situation under analysis in Frequent >30% Probable 10-30% To occur several mes during construcon of tunnels Occasional 5-10% To occur once during construcon of a tunnel Remote 1-5% To occur from project to project Improbable 0-1% Extremely rare to occur To occur repeatedly during construcon of tunnels Fig. 2, only one variable is being taken into account whereas two are absent. The “Present” state is further discretized into five levels of probability. Some variables can have more than two possible states, but few of the variables in the models needed to be expressed in such a way. To obtain trial information for the risk models, experts were requested to provide the chance of the risk factors in terms of qualitative probability estimates using a five-point scale. Terés Flores,(54) Visée,(55) and Bielza et al.(49) reported positive results using this approach and emphasized that most people find it easier to express probabilities qualitatively rather than quantitatively. This was supported by the experts that were consulted in this study. Using qualitative probabilities is also beneficial in reducing bias in data, as shown in Table II. In accordance with Dı́ez and Druzdzel,(56) specific questions were asked to each expert to obtain conditional probability estimates for each kind of relationship. Conditional probabilities on exclusive relationships were obtained by addressing the question What is the chance that a cause Xi (conditioning variable) results in an event Y (conditioned variable) when no other cause is present? For the inclusive interactions, the question addressed was What is the 102 chance that Xi (conditioning variable) lowers Y (conditioned variable) when no other factor has lowered it? Inclusive interactions are those in which, given a number of known causes of a variable occurring, a change in the conditional variable is necessarily due to a combination of changes in every cause occurring at the same time. In inclusive interactions, the conditioning variables might be mutually supportive. Conversely, in exclusive interactions, the causes are each assumed to be able to produce a change in the conditional variable in the absence of other causes, and their ability to cause a change in the conditional variable is assumed to be independent of the presence of other causes.(57) The explanations as to why these questions were worded in such a way are described in Onisko et al.(57) and Dı́ez and Druzdzel.(56) The experts were requested to provide estimates of the conditional probabilities in terms of qualitative probabilities using the same scale of five categories deployed for measuring the chance of variables described earlier. 3.6 Risk Models Evaluation To obtain the desired reliability in the models that integrate knowledge of the risks under study, a set of provisos was made. Probability estimates and conditional probabilities were used to define the occurrence of the variables and their relationships in the models. As discussed earlier, this input information was subjected to a discrepancy analysis. Discrepancy analysis provided information on which information pieces were appropriate to incorporate in the models and which should be rejected or further analyzed to assess the effect of epistemic uncertainties. In addition, a review of the models’ structures was carried out by various experts during the elicitation sessions. By using the networks that depicted the risks under consideration each expert had the chance to successively review the relationships among the variables in the models and estimate the strength of the influence of these relationships. This intensive review of the relationships can be seen as providing internal model validation. In a few instances, the experts had diverging views on the relationships, and the impact of these divergences on model performance was investigated using different relationship variants. The research steps thus ended with the evaluation of each risk network model representing each of the six identified major tunnel risks. Cárdenas et al. 4. EXAMPLE: TUNNEL FACE INSTABILITY RISK MODEL This section provides details of the developed tunnel face instability risk model for slurry shields in soft soils. When boring a tunnel using tunnel boring machines (TBMs), the soil is removed by a cutting head and transported back from the excavation face. To temporarily support the excavated ground, some types of TBMs have pressurized compartments at their front end to balance the ground and water pressure.(58) Face instability in bored tunnels occurs when the ground and water pressure, in conjunction with the shear strength available in the ground, are not in equilibrium with the pressure provided by the boring machine. This situation can potentially lead to a face collapse.(59) The integrated knowledge on the tunnel face instability risk covers issues related to the Dutch ground conditions and construction practices. Soft soils with low stiffness and a high groundwater table are the dominant ground conditions in the Netherlands, and, particularly, saturated medium-fine sandy soils are the usual ground conditions encountered when tunneling.(60,61) Tunnel boring is typically carried out in the Netherlands using closed shields such as slurry and earth pressure balance shields. These closed shields provide continuous support to the face during excavation. Slurry shields use bentonite suspensions as a supporting fluid. Earth pressure shields use the excavated soil as support, often conditioned with bentonite slurry and other additives.(62) Fig. 3 displays the components of the tunnel face instability risk model (variable states are not shown for reasons of clarity). Three major event-scenarios (the ovals in Fig. 3) can lead to face instability when using slurry shields: excessive loss of support, excessive support pressure, and loss of air pressure. These three scenarios share common causes that can be clustered into components such as faults in the excavation process, in design, or in monitoring, and mistakes by on-site operators, and are further affected by variables related to the ground conditions (the boxes in Fig. 3). Fig. 4 shows a fragment of the model that represents those variables associated with the eventscenario “excessive loss of support.” The submodel consists of subsets of variables that can be classified into causes, conditions, and inhibitors. Causes are those variables that can lead to the failure occurring. Conditions are the variables necessary to Capturing and Integrating Risk-Related Knowledge 103 Excessive loss of air pressure Excessive loss of support Ground condions Face instability Excessive support pressure Construcon process Design Monitoring Machine operator acons Response acons Fig. 3. Face instability risk: model prototype. Excessive pore pressure Unexpected variability of soil permeability Causes Incorrect properes of bentonite Ground condions Incorrect TBM penetraon rate Deficient control bentonite properes Incomplete ground model Underesmaon of support pressure Open wells/pipes going to surface Bentonite leakage into ground Insufficient penetraon of bentonite Excessive loss of support Face instability Local face instability Excessive loss air pressure (maintenance) Insufficient air supply Insufficient bentonite pressure Miscommunicaon operator monitoring Reacon to loss of support event Incomplete countermeasures Machine operator mistake Fig. 4. Excessive loss of support risk submodel. Inhibitors 104 produce the failure but that depend on the causes in the model to effectively drive the occurrence of failure. Inhibitors are those variables with the capability of reducing the chance of failure. An example inhibitor variable in the model is “reaction to excessive loss of support.” If an excessive loss of support occurs, the reaction determines whether the event can result in face instability. Ground conditions typically function as conditioning variables. For instance, to create an excessive loss of support event, “open wells or pipes extending to the surface” is a necessary condition for rising a bentonite leak into a ground event. The fragment of the face instability risk model shown in Fig. 4 takes into account other factors than those currently used in analytical methods to estimate tunnel-face instability. In these other methods, tunnel-face stability is estimated from factors such as overburden stresses and excess pore pressure in the ground, effective weight, strength, soil particle size, yield strength of the bentonite. As such, the presented model extends information on the risk of face instability by involving other relevant variables. The model was populated with the estimates of experts after performing the discrepancy analysis described earlier. As a result of the final discrepancy assessment, three pieces of evidence out of 136 estimates of conditional probabilities measuring the strength of the interactions among factors and 13 of the initial set of the (170) unconditional probability estimates associated with the likelihood of the factors occurring were eliminated. With this refinement in the data, most of the unconditional evidence probability distributions (25) and the conditional probabilities distributions (16) are spread over two to three categories of chance. To illustrate the functionality of the developed model, experimental data on the chance of risk factors is incorporated and propagated into the model. Unconditional distributions representing the likelihood of the risk factors occurring are attached to each variable in the model. The risk factors associated with ground conditions were modeled using uniform distributions. The model propagates this information and particular analysis can be performed, for instance, the identification of the most sensitive factors given certain conditions in a project. Details of this analysis will be presented in a separate paper. In this article an example of the analysis of a fragment of the developed model is described. The fragment under analysis corresponds to the set of risk factors directly associated with the event “excessive Cárdenas et al. loss of support” and is shown in Fig. 4. According to the sensitivity analysis carried out using experimental unconditional probabilities, it appears that the event “Underestimation of support pressure” is more critical to the occurrence of an excessive loss of support than the other influencing risk factors. The results of this analysis are represented by means of a tornado graph as shown in Fig. 4. The numbers indicate the relative importance of the risk factors based on the estimated values of the sensitivity indicators. This provides an indication of how resources can be apportioned to control the occurrence of excessive loss of support event. Such analysis can be repeated for any set of variables in the model. 5. CONTRIBUTIONS, LIMITATIONS, AND CONCLUSIONS The main objective of this study has been to develop and to demonstrate the application of an approach for collecting and integrating risk-related information. Six risk models were developed using this approach. In the remainder of this section, a number of important contributions and limitations of this study are discussed and the section ends by drawing some conclusions. 5.1 Contributions A recent review by Bielza et al.(49) draws attention to the fact that details on how to elicit expert judgments to construct Bayesian Network models have not been well documented. Renooij(50) reported that building Bayesian Belief Networks usually entails an elicitation of a large number of probabilities which is often referred to as a major obstacle to further demonstrate the feasibility of modeling complex real-life problem domains. This problem has been ameliorated with the development, for instance, of approximations of Conditional Probability Tables such as Noisy-MAX and Noisy-AND gates which require fewer estimates to model causal relationships. A comprehensive framework on the use of this approximations were provided by Dı́ez and Druzdzel.(56) This particular development and others have encouraged expert judgment to be encoded to develop Bayesian Belief Network models. Accordingly and as described by Renooij,(50) it is important to consider that issues related to human capabilities with respect to making judgments come into play when relying on experts for probability elicitation. Extensive psychological Capturing and Integrating Risk-Related Knowledge 105 research has shown that people, including experts, can deliver biased probability assessments. Decision analysis literature have provided a number of methods for the elicitation of judgment. These methods needed to be particularly customized for constructing Belief Networks to enable epistemic uncertainty to be captured while providing models building efficiency. An important first contribution of this study is therefore the detailed description and justification of the successive interactive steps needed to elicit expert knowledge for constructing risk models based on Bayesian Networks. To this end, criteria for identifying and selecting experts, the risk elicitation process itself, and data evaluation were reviewed and integrated. Further, a number of judgmental biases were identified as factors that potentially affect the development process of Bayesian Network risk models. To minimize the effect of these biases, a set of provisos were specified. These provisos include issues concerning the conditions of elicitation sessions, the application of discrepancy analysis, the operationalization of variables in the risk models, the process of review and feedback to experts, and the form of the questions addressed to the experts. Traditional elicitation procedures strive to reach consensus among participants, and hence eliminating information that might be associated with epistemic uncertainties which reflects a judgmental bias.(26,28) Unlike these traditional procedures, the proposed approach uses criteria that avoid epistemic uncertainties being arbitrarily eliminated. The assessment of the effect of epistemic uncertainties is important because it provides information on the uncertainties involved in risk management decisions.(29,63−66) In line with the above, the concept of an evidence probability distribution is used in this study to reflect the epistemic uncertainties in variables and relationships. Information on variables and relationships in the models was probabilistically “weighted” in such evidence probability distributions and then propagated into the Bayesian Networks. A second contribution of this research concerns the use of Bayesian Networks, in conjunction with the proposed approach, to capture risk-related knowledge. This combination is beneficial in avoiding or mitigating undesirable biases that are known to be common problems in expert judgment elicitation processes, as noted by Paté-Cornell,(29) Tah and Carr,(14) Garthwaite et al.,(26) Dikmen et al.,(17) and Taroun et al.(35) To summarize, in this process, the following benefits were achieved: • Evidence probability distributions: Existing methods to aggregate or average the judgments of multiple experts are questioned as they can lead to misrepresentation of uncertainty in models. The proposed evidence probability distributions, updated with the Bayesian scheme, however provide an alternative that does not truncate or mask divergent evidence. • Probability intervals: Arriving at probability distributions is often a difficult task. The way variables are represented here, using probability distributions derived from probability intervals, leads to a reduction in biases in the probability assessment and to a rapid elicitation process. • Bayesian Networks for multivariate elicitation: To elicit a joint probability distribution for two or more quantities, experts are required to somehow provide joint probability estimates, a process which is difficult and unreliable. Bayesian Networks enable risks to be broken down into simple tasks, allowing probability assessments and joint probability distributions to be obtained without “mental acrobatics.” • Risk diagrams: In any elicitation process, the raw data provided by experts are commonly expected to be inconsistent with the laws of probability. Bayesian Networks assist in checking and adjusting internal consistency. Similarly, the use of diagrams representing risks proved to be very beneficial in the elicitation process. Each expert could trace paths for different chains of factors allowing them to verify their quantitative judgments by comparing their probability estimate with others in the diagram and to add further information such as previously disregarded risk factors. Another contribution of this study concerns the treatment of epistemic uncertainties and the time involved in developing risk models. If the developed approach is compared to existing methods such as the Mixture approach,(67) Bayesian updating,(68) the NUREG-1150 approach,(69) the PACUA approach,(70) the Technical Facilitator-Integrator approach,(71) Risk Diagnosis Methodology(72) ; the developed approach uses Bayesian Networks to assemble evidence which has the effect of reducing the elicitation effort and avoiding epistemic uncertainty being misrepresented. In the developed approach, a 106 limited set of questions is addressed by experts leading to a less time-consuming elicitation process. 5.2 Limitations This study has concentrated on the development of models for six tunneling risks through eliciting risk-related knowledge from experts. Future research could extend the findings from this research by addressing its current limitations. A first limitation concerns the incompleteness of the developed tunnel risk models. Although the developed models do provide comprehensive information on risk factor interactions including those rarely measured during tunnel construction, these models remain inherently incomplete because they are unlikely to fully encompass the range of possible risk factors. Such incompleteness might be due to the experts having a limited understanding of the risks, or to undetected flaws in the elicitation procedure. As new information becomes available from further research, the models could be updated to reflect this enhanced information. A second reservation concerns the added value of the models when used in practice. The models will have to be evaluated on their performance in providing useful and consistent information for deriving risk measures while affording an assessment of the effect of discrepant data caused by epistemic uncertainties. In addition, one needs to assess the model performance when faced with different degrees of confidence in the information pieces provided by the experts. In these models, the same degree of confidence was assigned to each piece of evidence. The integrated knowledge used to characterize tunnel risks included issues limited to soft soils similar to Dutch ground conditions. Dutch ground conditions are characterized by saturated, low stiffness sandy soils with medium-fine size particles and a high groundwater table. The developed tunnel risk models refer only to tunnel boring with closed shields, such as slurry or earth pressure balance shields, and using concrete linings as the definitive support. Two of the developed models contain issues related to excavating shafts and cross passages, but these are also limited to the same ground conditions. Future studies could extend the research focus to include other ground conditions and also develop additional models for other tunnel works risks. A final remark on the limitations of this study concerns the question as to what extent the approach advocated in this article could be applicable for other Cárdenas et al. types of civil engineering projects and indeed other projects in general. We believe the methodology outlined in this article has the potential to assist in providing clear information on relevant scenarios that could lead to project failure elsewhere, but this remains to be proven. 5.3 Conclusions The aims of this study were twofold. First, the article describes the development of an approach to collect and integrate risk-related information that resulted in six models for major tunneling risks. The feasibility, in terms of cost-effectiveness and reliability, of developing such models for construction risks has been shown. The second aim was to demonstrate the applicability of the proposed approach. In this article, the approach was successfully applied to the risk of “face instability in soft soils when using slurry shields.” ACKNOWLEDGMENTS The authors are very grateful to the anonymous reviewers who provided some valuable and useful suggestions. This research has been supported by DELTARES, whose financial support is gratefully acknowledged. The authors also thank the experts who agreed to be involved in this research, without whose insights and support the work could not have been achieved. REFERENCES 1. Landrin H, Blückert C, Perrin JP, Stacey S, Stolfa A. ALOP/DSU coverage for tunnelling risks? Boston: International Association of Engineering Insurers, 2006. 19 p. Report No.: IMIA WGP 48 (06). 2. Muir Wood AM. Tunnel hazards: UK experience. Pp. 149–164 in Davis JS (ed.) Conference of Hazards in Tunnelling and on Falsework. London: Institution of Civil Engineers, 1975. 3. Muir Wood AM. Tunnelling: Management by Design. London and New York: Taylor & Francis Group, 2000. 256 p. 4. Sowers GF. Human factors in civil and geotechnical engineering failures. J Geotech Eng, 1993; 119(2):238–256. 5. Bea R. Reliability and human factors in geotechnical engineering. J Geotech and Geoenvir Engrg, 2006; 132(5):631–642. 6. van Tol AF. Schadegevallen bij bouwputten. [Damage caused by Deep Excavations]. Cement, 2007; 59(6):6–13. Dutch. 7. Wearne S. Organisational lessons from failures. Proc Instit Civil Eng-Civil Engrg, 2008; 161(6):4–7. 8. Mitchell JK. Geotechnical surprises-or are they? The 2004 H. Bolton Seed lecture. J Geotech and Geoenvir Engrg, 2009; 135(8):998–1010. 9. Bravery P, Cross S, Gallagher R, Hautefeuille O, Reiner H, Spencer M, Smith P, Stolfa A, Wannick H. A Code of Practice for Risk Management of Tunnel Works. Munich: The International Tunnelling Insurance Group, 2006. 28 p. Capturing and Integrating Risk-Related Knowledge 10. Eskesen SD, Tengborg P, Kampmann J, Holst Veicherts, T. Guidelines for tunneling risk management: International Tunneling Association, Working Group No. 2. Tunnelling Underground Space Technol, 2004; 19:217– 237. 11. European Parliament and the Council of the European Union. Directive 2004/54/EC of the European parliament and of the council of 29 April 2004 on minimum safety requirements for tunnels in the Trans-European Road Network. Official J Eur Union, 2004; L 167:39–91. 12. Meng Q, Qu X, Wang X, Yuanita V, Wong SC. Quantitative risk assessment modeling for nonhomogeneous urban road tunnels. Risk Analysis, 2011; 31(3):382–403. 13. Qu X, Meng Q, Yuanita V, Wong YH. Design and implementation of a quantitative risk assessment software tool for Singapore road tunnels. Expert Syst Appl, 2011; 38:13827– 13834. 14. Tah JHM, Carr V. Knowledge-based approach to construction project risk management. J Comput Civil Eng. 2001; 15(3):170–177. 15. De Zoysa S, Russell AD. Structuring of risk information to assist in knowledge based identification of the life cycle risks of civil engineering projects. Pp. 1–10 in Proceedings of the 5th Construction Specialty Conference of the Canadian Society for Civil Engineering. Moncton, Nouveau-Brunswick, Canada: Canadian Society for Civil Engineering, 2003. Report No.: COF-470. 16. Choi HH, Mahadevan S. Construction project risk assessment using existing database and project-specific information. J Constr Eng Manage, 2008; 134(11):894–903. 17. Dikmen I, Birgonul MT, Anac C, Tah JHM, Aouad G. Learning from risks: A tool for post-project risk assessment. Automat Constr, 2008; 18(1):42–50. 18. Tserng HP, Yin SYL, Dzeng RJ, Wou B, Tsai MD, Chen WY. A study of ontology-based risk management framework of construction projects through project life cycle. Automat Constr, 2009; 18(7): 994–1008. 19. Adams FK. Expert elicitation and Bayesian analysis of construction contract risks: An investigation. Constr Manage Econom, 2006; 24(1):81–96. 20. Bles T, Al-Jibouri S, van de Adel J. A risk model for pile foundations. Pp. 421–426 in Maas G, van Gassel F (eds.). Proceedings of the 20th International Symposium on Automation and Robotics in Construction ISARC2003 The Future Site. Eindhoven: IAARC, 2003. 21. Choi HH, Cho HN, Seo JW. Risk assessment methodology for underground construction projects. J Constr Eng Manage, 2004; 130(2):258–272. 22. Goossens LHJ, Cooke RM, Hale AR, Rodic-Wiersma LJ. Fifteen years of expert judgement at TUDelft. Safety Sci, 2008; 46(2):234–244. 23. Anandarajan A, Kleinman G, Palmon D. Novice and expert judgment in the presence of going concern uncertainty. The influence of heuristic biases and other relevant factors. Managerial Audit J, 2008; 23(4):345–366. 24. Cooke RM, Goossens LHJ. Procedures guide for structural expert judgement in accident consequence modelling. Radiat Prot Dosimetry. 2000; 90(3):303–309. 25. Ayyub BM. Elicitation of Expert Opinions for Uncertainty and Risks. 1st. ed. Boca Raton, London, New York, Washington, DC: CRC Press; 2001. 328 p. 26. Garthwaite PH, Kadane JB, O’Hagan A. Statistical methods for eliciting probability distributions. J Am Stat Assoc, 2005; 100(470):680–700. 27. Kynn M. The ‘heuristics and biases’ bias in expert elicitation. J R Stat Soc A 2008; 171(1):239–264. 28. Hallowell MR, Gambatese JA. Qualitative research: Application of the Delphi method to CEM research. J Constr Eng Manage. 2010; 136(1):99–107. 107 29. Pate-Cornell ME. Uncertainties in risk analysis: six levels of treatment. Reliability Engrg Syst Safety, 1996; 54(2–3):95– 111. 30. Zio E. Reliability engineering: Old problems and new challenges. Reliability Engrg Syst Safety, 2009;94(2):125–141. 31. Bedford T, Quigley J, Walls L. Expert elicitation for reliable system design. Stat Sci, 2006; 21(4):428–450. 32. Sigurdsson JH, Walls LA, Quigley JL. Bayesian belief nets for managing expert judgement and modelling reliability. Qual Reliab Engrg Intl, 2001; 17:181–190. 33. Weber P, Medina-Oliva G, Simon C, Iung B. Overview on Bayesian networks applications for dependability, risk analysis and maintenance areas. Engrg Appl Artif Intell, 2010. DOI:10.1016/j.engappai.2010.06.002. 34. Daniels DC, Hudson LD, Mahoney SM, Ware BS, Wright EJ. Terrorism risk management. Pp. 428 in Puorret O, Naim P, Marcot B (eds). Bayesian Networks: A Practical Guide to Applications. 1st ed. Padstow: John Wiley & Sons Ltd, 2008. 35. Taroun A, Yang JB, Lowe D. Construction risk modelling and assessment: Insights from a literature review. The Built Human Environ Rev, 2011; 4(1):87–97. 36. Simon Ch, Weber Ph, Levrat E. Bayesian networks and evidence theory to model complex systems reliability. J Comput, 2007; 2(1):33–43. 37. Dubois D. The role of fuzzy sets in decision sciences: Old techniques and new directions. Fuzzy Sets Syst, 2011; 184:3–28. 38. Novák V. Reasoning about mathematical fuzzy logic and its future. Fuzzy Sets Syst, 2012; 192:25–44. 39. Mamdani EH, Assilian S. An experiment in linguistic synthesis with a fuzzy logic controller. Intl J Man-Machine Stud, 1975; 7(1):1–13. 40. Sugeno M. Industrial Applications of Fuzzy Control. New York: Elsevier Science Inc., 1985. 269 p. 41. Ferson S, Ginzburg L. Different methods are needed to propagate ignorance and variability. Reliability Engrg Syst Safety, 1996; 54:133–144. 42. Baraldi P, Zio E. A combined Monte Carlo and possibilistic approach to uncertainty propagation in event tree analysis. Risk Analysis, 2008; 28(5):1309–1325. 43. Baraldi P, Zio E. A comparison between probabilistic and Dempster-Shafertheory approaches to model uncertainty analysis in the performance assessment of radioactive waste repositories. Risk Analysis, 2010; 30(7):1139–1155. 44. Nasir D, McCabe B, Hartono L. Evaluating risk in construction-schedule model (ERIC-S): Construction schedule risk model. J Constr Eng Manage. 2003; 129(5):518–527. 45. Marzouk M, Eldieb A. Risk assessment in pipeline projects using belief networks and computer simulation. Pp. 3384 in Proceedings of 33rd Annual Conference of the Canadian Society for Civil Engineering 2005: Gateway To Excellence. Toronto, Ontario: Curran Associates Inc., 2007. 46. Edieb A. DSS-PL: Decision support system for risk assessment of pipeline projects. Cost Engrg, 2007; 49(11):20–29. 47. Luu VT, Kim SY, Van Tuan N, Ogunlana O. Quantifying schedule risk in construction projects using Bayesian belief networks. Int J Project Manage, 2009; 27(1):39–50. 48. Bayraktar ME, Hastak M. A decision support system for selecting the optimal contracting strategy in highway work zone projects. Automat Constr, 2009; 18(6):834–843. 49. Bielza C, Gómez M, Shenoy PP. Modeling challenges with influence diagrams: Constructing probability and utility models. Decision Support Syst, 2010; 49(4):354–364. 50. Renooij S. Probability elicitation for belief networks: Issues to consider. Knowledge Engrg Rev, 2001; 16(3):255–269. 51. Clemen RT, Winkler RL. Combining probability distributions from experts in risk analysis. Risk Analysis, 1999; 19(2):187– 203. 52. Woodberry O, Nicholson AN, Korb KB, Pollino C. Parameterising Bayesian networks. Pp. 711–745 in Webb G, Yu X 108 53. 54. 55. 56. 57. 58. 59. 60. 61. 62. (eds). Proceedings of 17th Australian Joint Conference on Artificial Intelligence, 2004, Cairns (Australia). Berlin, Heidelberg: Springer-Verlag, 2005. van der Gaag LC. Bayesian belief networks: Odds and ends. Comput J, 1996; 39(2):97–113. Terés Flores JR. Risk management in Canadian Tunnelling Projects [Master’s thesis]. [Edmonton]: University of Alberta, 2006. 158 p. Visée JV. An adequate method to quantify probabilities: Case studies in the context of early reliability prediction using Bayesian networks restricted to subjective data [master’s thesis]. [Eindhoven]: Techische Universiteit Eindhoven, 2009. 65 p. Dı́ez FJ, Druzdzel MJ. Canonical Probabilistic Models for Knowledge Engineering. Version 0.9 (April 28, 2007), 2007. 59 p. Report No.: CISIAD-06-01. Onisko A, Druzdzel MJ, Wasyluk H. Learning Bayesian network parameters from small DataSets: Application of noisyOR gates. Intl J Approx Reason, 2001; 27(2):165–182. Anagnostou G, Kovári K. Face stability in slurry and EPB shield tunneling. Pp. 453–458 in Mair RJ, Taylor RN (eds). Proceedings of the International Symposium on Geotechnical Aspects of Underground Construction in Soft Ground. London, UK, Rotterdam: Balkema, Apr 15–17, 1996. Guglielmetti V, Grasso P, Mahtab A, Xu S. Mechanized Tunnelling in Urban Areas—Design Methodology and Construction Control. 1st ed. Leiden, The Netherlands: Taylor and Francis/Balkema, 2008. 507 p. Bakker KJ, Bezuijen A. Ten years of bored tunnels in the Netherlands: Part I: Geotechnical issues. Pp. 243–248 in Ng CWW, Huang HW, Liu GB (eds). Proceedings of the 6th International Symposium on Geotechnical Aspects of Underground Construction in Soft Ground (IS-Shanghai 2008); 2008 Apr 10–12; Shanghai (China). Leiden, The Netherlands: CRC Press/Balkema, 2009. Bezuijen A, Talmon AM. Processes around a TBM. Pp. 3–14 in Ng CWW, Huang HW, Liu GB (eds). Proceedings of the 6th International Symposium on Geotechnical Aspects of Underground Construction in Soft Ground (IS-Shanghai 2008); 2008 Apr 10–12; Shanghai (China). Leiden, The Netherlands: CRC Press/Balkema, 2009. Broere, W. Influence of excess pore pressures on the stability of the tunnel face. Pp. 759–765 in Saveur J (ed). (Re)Claiming Cárdenas et al. 63. 64. 65. 66. 67. 68. 69. 70. 71. 72. the underground space, Amsterdam (The Netherlands). Lisse, The Netherlands: Balkema, 2003. Aven T. Perspectives on risk in a decision-making context— review and discussion. Safety Sci, 2009; 47(10): 1355– 1364. Abt E, Rodricks JV, Levy JI, Zeise L, Burke TA. Science and decisions: Advancing risk assessment. Risk Analysis, 2010; 30(7):1028–1036. Dubois D, Guyonnet D. Risk-informed decision-making in the presence of epistemic uncertainty. Intl J Gen Syst, 2011; 40(2):145–167. Aven T, Zio E. Some considerations on the treatment of uncertainties in risk assessment for practical decision making. Reliab Engrg Syst Safety, 2011; 96(1):64–74. Apostolakis G. A commentary on model uncertainty. In: Mosleh A, Siu N, Smidts C, Lui C (eds), Proceedings of Workshop on Model Uncertainty: Its Characterization and Quantification, 1993 Oct 20–22; Annapolis (MD). Washington, DC: US Nuclear Regulatory Commission, 1993. Report No.: NUREG/CP-0138. Winkler R. Model uncertainty: Probabilities for models? In: Mosleh A, Siu N, Smidts C, Lui C (eds), Proceedings of Workshop on Model Uncertainty: Its Characterization and Quantification, 1993 Oct 20–22, Annapolis (MD). Washington, DC: US Nuclear Regulatory Commission, 1993. Report No.: NUREG/CP-0138. Keeney R, von Winterfeldt D. Eliciting probabilities from experts in complex technical problems. IEEE Trans Eng Manage, 1991; 38(3):191–201. Little MP, Muirhead CR, Goossens LHJ, Kraan BCP, Cooke RM, Harper FT, Hora SC. Probabilistic accident consequence uncertainty analysis. Washington, DC: US Nuclear Regulatory Commission and the Commission of European Communities, 1997. 62 p. Report No.: NUREG/CR-6555 and NUREG/CR6523. Budnitz R, Apostolakis G, Boore D, Cluff L, Coppersmith K, Cornell C, Morris P. Recommendations for probabilistic seismic hazard analysis: Guidance on uncertainty and the use of experts. Washington, DC: US Nuclear Regulatory Commission, 1997. 888 p. Report No.: NUREG/CR-6372. Halman JIM, Keizer JA. Diagnosing risks in productinnovation projects. Intl J Project Manage, 1994;12(2):75– 80.