Project Risk Simulation Methods - A Comparative Analysis: Abstract. Effective Risk Management
Project Risk Simulation Methods - A Comparative Analysis: Abstract. Effective Risk Management
Project Risk Simulation Methods - A Comparative Analysis: Abstract. Effective Risk Management
Constana-Nicoleta BODEA
Academy of Economic Studies, Bucharest
6 Roman Square, 1st district, Bucharest,
Romania
e-mail: bodea@ase.ro
Augustin PURNU
Technical University of Civil Engineering
124 Lacul Tei Blvd., Bucharest, Romania
e-mail: purnus@utcb.ro
1. Introduction
While the financial and economical crisis is present at the global level and the
competition in the market is more and more aggressive, the interest in risk
management increases (Hulett, 2009; Kendrick, 2003; Pritchard, 2011). Effective risk
management provides a solid basis for decision-making in projects, bringing important
benefits, such as: reduced costs, increased engagement with stakeholders and better
change management (Bayati, Gharabaghi and Ebrahimi, 2011). Different project risks
management approaches were defined in standards and guidelines such as:
ICB v3.0, International Competence Baseline, International Project
Management Association (IPMA, 2006);
AS/NZS 4360, Risk Management, Standards Australia/Standards New
Zealand (AS/NZS, 2004);
ISO 21500, Guidance on Project Management, STD v2, International
Organization for Standardization (ISO, 2012);
PMBOK Guide, Guide to the Project Management Body of Knowledge
(PMI, 2008);
SDPM, Success Driven Project Management, Spider Project Team
(Liberzon and Lobanov, 2000).
The project management standards and guidelines recommend similar
frameworks for project risk management. What it is really relevant for differentiating
between risk management approaches is not the process structure as such, but how the
integration of the risk management with all other project management processes is
reinforced. Chapman and Ward (2002) identify a lot of limitations and errors arising
when the risks are managed in projects, even if the project manager has considerable
experience in managing risks. The principal shortcomings relived by Chapman and
Ward are the following: the initial activities for managing risks are too detailed and
fails to underlay the connections with different projects elements in a balanced
manner; the risk identification fails to provide a good structure of the sources of
uncertainty, and to identify significant linkages and interdependences between issues;
the risk estimates are highly dependent on the project scope hypothesis and not on
other kind of assumptions; the risk estimations are costly, but not cost-effective; the
risk evaluation fails to combine properly different source of uncertainty because
crucial dependences are not captured and a weak implementation of the project plan.
The majority of these limitations are linked to the integration of the risk management
processes with each other and with the project management processes in large.
What it is considered by many professionals as being critical for an effective
risk management in projects it is the risk assessment (Bayati, Gharabaghi and
Ebrahimi, 2011; Makait, 2011; Andersen, 2011). If we accept that risk management is
all about identifying, measuring and minimizing uncertain events effecting projects,
then the secret for a good risk management lies in the ability to quality and quantify
the risk elements. This is why these professionals consider the qualitative and
quantitative project risk analysis as the core processes in risk management.
566
The project quantitative risk analysis is considered as the hardest part of the
risk management, because it is based on advanced statistics and mathematics methods.
A lot of deterministic and probabilistic methods were developed over time and made
available, especially through software implementation. But these methods are usually
not properly applied, or not applied at all. The main reasons for this are lake of
expertise, difficulties in collecting historical data, complexity of risk quantification
methods and also the computation effort. Only for a few project types, such as:
research and development projects and public and military capital Investments in
projects, the risk quantification is regularly performed.
In the absence of easy to use tools and techniques for project risk
quantification, most of the project team applies exclusively risk qualitative analysis. A
study conducted by the Chartered Institute of Building between December 2007 and
January 2008 highlights the fact that despite the development of sophisticated tools
and techniques, a large number of construction projects are delayed and over budget
and because the project quantitative risk analysis is missing (CIOB, 2008). The more
complex the projects are, the less likely it is to achieve the success.
2. Project quantitative risk analysis methods
The experience gained in a large number of projects reveals the fact that using
only deterministic methods of scheduling leads to a low probability of success. For this
reason project scheduling and monitoring must always include techniques for risk
simulation in order to obtain feasible results. Simulation is increasingly applied in
business. As a project management method, simulation depends on two essential
elements: a model for defining the project outcomes and outcome values and a
technique that repeatedly generates scenarios (Schyler, 2001). Variables whose values
are not known with certainty, but can be described by probability distributions are
called stochastic variables.
In the simulation, in order to emulate the variability of such variables is
necessary to generate possible values based on its distribution probability. The
information about probabilities is necessary both for building the simulation model and
for analyzing the simulation results. To construct the probability distribution of a
variable we can apply the following procedure:
Collect data on values of the stochastic variable;
Group values into intervals and develop the histogram of the relative
frequencies;
Analyze of the relative frequency histogram graph to determine whether a
shape resembling theoretical distributions known. The probability distribution type
can be appreciated using correlation tests, such as: Kolmogorov, Smirnov, Pearson (or
2), measuring the closeness between the theoretical and probabilistic distribution of
values obtained from historical data or expert estimations. Finally, the distribution
parameters are calculated.
The risk simulation methods can be semi-probabilistic and probabilistic ones.
PERT (Program Evaluation and Review Technique), originally developed in the late
567
50s is one of the first project planning approach addressing the project risks. PERT
takes into consideration the uncertainties using the three points estimation method.
Basing on their experience and the historical information the project team estimates
for each activity three durations: optimistic, pessimistic and most probable. The
probability to meet a project target date (time or cost) is computed considering only
the activities on the critical path. Different results in terms of accuracy may be
obtained if we use different probability distribution curves. It may be used Beta,
Normal or Triangular probability distribution. The critics of PERT are based mainly
on the hypothesis included in the method, which leads to errors in results, which are
considered optimistic in comparison with other methods.
By the 1960s, Monte Carlo simulation is embedded in PERT, in order to avoid
assuming that only one path may be critical and that the probability distribution of the
project duration must be normal. In the same time, GERT (Graphical Evaluation and
Review Technique) is defined, based on the decision trees embedded in Markov
processes. GERT enhances the project managers ability to understand how the project
is affected by the corrective actions, considered as repetitive processes, executed in a
specific timeframe window. In 1970s, Chapman developed SCERT (Synergistic
Contingency Planning and Review Technique) approach, based on the fault tree and
event tree concepts, for safety analysis (Chapman and Ward, 2003).
The next two paragraphs will shortly introduce Monte Carlo simulation and
three-scenario approach. In the final part of the paper, we will compare the
effectiveness of these two methods, by means of two experiments designed and ran by
the authors.
3. Monte Carlo simulation
Recognized by the accuracy of its results, Monte Carlo method is part of the
probabilistic methods used in risk simulation. The Monte Carlo method first generates
artificial variable values, using a random number generator uniformly distributed in
the interval [0, 1] and the associated cumulative distribution function. Then, the Monte
Carlo method uses the obtained results to extract values from the probability
distribution that describes the behavior of the stochastic variable.
3.1. Monte Carlo simulation with discrete stochastic variable
For discrete stochastic variables, the list of possible values and the
corresponding probabilities form a discrete probability distribution. In the terminology
of probability theory, one can note the stochastic variable as X, xi being a particular
value of the variable X. The probability that the value of a variable X equals to xi is
denoted as P(X = xi) = P(xi). The probability that the value of variable X exceeds a
certain value xi is called the cumulative distribution function and it is denoted as F (xi).
The most common theoretical discrete probability distributions are the discrete
uniform distribution, the binomial distribution and the Poisson distribution.
568
The
cumulative
distribution
function
is
defined
as:
xi
xi
Step 2. The random number intervals are associated to each value of to variable
discrete. This can be graphic or tabular.
Step 3. Generate a random number u uniformly distributed in the interval [0, 1]
using a random number generator.
Step 4. Calculate xi, the value of discrete stochastic variable. This can be done
graphic or tabular. Graphically, function F(xi) has a specific shape on x0y
coordinate. We can set the y value (the probability) and we can found the
stochastic variable value on x coordinate.
Steps 1-4 is being repeated until a predetermined number of trials are
completed.
569
exp
2
function of the normal distribution cannot be integrated exactly, it is not used directly.
Calculation of areas needed to determine the probabilities P(a x b) and the
570
571
risk response plans. As result, there will be developed not one, but three scenarios of
the same project schedule and budgets:
Optimistic Scenario includes the risk events with a probability exceeding
90%;
Most Probable Scenario is based on the most probable estimates and
includes the risk events with a probability exceeding 50%;
Pessimistic Scenario is based on the pessimistic estimates and includes all
selected risks.
Basing on the risk response plans, the most probable and pessimistic
scenarios will include additional activities, work volumes, durations, resources,
productivities, costs, calendars, financing and supply, other than the optimistic
scenario. The calculated scenarios with resource leveling and project budgeting are
used to rebuild the probability curves (for dates, costs, and material requirements)
with a predefined shape of probability curve (Figure 1). Defining the desired target
probabilities will allow us to obtain the desired dates for finishing the project, costs
and material requirements (Liberzon and Archibald, 2003). The project target
probabilities are usually defined by the organization risk tolerance, but for regular
projects, it should be in the range of 65%-75%. The three scenarios will establish the
buffers (for time, costs and materials) which will be available for the project
manager during the execution phase.
The probabilities to meet the target dates are called success probabilities and
they are used to measure the buffer penetration. If the success probability trend is
positive, than the buffer consumption was lower than expected, otherwise the buffer
consumption was higher than expected and are required corrective actions (Figure 2).
That means we will not focus on the activities total floats as it is in the actual approach
of project management, but on how the buffers are consumed during the execution
phase, as result of risk events.
The success probability trends measure and show not only project
performance but also project health taking into consideration both internal and
572
external factors. The main advantage of three scenarios approach is that it can be
applied easily in medium and large projects.
Probability Distribution Curve. Having the three scenarios we know the range
of most frequent values for project duration, total cost, material requirements, resources
etc. In order to build the probability distribution curve we rely on the three points with
the probabilities according to the three scenarios: the point with zero probability for the
optimistic scenario, the point with 100 % probability for the pessimistic scenario and the
point for which the probability distribution has the maximum value (Figure 3). What we
dont know is the shape of the probability distribution curve. This can be Beta, Normal
or Triangular. In fact, in the Three Scenario approach method the shape of the
probability distribution curve is not so important. We know from the very beginning that
the results are not accurate so they are not close to the exact value, but successive
measurements will give results very close each other. That means the results are
characterized by precision. During the project execution the same shape of the
probability distribution curve will be used. In this way, even the shape is not the correct
one; we will be able to establish if the probability will become greater or smaller as a
result of risk events or the application of preventive or corrective measures (Liberzon
and Souza Mello, 2011). To exemplify the probability computation for a certain target
date we will use the triangular distribution (Figure 4). The probability to achieve the
target date is represented by the area under the probability distribution curve at the left
side of the target date. P =
s
, where: P is the probability to achieve the target date; s
S
the area between the target date and the probabilistic date and S the total area under the
probability distribution curve.
573
will be: X = 1
(P T )2 .
(P O ) ( P M )
optimistic duration is 15 days, the most probable duration is 20 days and the
pessimistic duration is 29 days, presented in Figure 5.
If we define the desired target date as being the 23 days duration, the
probability to achieve it based on the triangular distribution curve will be 71.43%.
During the project execution the optimistic, most probable and pessimistic scenarios
durations will change. For each update stage we will compute the current probability
to achieve the target date, in our example, 23 days. The results of optimistic, most
probable and pessimistic scenarios duration and the current probability to achieve the
target date are presented in Table 1.
Table 1
Duration for optimistic, most probable and pessimistic scenarios and the current
probability to achieve the target date
Optimistic
duration
Most probable
duration
Pessimistic
duration
Target
data
Current
probability
15
20
29
23
71.43%
13.9
15.9
21
23
88.95%
12.25
14.25
20.5
23
87.88%
11.7
13.7
20.5
23
89.56%
11.15
13.15
19.75
23
81.39%
10.6
12.6
19.5
23
80.05%
10.05
12.05
19
23
74.28%
9.5
11.5
18.5
23
67.86%
574
575
project costs were modeled using 11 cost components grouped into 5 cost centers.
Three scenarios (optimistic, most probable and pessimistic) were developed taking
into account the identified risk. Different productivities, material consumption,
activity calendars, resource team's structures and costs resulted from the risk response
plan. The target dates were established for project duration and cost as following: the
target duration is 182 days and the target total c is 465.000 Euros.
Figure 8. Probability curves in (a) Monte Carlo and (b) Three Scenario approach for
parameter project duration
Figure 9. Probability curves in (a) Monte Carlo and (b) Three Scenario approach for
parameter project total cost
576
Using the Monte Carlo method, the probabilities obtained are higher than the
probabilities obtained using Three Scenario approach. The computation duration is
also much larger than in the Three Scenario approach (Table 2).
Table 2
Probabilities to achieve the target date with Monte Carlo and Three Scenario
approach
Method
Monte Carlo
Three Scenario Approach
Parameter
duration
88.30%
84.90%
Parameter
total cost
99.90%
71.90%
Computation duration
(seconds)
675
3
577
Figure 11. Probability curves in (a) Monte Carlo and (b) Three Scenario approach
for parameter project duration
Figure 12. Probability curves in (a) Monte Carlo and (b) Three Scenario approach
for parameter project total cost
Similar to the first project, the probabilities to achieve the target dates are
higher in Monte Carlo method than in Three Scenario approach (Table 3).
Table 3
Probabilities to achieve the target date with Monte Carlo and Three Scenario
approach
Method
Monte Carlo
Three Scenario
Approach
Duration parameter
99.30%
75.30%
Total cost
parameter
99.90%
64.20%
Computation duration
(seconds)
321
3
578
6. Conclusions
Project risk management represents a significant area in project management.
The authors analyzed several risk management standards and guidelines and
concluded that what it is really relevant for differentiating risk management
approaches is not the process structure as such, but how the integration of risk
management with all other project management processes is reinforced. The authors
focused on risk simulation methods recommended and used in project quantitative risk
analysis, investigating theoretical and in practice two methods, Monte Carlo and the
Three Scenario approach. In order to compare the effectiveness of the methods, two
experiments were designed based on real projects.
Even if the Monte Carlo method is very accurate, its practical application is
not feasible because of the iterations, which require effort and time for computation.
The application of Monte Carlo needs a great amount of time for the preparation of
input data when the projects are of medium and large size, with hundreds and
thousands activities, resources, materials and cost components. As time as the project
environment is in continuous changing the accuracy of the results using Monte Carlo
method does not help the project managers too much in decision making. Though the
Three Scenario approach is a semi-probabilistic method and it is not as accurate as
Monte Carlo, the use of the same shape of the probability distribution curve gives it
stability in results. It is easy to be applied in practice and requires a very short time for
computation. Time and cost buffers are set by defining reliable targets that have
reasonable probabilities to meet.
The application of management by trends of the probability to achieve the
target dates is in fact the management of buffers penetration allows project managers
to identify timely the risk events and to react properly. Trends of probabilities to meet
project targets (success probabilities) are most valuable and integrated project
performance indicators. They depend not only on project performance but also on
project environment.
References
Andersen, D.L. (2011), Project Risk management Inside Out: How to cope with complexity
generated risk, Future Trends in Project, Programme and Portfolio Management,
Proceedings International Expert Seminar, IPMA, Zrich
Archibald, R., Liberzon, V. and de Souza Mello, B.P. (2008), The Application of Success
Probabilities, Success Driven Project Management/SDPM, and Some Critical Chain Concepts
to the Oil & Gas Industry in Brazil, The Annual PMI College of Scheduling Conference,
Chicago
AS/NZS (2004). AS/NZS 4360: Risk Management, Standards Australia/Standards New
Zealand, available at: http://www.ucop.edu/riskmgt/erm/documents/as_stdrds4360_2004.pdf
Bayati, A., Gharabaghi, A.A. and Ebrahimi, M. (2011), Practical risk management in
petrochemical industries: an NPC case study, Future Trends in Project, Programme and
Portfolio Management, Proceedings International Expert Seminar in Zrich, IPMA
579
Software, The 3 PMI Europe Conference Proceedings, Jerusalem, Israel, 12-14 June
Makait, T. (2011), Qualitative and quantitative: EPC Project Risk Management is no gamble,
Future Trends in Project, Programme and Portfolio Management, Proceedings International
Expert Seminar in Zrich, IPMA
PMI (2008), PMBOK Guide: A Guide to the Project Management Body of Knowledge,
Newtown, PA: Project Management Institute
Pritchard, C.L. (2011), Risk Management-Concept and Guidance, 2nd edition, ESI International
Schyler, J. (2001), Risk and Decision Analysis in Projects, 2nd edition, PMI Inc
580