Accuracy in QRA
Accuracy in QRA
Accuracy in QRA
Abstract
This paper describes the results of a study stretching over 25 years, to determine how
accurate the risk assessment process can be made for process plants. During that time, risk
assessment has become a standard working tool for safety engineering, not just in land use
planning, but in design of safety shutdown and interlock systems, gas detection systems,
building blast resistance, and fire protection systems. It becomes highly desirable that QRA’s
should be just as accurate as calculations in other areas of engineering.
The study used several methods of approach including QRA followed by a 20 year serial
study of a number of plants; comparison of results from several methodologies and a large
number of models against each other; comparison with actual accident reports and statistics;
and third party reviews of QRA’s.
The main results are that while models have been developed which agree very well with
experiment, they are not always relevant to real life; that several very important accident
types are generally ignored; that the accuracy within the plant and especially at short
distances is much poorer than the accuracy at a distance; that the calculation methodologies
most commonly used introduce unnecessary additional errors; and most importantly, that
most of the deficiencies in current practice can be fairly easily corrected.
Acknowledgements
The authoer gratefully acknowledges the help in completing this study from Herlinde
Beerens, Y Weber, Abid Sayyed, Tamir Said, Shastri Ranjiputri
The most important property of any practical application of scientific theory is that the results
should agree with observation. Secondarily, the results should be repeatable.
Until a few years ago, even repeatability in QRA was not achievable to better than two orders
of magnitude (ref. 1). This was a saddening result, considering that QRA methodology has
been under development for over 30 years. The cooperative study organised by RIVM and
published in 2004 (ref. 2) showed that experienced analysts, using the same data, and a
carefully made guideline, could produce result in agreement to within a factor of 2, even
when using different software packages. This was encouraging in that it showed that
consistency, at least, can be achieved.
Even at the best, though, such calculations can only be “accurate on average”, that is, they
are accurate for an average plant. If you have an average plant, then you are in luck.
The data that are used in practice today for frequency determination are largely drawn from
just a few sources, and to a considerable extent, from the North Sea oil and gas production
systems (e. g. ref. 3). Others are often based on the original engineering judgements in the
Rijnmond study (ref. 4, with updates through the years, ref.5). Given that actual failure rates
for some items of equipment have vary between different plant types by up to three orders of
magnitude (ref. 6), this means that “accuracy on average” may be very inaccurate. An
example of this is the urea reactor on older urea plants. The historical frequency of vessel
rupture is about 2.6*10-3 per year (based on US experience), whereas generic vessel
frequencies in tables are of 5*10-5 to 5*10-7 per year i.e. a factor of 1000 inaccuracy.
To get over this problem, the RELBASE release frequency data base was developed (ref. 6)
this is a very extensive collection of data from in all about 2,500 plants of different types. It
allows plant specific release frequency values to be used, with dependency on technology,
feasible causes, and process fluid. It does not, so far, allow process safety management
variations to be taken into account.
Plant specific frequencies of major accidents derived using this approach were compared
with actual major hazard accident frequencies, for those plant types for which sufficient data
are available. Comparisons of prediction and observed frequencies are shown in Table 1,
with accident frequencies taken from ref. 7, and also derived as part of this project.
Plant specific data may be undesirable in regulatory circumstances, such as in land use
planning, because of the complexity of deriving fairly precise values. However, at least the
large variations arising from different technologies and different process fluids should be
taken into account. It simply does not make sense to use the same values for failure of piping
for the case of simple carbon steel and for 360 SS steel for example.
Note that the issue here is not uncertainty in failure rate values. The generic values used are
simply wrong for many cases, and this error is independent of any statistical uncertainty.
Looking to the future, many companies are already using risk based inspection as a tool, and
this is already generating a lot of plant specific failure rate data.
It is generally agreed that human error is one of the most important causes of major
accidents. This was confirmed from the accident reviews made as part of the present study.
Yet curiously, risk analysts working in the process industries spend large amounts of time
counting pipes, flanges and valves, and generally none at all in counting opportunities for
human error. By contrast, HRA is regarded as a standard part of nuclear power plant risk
assessments.
The kinds of operator and maintenance errors which appear as causes of major hazards
accidents are a fairly small set (ref. 8). It is possible to determine the frequency on a
statistical basis, given the number of accident reports available. The US RMP data which
was used as part of the input to the RELBASE database includes a significant percentage of
operator and maintenance related accidents. Alternatively human reliability analysis methods
can be used for analysis of the typical tasks which can give releases. The task risk analyses
(TRA’s or JSA’s) carried out by many companies provide a useful basis for such analyses.
It has been argued that release frequency data used in QRA’s already include an element of
human error. Examination of the sources of data used in commercial risk analyses show that
this is not actually the case. In any case, operator and maintenance errors often give
consequence which are quite different from leaks and pipe ruptures recorded in QRA
reference databases.
The impact of including human error into petroleum and chemical plant QRA have not so far
been investigated as part of the project.
By far the biggest human error contributions to major hazards accidents are design error and
management error (ref. 9). Accounting for these presents a philosophical problem – if you
can predict these errors it is best to remove them, not waste time calculate their frequency.
We did not find any good way out of this deadlock so far. Some possibilities for investigation
are described in ref. 9.
Consequence modelling
In order to assess the importance of modelling on QRA results, several approaches were
used:
The results from this exercise were encouraging. At distances over 100 to 200 m., many of
the models agreed with experiment to within a factor of 2 or better. The least consistent of
the models were those for which turbulence is important, i.e. gas dispersion and explosion,
but agreement with experiment to better than a factor 2 in 90% of cases seemed to be the
usual case.
Agreement at short distances, as needed for in-plant risk assessment for hazards to
employees and for domino effects, were less good. There were largely due to the common
use of oversimplified or inappropriate models. Examples were:
• Modelling jet fires as simple straight flames (in fact, horizontally and obliquely directed
flames always bend upwards)
• Neglect of flame drag and flame dip in tank fire and pool fire models.
• Use of a single surface radiation intensity for pool and jet fires, rater than varying
intensity
• Neglect of gas jet impingement
• Neglect of cross wind dispersion
• Assumption that all liquid releases occur on flat, non absorbent ground (in fact most
chemical plants have a slope of about 2˚, which has a very large effect on pool size.
Reduction in pool size is the prime purpose of the slope.)
Table 2 shows examples of accidents with their coverage in standard QRA methodologies.
Models for all of these effects were found in the literature. Especially the model reviews
performed by Deaves, Rew et al. for the UK HSE proved more accurate at short ranges
(refs.12, for example). (Perhaps not surprising since the work is relatively recent and could
draw on extensive earlier work).Omission of these effects was found to be important in many
cases. Most of the simplifications of the models result in underestimate of risk, so the
cumulative effects are important.
One perhaps surprising feature of modelling which became obvious from the comparisons
with actual accidents was that hazard zones for all pool fires, flash fires and explosions are
calculated as circles in commercial QRA packages. Not all accidents are circular. Examples
of accidents with extended linear geometries are the fire at Bellingham (ref. 13), Cubatao
(ref. 14), and Buncefield (ref.15), and San Rafael de la Laya.
One problem that requires considerable further research was identified from the comparisons
of models with accident data. That is the prediction of unconfined vapour cloud explosion
pressures. Flame acceleration is the key problem, and many different mechanisms for this
have been identified, only one of which has well researched full scale experimental support.
Many of the phenomena which occur in practice in process plant are not included at all in
standard methodologies and model sets. Examples are:
Models were available for all of these except fire induced tank explosion, which had to be
developed. The error induced by these lacunae varied from a significant percentage to a very
large factor, depending on the case.
It could be argued that the omitted scenarios are exceptional cases, which could be
calculated separately and incorporated into QRA’s. However, it appears especially that
spraying and splashing releases are the norm for liquids, rather than exceptions. A review
was made of major hazard accidents investigated by the US Chemical Safety Board and the
UK HSE. For petrochemical plant, less than 20% would be calculated by standard model
sets such as the Yellow and Green Books, or by commercial QRA calculation tools. For the
full range of major hazard accidents, only a small percentage would be calculated using
standard methodologies and tools, see table 2.
Choice of methodology
Methodologies for QRA specify which scenarios should be calculated, and often also many
of the parameters which should be used. Typical methodologies in use today are the World
Bank guideline(ref. 18), the Dutch purple Book (ref. 5) and the CCPS guideline (ref.19). For
this study, characteristics of methodologies were investigated in order to determine the effect
of methodology choice on results (ref. 21). The parameters investigated were:
• Choice of scenarios, just leaks and spontaneous ruptures, or a full range of scenarios
which could be identified by hazop studies such as overpressuring, overflow, confined
explosion etc. To keep the analyses comparable, the lacunae scenarios described in
the previous section were not included.
• The range of hole sizes considered (two, three, four or five, as provided for in
different methodologies. To support this study, an analysis of sizes for several
hundred cases was made).
• The location of holes along pipes (one representative location, two, three or four)
• The selection of vessels to be analysed, and whether each vessel should be
analysed, or groupings of vessels in an isolatable section.
• Whether to include domino effects at all, to include just calculation of the frequency of
domino effects, or the demanding calculation of both the sizes and frequency of
domino effects.
• Whether to include mitigation calculations in the methodology.
Conclusions
Quantitative risk assessment has become an important tool in plant safety engineering.
Earlier guidelines such as refs. 8 and 18 have provided an open and well researched basis
for the use of QRA. These methodologies, however, were developed during the 1990’s for
land use planning. The present study indicates that accuracy can be much improved by
using more recent models, using more thorough methodologies, and particularly, increasing
the scope of accident scenarios covered.
Can this make the analyses truly accurate, to the same level as other engineering
disciplines? It appears difficult to improve models for gas dispersion much until better
experimental results become available, which will need better instrumentation that used until
now; or possibly, improvements in CFD techniques will allow a priori calculation of
dispersion. Similar problems arise for explosion calculations. For now though, “factor of two
accuracy” seems the best to be achievable for most consequence calculations.
It also appears possible to improve frequency calculations, though factor of two accuracy
seems difficult to guarantee except under the best circumstances. Factor of five accuracy
seems achievable in a wide range of cases. Uncertainty bands in results have also narrowed
considerably as more data have become available.
Is accuracy really necessary? Could it be just accepted that repeatability and consistency is
sufficient, as is the case for regulatory models? After all, the criteria for risk acceptance can
always be adjusted to take into account an estimated factor of error. The main problem with
this approach is that it does not provide a good basis for risk reduction and ALARP analysis.
Also, the regulatory model approach is very inflexible, which means that it will lead to hazard
types being neglected unless the model set is complete. The most important improvement is
to make sure that the accident scenarios calculated are the ones which occur in practice.
The price for improvement in accuracy is that more thorough assessments are required. The
additional cost of calculations themselves is not a significant factor. The collection of data as
a basis for the calculations, including process parameters for all vessels containing
hazardous materials in a plant, however, represent a significant, though manageable effort.
References