ORX Exploring Risk Exposure Methodologies 2020 Summary Report
ORX Exploring Risk Exposure Methodologies 2020 Summary Report
ORX Exploring Risk Exposure Methodologies 2020 Summary Report
March 2021
Exploring
Risk Exposure
Methodologies
summary report
Structured methods for operational risk analysis
Become a member
of the world's 100 + research reports
and papers
leading operational
risk management
association 100 +
member
19
years’
firms experience
Joining ORX gives you access to
operational risk loss data, research
studies and initiatives and a global
network of over 100 financial firms. 800k + loss events in
our database
For more resources and To find out how ORX Membership will enhance
information on operational operational risk management at your firm visit
risk visit www.orx.org managingrisktogether.orx.org/orx-membership
Exploring Risk Exposure Methodologies
Contents
4 Executive summary
Area of op risk:
Risk measurement
5 Introduction
The Exploring Risk Exposure Methodologies study is
part of the ‘Risk Measurement’ area of our world-leading
6 Structured scenarios research programme, where we cover a wide range of
topics, including capital calculation and methodology,
8 Focus of this pilot models and stress testing. Taking part in and accessing
the full outputs from research is free to members. Other
8
firms can benefit for a small fee.
Bayesian networks for operational risk management
For more information about ORX research, visit
13 Pandemic structure
16
Disclaimer: ORX has prepared this document with care and attention.
Vendor failure structure ORX does not accept responsibility for any errors or omissions.
ORX does not warrant the accuracy of the advice, statement or
19
recommendations in this document. ORX shall not be liable for any
Conclusion loss, expense, damage or claim arising from this document. The
content of this document does not itself constitute a contractual
20
agreement, and ORX accepts no obligation associated with this
Report contacts document except as expressly agreed in writing. ©ORX 2021
3
Exploring Risk Exposure Methodologies
Executive summary
Much of operational risk quantification relies on two techniques – loss distribution
approaches (LDAs) and scenario analysis.
Each have their own merits and areas of application: LDAs The risk drivers relevant to a specific risk are typically not
are useful for familiar risk types for which sufficient data idiosyncratic as they relate to the fundamental processes
is available, while scenarios can be used to assess new or that give rise to operational risk events. Rather, the specific
emerging risks, or for which data may be sparse. values (or distributions) that these factors may take vary
across companies, as well as the strength of controls which
A third approach, which in some way tries to build a may mitigate their impact. As a result, the approach also
bridge between these two, are structured scenarios�. provides opportunities for industry collaboration to agree
The assumption behind these approaches is that the on the sets of relevant factors, and the mechanism by which
underlying exposure an institution has to a specific risk can they interact and lead to the exposure to risks.
be described by a set of factors. These variables drive the
frequency and severity of loss events, and by identifying and As well as banks and insurers from our membership, this
understanding them, you can both measure the exposure to collaboration also included the specialist consultancy,
a risk and know how best to control and mitigate it. Elseware3, who have extensive experience in creating
structure scenarios for operational risk. Structured scenarios
This paper reports on an industry initiative that ran require investment of time and expertise to create, but
throughout 2020 to create structured scenarios for by taking an industry approach, this burden is shared and
pandemic and vendor failure risks. The initiative used expertise is pooled, leading to a more efficient and robust
Bayesian networks2 as the modelling approach, but other outcome.
methods could also be applicable.
This report presents the results from the pilot project, which
By using Bayesian networks, the structured scenarios are aimed to identify the most important risk drivers for two
given a conceptual framework for gaining deeper insight very topical scenarios – Pandemic and Vendor Failure. It
into risk drivers and how they relate to each other. The provides background information on structured scenarios
relationships can easily be visualised and serve as basis for and Bayesian networks, before going into more detail on the
internal conversations involving a variety of stakeholders model structures that were built for the two scenarios.
– thereby bringing together risk modellers, risk managers,
and businesses, each of whom have much to gain from an The results are two models, which will deepen the
accessible basis for informed conversations on particular understanding of each of these risks and promote structured
risks. scenarios as a way of performing risk measurement.
1 Commonly also referred to as factor models or exposure models | 2 “Pearl, J. (1988) Probabilistic Reasoning in Intelligent Systems. San Francisco, CA: Elsevier | 3 https://www.elseware.fr/ 4
Exploring Risk Exposure Methodologies
Introduction
In early 2020, ORX launched a new initiative to explore the use of structured
scenarios for operational risk measurement, specifically structured scenarios based
on Bayesian networks.
The aim of the pilot project was to discover how financial • Can the industry work together to create a factor
institutions are using structured scenarios and gauge driven approach to measure operational risk that more
the interest among the ORX membership in joining an adequately reflects the organisational risk profile?
industry community to explore these methods and share
expertise. • What can ORX do to support members that are using,
or plan to implement, such an approach?
Structured scenarios using Bayesian networks are built
on risk drivers corresponding to both firm-specific and This meant exploring whether we could, together with a
external variables describing the environment. They are group of industry representatives, create a set of basic
developed using subject matter input to build structures models or structures4, which individual firms can use and
that capture loss generating processes that reflect adapt for their own purposes. The intention was to focus
the organisational risk profile to specific risk types and on two scenarios to explore the feasibility and interest in
scenarios. this kind of collaboration.
As they are data driven, they can be seen as building If successful, the pilot study would serve as a road map to
a bridge between conventional scenarios and purely create additional structures for a suite of scenarios that
quantitative approaches to measure operational risk ORX member institutions could benefit from.
exposure. They, therefore, promise to provide a more The project was run in collaboration with Elseware5, a
nuanced view of operational risk profile. Nevertheless, specialist consultancy with experience in structured
while structured approaches have been in use for a scenarios based on Bayesian networks, who provided
while and are increasing in popularity, they have yet to input and guidance during the project.
become a widely adopted approach for operational risk
quantification. It was also overseen by a steering group of 13 member
institutions, who were responsible for the general
The two central questions that the pilot aimed to explore direction of the project and provided input and feedback
were: on the developed structures.
6 https://www.bis.org/bcbs/publ/d508.htm 6
Exploring Risk Exposure Methodologies
Quantitative/objective Qualitative/subjective
Structured • Buidling a bridge between the two • Require more expertise • Are useful to • Allow for the inclusion
approaches: More use of data and time during inital assess rare events of controls and
scenarios and SME input where necessary development mitigation efforts
Structured scenarios typically require more effort and resources for their initial development but promise important benefits for risk management.
A key benefit is that, as they draw from firm Understanding the drivers of both well-known and lesser-
and business data, the structures can more known risks provides an understanding of exposure
accurately reflect organisational risk profiles to each, and a starting point for mitigating action.
Where this data is continuously monitored, this allows Where risk drivers are used across different structures
for the simple, regular updating of the model output, in the scenario suite, the creation of a risk driver
providing an up-to-date view of the risk profile. library provides an understanding of how individual
risk drivers create a dependence between risks.
7
Exploring Risk Exposure Methodologies
7 See for example Alexander, C. (2000). 'Bayesian Methods for Measuring Operational Risk.' Discussion
Papers in Finance 2000-02. The University of Reading. Reading: The Business School for Financial Markets.
8
Exploring Risk Exposure Methodologies
The method makes it feasible to derive a set of conditional distributions by greatly simplifying
calculations where relationship, or edges, do not exist. If two variables are not linked, it is implied
that they are conditionally independent of each other, i.e. the value of one variable does not
influence the value of the other.
9
Exploring Risk Exposure Methodologies
• The occurrence variable defines the probability that an event may occur Control variables and stressed variables
and includes any trigger variables that lead to an event. The exposure variable
A key advantage of structured scenarios over LDAs is that they provide a straight-
and the occurrence variable together are analogous to the event frequency in
forward way to include control variables and mitigating efforts. This leads to the
conventional scenarios.
updating of the probability distribution of the relevant risk drivers.
• The impact variable often combines several cost components that are typically
In addition, certain risk drivers can be stressed in order to consider how they
the result of a combination of risk drivers. The variable provides the estimated
might behave under particular environmental influences or circumstances. This is
impact per unit of exposure and is later scaled by the exposure variable to reflect
visualised in the form of the red node in figure 3. The possibility to stress risk drivers
the size of the organisation or, alternatively, the entity, business, or operation
makes structured scenarios particularly useful for stress testing exercises.
under consideration.
The inclusion of control variables and the stressing of variables provide examples
While not visually identified differently in the structure, for clarity, this report of a more general advantage of structured scenarios, which is that the models lend
uses the term cost component to describe those risk drivers that directly themselves to sensitivity analysis and “what-if” scenarios. By changing one variable
define the impact variable and assign a monetary value to the variable. while keeping the other assumptions unchanged, the structures provide insight into
the effects of the change in the variable on the overall exposure.
8 https://www.elseware.fr/xoi-method/ | 9 Naim, P. and Condamin, L. (2019). Operational Risk Modeling in Financial Services. Chichester: Wiley. 10
Exploring
Exploring Risk
risk exposure
Exposure methodologies
Methodologies
Global variable
Global
Risk driver Risk driver Risk driver Risk driver
variable
The exposure is the resource that exposes a firm The occurrence captures the probability of an event The impact captures the loss amount per unit of
to a risk. This can be, for example, the number of occurring and leads to a loss when a resource is exposure. It is an amount of loss which is further
employees (or a specific group of employees, such affected. The occurrence and exposure together are broken down into several components as necessary,
as traders), products, models, or certain assets. therefore analogous to the frequency of an event. e.g. direct loss, repair costs, indirect costs, loss of
income, and/or fines.
11
Exploring Risk Exposure Methodologies
initial set of structures, the participant reported that they could later be adapted and
Transparency repeatedly applied with ease. In addition, the assessments all started from the same
Structured scenarios provide a systematic and transparent method to consider set of risk drivers and ensured a consistent assessment across the organisation.
and capture risk drivers, their interaction, and their contribution to the overall
exposure to a risk type. They thereby provide risk managers with an effective means
to talk about specific risk types, and communicate results and findings to senior
Figure 4: Advantages of structured scenarios
management, businesses, and other stakeholders.
In addition, and as previously touched upon, running the analysis under different
assumptions can provide further insights. By gaining a better understanding of the
role of specific risk drivers through sensitivity and “what-if” analyses, risk managers
can be in a better position to identify internal control gaps or shortcomings.
Scalability Transparency
The calculation of an impact value per unit of exposure allows to scale the model
to firm size. Obtaining the value of the impact variable also means that they are
comparable within and across organisations, entities and business units.
Moreover, the risk drivers that firms would take into account for particular scenarios
are likely not firm-specific. Rather, specific values that these factors could take may
vary, as well as the strength of controls in place to mitigate their impact. This creates
opportunities for sharing industry knowledge and experiences.
Scalability Adaptability
Adaptability
Once established, the structures can be rolled out to different levels within
organisations, such as to different legal entities or businesses. Evidence from a
steering group participant, who initially established a set of structured scenarios
at group level, showed that the further roll out at other levels of the organisation
proved straight-forward. While more effort and time was required to establish the
12
Exploring Risk Exposure Methodologies
and Vendor
pandemic warrants a scenarios structure in its own right,
The rationale behind the structure presented in figure 5 is or constitutes a stress on other scenarios. To demonstrate
that a pandemic outbreak leads to: the case of the latter, the second structure on vendor failure
shows how a stress can be included in a scenario that
Failure
• Additional costs to protect employees from considers resources affected by the associated changed
becoming ill environment.
• Additional costs to enable employees to continue
their work and keep operations running, including
The following sections additional IT costs or other operational expenses Applications of the model
present the two structures, (for example, the hiring of additional staff)
The structure can be used to estimate the impact of a
Pandemic and Vendor • A reduction in productivity caused by a reduced pandemic in two situations.
labour force
Failure, that were developed
as part of the pilot project. The structure is therefore aligned with the ORX guidance for Interpandemic situation
recording coronavirus-related operational risk losses, which This describes a situation between two pandemics and
was developed as part of the work of the ORX Definitions sufficiently ahead of the next expected outbreak, i.e. no
Working Group (see page 15 for further details). It also significant epidemic is foreseeable in the near-term future. In
draws from the experience of the coronavirus pandemic, this case, the parameters, in particular those at the regional
which led to people being unable to work due to infections level, will be based on past experiences and/or on SME input.
and, in contrast to previous global outbreaks, led to
nationwide lockdowns in many countries that restricted and Pre-pandemic situation
changed working environments over long periods of time.
In this case, a significant outbreak has occurred and may
The loss of productivity does not usually fall under the remit develop into a pandemic, albeit is not yet officially considered
of operational risk loss and, as indicated in figure 5, has been as such. As a result, the probability of a pandemic occurring
added as an optional component. It was included to give should be increased, and regional parameters may be
firms the possibility to gain an understanding of the wider adjusted based on current developments.
10 https://managingrisktogether.orx.org/coronavirus/scenario-development-handbook-pandemic 13
Exploring Risk Exposure Methodologies
Basic variable
Region
Risk driver
Global variable
IT
Status Protection infrastructure Operational Fraction Slowdown
LMH Revenue
duration costs costs costs at risk factor
14
Exploring Risk Exposure Methodologies
2. It was an unexpected cost, i.e. was not planned • Building costs – immediate additional costs
for, budgeted for, or part of any strategic plan of building closure caused by pandemic (for
that was in place before the pandemic. example, additional security costs).
3. It was not a goodwill payment/cost incurred (e.g. • Cost of establishing a pandemic command
ex-gratia payments etc). centre – costs over and above existing
business continuity planning/control spend.
It was recommended to include the following costs:
The full guidance note can be downloaded via
• Working from home – one-off costs of https://managingrisktogether.orx.org/coronavirus/
additional work-at-home equipment (e.g. capturing-operational-risk-impacts-coronavirus
laptops) and services (e.g. increased VPN
capacity) required specifically as a result of the
pandemic.
15
Exploring Risk Exposure Methodologies
1 Information Security (inc. Cyber) Business Continuity Information Security (inc. Cyber)
3 Technology Transaction Processing & Execution (ranked 7th in September 2020) Business Continuity
4 Regulatory Compliance People (ranked 9th in Septmber 2020 Technology (ranked 7th in May 2020)
5 Financial Crime Third Party Regulatory Compliance (ranked 11th in May 2020)
Organisations across countries and industries have disruption (either technical or due to information further described below and after the description of
experienced financial distress due to coronavirus security incidents) of vendors, causing financial losses the basic variables and associated risk drivers.
and the restrictions on work, supply chains and experienced while the firm tries to return to operations
travel that came with it. Such circumstances can at previous service levels. Additionally, structured scenarios allow for the
have knock-on effects on client firms, including on inclusion for controls and mitigating efforts. An
financial institutions. The structure presented in figure To consider the additional stress of a pandemic, example of this has also been provided in the form of
6 therefore considers both the potential default and certain nodes in figure 6 are identified as risk drivers a node representing a resilience plan that firms might
that can be stressed. The rationale behind this is have in place. This is also further described below.
11 https://managingrisktogether.orx.org/research/operational-risk-horizon-2020 | 12 https://managingrisktogether.orx.org/operational-risk-research/top-risk-review 16
Exploring Risk Exposure Methodologies
Global variable
Number Daily
Technical InfoSec Return to
of key Default
disruption disruption impact operations
vendors
17
Exploring Risk Exposure Methodologies
HIGH
or default might have on an organisation.
While the fraction of vendors that are falling into those three
LOW
Conclusion
What-if and sensitivity In the case of the vendor failure structure, the
nodes that could be stressed are identified in
analysis figure 6. As a pandemic unrolls, the restricted
and changed working environments put
The pandemic outbreak in early 2020 had
far-reaching consequences for the global
pressure on vendors, who themselves might We believe that there is huge merit
be dependent on products and services that
economy, including slumped GDP and stark
have become difficult to procure. for the use of structured scenarios for
increases in unemployment in many countries.
Macroeconomic factors can be relevant and operational risk management.
important risk drivers to consider for certain Inclusion of mitigation
scenarios. Structured scenarios allow for the In light of the fast-changing environments in which
consideration of environmental factors, both in
variable: Resiliency plan financial institutions operate, structured scenarios have
the form of global variables that directly affect Structured scenarios also allow for the much to offer as they can provide a more accurate
other nodes, or as stresses on certain nodes. inclusion of controls or mitigation efforts. view of the organisational risk profile than a pure loss-
The resiliency plan variable included as an driven approach can provide. In addition, there are
For this kind of ‘what-if’ analysis, thought must clear benefits in using structured scenarios as a basis
be given to which nodes can be expected to indirect risk driver of the impact variable (see
figure 6) reflects any steps a firm has taken to lead informed conversations over risk profiles with
be affected by changes in the environment. stakeholders across the lines of defence and at different
The nodes are stressed by establishing to dampen the impact from vendor default
or disruption. This has a direct effect on the levels of seniority.
alternative distributions, typically with the
help of subject matter expert (SME) input. return to operations node and may even The pilot study showed the advantages of creating
The performance of sensitivity analysis for counteract some of the effects of a stressed a community to bring the industry together and
specific nodes can add further value and environment on the node. discuss structured approaches and examples of
provide a better understanding of the result of specific models. During the project, both structures
each change in the assumptions. were presented to wider audiences, including at the
2020 Analytics and Scenario forum and a webinar
with interested firms. The positive feedback and lively
discussions showed that there is a clear interest among
the ORX membership to learn more about structured
scenarios and, where firms have already used such
approaches, to share knowledge and experiences.
Extended ORX Reference Taxonomy This report presenting the two structures that were
developed as part of the pilot will be made available
ORX has leveraged the wisdom of crowds to create: to the ORX membership together with an appendix
• A new operational and non-financial risk reference taxonomy providing examples of the data quantifying each variable.
Further research that continues this work will be part of
• Complementary causal and impact categories
the risk measurement research stream and conducted in
cooperation with the ORX Scenarios service.
Download the ORX Reference Taxonomy
19
Exploring Risk Exposure Methodologies
Elseware
Patrick Naim
CEO
patrick.naim@elseware.fr
Laurent Condamin
Managing Director
laurent.condamin@elseware.fr