Calibration 1721242368
Calibration 1721242368
Calibration 1721242368
Data Treatment A further data treatment that is part of the model i.e., nontrivial filtering, joining, backfilling, imputation methods, proxies etc.
Data Representativeness Assessment of the representativeness conducted, to provide assurance that the data being used, appropriately reflects the environment on which the model will be applied
Model Segmentation Analysis to determine the product, business or sub-portfolio hierarchy level for model development in order to capture underlying sub-portfolio differences and maximise predicted power
of the model
PIT/Spot Model Build Data sampling Variable reduction, transformation and missing value treatment Segment model build and testing
Calibration Segmentation Design of calibration segments, grades or pool to achieve homogenous groups with similar risk levels
Calibration Process Calculation of long run average default rate and downturn LGD and EAD per calibration segment. Calibration process-alignment of PIT/Spot model output to long run downturn levels
MOC and Appropiate Adjustments Address identified issues and limitations via Margin of Conservatism (MoC) and appropriate adjustments (overrides) from the perspective of their influence on risk quantification.
Re-alignment and adjustments Adjusting the model for other use cases and purposes-for example, pricing
Development Documentation Model Development Document, Data Document, Rating Summary, ProForma, SAT, Model PreApproval Document
Development Documentation Note that this may not be the final code, which is covered under production
W W W . P E A K S 2 T A I L S . C O M
Development Cohorts/Snapshots
Period
Non-
PIT TTC Overlapping
Overlapping
W W W . P E A K S 2 T A I L S . C O M
Calibration Period – Full Economic cycle
• The economic cycle is identified to calibrate through the cycle (TTC) PD estimates to reflect the likely range
of variability in default rates for the type of exposure under consideration and should lead to calibrated PD
estimates representative of long-run average economic conditions.
• The following set of principles was considered in selecting the economic cycle for this purpose:
• The economic cycle should contain a minimum of one downturn period, identified using macroeconomic
factors relevant to the type of exposure under consideration, and in accordance with the EBA's regulatory
technical standards on the specification of an economic downturn.
• It is suggested that the economic cycle includes all key phases of a business cycle including the lead up to a
downturn period and the recovery from a downturn period as assessed against internal default rates.
• The economic cycle should contain at least the latest five years of data as at the time of model estimation.
W W W . P E A K S 2 T A I L S . C O M
Data Representativeness
• Data Representativeness Weighted Consecutive
#revol trades with
• Representativeness of Risk Characteristics Risk average # increases in
util > 90% in last
Bands payment in last Balance in last
• Test Purpose: To test whether key risk characteristics maintain stable 6M
6m 12 months
distributions between calibration data and application.
• Test Method: Check the distribution coverage of key drivers for PD (e.g. 1 100% 98% 100%
payment and balance related factors) by comparing those in the most recent 2 100% 100% 100%
scoring month (202305) vs. the LRA vintages (200806 - 201703).
3 96% 98% 100%
• In each PD band, compare the distributions of key risk drivers (including
"number of revolving trades with utilization above 90% in last 6 months", "payment 4 100% 100% 100%
in last 6 months" and "Consecutive increases in Balance in last 12 months". Two
steps were followed: 5 97% 100% 100%
• Calculate the range where majority (90%) of the observations fall based on 6 100% 100% 100%
calibration data, i.e. 5th percentile to 95th percentile 7 100% 100% 100%
• Calculate the proportion of accounts (from most recent vintage) falls into the
8 100% 100% 100%
range calculated in step 1.
• Test Results: The proportion falling into the range was above 85% across most 9 100% 100% 100%
of the 12 bands for all key drivers. 10 100% 100% 100%
• Key Drivers Distribution Comparison 11 100% 100% 100%
• Test Conclusion: Comparable range of values was observed for key risk drivers
12 100% 100% 100%
between model calibration period/development period and recent application
period.
W W W . P E A K S 2 T A I L S . C O M
Rating Philosophy – TTC, PIT or Hybrid
• For all portfolios generally we use TTC PD approach
• For banks governed by PRA, a hybrid approach is recommended for Mortgage portfolios- Problem with PITs is that capital
requirement , goes very high in downturn and very low in boom times. Mortgage industry is trillion pounds industry; if capital
requirement goes high in downturn, it would be a threat to the economic stability. With TTCs, the main issue is that drivers to it,
are very static in nature and might not reflect the actual economic cycle. So, PRA in UK has proposed that organization should
neither use the PITs approach nor TTCs approach. Rather organization should be using the hybrid approach.
• Now with hybrid approach, PRA has introduced the concept of cyclicality. PRA defines cyclicality as measure of PITness of the
model which can be measured portfolio to portfolio. Additionally, PRA has put a cap on cyclicality such that at no point model
cyclicality should exceed more that 30%.
• To calculate Hybrid PD, where sufficient historical data is available, the historical approach should be used to determine the long
run average PD to assign to each pool. Where this is not available, the same approach as provided by PRA in their stylized
example should be followed to calculate the LRA DR where the external data sources is used to impute the missing historical
default rates. The stressed period considered for the long run estimate should include the downturn period.
• E.g., The TTC back cast yields an inferred default rate of 11.85% for this risk grade during the historical economical period (1988
to 2007).
• Assume a cyclicality level of 30%.
• Risk Grade 1 hybrid PD is: (9.64% x 0.3) + (11.85% x 0.7) = 11.19%
W W W . P E A K S 2 T A I L S . C O M
Calibration bias adjustments - Seasonality
Seasonality Analysis
W W W . P E A K S 2 T A I L S . C O M
Calibration bias adjustments - Seasoning
• Firstly, the age of peak default rate is identified Time on book Default rate by Peak default rate Seasoned/ Seasoning adj.
distribution of time on book (a) (max of col (a)) Unsecured (c)=(b)/(a)
based on the portfolio distribution by time on Portfolio (b) accounts
book. Refer to below example, the peak default <2 MOB 0.5% 2% Unseasoned 4
3-4 MOB 1% 2% Unseasoned 2
rate (i.e. 2%, as indicated by the red circle in the 5-6 MOB 2% 2% Seasoned N/A (100% for
chart below) is the accounts with month on seasoned)
7-8 MOB 1.9% 2% Seasoned N/A (100% for
book (MOB) equal 5 and 6. Hence, accounts with seasoned)
9+ MOB 1.5% 2% Seasoned N/A (100% for
MOB<5 are unseasoned accounts and accounts seasoned)
with MOB>=5 are seasoned accounts.
Time on book Seasoned/ % of accounts (d) Seasoning adj. Seasoning adj.
distribution of Unsecured (c) factor (pool)
Pool accounts (e)=(d)x(c)
<2 MOB Unseasoned 20% 4 0.8
3-4 MOB Unseasoned 19% 2 0.38
5+ MOB Seasoned 61% 1 0.61
Seasoning adjustment factor for this pool= 1.79
Given the above seasoning adjustment in portfolio level, pool level adjustment
factor is calculated based on the distribution of accounts by time on book (see
below table). For a pool with PD estimate before seasoning adjustment is 1%, the
pool PD estimate with seasoning adjustment is 1.79% (=1%*1.79).
W W W . P E A K S 2 T A I L S . C O M
Calibration bias adjustments - Recency
• Background
• As a final adjustment to the PD estimation, the long run average PDs are further calibrated to account for the performance of holders
in recent time periods. This provides a more conservative assessment of the long run PD by taking into account any differences
observed in the recent periods.
• This calibration process established if recent performance was significantly above the adjusted LRPD and shifted the LRPD such that
the recent data was within 1 standard deviation of the long run. If any of the last four data points (quarters) fell outside of this
threshold (i.e. LRPD +/- 1standard deviation of the mean), then the LRPD was adjusted, otherwise no PD calibration was required
W W W . P E A K S 2 T A I L S . C O M
Calibration bias adjustments
– Short term contracts
W W W . P E A K S 2 T A I L S . C O M
Cat A & B
Institutions should identify all deficiencies related to the estimation of risk parameters that lead to a bias in
the quantification of those parameters or to an increased uncertainty that is not fully captured by the general
estimation error, and classify each deficiency into one of the following categories:
Category A: Identified data and methodological deficiencies;
Category B: Relevant changes to underwriting standards, risk appetite, collection and recovery policies and
any other source of additional uncertainty.
W W W . P E A K S 2 T A I L S . C O M
Overrides
• Business overrides
• Policy overrides
• Business overrides are made considering the • The following policy overrides are applicable to the Banks PD Model
following:
• Explicit Guarantee
• Limitations of the model for the counterparty
• If there is an irrevocable, written corporate guarantee from the parent that provides additional
in question risk mitigants, then the parental guarantor's rating can be used instead of the Subsidiary
borrower's standalone rating. This implies that the parental guarantor has been run through a
• External factors/information not captured by model suitable for that entity.
the model (e.g. recent profits warning, M&A • Parental support/government support adjustment
activity, or a recent, significant change in
- Parental adjustment framework has been performed, and an adjustment of the borrower's
prospects) Standalone CRR (notched from the parent CRR) is appropriate, or the CRR of the parent is
used Sovereign ceiling
• Expert judgment of the lending officer
- Override to ensure that the customer risk rating does not exceed the related sovereign
ratings.
W W W . P E A K S 2 T A I L S . C O M
Why Calibration - Issues in Rating based PD Estimation
• Bank’s create credit rating models for the primary purpose of
• Discrimination - Differentiating the credit quality (chance of default) across different borrowers /
counterparties
• Calibration - Quantifying the risk of default (predicting the probability of default) across rating
grades
• Based on actual historical performance of borrowers in different rating grades, we can derive
year-wise and long-run average default rates for each rating grade of the rating model
• However, the resultant default rate estimates derived from purely historical data may be
subject to various issues and may not be good predictors or future defaults:
• Zero defaults in some grades
• Lack of monotonicity of default rates / PDs across rating grades
• Change in the rating model during the historical observation period
• Change in the portfolio of borrowers
• Change in the forward-looking economic conditions
W W W . P E A K S 2 T A I L S . C O M
Example
LRA DR Across Rating Grades
70.00%
Historical Rating data (2010-2021) with performance till 2022
60.00%
50.00%
20.00%
1 19 11 57.89%
10.00%
2 200 27 13.50%
0.00%
3 1283 252 19.64% 0 2 4 6 8 10 12
4 3353 190 5.67%
5 3336 102 3.06% Frequency Distributions of Historical Good
6 3239 41 1.27% and Bad Accounts
40.00%
7 2384 40 1.68%
35.00%
8 2523 28 1.11% 30.00%
25.00%
9 2083 9 0.43%
20.00%
10 1093 5 0.46% 15.00%
%Bad %Good
W W W . P E A K S 2 T A I L S . C O M
PD Calibration
• PD calibration: The part of the process of the estimation of risk parameters which leads to
appropriate risk quantification by ensuring that when the PD ranking is applied to a
calibration sample, the resulting PD estimates correspond to a long-run average default rate
at the level relevant for the applied method.
• PD curve calibration: transformation of a set of rating grade level probabilities of default
(PDs) to an average PD level reflecting current or changed overall portfolio-wide PD
W W W . P E A K S 2 T A I L S . C O M
Steps in PD Calibration
• 1. Smoothening – to ensure that PD monotonically increases/decreases with grades. This
step is done using historical distribution of borrower across rating grades.
• 2. Shifting – Shifting the curve up or down so that the target central tendency observed in
historical data is met
• 3. Projections – Adjusting the PD curve based on forward looking expectations of PD
To build Forward Looking Expectations
a) Forecast ODR of each grade individually using MEVs
b) Use Current distribution of borrower across rating grades.
W W W . P E A K S 2 T A I L S . C O M
Objectives of PD Calibration
• Issues to be addressed:
• Monotonicity - Estimate appropriate grade-wise PDs, while ensuring that they are monotonically
increasing for worse grades and positive for all grades
• Conservatism: estimated PDs should not become statistically significantly lower than LR average
default rate (No underestimation)
• Discriminatory power of the rating model should not reduce significantly due to the estimated PDs
• Exponential relation - From the LR average default rate curve, while the rating scale is linear, the PDs
are non-linear – thus, non-linearity / exponential relationship should hold as far as possible
W W W . P E A K S 2 T A I L S . C O M
PD Calibration Methods
Linear Regression of
Tasche (2009) Quasi
Log-Odds with the Tasche (2009) Binormal
Moment Matching
scores as dependent Model
Model
variable
W W W . P E A K S 2 T A I L S . C O M
Regression of Log-Odds on Scores
• Convert observed default rate for each score / rating to its log-odds Observed Log Odds (good/bad)
equivalent 9.00
8.00
1−𝑂𝐷𝑅𝑆
• 𝐿𝑂𝑆 = ln( ) 7.00
y = 0.6574x - 0.2109
R² = 0.9117
𝑂𝐷𝑅𝑆
6.00
• LOs will range from negative to positive values; they will be positively 5.00
correlated to scores and the relationship will be close to linear 4.00
• The values of α and β are the constant and slope coefficient of the linear0.00
0 2 4 6 8 10 12
regression, which can be estimated. -1.00
1 50.00%
• 𝑃𝐷𝑠 =
1+𝑒 𝛼+𝛽𝑆+𝜀 40.00%
30.00%
20.00%
10.00%
0.00%
1 2 3 4 5 6 7 8 9 10 11
W W W . P E A K S 2 T A I L S . C O M
Regression of Log-Odds
60.00%
50.00%
40.00%
30.00%
20.00%
10.00%
0.00%
1 2 3 4 5 6 7 8 9 10 11
W W W . P E A K S 2 T A I L S . C O M
Issues in Regression of Log-Odds
W W W . P E A K S 2 T A I L S . C O M
Tasche Binormal PD Calibration
• Requires an ex-ante knowledge of the rating profile of borrowers at the start of the forecast
period and a forecasted overall PD for the portfolio
• Can be used for newly developed rating models if all borrowers can be re-rated with the new
model in a on-time exercise before the forecast period or atleast a large representative
sample of the borrowers in the portfolio are re-rated as per the new model
• Will be applicable for an existing rating model whose PDs need to be recalibrated
• Works also in cases where atleast an overall forecasted PD is known, even if the rating profile
of borrowers is not known
W W W . P E A K S 2 T A I L S . C O M
Tasche Binormal PD Calibration
• The objective is still to use a non-linear (exponential) Frequency Distribution of All Rated Accounts
transformation of scores to predict / estimate PDs 18.00%
16.00%
1
• 𝑃𝐷𝑠 = 1+𝑒 𝛼+𝛽𝑆+𝜀 14.00%
12.00%
%Bad %Good
W W W . P E A K S 2 T A I L S . C O M
Issues in Tasche Binormal PD Calibration
20.00%
0.00%
1 2 3 4 5 6 7 8 9 10 11
ODR #REF!
W W W . P E A K S 2 T A I L S . C O M
Tasche Quasi Moments Matching
• Requires an ex-ante knowledge of the rating profile of non-defaulted borrowers at the start of
the forecast period and a forecasted overall PD for the portfolio
• Based on the cumulative probability distribution of non-defaulted borrowers across rating
grades, normalized scores are generated through norms-inverse transformation to introduce
normality of the independent variable
• The mean and standard deviation of the normalized scores are used to estimate α and β
W W W . P E A K S 2 T A I L S . C O M
Tasche QMM PD Calibration
Normalised Score
• The objective is still to use a non-linear (exponential) 4
transformation of normalized scores (ST)to predict / estimate 3
PDs 2
1
• 𝑃𝐷𝑠 = 𝑇
1
1+𝑒 𝛼+𝛽𝑆 +𝜀 0
α 3.88 10.00%
β 1.12 0.00%
1 2 3 4 5 6 7 8 9 10 11
ODR Calibrated PD
W W W . P E A K S 2 T A I L S . C O M
Tasche PD Calibration Parameters
Parameter Tasche Binormal Tasche QMM
p Overall Observed / Target PD
AR Overall Observed / Target Accuracy Ratio
µ Mean Score estimated from known / observed Mean Normalized Score derived from norms-inverse of
distribution of all accounts cumulative distribution of good accounts
τ2 Variance of Score from known / observed Variance of Normalized Score
distribution of all accounts
c 𝐴𝑅 + 1
2 × 𝑁 −1
2
σ2 𝜏2
1 + 𝑝 × (1 − 𝑝) × 𝑐 2
µG 𝜇+𝑝×𝜎×𝑐
µB 𝜇 − (1 − 𝑝) × 𝜎 × 𝑐
W W W . P E A K S 2 T A I L S . C O M
Chi-Square Test for Goodness of Fit Test
• For any method of PD calibration, it is important to verify that broadly the predicted default
distribution is not statistically different from the observed default distribution
H0: Predicted Frequency Distribution of Defaults across Rating Grades = Observed Frequency
Distribution of Defaults across Rating Grades
H1: Predicted Frequency Distribution of Defaults across Rating Grades ≠ Observed Frequency
Distribution of Defaults across Rating Grades
• The Chi-square test at a predetermined level of significance (say 1%) can help to conclude
on this
𝑂𝐷𝑆 −𝑃𝐷𝑆 2
• Chi-square statistic =∑
𝑃𝐷𝑆
• If the Chi-square statistic > critical chi-square, then null is rejected, and the distributions are
different
• However, rejection based on chi-square does not necessarily invalidate the calibration
results and a second-level optimization of α can be carried out to minimum Chi-square
W W W . P E A K S 2 T A I L S . C O M
1. Rating grade → monotonic transformation → Z
scale
5. Find parameters of PD - 𝛼, 𝛽
1
Under QMM calibrated PD=
1+exp 𝛼+𝛽𝑋
6. Get calibrated PD
Parameters are found through Moment Matching
7. Run checks on model output- Calibration
accuracy, Discriminatory Power, GoF tests Under log odds Regression parameters are found
through regression i.e., log odds (G/B)= 𝛼 + 𝛽 X
Note : in case model fails calibration accuracy we can use delta add-ons
W W W . P E A K S 2 T A I L S . C O M
1. We want PD against each rating grade i.e., conditional PD Curve
4. Prob of rating given default follows ND with mean rating 𝜇𝐷 & 𝑉𝑎𝑟 𝜎 2
Prob of rating given non-default follows ND with mean rating
𝜇𝑁𝐷 & 𝑉𝑎𝑟 𝜎 2
1
5. Using Bayes formula P 𝐷|𝑋 =
1+exp 𝛼+𝛽𝑋
𝛼~𝑓𝑛 𝑈𝐷 , 𝑈𝑁 , 𝑝, 𝜎
𝛽~𝑓𝑛 𝑈𝐷 , 𝑈𝑁 , 𝜎
𝑈𝐷 , 𝑈𝑁 , 𝜌, 𝜎 ~fn (τ, 𝜇, 𝐴𝑅, 𝑝)
W W W . P E A K S 2 T A I L S . C O M
QMM
PD curve
𝑷 𝑿=𝒙 𝑫 ∙𝑷 𝑫
We want to find the PD curve i.e., probability of default given a particular rating i.e.,
P(D|X) 𝑷 𝑿=𝒙
1 𝛼~𝑓𝑛 𝑈𝐷 , 𝑈𝑁 , 𝑝, 𝜎
When you solve, you get P 𝐷|𝑋 = 𝛽~𝑓𝑛 𝑈𝐷 , 𝑈𝑁 , 𝜎
1 + exp 𝛼 + 𝛽𝑋
W W W . P E A K S 2 T A I L S . C O M
𝜌 is known, we must solve this for 𝑈𝐷 , 𝑈𝑁 , 𝜎
𝜇 = 𝑝 𝑈𝐷 + 1 − 𝑝 𝑈𝑁
From data
Known
𝑈𝑁 −𝑈𝐷
Also, AR = 2 N. 𝑖𝑛𝑣 −1
2𝜎
Known AUC
W W W . P E A K S 2 T A I L S . C O M