Credit Risk Model Monitoring Dashboard

By Jonas Osman Abdelghafour.

Ongoing monitoring is the continuous evidence that a credit risk model still performs as validated. Regulators — the ECB TRIM guide for internal ratings-based (IRB) banks and BCBS 239 for risk data — expect monitoring to be structured, timely, and actionable. A concise dashboard covering four dimensions is usually enough to catch material drift early: discrimination, calibration, stability, and overrides.

Discrimination

Discrimination measures whether the model still separates defaulters from non-defaulters. The standard metrics are the Gini coefficient (or equivalently AUC-ROC = (Gini + 1) / 2) and, for finer diagnostics, the Kolmogorov-Smirnov statistic and cumulative-accuracy-profile curves.

Practical thresholds depend on portfolio type: retail scorecards typically operate with Gini in the 0.55–0.75 range, corporate PD models often lower. A drop of more than a few percentage points from the validated baseline, or a sustained downward trend over several quarters, is a monitoring trigger.

Calibration

Calibration measures whether predicted PDs match observed default rates. The Hosmer-Lemeshow test and the Brier score are standard, but a simpler exhibit — predicted vs observed default rate by rating grade, with binomial confidence bounds — is what actually gets read at monitoring committees.

Systematic under-prediction on any grade, or a pattern where predicted PDs are conservative in benign years and optimistic in stress, indicates a through-the-cycle vs point-in-time misalignment. See IFRS 9 PD calibration: through-the-cycle vs point-in-time for the calibration frame.

Stability

Stability measures whether the population the model sees today resembles the population it was built on. The Population Stability Index (PSI) computed across score bands is the workhorse:

PSI = Σ (actual% − expected%) × ln(actual% / expected%)

Rules of thumb: PSI < 0.10 stable, 0.10–0.25 material shift, > 0.25 significant shift. Characteristic Stability Index (CSI) computed at the variable level explains where a PSI shift is coming from. A stability breach does not by itself invalidate a model, but it triggers a decision: monitor, adjust, or redevelop.

Overrides

Override monitoring is the least automated and often the most revealing. It tracks how frequently model output is overridden in underwriting or rating, in which direction (up-rating vs down-rating), by which teams, and with what subsequent default performance. High and asymmetric override rates typically signal either a model that has lost credibility with users or a portfolio segment the model does not cover well. Either finding is material and should be escalated.

Dashboard structure

A useful monthly pack is short: for each material model, one page with discrimination, calibration, stability, and override headline metrics against thresholds; a trend chart over the last eight quarters; and a red/amber/green status with the specific action required for anything not green. Deeper diagnostics live in an appendix. The board risk committee sees the RAG summary; the model risk committee reads the full pack.

Governance and escalation

Every metric on the dashboard should map to a documented threshold and escalation path. A single amber flag is typically noted; two consecutive periods or a red flag triggers a formal issue on the model inventory with an owner and a remediation plan. See model risk remediation and finding closure for what happens next.

Limitations

Monitoring metrics are lagging: default emergence often takes 12–24 months, so discrimination and calibration deteriorate before they are measured. Complementary early indicators — flow rates, forbearance take-up, macroeconomic overlays — reduce the lag but do not eliminate it. Monitoring is necessary but not sufficient; periodic full revalidation and, where warranted, redevelopment remain the definitive control.

Conclusion

A monitoring dashboard is not a compliance artefact. It is the operational feedback loop that keeps credit models honest between validation cycles. Kept short, tied to thresholds, and connected to a real escalation path, it repays its cost many times over. Related notes: LGD and EAD validation pitfalls and actuarial model governance and the three lines of defence.

Written by Jonas Osman Abdelghafour, actuary and financial risk manager. Background and contact details are on the about page.