What does model validation look like for stress testing models, and why is it harder than validating VaR models?

Question

AcadiFi · Accepted Answer

Validating stress test models is fundamentally more challenging than validating VaR models because stress scenarios are rare events with limited historical analogues. You cannot backtest a model against a scenario that hasn't occurred.

**Validation approaches for stress test models:**

1. **Benchmarking** — compare model outputs against industry estimates, regulatory results, or third-party vendor models for the same scenario
2. **Sensitivity testing** — verify that model outputs respond logically to changes in inputs (e.g., higher unemployment should increase loan losses)
3. **Historical replay** — run the model against past crises (2008, 2020) and compare projected losses to actual realized losses
4. **Component-level testing** — validate individual sub-models (PD model, LGD model, prepayment model) separately against observed data
5. **Expert judgment review** — senior risk officers assess whether outputs are economically intuitive

\`\`\`mermaid
flowchart TB
    A[Stress Test Model] --> B[Conceptual Soundness Review]
    A --> C[Sensitivity Analysis]
    A --> D[Historical Benchmarking]
    A --> E[Outcomes Analysis]
    B --> F[Documentation & Assumptions]
    C --> G[Input Perturbation Tests]
    D --> H[2008/2020 Replay Comparison]
    E --> I[Ongoing Monitoring]
    F & G & H & I --> J[Validation Report to Board]
\`\`\`

At Hartwick Bancshares, the validation team discovered that their commercial real estate loss model produced lower losses under the severely adverse scenario than under the adverse scenario for certain property types — a logical inconsistency indicating a coding error in the severity-mapping function. This type of "monotonicity check" is unique to stress test validation.

The core difficulty: VaR models generate daily predictions that can be backtested against thousands of observations, while stress models produce perhaps one or two comparable episodes per decade. Validators must therefore rely more heavily on conceptual soundness and less on statistical testing.

What does model validation look like for stress testing models, and why is it harder than validating VaR models?

Master Part I with our FRM Course

Related Questions

Practice Questions