What is process verification in model validation?

Question

AcadiFi · Accepted Answer

Process verification tests whether the model is correctly implemented, runs reliably in production, and its outputs flow appropriately into downstream processes. It's the operational glue between theory and use.

Five focus areas:

1. Code review. Production code matches the documented specification. Key math (e.g., Black-Scholes, Cholesky decomposition) is implemented correctly. No hardcoded values that should be parameters.

2. Data pipelines. Input data flows from source systems correctly — no dropped records, no timezone issues, no stale fixings. Data transformations (log returns, winsorization, missing-data treatment) match documentation.

3. Model environment. Version control, access controls, change management. Who can edit production code? Are changes peer-reviewed? Is the deployment pipeline auditable?

4. Output integrity. Model outputs flow to the correct consumers (risk systems, capital calculation, P&L attribution) without truncation or rounding errors. Downstream recipients interpret them correctly.

5. User controls. Are overrides possible? If so, are they logged, approved, and reviewed? Fat-fingering protections for trader-facing tools.

A concrete example: Copperleaf Investment reviews their production CVA engine. Process verification finds: (a) production code uses a 10-factor PCA but documentation says 12 — a prior refactor wasn't reflected; (b) holiday calendar for Japan hadn't been updated for 2 years, causing spurious fixings; (c) outputs fed to Basel capital with correct aggregation but fed to management reporting with a different (outdated) aggregation. None of these would show up in outcomes analysis because end-to-end CVA looks 'roughly right,' but each could cause real errors in specific scenarios.

For FRM exam: process verification complements conceptual soundness and outcomes. A theoretically sound, empirically accurate model can still produce wrong answers if implemented with a bug or if downstream consumption is flawed. Model risk includes implementation risk.

What is process verification in model validation?

Master Part II with our FRM Course

Related Questions

Practice Questions