Why do financial regulators require explainability in AI models, and what are the main techniques for making black-box models interpretable?

Explainable AI (XAI) refers to techniques that make model predictions understandable to humans. Financial regulators require explainability because credit decisions, insurance pricing, and investment recommendations directly affect consumers' lives, and affected parties have a right to understand why they were denied credit or charged higher premiums.\n\nRegulatory Drivers:\n- Adverse action notices: lenders must explain why an application was denied\n- Model risk management (SR 11-7): regulators expect banks to understand and validate all models\n- GDPR Article 22: individuals have a right to meaningful explanation of automated decisions\n- Fair lending laws: inability to explain decisions makes it impossible to prove non-discrimination\n\nExplainability Techniques:\n\n| Technique | Type | Scope | How It Works |\n|---|---|---|---|\n| SHAP values | Post-hoc | Local + Global | Game-theoretic feature attribution |\n| LIME | Post-hoc | Local | Local linear approximation |\n| Partial Dependence | Post-hoc | Global | Marginal effect of one feature |\n| Attention weights | Intrinsic | Local | Neural network self-explanation |\n| Decision rules | Intrinsic | Global | Inherently interpretable model |\n\nWorked Example with SHAP:\n\nWestbrook Financial uses a gradient boosting model to score mortgage applications. Applicant Thornton receives a denial. The SHAP decomposition shows:\n\n| Feature | SHAP Value | Direction |\n|---|---|---|\n| Debt-to-income ratio (48%) | -1.82 | Strong negative |\n| Payment history (2 late in 12 months) | -0.94 | Moderate negative |\n| Employment tenure (8 months) | -0.71 | Moderate negative |\n| Savings balance ($42,000) | +0.63 | Positive |\n| Income ($95,000) | +0.45 | Positive |\n| Net score | -2.39 | Below threshold (-1.0) |\n\nThe adverse action notice states: \"Primary reasons for denial: (1) debt-to-income ratio exceeds guidelines, (2) recent payment delinquencies, (3) insufficient employment history.\"\n\nThis individual explanation satisfies regulatory requirements because it identifies the specific factors driving the negative decision in order of importance.\n\nThe Accuracy-Explainability Spectrum:\n- Fully interpretable (logistic regression, decision trees): easy to explain but may miss complex patterns\n- Post-hoc explainable (gradient boosting + SHAP): high accuracy with adequate explanations\n- Black box (deep neural networks without XAI): highest potential accuracy but unacceptable for regulated decisions\n\nMost financial institutions land in the middle — using moderately complex models with post-hoc explainability tools.\n\nKey Exam Points:\n- Explainability is not optional in regulated financial services\n- Local explanations (why this specific decision) differ from global explanations (how the model generally works)\n- Model documentation must include intended use, limitations, validation results, and fairness assessments\n\nStudy AI governance requirements in our CFA Ethics and Professional Standards course.