What is MCMC and when do I need it?

Question

AcadiFi · Accepted Answer

Markov Chain Monte Carlo generates samples from a probability distribution by constructing a Markov chain whose stationary distribution is the target...

Unlock with Scholar — $19/month

Get full access to all Q&A answers, practice question explanations, and progress tracking.

Start 7-Day Free Trial Compare Plans

No credit card required for free trial

Safiya Buhari at Kano Quantitative Finance estimates a hierarchical credit model with 380 obligors and 12 hyperparameters. The posterior has no closed form; MCMC is the only practical option.

The canonical Metropolis-Hastings algorithm:
1. Start at theta_0
2. Propose theta* from proposal q(theta*|theta_t)
3. Compute acceptance ratio A = min(1, f(theta*) * q(theta_t|theta*) / (f(theta_t) * q(theta*|theta_t)))
4. Accept theta* with probability A, else stay at theta_t
5. Repeat — after burn-in, samples approximate posterior

Safiya runs 50,000 iterations, discards first 10,000 (burn-in), keeps every 5th (thinning to reduce autocorrelation). Diagnostic checks:
- Trace plots — should look like fuzzy caterpillars, not drift
- Autocorrelation — should decay within 30-50 lags
- Gelman-Rubin R-hat — multiple chains should converge; R-hat close to 1.0
- Effective sample size — should be several hundred at minimum

MCMC variants:
- Metropolis-Hastings: general-purpose, needs good proposal
- Gibbs sampler: draws from full conditionals one parameter at a time
- Hamiltonian Monte Carlo (HMC/NUTS): uses gradients, scales to hundreds of parameters (Stan, PyMC use this)
- Reversible jump MCMC: for model selection across dimensions

When you need it: hierarchical models, latent variables, non-conjugate priors, high-dimensional posteriors. Avoid when conjugate posterior suffices or Laplace approximation is adequate. MCMC is powerful but expensive — 10 minutes to 10 hours depending on problem.

What is MCMC and when do I need it?

Master Level III with our CFA Course

Related Questions

Practice Questions