How do I tune the proposal distribution in Metropolis-Hastings?

Q: How do I tune the proposal distribution in Metropolis-Hastings?

Acceptance rate diagnoses proposal quality. Target:\n- 1D problems: ~44% acceptance\n- Multi-dimensional: ~23.4% acceptance (Roberts-Gelman-Gilks result)\n- Too high (>50%): proposal too narrow, exploring slowly, high autocorrelation\n- Too low (<10%): proposal too wide, rejecting most, stuck at modes\n\nMateusz Brodziak at Gdynia Risk Analytics ran an MH chain for a 6-parameter credit model. Initial proposal: normal with covariance = 0.1 * Identity. Acceptance rate hit 78% — too high, chain barely moved. He scaled proposal covariance by 4x, dropping acceptance to 32% and autocorrelation time from 180 lags to 22 lags.\n\nTuning approaches:\n\n1. Adaptive Metropolis — Haario et al. updates proposal covariance using past samples. Converges to scaled posterior covariance. Be careful: adaptation must diminish (ergodic theorem requires it).\n\n2. Pilot tuning — run 2,000 iterations, compute sample covariance, use it (scaled by 2.38^2/d) as proposal covariance.\n\n3. Random-walk Metropolis vs independence proposals — for concentrated posteriors, independence proposals centered near the mode work well.\n\n4. Transformed variables — if posterior for variance is heavily right-skewed, sample log(variance) instead.\n\n```mermaid\nflowchart TD\n A[Pilot Run] --> B[Compute Sample Cov]\n B --> C[Scale by 2.38^2/d]\n C --> D[Production Run]\n D --> E[Check Acceptance ~23%]\n E --> F[Check R-hat, ESS]\n```\n\nModern practice uses Hamiltonian Monte Carlo (HMC) and No-U-Turn Sampler (NUTS) which leverage gradient information to propose states along Hamiltonian trajectories. Acceptance is much higher and mixing much better — NUTS is default in Stan and PyMC. But classical MH still shines for problems without easy gradients (discrete parameters, hard-to-differentiate likelihoods).

Question

AcadiFi · Accepted Answer

Acceptance rate diagnoses proposal quality. Target:
- 1D problems: ~44% acceptance
- Multi-dimensional: ~23.4% acceptance (Roberts-Gelman-Gilks result)
- Too high (>50%): proposal too narrow, exploring slowly, high autocorrelation
- Too low (<10%): proposal too wide, rejecting most, stuck at modes

Mateusz Brodziak at Gdynia Risk Analytics ran an MH chain for a 6-parameter credit model. Initial proposal: normal with covariance = 0.1 * Identity. Acceptance rate hit 78% — too high, chain barely moved. He scaled proposal covariance by 4x, dropping acceptance to 32% and autocorrelation time from 180 lags to 22 lags.

Tuning approaches:

1. Adaptive Metropolis — Haario et al. updates proposal covariance using past samples. Converges to scaled posterior covariance. Be careful: adaptation must diminish (ergodic theorem requires it).

2. Pilot tuning — run 2,000 iterations, compute sample covariance, use it (scaled by 2.38^2/d) as proposal covariance.

3. Random-walk Metropolis vs independence proposals — for concentrated posteriors, independence proposals centered near the mode work well.

4. Transformed variables — if posterior for variance is heavily right-skewed, sample log(variance) instead.

```mermaid
flowchart TD
  A[Pilot Run] --> B[Compute Sample Cov]
  B --> C[Scale by 2.38^2/d]
  C --> D[Production Run]
  D --> E[Check Acceptance ~23%]
  E --> F[Check R-hat, ESS]
```

Modern practice uses Hamiltonian Monte Carlo (HMC) and No-U-Turn Sampler (NUTS) which leverage gradient information to propose states along Hamiltonian trajectories. Acceptance is much higher and mixing much better — NUTS is default in Stan and PyMC. But classical MH still shines for problems without easy gradients (discrete parameters, hard-to-differentiate likelihoods).

How do I tune the proposal distribution in Metropolis-Hastings?

Master Level III with our CFA Course

Related Questions

Practice Questions