What is negative binomial regression and when do I prefer it over Poisson?

Question

AcadiFi · Accepted Answer

Negative binomial regression extends Poisson regression by adding a dispersion parameter that allows variance to exceed mean. This handles the common real-world problem of overdispersion where count data has more variability than pure Poisson predicts. Mathematical setup: Y ~ NegBin(μ, α) where E[Y] = μ and Var[Y] = μ + αμ². The α is the dispersion parameter. When α = 0, the model reduces to Poisson (variance = mean). When α > 0, variance exceeds mean — the hallmark of overdispersion. Intuition: negative binomial can be derived as a Poisson-gamma mixture. Suppose each observation has its own unobserved rate λᵢ drawn from a gamma distribution with mean μ and shape 1/α. Conditional on λᵢ, Yᵢ is Poisson(λᵢ). Marginally, Yᵢ follows negative binomial. The unobserved heterogeneity captures factors we haven't included (macro conditions, unobservable borrower quality). ```mermaid flowchart LR H[Unobserved
Heterogeneity] --> L[Rate
lambda_i Gamma] L --> C[Count
Y Poisson] C --> NB[Marginal
NegBin] ``` Example: Brookmere Bank models annual loan defaults per branch. Poisson regression shows Pearson chi-square / df = 3.8, indicating severe overdispersion. Refitting as negative binomial yields α̂ = 0.45, confirming substantial extra variability. Standard errors in the negative binomial model are about 2× larger than Poisson, making some 'significant' Poisson results no longer significant. Inferences change dramatically. Coefficient interpretation is identical to Poisson (rate ratios via exp(β)), but confidence intervals and p-values differ. Model selection: likelihood ratio test comparing α=0 against α>0 tests whether negative binomial is needed over Poisson. Other options for overdispersion include zero-inflated models (when excess zeros are present) and hurdle models (two-part: zero vs nonzero, then count given nonzero). Negative binomial is the standard starting point for credit default counts and claim frequencies with heterogeneous risk.

What is negative binomial regression and when do I prefer it over Poisson?

Master Level II with our CFA Course

Related Questions

Practice Questions