How do AIC, BIC, and HQC differ in penalizing model complexity, and which should I use?

Information criteria balance goodness of fit (log-likelihood) against model complexity (number of parameters). They differ in how severely they penalize additional parameters, which leads to different model selection behavior.\n\nFormulas:\n\nAll three have the form: IC = -2 x ln(L) + penalty\n\n- AIC = -2 ln(L) + 2k\n- BIC = -2 ln(L) + k x ln(n)\n- HQC = -2 ln(L) + 2k x ln(ln(n))\n\nwhere k is the number of estimated parameters, n is the sample size, and L is the maximized likelihood.\n\nPenalty Comparison:\n\nFor a model with k = 5 parameters and sample sizes:\n\n| Sample Size (n) | AIC Penalty | BIC Penalty | HQC Penalty |\n|---|---|---|---|\n| 50 | 10 | 19.56 | 12.51 |\n| 200 | 10 | 26.49 | 14.61 |\n| 1000 | 10 | 34.54 | 16.27 |\n| 10,000 | 10 | 46.05 | 18.53 |\n\nAIC penalty is constant regardless of n. BIC grows with ln(n), becoming increasingly harsh with larger samples. HQC falls between them, growing with ln(ln(n)).\n\nPractical Example:\n\nAnalyst Duncan at Greystone Partners evaluates three models for predicting GDP growth:\n\n| Model | Parameters (k) | -2 ln(L) | AIC | BIC (n=120) | HQC |\n|---|---|---|---|---|---|\n| AR(1) | 2 | 185.4 | 189.4 | 191.0 | 189.9 |\n| AR(2) + Inflation | 4 | 178.2 | 186.2 | 189.3 | 187.2 |\n| Kitchen Sink (8 vars) | 9 | 170.8 | 188.8 | 198.9 | 192.0 |\n\nAIC selects the AR(2) + Inflation model (186.2 is lowest). BIC also selects AR(2) + Inflation (189.3). But notice the Kitchen Sink model has the best fit (lowest -2 ln(L)) yet ranks last on all criteria because the penalty for 9 parameters outweighs its marginal fit improvement.\n\nWhen They Disagree:\n- BIC tends to select more parsimonious models, especially with large n\n- AIC can overfit by selecting models with too many parameters\n- BIC is consistent (selects the true model as n approaches infinity) if the true model is among the candidates\n- AIC is efficient (minimizes prediction error) even if the true model is not among candidates\n- HQC is consistent like BIC but penalizes less aggressively\n\nCFA Exam Guidance:\nFor the exam, know that BIC penalizes more heavily than AIC, leading to simpler model selection. Lower values are always preferred. Use AIC when prediction accuracy matters most; use BIC when you want the most parsimonious model.\n\nPractice information criteria problems in our CFA Quantitative Methods question bank.