What is stepwise regression and why has it fallen out of favor?

Question

AcadiFi · Accepted Answer

Both are partially right. Stepwise regression — forward selection, backward elimination, or hybrid — iteratively adds or removes predictors based on p-values or information criteria. It was standard practice for decades and is still taught in introductory courses. But modern statisticians identify serious problems.

How it works: Start with no variables (forward) or all variables (backward). At each step, add the variable with the lowest p-value (if < 0.05) or remove the one with the highest p-value (if > 0.05). Stop when no more changes improve the criterion.

Problems with stepwise:
1. Inflated R² and misleading p-values. Running hundreds of significance tests inflates Type I error massively. The final model's p-values are not valid because they ignore the selection process.
2. Instability. Tiny changes in the data can flip which variables end up in the model — a 1% resample can replace 'leverage' with 'operating margin.'
3. Overfitting. Stepwise selects variables that happen to correlate with noise in the training set.
4. Ignores prior knowledge. A theoretically important control might be dropped just because its p-value is 0.08.
5. Biased coefficient estimates on the survivors.

Better modern alternatives: LASSO and elastic net use penalized regression to do variable selection with honest statistics; cross-validation chooses the penalty. Best-subsets with AIC/BIC enumeration is thorough for moderate predictor counts. For causal inference, theory-driven variable selection beats any automated algorithm. Stepwise isn't useless — it's fine for preliminary exploration — but don't report stepwise-selected models as if their p-values were trustworthy.

What is stepwise regression and why has it fallen out of favor?

Master Level II with our CFA Course

Related Questions

Practice Questions