How does ensemble stacking combine multiple models, and why does it outperform individual learners in financial prediction?

Stacking (stacked generalization) trains a meta-learner to optimally combine predictions from diverse base models. Unlike simple averaging, it learns the relative strengths and weaknesses of each base model across different market conditions.\n\nStacking Architecture:\n\n`mermaid\ngraph TD\n A[\"Training Data\"] --> B[\"Base Model 1
Random Forest\"]\n A --> C[\"Base Model 2
Gradient Boosting\"]\n A --> D[\"Base Model 3
Linear Regression\"]\n A --> E[\"Base Model 4
Neural Network\"]\n B --> F[\"Out-of-fold
predictions\"]\n C --> F\n D --> F\n E --> F\n F --> G[\"Meta-Learner
(Ridge Regression)\"]\n G --> H[\"Final Prediction\"]\n`\n\nWhy Stacking Beats Averaging:\n\nSilverpeak Quantitative built four base models to predict sector rotation signals:\n\n| Base Model | Bull Market Accuracy | Bear Market Accuracy | Overall |\n|---|---|---|---|\n| Random Forest | 64% | 58% | 61% |\n| Gradient Boosting | 59% | 67% | 63% |\n| Logistic Regression | 62% | 55% | 58% |\n| SVM | 57% | 63% | 60% |\n\nSimple average accuracy: 60.5%. But notice that gradient boosting excels in bear markets while random forest excels in bull markets.\n\nThe meta-learner discovers these conditional strengths. It assigns higher weights to gradient boosting when volatility indicators suggest bearish conditions and leans on random forest during low-volatility expansions. The stacked ensemble achieved 68% accuracy — better than any individual model.\n\nImplementation Steps:\n1. Split training data into K folds (typically 5)\n2. For each fold, train base models on remaining K-1 folds and generate predictions on the held-out fold\n3. Collect all out-of-fold predictions as features for the meta-learner\n4. Train the meta-learner on these predictions with the true labels\n5. For new data, run all base models and feed their predictions to the meta-learner\n\nOverfitting Prevention:\nThe critical insight is using out-of-fold predictions. If base models predict on their own training data, the meta-learner sees artificially good predictions and overfits. Out-of-fold predictions simulate genuine out-of-sample performance.\n\nWhen to Use Stacking vs. Simpler Methods:\n- Use simple averaging when base models have similar accuracy across all conditions\n- Use stacking when models have complementary strengths (different market regimes, asset classes, or time horizons)\n- Keep the meta-learner simple (Ridge or linear) to avoid second-level overfitting\n\nDive deeper into ensemble methods in our CFA Quantitative Methods course.