What are the differences between grid search, random search, and Bayesian optimization for hyperparameter tuning in financial models?

Hyperparameter tuning selects the best configuration for a model's structural settings — values not learned from data but set before training. The three main approaches trade off thoroughness against computational efficiency.\n\nComparison:\n\n`mermaid\ngraph TD\n A[\"Hyperparameter Space\"] --> B[\"Grid Search\"]\n A --> C[\"Random Search\"]\n A --> D[\"Bayesian Optimization\"]\n B --> E[\"Tests every
combination on grid
Exhaustive but slow\"]\n C --> F[\"Samples randomly
Surprisingly effective
with many params\"]\n D --> G[\"Builds surrogate model
Explores intelligently
Fewest evaluations\"]\n E --> H{\"Budget: 100 evaluations\"}\n F --> H\n G --> H\n H --> I[\"Select best config
via cross-validation\"]\n`\n\nWorked Example:\nPinnacle Systematic has a gradient boosting model with these hyperparameters:\n\n| Hyperparameter | Range |\n|---|---|\n| n_estimators | [50, 100, 200, 500] |\n| learning_rate | [0.01, 0.05, 0.1, 0.2] |\n| max_depth | [3, 5, 7, 10] |\n| min_samples_leaf | [5, 10, 20, 50] |\n| subsample | [0.6, 0.8, 1.0] |\n\nGrid search: 4 x 4 x 4 x 4 x 3 = 768 combinations. At 2 minutes per 5-fold CV, that is 2,560 minutes (~43 hours).\n\nRandom search (100 iterations): Samples 100 random configurations. Research shows that with 100 random draws, you have a 95% probability of finding a configuration within the top 5% of all possibilities. Total time: ~333 minutes (~5.5 hours).\n\nBayesian optimization (50 iterations): Uses a Gaussian process to model the relationship between hyperparameters and validation performance. Each iteration selects the most promising configuration based on an acquisition function (expected improvement). Often finds the global optimum in 30-50 evaluations. Total time: ~170 minutes (~2.8 hours).\n\nResults:\n\n| Method | Best Validation Sharpe | Evaluations | Time |\n|---|---|---|---|\n| Grid | 1.42 | 768 | 43 hrs |\n| Random | 1.39 | 100 | 5.5 hrs |\n| Bayesian | 1.44 | 50 | 2.8 hrs |\n\nAvoiding Hyperparameter Overfitting:\n- Use nested cross-validation: outer loop evaluates generalization, inner loop tunes hyperparameters\n- Hold out a final test set that is never used during tuning\n- Prefer simpler configurations when multiple produce similar scores\n\nExplore hyperparameter tuning scenarios in our CFA Quantitative Methods course.