How do splines work in regression and what's the difference between natural vs cubic splines?

Question

AcadiFi · Accepted Answer

Spline regression fits a flexible non-linear function by joining together polynomial pieces (typically cubic) at specified points called knots. The resulting curve is smooth (continuous first and second derivatives) yet adapts to local data patterns — a powerful middle ground between global polynomials (too rigid or wiggly) and kernel smoothers (less interpretable).

Construction: suppose we want to model f(X) with knots at ξ₁, ξ₂, ..., ξₖ. A cubic spline has cubic polynomials on each interval [ξⱼ, ξⱼ₊₁], joined with continuity in the function value and its first two derivatives. This ensures smoothness without visual kinks.

Basis function representation: f(X) = β₀ + β₁X + β₂X² + β₃X³ + Σⱼ γⱼ(X − ξⱼ)³₊, where (X − ξⱼ)³₊ equals (X − ξⱼ)³ if X > ξⱼ, else 0. Each truncated cubic basis function contributes a smooth deviation after its knot, and the coefficients β and γ are estimated via ordinary least squares. This makes splines computationally easy.

Natural cubic splines add a constraint: the function is linear beyond the boundary knots. This prevents erratic behavior in the tails where data is sparse. Ordinary cubic splines can oscillate wildly at the extremes; natural splines are more reliable for extrapolation.

Knot placement matters. Fewer knots = smoother but may miss features. More knots = more flexible but risks overfitting. Common strategies: (1) place knots at quantiles of X; (2) use penalized regression splines with many knots plus a smoothness penalty (the GAM approach). The effective degrees of freedom, not the number of knots, controls complexity.

Example application: Fairwinds Asset Management models the non-linear relationship between yield curve slope and future bond returns using a natural cubic spline with 5 knots at quintiles. The resulting curve reveals threshold effects invisible to linear regression, improving predictive R² from 0.08 to 0.22. CFA Level II quant covers splines as a key non-linear modeling tool.

How do splines work in regression and what's the difference between natural vs cubic splines?

Master Level II with our CFA Course

Related Questions

Practice Questions