How is a ROC curve actually constructed from model scores?

Question

AcadiFi · Accepted Answer

ROC (Receiver Operating Characteristic) plots TPR (y-axis) vs. FPR (x-axis) as the classification threshold sweeps from 1.0 down to 0.0. Construction steps: 1. Score every observation with the model (probability 0 to 1). 2. Sort observations by score descending. 3. For each unique threshold, compute TPR = TP/P and FPR = FP/N. 4. Plot (FPR, TPR) points; connect them. 5. Curve starts at (0,0) (threshold=1, nothing flagged) and ends at (1,1) (threshold=0, everything flagged). ```mermaid flowchart TD A[Score all
observations] --> B[Sort by
score desc] B --> C[Sweep
thresholds] C --> D[Compute TPR
and FPR] D --> E[Plot
points] E --> F[Connect to
form curve] ``` Example: Solvermark Hedge Fund's equity-momentum classifier produces scores on 5,000 stocks (300 'winners'). At threshold 0.80, flags 200 stocks with 160 winners → TPR=160/300=0.53, FPR=40/4700=0.009. At threshold 0.50, flags 1,400 stocks with 240 winners → TPR=0.80, FPR=0.247. At threshold 0.20, flags 3,800 stocks with 295 winners → TPR=0.98, FPR=0.746. Plotting (0.009,0.53), (0.247,0.80), (0.746,0.98) plus endpoints and interpolating yields the ROC curve. AUC is numerically integrated — Solvermark's model AUC=0.78. Interpretation tips: steep initial rise (upper-left corner) = model finds positives without many FPs, ideal. Diagonal = random classifier. Below diagonal = worse than random (can invert predictions). Different curves may cross — a model better at low FPR may be worse at high FPR, so choose based on operating region relevant to your business.

How is a ROC curve actually constructed from model scores?

Master Level II with our CFA Course

Related Questions

Practice Questions