Is AUC really the probability that a positive outranks a negative? How is that equivalent to curve area?

Question

AcadiFi · Accepted Answer

Yes — AUC has two equivalent definitions that coincide mathematically.

Definition 1 (geometric): Integral of TPR with respect to FPR from 0 to 1.
Definition 2 (probabilistic): P(score_positive > score_negative) for a randomly drawn positive and negative.

Proof sketch (Mann-Whitney U): For continuous scores, consider all P×N pairs. AUC = (1/(P×N)) × Σ I(score_i > score_j) where i ranges over positives, j over negatives. This is the normalized Mann-Whitney U statistic. It equals the area under the empirical ROC because each step along the ROC reflects whether you've 'seen' another correctly-ranked positive vs negative pair.

Example: Framerton Asset Management tests a long/short ranking model on 100 stocks (20 winners, 80 losers). Out of 20×80 = 1,600 pairs, model correctly ranks winner-above-loser in 1,248 pairs, ties in 32 pairs (counted 0.5 each). AUC = (1248 + 16)/1600 = 0.79.

This is exactly the integrated ROC area. At threshold= highest winner score, you move up TPR by 1/20 = 0.05 for each winner without adding FPR — contributing 0.05 × 0 to area. Each loser ranked above a remaining winner shifts FPR right by 1/80 = 0.0125 at whatever TPR level, losing rectangle area equal to pairs that failed.

Practical consequence: AUC is invariant to monotonic score transformations (log, exp, rank), only cares about ordering. Framerton can use raw scores, z-scores, or decile ranks and AUC is unchanged. This makes AUC a robust model comparison tool across different probability calibration schemes.

Is AUC really the probability that a positive outranks a negative? How is that equivalent to curve area?

Master Level II with our CFA Course

Related Questions

Practice Questions