When should I use F1 score instead of accuracy or AUC?

Question

AcadiFi · Accepted Answer

F1 = 2 × (Precision × Recall) / (Precision + Recall). Use F1 when: (1) classes are imbalanced, (2) you need a single threshold-specific metric, (3) false positives and false negatives have similar costs, (4) accuracy is misleading due to class skew.

Why harmonic mean: it penalizes extreme imbalance between P and R. A model with Precision=95%, Recall=5% has arithmetic mean 50% but F1 = 9.5% — correctly reflecting poor practical value.

Example: Bannockwood Credit Union evaluates a loan fraud model on an imbalanced dataset (0.8% fraud in 45,000 apps = 360 positives).

Model A: accuracy 99.2%, but flags everything as non-fraud → Precision=undefined, Recall=0%, F1=0. Accuracy is misleading.

Model B: Precision=0.58, Recall=0.47 → F1 = 2(0.58×0.47)/(0.58+0.47) = 0.52.

Model C: Precision=0.71, Recall=0.39 → F1 = 0.50.

Bannockwood picks Model B by F1, but chief risk officer notes missed-fraud cost 3x higher than investigation cost, so they use F2 score (weights recall 2x): F2_B=0.49, F2_C=0.42 — still B wins, but margin narrows.

When NOT to use F1:
- When the operating threshold isn't fixed yet (use AUC instead)
- When TN count matters (F1 ignores true negatives)
- When you need calibration (F1 doesn't assess probability quality; use Brier score or log loss)

CFA curriculum highlights: always report F1 alongside AUC on imbalanced financial data, and consider Fβ variants when error costs are asymmetric.

When should I use F1 score instead of accuracy or AUC?

Master Level II with our CFA Course

Related Questions

Practice Questions