Evaluation & RL
Precision-Recall Curve
Drag the threshold slider to see how precision and recall respond, especially under severe class imbalance where ROC curves look falsely perfect.
Drag the threshold slider to see how precision and recall respond, especially under severe class imbalance where ROC curves look falsely perfect.
Stage 1 of 3: Balanced classes
With equal numbers of positives and negatives, both ROC and PR curves look reasonable. The PR curve traces the trade-off between finding every positive (recall) and only making correct positive predictions (precision). The threshold is 0.50. Accuracy is 72%.
- Actually positive
- Actually negative
- Your threshold
- Misclassified
Score above this is called positive.
How far apart the score distributions sit.
Confusion Matrix
With equal numbers of positives and negatives, both ROC and PR curves look reasonable. The PR curve traces the trade-off between finding every positive (recall) and only making correct positive predictions (precision).
The Illusion of ROC
Receiver Operating Characteristic (ROC) curves are the default choice for evaluating classifiers. But they have a blind spot: class imbalance. When negatives are extremely common and positives are rare, a model can guess ''negative'' almost every time, keep its False Positive Rate near zero, and produce a beautiful ROC curve — while completely failing to find the rare positives you care about.
Precision-Recall (PR) curves fix this by entirely ignoring true negatives. They only measure how well the model handles the positive class.
Use the controls above to introduce class imbalance and watch how the two curves respond.
Reference
- Precision
- TP / (TP + FP) — of what you flagged, how much was right
- Recall
- TP / (TP + FN) — of what was there, how much you found
- ROC
- every threshold at once, plotted as (FPR, TPR)
- PR Curve
- every threshold at once, plotted as (Recall, Precision)
Break it on purpose
When positives are rare (high class imbalance), the ROC curve can still look near-perfect because true negatives dominate. But the Precision-Recall curve collapses, revealing the model is actually performing poorly on the minority class.