About

I usually get confused about these ideas so I think it’s a good idea to get it clear & write it down.

Question

So we have a test set S, with a number of P positives and N negatives.

And we have a prediction method which assigns every sample with a label p and n.

True positive is the predicted positive and it’s true!

To give an evaluation of this methods:

  • MCC? AUC?

  • recall? precision?

  • sensitivity? specificity?

  • others?

Why bother using so many terminologies or methodologies?

I’ll get them straight one by one, in an arbitrary order.

Sensitivity & Specificity

Sensitivity is

\[ Sensitivity=\frac{TP}{P} \]

Specificity is

\[ Specificity=\frac{TN}{N} \]

Precision & Recall

Precision is

\[ Precision=\frac{TP}{p} \]

  • ratio of TP in all predicted p,

  • reduce the p size will increase precision

Recall is

\[ Recall=\frac{TP}{P} \]

  • ratio of TP in actual P

  • equivalent to Sensitivity

  • predict all as p, recall will reach 100%, and increase FP! So recall is not very useful while used alone.

MCC

AUC

Summary

It’s confusing because:

  • Recall = Sensitivity

but precision has nothing to do with specificity!

Since the four terms appear in pairs and meanings are designed (in some way) to appeear opposite for Sensitivity/specificity and precision/recall, and infact there are 3 meanings for the 4 term! oooooh you know your brain is gonna get fucked after a while!

Actually there are 12 terms for this, but not all of them are frequently used.

Why don’t people just stick to one thing?