a minipost

Some sports have doping issues and there is drug testing in many of these. There's always an issue of how good the tests are, but there's also an issue of how fair they are and how good they have to be to be fair. The best way to illustrate this is with an example.

Consider a sport with 1000 players - roughly the size of major league baseball in the US . Suppose steroid abuse is an issue and there's a test with 95% accuracy. And assume somehow it's known (or strongly suspected) that about 5% of the players are abusers. These are artificial numbers, but it's just an illustration.

What if the players are tested? We expect 50 to be abusers and 950 to be clean. The test is 95% accurate.. so 48 (rounding up from 47.5) are correctly identified. So far so good. Now consider the 950 clean players.. 5% or 48 (rounding up again) are incorrectly labeled as abusers. So of 1000 players there would be 96 positives and only a fifty-fifty chance of being right. Careers can be be ruined .. the stakes are high.

A way to sort out these conditional probabilities was described the The Reverend Thomas Bayes in the mid 1700s in a paper called *Essay Towards Solving a Problem in the Doctrine of Chances*. The good Reverend described the relationship between the probability an athlete is an abuser given a failed test and the probability an abuser fails the test. Consider these probabilities (apologies for notation)

P(A) = probability a drug test is positive

P(B) = probability a player is an abuser

P(A|B) = probability an abusing player tests positive

P(B|A) = probability a player has taken drugs if their test is positive

Perhaps the most important takeaway is that P(A|B) and P(B|A) are **not** the same! It's a very common prosecutor trick to fool the jury into thinking they are - so common that it's called *the prosecutor's fallacy*. But to the problem at hand.

We want to know P(B|A). We already know P(B) or at least we have the 5% estimate. So P(B) = 0.05. This also gives us P(not B) = (1 - 0.05) = 0.95 as P(B) and P(not B) must add to 1.0. We also have 47.5 (I'll go with the fractional athlete here) of 1000 drug free athletes tested positive, so P(A|not B) = 0.0475.

Bayes derived a formula showing how all of these relate. Sticking with our notation:

P(B|A) = {P(A|B) * * P(B)} / {P(A/B)* ** P(B) + P(A|not B)* * P(not B)}

= {0.95 * *0.05} / {0.95* * 0.05 + 0.0475 * 0.95}

= 0.513

This is a very unacceptable test! Although 95% accuracy sounds good on the surface, it only finds a bit better than half the abusers and incorrectly labels a lot of players. For fun try a 99% probability and a perfect 100% to see what's happening.

You can have much more complex conditionals, but this is the general idea. You really have to think things through a bit..

## Comments

You can follow this conversation by subscribing to the comment feed for this post.