2. The Importance of Being Bayes: Introducing Bayes' Theorem

Here is a fun fact: the inventor of Bayes' Theorem (Thomas Bayes) was not himself a Bayesian, as we understand it today. He simply saw the Bayesian Theorem as a way to calculate conditional probablities. So let's first have a look at Bayes' Theorem, the way it was intended by its creator.

For events

For events $A$ and $B$, we can write Bayes' Theorem as:

where

• $P(A)$ is the prior probability of $A$.
• $P(B|A)$ is the likelihood of $B$ given $A$.
• $P(B)$ is marginal probability or evidence, and,
• $P(A|B)$ is the posterior probability of $A$ given $B$.

The marginal likelihood can be expanded by summing up over all possible events ${A_i} \in \mathcal A$:

I appreciate that there is a lot to unpack here, and it doesn't get any simpler as we move from events to continuous random variables, and from conditional probabilities to Bayesian Inference. So let's take a moment to dive into an example of how to use Bayes' Theorem for a real-world example of conditional event estimation.

Example

Suppose that you have tested positive for COVID-19 using a lateral flow test, and you want to calculate the posterior probability that you actually have the disease. Let's introduce some notation first. The test can either be positive or negative, so we will call the event that the test is positive $t_+$, and, conversely, that it is negative $t_-$. Similarly, you can either be COVID-positive or COVID-negative, and we'll call those events $c_+$ and $c_-$, respectively:

• $t_+$ : Positive test
• $t_-$ : Negative test
• $c_+$ : COVID-positive
• $c_-$ : COVID-negative

If we encode this into Bayes' Theorem, we would like to find the posterior probability of having COVID-19, given that you just tested positive $P(c_+|t_+)$:

Let's unpack this. $P(c_+)$ is the prior probability of being COVID-positive. If we have no more information, let's assume that 10% of the population has COVID-19, so that $P(c_+)=0.1$.

The likelihood is a bit more involved, and we will have to look into the actual sensitivity and specificity of the lateral flow test. We'll have to take those numbers from somewhere, and this news article tells us that they are 76.8% and 99.7%, respectively. Another word for the sensitivity is the true positive rate, or, in other words, $P(t_+|c_+) = 0.768$. The specificity is also called the true negative rate, i.e. $P(t_-|c_-) = 0.997$.

The marginal likelihood expresses the likelihood of observing the data, considering all possible values of the variable of interest, in this case your COVID status. This can be written in the following way:

To calculate this, we need the two final ingredients, namely the prior probability of being COVID-negative $P(c_-)$ and the false positive rate of the lateral flow test $P(t_+|c_-)$. These can easily be obtained from the numbers we already have:

Putting it all together, we get:

or, 96.6%. Let's flip the calculations around and see what is the probability of being negative, given that you get a negative test $P(c_-|t_-)$:

or, 97.9%. This means that if you test negative using a lateral flow testing kit, you are almost certainly negative, and you can get on with your life.