The three moving parts: prior, likelihood, posterior
Every Bayesian update has three moving parts. The prior, P(H), is your confidence in the hypothesis before seeing the evidence. The likelihood, P(E | H), is how probable the evidence would be if the hypothesis were true. The posterior, P(H | E), is your updated confidence after taking the evidence into account. These three quantities form the skeleton of every Bayesian argument, whether the argument is conducted with precise numbers or rough qualitative judgments.
These three quantities are distinct, and keeping them distinct is the first discipline of Bayesian reasoning. A common error is to confuse the likelihood with the posterior — treating 'the evidence is likely if H is true' as the same thing as 'H is likely given the evidence.' They are not the same, and confusing them leads to serious mistakes we'll see in later lessons. The likelihood tells you about the predictive power of the hypothesis; the posterior tells you about the status of the hypothesis after the evidence is in.
Consider a concrete illustration. Suppose you are wondering whether a coin is fair or double-headed. Your prior might be 99% confidence that it is fair (most coins are). The likelihood of heads on a single flip, given a fair coin, is 50%; given a double-headed coin, it is 100%. After observing heads, the posterior shifts toward 'double-headed,' but only modestly — because the prior was so strong. After ten heads in a row, the posterior shifts dramatically. The prior, the likelihood, and the posterior each play a different role in this drama, and the update only makes sense when all three are tracked separately.
A useful mnemonic: the prior is what you brought to the table, the likelihood is how well the hypothesis predicted what you saw, and the posterior is what you should believe now. If you can state all three for a given problem, you have the scaffolding for a Bayesian analysis. If you find yourself stating only one or two, something is missing.
- Prior: what's the starting confidence in H?
- Likelihood: how well does H predict the evidence?
- Posterior: what's the updated confidence in H given the evidence?
- Verify that you have stated all three, not just one or two.
- Check that you have not accidentally treated the likelihood as the posterior.
Takeaway: Prior, likelihood, and posterior are three different things — never collapse them into one.