Conditional probability and the Monty Hall problem

When you learn that something happened, how should a probability change?

Keep only the outcomes in which it happened and rescale them to total 1. How far a probability moves depends on how likely the observation was under each possibility, so the rule that produced the observation matters as much as the observation itself.

Before you start · 3 quick checks on equally likely outcomes, complements and probability trees, about 3 min

Definitions and rules

Conditional probability. For events \(A\) and \(B\) with \(P(B) \gt 0\), the probability of \(A\) given \(B\) is

\[P(A \mid B) = \frac{P(A \cap B)}{P(B)}.\]

It is the share of the probability of \(B\) that also lies in \(A\). When all outcomes are equally likely, it is the number of outcomes in \(A \cap B\) divided by the number of outcomes in \(B\).

Multiplication rule. When \(P(A) \gt 0\),

\[P(A \cap B) = P(B \mid A)\,P(A).\]

On a probability tree, this is the product of the probabilities along a branch.

Law of total probability. Let \(A_1, \dots, A_k\) be a partition: events with positive probabilities, no two of which can happen together, one of which always happens. Then for every event \(B\),

\[P(B) = P(B \mid A_1)\,P(A_1) + \dots + P(B \mid A_k)\,P(A_k).\]

Bayes' rule. With the same partition and \(P(B) \gt 0\), for each \(i\),

\[P(A_i \mid B) = \frac{P(B \mid A_i)\,P(A_i)}{P(B)}.\]

Here \(P(A_i)\) is the prior, \(P(B \mid A_i)\) is the likelihood of what was observed, and \(P(A_i \mid B)\) is the posterior. In words: posterior equals prior times likelihood, divided by the total over all cases.

Fine print: events of probability 0, and independence

The ratio definition used here requires \(P(B) \gt 0\): the denominator cannot be 0. Every event this page conditions on has positive probability. (More advanced courses condition on some events of probability 0, such as a continuous random variable taking one exact value, by other definitions that this page does not need.)

Events \(A\) and \(B\) are independent when \(P(A \cap B) = P(A)\,P(B)\). If \(P(B) \gt 0\), this is the same as \(P(A \mid B) = P(A)\): learning \(B\) does not move the probability of \(A\).

Bayes' rule follows from the definition: the multiplication rule gives the numerator \(P(A_i \cap B) = P(B \mid A_i)\,P(A_i)\), and the law of total probability gives the denominator.

1 See

Stay or switch?

The game

There are \(n\) doors, with \(3 \le n \le 10\). A car is behind one door, placed at random with every door equally likely, and a goat is behind each of the others. You pick door 1. The host then opens doors by one of three rules:

A: the host knows. He knows where the car is. He opens \(n - 2\) of the other \(n - 1\) doors, all with goats, so that exactly one other door stays closed. If the car is behind door 1, he picks the door that stays closed at random, each of the \(n - 1\) other doors equally likely.
B: the host does not know. He picks \(n - 2\) of the other \(n - 1\) doors at random, every set of \(n - 2\) equally likely, and opens them. If one of them has the car, the game is discarded. Only games in which every opened door shows a goat are counted.
C: the host opens one door. He knows where the car is and opens just one door: a goat door other than door 1, picked at random among those. Switching then moves you to one of the \(n - 2\) other closed doors, picked at random.

You then stay with door 1 or switch. You win if the door you end with hides the car. With 3 doors, rules A and C describe the same game.

Set the number of doors and the rule. The figure shows the first game of a run as you would see it. Then choose once, stay or switch. The page plays 10,000 games with your choice and reports how often each strategy won.

How the games are simulated

Each run is 10,000 games drawn from one seed of a seeded random number generator, so the same games come back on every visit; the caption names the seed, and Play 10,000 new games moves to the next one. In each game the car's door is drawn first, then the host's random choices. Your choice is applied to every game, and the page also records what the other choice would have won.

For rule B, picking \(n - 2\) of the other \(n - 1\) doors at random is the same as picking at random the one door he leaves closed, and that is how the page draws it. The rates are shares of the kept games.

A simulated rate is an estimate. Its typical error, the standard error, is \(\sqrt{p(1-p)/m}\) for a true probability \(p\) and \(m\) counted games. With all 10,000 games counted it is at most half a percentage point. Under rule B with 10 doors only about 2,000 games are kept, and it grows to about 1.1 percentage points.

2 Try

Likelihoods decide the answer

Goals: enter the likelihoods

Three doors, you pick door 1, and the host opens door 3, which shows a goat. Call this event \(E\), and let \(C_i\) be the event that the car is behind door \(i\). For the host in each goal, enter the likelihood \(P(E \mid C_i)\) for each door, as a fraction such as 1/2 or a decimal. The page multiplies each likelihood by the prior \(\tfrac{1}{3}\) and divides by the total, which is Bayes' rule.

Key point Bayes' rule turns likelihoods into posteriors. Two hosts who open the same door can have different likelihoods, and then the same observation leads to different probabilities.

3 Prove

Bayes' rule for three doors

Theorem

Assumptions.

Three doors. The car is behind door 1, 2 or 3, each with probability \(\tfrac{1}{3}\). You pick door 1.
The host knows where the car is. He opens door 2 or door 3, never your door and never the door with the car.
When both doors 2 and 3 hide goats, he opens each with probability \(\tfrac{1}{2}\).

Conclusion. Let \(C_i\) be the event that the car is behind door \(i\) and \(O_3\) the event that the host opens door 3. Then \(P(C_2 \mid O_3) = \tfrac{2}{3}\) and \(P(C_1 \mid O_3) = \tfrac{1}{3}\): given that door 3 is opened, switching to door 2 wins with probability \(\tfrac{2}{3}\).

Scratch work, not part of the proof.

Same door opened, three hosts

In each row you pick door 1 and see door 3 opened with a goat behind it. Only the host's rule differs. The numbers come from the same three steps as the proof: likelihoods, total probability, Bayes' rule.

Key point Posterior is proportional to prior times likelihood. This host opens door 3 with probability 1 when the car is behind door 2 and with probability \(\tfrac{1}{2}\) when it is behind door 1. That factor of 2 turns equal priors into \(\tfrac{2}{3}\) against \(\tfrac{1}{3}\).

4 Apply

Problems

Each problem has hints and a full solution. In the door problems the car is placed at random, every door equally likely.