Conditional Probability, the Law of Total Probability, and Bayes' Formula

Conditional Probability

Definition of Conditional Probability

Intuitive Understanding of Conditional Probability

Understanding: the conditional probability $P(A|B)$ gives the probability that event $A$ occurs given that event $B$ has occurred. It reflects the influence of the occurrence of event $B$ on the probability of occurrence of event $A$.

Properties of Conditional Probability

Property 1: $0 \leq P(A|B) \leq 1$

Property 2: $P(\Omega|B) = 1$, $P(\varnothing|B) = 0$

Property 3: if $A_1, A_2, \dots, A_n$ are pairwise mutually exclusive, then: $P(A_1 \cup A_2 \cup \dots \cup A_n|B) = P(A_1|B) + P(A_2|B) + \dots + P(A_n|B)$

Property 4: $P(\overline{A}|B) = 1 - P(A|B)$

Multiplication Formula for Conditional Probability

Examples of Conditional Probability

Example 1: drawing one card at random from a deck

• Let $A = \{\text{a heart is drawn}\}$
• Let $B = \{\text{a red card is drawn}\}$
• $P(A) = \frac{13}{52} = \frac{1}{4}$
• $P(A|B) = \frac{P(A \cap B)}{P(B)} = \frac{\frac{13}{52}}{\frac{26}{52}} = \frac{1}{2}$

Example 2: rolling two dice

• Let $A = \{\text{the first die shows 6}\}$
• Let $B = \{\text{the sum of the two dice is 8}\}$
• $P(A) = \frac{1}{6}$
• $P(B) = \frac{5}{36}$ (there are 5 cases: (2,6), (3,5), (4,4), (5,3), (6,2))
• $P(A \cap B) = \frac{1}{36}$ (only the case (6,2))
• $P(A|B) = \frac{\frac{1}{36}}{\frac{5}{36}} = \frac{1}{5}$

The Law of Total Probability

Definition of the Law of Total Probability

Intuitive Understanding of the Law of Total Probability

Understanding: the law of total probability decomposes the probability of a complex event $B$ into a weighted sum of the probabilities of occurrence under different conditions. It embodies the idea of “divide and conquer”.

Proof of the Law of Total Probability

Proof:

1. $B = B \cap \Omega = B \cap (A_1 \cup A_2 \cup \dots \cup A_n)$
2. $= (B \cap A_1) \cup (B \cap A_2) \cup \dots \cup (B \cap A_n)$
3. Since $A_1, A_2, \dots, A_n$ are pairwise mutually exclusive, $(B \cap A_1), (B \cap A_2), \dots, (B \cap A_n)$ are also pairwise mutually exclusive
4. Therefore $P(B) = \sum_{i=1}^n P(B \cap A_i)$
5. By the multiplication formula, $P(B \cap A_i) = P(A_i)P(B|A_i)$
6. Hence $P(B) = \sum_{i=1}^n P(A_i)P(B|A_i)$

Example of the Law of Total Probability

Example 3: there are three boxes containing 2, 3, and 5 balls respectively; one box is chosen at random and one ball is drawn from it

• Let $A_i = \{\text{box } i \text{ is chosen}\}$, $i = 1, 2, 3$
• Let $B = \{\text{a white ball is drawn}\}$
• $P(A_1) = P(A_2) = P(A_3) = \frac{1}{3}$
• Suppose box 1 contains only white balls, box 2 contains only white balls, and box 3 contains 3 white and 2 black balls
• $P(B|A_1) = 1$, $P(B|A_2) = 1$, $P(B|A_3) = \frac{3}{5}$
• $P(B) = \frac{1}{3} \times 1 + \frac{1}{3} \times 1 + \frac{1}{3} \times \frac{3}{5} = \frac{1}{3} + \frac{1}{3} + \frac{1}{5} = \frac{11}{15}$

Bayes’ Formula

Definition of Bayes’ Formula

Intuitive Understanding of Bayes’ Formula

Understanding: Bayes’ formula is used to compute the probability that event $A_i$ occurs given that event $B$ has occurred. It embodies the idea of “inverse probability”, that is, inferring the cause from the result.

Proof of Bayes’ Formula

Proof:

1. By the definition of conditional probability, $P(A_i|B) = \frac{P(A_i \cap B)}{P(B)}$
2. By the multiplication formula, $P(A_i \cap B) = P(A_i)P(B|A_i)$
3. By the law of total probability, $P(B) = \sum_{j=1}^n P(A_j)P(B|A_j)$
4. Therefore $P(A_i|B) = \frac{P(A_i)P(B|A_i)}{\sum_{j=1}^n P(A_j)P(B|A_j)}$

Examples of Bayes’ Formula

Example 4: continuing the problem of Example 3, find the probability that the ball comes from box 3 given that a white ball is drawn

• $P(A_3|B) = \frac{P(A_3)P(B|A_3)}{P(B)}$
• $= \frac{\frac{1}{3} \times \frac{3}{5}}{\frac{11}{15}} = \frac{\frac{1}{5}}{\frac{11}{15}} = \frac{3}{11}$

Example 5: the products of a factory are produced by three workshops. The first workshop produces 40%, the second 35%, and the third 25%. The defective rate is 2% in the first workshop, 1% in the second, and 3% in the third. One product is drawn at random and is found to be defective. Find the probability that it comes from the first workshop.

Solution:

1. Let $A_i = \{\text{the product comes from the } i \text{-th workshop}\}$, $i = 1, 2, 3$
2. Let $B = \{\text{the product is defective}\}$
3. $P(A_1) = 0.4$, $P(A_2) = 0.35$, $P(A_3) = 0.25$
4. $P(B|A_1) = 0.02$, $P(B|A_2) = 0.01$, $P(B|A_3) = 0.03$
5. $P(B) = 0.4 \times 0.02 + 0.35 \times 0.01 + 0.25 \times 0.03 = 0.008 + 0.0035 + 0.0075 = 0.019$
6. $P(A_1|B) = \frac{0.4 \times 0.02}{0.019} = \frac{0.008}{0.019} \approx 0.421$

Independence of Events

Definition of Independence of Events

Equivalent Condition for Independence of Events

Equivalent condition: events $A$ and $B$ are independent if and only if: $P(A|B) = P(A) \quad \text{or} \quad P(B|A) = P(B)$

Properties of Independence of Events

Property 1: if $A$ and $B$ are independent, then $A$ and $\overline{B}$ are independent, $\overline{A}$ and $B$ are independent, and $\overline{A}$ and $\overline{B}$ are independent.

Property 2: if $A$ and $B$ are independent and $P(B) > 0$, then $P(A|B) = P(A)$.

Property 3: the impossible event is independent of any event.

Independence of Several Events

Example of Independence

Example 6: rolling two dice

• Let $A = \{\text{the first die shows 6}\}$
• Let $B = \{\text{the second die shows 6}\}$
• $P(A) = \frac{1}{6}$, $P(B) = \frac{1}{6}$
• $P(A \cap B) = \frac{1}{36} = \frac{1}{6} \times \frac{1}{6} = P(A)P(B)$
• Therefore $A$ and $B$ are independent

Exercises

Exercise 1

Given $P(A) = 0.3, P(B|A) = 0.5$, find $P(A \cap B)$.

Exercise 2

There are three boxes containing 2, 3, and 5 balls respectively; one box is chosen at random and one ball is drawn from it. Find the probability that the ball comes from box 3 given that a white ball is drawn (box 3 contains 3 white and 2 black balls, and the others contain only white balls).

Exercise 3

Given $P(A) = 0.6, P(B) = 0.5, P(A \cap B) = 0.3$, determine whether A and B are independent.

Exercise 4

The products of a factory are produced by two workshops. The first workshop produces 60% and the second 40%. The defective rate is 1% in the first workshop and 2% in the second. One product is drawn at random and is found to be defective. Find the probability that it comes from the first workshop.

Exercise 5

Roll two dice. Let $A = \{\text{the first die shows an even number}\}$ and $B = \{\text{the sum of the two dice is 7}\}$. Determine whether $A$ and $B$ are independent.