Definition and Properties of Probability

Definition of Probability

Intuitive Definition of Probability

Statistical Definition of Probability

Classical Probability Model

Definition of the Classical Probability Model

Probability Formula for the Classical Probability Model

Examples of the Classical Probability Model

Example 1: tossing a fair coin

• Sample space: $\Omega = \{\text{heads}, \text{tails}\}$
• $P(\{\text{heads}\}) = \frac{1}{2}$
• $P(\{\text{tails}\}) = \frac{1}{2}$

Example 2: rolling a fair die

• Sample space: $\Omega = \{1, 2, 3, 4, 5, 6\}$
• $P(\{\text{an even number appears}\}) = \frac{3}{6} = \frac{1}{2}$
• $P(\{\text{a number greater than 3 appears}\}) = \frac{3}{6} = \frac{1}{2}$

Example 3: randomly drawing $r$ balls from $n$ balls

• Sample space: all possible combinations of $r$ balls
• $|\Omega| = C_n^r = \frac{n!}{r!(n-r)!}$
• $P(A) = \frac{|A|}{C_n^r}$

Geometric Probability Model

Definition of the Geometric Probability Model

Probability Formula for the Geometric Probability Model

Examples of the Geometric Probability Model

Example 4: choosing a point at random on the interval $[0, 1]$

• Sample space: $\Omega = [0, 1]$
• $P(\text{landing in } [a, b]) = \frac{b-a}{1} = b-a$

Example 5: choosing a point at random inside the unit circle

• Sample space: all points inside the unit circle
• $P(\text{a point whose distance from the center is less than } r \text{ is obtained}) = \frac{\pi r^2}{\pi} = r^2$

Example 6: choosing a point at random inside the unit square

• Sample space: all points inside the unit square
• $P(\text{landing in the first quadrant}) = \frac{1}{4}$

Basic Properties of Probability

Non-negativity

Property 1: for any event $A$, $0 \leq P(A) \leq 1$.

Normalization

Property 2: $P(\Omega) = 1$, $P(\varnothing) = 0$.

Finite Additivity

Property 3: if the events $A_1, A_2, \dots, A_n$ are pairwise mutually exclusive, then: $P(A_1 \cup A_2 \cup \dots \cup A_n) = P(A_1) + P(A_2) + \dots + P(A_n)$

Probability of the Complement Event

Property 4: $P(\overline{A}) = 1 - P(A)$.

Monotonicity

Property 5: if $A \subset B$, then $P(A) \leq P(B)$.

Addition Formula for Probability

General Addition Formula

Generalization of the Addition Formula

Generalization: for any $n$ events $A_1, A_2, \dots, A_n$: $P(A_1 \cup A_2 \cup \dots \cup A_n) = \sum_{i=1}^n P(A_i) - \sum_{1 \leq i < j \leq n} P(A_i \cap A_j) + \sum_{1 \leq i < j < k \leq n} P(A_i \cap A_j \cap A_k) - \dots + (-1)^{n-1} P(A_1 \cap A_2 \cap \dots \cap A_n)$

Proof of the Addition Formula

Proof:

1. $A \cup B = A \cup (B - A)$
2. $A$ and $B - A$ are mutually exclusive
3. Hence $P(A \cup B) = P(A) + P(B - A)$
4. Also $B = (B - A) \cup (A \cap B)$
5. Hence $P(B) = P(B - A) + P(A \cap B)$
6. Therefore $P(B - A) = P(B) - P(A \cap B)$
7. Substituting gives $P(A \cup B) = P(A) + P(B) - P(A \cap B)$

Examples

Example 7: tossing a coin twice

Problem: toss a fair coin twice and find the probability of the event that exactly one head appears.

Solution:

1. Sample space: $\Omega = \{(\text{H},\text{H}), (\text{H},\text{T}), (\text{T},\text{H}), (\text{T},\text{T})\}$
2. Event $A = \{\text{exactly one head appears}\} = \{(\text{H},\text{T}), (\text{T},\text{H})\}$
3. $P(A) = \frac{2}{4} = \frac{1}{2}$

Example 8: rolling a die

Problem: roll a fair die and find the probability of getting an even number or a number greater than 3.

Solution:

1. Let $A = \{\text{an even number appears}\} = \{2, 4, 6\}$
2. Let $B = \{\text{a number greater than 3 appears}\} = \{4, 5, 6\}$
3. $A \cap B = \{4, 6\}$
4. $P(A) = \frac{3}{6} = \frac{1}{2}$
5. $P(B) = \frac{3}{6} = \frac{1}{2}$
6. $P(A \cap B) = \frac{2}{6} = \frac{1}{3}$
7. $P(A \cup B) = P(A) + P(B) - P(A \cap B) = \frac{1}{2} + \frac{1}{2} - \frac{1}{3} = \frac{2}{3}$

Exercises

Exercise 1

Toss a fair coin twice and find the probability of the event that exactly one head appears.

Exercise 2

Given $P(A) = 0.4, P(B) = 0.5, P(A \cap B) = 0.2$, find $P(A \cup B)$.

Exercise 3

Roll a fair die and find the probability of getting an even number or a number greater than 3.

Exercise 4

Choose a point at random on the interval $[0, 1]$ and find the probability of landing in $[0.3, 0.7]$.

Exercise 5

Prove that $P(\overline{A}) = 1 - P(A)$.