Chebyshev's Inequality

Definition of Chebyshev’s Inequality

Statement of Chebyshev’s Inequality

Equivalent Forms

Form 1: $P(|X - \mu| < \varepsilon) \geq 1 - \frac{\sigma^2}{\varepsilon^2}$

Form 2: $P(|X - \mu| \geq k\sigma) \leq \frac{1}{k^2}$, where $k > 0$

Intuitive Understanding of Chebyshev’s Inequality

Understanding: Chebyshev’s inequality gives an upper bound on the probability that a random variable deviates from its expected value. It tells us that the farther the values of a random variable deviate from the expectation, the smaller the probability of such a deviation.

Key Idea Behind the Proof

Key idea:

1. In the region $|X - \mu| \geq \varepsilon$, we have $(X - \mu)^2 \geq \varepsilon^2$
2. Therefore $\frac{(X - \mu)^2}{\varepsilon^2} \geq 1$
3. This inequality is used to bound the probability

Properties of Chebyshev’s Inequality

Generality

Property 1: Chebyshev’s inequality applies to random variables with any distribution, as long as their expectation and variance exist.

Conservativeness

Property 2: the upper bound given by Chebyshev’s inequality is usually conservative, that is, the actual probability may be far smaller than this upper bound.

Distribution-Free Character

Property 3: Chebyshev’s inequality places no special requirement on the distribution of the random variable and depends only on the expectation and the variance.

Applications of Chebyshev’s Inequality

Applications in Probability Estimation

Application 1: estimating the probability that a random variable deviates from its expectation

Example 1: let the expectation of a random variable $X$ be $0$ and its variance be $4$. Estimate an upper bound for $P(|X| \geq 3)$.

Solution: $P(|X| \geq 3) \leq \frac{4}{9} \approx 0.444$

Applications in Statistical Inference

Application 2: proving the law of large numbers

Application 3: estimating the precision of the sample mean

Applications in Quality Control

Application 4: estimating the range of fluctuation of product quality

Application 5: setting quality control standards

Limitations of Chebyshev’s Inequality

Conservativeness

Limitation 1: the upper bound given by Chebyshev’s inequality is usually conservative; for specific distributions there may be more precise estimates.

Incomplete Use of Information

Limitation 2: Chebyshev’s inequality uses only the information of the expectation and the variance, and does not use other features of the distribution.

Restricted Applicability

Limitation 3: for certain special distributions, there may be better inequalities.

Relations to Other Inequalities

Relation to Markov’s Inequality

Markov’s inequality: let $X$ be a non-negative random variable. Then for any $a > 0$: $P(X \geq a) \leq \frac{E(X)}{a}$

Relation: Chebyshev’s inequality is a special case of Markov’s inequality, obtained by replacing $X$ with $(X - \mu)^2$.

Relation to Hoeffding’s Inequality

Hoeffding’s inequality: for bounded random variables, Hoeffding’s inequality gives a more precise estimate.

Generalizations of Chebyshev’s Inequality

Multidimensional Chebyshev’s Inequality

Generalization 1: for multidimensional random variables, there is a corresponding Chebyshev’s inequality.

Conditional Chebyshev’s Inequality

Generalization 2: under given conditions, there is a corresponding Chebyshev’s inequality.

Exercises

Exercise 1

Use Chebyshev’s inequality to estimate an upper bound for $P(|X|\geq3)$ when $E(X)=0, D(X)=4$.

Exercise 2

Let the expectation of a random variable $X$ be $5$ and its variance be $9$. Use Chebyshev’s inequality to estimate a lower bound for $P(2 < X < 8)$.

Exercise 3

Let the expectation of a random variable $X$ be $10$ and its variance be $16$. Use Chebyshev’s inequality to estimate an upper bound for $P(|X - 10| \geq 8)$.

Exercise 4

Prove that for any random variable $X$, if $E(X) = \mu$ and $D(X) = \sigma^2$, then for any $k > 0$: $P(|X - \mu| \geq k\sigma) \leq \frac{1}{k^2}$

Exercise 5

Let a random variable $X$ follow the normal distribution $N(\mu, \sigma^2)$. Compare the upper bound given by Chebyshev’s inequality with the actual probability.

---

Summary

Symbols Used in This Article

---