Interval Estimation and Confidence Intervals

Basic Concepts of Interval Estimation

Definition of Interval Estimation

Intuitive Understanding of Confidence Intervals

Understanding: a confidence interval gives the range of values the parameter may take. A confidence level $1 - \alpha$ means that over a large number of repeated samplings, $100(1-\alpha)\%$ of the confidence intervals contain the true value.

Properties of Confidence Intervals

Property 1: the confidence interval is random, while the true value is fixed Property 2: the higher the confidence level, the wider the confidence interval Property 3: the larger the sample size, the narrower the confidence interval

Methods for Constructing Confidence Intervals

Pivotal Quantity Method

Idea: construct a statistic that involves both the parameter and the sample, whose distribution is known and does not depend on the parameter.

Steps:

1. Construct the pivotal quantity $G(X_1, X_2, \dots, X_n; \theta)$
2. Determine the distribution of the pivotal quantity
3. Find constants $a, b$ such that $P(a \leq G \leq b) = 1 - \alpha$
4. Solve the inequality $a \leq G \leq b$ to obtain the confidence interval

Confidence Interval for the Mean of a Normal Population

When the Variance Is Known

Population: $X \sim N(\mu, \sigma^2)$, with $\sigma^2$ known

Pivotal quantity: $Z = \frac{\overline{X} - \mu}{\sigma/\sqrt{n}} \sim N(0,1)$

Confidence interval: $P(-z_{\alpha/2} \leq \frac{\overline{X} - \mu}{\sigma/\sqrt{n}} \leq z_{\alpha/2}) = 1 - \alpha$

Solving gives: $\mu \in [\overline{X} - z_{\alpha/2}\frac{\sigma}{\sqrt{n}}, \overline{X} + z_{\alpha/2}\frac{\sigma}{\sqrt{n}}]$

When the Variance Is Unknown

Population: $X \sim N(\mu, \sigma^2)$, with $\sigma^2$ unknown

Pivotal quantity: $T = \frac{\overline{X} - \mu}{S/\sqrt{n}} \sim t(n-1)$

Confidence interval: $P(-t_{\alpha/2}(n-1) \leq \frac{\overline{X} - \mu}{S/\sqrt{n}} \leq t_{\alpha/2}(n-1)) = 1 - \alpha$

Solving gives: $\mu \in [\overline{X} - t_{\alpha/2}(n-1)\frac{S}{\sqrt{n}}, \overline{X} + t_{\alpha/2}(n-1)\frac{S}{\sqrt{n}}]$

Confidence Interval for the Variance of a Normal Population

Population: $X \sim N(\mu, \sigma^2)$

Pivotal quantity: $\chi^2 = \frac{(n-1)S^2}{\sigma^2} \sim \chi^2(n-1)$

Confidence interval: $P(\chi^2_{1-\alpha/2}(n-1) \leq \frac{(n-1)S^2}{\sigma^2} \leq \chi^2_{\alpha/2}(n-1)) = 1 - \alpha$

Solving gives: $\sigma^2 \in [\frac{(n-1)S^2}{\chi^2_{\alpha/2}(n-1)}, \frac{(n-1)S^2}{\chi^2_{1-\alpha/2}(n-1)}]$

Confidence Interval for the Difference of the Means of Two Normal Populations

When the Variances Are Known

Population: $X \sim N(\mu_1, \sigma_1^2)$, $Y \sim N(\mu_2, \sigma_2^2)$, with $\sigma_1^2, \sigma_2^2$ known

Pivotal quantity: $Z = \frac{(\overline{X} - \overline{Y}) - (\mu_1 - \mu_2)}{\sqrt{\frac{\sigma_1^2}{n_1} + \frac{\sigma_2^2}{n_2}}} \sim N(0,1)$

Confidence interval: $(\mu_1 - \mu_2) \in [(\overline{X} - \overline{Y}) - z_{\alpha/2}\sqrt{\frac{\sigma_1^2}{n_1} + \frac{\sigma_2^2}{n_2}}, (\overline{X} - \overline{Y}) + z_{\alpha/2}\sqrt{\frac{\sigma_1^2}{n_1} + \frac{\sigma_2^2}{n_2}}]$

When the Variances Are Unknown but Equal

Population: $X \sim N(\mu_1, \sigma^2)$, $Y \sim N(\mu_2, \sigma^2)$

Pivotal quantity: $T = \frac{(\overline{X} - \overline{Y}) - (\mu_1 - \mu_2)}{S_p\sqrt{\frac{1}{n_1} + \frac{1}{n_2}}} \sim t(n_1 + n_2 - 2)$

where $S_p^2 = \frac{(n_1-1)S_1^2 + (n_2-1)S_2^2}{n_1 + n_2 - 2}$

Confidence interval: $(\mu_1 - \mu_2) \in [(\overline{X} - \overline{Y}) - t_{\alpha/2}(n_1 + n_2 - 2)S_p\sqrt{\frac{1}{n_1} + \frac{1}{n_2}}, (\overline{X} - \overline{Y}) + t_{\alpha/2}(n_1 + n_2 - 2)S_p\sqrt{\frac{1}{n_1} + \frac{1}{n_2}}]$

Confidence Interval for the Ratio of the Variances of Two Normal Populations

Population: $X \sim N(\mu_1, \sigma_1^2)$, $Y \sim N(\mu_2, \sigma_2^2)$

Pivotal quantity: $F = \frac{S_1^2/\sigma_1^2}{S_2^2/\sigma_2^2} \sim F(n_1-1, n_2-1)$

Confidence interval: $P(F_{1-\alpha/2}(n_1-1, n_2-1) \leq \frac{S_1^2/\sigma_1^2}{S_2^2/\sigma_2^2} \leq F_{\alpha/2}(n_1-1, n_2-1)) = 1 - \alpha$

Solving gives: $\frac{\sigma_1^2}{\sigma_2^2} \in [\frac{S_1^2}{S_2^2}\frac{1}{F_{\alpha/2}(n_1-1, n_2-1)}, \frac{S_1^2}{S_2^2}\frac{1}{F_{1-\alpha/2}(n_1-1, n_2-1)}]$

Applications of Confidence Intervals

Determining the Sample Size

Problem: given a confidence level and a precision requirement, determine the sample size needed.

Example 1: for the estimation of the mean of a normal population, given the confidence level $1-\alpha$ and the precision $\delta$, find the sample size $n$.

Solution:

1. Length of the confidence interval: $2z_{\alpha/2}\frac{\sigma}{\sqrt{n}} \leq 2\delta$
2. Solving gives: $n \geq (\frac{z_{\alpha/2}\sigma}{\delta})^2$

Interpretation of Confidence Intervals

Correct interpretation: over a large number of repeated samplings, $100(1-\alpha)\%$ of the confidence intervals contain the true value.

Incorrect interpretation: the true value has a probability of $100(1-\alpha)\%$ of falling inside the confidence interval.

Choice of Confidence Interval

Two-sided confidence interval: suitable for the general case One-sided confidence interval: suitable for directional problems

Example 2: estimating the lower bound of the lifetime of a product

• One-sided confidence interval: $[\hat{\theta}_L, +\infty)$
• $P(\theta \geq \hat{\theta}_L) = 1 - \alpha$

Properties of Confidence Intervals

Relation Between the Confidence Level and the Length of the Interval

Relation: the higher the confidence level, the longer the confidence interval.

Reason: the higher the confidence level, the larger the probability that the true value is covered, so the interval must be wider.

Relation Between the Sample Size and the Length of the Interval

Relation: the larger the sample size, the shorter the confidence interval.

Reason: the larger the sample size, the more accurate the estimate, so the interval is narrower.

Symmetry of Confidence Intervals

Normal distribution: confidence intervals are usually symmetric Other distributions: confidence intervals may be asymmetric

Confidence Intervals for Common Distributions

Confidence Interval for the Parameter of a Binomial Distribution

Population: $X \sim B(1,p)$

Large-sample approximation: $p \in [\hat{p} - z_{\alpha/2}\sqrt{\frac{\hat{p}(1-\hat{p})}{n}}, \hat{p} + z_{\alpha/2}\sqrt{\frac{\hat{p}(1-\hat{p})}{n}}]$

where $\hat{p} = \frac{1}{n}\sum_{i=1}^n X_i$

Confidence Interval for the Parameter of a Poisson Distribution

Population: $X \sim Poisson(\lambda)$

Large-sample approximation: $\lambda \in [\hat{\lambda} - z_{\alpha/2}\sqrt{\frac{\hat{\lambda}}{n}}, \hat{\lambda} + z_{\alpha/2}\sqrt{\frac{\hat{\lambda}}{n}}]$

where $\hat{\lambda} = \overline{X}$

Exercises

Exercise 1

Write the $1-\alpha$ confidence interval for the mean $\mu$ of a normal population (with the variance known).

Exercise 2

Let $X_1, X_2, \dots, X_n$ be a sample from the population $X \sim N(\mu, \sigma^2)$ with $\sigma^2$ unknown. Find the $95\%$ confidence interval for $\mu$.

Exercise 3

Let $X_1, X_2, \dots, X_n$ be a sample from the population $X \sim N(\mu, \sigma^2)$. Find the $1-\alpha$ confidence interval for $\sigma^2$.

Exercise 4

Let the variances of two populations be $\sigma_1^2, \sigma_2^2$. To test $H_0: \sigma_1^2=\sigma_2^2$, which test should be used?

Exercise 5

Let $X_1, X_2, \dots, X_n$ be a sample from the population $X \sim N(\mu, \sigma^2)$ with $\sigma^2$ known. Find the $90\%$ confidence interval for $\mu$.

---

Summary

Symbols Used in This Article

---