Hypothesis Testing for Two Normal Populations

Testing the Difference of the Means of Two Normal Populations

When the Variances Are Known (Z Test)

Population: $X \sim N(\mu_1, \sigma_1^2)$, $Y \sim N(\mu_2, \sigma_2^2)$, with $\sigma_1^2, \sigma_2^2$ known

Hypotheses:

• $H_0: \mu_1 = \mu_2$
• $H_1: \mu_1 \neq \mu_2$ (two-sided test)
• $H_1: \mu_1 > \mu_2$ (right one-sided test)
• $H_1: \mu_1 < \mu_2$ (left one-sided test)

Test statistic: $Z = \frac{(\overline{X} - \overline{Y}) - (\mu_1 - \mu_2)}{\sqrt{\frac{\sigma_1^2}{n_1} + \frac{\sigma_2^2}{n_2}}} \sim N(0,1)$

Rejection region:

• Two-sided test: $|Z| > z_{\alpha/2}$
• Right one-sided test: $Z > z_{\alpha}$
• Left one-sided test: $Z < -z_{\alpha}$

Example 1: a factory produces parts with two different processes. The lengths of the parts produced by process A follow $N(\mu_1, 0.04)$, and the lengths of the parts produced by process B follow $N(\mu_2, 0.06)$. 16 parts are drawn from process A with an average length of 10.2cm, and 20 parts are drawn from process B with an average length of 10.0cm. At the significance level $\alpha = 0.05$, test whether the part lengths produced by the two processes differ.

Solution:

1. State the hypotheses: $H_0: \mu_1 = \mu_2$, $H_1: \mu_1 \neq \mu_2$
2. Test statistic: $Z = \frac{(10.2 - 10.0) - 0}{\sqrt{\frac{0.04}{16} + \frac{0.06}{20}}} = \frac{0.2}{\sqrt{0.0025 + 0.003}} = \frac{0.2}{0.074} = 2.70$
3. Rejection region: $|Z| > 1.96$
4. Decision: $|Z| = 2.70 > 1.96$, so reject the null hypothesis
5. Conclusion: the part lengths produced by the two processes differ significantly

When the Variances Are Unknown but Equal (Two-Sample T Test)

Population: $X \sim N(\mu_1, \sigma^2)$, $Y \sim N(\mu_2, \sigma^2)$, with $\sigma^2$ unknown but equal

Hypotheses:

• $H_0: \mu_1 = \mu_2$
• $H_1: \mu_1 \neq \mu_2$ (two-sided test)
• $H_1: \mu_1 > \mu_2$ (right one-sided test)
• $H_1: \mu_1 < \mu_2$ (left one-sided test)

Test statistic: $T = \frac{(\overline{X} - \overline{Y}) - (\mu_1 - \mu_2)}{S_p\sqrt{\frac{1}{n_1} + \frac{1}{n_2}}} \sim t(n_1 + n_2 - 2)$

where $S_p^2 = \frac{(n_1-1)S_1^2 + (n_2-1)S_2^2}{n_1 + n_2 - 2}$ is the pooled variance.

Rejection region:

• Two-sided test: $|T| > t_{\alpha/2}(n_1 + n_2 - 2)$
• Right one-sided test: $T > t_{\alpha}(n_1 + n_2 - 2)$
• Left one-sided test: $T < -t_{\alpha}(n_1 + n_2 - 2)$

Example 2: a hospital compares the effects of two drugs on high blood pressure. 15 patients are treated with drug A, with an average blood pressure reduction of 8.5mmHg and a sample standard deviation of 2.1mmHg; 12 patients are treated with drug B, with an average blood pressure reduction of 6.8mmHg and a sample standard deviation of 2.3mmHg. At the significance level $\alpha = 0.05$, test whether the effects of the two drugs differ.

Solution:

1. State the hypotheses: $H_0: \mu_1 = \mu_2$, $H_1: \mu_1 \neq \mu_2$
2. Pooled variance: $S_p^2 = \frac{(15-1) \times 2.1^2 + (12-1) \times 2.3^2}{15 + 12 - 2} = \frac{14 \times 4.41 + 11 \times 5.29}{25} = 4.82$
3. Test statistic: $T = \frac{(8.5 - 6.8) - 0}{\sqrt{4.82 \times (\frac{1}{15} + \frac{1}{12})}} = \frac{1.7}{\sqrt{4.82 \times 0.15}} = \frac{1.7}{0.85} = 2.00$
4. Rejection region: $|T| > t_{0.025}(25) = 2.060$
5. Decision: $|T| = 2.00 < 2.060$, so the null hypothesis is not rejected
6. Conclusion: there is no significant difference between the effects of the two drugs

When the Variances Are Unknown and Unequal (Welch’s T Test)

Population: $X \sim N(\mu_1, \sigma_1^2)$, $Y \sim N(\mu_2, \sigma_2^2)$, with $\sigma_1^2, \sigma_2^2$ unknown and unequal

Hypotheses:

• $H_0: \mu_1 = \mu_2$
• $H_1: \mu_1 \neq \mu_2$ (two-sided test)
• $H_1: \mu_1 > \mu_2$ (right one-sided test)
• $H_1: \mu_1 < \mu_2$ (left one-sided test)

Test statistic: $T = \frac{(\overline{X} - \overline{Y}) - (\mu_1 - \mu_2)}{\sqrt{\frac{S_1^2}{n_1} + \frac{S_2^2}{n_2}}} \sim t(df)$

where the degrees of freedom are $df = \frac{(\frac{S_1^2}{n_1} + \frac{S_2^2}{n_2})^2}{\frac{(S_1^2/n_1)^2}{n_1-1} + \frac{(S_2^2/n_2)^2}{n_2-1}}$ (the Welch-Satterthwaite formula).

Rejection region:

• Two-sided test: $|T| > t_{\alpha/2}(df)$
• Right one-sided test: $T > t_{\alpha}(df)$
• Left one-sided test: $T < -t_{\alpha}(df)$

Testing the Ratio of the Variances of Two Normal Populations (F Test)

When the Means Are Unknown

Population: $X \sim N(\mu_1, \sigma_1^2)$, $Y \sim N(\mu_2, \sigma_2^2)$, with $\mu_1, \mu_2$ unknown

Hypotheses:

• $H_0: \sigma_1^2 = \sigma_2^2$
• $H_1: \sigma_1^2 \neq \sigma_2^2$ (two-sided test)
• $H_1: \sigma_1^2 > \sigma_2^2$ (right one-sided test)
• $H_1: \sigma_1^2 < \sigma_2^2$ (left one-sided test)

Test statistic: $F = \frac{S_1^2/\sigma_1^2}{S_2^2/\sigma_2^2} = \frac{S_1^2}{S_2^2} \sim F(n_1-1, n_2-1)$

Rejection region:

• Two-sided test: $F < F_{1-\alpha/2}(n_1-1, n_2-1)$ or $F > F_{\alpha/2}(n_1-1, n_2-1)$
• Right one-sided test: $F > F_{\alpha}(n_1-1, n_2-1)$
• Left one-sided test: $F < F_{1-\alpha}(n_1-1, n_2-1)$

Example 3: a factory compares the stability of two production processes. 10 products are produced with process A, with a sample variance of 0.04; 12 products are produced with process B, with a sample variance of 0.06. At the significance level $\alpha = 0.05$, test whether the stability of the two processes differs.

Solution:

1. State the hypotheses: $H_0: \sigma_1^2 = \sigma_2^2$, $H_1: \sigma_1^2 \neq \sigma_2^2$
2. Test statistic: $F = \frac{0.04}{0.06} = 0.67$
3. Rejection region: $F < F_{0.975}(9, 11) = 0.25$ or $F > F_{0.025}(9, 11) = 3.59$
4. Decision: $0.25 < F = 0.67 < 3.59$, so the null hypothesis is not rejected
5. Conclusion: there is no significant difference between the stability of the two processes

Comparison of Testing Methods

Comparison of Methods for Testing a Difference of Means

Z test:

• Applicable condition: the variances are known
• Advantages: simple to compute, no need to consult the t distribution table
• Disadvantages: requires knowing the population variance

Two-sample T test:

• Applicable condition: the variances are unknown but equal
• Advantages: does not require knowing the population variance
• Disadvantages: requires assuming the variances are equal

Welch’s T test:

• Applicable condition: the variances are unknown and unequal
• Advantages: does not require assuming the variances are equal
• Disadvantages: relatively more complex to compute

Characteristics of the Test for the Ratio of Variances

F test:

• Object of the test: the ratio of the variances of two normal populations
• Application scenarios: comparing the precision and the stability of two methods
• Cautions: it is sensitive to the normality assumption

Assumptions of the Tests

The Normality Assumption

Importance: all the testing methods are based on the normality assumption.

Testing methods:

1. Graphical methods: Q-Q plots, histograms
2. Statistical tests: the Shapiro-Wilk test, the Kolmogorov-Smirnov test

Handling non-normal data:

1. Data transformation: logarithmic transformation, square root transformation, and so on
2. Nonparametric tests: the Mann-Whitney U test, the Kruskal-Wallis test, and so on

The Independence Assumption

Importance: the sample observations should be mutually independent.

Testing methods:

1. Time series analysis: the autocorrelation function
2. Spatial data analysis: spatial autocorrelation

The Homogeneity of Variance Assumption

Importance: the two-sample T test requires the assumption of equal variances.

Testing methods:

1. F test: tests the ratio of variances
2. Levene’s test: a test for homogeneity of variance that is insensitive to normality
3. Brown-Forsythe test: a test for homogeneity of variance based on the median

Power Analysis of Tests

Power Function

Factors that influence it:

1. Sample size: the larger the sample size, the higher the power
2. Significance level: the higher the significance level, the higher the power
3. Effect size: the larger the effect size, the higher the power

Determining the Sample Size

Problem: given the significance level, the power, and the effect size, determine the sample size required.

Example 4: for testing the difference of the means of two normal populations, given the significance level $\alpha = 0.05$, the power $1 - \beta = 0.8$, and the effect size $d = 0.5$, find the sample size $n$ for each group.

Solution:

1. Use the power analysis formula
2. Consult the table or use software to compute
3. We obtain $n \approx 32$ for each group

Cautions for Testing

The Multiple Comparison Problem

Problem: when performing several hypothesis tests, the probability of committing a Type I error increases.

Solution:

1. Bonferroni correction: divide the significance level by the number of tests
2. Holm correction: adjust the significance level step by step
3. FDR control: control the false discovery rate

Effect Size

Common effect sizes:

1. Cohen’s d: $d = \frac{\mu_1 - \mu_2}{\sigma}$
2. Correlation coefficient: $r = \frac{t}{\sqrt{t^2 + df}}$

Significance: even if the result is statistically significant, a small effect size may mean little practical significance.

Influence of the Sample Size

Influence: the larger the sample size, the higher the power of the test and the smaller the probability of committing a Type II error.

Reason: the larger the sample size, the more accurate the estimate, and the more easily real differences are detected.

Exercises

Exercise 1

For two normal populations whose variances are unknown but equal, which test should be used to test the difference of the means?

Exercise 2

Let the variances of two populations be $\sigma_1^2, \sigma_2^2$. To test $H_0: \sigma_1^2=\sigma_2^2$, which test should be used?

Exercise 3

Let $X \sim N(\mu_1, \sigma_1^2)$, $Y \sim N(\mu_2, \sigma_2^2)$ with $\sigma_1^2, \sigma_2^2$ known. To test $H_0: \mu_1 = \mu_2$, write the test statistic.

Exercise 4

Let $X \sim N(\mu_1, \sigma^2)$, $Y \sim N(\mu_2, \sigma^2)$ with $\sigma^2$ unknown but equal. To test $H_0: \mu_1 = \mu_2$, write the test statistic.

Exercise 5

Let $X \sim N(\mu_1, \sigma_1^2)$, $Y \sim N(\mu_2, \sigma_2^2)$ with $\mu_1, \mu_2$ unknown. To test $H_0: \sigma_1^2 = \sigma_2^2$, write the test statistic.