Basic Ideas and Steps of Hypothesis Testing

Basic Concepts of Hypothesis Testing

Definition of Hypothesis Testing

Intuitive Understanding of Hypothesis Testing

Understanding: hypothesis testing is similar to a court trial, where the null hypothesis corresponds to the presumption of innocence and the alternative hypothesis corresponds to the charge of guilt. We use evidence (sample data) to decide whether to reject the null hypothesis.

Basic Elements of Hypothesis Testing

Null hypothesis $H_0$: usually the hypothesis we want to reject, expressing “no difference”, “no effect”, and so on Alternative hypothesis $H_1$: usually the hypothesis we want to accept, expressing “there is a difference”, “there is an effect”, and so on Significance level $\alpha$: the probability of committing a Type I error, usually taken as 0.05 or 0.01 Test statistic: the statistic used to decide whether to reject the null hypothesis Rejection region: the region of values of the test statistic; the null hypothesis is rejected when the statistic falls in this region

Basic Steps of Hypothesis Testing

Step 1: State the Hypotheses

Null hypothesis $H_0$: $\theta = \theta_0$ or $\theta \leq \theta_0$ or $\theta \geq \theta_0$ Alternative hypothesis $H_1$: $\theta \neq \theta_0$ or $\theta > \theta_0$ or $\theta < \theta_0$

Example 1: testing whether the population mean equals a certain value

• $H_0: \mu = \mu_0$
• $H_1: \mu \neq \mu_0$

Step 2: Choose the Test Statistic

Principles of choice:

1. Under the condition that the null hypothesis holds, the distribution of the test statistic is known
2. The test statistic can distinguish the null hypothesis from the alternative hypothesis
3. The computation of the test statistic is relatively simple

Example 2: for testing the mean of a normal population (with the variance known)

• Test statistic: $Z = \frac{\overline{X} - \mu_0}{\sigma/\sqrt{n}} \sim N(0,1)$

Step 3: Determine the Rejection Region

Two-sided test: $|Z| > z_{\alpha/2}$ One-sided test: $Z > z_{\alpha}$ or $Z < -z_{\alpha}$

Example 3: for a two-sided test with significance level $\alpha = 0.05$

• Rejection region: $|Z| > 1.96$

Step 4: Compute the Sample Value

Computation: substitute the sample data into the test statistic to obtain a concrete numerical value.

Example 4: the sample mean is $\overline{x} = 52$, the population standard deviation is $\sigma = 10$, the sample size is $n = 25$, and we test $\mu_0 = 50$

• $z = \frac{52 - 50}{10/\sqrt{25}} = 1$

Step 5: Draw a Conclusion

Decision: if the value of the test statistic falls inside the rejection region, then reject the null hypothesis; otherwise accept the null hypothesis.

Example 5: for Example 4, $z = 1 < 1.96$, so the null hypothesis is not rejected.

The Two Types of Errors

Type I Error (False Rejection)

Probability: $P(\text{reject }H_0|H_0\text{ is true}) = \alpha$

Significance: a Type I error is the error of “wrongly convicting an innocent person”; its probability is exactly the significance level $\alpha$.

Example 6: in Example 4, if $\mu = 50$ (the null hypothesis is true) but $|Z| > 1.96$, then a Type I error has been committed.

Type II Error (False Acceptance)

Probability: $P(\text{accept }H_0|H_1\text{ is true}) = \beta$

Significance: a Type II error is the error of “letting a guilty person go”.

Example 7: in Example 4, if $\mu = 55$ (the null hypothesis is false) but $|Z| \leq 1.96$, then a Type II error has been committed.

Relation Between the Two Types of Errors

Relation: with the sample size fixed, decreasing $\alpha$ increases $\beta$, and decreasing $\beta$ increases $\alpha$.

Trade-off: the probability of the Type I error is usually controlled first, because the consequences of a Type I error are usually more serious.

Power Function

Significance: the larger the power function, the better the test performs.

Significance Level and P-Value

Significance Level

Common values: $\alpha = 0.05$ (5%), $\alpha = 0.01$ (1%), $\alpha = 0.10$ (10%)

Choice: choose the significance level according to how serious the problem is and what its consequences are.

P-Value

Computation:

• Two-sided test: $P = 2P(Z > |z|)$
• One-sided test: $P = P(Z > z)$ or $P = P(Z < z)$

Decision: if $P < \alpha$, then reject the null hypothesis; otherwise accept the null hypothesis.

Example 8: for Example 4, $z = 1$, $P = 2P(Z > 1) = 2 \times 0.1587 = 0.3174 > 0.05$, so the null hypothesis is not rejected.

Types of Hypothesis Tests

Two-Sided Test

Form: $H_0: \theta = \theta_0$, $H_1: \theta \neq \theta_0$

Rejection region: $|T| > t_{\alpha/2}$

Application: used when we care whether the parameter equals a certain value.

One-Sided Test

Left one-sided test: $H_0: \theta \geq \theta_0$, $H_1: \theta < \theta_0$ Right one-sided test: $H_0: \theta \leq \theta_0$, $H_1: \theta > \theta_0$

Rejection region: $T < -t_{\alpha}$ or $T > t_{\alpha}$

Application: used when we care whether the parameter is greater than or less than a certain value.

Decision Rules for Hypothesis Testing

Critical Value Method

Steps:

1. Determine the critical value from the significance level
2. Compute the value of the test statistic
3. Compare the test statistic with the critical value
4. Make a decision

Advantages: intuitive and explicit Disadvantages: requires consulting tables

P-Value Method

Steps:

1. Compute the P-value
2. Compare the P-value with the significance level
3. Make a decision

Advantages: no need to consult tables, allows precise judgement Disadvantages: relatively more complex to compute

Cautions for Hypothesis Testing

Influence of the Sample Size

Influence: the larger the sample size, the higher the power of the test and the smaller the probability of committing a Type II error.

Reason: the larger the sample size, the more accurate the estimate, and the more easily real differences are detected.

Effect Size

Significance: even if the result is statistically significant, a small effect size may mean little practical significance.

Multiple Comparisons

Problem: when performing several hypothesis tests, the probability of committing a Type I error increases.

Solution: use a multiple comparison correction method, such as the Bonferroni correction.

Applications of Hypothesis Testing

Applications in Quality Control

Application 1: testing whether product quality meets a standard Application 2: testing whether the production process is stable

Applications in Medical Research

Application 3: testing whether a new drug is effective Application 4: testing whether treatment methods differ

Applications in the Social Sciences

Application 5: testing whether survey results differ Application 6: testing whether the effect of a policy is significant

Exercises

Exercise 1

Briefly describe the general steps of hypothesis testing.

Exercise 2

Let $X \sim N(\mu, \sigma^2)$ with $\sigma$ known. To test $H_0: \mu=\mu_0$, write the test statistic.

Exercise 3

Explain the meaning of the Type I error and the Type II error.

Exercise 4

Let the significance level be $\alpha = 0.05$ and the P-value be $P = 0.03$. Should the null hypothesis be rejected or accepted?

Exercise 5

Let the test statistic be $Z = 2.5$ and the significance level be $\alpha = 0.05$. For a two-sided test, should the null hypothesis be rejected or accepted?

---

Summary

Symbols Used in This Article

---