Comprehensive Exercises on Parameter Estimation

Approach: Use the sample mean and the sample variance as unbiased estimators.

Steps:

1. Unbiased estimator of $\mu$: $\hat{\mu} = \overline{X} = \frac{1}{n}\sum_{i=1}^n X_i$
2. Unbiased estimator of $\sigma^2$: $\hat{\sigma}^2 = S^2 = \frac{1}{n-1}\sum_{i=1}^n (X_i - \overline{X})^2$

Answer: $\hat{\mu} = \overline{X}$, $\hat{\sigma}^2 = S^2$

Approach: Use the method of moments, equating the sample moments to the population moments.

Steps:

1. First population moment: $E(X) = \lambda$
2. First sample moment: $\frac{1}{n}\sum_{i=1}^n x_i = \overline{x}$
3. Set $\overline{x} = \lambda$
4. Solving gives: $\hat{\lambda} = \overline{x}$

Answer: $\hat{\lambda} = \overline{x}$

Approach: Use maximum likelihood estimation: construct the likelihood function and differentiate.

Steps:

1. Likelihood function: $L(\mu, \sigma^2) = \prod_{i=1}^n \frac{1}{\sqrt{2\pi}\sigma} e^{-\frac{(x_i-\mu)^2}{2\sigma^2}}$
2. Log-likelihood function: $\ln L(\mu, \sigma^2) = -\frac{n}{2}\ln(2\pi) - \frac{n}{2}\ln(\sigma^2) - \frac{1}{2\sigma^2}\sum_{i=1}^n (x_i-\mu)^2$
3. Differentiate with respect to $\mu$: $\frac{\partial \ln L}{\partial \mu} = \frac{1}{\sigma^2}\sum_{i=1}^n (x_i-\mu) = 0$
4. Solving gives: $\hat{\mu} = \overline{x}$
5. Differentiate with respect to $\sigma^2$: $\frac{\partial \ln L}{\partial \sigma^2} = -\frac{n}{2\sigma^2} + \frac{1}{2\sigma^4}\sum_{i=1}^n (x_i-\mu)^2 = 0$
6. Solving gives: $\hat{\sigma}^2 = \frac{1}{n}\sum_{i=1}^n (x_i-\overline{x})^2$

Answer: $\hat{\mu} = \overline{x}$, $\hat{\sigma}^2 = \frac{1}{n}\sum_{i=1}^n (x_i-\overline{x})^2$

Approach: Construct the confidence interval by the pivotal quantity method.

Steps:

1. Pivotal quantity: $Z = \frac{\overline{X} - \mu}{\sigma/\sqrt{n}} \sim N(0,1)$
2. $P(-z_{\alpha/2} \leq \frac{\overline{X} - \mu}{\sigma/\sqrt{n}} \leq z_{\alpha/2}) = 1 - \alpha$
3. Solving gives: $\mu \in [\overline{X} - z_{\alpha/2}\frac{\sigma}{\sqrt{n}}, \overline{X} + z_{\alpha/2}\frac{\sigma}{\sqrt{n}}]$

Answer: $[\overline{X} - z_{\alpha/2}\frac{\sigma}{\sqrt{n}}, \overline{X} + z_{\alpha/2}\frac{\sigma}{\sqrt{n}}]$

Approach: Compare the variances of the two estimators.

Steps:

1. If $Var(\hat{\theta}_1) < Var(\hat{\theta}_2)$
2. and both estimators are unbiased
3. then $\hat{\theta}_1$ is more efficient than $\hat{\theta}_2$

Answer: $\hat{\theta}_1$ is more efficient

Approach: Use the linearity of expectation.

Steps:

1. $E(\overline{X}) = E(\frac{1}{n}\sum_{i=1}^n X_i) = \frac{1}{n}\sum_{i=1}^n E(X_i)$
2. Since $X_i \sim Poisson(\lambda)$, we have $E(X_i) = \lambda$
3. $E(\overline{X}) = \frac{1}{n} \cdot n\lambda = \lambda$
4. Therefore $\overline{X}$ is an unbiased estimator of $\lambda$

Answer: the proof is complete

Approach: Compute the expectations of the two estimators.

Steps:

1. Let $S^2 = \frac{1}{n-1}\sum_{i=1}^n (X_i - \overline{X})^2$
2. It is known that $E(S^2) = \sigma^2$, that is, $\frac{1}{n-1}\sum_{i=1}^n (X_i - \overline{X})^2$ is unbiased
3. Let $\hat{\sigma}^2 = \frac{1}{n}\sum_{i=1}^n (X_i - \overline{X})^2$
4. $E(\hat{\sigma}^2) = E(\frac{n-1}{n}S^2) = \frac{n-1}{n}\sigma^2 \neq \sigma^2$
5. Therefore $\frac{1}{n}\sum_{i=1}^n (X_i - \overline{X})^2$ is not an unbiased estimator

Answer: $\frac{1}{n-1}\sum_{i=1}^n (X_i - \overline{X})^2$ is the unbiased estimator

Approach: Use the method of moments, equating the sample moments to the population moments.

Steps:

1. First population moment: $E(X) = \frac{\theta}{2}$
2. First sample moment: $\overline{X} = \frac{1}{n}\sum_{i=1}^n X_i$
3. Set $\overline{X} = \frac{\theta}{2}$
4. Solving gives: $\hat{\theta} = 2\overline{X}$

Answer: $\hat{\theta} = 2\overline{X}$

Approach: Use maximum likelihood estimation: construct the likelihood function and differentiate.

Steps:

1. Likelihood function: $L(\lambda) = \prod_{i=1}^n \lambda e^{-\lambda X_i} = \lambda^n e^{-\lambda \sum_{i=1}^n X_i}$
2. Log-likelihood function: $\ln L(\lambda) = n \ln \lambda - \lambda \sum_{i=1}^n X_i$
3. Differentiate: $\frac{\partial \ln L}{\partial \lambda} = \frac{n}{\lambda} - \sum_{i=1}^n X_i = 0$
4. Solving gives: $\hat{\lambda} = \frac{n}{\sum_{i=1}^n X_i} = \frac{1}{\overline{X}}$

Answer: $\hat{\lambda} = \frac{1}{\overline{X}}$

Approach: Construct the confidence interval using the t distribution.

Steps:

1. Pivotal quantity: $T = \frac{\overline{X} - \mu}{S/\sqrt{n}} \sim t(n-1)$
2. $P(-t_{0.025}(n-1) \leq \frac{\overline{X} - \mu}{S/\sqrt{n}} \leq t_{0.025}(n-1)) = 0.95$
3. Solving gives: $\mu \in [\overline{X} - t_{0.025}(n-1)\frac{S}{\sqrt{n}}, \overline{X} + t_{0.025}(n-1)\frac{S}{\sqrt{n}}]$

Answer: $[\overline{X} - t_{0.025}(n-1)\frac{S}{\sqrt{n}}, \overline{X} + t_{0.025}(n-1)\frac{S}{\sqrt{n}}]$

Approach: Construct the confidence interval using the chi-square distribution.

Steps:

1. Pivotal quantity: $\chi^2 = \frac{(n-1)S^2}{\sigma^2} \sim \chi^2(n-1)$
2. $P(\chi^2_{1-\alpha/2}(n-1) \leq \frac{(n-1)S^2}{\sigma^2} \leq \chi^2_{\alpha/2}(n-1)) = 1 - \alpha$
3. Solving gives: $\sigma^2 \in [\frac{(n-1)S^2}{\chi^2_{\alpha/2}(n-1)}, \frac{(n-1)S^2}{\chi^2_{1-\alpha/2}(n-1)}]$

Answer: $[\frac{(n-1)S^2}{\chi^2_{\alpha/2}(n-1)}, \frac{(n-1)S^2}{\chi^2_{1-\alpha/2}(n-1)}]$

Approach: Construct the confidence interval using the normal distribution.

Steps:

1. Pivotal quantity: $Z = \frac{\overline{X} - \mu}{\sigma/\sqrt{n}} \sim N(0,1)$
2. $P(-z_{0.05} \leq \frac{\overline{X} - \mu}{\sigma/\sqrt{n}} \leq z_{0.05}) = 0.90$
3. Solving gives: $\mu \in [\overline{X} - z_{0.05}\frac{\sigma}{\sqrt{n}}, \overline{X} + z_{0.05}\frac{\sigma}{\sqrt{n}}]$

Answer: $[\overline{X} - z_{0.05}\frac{\sigma}{\sqrt{n}}, \overline{X} + z_{0.05}\frac{\sigma}{\sqrt{n}}]$

Approach: Compare them in terms of computational complexity, estimation performance, applicability, and so on.

Steps:

1. Computational complexity:

   • Method of moments: simple to compute, only the sample moments need to be computed
   • Maximum likelihood estimation: relatively more complex to compute, requiring differentiation and solving equations

2. Estimation performance:

   • Method of moments: may not yield the optimal estimator
   • Maximum likelihood estimation: yields the optimal estimator under regularity conditions

3. Applicability:

   • Method of moments: applicable to all kinds of distributions, without needing to know the specific form
   • Maximum likelihood estimation: requires knowing the specific form of the distribution

4. Asymptotic properties:

   • Method of moments: has consistency, but does not necessarily have asymptotic efficiency
   • Maximum likelihood estimation: has consistency, asymptotic normality, and asymptotic efficiency

Answer: the method of moments is simple to compute but its performance may not be as good as maximum likelihood estimation, whereas maximum likelihood estimation is more complex to compute but performs better

Approach: Choose the appropriate test method according to the object being tested.

Steps:

1. Test the ratio of the variances of two normal populations
2. Use the F test
3. Test statistic: $F = \frac{S_1^2}{S_2^2} \sim F(n_1-1, n_2-1)$

Answer: the F test

Approach: Use maximum likelihood estimation: construct the likelihood function and differentiate.

Steps:

1. Likelihood function: $L(p) = \prod_{i=1}^n p^{X_i}(1-p)^{1-X_i} = p^{\sum_{i=1}^n X_i}(1-p)^{n-\sum_{i=1}^n X_i}$
2. Log-likelihood function: $\ln L(p) = \sum_{i=1}^n X_i \ln p + (n-\sum_{i=1}^n X_i) \ln(1-p)$
3. Differentiate: $\frac{\partial \ln L}{\partial p} = \frac{\sum_{i=1}^n X_i}{p} - \frac{n-\sum_{i=1}^n X_i}{1-p} = 0$
4. Solving gives: $\hat{p} = \frac{1}{n}\sum_{i=1}^n X_i = \overline{X}$

Answer: $\hat{p} = \overline{X}$