Bias

• Definition

  $$ B(\hat \theta)=E(\hat \theta)-\theta $$

  $\hat\theta$ is an unbiased estimator of $\theta$ if $E_\theta(\hat\theta)=\theta$ for all $\theta$, otherwise, biased.

  • $E(\bar X)=\mu, E(S^2)=E(\frac{1}{n-1}\sum(X_i-\bar X)^2)=\sigma^2$
  • $S = \sqrt{S^2}$ is not UE of $\sigma$. ($\because E(S^2) > \{E(S)\}^2$)

MSE

• Definition (Mean Square Error)

  $$ MSE(\hat\theta)=E_\theta[(\hat\theta-\theta)^2]=V(\hat\theta)+(B(\hat\theta))^2 $$

  The smaller, the better.

Relative efficiency

• Definition

  $$ eff(\hat\theta_1, \hat\theta_2)=\frac{V(\hat\theta_2)}{V(\hat\theta_1)} $$