Method of Moments (MM)

Idea: Sample moments provide good estimates of the corresponding population moments.

$$ m'_k=\frac{1}{n}\sum^n_1X^k_i \rightarrow \mu'_k=E(X^k) $$

Estimate population moments by sample moments. For any other parameter $\theta$, express $\theta$ in terms of $E(X^k)$

MME is

• easy to find and intuitive
• consistent if the parameter is a continuous function of moments
• not unbiased unless the parameter is a linear function of moments
• not necessarily the best estimator. MM often produces inferior estimators.

Maximum Likelihood Estimator (MLE)

Idea: For a fixed value of $\theta$, $L(\theta)$ is “probability” of getting $x_1,...,x_n$ when $\theta$ is true. It represents how likely the value $\theta$ is as the true parameter given $x_1,...,x_n$.

Definition

MLE $\hat \theta$ of $\theta$ is a value $\theta$ such that for given $x_1,...,x_n$,

$$ L(\hat\theta) = \max_\theta L(\theta) $$

• Check 1st derivative = 0, 2nd derivative < 0
• No need to check Hessian matrix

Invariance property of MLE

Theorem

If we have MLE $\hat\theta$ of $\theta$ and $\alpha=h(\theta)$ is a function of $\theta$, then MLE of $\alpha$ is $\hat \alpha=h(\hat\theta)$.

1-1 transform을 해도 MLE