Idea of Most Powerful Tests

1. Choose $H_0$ to be claim we want evidence against. ($H_a$ → 우리가 보여주고 싶은거)

2. We want to guard false rejection of $H_0$ ($=\alpha$) ⇒ Set a value of $\alpha$ we can live with, often 0.01, 0.05, 0.1. Call this fixed $\alpha$ the level of this test.

3. Ideally, among tests with fixed $\alpha$, use one with the smallest $\beta$ (highest power in $H_a(=1-\beta)$) against alternative of interest. ⇒ This is the most powerful test.

   $\alpha$ 고정시켜두고 가장 power($\theta_a$)가 큰 test = most powerful test

4. Once theory gives us from of MP tests, often use $p$-value rather than fixed $\alpha$ in practice.

What the theory tells us

Note: Reason we prefer to use $p$-value

• 상황에 따라 $\alpha$ 값을 다르게 정해야 함
• In discrete cases, cannot find a test of a specific size $\alpha$
• $p$-values provide more information than sayin reject/not reject at $\alpha$

Theory of Tests

With the parameter space $\Omega$,

$$ H_0 : \theta \in \Omega_0 \text{ vs. } H_a :\theta \in \Omega - \Omega_0 $$

Definition of Simple/Composite

Hypotheses are called as simple if it completely determines the underlying distribution. Otherwise, composite.

Definition of most powerful rejection region

A rejection region $R$ is most powerful of level $\alpha$ if

1. $\max_{\theta \in \Omega_0} P_\theta(R)=\alpha$
2. $P_\theta(R) \geq P_\theta(C)$ for all $\theta \in \Omega-\Omega_0$ for any other rejection region $C$ with $\max_{\theta\in\Omega_0} P_\theta(C)=\alpha$

→ For fixed $\alpha$, maximize power($\theta_a$) = $P(\text{reject }H_0|\theta \in H_a)=P_{\theta_a}(R)$