6. Hypothesis Testing

Typical Steps

Frequentist:

State your hypothesis
$H_{0} : μ = μ_{0}, H_{a} : μ \geq μ_{0}$
Find you test statistic
$z_{*} = \frac{\bar{x} - μ_{0}}{\frac{σ}{\sqrt{n}}}$
P-value = $P [z > z_{*}]$
Conclusions $α =$ significance level, if P-value $< α$ then reject $H_{0}$

Bayesian:

State Hypothesis
Find posterior distribution
Find the posterior probability of $H_{0}$ that is $P [H_{0} | Data]$
Conclusion if $P [H_{0} | Data] < α$ then we reject $H_{0}$ (the complement is the pro)

Example Normal

The local consumer watchdog group was concerned about the cost of electricity to residential customers over the summer months. They took a random sample of 25 residential electricity accounts and looked at the total cost of electricity used over the three months of June, July, and August. The costs were:
514 536 345 440 427
443 386 418 364 483
506 385 410 561 275
306 294 402 350 343
480 334 324 414 296

Assume that the amount of electricity used over the three months by a residential account is Normal(μ, σ2), where the known standard deviation σ = 80.

a) Use a Normal(325, $80^{2}$ ) prior for μ. Find the posterior distribution for μ.

Using the updating rules

\frac{1}{s_{*}^{2}} = \frac{25}{80^{2}} + \frac{1}{80^{2}} ⟹ \frac{80^{2}}{26} = 246.1538

\begin{aligned} m_{*} & = [\frac{\frac{1}{80^{2}}}{\frac{1}{80^{2}} + \frac{1}{s_{*}^{2}}}] \cdot 401.44 + [\frac{\frac{1}{s_{}^{2}}}{\frac{1}{80^{2}} + \frac{1}{s_{*}^{2}}}] \cdot 325 \\ = \frac{25}{56} \cdot 401.44 + \frac{31}{56} \cdot 325 \\ \approx 398.5 \end{aligned}

P (μ | Data) \sim N (398.5, 256.1588)

b) Find a 95% Bayesian credible interval for μ.

m_{*} \pm z_{\frac{α}{2}} \sqrt{s_{*}^{2}}

c) Perform a Bayesian test of the hypothesis
$H_{0} : μ = 350, H_{a} : μ \neq 350$ at the 5% level.

Not within the credible interval so no, reject $H_{0}$

d) Perform a Bayesian test of the hypothesis
$H_{0} : μ \leq 350, H a : μ > 350$ at the 5% level.

P [μ \leq 350 | Data] = P [z \leq \frac{350 - m_{*}}{\sqrt{s_{*}}}] = P [z \leq - 3.09] = 0.0010

Rejects $H_{0}$

Example Poisson

A local enforcement agency claims that the number of times that a patrol car passes through a particular neighbourhood follows a Poisson process with a mean of three times per nightly shift. Suppose that during a randomly selected night shift no patrol cars pass through the neighbourhood. Do you believe the agency’s claim? Perform a Bayesian test of the hypothesis

$H_{0} : λ \geq 3, H_{a} : λ < 3$ at the 5% level.

Use a $G a m m a (r = 1, v = 1 / 3)$ prior for λ.

by updating rules

p (λ | y = 0) is Gamma (r_{*} = 0 + 1, v_{*} = 1 + \frac{1}{3} = \frac{4}{3})

\begin{aligned} P [H_{0} | Data] & = P [λ \geq 3] \\ = \int_{3}^{\infty} \frac{\frac{4}{3}}{Γ (1)} λ^{1 - 1} e^{- 4 / 3 (λ)} d λ \\ = \int_{3}^{\infty} \frac{4}{3} e^{- 4 / 3 (λ)} d λ = e^{- 4} \end{aligned}

compare to earlier example

Difference In Parameters

Example Difference in Proportion

A new study indicates that tai chi, an ancient Chinese practice of exercise and med-
itation, may relieve symptoms of chronic painful fibromyalgia. The study assigned 66
fibromyalgia patients to take either a 12-week tai chi class $(n_{1} = 33)$ or a wellness educa-
tion class $(n_{2} = 33)$ . The results of the study are shown in the following table:
Tai Chi Wellness Education

\begin{array}{ccc} Thai Chi & Wellness Education \\ Number who felt better & 26 & 13 \end{array}

Show all your work and circle your final answers.
a) Find the posterior distribution of $π_{1}$ , the proportion of fibromyalgia patients who would
admit to feeling better after taking the tai chi class. (Use a Beta(1, 1) prior for π1. )

p (π_{1} | Data) \propto (π_{1})^{1 - 1} (1 - π)^{1 - 1} \cdot (π)^{26} \cdot (1 - π)^{7}

⟹ π_{1} | Data \sim B e t a (27, 8)

b) Find the posterior distribution of $π_{2}$ , the proportion of fibromyalgia patients who would
admit to feeling better after taking the wellness education class. (Use a Beta(1, 1) prior
for $π_{2}$ .)

π_{2} | Data \sim B e t a (14, 21)

c) Find the approximate posterior distribution of $π_{1} - π_{2}$ .

E [π_{1} - π_{2}] = E [π_{1}] - E [π_{2}] = \frac{27}{35} + \frac{14}{35}

V a r [π_{1} - π_{2}] = \frac{27 \cdot 8}{(35)^{2} \cdot (36)} + \frac{14 \cdot 21}{35^{2} \cdot (36)}

d) Test $H_{0} : π_{1} - π_{0} \geq 0$ vs $H_{a} : π_{1} - π_{2} < 0$ at the 1% level

$P [H_{0} | Data]$

\begin{array}{r} P [π_{1} - π_{2} \geq 0] \approx P [\frac{(π_{1} - π_{2}) - 0.3714}{\sqrt{0.0116}} \geq - \frac{0.3714}{\sqrt{0.0116}}] \approx 0.9997 \end{array}

Fail to reject

Example Difference in Means

Industrial wastes and sewage dumped into our rivers and streams absorb oxygen and thereby reduce the amount of dissolved oxygen available for fish and other forms of aquatic life. One provincial agency requires a minimum of 5 parts per million (ppm) of dissolved oxygen in order for the oxygen content to be sufficient to support aquatic life. A pollution control inspector suspected that a river community was releasing amounts of semitreated sewage into a river. To check her theory, she drew five randomly selected specimens of river water at a location above the town, and another five below. The dissolved oxygen readings (in parts per million) are as follows:

\begin{array}{cc} Above town (A) & Below town (B) \\ 4.8 & 5.0 \\ 5.2 & 4.7 \\ 5.0 & 4.9 \\ 4.9 & 4.8 \\ 5.1 & 4.9 \end{array}

(a) We will assume that the observations come from

Normal (μ_{A}, σ^{2}) and Normal (μ_{B}, σ^{2}),

where

σ^{2} = {0.2}^{2} .

Use independent “flat priors” for $μ_{A}$ and $μ_{B}$ .

Find the posterior distributions of $μ_{A}$ and $μ_{B}$ .

Answer:

Flat Prior: $g (μ) = 1$ , $N (0, s_{A}^{2} \to 0)$
Assumption: $σ^{2}$ is known

\begin{aligned} p (μ_{a} | y) & \propto e^{\frac{- 1}{2 (σ^{2} / n)} [μ - \bar{y}]^{2}} \end{aligned}

Updating Rules:

For $μ_{A}$

\begin{aligned} Posterior Precision & = Sample Precision + Prior Precision \\ \frac{1}{s_{A *}^{2}} & = {\frac{1}{s_{A}^{2}}}^{lim_{s_{A}^{2} \to \infty}} + \frac{n_{A}}{σ^{2}} \\ \frac{1}{s_{A *}^{2}} & = \frac{5}{{0.2}^{2}} = \frac{500}{4} \end{aligned}

\begin{aligned} Posterior Mean & = \frac{Sample Precision}{Posterior Precision} (Sample Mean) + \frac{Prior Precision}{Posterior Precision} (Prior Mean) \\ m_{A *} & = [\frac{\frac{n_{A}}{σ^{2}}}{(\frac{1}{s_{A}^{2}})}] {\bar{y}}_{A} + \frac{0}{(\frac{4}{500})} \bar{y} \\ m_{A *} & = {\bar{y}}_{A} \\ m_{A *} & = 5 \end{aligned}

$P (μ_{A} | Data) \sim N (5, \frac{4}{500})$

For $μ_{B}$

\frac{1}{s_{B *}^{2}} = \frac{n_{B}}{σ^{2}} = \frac{500}{4}

m_{B *} = {\bar{y}}_{B} = 4.86

$P (μ_{B} | Data) \sim N (5, \frac{4}{500})$

b) $P (μ_{A} - μ_{B} | Data) \sim Normal$
Mean = $5 - 4.86 = 0.14$
Variace = $\frac{8}{500}$

c) 95% C.I = $0.14 \pm 1.96 \sqrt{\frac{8}{500}}$

d) $H_{0} : μ_{A} - μ_{B} \leq 0$ vs $H_{a} : μ_{A} - μ_{B} > 0$

$P [\frac{μ_{A} - μ_{B} - 0.14}{\sqrt{\frac{8}{509}}} \leq - \frac{0.14}{\sqrt{\frac{8}{400}}}] = 0.133$

fail to reject