7. Correlation and Hypothesis Testing

Correlation Coefficient

Let $X, Y$ be random variables with $E (X) = μ_{x}$ and $E (Y) = μ_{y}$
otherwise $X$ is a constant

\begin{aligned} c o v (X, Y) & = E [(X - μ_{x}) (Y - μ_{y})] \\ c o v (X, Y) & = E [X Y] - E (X) E (Y) = E [X Y] - μ_{x} μ_{y} \end{aligned}

ρ = \frac{c o v (X, Y)}{σ_{x} σ_{y}}

$Y = β_{0} + β_{1} X + ε,$ $ε \sim N (0, σ^{2})$
$E [Y] = β_{0} + β_{1} μ_{x}$
$V a r (Y) = β_{1}^{2} σ_{x}^{2} + σ^{2}$ which is like $S S T = S S R + S S E$

$E [Y | X] = β_{0} + β_{1} X$
$V a r [Y | X] = σ^{2}$

$X \sim N (E (x) = μ_{x}, V a r (x) = σ_{x}^{2})$

\begin{aligned} ⟹ c o v (Y, X) & = E [(Y - μ_{y}) (X - μ_{x})] \\ = E [(β_{0} + β_{1} X + ε - β_{0} + β_{1} μ_{x}) (X - μ_{x})] \\ = E [(β_{1} (X - μ_{x}) + ε) (X - μ_{x})] \\ = E [β_{1} [X - μ_{x}]^{2} + ε (X - μ_{x})] \\ = β_{1} E [(x - μ_{x})^{2}] + E [ε (X - μ_{x})] \\ = β_{1} σ_{x}^{2} + E [ε] E [X - μ_{x}] \\ = β_{1} σ_{x}^{2} \\ (1) & ρ & = \frac{c o v (X, Y)}{σ_{x} σ_{y}} = \frac{β_{1} σ_{x}^{2}}{σ_{x} σ_{y}} = β_{1} (\frac{σ_{x}}{σ_{y}}) \\ (2) & ρ^{2} & = \frac{β_{1}^{2} σ_{x}^{2}}{σ_{y}^{2}} \\ σ_{y}^{2} & = β_{1}^{2} σ_{x}^{2} + σ^{2} \\ 1 & = \frac{σ_{y}^{2}}{σ_{y}^{2}} = \frac{β_{1}^{2} σ_{x}^{2}}{σ_{y}^{2}} + \frac{σ^{2}}{σ_{y}^{2}} \geq \frac{β_{1}^{2} σ_{x}^{2}}{σ_{y}^{2}} = ρ^{2} \\ ⟹ - 1 \leq ρ \leq 1 \end{aligned}

{\begin{cases} ρ^{2} = 1 & ⟺ σ^{2} = 0, perfect linear relationship \\ ρ = 0 & ⟺ β_{1} = 0 \\ ρ > 0 & ⟺ β_{1} > 0 \\ ρ < 0 & ⟺ β_{1} < 0 \end{cases}

Estimator

\begin{aligned} {\hat{ρ}}^{2} & = \frac{({\hat{β}}^{2} \cdot [\frac{\sum_{}^{} (x_{i} - \bar{x})^{2}}{n - 1}])}{\frac{\sum_{}^{} (y_{i} - \bar{y})^{2}}{n - 1}} \\ = \frac{({\hat{β}}^{2} \cdot [\sum_{}^{} (x_{i} - \bar{x})^{2}])}{\sum_{}^{} (y_{i} - \bar{y})^{2}} \\ = \frac{S S R}{S S T} \end{aligned}

note:

\begin{aligned} \frac{S S T}{S S T} & = \frac{S S R}{S S T} + \frac{S S E}{S S T} \\ 1 & = {\hat{ρ}}^{2} + \frac{S S E}{S S T} ⟹ 1 - \frac{S S E}{S S T} = {\hat{ρ}}^{2} \end{aligned}

let $r^{2} = {\hat{ρ}}^{2}$

Hypothesis Testing

H_{0} : β_{1} = 0 vs H_{a} : β_{1} > 0 ⟺ H_{0} : ρ = 0 vs H_{a} : ρ > 0

Looking at formula (1)

From earlier:

original test statistics

F-Test Restated

$F_{*} = \frac{\frac{{\hat{β}}_{1}^{2} S_{x x}}{| | Residual Vector | |^{2}}}{n - 2} = \frac{\frac{S S R}{S S E}}{n - 2}$

\begin{aligned} ⟹ (n - 2) [\frac{\frac{S S R}{S S T}}{\frac{S S E}{S S T}}] = (n - 2) [\frac{r^{2}}{1 - r^{2}}] \sim F_{n - 2} \end{aligned}

t_{*} = \sqrt{F_{*}} = \sqrt{n - 2} \cdot \frac{r}{\sqrt{1 - r^{2}}} \sim t_{n - 2}

by previous results

$p-val = p (t_{n - 2} > t_{*})$ , RR = ${t_{n - 2} > t_{n - 2, α}}$