3 Non-central Chi-Squared Theorems

Non-Central Chi-squared 16.2

If $y \sim N (μ, I)$ , then $q = y^{T} y$ has a non-central chi-squared distribution
with $N$ degrees of freedom and non-centrality parameter, $θ = \frac{μ^{T} μ}{2}$

The central case will be denoted by $χ^{2} (N, θ = 0)$ or, simply, $χ^{2} (N)$ .

Quadratic Form Distribution Theorem 16.4

If $y \sim N (μ, V)$ and $q = y^{T} A y$

$q \sim χ^{2} (r, θ)$ where $r = rank(A)$ and $θ = \frac{μ^{T} A μ}{2}$ $⟺$ $A \cdot V$ is idempotent

or $(A \cdot V)^{2} = A \cdot V$

Independence Condition for Quadratic Forms 16.5

$y \sim N (μ, V)$
$q_{1} = y^{T} A_{1} y$
$q_{2} = y^{T} A_{2} y$

$q_{1}$ and $q_{2}$ are independent iff $A_{1} V A_{2} = 0$

Simple Regression Example

For $y_{i} = β_{0} + ε_{i}$
WTS ${\hat{σ}}^{2} = s^{2} = \frac{\sum_{}^{} (y_{i} - {\hat{y}}_{i})^{2}}{n - 1}$ is unbiased for $σ^{2}$

and $\frac{(n - 1) s^{2}}{σ^{2}} \sim χ_{n - 1}^{2}$

\begin{aligned} ⟹ F_{*} & = \frac{\frac{S_{x x} {\hat{β_{1}}}^{2}}{σ^{2}}}{\frac{(n - 2) s^{2}}{(n - 2) σ^{2}}} \\ = \frac{S_{x x} {\hat{β_{1}}}^{2}}{s^{2}} \sim F_{(1, n - 2)} \end{aligned}

We have:

\begin{aligned} y & = [y_{1}, y_{2}, \dots, y_{n}], | u | = n \\ u_{1} = [1, 1, \dots, 1] \cdot \frac{1}{| | u_{1} | |} = [1, 1, \dots, 1] \cdot \frac{1}{\sqrt{n}} \\ (\begin{array}{c} \bar{y} \\ \bar{y} \\ ⋮ \\ \bar{y} \end{array}) & = ⟨ y, u ⟩ \cdot u \\ \bar{y} & = ⟨ y, u ⟩ \cdot \frac{1}{\sqrt{n}} \\ = \frac{y_{1} \cdot 1 + \dots + y_{n} \cdot 1}{n} \end{aligned}

Least Squares Estimator

\begin{aligned} min ⟨ y - (\begin{array}{c} β_{0} \\ β_{0} \\ ⋮ \\ β_{0} \end{array}), y - (\begin{array}{c} β_{0} \\ β_{0} \\ ⋮ \\ β_{0} \end{array}) ⟩ & = min \sum_{}^{} (y_{i} - β_{0})^{2} \\ ⟹ - \sum_{}^{} 2 [y_{i} - β_{0}] \overset{w a n t}{=} 0 \\ ⟹ β_{0} = \bar{y} \end{aligned}

\begin{aligned} y & = \underset{(1)}{\underset{―}{\bar{y} (\begin{array}{c} 1 \\ 1 \\ ⋮ \\ 1 \end{array})}} + \underset{(2)}{\underset{―}{(y - \bar{y} (\begin{array}{c} 1 \\ 1 \\ ⋮ \\ 1 \end{array}))}} \\ = {\hat{β}}_{0, MLE} (\begin{array}{c} 1 \\ 1 \\ ⋮ \\ 1 \end{array}) + \vec{ε} \\ = | | u | | \cdot {\hat{β}}_{0, M L E} \cdot u + \vec{ε} \end{aligned}

note (1) and (2) are orthogonal

If we have orthonormal basis in $R^{3}, {u_{1}, u_{2}, u_{3}}$

u_{1} = \frac{1}{\sqrt{3}} [\begin{matrix} 1 \\ 1 \\ 1 \end{matrix}], u_{2} = \frac{1}{\sqrt{6}} [\begin{matrix} - 2 \\ 1 \\ 1 \end{matrix}], u_{3} = \frac{1}{\sqrt{3}} [\begin{matrix} 0 \\ - 1 \\ 1 \end{matrix}]

\begin{aligned} y & = \sum_{i = 1}^{3} ⟨ y, u_{i} ⟩ \cdot u_{i} \\ w_{i} & = ⟨ y, u_{i} ⟩ \\ w_{1} & = \frac{y_{1} + y_{2} + y_{2}}{\sqrt{3}} \sim N (\sqrt{3} β_{0}, σ^{2}) \\ w_{2} & = \frac{- 2 y_{1} + y_{2} + y_{3}}{\sqrt{6}} \sim N (0, σ^{2}) \\ w_{3} & = \frac{- y_{2} + y_{3}}{\sqrt{2}} \sim N (0, σ^{2}) \end{aligned}

Hypothesis
$H_{0} : β_{0} = 0$
if true then:

\begin{aligned} ⟹ w_{1}, w_{2}, w_{3} \sim N (0, σ^{2}) \\ ⟹ \frac{⟨ y, u_{i} ⟩}{σ} \sim N (0, 1) \\ ⟹ \frac{⟨ y, u_{i} ⟩^{2}}{σ^{2}} \sim χ_{1}^{2} \end{aligned}

⟹ \frac{⟨ y, u_{1} ⟩^{2}}{(⟨ y, u_{2} ⟩^{2} + ⟨ y, u_{3} ⟩^{2}) / 2} \sim F_{1, 2}

numerator and denominator need to be independent

\begin{array}{r} y_{*} = [\begin{array}{c} \frac{y_{1}}{σ} \\ \frac{y_{2}}{σ} \\ \frac{y_{3}}{σ} \end{array}], μ = E [y_{*}] = [\begin{array}{c} 0 \\ 0 \\ 0 \end{array}], c o v . = (\begin{array}{c} 1 & 0 & 0 \\ 0 & 1 & 0 \\ 0 & 0 & 1 \end{array}) \end{array}

$c o v (y_{i}, y_{j}) = 0, i \neq j$ since they are independent

y_{*} \sim N (μ, c o v)

Applying 16.4:

goal: re-express ${[\frac{y_{1} + y_{2} + y_{3}}{\sqrt{3} σ}]}^{2}$ as $y_{*}^{T} A y_{*}$ and

\begin{aligned} ⟹ (y_{*} \cdot [\begin{array}{c} \frac{1}{\sqrt{3}} \\ \frac{1}{\sqrt{3}} \\ \frac{1}{\sqrt{3}} \end{array}]) \cdot (y_{*} \cdot [\begin{array}{c} \frac{1}{\sqrt{3}} \\ \frac{1}{\sqrt{3}} \\ \frac{1}{\sqrt{3}} \end{array}])^{T} \\ ⟹ y_{*}^{T} \cdot [\begin{array}{c} \frac{1}{\sqrt{3}} \\ \frac{1}{\sqrt{3}} \\ \frac{1}{\sqrt{3}} \end{array}] \cdot [\frac{1}{\sqrt{3}}, \frac{1}{\sqrt{3}}, \frac{1}{\sqrt{3}}] \cdot y_{*} \\ (1) & ⟹ y_{*}^{T} \cdot M_{3 \times 3} \cdot y_{*} = q_{1} \end{aligned}

M_{3 \times 3} = [\begin{matrix} \frac{1}{3} & \frac{1}{3} & \frac{1}{3} \\ \frac{1}{3} & \frac{1}{3} & \frac{1}{3} \\ \frac{1}{3} & \frac{1}{3} & \frac{1}{3} \end{matrix}] = A_{N}

rank = 1

\begin{matrix} (2) & θ = \frac{μ^{T} M_{3 \times 3} μ}{2} = 0 \end{matrix}

WTS $A_{N} \cdot V$ is idempotent:

(A_{N} \cdot V) (A_{N} \cdot V) = (A_{N} \cdot I_{3 \times 3}) (A_{N} \cdot I_{3 \times 3}) = A^{2}

= [\begin{matrix} \frac{1}{3} & \frac{1}{3} & \frac{1}{3} \\ \frac{1}{3} & \frac{1}{3} & \frac{1}{3} \\ \frac{1}{3} & \frac{1}{3} & \frac{1}{3} \end{matrix}] [\begin{matrix} \frac{1}{3} & \frac{1}{3} & \frac{1}{3} \\ \frac{1}{3} & \frac{1}{3} & \frac{1}{3} \\ \frac{1}{3} & \frac{1}{3} & \frac{1}{3} \end{matrix}] = [\begin{matrix} \frac{1}{3} & \frac{1}{3} & \frac{1}{3} \\ \frac{1}{3} & \frac{1}{3} & \frac{1}{3} \\ \frac{1}{3} & \frac{1}{3} & \frac{1}{3} \end{matrix}]

$⟹ q_{1} = [\frac{y_{1} + y_{2} + y_{3}}{\sqrt{3} σ}]^{2} \sim χ^{2} (0, 1)$

Applying 16.5

\begin{aligned} = (y_{*} \cdot [\begin{array}{c} \frac{- 2}{\sqrt{6}} \\ \frac{1}{\sqrt{6}} \\ \frac{1}{\sqrt{6}} \end{array}]) (y_{*} \cdot [\begin{array}{c} \frac{- 2}{\sqrt{6}} \\ \frac{1}{\sqrt{6}} \\ \frac{1}{\sqrt{6}} \end{array}])^{T} + (y_{*} \cdot [\begin{array}{c} 0 \\ \frac{- 1}{\sqrt{2}} \\ \frac{1}{\sqrt{2}} \end{array}]) (y_{*} \cdot [\begin{array}{c} 0 \\ \frac{- 1}{\sqrt{2}} \\ \frac{1}{\sqrt{2}} \end{array}])^{T} \\ = (y_{*} \cdot [\begin{array}{c} \frac{- 2}{\sqrt{6}} \\ \frac{1}{\sqrt{6}} \\ \frac{1}{\sqrt{6}} \end{array}]) \cdot ([- \frac{2}{\sqrt{6}}, \frac{1}{\sqrt{6}}, \frac{1}{\sqrt{6}}] \cdot y_{*}^{T}) + (y_{*} \cdot [\begin{array}{c} 0 \\ \frac{- 1}{\sqrt{2}} \\ \frac{1}{\sqrt{2}} \end{array}]) \cdot [0, \frac{- 1}{\sqrt{2}}, \frac{1}{\sqrt{2}}] \cdot y_{*}^{T} \\ = y_{*} [\begin{array}{c} \frac{4}{6} & \frac{2}{6} & \frac{2}{6} \\ \frac{- 2}{6} & \frac{1}{6} & \frac{1}{6} \\ \frac{- 2}{6} & \frac{1}{6} & \frac{1}{6} \end{array}] y_{*}^{T} + y_{*} [\begin{array}{c} 0 & 0 & 0 \\ 0 & \frac{- 1}{2} & \frac{- 1}{2} \\ 0 & \frac{1}{2} & \frac{1}{2} \end{array}] y_{*}^{T} \\ = y_{*} ([\begin{array}{c} \frac{4}{6} & \frac{2}{6} & \frac{2}{6} \\ \frac{- 2}{6} & \frac{1}{6} & \frac{1}{6} \\ \frac{- 2}{6} & \frac{1}{6} & \frac{1}{6} \end{array}] + [\begin{array}{c} 0 & 0 & 0 \\ 0 & \frac{- 1}{2} & \frac{- 1}{2} \\ 0 & \frac{1}{2} & \frac{1}{2} \end{array}]) y_{*}^{T} \\ (3) & = y_{*} \underset{A_{D}}{([\begin{array}{c} \frac{4}{6} & \frac{- 2}{6} & \frac{- 2}{6} \\ \frac{- 2}{6} & \frac{4}{6} & \frac{- 2}{6} \\ \frac{- 2}{6} & \frac{- 2}{6} & \frac{4}{6} \end{array}]}) y_{*}^{T} = q_{2} \end{aligned}

Now need to show

(A_{D} V) (A_{D} V) = A_{D} V

which is true

Then WTS $A_{N} V A_{D} = 0$

⟹ A_{N} I_{3 \times 3} A_{D} = 0

which is true. Thus $q_{1}$ and $q_{2}$ are independent by 16.5