10. Jeffreys' Prior

Jeffreys' Prior

Jeffreys (1961) suggested a default rule for generating a prior distribution of a parameter, $θ$ in a sampling model $p (Y | θ)$ . Jeffreys’ prior is given by

\begin{aligned} p_{J} \propto \sqrt{I (θ)} \\ where I (θ) = - E [\frac{\partial^{2} \log p (Y | θ)}{\partial θ^{2}}] \end{aligned}

is the Fisher information.

Example

$Y \sim N (μ, σ^{2})$ , Jefferys' prior distribution $p_{J} (θ)$ :

\begin{aligned} p ({y} | θ) & = {(\frac{1}{2 π})}^{n / 2} \cdot {(σ^{2})}^{- n / 2} \cdot \exp {\sum_{}^{} (y - μ)^{2} \cdot - \frac{1}{2 σ^{2}}} \\ l & = \frac{n}{2} \ln (1) - \frac{n}{2} \ln (2 π) - \frac{n}{2} \ln (σ^{2}) - \frac{1}{σ^{2}} \sum_{}^{} \frac{(y - μ)^{2}}{2} \\ \frac{\partial l}{\partial σ^{2}} & = - \frac{n}{2} (σ^{2})^{- 1} + \sum_{}^{} \frac{(y - μ)^{2}}{2} (σ^{2})^{- 2} \\ \frac{\partial^{2} l}{\partial (σ^{2})^{2}} & = \frac{n}{2} [\frac{1}{σ^{4}}] - \sum_{}^{} (y - μ)^{2} (\frac{1}{σ^{6}}) \\ = \frac{n}{2} [\frac{1}{σ^{4}}] - [\sum_{}^{} \underset{chi squared}{\underset{⏟}{{(\frac{(y - μ)}{σ})}^{2}}}] \frac{1}{σ^{4}} \\ ⟹ E [\frac{\partial^{2} l}{\partial (σ^{2})^{2}}] & = \frac{n}{2} [\frac{1}{σ^{2}}] - \frac{1}{σ^{4}} n = - \frac{n}{2} [\frac{1}{σ^{2}}] \\ ⟹ - E [\frac{\partial^{2} l}{\partial (σ^{2})^{2}}] & = \frac{n}{2} [\frac{1}{σ^{4}}] = I (σ^{2}) \end{aligned}

⟹ p_{J} (σ^{2}) \propto \sqrt{\frac{1}{σ^{4}}} = \frac{1}{σ^{2}}