8. Bayesian Analysis of Variance from Normal Likelihood
Gamma Distribution
g a m m a ( r , v )
p ( y ) = v r Γ ( r ) y r − 1 e − v y y > 0 , r > 0 , v > 0
Inverse Gamma Prior
If y ∼ g a m m a ( r , v ) find the pdf of W = 1 y
Using Change of Variables
⟹ y = 1 W ⟹ d y d W = 1 W 2
f W ( w ) = f Y ( 1 w ) ⋅ 1 w 2 = v r Γ ( r ) ( 1 w ) r − 1 e − v ⋅ 1 w ⋅ 1 w 2 = w − 2 v r Γ ( r ) w − r + 1 e − v ⋅ 1 w = v r Γ ( r ) w − r − 1 e − v ⋅ 1 w Using CDF technique:
P ( W ≤ w ) = P ( 1 Y ≤ w ) = P ( 1 w ≤ Y ) = 1 − P ( Y < 1 w ) = 1 − F y ( 1 w )
d d w F W ( w ) = f W ( w ) = ( − 1 ) f y ( w − 1 ) [ − w 2 ] Inverse Gamma to Chi-Squared
If x ∼ g a m m a ( r = k 2 , v = 1 2 ) ⟹ x ∼ χ 2 ( d f = k ) f X ( x ) = ( 1 2 ) k / 2 1 Γ ( k 2 ) x ( k 2 − 1 ) e − 1 2 x x > 0 Let y ∼ Inv. Gamma ( r = k 2 , v = S 2 ) and define W = S Y , S > 0 W ∼ G a m m a ( r = k 2 , v = 1 2 ) ≡ χ 2 ( k ) Case: W > 0
F W ( w ) = P [ W ≤ w ] = P [ S Y < w ] = P [ S w < Y ] = 1 − P [ Y ≤ S w ] d d w F W ( w ) = f W ( w ) = − 1 f y ( S w − 1 ) [ − S w − 2 ] ⟹ f W ( w ) = S w − 2 f y ( S w ) ⟹ f W ( w ) = S w − 2 ( S 2 ) k / 2 1 Γ ( k 2 ) ( S w ) − k 2 − 1 exp { − S 2 ( w S ) } ⟹ f W ( w ) = ( S ) k / 2 + 1 ( 1 2 ) k / 2 w − 2 1 Γ ( k 2 ) ( S ) − k / 2 − 1 w k / 2 + 1 exp { − w 2 } ⟹ f W ( w ) = ( 1 2 ) k / 2 1 Γ ( k 2 ) w k / 2 − 1 exp { − 1 2 w } Variance Posterior
Likelihood P ( y i | σ 2 ) = [ 1 2 π σ 2 ] 1 / 2 exp { − ( y i − μ ) 2 2 σ 2 } = [ 1 2 π ] 1 / 2 ( σ 2 ) − 1 / 2 exp { − ( y i − μ ) 2 2 σ 2 }
For multiple observations:
P ( { y i } | σ 2 ) = [ 1 2 π ] n / 2 ( σ 2 ) − n / 2 exp { ∑ − ( y i − μ ) 2 2 ( 1 σ 2 ) }
p ( σ 2 | { y i } ) ∝ ( σ 2 ) r − 1 exp { − v 1 σ 2 } ⋅ [ 1 2 π ] n / 2 ( σ 2 ) − n / 2 exp { ∑ − ( y i − μ ) 2 2 ( 1 σ 2 ) } ∝ ( σ 2 ) r − n / 2 − 1 exp { − v 1 σ 2 + ∑ − ( y i − μ ) 2 2 ( 1 σ 2 ) } ∝ ( σ 2 ) − ( r + n / 2 ) − 1 exp { ( − v + ∑ − ( y i − μ ) 2 2 ) ( 1 σ 2 ) } Inverse Gamma r ∗ = n 2 + r and v ∗ = ∑ ( y i − μ ) 2 2 + v
Example
The strength of an item is known to be Normally distributed with mean 200 and unknown variance σ 2 . A random sample of ten items is taken and their strength measured. The strengths are:
215 186 216 203 221 188 202 192 208 195
P ( { y i } | σ 2 ) = [ 1 2 π ] n / 2 ( σ 2 ) − n / 2 exp { ∑ − ( y i − μ ) 2 2 ( 1 σ 2 ) } ⟹ p ( { y i } | σ 2 ) ∝ ( σ 2 ) − 10 / 2 ⋅ exp { − [ 1428 2 ] [ 1 σ 2 ] } Uniform prior p ( σ 2 ) ∝ 1
p ( σ 2 | { y i } ) = p ( σ 2 ) p ( { y i } | σ 2 ) = p ( { y i } | σ 2 ) ∝ ( σ 2 ) − 10 / 2 ⋅ exp { − [ 1428 2 ] [ 1 σ 2 ] } = p ( { y i } | σ 2 ) ∝ ( σ 2 ) − 8 / 2 − 1 ⋅ exp { − [ 1428 2 ] [ 1 σ 2 ] } inv. gamma(r ∗ = 8 ⏞ k ∗ 2 , v ∗ = 1428 ⏞ S ∗ 2 )
by chi theorem W = S ∗ σ 2 ∼ χ 2 ( d f = k ∗ )
P [ χ 0.025 , 8 2 < 1428 σ 2 < χ 0.975 , 8 2 ] = 0.95 P [ 1428 χ 0.025 , 8 2 > σ 2 > 1428 χ 0.975 , 8 2 ] where we are doing area to the left for 0.025 and and 0.975.
For H 0 : σ ≤ 8 ⟹ σ 2 ≤ 64
P [ H 0 | D a t a ] = P [ σ 2 ≤ 64 ] = P [ 1 64 ≤ 1 σ 2 ] = P [ 1428 64 ≤ 1428 σ 2 ] = P [ 1428 64 ≤ χ 8 2 ] ≤ 0.005 reject H 0
Example
Using Jefferys' Prior p ( σ 2 ) ∝ 1 σ 2 = ( σ 2 ) − 1
p ( σ 2 | { y } ) ∝ ( σ 2 ) − n / 2 − 1 exp { − [ ∑ ( y − μ ) 2 2 ] [ 1 σ 2 ] }