8. Bayesian Analysis of Variance from Normal Likelihood

Gamma Distribution

gamma(r,v)

p(y)=vrΓ(r)yr1evy

y>0,r>0,v>0

Inverse Gamma Prior

If ygamma(r,v) find the pdf of W=1y

Using Change of Variables
y=1WdydW=1W2

fW(w)=fY(1w)1w2=vrΓ(r)(1w)r1ev1w1w2=w2vrΓ(r)wr+1ev1w=vrΓ(r)wr1ev1w

Using CDF technique:

P(Ww)=P(1Yw)=P(1wY)=1P(Y<1w)=1Fy(1w)

ddwFW(w)=fW(w)=(1)fy(w1)[w2]

Inverse Gamma to Chi-Squared

If xgamma(r=k2,v=12)xχ2(df=k)fX(x)=(12)k/21Γ(k2)x(k21)e12xx>0Let yInv. Gamma(r=k2,v=S2) and define W=SY,S>0WGamma(r=k2,v=12)χ2(k)

Case: W>0

FW(w)=P[Ww]=P[SY<w]=P[Sw<Y]=1P[YSw]ddwFW(w)=fW(w)=1fy(Sw1)[Sw2]fW(w)=Sw2fy(Sw)fW(w)=Sw2(S2)k/21Γ(k2)(Sw)k21exp{S2(wS)}fW(w)=(S)k/2+1(12)k/2w21Γ(k2)(S)k/21wk/2+1exp{w2}fW(w)=(12)k/21Γ(k2)wk/21exp{12w}

Variance Posterior

Likelihood P(yi|σ2)=[12πσ2]1/2exp{(yiμ)22σ2}=[12π]1/2(σ2)1/2exp{(yiμ)22σ2}
For multiple observations:
P({yi}|σ2)=[12π]n/2(σ2)n/2exp{(yiμ)22(1σ2)}

p(σ2|{yi})(σ2)r1exp{v1σ2}[12π]n/2(σ2)n/2exp{(yiμ)22(1σ2)}(σ2)rn/21exp{v1σ2+(yiμ)22(1σ2)}(σ2)(r+n/2)1exp{(v+(yiμ)22)(1σ2)}

Inverse Gamma r=n2+r and v=(yiμ)22+v

Example

The strength of an item is known to be Normally distributed with mean 200 and unknown variance σ2. A random sample of ten items is taken and their strength measured. The strengths are:
215 186 216 203 221 188 202 192 208 195

P({yi}|σ2)=[12π]n/2(σ2)n/2exp{(yiμ)22(1σ2)}p({yi}|σ2)(σ2)10/2exp{[14282][1σ2]}

Uniform prior p(σ2)1

p(σ2|{yi})=p(σ2)p({yi}|σ2)=p({yi}|σ2)(σ2)10/2exp{[14282][1σ2]}=p({yi}|σ2)(σ2)8/21exp{[14282][1σ2]}

inv. gamma(r=8k2,v=1428S2)

by chi theorem W=Sσ2χ2(df=k)

P[χ0.025,82<1428σ2<χ0.975,82]=0.95P[1428χ0.025,82>σ2>1428χ0.975,82]

where we are doing area to the left for 0.025 and and 0.975.

For H0:σ8σ264

P[H0|Data]=P[σ264]=P[1641σ2]=P[1428641428σ2]=P[142864χ82]0.005

reject H0

Example

Using Jefferys' Prior p(σ2)1σ2=(σ2)1

p(σ2|{y})(σ2)n/21exp{[(yμ)22][1σ2]}