Where is my mistake in this definition of Bayes Factor?
.everyoneloves__top-leaderboard:empty,.everyoneloves__mid-leaderboard:empty{ margin-bottom:0;
}
up vote
5
down vote
favorite
From "The Bayesian Choice" by Christian P. Robert.
The definition of the Bayes factor is given to be the ratio of the posterior probabilities of the null and the alternative hypothesis over the ratio of the prior probabilities of the null and alternative.
ie $$B_{01}^{pi}= frac{ frac{P(theta in Theta_{0}|x)}{P(theta in Theta_{1}|x)}}{frac{p(theta in Theta_{0})}{ p(theta in Theta_{1})}}$$
Which the author shortly simplifies to be
$$B_{01}^{pi}=frac{int_{Theta_{0}}f(x|theta_{0})pi_{0}(theta)dtheta}{int_{Theta_{1}}f(x|theta_{1})pi_{1}(theta)dtheta}$$
But when I try to write out all the terms and use that
$$pi(theta|x)=frac{f(x|theta) pi(theta)}{int_{Theta}f(x|theta) pi(theta)dtheta}$$
I get
$$B_{01}^{pi}=frac{f(x|theta_{0})int_{Theta_{1}}f(x|theta_{1})pi_{1}(theta)dtheta}{f(x|theta_{1})int_{Theta_{0}}f(x|theta_{0})pi_{0}(theta)dtheta}$$
instead.
Anyone have any idea of where I went wrong? Probably some simple mistake I made or error.
hypothesis-testing bayesian mathematical-statistics odds-ratio marginal
add a comment |
up vote
5
down vote
favorite
From "The Bayesian Choice" by Christian P. Robert.
The definition of the Bayes factor is given to be the ratio of the posterior probabilities of the null and the alternative hypothesis over the ratio of the prior probabilities of the null and alternative.
ie $$B_{01}^{pi}= frac{ frac{P(theta in Theta_{0}|x)}{P(theta in Theta_{1}|x)}}{frac{p(theta in Theta_{0})}{ p(theta in Theta_{1})}}$$
Which the author shortly simplifies to be
$$B_{01}^{pi}=frac{int_{Theta_{0}}f(x|theta_{0})pi_{0}(theta)dtheta}{int_{Theta_{1}}f(x|theta_{1})pi_{1}(theta)dtheta}$$
But when I try to write out all the terms and use that
$$pi(theta|x)=frac{f(x|theta) pi(theta)}{int_{Theta}f(x|theta) pi(theta)dtheta}$$
I get
$$B_{01}^{pi}=frac{f(x|theta_{0})int_{Theta_{1}}f(x|theta_{1})pi_{1}(theta)dtheta}{f(x|theta_{1})int_{Theta_{0}}f(x|theta_{0})pi_{0}(theta)dtheta}$$
instead.
Anyone have any idea of where I went wrong? Probably some simple mistake I made or error.
hypothesis-testing bayesian mathematical-statistics odds-ratio marginal
add a comment |
up vote
5
down vote
favorite
up vote
5
down vote
favorite
From "The Bayesian Choice" by Christian P. Robert.
The definition of the Bayes factor is given to be the ratio of the posterior probabilities of the null and the alternative hypothesis over the ratio of the prior probabilities of the null and alternative.
ie $$B_{01}^{pi}= frac{ frac{P(theta in Theta_{0}|x)}{P(theta in Theta_{1}|x)}}{frac{p(theta in Theta_{0})}{ p(theta in Theta_{1})}}$$
Which the author shortly simplifies to be
$$B_{01}^{pi}=frac{int_{Theta_{0}}f(x|theta_{0})pi_{0}(theta)dtheta}{int_{Theta_{1}}f(x|theta_{1})pi_{1}(theta)dtheta}$$
But when I try to write out all the terms and use that
$$pi(theta|x)=frac{f(x|theta) pi(theta)}{int_{Theta}f(x|theta) pi(theta)dtheta}$$
I get
$$B_{01}^{pi}=frac{f(x|theta_{0})int_{Theta_{1}}f(x|theta_{1})pi_{1}(theta)dtheta}{f(x|theta_{1})int_{Theta_{0}}f(x|theta_{0})pi_{0}(theta)dtheta}$$
instead.
Anyone have any idea of where I went wrong? Probably some simple mistake I made or error.
hypothesis-testing bayesian mathematical-statistics odds-ratio marginal
From "The Bayesian Choice" by Christian P. Robert.
The definition of the Bayes factor is given to be the ratio of the posterior probabilities of the null and the alternative hypothesis over the ratio of the prior probabilities of the null and alternative.
ie $$B_{01}^{pi}= frac{ frac{P(theta in Theta_{0}|x)}{P(theta in Theta_{1}|x)}}{frac{p(theta in Theta_{0})}{ p(theta in Theta_{1})}}$$
Which the author shortly simplifies to be
$$B_{01}^{pi}=frac{int_{Theta_{0}}f(x|theta_{0})pi_{0}(theta)dtheta}{int_{Theta_{1}}f(x|theta_{1})pi_{1}(theta)dtheta}$$
But when I try to write out all the terms and use that
$$pi(theta|x)=frac{f(x|theta) pi(theta)}{int_{Theta}f(x|theta) pi(theta)dtheta}$$
I get
$$B_{01}^{pi}=frac{f(x|theta_{0})int_{Theta_{1}}f(x|theta_{1})pi_{1}(theta)dtheta}{f(x|theta_{1})int_{Theta_{0}}f(x|theta_{0})pi_{0}(theta)dtheta}$$
instead.
Anyone have any idea of where I went wrong? Probably some simple mistake I made or error.
hypothesis-testing bayesian mathematical-statistics odds-ratio marginal
hypothesis-testing bayesian mathematical-statistics odds-ratio marginal
edited Nov 24 at 5:48
Xi'an
52.4k688340
52.4k688340
asked Nov 24 at 1:13
Quality
1979
1979
add a comment |
add a comment |
1 Answer
1
active
oldest
votes
up vote
5
down vote
accepted
My deepest apologies, there is a typo in the final expression! Indeed, here is the complete text from my book (p.231):
Definition 5.5. The Bayes factor is the ratio of the posterior probabilities of the null and the alternative hypotheses over the
ratio of the prior probabilities of the null and the alternative
hypotheses, i.e., $$ B^pi_{01}(x) = {P(theta in Theta_ 0mid x)
over P(theta in Theta_1mid x)} bigg/ {pi(theta in Theta_ 0)
over pi(theta in Theta_ 1)}. $$ This ratio evaluates the
modification of the odds of $Theta_0$ against $Theta_1$ due to the
obs and can naturally be compared to $1$, although an exact
comparison scale can only be based upon a loss function. In the
particular case where $Theta_0={theta_0}$ and
$Theta_1={theta_1}$, the Bayes factor simplifies to the usual
likelihood ratio $$ B^pi_{01} (x) = {f(x|theta_0)over
f(x|theta_1)}. $$ In general, the Bayes factor depends on prior
information, but is still proposed as an ``objective'' Bayesian
answer, since it partly eliminates the influence of the prior modeling
and emphasizes the role of the observations. Actually, it can be
perceived as a Bayesian likelihood ratio since, if $pi_0$ is the
prior distribution under $H_0$ and $pi_1$ the prior distribution
under $H_1$, $B^pi_{01}(x)$ can be written as begin{equation}
B^pi_{01} (x) = {int_{Theta_0} f(x|theta_0)pi_0(theta)
,text{d}theta over int_{Theta_1} f(x|theta_1)pi_1(theta)
,text{d}theta}
=frac{m_0(x)}{m_1(x)},, end{equation} thus replacing the likelihoods with the marginals under both hypotheses.
The prior is thus defined as a mixture:
$$pi(theta)=pi(theta in Theta_ 0)timespi_0(theta)timesmathbb{I}_{Theta_0}(theta)+pi(theta in Theta_ 1)timespi_1(theta)timesmathbb{I}_{Theta_1}(theta)$$where
$$pi(theta in Theta_ 0)=rho_0qquadtext{and}qquadpi(theta in Theta_ 1)=1-rho_0stackrel{text{def}}{=}rho_1$$
are the prior weights of both hypotheses and
$$int_{Theta_0} pi_0(theta_0)text{d}theta_0=int_{Theta_1} pi_1(theta_1)text{d}theta_1=1$$Therefore
begin{align*}P(theta in Theta_ 0|x)=int_{Theta_0} pi(theta_0|x)text{d}theta_0&=int_{Theta_0} pi(theta_0|x)text{d}theta_0\
&= frac{int_{Theta_0} pi(theta_0)f(x|theta_0)text{d}theta_0}{int_{Theta_0} pi(theta_0)f(x|theta_0)text{d}theta_0+int_{Theta_1} pi(theta_1)f(x|theta_1)text{d}theta_1}\
&=frac{int_{Theta_0} rho_0pi_0(theta_0)f(x|theta_0)text{d}theta_0}{int_{Theta_0} rho_0pi_0(theta_0)f(x|theta_0)text{d}theta_0+int_{Theta_1} rho_1pi_1(theta_1)f(x|theta_1)text{d}theta_1}
end{align*}
and
$$P(theta in Theta_1mid x)=frac{int_{Theta_1} rho_1pi_1(theta_1)f(x|theta_1)text{d}theta_1}{int_{Theta_0} rho_0pi_0(theta_0)f(x|theta_0)text{d}theta_0+int_{Theta_1} rho_1pi_1(theta_1)f(x|theta_1)text{d}theta_1}$$Hence,
$$frac{P(theta in Theta_ 0|x)}{P(theta in Theta_1mid x)}=frac{rho_0int_{Theta_0} pi_0(theta_0)f(x|theta_0)text{d}theta_0}{rho_1int_{Theta_1} pi_1(theta_1)f(x|theta_1)text{d}theta_1}=frac{rho_0}{1-rho_0}B^pi_{01}(x)$$
This hopefully explains where the final expression comes from.
Alas, there is a mistake in the last ratio of integrals in the quoted text (presumably due to a cut & paste)
$$B_{01}^{pi}=frac{int_{Theta_{0}}f(x|theta_{0})pi_{0}(theta)dtheta}{int_{Theta_{1}}f(x|theta_{1})pi_{1}(theta)dtheta}$$
which should be (when identifying the integrands differently in the two integrals to signify that the integrals are over two different parameter spaces)
$$B_{01}^{pi}=frac{int_{Theta_{0}}f(x|theta_{0})pi_{0}(theta_0)dtheta_0}{int_{Theta_{1}}f(x|theta_{1})pi_{1}(theta_1)dtheta_1}$$or (with another choice of representation of the integrands)
$$B_{01}^{pi}=frac{int_{Theta_{0}}f(x|theta)pi_{0}(theta)dtheta}{int_{Theta_{1}}f(x|theta)pi_{1}(theta)dtheta}$$
The only case when the notations $theta_0$ and $theta_1$ are important is when both null and alternative hypotheses are point hypotheses, i.e., when $Theta_0={theta_0}$ and $Theta_1={theta_1}$, since both symbols then take specific values, like $theta_0=3$ and $theta_1=-2$. In this specific case, the posterior is concentrated on ${theta_0,theta_1}$ and the Bayes factor writes $$ B^pi_{01} (x) = {f(x|theta_0)over f(x|theta_1)}$$ since
$$P(theta=theta_0|x)=frac{overbrace{pi(theta_0)}^{rho_0}f(x|theta_0)}{pi(theta_0)f(x|theta_0)+underbrace{pi(theta_1)}_{rho_1}f(x|theta_1)}$$
Thank you for pointing out this error, to be added to the list of typos.
add a comment |
1 Answer
1
active
oldest
votes
1 Answer
1
active
oldest
votes
active
oldest
votes
active
oldest
votes
up vote
5
down vote
accepted
My deepest apologies, there is a typo in the final expression! Indeed, here is the complete text from my book (p.231):
Definition 5.5. The Bayes factor is the ratio of the posterior probabilities of the null and the alternative hypotheses over the
ratio of the prior probabilities of the null and the alternative
hypotheses, i.e., $$ B^pi_{01}(x) = {P(theta in Theta_ 0mid x)
over P(theta in Theta_1mid x)} bigg/ {pi(theta in Theta_ 0)
over pi(theta in Theta_ 1)}. $$ This ratio evaluates the
modification of the odds of $Theta_0$ against $Theta_1$ due to the
obs and can naturally be compared to $1$, although an exact
comparison scale can only be based upon a loss function. In the
particular case where $Theta_0={theta_0}$ and
$Theta_1={theta_1}$, the Bayes factor simplifies to the usual
likelihood ratio $$ B^pi_{01} (x) = {f(x|theta_0)over
f(x|theta_1)}. $$ In general, the Bayes factor depends on prior
information, but is still proposed as an ``objective'' Bayesian
answer, since it partly eliminates the influence of the prior modeling
and emphasizes the role of the observations. Actually, it can be
perceived as a Bayesian likelihood ratio since, if $pi_0$ is the
prior distribution under $H_0$ and $pi_1$ the prior distribution
under $H_1$, $B^pi_{01}(x)$ can be written as begin{equation}
B^pi_{01} (x) = {int_{Theta_0} f(x|theta_0)pi_0(theta)
,text{d}theta over int_{Theta_1} f(x|theta_1)pi_1(theta)
,text{d}theta}
=frac{m_0(x)}{m_1(x)},, end{equation} thus replacing the likelihoods with the marginals under both hypotheses.
The prior is thus defined as a mixture:
$$pi(theta)=pi(theta in Theta_ 0)timespi_0(theta)timesmathbb{I}_{Theta_0}(theta)+pi(theta in Theta_ 1)timespi_1(theta)timesmathbb{I}_{Theta_1}(theta)$$where
$$pi(theta in Theta_ 0)=rho_0qquadtext{and}qquadpi(theta in Theta_ 1)=1-rho_0stackrel{text{def}}{=}rho_1$$
are the prior weights of both hypotheses and
$$int_{Theta_0} pi_0(theta_0)text{d}theta_0=int_{Theta_1} pi_1(theta_1)text{d}theta_1=1$$Therefore
begin{align*}P(theta in Theta_ 0|x)=int_{Theta_0} pi(theta_0|x)text{d}theta_0&=int_{Theta_0} pi(theta_0|x)text{d}theta_0\
&= frac{int_{Theta_0} pi(theta_0)f(x|theta_0)text{d}theta_0}{int_{Theta_0} pi(theta_0)f(x|theta_0)text{d}theta_0+int_{Theta_1} pi(theta_1)f(x|theta_1)text{d}theta_1}\
&=frac{int_{Theta_0} rho_0pi_0(theta_0)f(x|theta_0)text{d}theta_0}{int_{Theta_0} rho_0pi_0(theta_0)f(x|theta_0)text{d}theta_0+int_{Theta_1} rho_1pi_1(theta_1)f(x|theta_1)text{d}theta_1}
end{align*}
and
$$P(theta in Theta_1mid x)=frac{int_{Theta_1} rho_1pi_1(theta_1)f(x|theta_1)text{d}theta_1}{int_{Theta_0} rho_0pi_0(theta_0)f(x|theta_0)text{d}theta_0+int_{Theta_1} rho_1pi_1(theta_1)f(x|theta_1)text{d}theta_1}$$Hence,
$$frac{P(theta in Theta_ 0|x)}{P(theta in Theta_1mid x)}=frac{rho_0int_{Theta_0} pi_0(theta_0)f(x|theta_0)text{d}theta_0}{rho_1int_{Theta_1} pi_1(theta_1)f(x|theta_1)text{d}theta_1}=frac{rho_0}{1-rho_0}B^pi_{01}(x)$$
This hopefully explains where the final expression comes from.
Alas, there is a mistake in the last ratio of integrals in the quoted text (presumably due to a cut & paste)
$$B_{01}^{pi}=frac{int_{Theta_{0}}f(x|theta_{0})pi_{0}(theta)dtheta}{int_{Theta_{1}}f(x|theta_{1})pi_{1}(theta)dtheta}$$
which should be (when identifying the integrands differently in the two integrals to signify that the integrals are over two different parameter spaces)
$$B_{01}^{pi}=frac{int_{Theta_{0}}f(x|theta_{0})pi_{0}(theta_0)dtheta_0}{int_{Theta_{1}}f(x|theta_{1})pi_{1}(theta_1)dtheta_1}$$or (with another choice of representation of the integrands)
$$B_{01}^{pi}=frac{int_{Theta_{0}}f(x|theta)pi_{0}(theta)dtheta}{int_{Theta_{1}}f(x|theta)pi_{1}(theta)dtheta}$$
The only case when the notations $theta_0$ and $theta_1$ are important is when both null and alternative hypotheses are point hypotheses, i.e., when $Theta_0={theta_0}$ and $Theta_1={theta_1}$, since both symbols then take specific values, like $theta_0=3$ and $theta_1=-2$. In this specific case, the posterior is concentrated on ${theta_0,theta_1}$ and the Bayes factor writes $$ B^pi_{01} (x) = {f(x|theta_0)over f(x|theta_1)}$$ since
$$P(theta=theta_0|x)=frac{overbrace{pi(theta_0)}^{rho_0}f(x|theta_0)}{pi(theta_0)f(x|theta_0)+underbrace{pi(theta_1)}_{rho_1}f(x|theta_1)}$$
Thank you for pointing out this error, to be added to the list of typos.
add a comment |
up vote
5
down vote
accepted
My deepest apologies, there is a typo in the final expression! Indeed, here is the complete text from my book (p.231):
Definition 5.5. The Bayes factor is the ratio of the posterior probabilities of the null and the alternative hypotheses over the
ratio of the prior probabilities of the null and the alternative
hypotheses, i.e., $$ B^pi_{01}(x) = {P(theta in Theta_ 0mid x)
over P(theta in Theta_1mid x)} bigg/ {pi(theta in Theta_ 0)
over pi(theta in Theta_ 1)}. $$ This ratio evaluates the
modification of the odds of $Theta_0$ against $Theta_1$ due to the
obs and can naturally be compared to $1$, although an exact
comparison scale can only be based upon a loss function. In the
particular case where $Theta_0={theta_0}$ and
$Theta_1={theta_1}$, the Bayes factor simplifies to the usual
likelihood ratio $$ B^pi_{01} (x) = {f(x|theta_0)over
f(x|theta_1)}. $$ In general, the Bayes factor depends on prior
information, but is still proposed as an ``objective'' Bayesian
answer, since it partly eliminates the influence of the prior modeling
and emphasizes the role of the observations. Actually, it can be
perceived as a Bayesian likelihood ratio since, if $pi_0$ is the
prior distribution under $H_0$ and $pi_1$ the prior distribution
under $H_1$, $B^pi_{01}(x)$ can be written as begin{equation}
B^pi_{01} (x) = {int_{Theta_0} f(x|theta_0)pi_0(theta)
,text{d}theta over int_{Theta_1} f(x|theta_1)pi_1(theta)
,text{d}theta}
=frac{m_0(x)}{m_1(x)},, end{equation} thus replacing the likelihoods with the marginals under both hypotheses.
The prior is thus defined as a mixture:
$$pi(theta)=pi(theta in Theta_ 0)timespi_0(theta)timesmathbb{I}_{Theta_0}(theta)+pi(theta in Theta_ 1)timespi_1(theta)timesmathbb{I}_{Theta_1}(theta)$$where
$$pi(theta in Theta_ 0)=rho_0qquadtext{and}qquadpi(theta in Theta_ 1)=1-rho_0stackrel{text{def}}{=}rho_1$$
are the prior weights of both hypotheses and
$$int_{Theta_0} pi_0(theta_0)text{d}theta_0=int_{Theta_1} pi_1(theta_1)text{d}theta_1=1$$Therefore
begin{align*}P(theta in Theta_ 0|x)=int_{Theta_0} pi(theta_0|x)text{d}theta_0&=int_{Theta_0} pi(theta_0|x)text{d}theta_0\
&= frac{int_{Theta_0} pi(theta_0)f(x|theta_0)text{d}theta_0}{int_{Theta_0} pi(theta_0)f(x|theta_0)text{d}theta_0+int_{Theta_1} pi(theta_1)f(x|theta_1)text{d}theta_1}\
&=frac{int_{Theta_0} rho_0pi_0(theta_0)f(x|theta_0)text{d}theta_0}{int_{Theta_0} rho_0pi_0(theta_0)f(x|theta_0)text{d}theta_0+int_{Theta_1} rho_1pi_1(theta_1)f(x|theta_1)text{d}theta_1}
end{align*}
and
$$P(theta in Theta_1mid x)=frac{int_{Theta_1} rho_1pi_1(theta_1)f(x|theta_1)text{d}theta_1}{int_{Theta_0} rho_0pi_0(theta_0)f(x|theta_0)text{d}theta_0+int_{Theta_1} rho_1pi_1(theta_1)f(x|theta_1)text{d}theta_1}$$Hence,
$$frac{P(theta in Theta_ 0|x)}{P(theta in Theta_1mid x)}=frac{rho_0int_{Theta_0} pi_0(theta_0)f(x|theta_0)text{d}theta_0}{rho_1int_{Theta_1} pi_1(theta_1)f(x|theta_1)text{d}theta_1}=frac{rho_0}{1-rho_0}B^pi_{01}(x)$$
This hopefully explains where the final expression comes from.
Alas, there is a mistake in the last ratio of integrals in the quoted text (presumably due to a cut & paste)
$$B_{01}^{pi}=frac{int_{Theta_{0}}f(x|theta_{0})pi_{0}(theta)dtheta}{int_{Theta_{1}}f(x|theta_{1})pi_{1}(theta)dtheta}$$
which should be (when identifying the integrands differently in the two integrals to signify that the integrals are over two different parameter spaces)
$$B_{01}^{pi}=frac{int_{Theta_{0}}f(x|theta_{0})pi_{0}(theta_0)dtheta_0}{int_{Theta_{1}}f(x|theta_{1})pi_{1}(theta_1)dtheta_1}$$or (with another choice of representation of the integrands)
$$B_{01}^{pi}=frac{int_{Theta_{0}}f(x|theta)pi_{0}(theta)dtheta}{int_{Theta_{1}}f(x|theta)pi_{1}(theta)dtheta}$$
The only case when the notations $theta_0$ and $theta_1$ are important is when both null and alternative hypotheses are point hypotheses, i.e., when $Theta_0={theta_0}$ and $Theta_1={theta_1}$, since both symbols then take specific values, like $theta_0=3$ and $theta_1=-2$. In this specific case, the posterior is concentrated on ${theta_0,theta_1}$ and the Bayes factor writes $$ B^pi_{01} (x) = {f(x|theta_0)over f(x|theta_1)}$$ since
$$P(theta=theta_0|x)=frac{overbrace{pi(theta_0)}^{rho_0}f(x|theta_0)}{pi(theta_0)f(x|theta_0)+underbrace{pi(theta_1)}_{rho_1}f(x|theta_1)}$$
Thank you for pointing out this error, to be added to the list of typos.
add a comment |
up vote
5
down vote
accepted
up vote
5
down vote
accepted
My deepest apologies, there is a typo in the final expression! Indeed, here is the complete text from my book (p.231):
Definition 5.5. The Bayes factor is the ratio of the posterior probabilities of the null and the alternative hypotheses over the
ratio of the prior probabilities of the null and the alternative
hypotheses, i.e., $$ B^pi_{01}(x) = {P(theta in Theta_ 0mid x)
over P(theta in Theta_1mid x)} bigg/ {pi(theta in Theta_ 0)
over pi(theta in Theta_ 1)}. $$ This ratio evaluates the
modification of the odds of $Theta_0$ against $Theta_1$ due to the
obs and can naturally be compared to $1$, although an exact
comparison scale can only be based upon a loss function. In the
particular case where $Theta_0={theta_0}$ and
$Theta_1={theta_1}$, the Bayes factor simplifies to the usual
likelihood ratio $$ B^pi_{01} (x) = {f(x|theta_0)over
f(x|theta_1)}. $$ In general, the Bayes factor depends on prior
information, but is still proposed as an ``objective'' Bayesian
answer, since it partly eliminates the influence of the prior modeling
and emphasizes the role of the observations. Actually, it can be
perceived as a Bayesian likelihood ratio since, if $pi_0$ is the
prior distribution under $H_0$ and $pi_1$ the prior distribution
under $H_1$, $B^pi_{01}(x)$ can be written as begin{equation}
B^pi_{01} (x) = {int_{Theta_0} f(x|theta_0)pi_0(theta)
,text{d}theta over int_{Theta_1} f(x|theta_1)pi_1(theta)
,text{d}theta}
=frac{m_0(x)}{m_1(x)},, end{equation} thus replacing the likelihoods with the marginals under both hypotheses.
The prior is thus defined as a mixture:
$$pi(theta)=pi(theta in Theta_ 0)timespi_0(theta)timesmathbb{I}_{Theta_0}(theta)+pi(theta in Theta_ 1)timespi_1(theta)timesmathbb{I}_{Theta_1}(theta)$$where
$$pi(theta in Theta_ 0)=rho_0qquadtext{and}qquadpi(theta in Theta_ 1)=1-rho_0stackrel{text{def}}{=}rho_1$$
are the prior weights of both hypotheses and
$$int_{Theta_0} pi_0(theta_0)text{d}theta_0=int_{Theta_1} pi_1(theta_1)text{d}theta_1=1$$Therefore
begin{align*}P(theta in Theta_ 0|x)=int_{Theta_0} pi(theta_0|x)text{d}theta_0&=int_{Theta_0} pi(theta_0|x)text{d}theta_0\
&= frac{int_{Theta_0} pi(theta_0)f(x|theta_0)text{d}theta_0}{int_{Theta_0} pi(theta_0)f(x|theta_0)text{d}theta_0+int_{Theta_1} pi(theta_1)f(x|theta_1)text{d}theta_1}\
&=frac{int_{Theta_0} rho_0pi_0(theta_0)f(x|theta_0)text{d}theta_0}{int_{Theta_0} rho_0pi_0(theta_0)f(x|theta_0)text{d}theta_0+int_{Theta_1} rho_1pi_1(theta_1)f(x|theta_1)text{d}theta_1}
end{align*}
and
$$P(theta in Theta_1mid x)=frac{int_{Theta_1} rho_1pi_1(theta_1)f(x|theta_1)text{d}theta_1}{int_{Theta_0} rho_0pi_0(theta_0)f(x|theta_0)text{d}theta_0+int_{Theta_1} rho_1pi_1(theta_1)f(x|theta_1)text{d}theta_1}$$Hence,
$$frac{P(theta in Theta_ 0|x)}{P(theta in Theta_1mid x)}=frac{rho_0int_{Theta_0} pi_0(theta_0)f(x|theta_0)text{d}theta_0}{rho_1int_{Theta_1} pi_1(theta_1)f(x|theta_1)text{d}theta_1}=frac{rho_0}{1-rho_0}B^pi_{01}(x)$$
This hopefully explains where the final expression comes from.
Alas, there is a mistake in the last ratio of integrals in the quoted text (presumably due to a cut & paste)
$$B_{01}^{pi}=frac{int_{Theta_{0}}f(x|theta_{0})pi_{0}(theta)dtheta}{int_{Theta_{1}}f(x|theta_{1})pi_{1}(theta)dtheta}$$
which should be (when identifying the integrands differently in the two integrals to signify that the integrals are over two different parameter spaces)
$$B_{01}^{pi}=frac{int_{Theta_{0}}f(x|theta_{0})pi_{0}(theta_0)dtheta_0}{int_{Theta_{1}}f(x|theta_{1})pi_{1}(theta_1)dtheta_1}$$or (with another choice of representation of the integrands)
$$B_{01}^{pi}=frac{int_{Theta_{0}}f(x|theta)pi_{0}(theta)dtheta}{int_{Theta_{1}}f(x|theta)pi_{1}(theta)dtheta}$$
The only case when the notations $theta_0$ and $theta_1$ are important is when both null and alternative hypotheses are point hypotheses, i.e., when $Theta_0={theta_0}$ and $Theta_1={theta_1}$, since both symbols then take specific values, like $theta_0=3$ and $theta_1=-2$. In this specific case, the posterior is concentrated on ${theta_0,theta_1}$ and the Bayes factor writes $$ B^pi_{01} (x) = {f(x|theta_0)over f(x|theta_1)}$$ since
$$P(theta=theta_0|x)=frac{overbrace{pi(theta_0)}^{rho_0}f(x|theta_0)}{pi(theta_0)f(x|theta_0)+underbrace{pi(theta_1)}_{rho_1}f(x|theta_1)}$$
Thank you for pointing out this error, to be added to the list of typos.
My deepest apologies, there is a typo in the final expression! Indeed, here is the complete text from my book (p.231):
Definition 5.5. The Bayes factor is the ratio of the posterior probabilities of the null and the alternative hypotheses over the
ratio of the prior probabilities of the null and the alternative
hypotheses, i.e., $$ B^pi_{01}(x) = {P(theta in Theta_ 0mid x)
over P(theta in Theta_1mid x)} bigg/ {pi(theta in Theta_ 0)
over pi(theta in Theta_ 1)}. $$ This ratio evaluates the
modification of the odds of $Theta_0$ against $Theta_1$ due to the
obs and can naturally be compared to $1$, although an exact
comparison scale can only be based upon a loss function. In the
particular case where $Theta_0={theta_0}$ and
$Theta_1={theta_1}$, the Bayes factor simplifies to the usual
likelihood ratio $$ B^pi_{01} (x) = {f(x|theta_0)over
f(x|theta_1)}. $$ In general, the Bayes factor depends on prior
information, but is still proposed as an ``objective'' Bayesian
answer, since it partly eliminates the influence of the prior modeling
and emphasizes the role of the observations. Actually, it can be
perceived as a Bayesian likelihood ratio since, if $pi_0$ is the
prior distribution under $H_0$ and $pi_1$ the prior distribution
under $H_1$, $B^pi_{01}(x)$ can be written as begin{equation}
B^pi_{01} (x) = {int_{Theta_0} f(x|theta_0)pi_0(theta)
,text{d}theta over int_{Theta_1} f(x|theta_1)pi_1(theta)
,text{d}theta}
=frac{m_0(x)}{m_1(x)},, end{equation} thus replacing the likelihoods with the marginals under both hypotheses.
The prior is thus defined as a mixture:
$$pi(theta)=pi(theta in Theta_ 0)timespi_0(theta)timesmathbb{I}_{Theta_0}(theta)+pi(theta in Theta_ 1)timespi_1(theta)timesmathbb{I}_{Theta_1}(theta)$$where
$$pi(theta in Theta_ 0)=rho_0qquadtext{and}qquadpi(theta in Theta_ 1)=1-rho_0stackrel{text{def}}{=}rho_1$$
are the prior weights of both hypotheses and
$$int_{Theta_0} pi_0(theta_0)text{d}theta_0=int_{Theta_1} pi_1(theta_1)text{d}theta_1=1$$Therefore
begin{align*}P(theta in Theta_ 0|x)=int_{Theta_0} pi(theta_0|x)text{d}theta_0&=int_{Theta_0} pi(theta_0|x)text{d}theta_0\
&= frac{int_{Theta_0} pi(theta_0)f(x|theta_0)text{d}theta_0}{int_{Theta_0} pi(theta_0)f(x|theta_0)text{d}theta_0+int_{Theta_1} pi(theta_1)f(x|theta_1)text{d}theta_1}\
&=frac{int_{Theta_0} rho_0pi_0(theta_0)f(x|theta_0)text{d}theta_0}{int_{Theta_0} rho_0pi_0(theta_0)f(x|theta_0)text{d}theta_0+int_{Theta_1} rho_1pi_1(theta_1)f(x|theta_1)text{d}theta_1}
end{align*}
and
$$P(theta in Theta_1mid x)=frac{int_{Theta_1} rho_1pi_1(theta_1)f(x|theta_1)text{d}theta_1}{int_{Theta_0} rho_0pi_0(theta_0)f(x|theta_0)text{d}theta_0+int_{Theta_1} rho_1pi_1(theta_1)f(x|theta_1)text{d}theta_1}$$Hence,
$$frac{P(theta in Theta_ 0|x)}{P(theta in Theta_1mid x)}=frac{rho_0int_{Theta_0} pi_0(theta_0)f(x|theta_0)text{d}theta_0}{rho_1int_{Theta_1} pi_1(theta_1)f(x|theta_1)text{d}theta_1}=frac{rho_0}{1-rho_0}B^pi_{01}(x)$$
This hopefully explains where the final expression comes from.
Alas, there is a mistake in the last ratio of integrals in the quoted text (presumably due to a cut & paste)
$$B_{01}^{pi}=frac{int_{Theta_{0}}f(x|theta_{0})pi_{0}(theta)dtheta}{int_{Theta_{1}}f(x|theta_{1})pi_{1}(theta)dtheta}$$
which should be (when identifying the integrands differently in the two integrals to signify that the integrals are over two different parameter spaces)
$$B_{01}^{pi}=frac{int_{Theta_{0}}f(x|theta_{0})pi_{0}(theta_0)dtheta_0}{int_{Theta_{1}}f(x|theta_{1})pi_{1}(theta_1)dtheta_1}$$or (with another choice of representation of the integrands)
$$B_{01}^{pi}=frac{int_{Theta_{0}}f(x|theta)pi_{0}(theta)dtheta}{int_{Theta_{1}}f(x|theta)pi_{1}(theta)dtheta}$$
The only case when the notations $theta_0$ and $theta_1$ are important is when both null and alternative hypotheses are point hypotheses, i.e., when $Theta_0={theta_0}$ and $Theta_1={theta_1}$, since both symbols then take specific values, like $theta_0=3$ and $theta_1=-2$. In this specific case, the posterior is concentrated on ${theta_0,theta_1}$ and the Bayes factor writes $$ B^pi_{01} (x) = {f(x|theta_0)over f(x|theta_1)}$$ since
$$P(theta=theta_0|x)=frac{overbrace{pi(theta_0)}^{rho_0}f(x|theta_0)}{pi(theta_0)f(x|theta_0)+underbrace{pi(theta_1)}_{rho_1}f(x|theta_1)}$$
Thank you for pointing out this error, to be added to the list of typos.
edited Nov 25 at 5:54
answered Nov 24 at 5:33
Xi'an
52.4k688340
52.4k688340
add a comment |
add a comment |
Thanks for contributing an answer to Cross Validated!
- Please be sure to answer the question. Provide details and share your research!
But avoid …
- Asking for help, clarification, or responding to other answers.
- Making statements based on opinion; back them up with references or personal experience.
Use MathJax to format equations. MathJax reference.
To learn more, see our tips on writing great answers.
Some of your past answers have not been well-received, and you're in danger of being blocked from answering.
Please pay close attention to the following guidance:
- Please be sure to answer the question. Provide details and share your research!
But avoid …
- Asking for help, clarification, or responding to other answers.
- Making statements based on opinion; back them up with references or personal experience.
To learn more, see our tips on writing great answers.
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
StackExchange.ready(
function () {
StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2fstats.stackexchange.com%2fquestions%2f378496%2fwhere-is-my-mistake-in-this-definition-of-bayes-factor%23new-answer', 'question_page');
}
);
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown