Where is my mistake in this definition of Bayes Factor?





.everyoneloves__top-leaderboard:empty,.everyoneloves__mid-leaderboard:empty{ margin-bottom:0;
}






up vote
5
down vote

favorite
4












From "The Bayesian Choice" by Christian P. Robert.



The definition of the Bayes factor is given to be the ratio of the posterior probabilities of the null and the alternative hypothesis over the ratio of the prior probabilities of the null and alternative.



ie $$B_{01}^{pi}= frac{ frac{P(theta in Theta_{0}|x)}{P(theta in Theta_{1}|x)}}{frac{p(theta in Theta_{0})}{ p(theta in Theta_{1})}}$$



Which the author shortly simplifies to be



$$B_{01}^{pi}=frac{int_{Theta_{0}}f(x|theta_{0})pi_{0}(theta)dtheta}{int_{Theta_{1}}f(x|theta_{1})pi_{1}(theta)dtheta}$$



But when I try to write out all the terms and use that



$$pi(theta|x)=frac{f(x|theta) pi(theta)}{int_{Theta}f(x|theta) pi(theta)dtheta}$$



I get



$$B_{01}^{pi}=frac{f(x|theta_{0})int_{Theta_{1}}f(x|theta_{1})pi_{1}(theta)dtheta}{f(x|theta_{1})int_{Theta_{0}}f(x|theta_{0})pi_{0}(theta)dtheta}$$



instead.



Anyone have any idea of where I went wrong? Probably some simple mistake I made or error.










share|cite|improve this question






























    up vote
    5
    down vote

    favorite
    4












    From "The Bayesian Choice" by Christian P. Robert.



    The definition of the Bayes factor is given to be the ratio of the posterior probabilities of the null and the alternative hypothesis over the ratio of the prior probabilities of the null and alternative.



    ie $$B_{01}^{pi}= frac{ frac{P(theta in Theta_{0}|x)}{P(theta in Theta_{1}|x)}}{frac{p(theta in Theta_{0})}{ p(theta in Theta_{1})}}$$



    Which the author shortly simplifies to be



    $$B_{01}^{pi}=frac{int_{Theta_{0}}f(x|theta_{0})pi_{0}(theta)dtheta}{int_{Theta_{1}}f(x|theta_{1})pi_{1}(theta)dtheta}$$



    But when I try to write out all the terms and use that



    $$pi(theta|x)=frac{f(x|theta) pi(theta)}{int_{Theta}f(x|theta) pi(theta)dtheta}$$



    I get



    $$B_{01}^{pi}=frac{f(x|theta_{0})int_{Theta_{1}}f(x|theta_{1})pi_{1}(theta)dtheta}{f(x|theta_{1})int_{Theta_{0}}f(x|theta_{0})pi_{0}(theta)dtheta}$$



    instead.



    Anyone have any idea of where I went wrong? Probably some simple mistake I made or error.










    share|cite|improve this question


























      up vote
      5
      down vote

      favorite
      4









      up vote
      5
      down vote

      favorite
      4






      4





      From "The Bayesian Choice" by Christian P. Robert.



      The definition of the Bayes factor is given to be the ratio of the posterior probabilities of the null and the alternative hypothesis over the ratio of the prior probabilities of the null and alternative.



      ie $$B_{01}^{pi}= frac{ frac{P(theta in Theta_{0}|x)}{P(theta in Theta_{1}|x)}}{frac{p(theta in Theta_{0})}{ p(theta in Theta_{1})}}$$



      Which the author shortly simplifies to be



      $$B_{01}^{pi}=frac{int_{Theta_{0}}f(x|theta_{0})pi_{0}(theta)dtheta}{int_{Theta_{1}}f(x|theta_{1})pi_{1}(theta)dtheta}$$



      But when I try to write out all the terms and use that



      $$pi(theta|x)=frac{f(x|theta) pi(theta)}{int_{Theta}f(x|theta) pi(theta)dtheta}$$



      I get



      $$B_{01}^{pi}=frac{f(x|theta_{0})int_{Theta_{1}}f(x|theta_{1})pi_{1}(theta)dtheta}{f(x|theta_{1})int_{Theta_{0}}f(x|theta_{0})pi_{0}(theta)dtheta}$$



      instead.



      Anyone have any idea of where I went wrong? Probably some simple mistake I made or error.










      share|cite|improve this question















      From "The Bayesian Choice" by Christian P. Robert.



      The definition of the Bayes factor is given to be the ratio of the posterior probabilities of the null and the alternative hypothesis over the ratio of the prior probabilities of the null and alternative.



      ie $$B_{01}^{pi}= frac{ frac{P(theta in Theta_{0}|x)}{P(theta in Theta_{1}|x)}}{frac{p(theta in Theta_{0})}{ p(theta in Theta_{1})}}$$



      Which the author shortly simplifies to be



      $$B_{01}^{pi}=frac{int_{Theta_{0}}f(x|theta_{0})pi_{0}(theta)dtheta}{int_{Theta_{1}}f(x|theta_{1})pi_{1}(theta)dtheta}$$



      But when I try to write out all the terms and use that



      $$pi(theta|x)=frac{f(x|theta) pi(theta)}{int_{Theta}f(x|theta) pi(theta)dtheta}$$



      I get



      $$B_{01}^{pi}=frac{f(x|theta_{0})int_{Theta_{1}}f(x|theta_{1})pi_{1}(theta)dtheta}{f(x|theta_{1})int_{Theta_{0}}f(x|theta_{0})pi_{0}(theta)dtheta}$$



      instead.



      Anyone have any idea of where I went wrong? Probably some simple mistake I made or error.







      hypothesis-testing bayesian mathematical-statistics odds-ratio marginal






      share|cite|improve this question















      share|cite|improve this question













      share|cite|improve this question




      share|cite|improve this question








      edited Nov 24 at 5:48









      Xi'an

      52.4k688340




      52.4k688340










      asked Nov 24 at 1:13









      Quality

      1979




      1979






















          1 Answer
          1






          active

          oldest

          votes

















          up vote
          5
          down vote



          accepted










          My deepest apologies, there is a typo in the final expression! Indeed, here is the complete text from my book (p.231):




          Definition 5.5. The Bayes factor is the ratio of the posterior probabilities of the null and the alternative hypotheses over the
          ratio of the prior probabilities of the null and the alternative
          hypotheses, i.e., $$ B^pi_{01}(x) = {P(theta in Theta_ 0mid x)
          over P(theta in Theta_1mid x)} bigg/ {pi(theta in Theta_ 0)
          over pi(theta in Theta_ 1)}. $$
          This ratio evaluates the
          modification of the odds of $Theta_0$ against $Theta_1$ due to the
          obs and can naturally be compared to $1$, although an exact
          comparison scale can only be based upon a loss function. In the
          particular case where $Theta_0={theta_0}$ and
          $Theta_1={theta_1}$, the Bayes factor simplifies to the usual
          likelihood ratio $$ B^pi_{01} (x) = {f(x|theta_0)over
          f(x|theta_1)}. $$
          In general, the Bayes factor depends on prior
          information, but is still proposed as an ``objective'' Bayesian
          answer, since it partly eliminates the influence of the prior modeling
          and emphasizes the role of the observations. Actually, it can be
          perceived as a Bayesian likelihood ratio since, if $pi_0$ is the
          prior distribution under $H_0$ and $pi_1$ the prior distribution
          under $H_1$, $B^pi_{01}(x)$ can be written as begin{equation}
          B^pi_{01} (x) = {int_{Theta_0} f(x|theta_0)pi_0(theta)
          ,text{d}theta over int_{Theta_1} f(x|theta_1)pi_1(theta)
          ,text{d}theta}
          =frac{m_0(x)}{m_1(x)},, end{equation}
          thus replacing the likelihoods with the marginals under both hypotheses.




          The prior is thus defined as a mixture:
          $$pi(theta)=pi(theta in Theta_ 0)timespi_0(theta)timesmathbb{I}_{Theta_0}(theta)+pi(theta in Theta_ 1)timespi_1(theta)timesmathbb{I}_{Theta_1}(theta)$$where
          $$pi(theta in Theta_ 0)=rho_0qquadtext{and}qquadpi(theta in Theta_ 1)=1-rho_0stackrel{text{def}}{=}rho_1$$
          are the prior weights of both hypotheses and
          $$int_{Theta_0} pi_0(theta_0)text{d}theta_0=int_{Theta_1} pi_1(theta_1)text{d}theta_1=1$$Therefore
          begin{align*}P(theta in Theta_ 0|x)=int_{Theta_0} pi(theta_0|x)text{d}theta_0&=int_{Theta_0} pi(theta_0|x)text{d}theta_0\
          &= frac{int_{Theta_0} pi(theta_0)f(x|theta_0)text{d}theta_0}{int_{Theta_0} pi(theta_0)f(x|theta_0)text{d}theta_0+int_{Theta_1} pi(theta_1)f(x|theta_1)text{d}theta_1}\
          &=frac{int_{Theta_0} rho_0pi_0(theta_0)f(x|theta_0)text{d}theta_0}{int_{Theta_0} rho_0pi_0(theta_0)f(x|theta_0)text{d}theta_0+int_{Theta_1} rho_1pi_1(theta_1)f(x|theta_1)text{d}theta_1}
          end{align*}

          and
          $$P(theta in Theta_1mid x)=frac{int_{Theta_1} rho_1pi_1(theta_1)f(x|theta_1)text{d}theta_1}{int_{Theta_0} rho_0pi_0(theta_0)f(x|theta_0)text{d}theta_0+int_{Theta_1} rho_1pi_1(theta_1)f(x|theta_1)text{d}theta_1}$$Hence,
          $$frac{P(theta in Theta_ 0|x)}{P(theta in Theta_1mid x)}=frac{rho_0int_{Theta_0} pi_0(theta_0)f(x|theta_0)text{d}theta_0}{rho_1int_{Theta_1} pi_1(theta_1)f(x|theta_1)text{d}theta_1}=frac{rho_0}{1-rho_0}B^pi_{01}(x)$$
          This hopefully explains where the final expression comes from.



          Alas, there is a mistake in the last ratio of integrals in the quoted text (presumably due to a cut & paste)




          $$B_{01}^{pi}=frac{int_{Theta_{0}}f(x|theta_{0})pi_{0}(theta)dtheta}{int_{Theta_{1}}f(x|theta_{1})pi_{1}(theta)dtheta}$$




          which should be (when identifying the integrands differently in the two integrals to signify that the integrals are over two different parameter spaces)
          $$B_{01}^{pi}=frac{int_{Theta_{0}}f(x|theta_{0})pi_{0}(theta_0)dtheta_0}{int_{Theta_{1}}f(x|theta_{1})pi_{1}(theta_1)dtheta_1}$$or (with another choice of representation of the integrands)
          $$B_{01}^{pi}=frac{int_{Theta_{0}}f(x|theta)pi_{0}(theta)dtheta}{int_{Theta_{1}}f(x|theta)pi_{1}(theta)dtheta}$$
          The only case when the notations $theta_0$ and $theta_1$ are important is when both null and alternative hypotheses are point hypotheses, i.e., when $Theta_0={theta_0}$ and $Theta_1={theta_1}$, since both symbols then take specific values, like $theta_0=3$ and $theta_1=-2$. In this specific case, the posterior is concentrated on ${theta_0,theta_1}$ and the Bayes factor writes $$ B^pi_{01} (x) = {f(x|theta_0)over f(x|theta_1)}$$ since
          $$P(theta=theta_0|x)=frac{overbrace{pi(theta_0)}^{rho_0}f(x|theta_0)}{pi(theta_0)f(x|theta_0)+underbrace{pi(theta_1)}_{rho_1}f(x|theta_1)}$$



          Thank you for pointing out this error, to be added to the list of typos.






          share|cite|improve this answer























            Your Answer





            StackExchange.ifUsing("editor", function () {
            return StackExchange.using("mathjaxEditing", function () {
            StackExchange.MarkdownEditor.creationCallbacks.add(function (editor, postfix) {
            StackExchange.mathjaxEditing.prepareWmdForMathJax(editor, postfix, [["$", "$"], ["\\(","\\)"]]);
            });
            });
            }, "mathjax-editing");

            StackExchange.ready(function() {
            var channelOptions = {
            tags: "".split(" "),
            id: "65"
            };
            initTagRenderer("".split(" "), "".split(" "), channelOptions);

            StackExchange.using("externalEditor", function() {
            // Have to fire editor after snippets, if snippets enabled
            if (StackExchange.settings.snippets.snippetsEnabled) {
            StackExchange.using("snippets", function() {
            createEditor();
            });
            }
            else {
            createEditor();
            }
            });

            function createEditor() {
            StackExchange.prepareEditor({
            heartbeatType: 'answer',
            convertImagesToLinks: false,
            noModals: true,
            showLowRepImageUploadWarning: true,
            reputationToPostImages: null,
            bindNavPrevention: true,
            postfix: "",
            imageUploader: {
            brandingHtml: "Powered by u003ca class="icon-imgur-white" href="https://imgur.com/"u003eu003c/au003e",
            contentPolicyHtml: "User contributions licensed under u003ca href="https://creativecommons.org/licenses/by-sa/3.0/"u003ecc by-sa 3.0 with attribution requiredu003c/au003e u003ca href="https://stackoverflow.com/legal/content-policy"u003e(content policy)u003c/au003e",
            allowUrls: true
            },
            onDemand: true,
            discardSelector: ".discard-answer"
            ,immediatelyShowMarkdownHelp:true
            });


            }
            });














            draft saved

            draft discarded


















            StackExchange.ready(
            function () {
            StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2fstats.stackexchange.com%2fquestions%2f378496%2fwhere-is-my-mistake-in-this-definition-of-bayes-factor%23new-answer', 'question_page');
            }
            );

            Post as a guest















            Required, but never shown

























            1 Answer
            1






            active

            oldest

            votes








            1 Answer
            1






            active

            oldest

            votes









            active

            oldest

            votes






            active

            oldest

            votes








            up vote
            5
            down vote



            accepted










            My deepest apologies, there is a typo in the final expression! Indeed, here is the complete text from my book (p.231):




            Definition 5.5. The Bayes factor is the ratio of the posterior probabilities of the null and the alternative hypotheses over the
            ratio of the prior probabilities of the null and the alternative
            hypotheses, i.e., $$ B^pi_{01}(x) = {P(theta in Theta_ 0mid x)
            over P(theta in Theta_1mid x)} bigg/ {pi(theta in Theta_ 0)
            over pi(theta in Theta_ 1)}. $$
            This ratio evaluates the
            modification of the odds of $Theta_0$ against $Theta_1$ due to the
            obs and can naturally be compared to $1$, although an exact
            comparison scale can only be based upon a loss function. In the
            particular case where $Theta_0={theta_0}$ and
            $Theta_1={theta_1}$, the Bayes factor simplifies to the usual
            likelihood ratio $$ B^pi_{01} (x) = {f(x|theta_0)over
            f(x|theta_1)}. $$
            In general, the Bayes factor depends on prior
            information, but is still proposed as an ``objective'' Bayesian
            answer, since it partly eliminates the influence of the prior modeling
            and emphasizes the role of the observations. Actually, it can be
            perceived as a Bayesian likelihood ratio since, if $pi_0$ is the
            prior distribution under $H_0$ and $pi_1$ the prior distribution
            under $H_1$, $B^pi_{01}(x)$ can be written as begin{equation}
            B^pi_{01} (x) = {int_{Theta_0} f(x|theta_0)pi_0(theta)
            ,text{d}theta over int_{Theta_1} f(x|theta_1)pi_1(theta)
            ,text{d}theta}
            =frac{m_0(x)}{m_1(x)},, end{equation}
            thus replacing the likelihoods with the marginals under both hypotheses.




            The prior is thus defined as a mixture:
            $$pi(theta)=pi(theta in Theta_ 0)timespi_0(theta)timesmathbb{I}_{Theta_0}(theta)+pi(theta in Theta_ 1)timespi_1(theta)timesmathbb{I}_{Theta_1}(theta)$$where
            $$pi(theta in Theta_ 0)=rho_0qquadtext{and}qquadpi(theta in Theta_ 1)=1-rho_0stackrel{text{def}}{=}rho_1$$
            are the prior weights of both hypotheses and
            $$int_{Theta_0} pi_0(theta_0)text{d}theta_0=int_{Theta_1} pi_1(theta_1)text{d}theta_1=1$$Therefore
            begin{align*}P(theta in Theta_ 0|x)=int_{Theta_0} pi(theta_0|x)text{d}theta_0&=int_{Theta_0} pi(theta_0|x)text{d}theta_0\
            &= frac{int_{Theta_0} pi(theta_0)f(x|theta_0)text{d}theta_0}{int_{Theta_0} pi(theta_0)f(x|theta_0)text{d}theta_0+int_{Theta_1} pi(theta_1)f(x|theta_1)text{d}theta_1}\
            &=frac{int_{Theta_0} rho_0pi_0(theta_0)f(x|theta_0)text{d}theta_0}{int_{Theta_0} rho_0pi_0(theta_0)f(x|theta_0)text{d}theta_0+int_{Theta_1} rho_1pi_1(theta_1)f(x|theta_1)text{d}theta_1}
            end{align*}

            and
            $$P(theta in Theta_1mid x)=frac{int_{Theta_1} rho_1pi_1(theta_1)f(x|theta_1)text{d}theta_1}{int_{Theta_0} rho_0pi_0(theta_0)f(x|theta_0)text{d}theta_0+int_{Theta_1} rho_1pi_1(theta_1)f(x|theta_1)text{d}theta_1}$$Hence,
            $$frac{P(theta in Theta_ 0|x)}{P(theta in Theta_1mid x)}=frac{rho_0int_{Theta_0} pi_0(theta_0)f(x|theta_0)text{d}theta_0}{rho_1int_{Theta_1} pi_1(theta_1)f(x|theta_1)text{d}theta_1}=frac{rho_0}{1-rho_0}B^pi_{01}(x)$$
            This hopefully explains where the final expression comes from.



            Alas, there is a mistake in the last ratio of integrals in the quoted text (presumably due to a cut & paste)




            $$B_{01}^{pi}=frac{int_{Theta_{0}}f(x|theta_{0})pi_{0}(theta)dtheta}{int_{Theta_{1}}f(x|theta_{1})pi_{1}(theta)dtheta}$$




            which should be (when identifying the integrands differently in the two integrals to signify that the integrals are over two different parameter spaces)
            $$B_{01}^{pi}=frac{int_{Theta_{0}}f(x|theta_{0})pi_{0}(theta_0)dtheta_0}{int_{Theta_{1}}f(x|theta_{1})pi_{1}(theta_1)dtheta_1}$$or (with another choice of representation of the integrands)
            $$B_{01}^{pi}=frac{int_{Theta_{0}}f(x|theta)pi_{0}(theta)dtheta}{int_{Theta_{1}}f(x|theta)pi_{1}(theta)dtheta}$$
            The only case when the notations $theta_0$ and $theta_1$ are important is when both null and alternative hypotheses are point hypotheses, i.e., when $Theta_0={theta_0}$ and $Theta_1={theta_1}$, since both symbols then take specific values, like $theta_0=3$ and $theta_1=-2$. In this specific case, the posterior is concentrated on ${theta_0,theta_1}$ and the Bayes factor writes $$ B^pi_{01} (x) = {f(x|theta_0)over f(x|theta_1)}$$ since
            $$P(theta=theta_0|x)=frac{overbrace{pi(theta_0)}^{rho_0}f(x|theta_0)}{pi(theta_0)f(x|theta_0)+underbrace{pi(theta_1)}_{rho_1}f(x|theta_1)}$$



            Thank you for pointing out this error, to be added to the list of typos.






            share|cite|improve this answer



























              up vote
              5
              down vote



              accepted










              My deepest apologies, there is a typo in the final expression! Indeed, here is the complete text from my book (p.231):




              Definition 5.5. The Bayes factor is the ratio of the posterior probabilities of the null and the alternative hypotheses over the
              ratio of the prior probabilities of the null and the alternative
              hypotheses, i.e., $$ B^pi_{01}(x) = {P(theta in Theta_ 0mid x)
              over P(theta in Theta_1mid x)} bigg/ {pi(theta in Theta_ 0)
              over pi(theta in Theta_ 1)}. $$
              This ratio evaluates the
              modification of the odds of $Theta_0$ against $Theta_1$ due to the
              obs and can naturally be compared to $1$, although an exact
              comparison scale can only be based upon a loss function. In the
              particular case where $Theta_0={theta_0}$ and
              $Theta_1={theta_1}$, the Bayes factor simplifies to the usual
              likelihood ratio $$ B^pi_{01} (x) = {f(x|theta_0)over
              f(x|theta_1)}. $$
              In general, the Bayes factor depends on prior
              information, but is still proposed as an ``objective'' Bayesian
              answer, since it partly eliminates the influence of the prior modeling
              and emphasizes the role of the observations. Actually, it can be
              perceived as a Bayesian likelihood ratio since, if $pi_0$ is the
              prior distribution under $H_0$ and $pi_1$ the prior distribution
              under $H_1$, $B^pi_{01}(x)$ can be written as begin{equation}
              B^pi_{01} (x) = {int_{Theta_0} f(x|theta_0)pi_0(theta)
              ,text{d}theta over int_{Theta_1} f(x|theta_1)pi_1(theta)
              ,text{d}theta}
              =frac{m_0(x)}{m_1(x)},, end{equation}
              thus replacing the likelihoods with the marginals under both hypotheses.




              The prior is thus defined as a mixture:
              $$pi(theta)=pi(theta in Theta_ 0)timespi_0(theta)timesmathbb{I}_{Theta_0}(theta)+pi(theta in Theta_ 1)timespi_1(theta)timesmathbb{I}_{Theta_1}(theta)$$where
              $$pi(theta in Theta_ 0)=rho_0qquadtext{and}qquadpi(theta in Theta_ 1)=1-rho_0stackrel{text{def}}{=}rho_1$$
              are the prior weights of both hypotheses and
              $$int_{Theta_0} pi_0(theta_0)text{d}theta_0=int_{Theta_1} pi_1(theta_1)text{d}theta_1=1$$Therefore
              begin{align*}P(theta in Theta_ 0|x)=int_{Theta_0} pi(theta_0|x)text{d}theta_0&=int_{Theta_0} pi(theta_0|x)text{d}theta_0\
              &= frac{int_{Theta_0} pi(theta_0)f(x|theta_0)text{d}theta_0}{int_{Theta_0} pi(theta_0)f(x|theta_0)text{d}theta_0+int_{Theta_1} pi(theta_1)f(x|theta_1)text{d}theta_1}\
              &=frac{int_{Theta_0} rho_0pi_0(theta_0)f(x|theta_0)text{d}theta_0}{int_{Theta_0} rho_0pi_0(theta_0)f(x|theta_0)text{d}theta_0+int_{Theta_1} rho_1pi_1(theta_1)f(x|theta_1)text{d}theta_1}
              end{align*}

              and
              $$P(theta in Theta_1mid x)=frac{int_{Theta_1} rho_1pi_1(theta_1)f(x|theta_1)text{d}theta_1}{int_{Theta_0} rho_0pi_0(theta_0)f(x|theta_0)text{d}theta_0+int_{Theta_1} rho_1pi_1(theta_1)f(x|theta_1)text{d}theta_1}$$Hence,
              $$frac{P(theta in Theta_ 0|x)}{P(theta in Theta_1mid x)}=frac{rho_0int_{Theta_0} pi_0(theta_0)f(x|theta_0)text{d}theta_0}{rho_1int_{Theta_1} pi_1(theta_1)f(x|theta_1)text{d}theta_1}=frac{rho_0}{1-rho_0}B^pi_{01}(x)$$
              This hopefully explains where the final expression comes from.



              Alas, there is a mistake in the last ratio of integrals in the quoted text (presumably due to a cut & paste)




              $$B_{01}^{pi}=frac{int_{Theta_{0}}f(x|theta_{0})pi_{0}(theta)dtheta}{int_{Theta_{1}}f(x|theta_{1})pi_{1}(theta)dtheta}$$




              which should be (when identifying the integrands differently in the two integrals to signify that the integrals are over two different parameter spaces)
              $$B_{01}^{pi}=frac{int_{Theta_{0}}f(x|theta_{0})pi_{0}(theta_0)dtheta_0}{int_{Theta_{1}}f(x|theta_{1})pi_{1}(theta_1)dtheta_1}$$or (with another choice of representation of the integrands)
              $$B_{01}^{pi}=frac{int_{Theta_{0}}f(x|theta)pi_{0}(theta)dtheta}{int_{Theta_{1}}f(x|theta)pi_{1}(theta)dtheta}$$
              The only case when the notations $theta_0$ and $theta_1$ are important is when both null and alternative hypotheses are point hypotheses, i.e., when $Theta_0={theta_0}$ and $Theta_1={theta_1}$, since both symbols then take specific values, like $theta_0=3$ and $theta_1=-2$. In this specific case, the posterior is concentrated on ${theta_0,theta_1}$ and the Bayes factor writes $$ B^pi_{01} (x) = {f(x|theta_0)over f(x|theta_1)}$$ since
              $$P(theta=theta_0|x)=frac{overbrace{pi(theta_0)}^{rho_0}f(x|theta_0)}{pi(theta_0)f(x|theta_0)+underbrace{pi(theta_1)}_{rho_1}f(x|theta_1)}$$



              Thank you for pointing out this error, to be added to the list of typos.






              share|cite|improve this answer

























                up vote
                5
                down vote



                accepted







                up vote
                5
                down vote



                accepted






                My deepest apologies, there is a typo in the final expression! Indeed, here is the complete text from my book (p.231):




                Definition 5.5. The Bayes factor is the ratio of the posterior probabilities of the null and the alternative hypotheses over the
                ratio of the prior probabilities of the null and the alternative
                hypotheses, i.e., $$ B^pi_{01}(x) = {P(theta in Theta_ 0mid x)
                over P(theta in Theta_1mid x)} bigg/ {pi(theta in Theta_ 0)
                over pi(theta in Theta_ 1)}. $$
                This ratio evaluates the
                modification of the odds of $Theta_0$ against $Theta_1$ due to the
                obs and can naturally be compared to $1$, although an exact
                comparison scale can only be based upon a loss function. In the
                particular case where $Theta_0={theta_0}$ and
                $Theta_1={theta_1}$, the Bayes factor simplifies to the usual
                likelihood ratio $$ B^pi_{01} (x) = {f(x|theta_0)over
                f(x|theta_1)}. $$
                In general, the Bayes factor depends on prior
                information, but is still proposed as an ``objective'' Bayesian
                answer, since it partly eliminates the influence of the prior modeling
                and emphasizes the role of the observations. Actually, it can be
                perceived as a Bayesian likelihood ratio since, if $pi_0$ is the
                prior distribution under $H_0$ and $pi_1$ the prior distribution
                under $H_1$, $B^pi_{01}(x)$ can be written as begin{equation}
                B^pi_{01} (x) = {int_{Theta_0} f(x|theta_0)pi_0(theta)
                ,text{d}theta over int_{Theta_1} f(x|theta_1)pi_1(theta)
                ,text{d}theta}
                =frac{m_0(x)}{m_1(x)},, end{equation}
                thus replacing the likelihoods with the marginals under both hypotheses.




                The prior is thus defined as a mixture:
                $$pi(theta)=pi(theta in Theta_ 0)timespi_0(theta)timesmathbb{I}_{Theta_0}(theta)+pi(theta in Theta_ 1)timespi_1(theta)timesmathbb{I}_{Theta_1}(theta)$$where
                $$pi(theta in Theta_ 0)=rho_0qquadtext{and}qquadpi(theta in Theta_ 1)=1-rho_0stackrel{text{def}}{=}rho_1$$
                are the prior weights of both hypotheses and
                $$int_{Theta_0} pi_0(theta_0)text{d}theta_0=int_{Theta_1} pi_1(theta_1)text{d}theta_1=1$$Therefore
                begin{align*}P(theta in Theta_ 0|x)=int_{Theta_0} pi(theta_0|x)text{d}theta_0&=int_{Theta_0} pi(theta_0|x)text{d}theta_0\
                &= frac{int_{Theta_0} pi(theta_0)f(x|theta_0)text{d}theta_0}{int_{Theta_0} pi(theta_0)f(x|theta_0)text{d}theta_0+int_{Theta_1} pi(theta_1)f(x|theta_1)text{d}theta_1}\
                &=frac{int_{Theta_0} rho_0pi_0(theta_0)f(x|theta_0)text{d}theta_0}{int_{Theta_0} rho_0pi_0(theta_0)f(x|theta_0)text{d}theta_0+int_{Theta_1} rho_1pi_1(theta_1)f(x|theta_1)text{d}theta_1}
                end{align*}

                and
                $$P(theta in Theta_1mid x)=frac{int_{Theta_1} rho_1pi_1(theta_1)f(x|theta_1)text{d}theta_1}{int_{Theta_0} rho_0pi_0(theta_0)f(x|theta_0)text{d}theta_0+int_{Theta_1} rho_1pi_1(theta_1)f(x|theta_1)text{d}theta_1}$$Hence,
                $$frac{P(theta in Theta_ 0|x)}{P(theta in Theta_1mid x)}=frac{rho_0int_{Theta_0} pi_0(theta_0)f(x|theta_0)text{d}theta_0}{rho_1int_{Theta_1} pi_1(theta_1)f(x|theta_1)text{d}theta_1}=frac{rho_0}{1-rho_0}B^pi_{01}(x)$$
                This hopefully explains where the final expression comes from.



                Alas, there is a mistake in the last ratio of integrals in the quoted text (presumably due to a cut & paste)




                $$B_{01}^{pi}=frac{int_{Theta_{0}}f(x|theta_{0})pi_{0}(theta)dtheta}{int_{Theta_{1}}f(x|theta_{1})pi_{1}(theta)dtheta}$$




                which should be (when identifying the integrands differently in the two integrals to signify that the integrals are over two different parameter spaces)
                $$B_{01}^{pi}=frac{int_{Theta_{0}}f(x|theta_{0})pi_{0}(theta_0)dtheta_0}{int_{Theta_{1}}f(x|theta_{1})pi_{1}(theta_1)dtheta_1}$$or (with another choice of representation of the integrands)
                $$B_{01}^{pi}=frac{int_{Theta_{0}}f(x|theta)pi_{0}(theta)dtheta}{int_{Theta_{1}}f(x|theta)pi_{1}(theta)dtheta}$$
                The only case when the notations $theta_0$ and $theta_1$ are important is when both null and alternative hypotheses are point hypotheses, i.e., when $Theta_0={theta_0}$ and $Theta_1={theta_1}$, since both symbols then take specific values, like $theta_0=3$ and $theta_1=-2$. In this specific case, the posterior is concentrated on ${theta_0,theta_1}$ and the Bayes factor writes $$ B^pi_{01} (x) = {f(x|theta_0)over f(x|theta_1)}$$ since
                $$P(theta=theta_0|x)=frac{overbrace{pi(theta_0)}^{rho_0}f(x|theta_0)}{pi(theta_0)f(x|theta_0)+underbrace{pi(theta_1)}_{rho_1}f(x|theta_1)}$$



                Thank you for pointing out this error, to be added to the list of typos.






                share|cite|improve this answer














                My deepest apologies, there is a typo in the final expression! Indeed, here is the complete text from my book (p.231):




                Definition 5.5. The Bayes factor is the ratio of the posterior probabilities of the null and the alternative hypotheses over the
                ratio of the prior probabilities of the null and the alternative
                hypotheses, i.e., $$ B^pi_{01}(x) = {P(theta in Theta_ 0mid x)
                over P(theta in Theta_1mid x)} bigg/ {pi(theta in Theta_ 0)
                over pi(theta in Theta_ 1)}. $$
                This ratio evaluates the
                modification of the odds of $Theta_0$ against $Theta_1$ due to the
                obs and can naturally be compared to $1$, although an exact
                comparison scale can only be based upon a loss function. In the
                particular case where $Theta_0={theta_0}$ and
                $Theta_1={theta_1}$, the Bayes factor simplifies to the usual
                likelihood ratio $$ B^pi_{01} (x) = {f(x|theta_0)over
                f(x|theta_1)}. $$
                In general, the Bayes factor depends on prior
                information, but is still proposed as an ``objective'' Bayesian
                answer, since it partly eliminates the influence of the prior modeling
                and emphasizes the role of the observations. Actually, it can be
                perceived as a Bayesian likelihood ratio since, if $pi_0$ is the
                prior distribution under $H_0$ and $pi_1$ the prior distribution
                under $H_1$, $B^pi_{01}(x)$ can be written as begin{equation}
                B^pi_{01} (x) = {int_{Theta_0} f(x|theta_0)pi_0(theta)
                ,text{d}theta over int_{Theta_1} f(x|theta_1)pi_1(theta)
                ,text{d}theta}
                =frac{m_0(x)}{m_1(x)},, end{equation}
                thus replacing the likelihoods with the marginals under both hypotheses.




                The prior is thus defined as a mixture:
                $$pi(theta)=pi(theta in Theta_ 0)timespi_0(theta)timesmathbb{I}_{Theta_0}(theta)+pi(theta in Theta_ 1)timespi_1(theta)timesmathbb{I}_{Theta_1}(theta)$$where
                $$pi(theta in Theta_ 0)=rho_0qquadtext{and}qquadpi(theta in Theta_ 1)=1-rho_0stackrel{text{def}}{=}rho_1$$
                are the prior weights of both hypotheses and
                $$int_{Theta_0} pi_0(theta_0)text{d}theta_0=int_{Theta_1} pi_1(theta_1)text{d}theta_1=1$$Therefore
                begin{align*}P(theta in Theta_ 0|x)=int_{Theta_0} pi(theta_0|x)text{d}theta_0&=int_{Theta_0} pi(theta_0|x)text{d}theta_0\
                &= frac{int_{Theta_0} pi(theta_0)f(x|theta_0)text{d}theta_0}{int_{Theta_0} pi(theta_0)f(x|theta_0)text{d}theta_0+int_{Theta_1} pi(theta_1)f(x|theta_1)text{d}theta_1}\
                &=frac{int_{Theta_0} rho_0pi_0(theta_0)f(x|theta_0)text{d}theta_0}{int_{Theta_0} rho_0pi_0(theta_0)f(x|theta_0)text{d}theta_0+int_{Theta_1} rho_1pi_1(theta_1)f(x|theta_1)text{d}theta_1}
                end{align*}

                and
                $$P(theta in Theta_1mid x)=frac{int_{Theta_1} rho_1pi_1(theta_1)f(x|theta_1)text{d}theta_1}{int_{Theta_0} rho_0pi_0(theta_0)f(x|theta_0)text{d}theta_0+int_{Theta_1} rho_1pi_1(theta_1)f(x|theta_1)text{d}theta_1}$$Hence,
                $$frac{P(theta in Theta_ 0|x)}{P(theta in Theta_1mid x)}=frac{rho_0int_{Theta_0} pi_0(theta_0)f(x|theta_0)text{d}theta_0}{rho_1int_{Theta_1} pi_1(theta_1)f(x|theta_1)text{d}theta_1}=frac{rho_0}{1-rho_0}B^pi_{01}(x)$$
                This hopefully explains where the final expression comes from.



                Alas, there is a mistake in the last ratio of integrals in the quoted text (presumably due to a cut & paste)




                $$B_{01}^{pi}=frac{int_{Theta_{0}}f(x|theta_{0})pi_{0}(theta)dtheta}{int_{Theta_{1}}f(x|theta_{1})pi_{1}(theta)dtheta}$$




                which should be (when identifying the integrands differently in the two integrals to signify that the integrals are over two different parameter spaces)
                $$B_{01}^{pi}=frac{int_{Theta_{0}}f(x|theta_{0})pi_{0}(theta_0)dtheta_0}{int_{Theta_{1}}f(x|theta_{1})pi_{1}(theta_1)dtheta_1}$$or (with another choice of representation of the integrands)
                $$B_{01}^{pi}=frac{int_{Theta_{0}}f(x|theta)pi_{0}(theta)dtheta}{int_{Theta_{1}}f(x|theta)pi_{1}(theta)dtheta}$$
                The only case when the notations $theta_0$ and $theta_1$ are important is when both null and alternative hypotheses are point hypotheses, i.e., when $Theta_0={theta_0}$ and $Theta_1={theta_1}$, since both symbols then take specific values, like $theta_0=3$ and $theta_1=-2$. In this specific case, the posterior is concentrated on ${theta_0,theta_1}$ and the Bayes factor writes $$ B^pi_{01} (x) = {f(x|theta_0)over f(x|theta_1)}$$ since
                $$P(theta=theta_0|x)=frac{overbrace{pi(theta_0)}^{rho_0}f(x|theta_0)}{pi(theta_0)f(x|theta_0)+underbrace{pi(theta_1)}_{rho_1}f(x|theta_1)}$$



                Thank you for pointing out this error, to be added to the list of typos.







                share|cite|improve this answer














                share|cite|improve this answer



                share|cite|improve this answer








                edited Nov 25 at 5:54

























                answered Nov 24 at 5:33









                Xi'an

                52.4k688340




                52.4k688340






























                    draft saved

                    draft discarded




















































                    Thanks for contributing an answer to Cross Validated!


                    • Please be sure to answer the question. Provide details and share your research!

                    But avoid



                    • Asking for help, clarification, or responding to other answers.

                    • Making statements based on opinion; back them up with references or personal experience.


                    Use MathJax to format equations. MathJax reference.


                    To learn more, see our tips on writing great answers.





                    Some of your past answers have not been well-received, and you're in danger of being blocked from answering.


                    Please pay close attention to the following guidance:


                    • Please be sure to answer the question. Provide details and share your research!

                    But avoid



                    • Asking for help, clarification, or responding to other answers.

                    • Making statements based on opinion; back them up with references or personal experience.


                    To learn more, see our tips on writing great answers.




                    draft saved


                    draft discarded














                    StackExchange.ready(
                    function () {
                    StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2fstats.stackexchange.com%2fquestions%2f378496%2fwhere-is-my-mistake-in-this-definition-of-bayes-factor%23new-answer', 'question_page');
                    }
                    );

                    Post as a guest















                    Required, but never shown





















































                    Required, but never shown














                    Required, but never shown












                    Required, but never shown







                    Required, but never shown

































                    Required, but never shown














                    Required, but never shown












                    Required, but never shown







                    Required, but never shown







                    Popular posts from this blog

                    "Incorrect syntax near the keyword 'ON'. (on update cascade, on delete cascade,)

                    Alcedinidae

                    RAC Tourist Trophy