Sampling from Gaussian mixture models, when are the sampled data independent?












2












$begingroup$


Suppose I generate a Gaussian mixture model with $N$ Gaussian distributions



$p(x) = sumlimits_{i = 1}^N w_i mathcal{N}(x;mu_i, Sigma_i)$



where $w_i$ are the weights.



Now I sample some points ${x_n}$ from $p(x)$



What condition is needed to ensure that the points are independent from each other?



Is it sufficient to assume that each component $mathcal{N}(x;mu_i, Sigma_i)$ have diagonal covariance matrix $Sigma_i$?






normal-distribution mixed-model random-variable independence gaussian-mixture







share|cite|improve this question









$endgroup$

















    2












    $begingroup$


    Suppose I generate a Gaussian mixture model with $N$ Gaussian distributions



    $p(x) = sumlimits_{i = 1}^N w_i mathcal{N}(x;mu_i, Sigma_i)$



    where $w_i$ are the weights.



    Now I sample some points ${x_n}$ from $p(x)$



    What condition is needed to ensure that the points are independent from each other?



    Is it sufficient to assume that each component $mathcal{N}(x;mu_i, Sigma_i)$ have diagonal covariance matrix $Sigma_i$?






    normal-distribution mixed-model random-variable independence gaussian-mixture







    share|cite|improve this question









    $endgroup$















      2












      2








      2





      $begingroup$


      Suppose I generate a Gaussian mixture model with $N$ Gaussian distributions



      $p(x) = sumlimits_{i = 1}^N w_i mathcal{N}(x;mu_i, Sigma_i)$



      where $w_i$ are the weights.



      Now I sample some points ${x_n}$ from $p(x)$



      What condition is needed to ensure that the points are independent from each other?



      Is it sufficient to assume that each component $mathcal{N}(x;mu_i, Sigma_i)$ have diagonal covariance matrix $Sigma_i$?






      normal-distribution mixed-model random-variable independence gaussian-mixture







      share|cite|improve this question









      $endgroup$




      Suppose I generate a Gaussian mixture model with $N$ Gaussian distributions



      $p(x) = sumlimits_{i = 1}^N w_i mathcal{N}(x;mu_i, Sigma_i)$



      where $w_i$ are the weights.



      Now I sample some points ${x_n}$ from $p(x)$



      What condition is needed to ensure that the points are independent from each other?



      Is it sufficient to assume that each component $mathcal{N}(x;mu_i, Sigma_i)$ have diagonal covariance matrix $Sigma_i$?






      normal-distribution mixed-model random-variable independence gaussian-mixture




      normal-distribution mixed-model random-variable independence gaussian-mixture






      share|cite|improve this question













      share|cite|improve this question











      share|cite|improve this question




      share|cite|improve this question










      asked 7 hours ago









      Shamisen ExpertShamisen Expert

      126112




      126112






















          1 Answer
          1






          active

          oldest

          votes


















          5












          $begingroup$

          You are confusing independence between the simulated vectors and independence between the components of a single vector. Unless otherwise stated, two simulations $X_1,X_2$ from$$sumlimits_{i = 1}^N w_i mathcal{N}(x;mu_i, Sigma_i)tag{1}$$will be independent, while the components of the vector $X_1$ will most likely be dependent, even when the matrices $Sigma_i$ are diagonal. (The intuition about that last point is that if $X_{11}$ is closer to $mu_{1k}$ than to the other Normal means, then the other components of $X_1$ are also more likely to originate from this same $k$-th element of the mixture.) For instance a most standard way to simulate from (1) is to simulate
          $$Z_1simmathcal M(1;w_1,ldots,w_N)qquad X_1|Z_1=k sim mathcal{N}(x;mu_k, Sigma_k)$$and$$Z_2simmathcal M(1;w_1,ldots,w_N)qquad X_2|Z_2=k sim mathcal{N}(x;mu_k, Sigma_k)$$If the four variables are mutually independent then $X_1$ and $X_2$ are independent.






          share|cite|improve this answer











          $endgroup$













          • $begingroup$
            Thanks for your answer. What is the meaning of $mathcal{M}$? Can you justify intuitively why the $X_1, X_2$ will most likely to be independent (this is clearer to me), whereas the components of $X_1$ will be dependent (this is not as clear to me)?
            $endgroup$
            – Shamisen Expert
            7 hours ago










          • $begingroup$
            Xi'an is using $mathcal{M}$ to denote the multinomial distribution. With the first parameter $n$ set to 1, the multinomial is also known as the categorical distribution. A categorical random variable takes on one of a finite set of categorical values with probability determined by the second parameter, the $w$ vector. In other words, $Z_i$ represents the group membership of observation $i$.
            $endgroup$
            – mb7744
            59 mins ago













          Your Answer





          StackExchange.ifUsing("editor", function () {
          return StackExchange.using("mathjaxEditing", function () {
          StackExchange.MarkdownEditor.creationCallbacks.add(function (editor, postfix) {
          StackExchange.mathjaxEditing.prepareWmdForMathJax(editor, postfix, [["$", "$"], ["\\(","\\)"]]);
          });
          });
          }, "mathjax-editing");

          StackExchange.ready(function() {
          var channelOptions = {
          tags: "".split(" "),
          id: "65"
          };
          initTagRenderer("".split(" "), "".split(" "), channelOptions);

          StackExchange.using("externalEditor", function() {
          // Have to fire editor after snippets, if snippets enabled
          if (StackExchange.settings.snippets.snippetsEnabled) {
          StackExchange.using("snippets", function() {
          createEditor();
          });
          }
          else {
          createEditor();
          }
          });

          function createEditor() {
          StackExchange.prepareEditor({
          heartbeatType: 'answer',
          autoActivateHeartbeat: false,
          convertImagesToLinks: false,
          noModals: true,
          showLowRepImageUploadWarning: true,
          reputationToPostImages: null,
          bindNavPrevention: true,
          postfix: "",
          imageUploader: {
          brandingHtml: "Powered by u003ca class="icon-imgur-white" href="https://imgur.com/"u003eu003c/au003e",
          contentPolicyHtml: "User contributions licensed under u003ca href="https://creativecommons.org/licenses/by-sa/3.0/"u003ecc by-sa 3.0 with attribution requiredu003c/au003e u003ca href="https://stackoverflow.com/legal/content-policy"u003e(content policy)u003c/au003e",
          allowUrls: true
          },
          onDemand: true,
          discardSelector: ".discard-answer"
          ,immediatelyShowMarkdownHelp:true
          });


          }
          });














          draft saved

          draft discarded


















          StackExchange.ready(
          function () {
          StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2fstats.stackexchange.com%2fquestions%2f396582%2fsampling-from-gaussian-mixture-models-when-are-the-sampled-data-independent%23new-answer', 'question_page');
          }
          );

          Post as a guest















          Required, but never shown

























          1 Answer
          1






          active

          oldest

          votes








          1 Answer
          1






          active

          oldest

          votes









          active

          oldest

          votes






          active

          oldest

          votes









          5












          $begingroup$

          You are confusing independence between the simulated vectors and independence between the components of a single vector. Unless otherwise stated, two simulations $X_1,X_2$ from$$sumlimits_{i = 1}^N w_i mathcal{N}(x;mu_i, Sigma_i)tag{1}$$will be independent, while the components of the vector $X_1$ will most likely be dependent, even when the matrices $Sigma_i$ are diagonal. (The intuition about that last point is that if $X_{11}$ is closer to $mu_{1k}$ than to the other Normal means, then the other components of $X_1$ are also more likely to originate from this same $k$-th element of the mixture.) For instance a most standard way to simulate from (1) is to simulate
          $$Z_1simmathcal M(1;w_1,ldots,w_N)qquad X_1|Z_1=k sim mathcal{N}(x;mu_k, Sigma_k)$$and$$Z_2simmathcal M(1;w_1,ldots,w_N)qquad X_2|Z_2=k sim mathcal{N}(x;mu_k, Sigma_k)$$If the four variables are mutually independent then $X_1$ and $X_2$ are independent.






          share|cite|improve this answer











          $endgroup$













          • $begingroup$
            Thanks for your answer. What is the meaning of $mathcal{M}$? Can you justify intuitively why the $X_1, X_2$ will most likely to be independent (this is clearer to me), whereas the components of $X_1$ will be dependent (this is not as clear to me)?
            $endgroup$
            – Shamisen Expert
            7 hours ago










          • $begingroup$
            Xi'an is using $mathcal{M}$ to denote the multinomial distribution. With the first parameter $n$ set to 1, the multinomial is also known as the categorical distribution. A categorical random variable takes on one of a finite set of categorical values with probability determined by the second parameter, the $w$ vector. In other words, $Z_i$ represents the group membership of observation $i$.
            $endgroup$
            – mb7744
            59 mins ago


















          5












          $begingroup$

          You are confusing independence between the simulated vectors and independence between the components of a single vector. Unless otherwise stated, two simulations $X_1,X_2$ from$$sumlimits_{i = 1}^N w_i mathcal{N}(x;mu_i, Sigma_i)tag{1}$$will be independent, while the components of the vector $X_1$ will most likely be dependent, even when the matrices $Sigma_i$ are diagonal. (The intuition about that last point is that if $X_{11}$ is closer to $mu_{1k}$ than to the other Normal means, then the other components of $X_1$ are also more likely to originate from this same $k$-th element of the mixture.) For instance a most standard way to simulate from (1) is to simulate
          $$Z_1simmathcal M(1;w_1,ldots,w_N)qquad X_1|Z_1=k sim mathcal{N}(x;mu_k, Sigma_k)$$and$$Z_2simmathcal M(1;w_1,ldots,w_N)qquad X_2|Z_2=k sim mathcal{N}(x;mu_k, Sigma_k)$$If the four variables are mutually independent then $X_1$ and $X_2$ are independent.






          share|cite|improve this answer











          $endgroup$













          • $begingroup$
            Thanks for your answer. What is the meaning of $mathcal{M}$? Can you justify intuitively why the $X_1, X_2$ will most likely to be independent (this is clearer to me), whereas the components of $X_1$ will be dependent (this is not as clear to me)?
            $endgroup$
            – Shamisen Expert
            7 hours ago










          • $begingroup$
            Xi'an is using $mathcal{M}$ to denote the multinomial distribution. With the first parameter $n$ set to 1, the multinomial is also known as the categorical distribution. A categorical random variable takes on one of a finite set of categorical values with probability determined by the second parameter, the $w$ vector. In other words, $Z_i$ represents the group membership of observation $i$.
            $endgroup$
            – mb7744
            59 mins ago
















          5












          5








          5





          $begingroup$

          You are confusing independence between the simulated vectors and independence between the components of a single vector. Unless otherwise stated, two simulations $X_1,X_2$ from$$sumlimits_{i = 1}^N w_i mathcal{N}(x;mu_i, Sigma_i)tag{1}$$will be independent, while the components of the vector $X_1$ will most likely be dependent, even when the matrices $Sigma_i$ are diagonal. (The intuition about that last point is that if $X_{11}$ is closer to $mu_{1k}$ than to the other Normal means, then the other components of $X_1$ are also more likely to originate from this same $k$-th element of the mixture.) For instance a most standard way to simulate from (1) is to simulate
          $$Z_1simmathcal M(1;w_1,ldots,w_N)qquad X_1|Z_1=k sim mathcal{N}(x;mu_k, Sigma_k)$$and$$Z_2simmathcal M(1;w_1,ldots,w_N)qquad X_2|Z_2=k sim mathcal{N}(x;mu_k, Sigma_k)$$If the four variables are mutually independent then $X_1$ and $X_2$ are independent.






          share|cite|improve this answer











          $endgroup$



          You are confusing independence between the simulated vectors and independence between the components of a single vector. Unless otherwise stated, two simulations $X_1,X_2$ from$$sumlimits_{i = 1}^N w_i mathcal{N}(x;mu_i, Sigma_i)tag{1}$$will be independent, while the components of the vector $X_1$ will most likely be dependent, even when the matrices $Sigma_i$ are diagonal. (The intuition about that last point is that if $X_{11}$ is closer to $mu_{1k}$ than to the other Normal means, then the other components of $X_1$ are also more likely to originate from this same $k$-th element of the mixture.) For instance a most standard way to simulate from (1) is to simulate
          $$Z_1simmathcal M(1;w_1,ldots,w_N)qquad X_1|Z_1=k sim mathcal{N}(x;mu_k, Sigma_k)$$and$$Z_2simmathcal M(1;w_1,ldots,w_N)qquad X_2|Z_2=k sim mathcal{N}(x;mu_k, Sigma_k)$$If the four variables are mutually independent then $X_1$ and $X_2$ are independent.







          share|cite|improve this answer














          share|cite|improve this answer



          share|cite|improve this answer








          edited 6 hours ago

























          answered 7 hours ago









          Xi'anXi'an

          57.9k897360




          57.9k897360












          • $begingroup$
            Thanks for your answer. What is the meaning of $mathcal{M}$? Can you justify intuitively why the $X_1, X_2$ will most likely to be independent (this is clearer to me), whereas the components of $X_1$ will be dependent (this is not as clear to me)?
            $endgroup$
            – Shamisen Expert
            7 hours ago










          • $begingroup$
            Xi'an is using $mathcal{M}$ to denote the multinomial distribution. With the first parameter $n$ set to 1, the multinomial is also known as the categorical distribution. A categorical random variable takes on one of a finite set of categorical values with probability determined by the second parameter, the $w$ vector. In other words, $Z_i$ represents the group membership of observation $i$.
            $endgroup$
            – mb7744
            59 mins ago




















          • $begingroup$
            Thanks for your answer. What is the meaning of $mathcal{M}$? Can you justify intuitively why the $X_1, X_2$ will most likely to be independent (this is clearer to me), whereas the components of $X_1$ will be dependent (this is not as clear to me)?
            $endgroup$
            – Shamisen Expert
            7 hours ago










          • $begingroup$
            Xi'an is using $mathcal{M}$ to denote the multinomial distribution. With the first parameter $n$ set to 1, the multinomial is also known as the categorical distribution. A categorical random variable takes on one of a finite set of categorical values with probability determined by the second parameter, the $w$ vector. In other words, $Z_i$ represents the group membership of observation $i$.
            $endgroup$
            – mb7744
            59 mins ago


















          $begingroup$
          Thanks for your answer. What is the meaning of $mathcal{M}$? Can you justify intuitively why the $X_1, X_2$ will most likely to be independent (this is clearer to me), whereas the components of $X_1$ will be dependent (this is not as clear to me)?
          $endgroup$
          – Shamisen Expert
          7 hours ago




          $begingroup$
          Thanks for your answer. What is the meaning of $mathcal{M}$? Can you justify intuitively why the $X_1, X_2$ will most likely to be independent (this is clearer to me), whereas the components of $X_1$ will be dependent (this is not as clear to me)?
          $endgroup$
          – Shamisen Expert
          7 hours ago












          $begingroup$
          Xi'an is using $mathcal{M}$ to denote the multinomial distribution. With the first parameter $n$ set to 1, the multinomial is also known as the categorical distribution. A categorical random variable takes on one of a finite set of categorical values with probability determined by the second parameter, the $w$ vector. In other words, $Z_i$ represents the group membership of observation $i$.
          $endgroup$
          – mb7744
          59 mins ago






          $begingroup$
          Xi'an is using $mathcal{M}$ to denote the multinomial distribution. With the first parameter $n$ set to 1, the multinomial is also known as the categorical distribution. A categorical random variable takes on one of a finite set of categorical values with probability determined by the second parameter, the $w$ vector. In other words, $Z_i$ represents the group membership of observation $i$.
          $endgroup$
          – mb7744
          59 mins ago




















          draft saved

          draft discarded




















































          Thanks for contributing an answer to Cross Validated!


          • Please be sure to answer the question. Provide details and share your research!

          But avoid …



          • Asking for help, clarification, or responding to other answers.

          • Making statements based on opinion; back them up with references or personal experience.


          Use MathJax to format equations. MathJax reference.


          To learn more, see our tips on writing great answers.




          draft saved


          draft discarded














          StackExchange.ready(
          function () {
          StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2fstats.stackexchange.com%2fquestions%2f396582%2fsampling-from-gaussian-mixture-models-when-are-the-sampled-data-independent%23new-answer', 'question_page');
          }
          );

          Post as a guest















          Required, but never shown





















































          Required, but never shown














          Required, but never shown












          Required, but never shown







          Required, but never shown

































          Required, but never shown














          Required, but never shown












          Required, but never shown







          Required, but never shown







          -

          Popular posts from this blog

          "Incorrect syntax near the keyword 'ON'. (on update cascade, on delete cascade,)

          Image
          .everyoneloves__top-leaderboard:empty,.everyoneloves__mid-leaderboard:empty,.everyoneloves__bot-mid-leaderboard:empty{ height:90px;width:728px;box-sizing:border-box; } 2 total noob sorry any help greatly appreciated. I keep getting this "Incorrect syntax near the keyword 'ON'." heres my codes code one CREATE TABLE supplier ( supplierID INT PRIMARY KEY, ISBN INT FOREIGN KEY REFERENCES book(ISBN), price DECIMAL (5,2), ON UPDATE CASCADE, ON DELETE CASCADE, ); code two CREATE TABLE orders ( orderID INT PRIMARY KEY, customerID INT FOREIGN KEY REFERENCES customer(customerID), ISBN INT FOREIGN KEY REFERENCES book(ISBN), orderDate date, ON UPDATE CASCADE, ON DELETE CASCADE, ); Ive run out of stuff to try other than dumbing it down to uselessness. ...
          Read more

          Alcedinidae

          Image
          Alcedinidae Martin-pêcheur à dos bleu ( Alcedo azurea ) Classification (COI) Règne Animalia Embranchement Chordata Sous-embr. Vertebrata Classe Aves Ordre Coraciiformes Famille Alcedinidae Rafinesque, 1815 Les Alcédinidés ou Alcedinidae forment une famille d'oiseaux plus connus sous les noms de martins-pêcheurs et martins-chasseurs. Sommaire 1 Description 2 Répartition et habitat 3 Systématique 3.1 Liste alphabétique des genres 3.2 Liste des espèces 4 Chez les Anciens : l'alcyon 4.1 Étymologie (Littré) 4.2 Oiseau « fabuleux » ? 5 Notes 6 Articles connexes 7 Liens externes 7.1 Bibliographie Description | Ce sont des oiseaux compacts de taille petite à moyenne (10 à 46 cm), au bec droit et long, en forme de poignard. Leurs pattes sont courtes, et ils portent un plumage aux couleurs vives. Répartition et habitat | Cosmopolites, ils fréquentent surtout ...
          Read more

          Origin of the phrase “under your belt”?

          Image
          .everyoneloves__top-leaderboard:empty,.everyoneloves__mid-leaderboard:empty,.everyoneloves__bot-mid-leaderboard:empty{ margin-bottom:0; } 17 1 Today, I found myself discussing what students should have "under their belts" during a lecture, and I wondered to myself if there was some inappropriate undertone here that I might be unaware of. Question : What is the origin of the phrase "under your belt"? A google search revealed some webpages (e.g. [1]) that assert its origin is related to consumption of food -- once a meal has been eaten, it's under your belt (which is a relief). However, with such websites, I have no way to determine fact from "random guy on the internet making stuff up". etymology idioms ...
          Read more