How to check the presence of a given numpy array in a larger-shape numpy array?











up vote
1
down vote

favorite












I guess the title of my question might not be very clear..



I have a small array, say a = ([[0,0,0],[0,0,1],[0,1,1]]). Then I have a bigger array of a higher dimension, say b = ([[[2,2,2],[2,0,1],[2,1,1]],[[0,0,0],[3,3,1],[3,1,1]],[...]]).



I'd like to check if one of the elements of a can be found in b. In this case, I'd find that the first element of a [0,0,0] is indeed in b, and then I'd like to retrieve the corresponding index in b.



I'd like to do that avoiding looping, since from the very little I understood from numpy arrays, they are not meant to be iterated over in a classic way. In other words, I need it to be very fast, because my actual arrays are quite big.



Any idea?
Thanks a lot!



Arnaud.










share|improve this question




























    up vote
    1
    down vote

    favorite












    I guess the title of my question might not be very clear..



    I have a small array, say a = ([[0,0,0],[0,0,1],[0,1,1]]). Then I have a bigger array of a higher dimension, say b = ([[[2,2,2],[2,0,1],[2,1,1]],[[0,0,0],[3,3,1],[3,1,1]],[...]]).



    I'd like to check if one of the elements of a can be found in b. In this case, I'd find that the first element of a [0,0,0] is indeed in b, and then I'd like to retrieve the corresponding index in b.



    I'd like to do that avoiding looping, since from the very little I understood from numpy arrays, they are not meant to be iterated over in a classic way. In other words, I need it to be very fast, because my actual arrays are quite big.



    Any idea?
    Thanks a lot!



    Arnaud.










    share|improve this question


























      up vote
      1
      down vote

      favorite









      up vote
      1
      down vote

      favorite











      I guess the title of my question might not be very clear..



      I have a small array, say a = ([[0,0,0],[0,0,1],[0,1,1]]). Then I have a bigger array of a higher dimension, say b = ([[[2,2,2],[2,0,1],[2,1,1]],[[0,0,0],[3,3,1],[3,1,1]],[...]]).



      I'd like to check if one of the elements of a can be found in b. In this case, I'd find that the first element of a [0,0,0] is indeed in b, and then I'd like to retrieve the corresponding index in b.



      I'd like to do that avoiding looping, since from the very little I understood from numpy arrays, they are not meant to be iterated over in a classic way. In other words, I need it to be very fast, because my actual arrays are quite big.



      Any idea?
      Thanks a lot!



      Arnaud.










      share|improve this question















      I guess the title of my question might not be very clear..



      I have a small array, say a = ([[0,0,0],[0,0,1],[0,1,1]]). Then I have a bigger array of a higher dimension, say b = ([[[2,2,2],[2,0,1],[2,1,1]],[[0,0,0],[3,3,1],[3,1,1]],[...]]).



      I'd like to check if one of the elements of a can be found in b. In this case, I'd find that the first element of a [0,0,0] is indeed in b, and then I'd like to retrieve the corresponding index in b.



      I'd like to do that avoiding looping, since from the very little I understood from numpy arrays, they are not meant to be iterated over in a classic way. In other words, I need it to be very fast, because my actual arrays are quite big.



      Any idea?
      Thanks a lot!



      Arnaud.







      numpy






      share|improve this question















      share|improve this question













      share|improve this question




      share|improve this question








      edited Nov 19 at 8:09









      Aqueous Carlos

      301213




      301213










      asked Nov 19 at 7:51









      Arnaud

      12813




      12813
























          1 Answer
          1






          active

          oldest

          votes

















          up vote
          1
          down vote













          I don't know of a direct way, but I here's a function that works around the problem:



          import numpy as np

          def find_indices(val, arr):
          # first take a mean at the lowest level of each array,
          # then compare these to eliminate the majority of entries
          mb = np.mean(arr, axis=2); ma = np.mean(val)
          Y = np.argwhere(mb==ma)
          indices =
          # Then run a quick loop on the remaining elements to
          # eliminate arrays that don't match the order
          for i in range(len(Y)):
          idx = (Y[i,0],Y[i,1])
          if np.array_equal(val, arr[idx]):
          indices.append(idx)

          return indices

          # Sample arrays
          a = np.array([[0,0,0],[0,0,1],[0,1,1]])
          b = np.array([ [[6,5,4],[0,0,1],[2,3,3]],
          [[2,5,4],[6,5,4],[0,0,0]],
          [[2,0,2],[3,5,4],[5,4,6]],
          [[6,5,4],[0,0,0],[2,5,3]] ])



          print(find_indices(a[0], b))
          # [(1, 2), (3, 1)]
          print(find_indices(a[1], b))
          # [(0, 1)]


          The idea is to use the mean of each array and compare this with the mean of the input. np.argwhere() is the key here. That way you remove most of the unwanted matches, but I did need to use a loop on the remainder to avoid the unsorted matches (this shouldn't be too memory-consuming). You'll probably want to customise it further, but I hope this helps.






          share|improve this answer





















          • Thanks for the suggestion! Indeed such kind of workaround is a help!
            – Arnaud
            Nov 20 at 15:20











          Your Answer






          StackExchange.ifUsing("editor", function () {
          StackExchange.using("externalEditor", function () {
          StackExchange.using("snippets", function () {
          StackExchange.snippets.init();
          });
          });
          }, "code-snippets");

          StackExchange.ready(function() {
          var channelOptions = {
          tags: "".split(" "),
          id: "1"
          };
          initTagRenderer("".split(" "), "".split(" "), channelOptions);

          StackExchange.using("externalEditor", function() {
          // Have to fire editor after snippets, if snippets enabled
          if (StackExchange.settings.snippets.snippetsEnabled) {
          StackExchange.using("snippets", function() {
          createEditor();
          });
          }
          else {
          createEditor();
          }
          });

          function createEditor() {
          StackExchange.prepareEditor({
          heartbeatType: 'answer',
          convertImagesToLinks: true,
          noModals: true,
          showLowRepImageUploadWarning: true,
          reputationToPostImages: 10,
          bindNavPrevention: true,
          postfix: "",
          imageUploader: {
          brandingHtml: "Powered by u003ca class="icon-imgur-white" href="https://imgur.com/"u003eu003c/au003e",
          contentPolicyHtml: "User contributions licensed under u003ca href="https://creativecommons.org/licenses/by-sa/3.0/"u003ecc by-sa 3.0 with attribution requiredu003c/au003e u003ca href="https://stackoverflow.com/legal/content-policy"u003e(content policy)u003c/au003e",
          allowUrls: true
          },
          onDemand: true,
          discardSelector: ".discard-answer"
          ,immediatelyShowMarkdownHelp:true
          });


          }
          });














          draft saved

          draft discarded


















          StackExchange.ready(
          function () {
          StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2fstackoverflow.com%2fquestions%2f53370378%2fhow-to-check-the-presence-of-a-given-numpy-array-in-a-larger-shape-numpy-array%23new-answer', 'question_page');
          }
          );

          Post as a guest















          Required, but never shown

























          1 Answer
          1






          active

          oldest

          votes








          1 Answer
          1






          active

          oldest

          votes









          active

          oldest

          votes






          active

          oldest

          votes








          up vote
          1
          down vote













          I don't know of a direct way, but I here's a function that works around the problem:



          import numpy as np

          def find_indices(val, arr):
          # first take a mean at the lowest level of each array,
          # then compare these to eliminate the majority of entries
          mb = np.mean(arr, axis=2); ma = np.mean(val)
          Y = np.argwhere(mb==ma)
          indices =
          # Then run a quick loop on the remaining elements to
          # eliminate arrays that don't match the order
          for i in range(len(Y)):
          idx = (Y[i,0],Y[i,1])
          if np.array_equal(val, arr[idx]):
          indices.append(idx)

          return indices

          # Sample arrays
          a = np.array([[0,0,0],[0,0,1],[0,1,1]])
          b = np.array([ [[6,5,4],[0,0,1],[2,3,3]],
          [[2,5,4],[6,5,4],[0,0,0]],
          [[2,0,2],[3,5,4],[5,4,6]],
          [[6,5,4],[0,0,0],[2,5,3]] ])



          print(find_indices(a[0], b))
          # [(1, 2), (3, 1)]
          print(find_indices(a[1], b))
          # [(0, 1)]


          The idea is to use the mean of each array and compare this with the mean of the input. np.argwhere() is the key here. That way you remove most of the unwanted matches, but I did need to use a loop on the remainder to avoid the unsorted matches (this shouldn't be too memory-consuming). You'll probably want to customise it further, but I hope this helps.






          share|improve this answer





















          • Thanks for the suggestion! Indeed such kind of workaround is a help!
            – Arnaud
            Nov 20 at 15:20















          up vote
          1
          down vote













          I don't know of a direct way, but I here's a function that works around the problem:



          import numpy as np

          def find_indices(val, arr):
          # first take a mean at the lowest level of each array,
          # then compare these to eliminate the majority of entries
          mb = np.mean(arr, axis=2); ma = np.mean(val)
          Y = np.argwhere(mb==ma)
          indices =
          # Then run a quick loop on the remaining elements to
          # eliminate arrays that don't match the order
          for i in range(len(Y)):
          idx = (Y[i,0],Y[i,1])
          if np.array_equal(val, arr[idx]):
          indices.append(idx)

          return indices

          # Sample arrays
          a = np.array([[0,0,0],[0,0,1],[0,1,1]])
          b = np.array([ [[6,5,4],[0,0,1],[2,3,3]],
          [[2,5,4],[6,5,4],[0,0,0]],
          [[2,0,2],[3,5,4],[5,4,6]],
          [[6,5,4],[0,0,0],[2,5,3]] ])



          print(find_indices(a[0], b))
          # [(1, 2), (3, 1)]
          print(find_indices(a[1], b))
          # [(0, 1)]


          The idea is to use the mean of each array and compare this with the mean of the input. np.argwhere() is the key here. That way you remove most of the unwanted matches, but I did need to use a loop on the remainder to avoid the unsorted matches (this shouldn't be too memory-consuming). You'll probably want to customise it further, but I hope this helps.






          share|improve this answer





















          • Thanks for the suggestion! Indeed such kind of workaround is a help!
            – Arnaud
            Nov 20 at 15:20













          up vote
          1
          down vote










          up vote
          1
          down vote









          I don't know of a direct way, but I here's a function that works around the problem:



          import numpy as np

          def find_indices(val, arr):
          # first take a mean at the lowest level of each array,
          # then compare these to eliminate the majority of entries
          mb = np.mean(arr, axis=2); ma = np.mean(val)
          Y = np.argwhere(mb==ma)
          indices =
          # Then run a quick loop on the remaining elements to
          # eliminate arrays that don't match the order
          for i in range(len(Y)):
          idx = (Y[i,0],Y[i,1])
          if np.array_equal(val, arr[idx]):
          indices.append(idx)

          return indices

          # Sample arrays
          a = np.array([[0,0,0],[0,0,1],[0,1,1]])
          b = np.array([ [[6,5,4],[0,0,1],[2,3,3]],
          [[2,5,4],[6,5,4],[0,0,0]],
          [[2,0,2],[3,5,4],[5,4,6]],
          [[6,5,4],[0,0,0],[2,5,3]] ])



          print(find_indices(a[0], b))
          # [(1, 2), (3, 1)]
          print(find_indices(a[1], b))
          # [(0, 1)]


          The idea is to use the mean of each array and compare this with the mean of the input. np.argwhere() is the key here. That way you remove most of the unwanted matches, but I did need to use a loop on the remainder to avoid the unsorted matches (this shouldn't be too memory-consuming). You'll probably want to customise it further, but I hope this helps.






          share|improve this answer












          I don't know of a direct way, but I here's a function that works around the problem:



          import numpy as np

          def find_indices(val, arr):
          # first take a mean at the lowest level of each array,
          # then compare these to eliminate the majority of entries
          mb = np.mean(arr, axis=2); ma = np.mean(val)
          Y = np.argwhere(mb==ma)
          indices =
          # Then run a quick loop on the remaining elements to
          # eliminate arrays that don't match the order
          for i in range(len(Y)):
          idx = (Y[i,0],Y[i,1])
          if np.array_equal(val, arr[idx]):
          indices.append(idx)

          return indices

          # Sample arrays
          a = np.array([[0,0,0],[0,0,1],[0,1,1]])
          b = np.array([ [[6,5,4],[0,0,1],[2,3,3]],
          [[2,5,4],[6,5,4],[0,0,0]],
          [[2,0,2],[3,5,4],[5,4,6]],
          [[6,5,4],[0,0,0],[2,5,3]] ])



          print(find_indices(a[0], b))
          # [(1, 2), (3, 1)]
          print(find_indices(a[1], b))
          # [(0, 1)]


          The idea is to use the mean of each array and compare this with the mean of the input. np.argwhere() is the key here. That way you remove most of the unwanted matches, but I did need to use a loop on the remainder to avoid the unsorted matches (this shouldn't be too memory-consuming). You'll probably want to customise it further, but I hope this helps.







          share|improve this answer












          share|improve this answer



          share|improve this answer










          answered Nov 19 at 18:37









          John Kealy

          362




          362












          • Thanks for the suggestion! Indeed such kind of workaround is a help!
            – Arnaud
            Nov 20 at 15:20


















          • Thanks for the suggestion! Indeed such kind of workaround is a help!
            – Arnaud
            Nov 20 at 15:20
















          Thanks for the suggestion! Indeed such kind of workaround is a help!
          – Arnaud
          Nov 20 at 15:20




          Thanks for the suggestion! Indeed such kind of workaround is a help!
          – Arnaud
          Nov 20 at 15:20


















          draft saved

          draft discarded




















































          Thanks for contributing an answer to Stack Overflow!


          • Please be sure to answer the question. Provide details and share your research!

          But avoid



          • Asking for help, clarification, or responding to other answers.

          • Making statements based on opinion; back them up with references or personal experience.


          To learn more, see our tips on writing great answers.





          Some of your past answers have not been well-received, and you're in danger of being blocked from answering.


          Please pay close attention to the following guidance:


          • Please be sure to answer the question. Provide details and share your research!

          But avoid



          • Asking for help, clarification, or responding to other answers.

          • Making statements based on opinion; back them up with references or personal experience.


          To learn more, see our tips on writing great answers.




          draft saved


          draft discarded














          StackExchange.ready(
          function () {
          StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2fstackoverflow.com%2fquestions%2f53370378%2fhow-to-check-the-presence-of-a-given-numpy-array-in-a-larger-shape-numpy-array%23new-answer', 'question_page');
          }
          );

          Post as a guest















          Required, but never shown





















































          Required, but never shown














          Required, but never shown












          Required, but never shown







          Required, but never shown

































          Required, but never shown














          Required, but never shown












          Required, but never shown







          Required, but never shown







          Popular posts from this blog

          "Incorrect syntax near the keyword 'ON'. (on update cascade, on delete cascade,)

          Alcedinidae

          Origin of the phrase “under your belt”?