Python - how to read txt file every xxx time and add to multiprocess if new text added












0















So basically I am trying to do something that I am not sure if it is possible to do nor not.



My idea is that I am using a multiprocessing which means it runs its separated process on the background for each "task" done. Similar to:



names.txt



Oskar Baldwin

Khalil Whittle

Jevon Burn

Paddy Wilkinson

Jocelyn Weiss

Ishaq Glenn

Zahraa Macfarlane

Marianna Roy

Humera Schultz

Luther Pugh





import json, time, sys, os, timeit, random, multiprocessing.dummy, re

import threading





def main(names):

    print(names)

    time.sleep(5)





if __name__ == '__main__':



    try:

        jobs = 

        for names in [line.rstrip('n') for line in open('names.txt')]:

            p = multiprocessing.dummy.Process(target=main, args=(names,))

            jobs.append(p)

            p.start()



    except KeyboardInterrupt:

        print('Keyboard - Interrupted)

        sys.exit()


So the output of this will be that each process will take care of each of names for themselves so process 1 will use the first line in txt file and the 2nd process will use the second name and so on.



My question is how can I move on and be able to check the names.txt xxx time and check if there is a new name added then add it to a new process and if there is no new names then just continue the process and sleep again until another xxx time while the other process is running in the background?






python multithreading process multiprocessing







share|improve this question



























    0















    So basically I am trying to do something that I am not sure if it is possible to do nor not.



    My idea is that I am using a multiprocessing which means it runs its separated process on the background for each "task" done. Similar to:



    names.txt
    

    
Oskar Baldwin
    
Khalil Whittle
    
Jevon Burn
    
Paddy Wilkinson
    
Jocelyn Weiss
    
Ishaq Glenn
    
Zahraa Macfarlane
    
Marianna Roy
    
Humera Schultz
    
Luther Pugh
    




    import json, time, sys, os, timeit, random, multiprocessing.dummy, re
    
import threading
    

    

    
def main(names):
    
    print(names)
    
    time.sleep(5)
    

    

    
if __name__ == '__main__':
    

    
    try:
    
        jobs = 
    
        for names in [line.rstrip('n') for line in open('names.txt')]:
    
            p = multiprocessing.dummy.Process(target=main, args=(names,))
    
            jobs.append(p)
    
            p.start()
    

    
    except KeyboardInterrupt:
    
        print('Keyboard - Interrupted)
    
        sys.exit()


    So the output of this will be that each process will take care of each of names for themselves so process 1 will use the first line in txt file and the 2nd process will use the second name and so on.



    My question is how can I move on and be able to check the names.txt xxx time and check if there is a new name added then add it to a new process and if there is no new names then just continue the process and sleep again until another xxx time while the other process is running in the background?






    python multithreading process multiprocessing







    share|improve this question

























      0












      0








      0


      1






      So basically I am trying to do something that I am not sure if it is possible to do nor not.



      My idea is that I am using a multiprocessing which means it runs its separated process on the background for each "task" done. Similar to:



      names.txt
      

      
Oskar Baldwin
      
Khalil Whittle
      
Jevon Burn
      
Paddy Wilkinson
      
Jocelyn Weiss
      
Ishaq Glenn
      
Zahraa Macfarlane
      
Marianna Roy
      
Humera Schultz
      
Luther Pugh
      




      import json, time, sys, os, timeit, random, multiprocessing.dummy, re
      
import threading
      

      

      
def main(names):
      
    print(names)
      
    time.sleep(5)
      

      

      
if __name__ == '__main__':
      

      
    try:
      
        jobs = 
      
        for names in [line.rstrip('n') for line in open('names.txt')]:
      
            p = multiprocessing.dummy.Process(target=main, args=(names,))
      
            jobs.append(p)
      
            p.start()
      

      
    except KeyboardInterrupt:
      
        print('Keyboard - Interrupted)
      
        sys.exit()


      So the output of this will be that each process will take care of each of names for themselves so process 1 will use the first line in txt file and the 2nd process will use the second name and so on.



      My question is how can I move on and be able to check the names.txt xxx time and check if there is a new name added then add it to a new process and if there is no new names then just continue the process and sleep again until another xxx time while the other process is running in the background?






      python multithreading process multiprocessing







      share|improve this question














      So basically I am trying to do something that I am not sure if it is possible to do nor not.



      My idea is that I am using a multiprocessing which means it runs its separated process on the background for each "task" done. Similar to:



      names.txt
      

      
Oskar Baldwin
      
Khalil Whittle
      
Jevon Burn
      
Paddy Wilkinson
      
Jocelyn Weiss
      
Ishaq Glenn
      
Zahraa Macfarlane
      
Marianna Roy
      
Humera Schultz
      
Luther Pugh
      




      import json, time, sys, os, timeit, random, multiprocessing.dummy, re
      
import threading
      

      

      
def main(names):
      
    print(names)
      
    time.sleep(5)
      

      

      
if __name__ == '__main__':
      

      
    try:
      
        jobs = 
      
        for names in [line.rstrip('n') for line in open('names.txt')]:
      
            p = multiprocessing.dummy.Process(target=main, args=(names,))
      
            jobs.append(p)
      
            p.start()
      

      
    except KeyboardInterrupt:
      
        print('Keyboard - Interrupted)
      
        sys.exit()


      So the output of this will be that each process will take care of each of names for themselves so process 1 will use the first line in txt file and the 2nd process will use the second name and so on.



      My question is how can I move on and be able to check the names.txt xxx time and check if there is a new name added then add it to a new process and if there is no new names then just continue the process and sleep again until another xxx time while the other process is running in the background?






      python multithreading process multiprocessing




      python multithreading process multiprocessing






      share|improve this question













      share|improve this question











      share|improve this question




      share|improve this question










      asked Nov 20 '18 at 11:24









      HellosiroverthereHellosiroverthere

      10818




      10818
























          1 Answer
          1






          active

          oldest

          votes


















          1














          Quick and dirty way of doing it. I didn't research whether there's a way to query an item in the jobs list for its "args" attribute.



          Beware that it does not correctly handle the case where a duplicate name is added after that process has finished executing. You'll have to come up with a way to remove finished processes from the used_names list.



          jobs = 
          
used_names = 
          
for i in range(0, 10): # repeat as often as necessary
          
    try:
          
        for names in [line.rstrip('n') for line in open('names.txt')]:
          
            if names not in used_names: # dont spawn new process if one already exists
          
                p = multiprocessing.dummy.Process(target=main, args=(names,))
          
                jobs.append(p)
          
                used_names.append(p)
          
                p.start()
          
        time.sleep(0.1) # arbitrary sleep
          

          
    except KeyboardInterrupt:
          
        print('Keyboard - Interrupted')
          
        sys.exit(0)





          share|improve this answer























            Your Answer






            StackExchange.ifUsing("editor", function () {
            StackExchange.using("externalEditor", function () {
            StackExchange.using("snippets", function () {
            StackExchange.snippets.init();
            });
            });
            }, "code-snippets");

            StackExchange.ready(function() {
            var channelOptions = {
            tags: "".split(" "),
            id: "1"
            };
            initTagRenderer("".split(" "), "".split(" "), channelOptions);

            StackExchange.using("externalEditor", function() {
            // Have to fire editor after snippets, if snippets enabled
            if (StackExchange.settings.snippets.snippetsEnabled) {
            StackExchange.using("snippets", function() {
            createEditor();
            });
            }
            else {
            createEditor();
            }
            });

            function createEditor() {
            StackExchange.prepareEditor({
            heartbeatType: 'answer',
            autoActivateHeartbeat: false,
            convertImagesToLinks: true,
            noModals: true,
            showLowRepImageUploadWarning: true,
            reputationToPostImages: 10,
            bindNavPrevention: true,
            postfix: "",
            imageUploader: {
            brandingHtml: "Powered by u003ca class="icon-imgur-white" href="https://imgur.com/"u003eu003c/au003e",
            contentPolicyHtml: "User contributions licensed under u003ca href="https://creativecommons.org/licenses/by-sa/3.0/"u003ecc by-sa 3.0 with attribution requiredu003c/au003e u003ca href="https://stackoverflow.com/legal/content-policy"u003e(content policy)u003c/au003e",
            allowUrls: true
            },
            onDemand: true,
            discardSelector: ".discard-answer"
            ,immediatelyShowMarkdownHelp:true
            });


            }
            });














            draft saved

            draft discarded


















            StackExchange.ready(
            function () {
            StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2fstackoverflow.com%2fquestions%2f53391973%2fpython-how-to-read-txt-file-every-xxx-time-and-add-to-multiprocess-if-new-text%23new-answer', 'question_page');
            }
            );

            Post as a guest















            Required, but never shown

























            1 Answer
            1






            active

            oldest

            votes








            1 Answer
            1






            active

            oldest

            votes









            active

            oldest

            votes






            active

            oldest

            votes









            1














            Quick and dirty way of doing it. I didn't research whether there's a way to query an item in the jobs list for its "args" attribute.



            Beware that it does not correctly handle the case where a duplicate name is added after that process has finished executing. You'll have to come up with a way to remove finished processes from the used_names list.



            jobs = 
            
used_names = 
            
for i in range(0, 10): # repeat as often as necessary
            
    try:
            
        for names in [line.rstrip('n') for line in open('names.txt')]:
            
            if names not in used_names: # dont spawn new process if one already exists
            
                p = multiprocessing.dummy.Process(target=main, args=(names,))
            
                jobs.append(p)
            
                used_names.append(p)
            
                p.start()
            
        time.sleep(0.1) # arbitrary sleep
            

            
    except KeyboardInterrupt:
            
        print('Keyboard - Interrupted')
            
        sys.exit(0)





            share|improve this answer




























              1














              Quick and dirty way of doing it. I didn't research whether there's a way to query an item in the jobs list for its "args" attribute.



              Beware that it does not correctly handle the case where a duplicate name is added after that process has finished executing. You'll have to come up with a way to remove finished processes from the used_names list.



              jobs = 
              
used_names = 
              
for i in range(0, 10): # repeat as often as necessary
              
    try:
              
        for names in [line.rstrip('n') for line in open('names.txt')]:
              
            if names not in used_names: # dont spawn new process if one already exists
              
                p = multiprocessing.dummy.Process(target=main, args=(names,))
              
                jobs.append(p)
              
                used_names.append(p)
              
                p.start()
              
        time.sleep(0.1) # arbitrary sleep
              

              
    except KeyboardInterrupt:
              
        print('Keyboard - Interrupted')
              
        sys.exit(0)





              share|improve this answer


























                1












                1








                1







                Quick and dirty way of doing it. I didn't research whether there's a way to query an item in the jobs list for its "args" attribute.



                Beware that it does not correctly handle the case where a duplicate name is added after that process has finished executing. You'll have to come up with a way to remove finished processes from the used_names list.



                jobs = 
                
used_names = 
                
for i in range(0, 10): # repeat as often as necessary
                
    try:
                
        for names in [line.rstrip('n') for line in open('names.txt')]:
                
            if names not in used_names: # dont spawn new process if one already exists
                
                p = multiprocessing.dummy.Process(target=main, args=(names,))
                
                jobs.append(p)
                
                used_names.append(p)
                
                p.start()
                
        time.sleep(0.1) # arbitrary sleep
                

                
    except KeyboardInterrupt:
                
        print('Keyboard - Interrupted')
                
        sys.exit(0)





                share|improve this answer













                Quick and dirty way of doing it. I didn't research whether there's a way to query an item in the jobs list for its "args" attribute.



                Beware that it does not correctly handle the case where a duplicate name is added after that process has finished executing. You'll have to come up with a way to remove finished processes from the used_names list.



                jobs = 
                
used_names = 
                
for i in range(0, 10): # repeat as often as necessary
                
    try:
                
        for names in [line.rstrip('n') for line in open('names.txt')]:
                
            if names not in used_names: # dont spawn new process if one already exists
                
                p = multiprocessing.dummy.Process(target=main, args=(names,))
                
                jobs.append(p)
                
                used_names.append(p)
                
                p.start()
                
        time.sleep(0.1) # arbitrary sleep
                

                
    except KeyboardInterrupt:
                
        print('Keyboard - Interrupted')
                
        sys.exit(0)






                share|improve this answer












                share|improve this answer



                share|improve this answer










                answered Nov 20 '18 at 22:43









                JavonJavon

                513




                513






























                    draft saved

                    draft discarded




















































                    Thanks for contributing an answer to Stack Overflow!


                    • Please be sure to answer the question. Provide details and share your research!

                    But avoid …



                    • Asking for help, clarification, or responding to other answers.

                    • Making statements based on opinion; back them up with references or personal experience.


                    To learn more, see our tips on writing great answers.




                    draft saved


                    draft discarded














                    StackExchange.ready(
                    function () {
                    StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2fstackoverflow.com%2fquestions%2f53391973%2fpython-how-to-read-txt-file-every-xxx-time-and-add-to-multiprocess-if-new-text%23new-answer', 'question_page');
                    }
                    );

                    Post as a guest















                    Required, but never shown





















































                    Required, but never shown














                    Required, but never shown












                    Required, but never shown







                    Required, but never shown

































                    Required, but never shown














                    Required, but never shown












                    Required, but never shown







                    Required, but never shown







                    -

                    Popular posts from this blog

                    If I really need a card on my start hand, how many mulligans make sense? [duplicate]

                    Image
                    5 This question already has an answer here: How do you calculate the likelihood of drawing certain cards in your opening hand? 3 answers There is a funny modern deck where you play Treasure Hunt, but you really need to have it on the starting hand. How many mulligans should I take, if my deck contains 60 cards and contains 4 copies of Treasure Hunt? magic-the-gathering probability share | improve this question asked Nov 23 '18 at 12:12 dan dan 152 4
                    Read more

                    Alcedinidae

                    Image
                    Alcedinidae Martin-pêcheur à dos bleu ( Alcedo azurea ) Classification (COI) Règne Animalia Embranchement Chordata Sous-embr. Vertebrata Classe Aves Ordre Coraciiformes Famille Alcedinidae Rafinesque, 1815 Les Alcédinidés ou Alcedinidae forment une famille d'oiseaux plus connus sous les noms de martins-pêcheurs et martins-chasseurs. Sommaire 1 Description 2 Répartition et habitat 3 Systématique 3.1 Liste alphabétique des genres 3.2 Liste des espèces 4 Chez les Anciens : l'alcyon 4.1 Étymologie (Littré) 4.2 Oiseau « fabuleux » ? 5 Notes 6 Articles connexes 7 Liens externes 7.1 Bibliographie Description | Ce sont des oiseaux compacts de taille petite à moyenne (10 à 46 cm), au bec droit et long, en forme de poignard. Leurs pattes sont courtes, et ils portent un plumage aux couleurs vives. Répartition et habitat | Cosmopolites, ils fréquentent surtout
                    Read more

                    Can an atomic nucleus contain both particles and antiparticles? [duplicate]

                    Image
                    9 $begingroup$ This question already has an answer here: What happens if we put together a proton and an antineutron? 2 answers Is it theoretically possible to make a "deuterium" atom containing a proton and an antineutron in its nucleus? Would the strong nuclear force cause attraction between a proton and an antineutron? Would such a nucleus be stable, or would the proton somehow annihilate the antineutron when close enough? nuclear-physics antimatter share | cite | improve this question asked yesterday
                    Read more