Multithreading and concurrent programming (2)

Catalog

Multithreading and concurrent programming （ Two ）

Process creation -Proccess Subclass

The process of pool

Pool Example method table

Multithreading and concurrent programming （ Two ）

Process creation -Proccess Subclass

  You can also create processes by using classes , You can customize - One class , Inherit Process class , Every time this class is instantiated , Is equivalent to instantiating a process object .

[ Example ] Inherit Process Class , rewrite run0 Method to create a process  

#  The import module 
from multiprocessing import Process
from time import sleep
import time
#  Defining classes 
class ClockProcess(Process):
    #  Reinitialize the method 
    def __init__(self,interval):
        Process.__init__(self)
        self.interval=interval

    def run(self):
        #  Create child process 
        print(" The time when the subprocess starts executing ：{}".format(time.ctime()))
        sleep(self.interval)
        print(" Sub process execution end time ：{}".format(time.ctime()))

if __name__ =="__main__":
    p=ClockProcess(4)
    p.start()
    p.join()
    print(" End of main process ")

The process of pool

Using Python When it comes to system management , In particular, operating multiple file directories at the same time , Or remote control of multiple hosts , Parallel operation can save a lot of time . When the number of objects to be manipulated is small , Can be used directly multiprocessing Medium Process Dynamic generation of multiple processes , A dozen are fine , But if it's a hundred , Thousands of goals , Manually limiting the number of processes is too cumbersome , At this time, it can play the role of process pool . 

Pool Can provide a specified number of processes , For users to call , When a new request is submitted to pool In the middle of the day , If the pool is not full , Then it will create - A new process Used to execute the request ; But if the number of processes in the pool has reached the specified maximum , Then the request will wait , Until there are processes in the pool , To create a new process .Pool The syntax of is as follows : 

Pool ([numprocess [ initializer [, initrg]])

among numprocess Is the number of processes to create . If you omit this parameter , Will use cpu _count() Value .Initializer Is the callable object to be executed when each worker process starts .Initargs It's to pass on to initializer Parameter Yuanzu .Initializer The default is None. 

Pool Example method table

Method	describe
apply (fune [, args, [, kwargs]])	Execute functions in a pool worker process (*arss, **kwares), And then return the result .
apply_async(func,[,args[,kwargs[,callback]]])	stay Functions are executed asynchronously in a pool worker process (*arss, **kwarss), And then return the result . The result of this method is AsyncResult Class , It can be used later to get the final result .Callback Is a callable object , Accept input parameters . When func When the result of becomes available , Pass on to immediately callback. Callback Do not perform any blocking operations , Otherwise, it will block receiving the results of other asynchronous operations .
close()	Close process pool , Prevent from entering - - Step by step . If there are still pending operations , They will be completed before the work process is terminated
join()	Wait for all worker processes to exit . This method can only be used in close () perhaps terminate () Call after method
imap( func, iterable [,chunksize] )	map () One of the versions of the function , Return iterators instead of result lists
imap unordered( fune. ,iterable [ ,chunksize] )	Same as imap() The function is the same , Only the order of results is arbitrarily determined according to the time received from the work process
map( func, iterable [,chunksize] )	The callable object fune Apply to iterable. All items in , Then return the results in the form of a list . By way of iterable Divide into blocks and assign work to work processes , This operation can be performed in parallel .chunksize. Specify the number of items in each block . If the quantity is large , Can increase chunksize. To improve performance
map_ asyne( fune, iterable[, chunksize[,callback]] )	Same as map () function , But the result is returned asynchronously . The return value is AsyncResult Class , Available later and get results .Callback An adjustable object that accepts a parameter . Provided callable, When the results become available , Will call... With the result callable.
terminate()	That is, terminate all working processes , At the same time, do not perform any cleanup or end any pending work . If p Be recycled , This function will be called automatically .
get( [ timeout] )	Return results , Wait for results to arrive if necessary .Timeout Is an optional timeout . If the result does not arrive within the specified time , Will lead to multiprocessing. TimeoutError abnormal . If an exception is thrown in a remote operation , It will be raised again when this method is called
ready()	If the call is complete , Then return to True
sucessful()	If the call completes without throwing an exception , return True. Call this method before the result is ready. , Will lead to AssertionError abnormal
wait( [timeout] )	Wait for results to become available .Timeout Is an optional timeout

Be careful : 

apply_ async(func[, args[, kwds[, callback]]) It's non blocking ,apply(func[, args[, kwds]) It's blocked  

[ Example ] Use of process pools ( Non blocking ) 

#  The import module 
import multiprocessing
import time
#  The task function executed by the process 
def func(msg):
    print('start',msg)
    time.sleep(3)
    print('end',msg)

if __name__=='__main__':
    #  Create initialization 3 The process pool of 
    pool=multiprocessing.Pool(3)
    #  Add tasks 
    for i in range(1,6):
        msg=' Mission %d'%i
        pool.apply_async(func,(msg,))
    #  If the process no longer receives new requests   call close
    pool.close()
    #  Wait for the subprocess to finish 
    pool.join()