47. Process lock & process pool & Collaboration

1. Deadlock

 A thread took b lock , A process took it a lock , Each other needs to get the other's lock and get stuck .

from threading import Thread, Lock
import time

def task1(lock_a, lock_b, i):
    lock_a.acquire()
    print('%s Grab lock_a Lock the ！' % i)
    lock_b.acquire()
    print('%s Grab lock_b Lock the ！' % i)

    lock_a.release()
    print('%s Release lock_a Lock the ！' % i)

    lock_b.release()
    print('%s Release lock_b Lock the ！' % i)


def task2(lock_a, lock_b, i):

    lock_b.acquire()
    print('%s Grab lock_b Lock the ！' % i)
    lock_a.acquire()
    print('%s Grab lock_a Lock the ！' % i)

    time.sleep(2)

    lock_b.release()
    print('%s Release lock_a Lock the ！' % i)
    lock_a.release()
    print('%s Release lock_b Lock the ！' % i)


def task0(lock_a, lock_b, i):
    time.sleep(0.2)
    task1(lock_a, lock_b, i)
    task2(lock_a, lock_b, i)


if __name__ == '__main__':
    lock_a = Lock()
    lock_b = Lock()
    for i in range(10):
        t = Thread(target=task0, args=(lock_a, lock_b, i))
        t.start()

2 Grab lock_a Lock the ！
2 Grab lock_b Lock the ！
2 Release lock_a Lock the ！
1 Grab lock_a Lock the ！
2 Release lock_b Lock the ！
2 Grab lock_b Lock the ！

2. Recursive lock

 The characteristic of recursive lock is that the first one who grabs the lock can be continuous acpuire() and release().
 There is a calculator inside , Every time acquire Add one calculator at a time ,
 Every time release One counter minus one , As long as you don't 0, No one else can grab the lock .

lock_a = lock_b  = RLock() 
 Two locks point to the same memory address , encounter .acquire, RLock Add 1.

from threading import Thread, RLock
import time

def task1(lock_a, lock_b, i):
    lock_a.acquire()
    print('%s Grab lock_a Lock the ！' % i)
    lock_b.acquire()
    print('%s Grab lock_b Lock the ！' % i)

    lock_a.release()
    print('%s Release lock_a Lock the ！' % i)

    lock_b.release()
    print('%s Release lock_b Lock the ！' % i)


def task2(lock_a, lock_b, i):

    lock_b.acquire()
    print('%s Grab lock_b Lock the ！' % i)
    lock_a.acquire()
    print('%s Grab lock_a Lock the ！' % i)

    time.sleep(2)

    lock_b.release()
    print('%s Release lock_a Lock the ！' % i)
    lock_a.release()
    print('%s Release lock_b Lock the ！' % i)


def task0(lock_a, lock_b, i):
    time.sleep(0.2)
    task1(lock_a, lock_b, i)
    task2(lock_a, lock_b, i)


if __name__ == '__main__':
    lock_a = lock_b  = RLock()

    for i in range(10):
        t = Thread(target=task0, args=(lock_a, lock_b, i))
        t.start()

3. Process pool thread pool

 There are usually many tasks that need to be executed concurrently in program design , Often greater than the number of cores ,
 An operating system cannot open processes indefinitely , Too many processes , Efficiency will decline .
 Starting the process needs to preempt system resources , Start the process with extra cores , It can't be done in parallel .

 The process of pool ： Set a process number , Control the number of processes that can run simultaneously ,
 If there is a new request to start a process and the process pool is full , Then the request can only wait for the end of other processes in the pool , Do it again. .

from concurrent.futures import ProcessPoolExecutor, ThreadPoolExecutor
import time

def task(i):
    print(' I am a %s Number thread ' % i)
    time.sleep(3)

if __name__ == '__main__':
    p_pool = ProcessPoolExecutor(3)

    for i in range(10):
        p_pool.submit(task, i)

    p_pool.shutdown()   # join The function of 
    print(' I'm the main process ')

from concurrent.futures import ProcessPoolExecutor, ThreadPoolExecutor
import time

def task(i):
    print(' I am a %s Number thread ' % i)
    time.sleep(3)
    return i

def info(res):
    print(res.result()) # result() Print out the content 

if __name__ == '__main__':
    p_pool = ProcessPoolExecutor(3)

    for i in range(10):
        p_pool.submit(task, i).add_done_callback(info)  # dd_done_callback Give the return value to info

    print(' I'm the main process ')

4. coroutines

 coroutines ： It's concurrency under single thread , Also called tasklet , fibers . English name Coroutine.
 In a word, what is thread ： Coroutine is a lightweight thread in user mode , That is, the scheduling is controlled by the user program itself .

 It's important to note that ：
#1. python The threads of are kernel level , That is, scheduling is controlled by the operating system 
（ If a single thread encounters io Or if the execution time is too long, it will be forced to hand over cpu Executive authority , Switch other threads to run ）

#2.  Start the process in a single thread , Once encountered io, From the application level （
 Not the operating system ） Control switch , To improve efficiency （！！！ Not io The switching of operation has nothing to do with efficiency ）

 Compare the switching of operating system control threads , The user controls the switch of the process in a single thread 

 Advantages as follows ：
#1.  The switching cost of the cooperation process is less , Switching at the program level , The operating system is completely unaware of , So it's more lightweight 
#2.  Single thread can achieve the effect of concurrency , Make the most of cpu

 Drawbacks as follows ：
#1.  The essence of the process is single thread , Can't use multi-core , Can be a program to open multiple processes , Open multiple threads in each process , Start the process in each thread 
#2.  A process is a single thread , So once the process is blocked , Will block the entire thread 

 Summarize the characteristics of the process ：

1.   Concurrency must be implemented in only one single thread 
2.   No lock is needed to modify shared data 
3.   The user program stores multiple control flow context stacks 
4.   additional ： A process meets IO The operation automatically switches to other processes （
 How to achieve detection IO,yield、greenlet Can't achieve , It uses gevent modular （select Mechanism ））

5.Greenlet modular

 install  ：pip3 install greenlet

from greenlet import greenlet

def eat(name):
    print('%s eat 1' %name)
    g2.switch('egon')
    print('%s eat 2' %name)
    g2.switch()
def play(name):
    print('%s play 1' %name)
    g1.switch()
    print('%s play 2' %name)

g1=greenlet(eat)
g2=greenlet(play)

g1.switch('egon')# For the first time switch The parameter is passed in , No need for 

 Simple switching （ In the absence of io In case of or without repeated operation of opening memory space ）, On the contrary, it will reduce the execution speed of the program 

# Sequential execution 
import time
def f1():
    res=1
    for i in range(100000000):
        res+=i

def f2():
    res=1
    for i in range(100000000):
        res*=i

start=time.time()
f1()
f2()
stop=time.time()
print('run time is %s' %(stop-start)) #10.985628366470337

# Switch 
from greenlet import greenlet
import time
def f1():
    res=1
    for i in range(100000000):
        res+=i
        g2.switch()

def f2():
    res=1
    for i in range(100000000):
        res*=i
        g1.switch()

start=time.time()
g1=greenlet(f1)
g2=greenlet(f2)
g1.switch()
stop=time.time()
print('run time is %s' %(stop-start))  # 52.763017892837524

greenlet It just provides a comparison generator More convenient switching mode , When cutting to a task execution, if io,
 Then block in place , It's still not solved IO The problem of automatic switching to improve efficiency .

 This one in a single thread 20 The code of tasks usually has both calculation and blocking operations , We could be on a mission 1 When there's a jam ,
 Just use the blocked time to perform the task 2.... such , To improve efficiency , And that's where it comes in Gevent modular .