Understanding of processes and threads

One 、 The concept of process and thread （ A word summary ）

         process ： The program we run usually corresponds to one or more processes , Process is the basic unit of memory allocated by the operating system .

         Threads ： A process usually contains one or more threads , Threads are operating system allocations CPU The basic unit of .

         Single threaded program ---> There is only one execution clue in our program （ The main thread ）---> Create multiple threads （ There are multiple concurrent parts ）

        Technology exchange group ：830709780

notes ： A program has at least one process , A process has at least one thread （ The main thread ）, Processes have independent memory units during execution , Multiple threads share memory , Thus greatly improving the efficiency of the program .

Two 、 Difference between process and thread

        1. The way of operating system resource management is different , The process has a separate address space , After a process crashes, there will be a protection mode so that it will not affect other processes . Threads are not , Threads have their own stack and local variables , But there is no separate address space between threads , So a thread hanging may affect the whole process hanging .

        2. The concurrency of processes is not as high as that of threads .

        3. Each independent thread has an entry for the program to run 、 Exit of sequential execution sequence and program . But threads can't execute independently , Must exist in the application . Multiple thread execution control provided by the application .

        4. For applications , Multithreading means that multiple execution parts can be executed at the same time . But for the operating system, it does not regard multiple threads as multiple independent applications , To achieve process scheduling and management and resource allocation

notes ： Multithreading is easy to schedule , Effectively achieve concurrency . The overhead of memory is relatively small . Creating a thread is faster than creating a process .

3、 ... and 、 The concept of thread pools

         When writing multithreaded programs , It must be noted that the creation and release of threads have a large overhead , And if you create too many threads , Scheduling switching between threads also has overhead , So the more threads, the better , The best use is to create several threads , Then reuse them .

         Thread pool ： First use a container , Create several threads in advance , When using threads, borrow a thread from the thread pool , After use , Do not release threads , Instead, put the thread back in the pool , Let threads be reused .

notes ： Pooling technology is basically space for time .

Four 、 Code demonstration （ Call the third-party interface to download ( Multithreading and thread pooling ) picture ）

# coding=utf8
# @time:2022/5/19 17:02

import time
from concurrent.futures import ThreadPoolExecutor
import requests


def down_picture(picture_url):
    resp = requests.get(picture_url)
    if resp.status_code == 200:
        filename = picture_url[picture_url.rfind('/'):]
        with open(f'images/{filename}', 'wb') as f:
            f.write(resp.content)


def read_picture():
    futures = []
    with ThreadPoolExecutor(max_workers=10) as pool:	#  Thread pool , The biggest process 10
        # threads = []
        star = time.time()
        format = 'png'
        header = {
            'User-Agent': 'Mozilla/5.0 (compatible; U; ABrowse 0.6; Syllable) AppleWebKit/420+ (KHTML, like Gecko)'}
        data = {'key': '919928ff6d14cXXXXXXXXXXXXXX', 'url': 'https://www.hXXXXXXXXXXXXXX.com'}	#  Fill in the parameters according to the actual situation 
        for page in range(1, 6):
            resp = requests.get('http://api.xxxxxxx.com', params=data, headers=header)	#  Fill in the parameters according to the actual situation 
            picture_list = resp.json()['newslist']
            # print(picture_list)
            for picture in picture_list:
                picture_url = picture['picUrl']
                # print(picture_url)
                # down_picture(picture_url)
                # Thread pool 
                f = pool.submit(down_picture, picture_url)
    '''''''''           
                #  Common multithreading methods 
                thd = threading.Thread(target=down_picture,args=(picture_url,))
                threads.append(thd)
                thd.start()
        # print(threads)
    
        #  Wait for all the execution time of the child thread just started 
        for t in threads:
            t.join()
         '''''''''
    for f in futures:
        f.result()
    end = time.time()
    print(end - star)    # 4.877447128295898


if __name__ == '__main__':
    read_picture()

Download situation ：

  More secure sharing , Please pay attention to 【 Security info】 WeChat official account ！