Five network IO models

Preface

   This article focuses on introducing five kinds of networks IO Model

   The Internet IO, Two system objects will be involved , One is user space call IO Process or thread of , The other is the kernel system in kernel space , For example, when IO operation read when , It goes through two stages .

1. Wait for the data to be ready
2. Copy data from the kernel to processes or threads

   Because there are different situations in the above two stages , So there are many kinds of networks IO Model .

   The knowledge points of this column are through Zero sound education Online learning , Make a summary and write an article , Yes c/c++linux Readers interested in the course , You can click on the link C/C++ Background advanced server course introduction Check the service of the course in detail .

Blocking IO（blocking IO）

   stay linux in , By default, all socket All are blocking, A typical read operation flow

   When the user process calls read This system call ,kernel He began IO The first stage of ： Prepare the data . about network io Come on , Many times the data hasn't arrived at the beginning （ such as , I haven't received a complete packet yet ） This is the time kernel Just wait for enough data to come . On the user process side , The whole process will be blocked . When kernel Wait until the data is ready , It will take data from kernel Copy to user memory , then kernel Return results , The user process is released block The state of , Run it again .
   therefore ,blocking IO It is characterized by IO Two stages of implementation （ Waiting for data and copying data ） All be block 了 .

   The first time I came into contact with network programming was from listen()、send()、recv() Wait for the interface to start , These interfaces are blocked . Using these interfaces makes it easy to build servers / The model of the client . Here is a simple “ One question and one answer ” Server for .

   Most of socket Interfaces are all blocking . A blocking interface is a system call （ It's usually IO Interface ） Do not return the call result and keep the current thread blocked , Only when the system call gets the result or the timeout error returns .
   actually , Unless otherwise specified , Almost all IO Interface ( Include socket Interface ) It's all blocking . This brings a big problem to network programming , As in the call send() At the same time , Threads will be blocked , in the meantime , Threads will not be able to perform any operations or respond to any network requests .
   A simple improvement is to use multithreading on the server side （ Or multiple processes ）. Multithreading （ Or multiple processes ） The goal is to have separate threads for each connection （ Or the process ）, In this way, the blocking of any connection will not affect other connections . How to use multi process or multi thread , There is no specific pattern . In the traditional sense , The cost of a process is much greater than that of a thread , So if you need to serve more clients at the same time , Multiple processes are not recommended ; If a single service execution needs to consume more CPU resources , Such as the need for large-scale or long-term data operations or file access , The process is more secure . Usually , Use pthread_create () Create a new thread ,fork() Create a new process .
   We assume that for the above server / The client model , Put forward higher requirements , That is, let the server provide a question and answer service for multiple clients at the same time . So we have the following model .

   In the above model , The main thread continues to wait for the connection request from the client , If there is a connection , Then create a new thread , And provide the same Q & a service for the previous example in the new thread .

   Many people may not understand why one socket Sure accept many times . actually socket The designer of may have deliberately foreshadowed the situation of multiple clients , Give Way accept() Can return a new socket. Here is accept Prototypes of interfaces ：

int accept(int s, struct sockaddr *addr, socklen_t *addrlen);

   Input parameters s It's from socket(),bind() and listen() Used in socket Handle value . After execution bind() and listen() after , The operating system has started listening for all connection requests at the specified port , If there is a request , Then add the connection request to the request queue . call accept() The interface is from socket s The request queue extracts the first connection information , Create a and s Similar new socket Return handle . new socket The handle is the following read() and recv() Input parameters of . If the request queue currently has no requests , be accept() Will enter the blocking state until a request enters the queue .

   The above multithreaded server model seems to perfectly solve the requirement of providing Q & a service for multiple clients , But not really . If you want to respond to hundreds of connection requests at the same time , No matter multithreading or multiprocessing, it will occupy system resources seriously , Reduce the response efficiency of the system to the outside world , Threads and processes themselves are more likely to enter the state of suspended animation .

   Many people may consider using “ Thread pool ” or “ Connection pool ”.“ Thread pool ” Designed to reduce the frequency of thread creation and destruction , It maintains a reasonable number of threads , And let the idle thread assume the new execution task again .“ Connection pool ” Maintain the connected cache pool , Try to reuse existing connections 、 Reduce the frequency of creating and closing connections . Both of these technologies can reduce the system overhead very well , Are widely used in many large systems , Such as websphere、tomcat And various databases . however ,“ Thread pool ” and “ Connection pool ” Technology only alleviates frequent calls to a certain extent IO The resource occupation brought by the interface . and , So-called “ pool ” There is always a ceiling , When the request is well over the upper limit ,“ pool ” The response of the constructed system to the outside world is not much better than when there is no pool . So use “ pool ” The scale of the response it faces... Has to be considered , And adjust according to the response scale “ pool ” Size .

   Corresponding to the possible simultaneous tens or even tens of thousands of client requests in the above example ,“ Thread pool ” or “ Connection pool ” Maybe it can relieve some of the pressure , But it doesn't solve all the problems . All in all , Multithreading model can solve small-scale service request conveniently and efficiently , But in the face of large-scale service requests , The multithreading model also has bottlenecks , You can try to solve this problem with a non blocking interface .

Blocking IO Advantages and disadvantages ：

advantage ： Development of simple , Easy to get started . During blocking wait , User thread suspended , Will not occupy... During suspend CPU resources

shortcoming ： One thread maintains one IO , Not suitable for large concurrent , When there is a large amount of concurrency, a large number of threads need to be created to maintain the network connection , Memory 、 Thread overhead is very large

Non blocking IO（non-blocking IO）

  Linux Next , Can be set by socket Turn it into non-blocking. When it comes to one non-blocking socket When reading , The process is like this ：

   As you can see from the diagram , When the user process issues read In operation , If kernel The data in is not ready , Then it won't block User process , But immediately return to a error. From the perspective of user process , It launched a read After the operation , There is no need to wait , It's about getting an immediate result . The result of user process judgment is a error when , It knows the data is not ready , So it can send... Again read operation . once kernel The data in is ready , And again received the user process's system call, Then it immediately copies the data to the user memory , Then return , therefore , In non blocking IO in , In fact, the user process needs continuous active inquiry kernel Is the data ready .

   In a non blocking state ,recv() The interface returns... Immediately after being called , The return value represents different meanings . In this case

• recv() Return value greater than 0, Indicates that data acceptance is complete , The return value is the number of bytes received ;
• recv() return 0, Indicates that the connection has been disconnected normally ;
• recv() return -1, And errno be equal to EAGAIN, Express recv The operation has not been completed yet ;
• recv() return -1, And errno It's not equal to EAGAIN, Express recv The operating system encountered an error errno.

   The significant difference between non blocking interfaces and blocking interfaces is , Return immediately after being called . You can use the following function to put a handle fd Set to non blocking state .

fcntl( fd, F_SETFL, O_NONBLOCK );

   The following will show that only one thread , However, it can detect whether data is delivered from multiple connections at the same time , And the model of receiving data .

   You can see that the server thread can call... Through a loop recv() Interface , You can receive all connected data in a single thread . But the above model is never recommended . because , Cycle call recv() Will push up by a large margin CPU Occupancy rate ; Besides , In this scheme recv() It's more about detection “ Is the operation complete ” The role of , The actual operating system provides more efficient detection “ Is the operation complete “ Interface of action , for example select() Multiplexing mode , It can detect whether multiple connections are active at one time .

Non blocking IO Advantages and disadvantages ：

Synchronous nonblocking IO advantage ： Every time I initiate IO call , While the kernel is waiting for data, it can immediately return , User threads don't block , Good real-time

Synchronous nonblocking IO shortcoming ： Multiple threads constantly poll the kernel for data , Take up a lot of CPU resources , The efficiency is not high . commonly Web The server will not adopt this mode

Multiplexing IO（IO multiplexing）

  IO multiplexing The word may be a little strange , But mentioned select/epoll, I think we can all understand . Some places also call this IO It's event driven IO(event driven IO). We all know ,select/epoll The good thing about it is that it's a single process You can handle multiple network connections at the same time IO. Its basic principle is select/epoll This function Will constantly poll all the socket, When a socket There's data coming in , Just inform the user of the process . Its flow chart is as follows ：

   When the user process calls select, Then the whole process will be block, At the same time ,kernel Meeting “ monitor ” all select conscientious socket, When any one socket The data in is ready ,select It will return . At this time, the user process calls read operation , Take data from kernel Copy to user process .

   This figure and blocking IO It's not that different , In fact, it's worse . Because here we need to use two system calls (select and read), and blocking IO Only one system call was called (read). But use select In the future, the biggest advantage is that the user can process multiple socket Of IO request . Users can register multiple socket, And then call again and again select Read active socket, It can handle multiple threads at the same time in the same thread IO Purpose of request . In the synchronous blocking model , You have to Multithread to do this .（ Many say ： therefore , If the number of connections processed is not very high , Use select/epoll Of web server It's not necessarily better than using multi-threading + blocking IO Of webserver Better performance , Maybe the delay is even greater .select/epoll The advantage is not that you can handle a single connection faster , It's about being able to handle more connections .）

   In the multiplexing model , For each of these socket, It's usually set to non-blocking, however , As shown in the figure above , For the entire user process In fact, it has been block Of . It's just process Be being select This function block, Rather than being socket IO to block. therefore select() And non blocking IO similar .

Most of the Unix/Linux All support select function , This function is used to detect the state change of multiple file handles .

Here's how select Prototypes of interfaces ：

FD_ZERO(int fd, fd_set* fds)
FD_SET(int fd, fd_set* fds)
FD_ISSET(int fd, fd_set* fds)
FD_CLR(int fd, fd_set* fds)
int select(int nfds, fd_set *readfds, fd_set *writefds, fd_set *exceptfds,struct timeval *timeout)

   here ,fd_set Type can be simply understood as bit Queue of bit marker handles , For example, in a fd_set A value in the tag is 16 The handle of , Then fd_set Of the 16 individual bit Bits are marked with 1. Specific setting 、 Verify that you can use FD_SET、FD_ISSET Wait for the macro to realize . stay select() Function readfds、writefds and exceptfds As both input and output parameters . If you enter readfds The tag 16 Handle to the , be select() Will detect 16 Whether the handle number is readable . stay select() After the return , It can pass the inspection readfds Whether there is a mark 16 Handle to the , To judge the right “ Can be read ” Whether the incident happened . in addition , Users can set timeout Time .

Next, we will re simulate the model of receiving data from multiple clients in the above example .

   The above model only describes the use of select() The process of receiving data from multiple clients at the same time ; because select() The interface can read the state of multiple handles at the same time 、 Detection of write status and error status , Therefore, it is easy to build a server system that provides independent question and answer services for multiple clients .

   What needs to be pointed out here is , One of the clients connect() operation , A will be fired on the server side “ Can read the event ”, therefore select() It can also detect from the client connect() Behavior .
   In the above model , The key is how to maintain dynamically select() Three parameters of readfds、writefds and exceptfds. As input parameter ,readfds All that need to be detected should be marked “ Can read the event ” The handle of , It always includes the detection connect() the “ mother ” Handle ; meanwhile ,writefds and exceptfds All that need to be detected should be marked “ Can write event ” and “ Error events ” The handle of ( Use FD_SET() Mark ).
   As an output parameter ,readfds、writefds and exceptfds Saved in select() Handle values of all events captured . All the marker bits that the programmer needs to check ( Use FD_ISSET() Check ), To determine which handles have events .

   The above model mainly simulates “ One question and one answer ” The service flow of , So if select() Found that a handle caught “ Can read the event ”, The server program should do it in time recv() operation , And prepare the data to be sent according to the received data , And add the corresponding handle value to writefds, Prepare for the next “ Can write event ” Of select() Probe . Again , If select() Found a handle that caught “ Can write event ”, Then the procedure should be done in time send() operation , And get ready for the next “ Can read the event ” Detection preparation .

   The characteristic of this model is that each execution cycle will detect one or a group of events , A specific event triggers a specific response . We can classify this model as “ Event driven model ”. Compared with other models , Use select() The event driven model of is single threaded （ process ） perform , Less occupied resources , It doesn't cost much CPU, At the same time, it can provide services for multiple clients . If you try to build a simple event driven server program , This model has certain reference value .

   But this model still has many problems . First select() An interface is not an implementation “ Event driven ” The best choice . Because when the handle value to be detected is large ,select() The interface itself takes a lot of time to poll the handles . Many operating systems provide more efficient interfaces , Such as linux Provides epoll,BSD Provides kqueue,Solaris Provides /dev/poll,…. If you need to implement more efficient server programs , similar epoll Such an interface is more recommended . Unfortunately, different operating systems are special for epoll The interface is very different , So use something like epoll It is difficult to implement a server with better cross platform capability .

   secondly , The model combines event detection with event response , Once the execution body of event response is huge , It's catastrophic for the whole model . Here's an example , A huge executive body 1 Will directly lead to response events 2 The executor of the has not been implemented for a long time , And reduce the timeliness of event detection to a great extent .

   Fortunately, , There are many efficient event driven libraries that can mask the above difficulties , Common event driven libraries are libevent library , And as libevent The replacement's libev library . These libraries will select the most appropriate event detection interface according to the characteristics of the operating system , And added a signal (signal) And other technologies to support asynchronous response , This makes these libraries the best choice for building event driven models . The next chapter describes how to use libev Library replacement select or epoll Interface , Achieve an efficient and stable server model .

Multiplexing IO Advantages and disadvantages ：

advantage ： The system does not have to create and maintain a large number of threads , Use only one thread 、 A selector can handle thousands of connections at the same time , Greatly reduces system overhead

shortcoming ： Essentially ,select and epoll The system call of is blocking , It's synchronization IO, After the read-write time is ready , Blocked reads and writes by system calls

actually ,Linux Kernel from 2.6 Start , It also introduces a method to support asynchronous response IO operation , Such as aio_read,aio_write, This is asynchronous IO.

asynchronous IO（Asynchronous I/O）

Linux Under the asynchronous IO Used on disk IO Read and write operations , Not used for network IO, From the kernel 2.6 The version just started to introduce . Let's take a look at the process

   User process initiation read After the operation , You can start doing other things right away . And on the other hand , from kernel The angle of , When it receives a asynchronous read after , First of all, it will return to , So it doesn't produce anything to the user process block. then ,kernel Waiting for the data to be ready , Then copy the data to user memory , When all this is done ,kernel A... Will be sent to the user process signal, Tell it read Operation completed .

   Use asynchronous IO The server implemented here is not an example , There will be time to write another article about . asynchronous IO It's really non blocking , It doesn't block the request process in any way , Therefore, the implementation of high concurrency network server is very important .

   up to now , There are already four IO The models are all introduced . Now there are a few questions ：blocking and non-blocking What's the difference between ,synchronous IO and asynchronous IO What's the difference between .

   Answer the simplest one first ：blocking And non-blocking. In fact, the previous introduction has clearly explained the difference between the two . call blocking IO Will always be block Live in the corresponding process until the operation is completed , and non-blocking IO stay kernel When the data is still being prepared, it will return immediately .

  synchronous IO and asynchronous IO The difference is synchronous IO do ”IO operation” Will be process Blocking . By this definition , As mentioned before blocking IO,non-blocking IO,IO multiplexing All belong to synchronous IO. Someone might say ,non-blocking IO Not by block ah . Here's a very “ The cunning ” The place of , The definition of ”IO operation” It means the real IO operation , It's in the example read This system call .non-blocking IO In execution read When this system is called , If kernel The data is not ready , Not at this time block process . But when kernel When the Chinese data is ready ,read Will transfer data from kernel Copy to user memory , At this time the process is being block 了 , During this time, the process is block Of . and asynchronous IO Is not the same , When the process starts IO After the operation , Just go straight back and ignore , until kernel Send a signal , Tell the process to say IO complete . In the whole process , The process has not been block.

asynchronous IO Is really non blocking （ Both phases are non blocking ）

Signal driven IO（signal driven I/O, SIGIO）

   First, we allow the socket to signal drive I/O, And install a signal processing function , The process continues to run without blocking . When the data is ready , The process will receive a SIGIO The signal , It can be called in the signal processing function I/O Operating functions process data . When the datagram is ready to read , The kernel generates a... For this process SIGIO The signal . We can then call it in the signal processing function read Read datagram , And inform the main cycle that the data is ready for processing , You can also notify the main loop immediately , Let it read datagrams . Deal with it anyway SIGIO The signal , The advantage of this model is to wait for the datagram to arrive ( The first stage ) period , The process can continue , Don't be blocked . Save it. select Blocking and polling , When there is an active socket , Registered by handler Handle .

   After the introduction above , Will find non-blocking IO and asynchronous IO The difference is still obvious . stay non-blocking IO in , Although the process will not be block, But it still requires the process to be proactive check, And when the data is ready , It also needs the process to call again recvfrom To copy data to user memory . and asynchronous IO It's totally different . It's like the user process will take the whole IO The operation was handed over to others （kernel） complete , And then they signal when they're done . in the meantime , User processes don't need to check IO State of operation , There's no need to actively copy data .

Five kinds of networks IO Model comparison

   Blocking IO, Non blocking IO, Multiplexing IO, Signal driven IO The main differences of these four kinds are in the first stage , They are the same in the second stage ： Data is blocked during copying from the kernel buffer to the caller buffer . They're all in sync IO, Only synchronization IO The model considers blocking and non blocking . asynchronous IO It must be non blocking .