Talk about how redis handles requests

High quality resource sharing

Learning route guidance （ Click unlock ）	Knowledge orientation	Crowd positioning
🧡 Python Actual wechat ordering applet 🧡	Progressive class	This course is python flask+ Perfect combination of wechat applet , From the deployment of Tencent to the launch of the project , Create a full stack ordering system .
Python Quantitative trading practice	beginner	Take you hand in hand to create an easy to expand 、 More secure 、 More efficient quantitative trading system

Please state the source of reprint ~, This article was published at luozhiyun The blog of ：https://www.luozhiyun.com/archives/674

This article uses Redis 5.0 Source code

I think this part of the code is quite interesting , I try to explain it in a more popular way

summary

I remember I was Let's make it clear Go Language HTTP Standard library In this article, it is analyzed that Go How to create a Server End program ：

• The first is to register the processor ;
• Open the loop listening port , Every time you hear a connection, you create one Goroutine;
• Then is Goroutine There will be a loop waiting to receive the request data , Then according to the requested address to match the corresponding processor in the processor routing table , Then the request is sent to the processor for processing ;

In code, that's it ：

Copyfunc (srv *Server) Serve(l net.Listener) error { 
    ...
    baseCtx := context.Background()  
    ctx := context.WithValue(baseCtx, ServerContextKey, srv)
    for {
        //  receive  listener  Network connection over here 
        rw, err := l.Accept()
        ... 
        tempDelay = 0
        c := srv.newConn(rw)
        c.setState(c.rwc, StateNew) 
        //  Create coprocessing connection 
        go c.serve(connCtx)
    }
}

about Redis It's a little different , Because it's single threaded , Cannot use multithreading to process connections , therefore Redis Select Use Based on Reactor Mode event driver to realize the concurrent processing of events .

stay Redis Zhongso Reactor The pattern is through epoll To monitor multiple fd, Whenever these fd When there is a response, it will be notified in the form of an event epoll Make a callback , Each event has a corresponding event handler .

Such as ： accept Corresponding acceptTCPHandler Event handler 、read & write Corresponding readQueryFromClient Event handlers, etc , Then, the event is allocated to the event processor for processing in the form of circular distribution of events .

So this one above Reactor Patterns are all through epoll To achieve , about epoll There are three main methods ：

Copy// Create a epoll The handle of ,size Used to tell the kernel how many monitors there are in total 
int epoll\_create(int size);

/*
 *  It can be understood as , Additions and deletions  fd  Events that need to be monitored 
 * epfd  yes  epoll\_create()  Created handle .
 * op  Express   Additions and deletions 
 * epoll\_event  Indicates the event to be monitored ,Redis  Only readable , Can write , error , Hang up   Four states 
 */
int epoll\_ctl(int epfd, int op, int fd, struct epoll\_event *event);

/*
 *  It can be understood as querying qualified Events 
 * epfd  yes  epoll\_create()  Created handle .
 * epoll\_event  Used to store the collection of events obtained from the kernel 
 * maxevents  Maximum number of events obtained 
 * timeout  Waiting timeout 
 */
int epoll\_wait(int epfd, struct epoll\_event * events, int maxevents, int timeout);

So we can implement a simple server：

Copy//  Create a listener 
int listenfd = ::socket();

//  binding ip And port 
int r = ::bind();  
//  establish  epoll  example 
int epollfd = epoll_create(xxx); 
//  add to epoll The type of event to listen for 
int r = epoll_ctl(..., listenfd, ...);
 
struct epoll\_event* alive\_events =  static_cast(calloc(kMaxEvents, sizeof(epoll\_event)));

while (true) {
 //  Events wait 
 int num = epoll\_wait(epollfd, alive\_events, kMaxEvents, kEpollWaitTime);
 //  Traversal Events , And handle the event 
 for (int i = 0; i < num; ++i) {
 int fd = alive\_events[i].data.fd;
 //  Get events 
 int events = alive\_events[i].events;
 //  Distribution of events 
 if ( (events & EPOLLERR) || (events & EPOLLHUP) ) {
 ...
 } else if (events & EPOLLRDHUP) {
 ...
 } 
 ...
 } 
}

Calling process

So according to the above introduction , You know, for Redis There are only a few steps in an event cycle ：

1. Register event listening and callback functions ;
2. Loop waiting to get the event and handle ;
3. Call callback function , Processing data logic ;
4. Write back data to Client;

1. register fd To epoll in , And set the callback function acceptTcpHandler, If there is a new connection, the callback function will be called ;
2. Start an endless loop call epoll_wait Wait and continue to handle the event , We will return to aeMain Loop call in function aeProcessEvents function ;
3. When a network incident comes , Will follow the callback function acceptTcpHandler Call all the way to readQueryFromClient Data processing ,readQueryFromClient Can parse client The data of , Find the corresponding cmd Function execution ;
4. Redis After the instance receives the client request , After processing the client command , Write the data to be returned to the client output buffer instead of returning immediately ;
5. And then in aeMain Every time the function loops, it calls beforeSleep Function writes the data in the buffer back to the client ;

In fact, the code steps of the whole event cycle above have been written very clearly , There are also many articles on the Internet , I won't talk more .

Command execution process & Write back client

Command execution

Now let's talk about some things that are not mentioned in many online articles , have a look Redis How to execute commands , And then it's stored in the cache , And write data back from the cache Client This process .

In the previous section, we also mentioned , If a network event comes, it will call readQueryFromClient function , It's where the orders are actually executed . We just follow this method and keep looking down ：

1. readQueryFromClient It will call processInputBufferAndReplicate Function to process the requested command ;
2. stay processInputBufferAndReplicate It's called inside the function processInputBuffer And judge if it is a cluster mode , Whether you need to copy commands to other nodes ;
3. processInputBuffer The function will cycle through the requested commands , And call according to the requested Protocol processInlineBuffer function , take redisObject Call after object processCommand Carry out orders ;
4. processCommand When executing the order, it will pass lookupCommand Go to server.commands Find the corresponding execution function according to the command in the table , Then after a series of checks , Call the corresponding function to execute the command , call addReply Write the data to be returned to the client output buffer ;

server.commands Will be in populateCommandTable In the function, all Redis Order to register , As a table that gets command functions according to the command name .

for instance , To execute get command , Then it will call getCommand function ：

Copyvoid getCommand(client *c) {
    getGenericCommand(c);
}

int getGenericCommand(client *c) {
    robj *o;
	//  Find data 
    if ((o = lookupKeyReadOrReply(c,c->argv[1],shared.nullbulk)) == NULL)
        return C_OK;
    ...
}

robj *lookupKeyReadOrReply(client *c, robj *key, robj *reply) {
    // To db Find data in 
    robj *o = lookupKeyRead(c->db, key);
    //  Write to cache 
    if (!o) addReply(c,reply);
    return o;
}

stay getCommand Data found in function , And then call addReply Write the data to be returned to the client output buffer .

Data write back client

After executing the command above and writing to the buffer , You also need to fetch data from the buffer and return it to Client. For the process of writing data back to the client , In fact, it is also completed in the event cycle of the server .

1. First Redis Will be in main Call in function aeSetBeforeSleepProc The function will write back to the function of the package beforeSleep Sign up to eventLoop In the middle ;
2. then Redis Calling aeMain Functions will judge when they cycle events beforesleep Is it set , If there is , Then it will be called ;
3. beforesleep Function will call handleClientsWithPendingWrites function , It will be called writeToClient Write data back from the buffer to the client ;

summary

This article introduces the whole Redis What is the request processing model of . Listen from registration fd Event to execute command , At last, I wrote the data back to the client and made a general analysis . Of course, this article is also a little different from my previous articles , No lengthy post code , Mainly, I don't think it's necessary , Interested can follow the flow chart to see the code .

Reference

http://www.dre.vanderbilt.edu/~schmidt/PDF/reactor-siemens.pdf

https://time.geekbang.org/column/article/408491

http://remcarpediem.net/article/1aa2da89/

https://github.com/Junnplus/blog/issues/37

https://blog.csdn.net/neooelric/p/9629948.html