Detailed explanation of five types of load balancing principle scenarios

1  Classification of common load balancing

Speaking of common load balancing classifications , Let's see what is OSI Seven layer model . We often say the second floor / Three layers / Four layer switch , Is based on OSI The layer corresponding to the seven layer model . When I first came into contact with network devices , The first time I heard of a layer 2 switch 、 Layer 3 switch , I thought the hardware thickness of the switch had two layers 、 There are three floors , Only later did I know , In fact, according to OSI Seven layer model to distinguish the principle and function of different switches .OSI The seven layer model is mainly used as a general model for theoretical analysis , It is a theoretical model . and TCP/IP Communication protocol model （ four layers ） Is the actual communication protocol of the Internet , The two are generally mapped, compared and analyzed .

OSI Seven layer model and corresponding transmission protocol

There are many common load balancing methods , There is a famous hardware load balancing , Such as F5、Netscaler; There are also common open source load balancing , such as LVS、HAProxy、Nginx, even to the extent that Apache Can also do load balancing （ Not recommended ）. We often talk about seven layer load balancing 、 Four layer load balancing , Is based on OSI Seven layer model . be based on OSI The underlying principle of the seven layer model , We divide the common load balancing into the following types ： On the second floor 、 Three layers 、 four layers 、 Seven layers . Finally, add a division OSI Beyond the seven tier model DNS, adopt DNS Load balancing scenarios are also very common .

Classification of load balancing

The current second floor 、 The implementation of three-layer load balancing is only LVS Can do . Compared with the fourth floor 、 Seven layer load balancing is more widely used , Such as hardware load balancing 、Nginx、HAProxy Popular middleware supports four layers 、 Seven layer load balancing .

2  The originator of load balancing —LVS How it works

As mentioned above LVS Application scenarios and features of , But on the second floor / Three layers / four layers / Before the seven layer load balancing comparison , It's necessary to say LVS Principle . In the field of operations and maintenance , I'm sure you're right LVS No stranger , Almost mention load balancing , When it comes to operation and maintenance , All will mention LVS. In the practical application of load balancing , I saw some interesting phenomena , Next, I will share with you the relevant technical practices in detail . Before that , Let's take a look at the abbreviations of load balancing technology .

Abbreviations of terms related to load balancing technology

LVS Implementation of load balancing technology , Mainly by IPVS and Ipvsadm Realization .

• IPVS： yes LVS The core part of the cluster system , Is based on Linux Netfilter A kernel module implemented by the framework , Mainly works in kernel space INPUT On the chain . Its hook functions are HOOK stay LOCAL_IN and FORWARD Two HOOK spot .

Here's the thing to watch out for ,IPVS It directly acts on the kernel space to modify and forward data packets . and Nginx/HAProxy Act in user space , This makes LVS More robust performance （ Be able to catch up with the performance of hardware load balancing ）, and Nginx/HAProxy It's not an order of magnitude at all . And now IPVS It's already Linux Part of the kernel standard , Early Linux In the system version , install LVS You also need to recompile the kernel , Of course, it is no longer necessary now .

kernel LVS Packets flow to

• Ipvsadm： and Ipvsadm Working in user space , It is mainly used to define and manage cluster services . So the actual installation configuration LVS when , Mainly installation and configuration Ipvsadm. For example Redhat A simple command is installed in ：

yum install ipvsadm

LVS Principle framework

stay Director Server On ,IPVS Create a virtual server that provides external access IP Address , Users have to go through this virtual VIP Address access server . because LVS Is to use VIP（ Three layers IP Address ） As a request entry , This is also what many people like to put LVS Collectively referred to as IP The reason for load balancing , That is, three-layer load balancing . but LVS Yes DR/IP TUN/NAT3 Patterns , The core principles of each mode act on the second layer / Three layers / four layers , So the LVS It is called the second floor / Three layers / Four layer load balancing is more appropriate .

The direction of specific data packets is as follows ：

1. The access request first passes through VIP Reach the kernel space of the load scheduler .

2. PREROUTING After receiving the user's request, the chain , Will judge the target IP, Make sure it's local IP, Send packets to INPUT chain .

3. When the user request arrives INPUT when ,IPVS The user request and Ipvsadm Compare the defined rules . If the user requests a defined cluster service , So at this time IPVS Will forcibly modify the packet , And send new packets to POSTROUTING chain .

4. POSTROUTING Link to receive packet and find target IP The address happens to be your own back-end server , Finally, the data packet is sent to the back-end server . Modification of data package , Based on different modification methods , To form the LVS Of 3 Patterns .

LVS Three models

3  Second level load balancing

The current load balancing world , Only LVS Two layer load balancing is realized . So the two-layer load balancing practice is mainly LVS Of DR Pattern Practice .

Two layer load balancing packet routing principle

The following is a detailed description of the packet routing principle of layer 2 load balancing ：

1. The source address of the client request packet message is CIP, The destination address is VIP.

2. Load balancing will the source of the client request packet message MAC Change the address to yourself DIP Of MAC Address , The goal is MAC Change it to RIP Of MAC Address , And send this package to the back-end server . It is required that all back-end servers and load balancing servers can only be in one server VLAN（ LAN ） Inside , That is, you can't cross VLAN. According to the two-layer principle, we can see that , The source of the client can be directly obtained from the back-end server IP Address ,netstat -n You can directly view the source of client request communication IP. The sample code is as follows ：># netstat -nActive Internet connections (w/o servers)Proto Recv-Q Send-Q Local Address Foreign Address Statetcp 0 0 192.168.3.33:80 47.100.187.71:13537 ESTABLISHEDtcp 0 0 192.168.3.33:80 115.29.209.104:49435 ESTABLISHEDtcp 0 0 192.168.3.33:80 115.19.244.124:45418 TIME_WAITtcp 0 0 192.168.3.33:80 115.29.233.224:59310 ESTABLISHED

3. The backend server finds the destination in the request packet MAC Is your own , Will receive the packet message . Due to the of the packet MAC The address has been modified , Therefore, the back-end server needs to be in lo Network port binding VIP, In this way, packets will have “ Sense of belonging ”. After processing the request message , Pass the response message lo Interface to eth0 The network card is sent directly to the client . Because the data packets are directly returned to the client by the back-end server , Therefore, the backend server must be bound to the public network IP.

4  Three layer load balancing

Again , In the current load balancing world , Only LVS Three layer load balancing is realized . Therefore, the three-tier load balancing practices are mainly LVS Of IP-TUN Pattern Practice .

Three layer load balancing packet routing principle

The following is a detailed description of the packet routing principle of three-layer load balancing ：

1. The source address of the client request packet message is CIP, The destination address is VIP.

2. Load balancing encapsulates the header of the client request packet IP message , Change the source address to DIP, Target address changed to RIP, And send the packet to the back-end server . Unlike layer 2 Load Balancing , Packets communicate through TUNNEL Pattern realization , Therefore, both internal and external networks can communicate , So no need LVS VIP In the same network segment as the back-end server , Immediate span VLAN. However, the three-tier load balancing principle makes it impossible to directly obtain the source of the client in the back-end server IP Address ,netstat -n What you can see is the communication with load balancing IP. The sample code is as follows ：># netstat -nActive Internet connections (w/o servers)Proto Recv-Q Send-Q Local Address Foreign Address Statetcp 0 0 192.168.3.33:80 172.16.2.2:13537 ESTABLISHEDtcp 0 0 192.168.3.33:80 172.16.2.2:49435 ESTABLISHEDtcp 0 0 192.168.3.33:80 172.16.2.2:45418 TIME_WAITtcp 0 0 192.168.3.33:80 172.16.2.2:59310 ESTABLISHED

3. After the back-end server receives the request message , The first layer of packaging is broken down , And then there's another layer IP The target address of the header is itself lo On the interface VIP, So it's going to process the sub-request packet , And pass the response message lo Interface to eth0 The network card is sent directly to the client .

TUNNEL Mode, tunnel mode , Operation and maintenance are difficult , Not commonly used in practical applications .

5  Four layer load balancing

LVS Of NAT Pattern 、 The four layers of Alibaba cloud SLB、Nginx/HAProxy Four floors , Although they are all four layer load balancing , But their underlying principles are very different , This leads to a great difference in their functional characteristics .

5.1 LVS-NAT Four layer load balancing under

Four layer load balancing LVS-NAT The principle of packet routing is shown in the figure below . The following is the four layer load balancing LVS-NAT A detailed description of the packet routing principle .

1. The source address of the client request packet message is CIP, The destination address is VIP.

2. Load balancing changes the target address of the client request packet message to RIP Address , And send the packet to the back-end server . It is also required that all back-end servers and load balancing servers can only be in one server VLAN（ LAN ） Inside , That is, you can't cross VLAN. Again , according to LVS Of NAT The principle of mode modification packet message , The source of the client can be directly obtained from the back-end server IP Address ,netstat -n You can directly view the source of client request communication IP.

3. After the message is sent to the back-end server , The target server will respond to the request , And return the response packet message to the load balancer . The gateway address of each internal back-end server must be the intranet address of the server where the load balancing is located , To configure SNAT, So that the packet can pass through LVS Return to the client .

4. Then the load balancer modifies the source address of the packet to the local machine and sends it to the client .

Four layer load balancing LVS-NAT Packet routing principle

We can see , When the target server finishes processing packets and returns them to the client , Pass by LVS, Then send the packet back to the client . From this we can see that ,LVS NAT The pattern has a big performance bottleneck （ It's about LVS This end ）, Compared to DR And IP-TUN Pattern , This problem does not exist when the packet back-end server directly returns it to the client . In practice , I have also encountered this real case of performance bottleneck . I worked in a company that mainly engaged in e-commerce , In the preparation for the annual double 11 , During the peak traffic period, the client is seriously stuck 、 Packet loss and delay . At that time, the load balancing method was LVS Of NAT Pattern , We found that in high concurrency and high traffic mode , The intranet traffic can reach Gigabit . Then the emergency switch to LVS DR Pattern , The fault was quickly eliminated .

5.2  Alibaba cloud SLB Four layer load balancing under

Alicloud four layer load balancing is aimed at LVS Some problems are customized and optimized , New forwarding mode FULLNAT.

Alicloud four layer load balancing is right for LVS The optimization of the

This shows that based on LVS, Alicloud's four layer load balancing has realized cross VLAN、 Have defense DDoS Ability to attack 、 Deploy in cluster mode . this 3 Compared with the original ecological LVS, The function has been greatly improved .

In practical application , We found that Alibaba cloud's four layer load balancing , The direction of the packets is the same as that of the backend ECS Whether the public network is bound IP There is a direct connection （ Classic networks only ）.

• When the back end ECS When the public network is not bound , Load balancing forwards packets to the backend ECS. Back end ECS After processing, the data packet will be returned to the load balancer , Load balancing returns packets to the client . This way is similar to LVS Of NAT Pattern .

• When the back-end classic network ECS When binding the public network （ Only the classic network ECS Bound public network IP The public network card will be allocated separately eth1）, Load balancing forwards packets to the backend ECS. Back end ECS After processing, the data packet is directly returned to the client through the public network card . This way is similar to LVS Of DR Pattern . stay ECS The traffic details can not be seen in the monitoring of the public network card , This traffic content is directly merged into SLB It is calculated in this paper . But we go through Zabbix Related monitoring , You can see eth1 Public network card traffic , And can catch the corresponding details . Sometimes the actual public network card traffic even far exceeds the peak public network bandwidth , From this we can see that , This part of the data packets returned to the client go to the public network card , Not subject to backend ECS Bound public network bandwidth limit , And not included in ECS Traffic billing , Put it alone SLB Traffic billing is in progress .

If ECS There are public IP And private network IP, Disabling the public network adapter will affect the load balancing service , Because when there is a public network card , The default route will go through the public network , If disabled, you cannot repack , Therefore, the load balancing service will be affected . It is recommended not to prohibit , If you must do so , You need to change the default route to private network so that the service will not be affected . However, it is necessary to consider whether the business depends on the public network , Such as access through the public network RDS etc. .

5.3 Nginx/HAProxy Four layer load balancing under

Four layer load balancing Nginx/HAProxy The principle of packet routing is shown in the figure below . The following is the four layer load balancing Nginx/HAProxy A detailed description of the packet routing principle .

Four layer load balancing Nginx/HAProxy Packet routing principle

It's important to pay attention to , This has been followed by LVS There are essential differences .LVS Of NAT The external mode is virtual VIP As a request entry （IP There are three layers ）, Then, on the basis of three-layer load balancing, use IP+PORT Receiving request , And then forward to the corresponding back-end server , therefore LVS Of NAT The mode is three-tier load balancing + Four layer load balancing . and Nginx/HAProxy The four layers of are directly exposed to the outside world DIP+TCP/UDP IP Port services .

1. The source address of the client request packet message is CIP, The destination address is DIP+IP port .

2. Load balancing receives packets in user space , And load balancing and back-end servers initiate new TCP Three handshakes , Set up new TCP Connect . So at this time, it is load balancing that replaces the client and the back-end to initiate new TCP Request connection . The request packet is a new request packet , The source address of the message is DIP, The destination address is RIP. So in the back-end server ,netstat -n What you see is the communication with load balancing IP. But load balancing and back-end servers are built new TCP Connect , Therefore, the back-end server and the server where the load balancer is located can cross VLAN（ LAN ） signal communication .

3. After the message is sent to the back-end server , The server responds to the request , And return the response packet message to the load balancer . comparison LVS Of NAT Pattern ,Nginx/HAProxy The four layer load balancing under does not need to set the gateway address of each internal back-end server as the intranet address of the server where the load balancing is located , That is, no configuration is required SNAT. Because it is load balancing and back-end servers that set up new TCP Connect , Don't worry that packets won't be returned to the load balancer .

4. Then the load balancer repackages the response content of the packet and returns it to the client .

6  Seven layer load balancing

In common load balancing classifications , In fact, layer 7 and layer 4 load balancing are the most widely used . The main difference between layer 7 and layer 4 load balancing is , In the seven layer load balancing, users' requests can be obtained HTTP Header information . What exactly is HTTP Header information , We use nginx.conf Defined in the configuration Nginx The format and content of the log file are explained from the side ：

log_format '$remote_addr - $remote_user [$time_local] "$request" ''$status $body_bytes_sent "$http_referer" ''"$http_user_agent" "$http_x_forwarded_for"';

Then we can see the request header information of the specific client in the corresponding log file , as follows ：

45.127.220.171 - - [02/Dec/2018:14:10:17 +0800] "GET https://qiaobangzhu.cn ( https://qiaobangzhu.cn ) HTTP/1.1" 301 184 "-" "Mozilla/5.0 (Windows NT 6.1; WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/51.0.2704.103 Safari/537.36"

You can see that in the seven layer load balancing , We can get the client access IP、HTTP Request type （GET/POST）、 The host address of the domain name visited and the requested URL Detailed 、 Browser type, etc , This is HTTP Request header information . The principle of seven layer load balancing is based on HTTP The request header is used for judging and forwarding , The virtual host function is the most widely used in the seven layer load balancing . In fact, the core of the virtual host function is to obtain HTTP In the request header HOST Field to correspond to the matching forwarding .

Seven layer load balancing packet routing principle

The following is a detailed description of the seven layer load balancing packet routing principle .

1. and Nginx/HAProxy It's the same as the fourth floor , The source address of the client request packet message is CIP. But in practice , The destination address accessed here is not DIP+IP port , It is URL.

2. Same as Nginx/HAProxy It's the same as the fourth floor , Load balancing receives packets in user space , And load balancing and back-end servers initiate new TCP Three handshakes , Set up new TCP Connect . So at this time, it is load balancing that replaces the client and the back-end to initiate new TCP Request connection . The request packet is new , The source address of the message is DIP, The destination address is RIP, And there is the target of the client request URL（ Here you can see , For seven layer load balancing , The destination address may be changing all the time , But the goal of the visit URL Always the same , Unless the client request content is modified ）.

3. After the message is sent to the back-end server , The server responds to the request , And return the response packet message to the load balancer . Follow Nginx/HAProxy equally , There is no need to configure the gateway of the back-end server , namely SNAT Related configuration .

4. Then the load balancer repackages the response content of the packet and returns it to the client .

in summary , Because the application layer can be obtained from the seven layers HTTP Request content for , Therefore, the seven layer load balancing has the following common application scenarios ：

• Seven layers act on HTTP application layer , Therefore, the load balancing of layer 7 can only follow Tomcat、PHP etc. Web Service load balancing .

• You can forward according to the requested domain name . For example, the requester visits A domain name , Forward to the back end A The server ; Request access B domain name , Forward to the back end B The server . This function , It is called virtual host function on the seventh floor , It is the most popular application practice in the seven tier application .

• According to the request URL To do forwarding . For example, the requester accesses URL contain A Catalog , Just forward it to A The server ; Requested access to URL contain B Catalog , Just forward it to B The server .

• It can be forwarded according to the browser type of the request . For example, the browser type used by the requester is Chrome, Just forward it to A The server ; The type of browser used by the requester is Firefox, Just forward it to B The server .

And the four layers can only get the access target / Source IP Address and port . So four layers of load balancing , Simply forward the request poll to the back-end target server . It can't be the same as the seventh floor , Make a logical judgment , Finally, it is forwarded to the back-end target server that meets the requirements . Compared with the seventh floor , The four tier application scenarios include ：

• Yes MySQL、LDAP、Redis、Memcache And other four tier applications for load balancing .

• Yes, the seventh floor HTTP Of Web Class applications do load balancing , Such as Tomcat、PHP.

• Yes, the seventh floor Nginx、HAProxy Do load balancing （ In practice , How to use the fourth floor and the seventh floor together , See the relevant sections of load balancing in Chapter 2 for details ）.

But there is no way that the fourth floor is the same as the seventh floor , Do the application of virtual host . I once asked this question in an interview , Namely LVS Can you identify the domain name for forwarding . For example, the requester visits A domain name , Forward to the back end A The server ; Request access B domain name , Forward to the back end B The server . What's interesting is that , Some experienced operation and maintenance personnel still can't answer this question , This requires a good look at the underlying principles and the corresponding load balancing application scenarios . The answer must be No , because LVS On the fourth and second floors , There is no way to identify the contents of packets encapsulated in seven layers . Another example of a seven layer load balancing practice , To access a system in the cloud , The client must use Chrome browser , Refer to the following core configurations ：

location / {if ( $http_user_agent !~ ^.Chrome/5|6. ){rewrite ^(.)$ https://qiaobangzhu.cn/error.html ( https://qiaobangzhu.cn/error.html ) permanent;}proxy_pass http://192.168.2.2:81/ ( http://192.168.2.2:81/ );proxy_set_header Host $host;proxy_set_header X-Real-IP $remote_addr;proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;}

The core configuration of the implementation is to obtain the data in the seven layer header information for comparison and judgment , If meet , Just execute the corresponding operation . This is the fourth floor / Functions that cannot be realized by layer-2 load balancing .

7  Load balanced “ One connection ” and “ Two connections ” Core summary

7.1 LVS On the second floor / Four story “ One connection ”

LVS Of DR Pattern 、NAT The mode only does the processing of data packets “ One connection ”, That is, load balancing only forwards packets .

LVS The third handshake packet of

LVS Able to “ One connection ” The essential reason for this is LVS Work in kernel space .LVS3 Both modes work in kernel space , The processing of data packets is only in kernel space , This is also LVS Lightweight and efficient 、 The most essential reason for high performance .

LVS Able to “ One connection ”, So through the four layers SLB, We can go directly to the back-end server ECS in netstat -n View the source of client communication IP Address .

LVS Request the packet to

7.2 Nginx/HAProxy Four story “ Secondary connection ”

Compared with LVS,Nginx/HAProxy The fourth floor should be established “ Secondary connection ”.

Nginx/HAProxy Four layer three-time handshake packet direction

The client is moving towards load balancing TCP After three handshakes , Load balancing will launch a new TCP Connect , That is to say “ Secondary connection ”. Since it is load balancing with the back-end to build new TCP Three handshakes and forwarding the data requested by the client , So on the back-end server netstat -n View the of the requested communication IP It 's load balanced IP. Compared to LVS,Nginx/HAProxy The fourth floor works in the user space , The processing of packets is done in user space , The flow and processing of data packets are increasing , This is also Nginx/HAProxy The performance and performance of LVS The essential reason for this magnitude .

Nginx/HAProxy Request the packet to

7.3 Nginx/HAProxy Seven story “ Secondary connection ”

Compared with Nginx/HAProxy Secondary connection of four layers , and Nginx/HAProxy The secondary connection of the seventh floor is somewhat different .Nginx/HAProxy Secondary connection of four layers , Client side and load balancing TCP After three handshakes , Load balancing and back-end servers will be updated immediately TCP Three handshakes . and Nginx/HAProxy The secondary connection of the seventh floor , On the client side and load balancing TCP After three handshakes , You still need to wait for the client Pushdata To transmit data , After that, load balancing and back-end servers will establish new servers TCP Three handshakes . thus it can be seen ,Nginx/HAProxy The secondary connection forwarding efficiency of the four layers will be higher . add Nginx/HAProxy The seventh floor will have some Rewrite Judgment of rules , Will lose some CPU And memory performance . So comparatively speaking ,Nginx/HAProxy The performance of the four layers is much higher . Again , Because it is load balancing to establish a new network with the backend TCP Three handshakes and forwarding the data requested by the client , So on the back-end server netstat -n View the of the requested communication IP It 's load balanced IP. So in the back-end server ,HTTP The head of the remote_addr Although it represents the client IP, But the actual value is load balanced IP. To avoid this situation , Layer 7 load balancing usually adds a new layer called x_forwarded_for The header information , Connect its client IP（ Internet machine IP） Add to this header , This will ensure that the back-end server can obtain the real IP. But actually , We will encounter the case that the back-end server forwards the request to the next target server , namely ：“ Client request >>> Nginx >>> Nginx1 >>> Nginx2” Structure , We are Nginx1 Pass on server x_forwarded_for The header information of gets the real source of the client IP, How is that Nginx2 Go to get the source of the client IP Well ？ stay Nginx1 In the forwarding rules on , Configure the following code ：

proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;

This will make Nginx1 Forward to the back end HTTP In the header x_forwarded_for Keep the real source of the customer in the information IP.

Nginx/HAProxy Layer 7 triple handshake packet trend

Nginx The seventh floor can be “ Secondary connection ”, So through the seventh floor SLB, We are on the back-end server ECS in netstat -n View the communication with the back-end server IP by SLB Of IP Address , Instead, you cannot directly obtain the source of client communication IP.

8 DNS Load balancing

from OSI From the seven layer model ,DNS Domain name resolution is actually used in the seventh application layer .

DNS Request flow to

In the picture , Essentially DNS The load balancing of is not a seven layer load balancing , because DNS Analysis is in TCP/IP Before communication . That is, if the domain name is used in the application , You need to ask first DNS The server gets the target IP Address , To build TCP/IP signal communication . Compared with the widely used four layer and seven layer load balancing ,DNS Load balancing scenarios are less common .DNS Load balancing brings two big problems , The two questions are as follows ：

1. In practice , adopt DNS Resolve multiple configurations A Record the address . Different clients to request DNS, return A Record the different sources of the configuration IP（ It can also be load balanced VIP Address ）, So as to achieve the effect of load balancing . But what we found was that , Set up DNS Load balancing of , Fall to different sources IP（ It can also be load balanced VIP Address ） The request traffic is often unevenly distributed . There may be a large number of requests for a certain back-end address , However, the request volume of the other back-end address is very small .

2. Because the client often has DNS Corresponding cache , If so DNS A source of parsing IP Service exception , Generally, it will not actively eliminate the abnormal source IP analysis . This may lead to the resolution access of some customers or the abnormal service address . Although now DNS It has advanced functions of intelligent parsing , Be able to actively monitor the availability of back-end services . But the only thing we can't control is the client DNS cache , Most client computers DNS There are caches . It could be DNS It has been resolved to the latest IP, But at this time, the client DNS The cache will still get the resolved old IP, This will cause the client to parse and access the service with exception for a period of time IP.

DNS Load balancing of , Generally in very large-scale applications , In particular, it is widely used in cross regional distributed applications , Regular small and medium-sized 、 Medium and large scale applications , It is not particularly recommended to try DNS Load balancing of .DNS As an application scenario of load balancing, it will be introduced in detail in the second article .

9  summary

Summary of functional characteristics of different load balancing types .