LVS load balancing cluster deployment - Dr direct routing mode

To be continued

One 、 LVS-DR working principle

LVS-DR（Linux Virtual Server Director Server） Working mode , Is one of the most commonly used in production environments Working mode .

• LVS-DR Pattern ,Director Server As an access portal to the cluster , Not used as a gateway
• node Director Server And Real Server Need to be in the same network , The data returned to the client does not need to go through Director Server.
• In response to access to the entire cluster ,Director Server And Real Server They all need to be configured VIP Address .

• The client initiates the request , Through the dispatching server （lvs）, After algorithm scheduling , To access the real server （RS）

• Because you don't go back the same way , The client does not know , Real host ip Address ,

• Therefore, it can only be through the external network of the scheduling server ip（vip） Go back to the message information .

Two 、 Packet flow analysis

• Client sends request to Director Server, The requested datagram （ Source IP yes CIP, The goal is IP yes VIP） Get to kernel space .
• Director Server and Real Server In the same network , Data is transmitted through the layer 2 data link layer .
• Kernel space determines the destination of the packet IP It's local VIP, here IPVS Compare whether the service requested by the packet is a cluster service , If it's a cluster service, repackage the data package . Modification source MAC The address is Director Server Of MAC Address , Modify target MAC The address is Real Server Of MAC Address , Source IP Address and destination IP The address hasn't been changed change , The packet is then sent to the Real Server.
• arrive Real Server Of the request message MAC The address is its own MAC Address , This message is received . Count Reseal the message according to the packet ( Source IP The address is VIP, The goal is IP by CIP), Pass the response message lo The interface is transmitted to the physical server The network card is then sent out .
• Real Server Send the response message to the client directly .

3、 ... and 、LVS-DR Characteristics of the pattern

• Director Server and Real Server Must be in the same physical network .
• Real Server Private addresses can be used , You can also use a public address . If you use a public address , Can pass The Internet is good for RIP Make a direct visit .
• All request messages are sent through Director Server, But the response message cannot pass through Director Server.
• Real Server The gateway of is not allowed to point to Director Server IP, That is, packets are not allowed to pass through Director S erver.
• Real Server Upper lo Interface configuration VIP Of IP Address .

Four 、ARP problem

4.1 First visit complete （ Regardless of practical problems ）

 client ----> Internet address 12.0.0.188      12.0.0.188----> client  
#12.0.0.18----- client     The client will directly discard 
 Configure the Internet address for each real server  12.0.0.188
12.0.0.188------> client    

4.2 Question 1 ：IP Address conflict

stay LVS-DR Load balancing cluster , Load balancer and node server should be configured the same VIP Address , Have the same... In a LAN IP The earth site . It's bound to cause servers ARP Communication disorder

• When ARP Broadcast to LVS-DR When the cluster , because Load balancing Both the server and the node server are connected to the same network , They will all receive ARP radio broadcast

• Only the front-end load balancer responds , Other node servers should not respond ARP radio broadcast

resolvent ：

• Process the node server , Make it unresponsive to VIP Of ARP request

• Virtual interface lo:0 bearing VIP Address

• Set kernel parameters arp_ ignore=1: The system only responds to the purpose IP For the local IP Of ARP request

 Router sends ARP request （ radio broadcast ）
ARP----> Broadcast to find ip Address resolution into mac Address 
 The external network address on the dispatching server is used by default （vip Address ） Respond to ,
 You need to modify the kernel parameters on the real server 
 Make the real server only for the real server on its own server IP Address response ARP analysis .

4.3 Question two ： The second time there is another access request

RealServer Return message （ Source IP yes VIP） Forward via router , When repacking a message , You need to get the router first MAC Address , send out ARP When asked ,Linux By default IP Source of package IP Address （ namely VIP） As ARP Request source in package IP Address , Instead of using the sending interface IP Address , Router received ARP After the request , Will be updated ARP Table item , The original VIP Corresponding Director Of MAC The address will be updated to VIP Corresponding RealServer Of MAC Address . The router is based on ARP Table item , Will forward the new request message to RealServer, Lead to Director Of VIP invalid

resolvent ：

Process the node server , Set kernel parameters arp_announce=2： The system doesn't use IP Set the source address of the package ARP The source address of the request , And choose the send interface IP Address

 The router is bound with   Real server 1 Of mac Information ,
# The request reaches the real server 
 Modify kernel parameters on a real server 
 Only the addresses on the real network cards of all servers are fed back , analysis 

5、 ... and 、 Deploy LVS-DR colony

# Introduction to the environment 
DR  The server ：192.168.80.129
web  The server 1：192.168.80.128
web  The server 2：192.168.80.130
vip（ Virtual loop ）：192.168.80.188
 client ：192.168.80.133

5.1 Configure the load scheduler

1. # Turn off firewall 
systemctl stop firewalld.service
setenforce 0
 
2. # install ipvsadm Tools 
yum install ipvsadm.x86_64 -y
 
3. # Configure virtual IP Address （VIP：192.168.80.188）
cd /etc/sysconfig/network-scripts/
cp ifcfg-ens33 ifcfg-ens33:0
vim ifcfg-ens33:0
# Delete UUID,dns With gateway , Attention subnet 
NAME=ens33:0
DEVICE=ens33:0
IPADDR=192.168.80.188
NETMASK=255.255.255.255
 
4. # Restart network service 、 Start the network card 
systemctl restart network
ifup ifcfg-ens33:0
 
5. # adjustment /proc Response parameter    
   # about  DR  For cluster mode , because  LVS  The load scheduler and each node need to share  VIP  Address , It should be shut down  Linux  The redirection parameter response server of the kernel is not a router , Then it will not send a redirect , So you can turn off the function 
vi /etc/sysctl.conf
net.ipv4.ip_forward = 0
net.ipv4.conf.all.send_redirects = 0
net.ipv4.conf.default.send_redirects = 0
net.ipv4.conf.ens33.send_redirects = 0
 
6. # Refresh configuration 
sysctl -p
 
7. # Load module 
modprobe ip_vs
cat /proc/net/ip_vs
 
8. # Configure load distribution policies , And start the service 
ipvsadm-save >/etc/sysconfig/ipvsadm
systemctl start ipvsadm.service
 
9. # Empty ipvsadm, And make strategies 
## Add real servers -a   Appoint VIP Address and TCP port -t    Appoint RIP Address and TCP port  -r  Appoint DR Pattern -g
ipvsadm -C
ipvsadm -A -t 192.168.80.188:80 -s rr
ipvsadm -a -t 192.168.80.188:80 -r 192.168.80.128:80 -g
ipvsadm -a -t 192.168.80.188:80 -r 192.168.80.130:80 -g
 
10. # Save settings 
ipvsadm
ipvsadm -ln
ipvsadm-save >/etc/sysconfig/ipvsadm

Turn off firewall , install ipvsadm Tools

Configure virtual IP Address （VIP：192.168.80.188）

Restart network service 、 Start the network card  

adjustment /proc Response parameter  

Refresh configuration

Load module

 

  Configure load distribution policies , And start the service