LVS load balancing cluster of efficient multi-purpose cluster (NAT mode)

One 、 Understand the principle of load balancing clustering

1.1、 Enterprise cluster application concept

The meaning of clustering

• Cluster, colony 、 to cluster around
• Composed of multiple hosts , But externally, it is only a whole , Only one
  Access portal ( Domain name or IP Address ),, It's like a mainframe computer .

problem

• In Internet applications , As the site improves hardware performance 、 Response speed 、 Service stability 、 Data reliability and other requirements are getting higher and higher , A single server can no longer meet the requirements of load balancing and high availability .

resolvent

• Using expensive minicomputers 、 The mainframe

• Build a service cluster with several relatively cheap ordinary servers

  By consolidating multiple servers , Use LVS To achieve high availability and load balancing of the server , And with the same IP The address provides the same service .
  A cluster technology commonly used in enterprises LVS (Linux Virtual Server, Linux Virtual server )

Two 、 Enterprise cluster classification

Depending on the target of the cluster , It can be divided into three types

• Load balancing cluster
• High availability cluster
• High performance computing cluster

2.1、 Load balancing cluster （Load Balance Cluster）

• Improve the response ability of the application system 、 Handle as many access requests as possible 、
  The goal is to reduce delay , Get high concurrency 、 load (LB) Overall performance of
• LB The load distribution depends on the shunting algorithm of the master node , From the client
  Access requests are shared among multiple server nodes , So as to alleviate the negative impact of the whole system
  Loading pressure . for example ,“DNS polling ” " Reverse proxy " etc.

2.2、 High availability cluster （High Availability Cluster）

• Improve the reliability of the application system 、 The goal is to minimize interruption time , Ensure continuity of service , To achieve high availability (HA) Fault tolerant effect of
• HA There are two modes of operation of the system: duplex mode and master-slave mode , Duplex means that all nodes are online at the same time ; Only the master node is online , But when there is a failure, the slave node can automatically switch to the master node .
  for example ,” Fail over ” “ hot standby ” etc.

2.3、 High performance computing cluster

• In order to improve the efficiency of the application system CPU speed 、 Expand hardware resources and analysis capabilities , Get the equivalent of large 、 High performance computing in supercomputers (HPC) Ability
• High performance depends on " Distributed computing ”、“ Parallel computing ” , Through dedicated hardware and software, multiple servers will be CPU、 Memory and other resources are integrated together , Implementation is only large 、 The computing power of supercomputers . for example ,“ Cloud computing ” " Grid computing " etc.

3、 ... and 、 Load balancing cluster architecture and working mode

3.1、 The structure of load balancing

• first floor , Load scheduler （Load Balancer or Director）

The only access to the entire cluster system , It is common for all servers to be used externally VIP Address , Also known as clustering IP Address . The main 、 Backup two dispatchers for hot backup , When the main dispatcher fails, it can be smoothly replaced by the standby dispatcher , Ensure high availability .

• The second floor , The server pool （Server Pool）

Application services provided by cluster 、 It's up to the server pool , Each node has its own RIP Address ( real IP), Only handle client requests distributed by the scheduler . When a node fails temporarily , The load scheduler's fault tolerance mechanism isolates it , Wait for the error to be eliminated before being brought back into the server pool .

• The third level , Shared memory （Share Storage）

Provide stability for all nodes in the server pool 、 Consistent file access service , Ensure the consistency of the entire cluster . Shared storage can use NAS equipment , Or offer NFS A dedicated server for shared services .

3.2、 Analysis of load balancing cluster working mode

• Load balancing cluster is the most commonly used cluster type in enterprises

• The load scheduling technology of cluster has three working modes （LVS）

  • address translation （NAT Pattern ）
  • IP Tunnel （TUN Pattern ）
  • Direct routing （DR Pattern ）

address translation （NAT）

• Network Address Translation, abbreviation NAT Pattern

• A private network structure similar to a firewall , The load scheduler acts as a gateway to all server nodes , That is, as the access portal of the client , It is also the access exit for each node to respond to the client , It will become the bottleneck of the whole load balancing

• The server node uses private IP Address , On the same physical network as the load scheduler , Security is better than the other two

IP Tunnel （TUN Pattern ）

• IP Tunnel, abbreviation TUN Pattern
• Adopt an open network structure , The load scheduler serves only as the access portal for the client , Each node through its own Internet Connect to the direct response client , Instead of going through the load scheduler
• Server nodes are scattered in different locations in the Internet , It has an independent public network IP Address , Through dedicated IP The tunnel communicates with the load scheduler

Direct routing （DR Pattern ）

• Direct Routing, abbreviation DR Pattern
• Adopt semi open network structure , And TUN The structure of the pattern is similar to , But the nodes are not scattered all over the place , It's on the same physical network as the scheduler
• The load scheduler is connected to each node server through the local network , There is no need to create a dedicated IP Tunnel

Four 、LVS Virtual server

Linux Virtual Server

• in the light of Linux Load balancing solution developed by kernel
• 1998 year 5 month , Founded by Dr. Zhang wensong of China
• Official website : http://www.linuxvirtualserver.orgl
• LVS It's actually equivalent to based on IP Address virtualization applications , Based on IP Load balancing of address and content request distribution presents an efficient solution

LVS Now it has become Linux Part of the kernel , The default compilation is ip_vs modular , Automatically call... When necessary . stay CentOS 7 In the system , The following operations can be manually loaded ip_vs modular , And look at the current system ip_vs Module version information .

5、 ... and 、LVS Load scheduling algorithm of

1） polling (Round Robin)

• The received access requests are assigned to each node in the cluster in turn ( Real server ), Treat each server equally , Regardless of the actual number of connections to the server and the system negative

2） Weighted polling (Weighted Round Robin)

• Distribute the request according to the weight value set by the scheduler , The node with high weight gets the task first , The more requests are allocated
• Ensure that the server with strong performance takes on more access traffic

3） The minimum connection (Least Connections)

• Allocate according to the number of connections established by the real server , Assign the received access request to the node with the least number of connections first

4） Weighted least connected (Weighted Least Connections)

• When the performance of server nodes varies greatly , You can automatically adjust weights for real servers
• The nodes with higher performance will bear a larger proportion of the active connection load
• Source address hash value （source hash）
  • dh: Destination address hash scheduling (destination hashing) Find a static for the keyword with the destination address hash Table to get what you need RS.
  • Source address hash scheduling (source hashing) Find a static keyword with the source address hash To get what you need RS.

Dynamic scheduling algorithm : wlc, lc, lblc

wlc: Weighted minimum connection number scheduling , Assuming that the machine RS The weight of Wi, At present tcp The number of connections is Ti, In order to Ti/Wi For the smallest RS As the next – One assigned RS.

lc: Minimum connection number scheduling (least-connection),IPVS The table stores all the active connections .LB The connection request is compared to the one with the fewest connections RS.

lblc: Minimum number of connections based on address scheduling (locality-based

least-connection): Assign requests from the same destination address to the same station RS, This server is not yet fully loaded . Otherwise, assign the request to the number of connections
The smallest RS, And use it as the first consideration for the next assignment .

6、 ... and 、 Use ipvsadm Tools

To use LVS The ability of , Just install one LVS Management tools ：ipvsadm

ipvsadm Tool options description :

Options	function
-A	Add a virtual server
-D	Delete the entire virtual server
-s	Specify the load scheduling algorithm ( polling : rr、 Weighted polling : wrr、 The minimum connection : lc、 Weighted least connected : wlc)
-a	Represents adding a real server ( Node server )
-d	Delete a node
-t	Appoint VIP Address and TCP port
-r	Appoint RIP Address and TCP port
-m	Said the use of NAT Cluster mode
-g	Said the use of DR Pattern
-i	Said the use of TUN Pattern
-W	Set weight （ The weight of 0 Indicates that the node is suspended ）
-p 60	Keep a long connection 60 second
-l	List view LVS Virtual server （ The default is to view all ）
-n	Show the address in digital form 、 Port and other information , Often with “-l” Option combination .ipvsadm -ln

7、 ... and 、 master LVS-NAT Deployment of patterns

LVS Scheduler as Web Gateway to server pool ,LVS With two network cards , Connect the intranet and Intranet separately , Use polling （rr） Scheduling algorithm

Environment configuration

host	operating system	IP Address	Services needed
Load scheduler	CentOS7	Intranet gateway :192.168.100.10 External gateway :12.0.0.1	ipvsadm
NFS The server 1	CentOS7	192.168.100.11	rpcbind、nfs-utils
Web Node server 1	CentOS7	192.168.100.12	rpcbind、nfs-utils、httpd
Web Node server 2	CentOS7	192.168.100.13	rpcbind、nfs-utils、httpd
client	Windows10	12.0.0.10

The gateway of server and client should be set up , The address is the internal and external gateway address of the load scheduler .（ Gateway is necessary for cross segment access

7.1、 Deploy shared storage

NFS The server ：192.168.100.11

systemctl stop firewalld.service
systemctl disable firewalld.service
setenforce 0

yum -y install nfs-utils rpcbind

systemctl start rpcbind.service
systemctl start nfs.service

systemctl enable nfs.service
systemctl enable rpcbind.service

mkdir /opt/test1
mkdir /opt/test2

chmod 777 /opt/test1
chmod 777 /opt/test2

vim /etc/exports
/opt/test1 192.168.100.0/24(rw,sync)
/opt/test2 192.168.100.0/24(rw,sync)

exportfs -rv

7.2、 Configure the node server

Web Node server 1:192.168.100.12
Web Node server 2:192.168.100.13
Both need installation services , The steps are the same

systemctl stop firewalld.service
systemctl disable firewalld.service
setenforce 0

yum install httpd -y
systemctl start httpd.service
systemctl enable httpd.service

yum -y install nfs-utils rpcbind

systemctl start rpcbind
systemctl enable rpcbind

showmount -e 192.168.100.11

Web Node server 1：192.168.100.12

mount.nfs 192.168.100.11:/opt/test1 /var/www/html
echo 'this is test1 web!' > /var/www/html/index.html

Web Node server 2：192.168.100.13

mount.nfs 192.168.100.11:/opt/test2 /var/www/html
echo 'this is test2 Web!' > /var/www/html/index.html

7.3、 Configure the load scheduler

Load scheduler ： Intranet gateway ens33：192.168.100.10, External gateway ens37：12.0.0.1

systemctl stop firewalld.service
systemctl disable firewalld.service
setenforce 0

1)、 To configure SNAT Forwarding rules

vim /etc/sysctl.conf
net.ipv4.ip_forward = 1
 or 
echo '1' > /proc/sys/net/ipv4/ip_forward
sysctl -p

iptables -t nat -F
iptables -F
iptables -t nat -A POSTROUTING -s 192.168.100.0/24 -o ens37 -j SNAT --to-source 12.0.0.1

2)、 load LVS The kernel module

# load  ip_vs modular 
modprobe ip_vs
# see  ip_vs Version information 
cat /proc/net/ip_vs

3)、 install ipvsadm Management tools

yum -y install ipvsadm

# Save the load distribution policy before starting the service 
ipvsadm-save > /etc/sysconfig/ipvsadm
 or 
ipvsadm --save > /etc/sysconfig/ipvsadm

systemctl start ipvsadm.service

4)、 Set load distribution strategy （NAT Mode is configured on the server , Node servers don't need special configuration ）

# Remove the old strategy 
ipvsadm -C

ipvsadm -A -t 12.0.0.1:80 -s rr
ipvsadm -a -t 12.0.0.1:80 -r 192.168.163.12:80 -m
ipvsadm -a -t 12.0.0.1:80 -r 192.168.163.13:80 -m
# Enable policy 
ipvsadm

# View node status ,Masq representative  NAT Pattern 
ipvsadm -ln
# Save strategy 
ipvsadm-save > /etc/sysconfig/ipvsadm

7.4、 test

At one IP by 12.0.0.10 Your client uses a browser to access http://12.0.0.1/ , Constantly refresh the browser to test the effect of load balancing , The refresh interval needs to be longer .