Introduction and principle analysis of cluster and LVS

Catalog

One 、 Cluster and LVS Introduce

1、 Introduction of the cluster

1. Cluster description

2. The purpose of the cluster

3. Cluster classification

2、LVS summary

1. LVS Introduce

2. LVS The cluster consists of

3. LVS The term

4. LVS Working mode

5. Load balancing scheduling algorithm （10 Kind of ）

Two 、LVS-NAT Deployment and test validation

1. Install the software

2. LVS-NAT Cluster topology

3. Operation process

4. Deploy LVS Scheduler

5. To configure web The server

6. Client side test

3、 ... and 、LVS-DR colony

1、LVS-DR Introduction and principle

1. LVS-DR Cluster topology

2. Operation process

3. ARP The problem of broadcasting

4. Kernel parameter description

2、LVS-DR Deploy

1. Configure the experimental network environment

2. To configure web1（web2 Same as web1 Do the same thing ） The server

3. proxy The scheduler installs the software and deploys it LVS-DR Pattern

4. Client side test （ Can no longer proxy Host verification ）

---

One 、 Cluster and LVS Introduce

1、 Introduction of the cluster

1. Cluster description

Centralize many servers through high-speed network     - Providing the same service , On the client side, it's like there's only one server

Can be achieved at a lower cost in the case of performance 、 reliability 、 Relatively high benefits in terms of flexibility

Task scheduling It is in the cluster system The core technology

2. The purpose of the cluster

① Improve performance ：         - For example, computing intensive applications , Like the weather forecast 、 Nuclear test simulation, etc

② cost reduction         - Compared to a million dollar supercomputer , Cheapness

③ Improve scalability         - Just add cluster nodes

④ Enhance reliability         - Multiple nodes complete the same function , Avoid a single point of failure

3. Cluster classification

① High performance computing cluster HPC（High Performance Computing）         - Through parallel applications developed in clusters , Solve complex scientific problems

② Load balancing cluster LBC（load balancing Cluster）         - The client load is in the computer cluster as much as possible Average share

③ High availability cluster HAC（High Availability Cluster）         - Avoid single point of failure , When a system fails , It can migrate quickly

2、LVS summary

1. LVS Introduce

① Linux Virtual server (LVS) It was founded by zhangwensong during his doctoral study at the National University of defense technology
② LVS Can achieve high availability 、 scalable Web、Mail、Cache and Media Wait for the Internet service
③ The ultimate goal is to use Linux Operating system and LVS Cluster software to achieve a high availability 、 High performance 、 Low cost server application cluster

2. LVS The cluster consists of

① front end ： Load balancing layer
- It consists of one or more load schedulers
② middle ： Server group layer
- It consists of a group of servers that actually apply services
③ Bottom ： Data sharing storage layer
- A storage area that provides shared storage space

3. LVS The term

Directory Server： Scheduling server
- Distribute the load to Real  Server Server for
Real  Server：         Real servers
- Servers that really provide application services
VIP： fictitious IP Address
- A virtual device published to users for access IP Address
DIP： The scheduler connects to the backend node server IP Address
RIP： real IP Address
- Used on cluster nodes IP Address

4. LVS Working mode

 

① VS/NAT：        - Virtual server through network address translation
- Large concurrent access , The performance of the scheduler becomes a bottleneck
② VS/DR：          - Use routing technology to realize virtual server directly
- The node server needs to be configured VIP（ Be careful ：MAC Address broadcast ）
③ VS/TUN：        - Realize the virtual server through the tunnel

5. Load balancing scheduling algorithm （10 Kind of ）

① polling （Round Robin）                                        Commonly used   rr
② Weighted polling （Weighted Round Robin）                 Commonly used wrr
③ The minimum connection （Least Connections）                         Commonly used lc
④ Weighted least connected （Weighted Least Connections） Commonly used wlc
⑤ Source address hash （Source Hashing）
⑥ Destination address hash （Destination Hashing）
⑦ Minimal linking based on locality
⑧ Locally based links with replication are minimal
⑨ The shortest expected delay
⑩ Minimum queue scheduling

Two 、LVS-NAT Deployment and test validation

1. Install the software

① install ipvsadm

yum -y install ipvsadm    # install LVS Policy management tools 

② ipvsadm usage

Create a virtual server
-A	Add a virtual server
-t	Set the cluster address （VIP）
-s	Specify the load scheduling algorithm [ rr | wrr | lc | wlc | sh ]
Case study ：ipvsadm  -A  -t  VIP Address : port   -s  rr         # Specifies that the algorithm is polling
add to 、 Delete 、 Modify the server node
-D	Delete virtual server
-E	Modify the virtual server
-C	Clean all
-a	Add real servers
-e	Modify the real server
-d	Delete the real server
-r	Specify the real server （Real Server） The address of
-m	Use NAT Pattern ;（-g、-i They correspond to each other DR、TUN Pattern ）
-w	Set weights for node servers , The default is 1
-L	see LVS Rule table
ipvsadm  -a  -t  VIP Address : Port number   -r  RIP Address : Port number   -m        # Add server nodes ipvsadm  -d  -r  RIP Address : Port number   -t  VIP Address : Port number               # Delete server node ipvsadm  -Ln        # see IPVS

2. LVS-NAT Cluster topology

3. Operation process

Real   Server：
	- To configure WEB The server
Directory   Server：
	- stay proxy Install and enable ipvsadm
	- Create a virtual server
	- Add nodes to the virtual server
Client：
	Connect virtual server test

4. Deploy LVS Scheduler

[[email protected] ~]# yum -y install ipvsadm    # install LVS Policy management tools 
[[email protected] ~]# echo "net.ipv4.ip_forward=1" >> /etc/sysctl.conf  # Turn on the routing function 
[[email protected] ~]# sysctl -p     #-p Load system parameters from the specified file , The default from the /etc/sysctl.conf Load in 
[[email protected] ~]# ipvsadm -A -t 192.168.4.5:80 -s wrr  # Create add virtual server cluster 、tcp agreement 、 Weighted polling 
[[email protected] ~]# ipvsadm -a -t 192.168.4.5:80 -r 192.168.5.11 -w 1 -m # Add real servers , Weight arbitrary 
[[email protected] ~]# ipvsadm -a -t 192.168.4.5:80 -r 192.168.5.12 -w 1 -m

5. To configure web The server

 To configure WEB1 The server 
[[email protected] ~]# yum -y install httpd        # Install the software 
[[email protected] ~]# echo "192.168.5.11" > /var/www/html/index.html    # Create web file 
[[email protected] ~]# systemctl enable --now httpd    # start-up httpd
[[email protected] ~]# curl http://192.168.5.11    # verification web1 Whether it can be accessed normally 
192.168.5.11
[[email protected] ~]# nmcli connection modify ens33 ipv4.method manual ipv4.gateway 192.168.4.5
# configure gateway ;（ Be careful ： The network card name cannot be copied , Fill in according to the local network card name ）
[[email protected] ~]# nmcli connection up ens33 # Start the network card 
[[email protected] ~]# ip route show    # View default gateway 
default via 192.168.5.5 dev ens33 proto static metric 100   #default The latter is the default gateway 

 To configure WEB2 The server 
[[email protected] ~]# yum -y install httpd        # Install the software 
[[email protected] ~]# echo "192.168.5.12" > /var/www/html/index.html    # Create web file 
[[email protected] ~]# systemctl enable --now httpd    # start-up httpd
[[email protected] ~]# curl http://192.168.5.12    # verification web1 Whether it can be accessed normally 
192.168.5.12
[[email protected] ~]# nmcli connection modify ens33 ipv4.method manual ipv4.gateway 192.168.4.5
# configure gateway ;（ Be careful ： The network card name cannot be copied , Fill in according to the local network card name ）
[[email protected] ~]# nmcli connection up ens33 # Start the network card 
[[email protected] ~]# ip route show    # View default gateway 
default via 192.168.5.5 dev ens33 proto static metric 100   #default The latter is the default gateway 

6. Client side test

 test ： Repeated visits 4.5 Will poll different back-end real servers 
[[email protected] ~]# curl 192.168.4.5
192.168.5.12
[[email protected] ~]# curl 192.168.4.5
192.168.5.11
[[email protected] ~]# curl 192.168.4.5
192.168.5.12
[[email protected] ~]# curl 192.168.4.5
192.168.5.11

3、 ... and 、LVS-DR colony

1、LVS-DR Introduction and principle

1. LVS-DR Cluster topology

2. Operation process

Real  Server：
	- To configure WEB The server
	- Configuration assistant IP Address 、 Adjust kernel parameters
Directory  Server：
	- Install and enable the agent ipvsadm
	- Configuration assistant IP Address
	- Create a virtual server 、 Add nodes to the virtual server
Client
	- Connect virtual server test

3. ARP The problem of broadcasting

ARP The purpose of the agreement is to be based on IP Get its corresponding address MAC Address

① When a client initiates an access VIP When requesting the corresponding domain name , According to the principle of network communication, there will be ARP radio broadcast
② Because the load balancer and the real server are on the same network and VIP Set on each node in the cluster
③ At this time, the real server in the cluster will try to answer from the client ARP radio broadcast , This will create problems , everyone They say I am “VIP”

4. Kernel parameter description

arp_ignore（ Define reply ARP Way of broadcasting ）
0（ Default ）	Respond to all local addresses ARP radio broadcast , The local address can be configured on any network interface
1	Only respond to any... Configured on the inbound network card interface IP Address of the ARP radio broadcast
2	In addition to satisfaction 1 In addition to the conditions , And satisfy ARP The sender of the request packet IP The address also belongs to the subnet to which the current network card belongs , Only in this way can we respond ARP Response packets
3	If ARP Requested by the request packet IP The local address corresponding to the address has its scope （scope） For the host （host）, No response ARP Response packets , If the scope is global （global） Or link （link）, Response ARP Response packets
8	Even if ARP Request data requested IP The address belongs to any local address , No response ARP Response packets
arp_announce（ Define host send ARP How to select the sender used in the packet when requesting the packet IP Address ）
0（ Default ）	Use the local port configured on any network card interface IP Address （ Any local address ）; It's usually just to be sent IP The source of the packet IP Address
1	Try to avoid using interfaces that do not belong to the network （ namely Network interface for sending packets ） The local address of the subnet is used as the sender IP Address
2	To query the target Use the most appropriate local address . stay This... Will be ignored in this mode IP The source address of the packet and try to select a local address that can communicate with that address . The first is to select all the subnets of the network interface, and the outbound access subnet contains the target IP The local address of the address . If no suitable address is found , The current sending network interface or other network interface that may receive the message will be selected ARP The network interface of the response to send

2、LVS-DR Deploy

1. Configure the experimental network environment

[[email protected] ~]# cp /etc/sysconfig/network-scripts/ifcfg-eth0 /etc/sysconfig/network-scripts/ifcfg-eth0:0
# The network card name should be filled in according to the actual situation 
[[email protected] ~]# vim /etc/sysconfig/network-scripts/ifcfg-eth0:0
TYPE=Ethernet        # Network card type is ： Ethernet card 
BOOTPROTO=none       #none Manual configuration IP, perhaps dhcp Automatic configuration IP
NAME=eth0:0          # The network card name 
DEVICE=eth0:0        # Equipment name 
ONBOOT=yes           # Whether to automatically activate the network card when starting up 
IPADDR=192.168.5.50  #IP Address 
PREFIX=24            # Subnet mask 
[[email protected] ~]# systemctl restart network  # Restart network service 
[[email protected] ~]# ip a s eth0      # Will see eth0 There's even one IP Address 
2: eth0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast state UP group default qlen 1000
    link/ether 00:0c:29:17:f5:46 brd ff:ff:ff:ff:ff:ff
    inet 192.168.5.5/24 brd 192.168.5.255 scope global noprefixroute eth0
       valid_lft forever preferred_lft forever
    inet 192.168.5.50/24 brd 192.168.5.255 scope global secondary noprefixroute eth0:0
       valid_lft forever preferred_lft forever
    inet6 fe80::da6d:94a0:3f30:531/64 scope link noprefixroute 
       valid_lft forever preferred_lft forever
 Be careful ： If there is a conflict when restarting the network , You can turn it off first NetworkManager Restart after service network Can solve 

2. To configure web1（web2 Same as web1 Do the same thing ） The server

[[email protected] ~]# nmcli connection modify ens33 ipv4.method manual ipv4.gateway ''
# take NAT The experimental environment gateway is cleared 
[[email protected] ~]# nmcli connection up ens33 # Activate ens33 network card 
[[email protected] ~]# cp /etc/sysconfig/network-scripts/ifcfg-lo /etc/sysconfig/network-scripts/ifcfg-lo:0
[[email protected] ~]# vim /etc/sysconfig/network-scripts/ifcfg-lo:0
DEVICE=lo:0            # Equipment name 
IPADDR=192.168.5.50    #IP Address 
NETMASK=255.255.255.255# Subnet mask 
NETWORK=192.168.5.50   # network address 
BROADCAST=192.168.5.50 # Broadcast address 
ONBOOT=yes             # Whether the network card is activated after startup 
NAME=lo:0              # The network card name 

Preventing address conflicts ：

Here because web1 It is also configured with The scheduler is the same VIP Address , By default, there is bound to be an address conflict ; modify sysctl.conf Document purpose ： visit 192.168.5.50 Data packets of , Only the scheduler will respond , Other hosts do not respond

[[email protected] ~]# vim /etc/sysctl.conf  # Modify file , The following is added 4 That's ok 
net.ipv4.conf.all.arp_ignore = 1
net.ipv4.conf.lo.arp_ignore = 1
net.ipv4.conf.lo.arp_announce = 2
net.ipv4.conf.all.arp_announce = 2
# When there is arp The radio asked who was 192.168.5.50 when , The machine ignores the ARP radio broadcast , No response （ Prevent inbound conflicts ）
# Don't announce your lo The loopback address is 192.168.5.50（ Prevent outbound conflicts ）
[[email protected] ~]# sysctl -p            # Load system parameters 
net.ipv4.conf.all.arp_ignore = 1
net.ipv4.conf.lo.arp_ignore = 1
net.ipv4.conf.lo.arp_announce = 2
net.ipv4.conf.all.arp_announce = 2
[[email protected] ~]# systemctl restart network
[[email protected] ~]# ip a s lo     # Will see lo There are two network cards IP Address 
1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000
    link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
    inet 127.0.0.1/8 scope host lo
       valid_lft forever preferred_lft forever
    inet 192.168.5.50/32 brd 192.168.5.50 scope global lo:0
       valid_lft forever preferred_lft forever
    inet6 ::1/128 scope host 
       valid_lft forever preferred_lft forever

3. proxy The scheduler installs the software and deploys it LVS-DR Pattern

[[email protected] ~]# ipvsadm -C  # Clear all rules , Because the last experiment set rules 
[[email protected] ~]# ipvsadm -A -t 192.168.5.50:80 -s wrr 
# Add a virtual server cluster 、tcp agreement 、VIP Address and port 、 Algorithm weighted polling 
[[email protected] ~]# ipvsadm -a -t 192.168.5.50:80 -r 192.168.5.11 -g -w 2
[[email protected] ~]# ipvsadm -a -t 192.168.5.50:80 -r 192.168.5.12 -g -w 1
#-a(add) Add back-end real servers to the virtual server cluster IP, Designated to -t 192.168.5.50:80 Add... To this cluster 
#-r(real) Followed by the back-end real server IP And port , No port is written here. The default is 80 port 
#-w(weight) Specify the weight of the server , The greater the weight, the more times you are visited 
#-m Specify that the working mode of the cluster is NAT Pattern , If it is -g For use DR Pattern ,-i representative TUN Pattern 
[[email protected] ~]# ipvsadm -Ln                    # Look at the list of rules 
IP Virtual Server version 1.2.1 (size=4096)
Prot LocalAddress:Port Scheduler Flags
  -> RemoteAddress:Port           Forward Weight ActiveConn InActConn
TCP  192.168.5.50:80 wrr
  -> 192.168.5.11:80              Route   2      0          0         
  -> 192.168.5.12:80              Route   1      0          0         

4. Client side test （ Can no longer proxy Host verification ）

[[email protected] ~]# nmcli connection modify ens33 ipv4.method manual ipv4.address 192.168.5.2/24 ipv4.gateway 192.168.5.254 connection.autoconnect yes
# Configure network card address 
[[email protected] ~]# curl http://192.168.5.50    # The client scans repeatedly , Because it is weighted polling 
192.168.5.12                                 #11 The weight is 2, So it will poll 2 Time 
[[email protected] ~]# curl http://192.168.5.50
192.168.5.11
[[email protected] ~]# curl http://192.168.5.50
192.168.5.11
[[email protected] ~]# curl http://192.168.5.50
192.168.5.12