Load balancing strategy graphic explanation

One 、「 Load balancing 」 What is it?

As the picture above shows , The flow is converged by an independent unified inlet , The process of secondary distribution is 「 Load balancing 」, Its essence and 「 Distributed systems 」 equally , yes 「 Divide and conquer 」.

If you are used to using some navigation software while driving , We will find that , There is an upper limit on the number of recommended route schemes of the navigation software , such as 3 strip 、5 strip . therefore , In fact, it plays a similar role in essence 「 Load balancing 」 The role of , Because if you can only take Top3 The unobstructed route , The route with serious natural congestion cannot be recommended to you , So that the pressure of traffic flow is distributed to the relatively idle route .

The same is true in software systems , To avoid uneven flow sharing , Cause local nodes to be overloaded （ Such as CPU Tight, etc ）, Therefore, an independent unified portal is introduced to do the above “ Navigation ” The job of . however , In the software system 「 Load balancing 」 The difference from navigation is , Navigation is a flexible strategy , Ultimately, the user needs to make a choice , The former is different .

Behind the balance is the strategy , Behind the strategy is some algorithm or logic . such as , The algorithm in navigation belongs to 「 Path planning 」 Category , In this category, it is subdivided into 「 Static path planning 」 and 「 Dynamic path planning 」, also , In different branches, there are also various algorithms for specific calculations , Such as Dijikstra、A* etc. . alike , Load balancing in software systems , There are also many algorithms or logic supporting these strategies , Coincidentally, there are also static and dynamic .

Two 、 Commonly used 「 Load balancing 」 Strategy diagram

Here is a list of the most common things in daily work 5 Strategies .

1、 polling

This is the most common and simplest strategy , Average distribution , Everyone has 、 One person at a time . The general code is as follows .

2、 Weighted polling

On the basis of polling , Added a concept of weight . Weight is a generalized concept , It can be embodied in any way , In essence, it is a thought that those who can do more work . such as , Different weights can be configured according to the performance difference of the host . The general code is as follows :

The process of this code is shown in the table below ."()" The number in is a self increasing number , In code cur_weight.

It is worth noting that , Weighted polling itself can be implemented in different ways , Although the final proportion is 2：1：2. But the order in which requests are sent can be all different . such as 「5-4,3,2-1」 Compared with the above case , The final ratio is the same , But the effect is different .「5-4,3,2-1」 More likely to cause concurrency problems , Cause server congestion , And this problem becomes more and more serious as the weight number is larger . Example ：10：5：3 The result is 「18-17-16-15-14-13-12-11-10-9,8-7-6-5-4,3-2-1」

Here I would like to recommend one Java Learning exchange group . Exchange learning group number ：874811168 It will share some videos recorded by senior architects ： Yes Spring,MyBatis,Netty Source code analysis , High concurrency 、 High performance 、 Distributed 、 Principles of microservice architecture ,JVM performance optimization 、 Distributed architecture and other necessary knowledge systems for Architects . You can also get free learning resources , Learning together , Progress together , Now it's been a lot of good .

3、 Minimum number of connections

This is based on real-time load conditions , Dynamic load balancing . Maintain the number of connections in the activity , Then take the smallest return . The general code is as follows .

4、 The fastest response

This is also a dynamic load balancing strategy , Its essence is to allocate according to the response of each node in the past period of time , The faster the response, the more allocated . There are many specific ways of operation , The above picture can be understood as , Record the average time spent on requests in the most recent period , Combine the preceding 「 Weighted polling 」 To deal with it , So it's equivalent to 2：1：3 Weighted polling for .

Digression ： Generally speaking , The delay in the same computer room is almost the same , The difference in response time mainly lies in the processing capacity of the service . If across regions （ example ： Zhejiang -> Shanghai , Or Zhejiang -> Beijing ） Is used in some request processing , Timing is used in most cases 「ping」 To get the delay condition , the reason being that OSI Of L3 forward , The data is cleaner , More accurate .

5、Hash Law

hash The load balancing of the method differs from the previous methods in that , The result is determined by the client . Through a standardized hash function, a certain identifier brought by the client is scattered and allocated .

The hash function in the figure above is the simplest and most crude 「 Surplus method 」.

Digression ： Hash function except for remainder , And things like 「 Variable base 」、「 Fold 」、「 Square with the middle method 」 wait , No expansion here , Interested partners can consult the information by themselves .

in addition , The parameters of the remainder can be arbitrary , As long as it is finally converted into an integer to participate in the operation . The most common is to use the source ip Address as a parameter , This ensures that the same client requests fall on the same server as much as possible .

3、 ... and 、 Commonly used 「 Load balancing 」 Advantages and disadvantages of the strategy and applicable scenarios

We know , There is no perfect thing , The same goes for load balancing strategies . The most common strategies listed above also have their own advantages, disadvantages and applicable scenarios , I did a little sorting , as follows .

These load balancing algorithms are commonly used because they are simple , Want better results , Higher complexity is necessary . such as , Simple strategies can be combined 、 Or comprehensive evaluation through more dimensional data sampling 、 Even based on the prediction algorithm after data mining .

Four 、 use 「 Health detection 」 To ensure high availability

Whatever the strategy , It is inevitable to encounter machine failure or program failure . So make sure that load balancing works better , Also need to combine some 「 Health detection 」 Mechanism . Regularly detect whether the server can still be connected , Is the response slower than expected . If the node belongs to “ Unavailable ” The state of the word , You need to temporarily remove this node from the list to be selected , To improve usability . Commonly used 「 Health detection 」 There are ways 3 Kind of .

1、HTTP Probe

Use Get/Post Request a fixed... Of the server URL, Judge whether the returned content meets the expectation . In general use Http Status code 、response To judge .

2、TCP Probe

be based on Tcp Three times handshake system to detect the specified IP + port . Best practices can be learned from Alibaba cloud SLB Mechanism , Here's the picture .

It is worth noting that , In order to release the connection as soon as possible , Follow up immediately after three handshakes RST To interrupt TCP Connect .

3、UDP Probe

It may be used by some applications UDP agreement . Under this protocol, the specified message can be detected through the message IP + port . Alibaba cloud's best practices can also be used for reference SLB Mechanism , Here's the picture .

The way to judge the result is ： When the server does not return any information , Default normal state . Otherwise, it will return a ICMP Error message .

5、 ... and 、 Conclusion

In one sentence, the essence of load balancing is ：

Will request or traffic , Allocate the desired rules to multiple operation units for execution .

Through it, horizontal expansion can be realized （scale out）, Make redundancy work as 「 High availability 」. in addition , It can also make the best use of everything , Improve resource utilization .