Microservice stability guarantee

Microservice transformation , The biggest challenge is the stability guarantee after the split , The link is complex after splitting 、 Many fault points , A systematic stability guarantee mechanism is required .

1. The goal of stability guarantee

Microservice stability assurance needs to be carried out before 、 Consider all aspects during and after the event . Microservices architecture , Applications 、 dependent service 、 The Internet 、 Hardware, etc. may fail , The specific objectives of stability design and support are as follows .

Failure prevention , Minimize the occurrence of faults , The vast majority of stability problems and stability failures have certain incentives , And generally, the fault will occur only when a variety of interception means fail , If we develop complete stability guarantee measures before the failure occurs , It can minimize the occurrence of stability failure .

Fast fault location , There is no business without failure , The key is to be able to quickly find the fault when it breaks down , Only in time , In order to take corresponding solutions in the shortest time .

Fault quick stop , Business stop loss shall be carried out immediately after failure , Restore the normal operation of the business , The deep-seated specific causes of the fault can be analyzed and solved after the event .

2. Stability guaranteed 6 Dimensions

There are many system failure points , Stability guarantee is the process of managing failure points . The whole stability design and guarantee can be divided into the following isolation from the perspective of fault point management 、 redundancy 、 Disaster tolerance 、 Change management 、 Time related fault management is friendly to operation and maintenance 6 Dimensions .

• Isolation

The first principle of stability design is “ Isolation ”, Through various isolation mechanisms , Isolate the failure point before the core service , Ensure the availability of core services .

The guiding principle of the isolation mechanism is to change and not change 、 Distinguish between important and unimportant , Change is the main source of stability failure , Separate the changeable parts from the core services and core processes , Reduce core changes , It can guarantee the stability of the core system . A major means of isolation mechanism is decoupling , Core services and non core services can be separated by decoupling , When core services access non core services , By fusing 、 Timeout and retry mechanisms , Ensure that the non core service failure will not affect the overall stability to the greatest extent .

• redundancy

Through service levels 、 Machine level 、 Cluster level 、 Redundancy of multiple dimensions such as machine room level , We can guarantee that ： Even if the core services go wrong , You can also use the corresponding traffic switching strategy , Switch traffic to redundant nodes , Ensure that the business is not affected .

In order to avoid redundancy failure at the same time , Redundant replicas need to be independent of each other , Completely equivalent , Can't depend on each other , The replicas in the computer room are deployed across switches （ At this time, it is generally possible to cross cabinets ）, If there is redundancy in multiple machine rooms , Each machine room is independent , There can be no identical dependencies .

• Disaster tolerance

The third principle of stability design is “ Disaster tolerance ”, By building a multi-dimensional disaster tolerance system , Ensure that the system faces abnormal input , It can still improve the stable output capacity .

Services can be degraded and throttled , Reduce the impact of sudden large traffic on the system , Ensure the stable output of the system , To ensure the immediacy of degradation and current limiting operation , The system needs to support dynamic modification and validation of configuration .

• Change management

Most stability failures are caused by changes , If the system has not changed for a long time , There are few stability problems , Therefore, the key to ensuring service stability is to strictly control change , Ensure change quality .

For changes , Complete change specifications need to be developed , Changes shall be made in strict accordance with the specifications , No matter how small a change is, it may cause stability problems , Therefore, we must strengthen the awareness of stability when changing , Each step of the change operation shall be subject to various monitoring items , If there is a problem, roll back immediately .

• Time related fault management

When the service is not changed , There is a kind of fault that seldom occurs and is difficult to find , It changes with time ID Out of bounds and overflow , This kind of fault is difficult to find during normal test , And when it happens, it will have a great impact on the whole system .

• Friendly operation and maintenance

In order to realize the operation and maintenance friendly system design , The system needs to monitor all relevant information involved in fault analysis and location , Build a perfect monitoring closed loop , For the system layer 、 Service layer 、 The interface layer 、 The business layer and other dimensions conduct monitoring, collection and alarm . In order to reduce the hidden trouble of system stability , Try to follow simple design principles in microservice architecture design , Starting from the real needs of the business , Avoid tall technical solutions that are purely technical , If not the core function of the business , If necessary, some compromises and cuts can be made , Try to keep the system simple and concise .