Evolution of zhenai microservice underlying framework from open source component encapsulation to self-development

edit | Xue Liang

The guest | Peng Wanliang

Owned as a family 17 Dating website with a history of years , Zhenai has experienced explosive growth in business scale , There is a bottleneck in the system , The business has experienced micro service splitting 、 Link optimization 、 On the cloud 、 Containerization 、 Development history of double cloud and double life , There are corresponding thoughts in each technical link , Exposing problems repeatedly 、 After solving the problem , Finally, a set of cherished and characteristic microservice governance architecture is precipitated . The following is treasure net CTO Peng Wanliang's interpretation of the structure .

The following figure is the overall architecture of zhenai's bottom framework , The yellow part is the basic tool class , Some configurations common to the project ; The purple part is the extension plug-in , Using plug-ins is because it involves process control , For example, containerization 、 unitized 、 Publishing process 、 journal （ monitor ）; The blue part is used Springboot The quick start of is convenient for everyone to access the project .

The blue part is the self-developed component of treasure net , For example, graceful shutdown , Delay callback , Database cache synchronization , Report , Asynchronous task , Bytecode enhancement , Scheduling tasks ; The green part is the encapsulation or secondary development of excellent open source components , For the database 、 cache 、 queue 、 Asynchronous and other operations are managed , Greatly reduce the number of problems .

Service framework layer —— Micro service splitting

Architecturally, the independent business is a service , There are many rules for splitting , Need theory + Experience + practice , What you need to pay attention to , Change the business into a tree call , Remove the mesh layer call , In addition, the actual effect needs to be verified , We will Spring Cloud Of RPC Scheme and Dubbo Of RPC The scheme has made a pressure test ,Dubbo Of TPS It's better than Spring Cloud, But it is not as obvious as many online materials , There are differences between the actual effect and the laboratory effect , The core of our test is to enable you to better predict the ability of the system .

Gateway layer construction

In addition to its core routing function , It also expands some capabilities ： be based on Sentinel Fusing and current limiting （ Secondary development ）, United return （ Standardization of input and output parameters , Inclusion structure , Exception return code ）, Retry optimization （Connect It's abnormal ）, Elegant shutdown , Start the warm-up .

Data framework layer

The first thing to do is to split the data layer according to business , We also happen to take advantage of the opportunity of going to the cloud , Data migration is required as a whole , Re plan the data .

second , Remove the database middle tier , To improve performance and stability , introduce Sharding-JDBC（3.1.0, Some of the current versions have been improved ）, We also provide a database and table algorithm on the underlying framework 、ID Generate rules 、 Routing tools and other tools , And we have developed a set of configuration management , It also brings great convenience to our subsequent governance work , First, the configuration of the database , We hope to conduct unified management , All databases are configured under one directory , Leave it to the architecture team to maintain , Project configuration can be the responsibility of the business team （Nacos Realization ）, stay Git There's a project built on , Inside is the database layer configuration of all projects , everyone Git After submitting the configuration on , Published by the architecture group , There will be some configuration checks , Especially in the configuration of fragmentation rules , It is particularly easy to be misconfigured , After the release , We transform the configuration （ Secondary development ）, Use ZK Monitoring enables the application layer and agent layer to update at the same time , Make seamless switching of large watches for us , Brought great convenience , Involves configuration updates .

How to refresh the database link , As shown in the figure ：

There will be holes in it , The database is instantly facing doubled connections , For databases with too many connections , It will directly cause the database to crash , So we need to balance according to the business .

Another common requirement is how to divide a large table and complete it without downtime , The full data adopts Dump The way , Incremental data we pass otter Synchronous data , use ShardingProxy（ The above configuration sub database sub table rules ） Write to the new table , After verifying that the data is consistent , Let's refresh the configuration directly , The application end can switch directly , What happens if there are still connections that have not been switched , The old data will still pass ShardingProxy Write it down （ Now? Sharding-Scaling）, As shown in the figure

In addition, we have some exchanges with the original author , There are some places they don't support , We have also done some secondary development , Some of their subsequent versions have also been adopted , For example, one watch has multiple slice keys （ scene ： Dynamic tables need to be based on dynamic ID Inquire about , It also needs to be based on users ID Inquire about , Avoid full table scanning without partition ）、 How do time slicing keys do inequality routing （ scene ： Log business , Avoid the full table scan of interval query ）, We recognize that the sub table key is time , And there is a dynamic structure , It can also automatically create tables on a regular basis , We found performance problems after debugging .

Service governance

Release framework

be based on Maven Plug in development for , Since the research Zhenai-Dockerx-PlugIn plug-in unit , Let applications access transparently Docker、Kubernetes.

Parameter checking , Build a mirror image , The release and deployment are completed by the plug-in , Development only requires execution MVN Test command , Then the container deployment to the test environment is completed , The efficiency is greatly improved .

Release tests in parallel

Microservices architecture , How to achieve parallel development , We need a set of stable services , Deploy only the micro services involved in the modified Branch , adopt Zone Flow dyeing , The client can be accessed through Header Buried inside Zone The way , It can also be done through Nginx Exposure fixation IP Generate Zone The way , Conduct full link routing of traffic , In addition, it should be noted that the middleware also needs to follow Zone Routing .

Release process transformation

Go back to the whole release process , Developers only need to execute MVN Test, The rest of the links passed Webhook Hook to achieve linkage , After the acceptance personnel release the generation environment , Will make a formal tag, And trigger the hook to update the formal environment to the stable service of the test .

Unit transformation

If a business causes basic service problems , Then it will affect all businesses , Take advantage of the convenience of container deployment , As long as the project depends on unitized components , Add the project name while producing the deployment file , Run an additional set of environment , You can realize unitization , It should be noted that the registration center routes correctly and the configuration center distinguishes configuration items , Gateways also need to be unitized , Later, it can even be extended to the whole business for unitization .

Dual Cloud Architecture

The above content is only a part of the upgrading of zhenai's microservice architecture , I have encountered many pain points in the process of practice , For example, business technology is developing rapidly, but I don't know how to cut into business , We need to precipitate a set of our own underlying framework for microservice governance , Only when the bottom is built , To support the diversified development of technology and business , What kind of technical solution will be used in the future , Just extend the underlying architecture .

For more details , Teacher Peng Wanliang will be in 7 Held in ArchSummit Global architects Summit （ ShenZhen Railway Station ）2022 Elaborate on , I hope to help the audience when facing a huge system problem , Be able to conduct multi-dimensional analysis and continuous improvement , meanwhile , Build an effective self owned structure .

Activity Introduction

This year, 7 month 15 Japan -7 month 16 Japan ,ArchSummit The global architects summit will be held in Shenzhen . Our guest teacher Peng Wanliang will be in 「 Microservice architecture implementation practice 」 Share more topics , This topic also invited Alibaba Alipay / Global Funds / Senior technical expert gaoxianglong （ Nine uncles ） teacher 、 ByteDance senior architect hongzenglin 、 Mr. Liu Feng, deputy general manager of China post consumer finance technology development department, and many other senior architecture experts in the industry shared with you topics related to the implementation of micro service architecture , Click to read the original to learn more .

The conference has entered the countdown , There are limited seats on site , Tickets are about to sell out , For more details, please scan the code or consult ：18514549229（ Wechat is on the same phone ).

I'm looking at one less bug