Practice of implementing cloud native Devops based on rainbow library app

Library App be based on Rainbond Implement cloud native DevOps Practice

What we need is not mastery Kubernetes The engineer , We need a management tool that Xiaobai can use well .

​ —— Person in charge of operation and maintenance of Xiamen Zhengguan Yizhi Technology Co., Ltd Guo Chuanhao

Hello everyone , I'm Guo Chuanhao, the operation and Maintenance Director of Xiamen Zhengguan Yizhi Technology Co., Ltd .

The library is a cloud private library that focuses on users' self-growth , Reading of e-books 、 commend 、 borrow 、 purchase 、 Storage and knowledge management functions in one , Committed to the cognitive empowerment of users , Through the cultivation of reading habits , Achieve self growth . At present, the total number of registered users has reached 1500W , Platform book resources exceed 200W book .

We use Rainbond There has been a 2 year , I'll share our experience with you .

1. Previous difficulties

Libraries in the cloud age have a high starting point , We chose the microservice architecture from the beginning 、Kubernetes、 Containerization and other technical systems in line with the trend of the times . But the original kubernetes The functions provided by the management platform do not fully meet our expectations , We can't afford the huge technical cost of secondary development .

In learning Rainbond Before , In the early stage of entrepreneurship, we have been trapped in the trivia caused by frequent product iteration and change . The trivia I'm talking about includes... Under the microservice architecture 50 Version management of a service 、 Replacement of construction products 、 The release of online environment and all kinds of cumbersome work around the application from development to online and then to operation and maintenance .

We hope to get out of these trivial things , Focus on the business itself , Explore a path suitable for kubernetes Continuous development in the environment 、 An easy path to continuous delivery .

In view of this , We are actively looking for an open source and easy-to-use application management platform .

2. First time to know Rainbond

stay Rainbond Before , We've tried Rancher Products such as , But there is a big gap between the product function and our expectation .

2 Years ago , We go through Github I learned for the first time Rainbond This product , The surprise is that the function is very suitable for the needs of the library . List a few impressive abilities ：

• The overall topology of the application architecture

From God's point of view , The running state and dependencies of all service components can be observed at a glance . Obsessive compulsive disorder will force our engineers to eliminate the red abnormal state , And the green that reflects the running state is really reassuring .

• Visual resource usage

Resource occupancy is an important indicator of observability . For startups , Know whether the resource allocation is reasonable , It is very important to put an end to the waste of resources .Rainbond Every dimension from team to service component provides good observability . Team level resource quota capability is very practical , It solves the problem that the operation and maintenance team cannot effectively control the resource allocation .

• Automatic telescopic

The library is a typical 2C scene , How to make good use of the flexibility provided by Cloud Computing , Flexible response to peak traffic ？ The answer is to use auto scaling .Rainbond Automatic scaling function provided , It highlights the characteristics of simple and easy configuration . The automatic scaling capability greatly liberates the working pressure of the operation and maintenance team , From then on, we will stay away from mobilizing the masses and preparing for the battle . Key business will follow our heart , Auto expansion , Until it can handle all traffic .

• Centralized gateway policy management

2C Service publishing under scenario , I can't get around it anyway . Whether it's blue or green 、 Grayscale publishing or A/B test , The 18 martial arts related to this service release will encounter more or less . Native Kubernetes Ingress These publishing strategies can indeed be implemented , However, we prefer to get a product centralized management page to deal with these problems , Instead of bothering the O & M engineers to touch those strict formats Yaml file .Rainbond Gateway policy management perfectly meets our requirements .

• Apply replication to quickly build the environment

In the development process of the library , We often need to build various environments , To distinguish development 、 test 、 Pre release scenario , Corresponding to different versions of microservice components , such as Dev、Prod And so on. . But if you have to manually create service components every time you build the environment , That's really inefficient . Application replication is very useful in this scenario , It helps me quickly copy a set of environment . Select the version of the build source during the copy process , It's mirrored to me Tag. New environment after replication , All service component meta information and dependencies are preserved .

In the process of gradual exploration , Interaction with the official team in the community , Let's avoid many detours . If an open source product is accompanied by a viable community , It will be very reassuring .

3. Development test environment deployment

First step , Deploy microservices

Get started Rainbond , It starts with deploying a single microservice , The process is very simple , There is no need to learn Kubernetes Of Yaml . Components in the development environment are built based on images , Just fill in the image address and related information according to the guide of the interface .

We're going to use Spring Cloud Microservice framework , stay Rainbond The system can run well , I was influenced by a series of documents in the deployment process , Here to share :

• Spring Cloud Microservices are deployed in Rainbond The advantages of
• Spring Cloud Micro service and Service Mesh Fusion
• Spring Cloud Microservices are deployed in Rainbond The case of

The second step , Service orchestration through visualization

In the process of choreographing microservices , This function is based on the graphical editing of component dependencies , It really surprised me . This arrangement and other platforms are based on complex Yaml The way documents are organized is very different . Interested friends can read Service Orchestration Related description , This is indeed a way of micro service arrangement that Xiaobai can control .

The third step , Connect with external services

For a start-up like us , Hosting services such as databases to cloud service providers , Use it directly RDS Service is an economical and sound choice . stay Rainbond In the system , I added third-party components to the cloud RDS Services are included in Management . This inclusion allows third-party services to be deployed on the platform , Can be relied on by other microservice components .

thus , My development environment has been deployed .

Step four , Copied the test environment 、 Pre release environment and production environment

In the past development process , Building an environment is a very cumbersome thing . For a product in the process of rapid iteration , We need at least a development environment 、 Test environment 、 Production environment , In our own actual scene , A pre release environment is also introduced . For code , I just need Fork Multiple branches , To distinguish between different environments . By defining the pipeline , We have also completed different packaging images of different code branches tag. But when it comes to setting up the environment , Can it only be based on different images tag , Manually create components a little bit ？ Think of the near future of the library business 50 Microservice components , And dependencies on each other , My head is big ,IT What practitioners can't stand most is repetitive work .

exploring Rainbond In the process of using the method , The function of quick copy caught my eye . The quick copy function can check out the construction source information of all components , For components built from source code , The build source is its code warehouse address 、 programing language 、 Code branch ; And for components built with images , The build source corresponds to the image address and tag. In such an interface , I can select the components that need to be copied , Modify it tag edition . Such replication capability can be implemented in different clusters 、 Replication under different teams . The new environment inherits the image from the original environment tag All settings except ： Dependency relationship 、 Component name 、 Environment variable configuration 、 Persistence settings, etc .

Take advantage of this ability , I'm based on the development environment , image Fork Like a code , By modifying the tag The test environment is copied in a way 、 Pre release environment and production environment .

This ability greatly saves us from using Rainbond when , The time cost of deploying various environments . at present , We also use this function to build a new development environment , In the past, teaching newcomers how to build their own development environment was very laborious .

4. Collude with continuous delivery process

earlier , We have used Jenkins , Customized a complete set of pipeline , To implement the construction of all microservice components . The final build product will be customized as an image and pushed to the image warehouse . We are quite satisfied with this part of the work , We hope Rainbond It can be integrated after the mirror warehouse , Complete the continuous construction and deployment of microservice components .

Rainbond It's very open in this part , Provides an interface to implement communication with third parties CI Docking of tools . All we need to do is Jenkins After the pipeline completes the image push , Add a step to simply call the next Rainbond Provided interface , The corresponding microservice component will automatically pull the latest image , Complete the online operation . up to now , The entire technical team has adapted to this use .

In fact, this is a very common interface calling method , No matter what kind of third party CI Tools , Can be very convenient to call .

Each one runs in Rainbond Microservice component on , Automatic build settings can be turned on at the build source . There are two implementations of automatic build settings ：

• be based on Git-Webhook , For microservice components built based on source code , With the help of code warehouse Webhook Ability , Implement code push , This triggers the service component to automatically build and go online . There are a wide range of supported code warehouse types ,GItlab、Github、Gitee、Gitea And other code warehouses support .
• Based on image warehouse Webhook , For microservice components built based on image , With the help of mirror warehouse Webhook Ability , Realize image push , This triggers the service component to automatically build and go online .Harbor、Dockerhub And other image warehouses support Webhook function .
• Customize API, This is the most common way to call interfaces , Users only need to be based on Http Protocol call , You can trigger the automatic construction of the service component and go online .

Trigger the auto build... In the figure above API, The simplest way is to execute a command on the command line ：

curl -d '{"secret_key":"6GvowlHX"}' -H "Content-type: application/json" -X POST https://<Rainbond Console address >/console/custom/deploy/c4e7b60bd800986df940d8103f22d271

at present , We can already do it in a very simple way , Trigger exactly to the specified pipeline , Complete the update of corresponding microservice components .

5. The others are through Rainbond Problem solved

As for Rainbond The understanding of this product continues to deepen , We began to throw away some trivial things , Some problems that are difficult to avoid under the traditional deployment mode , With the help of Rainbond The ability to solve ：

• Internal dependency configuration cannot be queried

Under the traditional deployment mode , Dependencies between all components , Are a series of configurations written in the configuration file . It is difficult for engineers who do not know enough about the whole product to master the reference address of all dependencies , Wrong writing IP Errors that lead to impassable calls occur from time to time . With the help of the presentation of application Topology , Now every novice Engineer , After seeing the topology diagram , Will immediately have an intuitive understanding of the overall structure of the product .

• Troubleshooting after multi instance deployment exceptions is inconvenient

Under the traditional deployment mode , If multi instance deployment is required for each microservice component , Engineers need to manually operate the server to upload jar Package configuration . In case of upgrade adjustment , Sometimes mistakes and omissions . Once a problem occurs, it needs to be checked , How to locate the correct instance is already very troublesome .Rainbond In the environment , The multi instance version consistency of each microservice component does not need to be concerned , When troubleshooting problems , Real time log push and Web The console helped a lot .

• Server resources cannot be shared

Under the traditional deployment mode , We divide virtual machines to avoid the waste of computing resources , But that's not enough . We want computing resources to be fully pooled , For each microservice component . This is all based on Kubernetes The implemented application management platform can be realized . and Rainbond Scaling settings for , It's the best product I've ever seen , The easiest to use .Rainbond After the platform went online, the server resources were reduced by two-thirds .

• The same listening port cannot be deployed on the same platform

Port is also an important resource , Under the same operating system , The occupation of ports cannot conflict . This problem is particularly prominent when deploying large-scale micro service components .Rainbond It's done very well , Each microservice does not directly occupy the server port , Our developers have more freedom to define component listening .

• Developer rights management

In a real business scenario , The problems of the software system itself are not handled by the operation and maintenance personnel , In more cases, it needs to be checked and handled by the developer . However, permission management requires that developers do not have the permission to log in to the production server as much as possible . This leads to a dilemma , Quickly solve problems or strictly control permissions ？ This is the point where developers and operation and maintenance personnel are prone to conflict . introduce Rainbond after , This problem has been well solved , All the actions of developers are in Rainbond Management interface , Even if you need to troubleshoot the problem through the command line , It can also be done through Web Console login container environment , Not the host server . at present , We have formed an application developer based on Rainbond O & M develops its own mode of microservice components .

• Apply version rollback

In the traditional mode , How complex is the deployment of microservice components , Then rolling back to the previous version will only be more complex .Rainbond This product is very considerate and provides one click rollback at the service component level , Managers can select the required version from the version list at will , One click rollback , It's really convenient .

6. At the end

Just like other products , Deep use Rainbond It also needs some running in process . A situation that impressed me , yes Rainbond The use of Glusterfs distributed file system , After a long period of use , Storage capacity exhausted , It leads to a series of problems . Our O & M personnel are lack of understanding of Glusterfs Understanding , Yes Rainbond How to communicate with Glusterfs Interaction is a little understood . In desperation, he turned to the authorities , Unexpectedly, the official engineers were very enthusiastic and supported us to solve the problem , And left the operation documents attentively .

I am right. Rainbond The biggest feeling is that its ease of use is very good . hope Rainbond The team can carry this through , Provide more practical features that can solve practical problems . We learned that Rainbond Of Service Mesh The next step is to support istio, In the next stage, we are going to try .