Managed service network: application architecture evolution in the cloud native Era

author ： Wang Xining

The content of this article is based on the author's 2022 The speech at the cloud native industry conference in .

background

Review the evolution of application service architecture . Judging from the processing methods of service callers and providers , Can be divided into 3 Stages .

The first stage is Centralized load balancing , That is to say, the service caller routes to the corresponding service provider through an external load balancing . The advantages are obvious , No intrusion into the application itself , It can support multi language and multi framework to develop and implement the application itself , Load balancing unified and centralized management , The whole deployment is simple . But the disadvantages are also very significant , Because it is centralized, the scalability is limited , At the same time, the service governance ability of centralized load balancing is relatively weak .

The second stage refers to Distributed governance of microservices , That is, the built-in governance capability of the service caller , With SDK The way of library is integrated into the application . The advantage is that the whole system has good scalability , Strong service management ability , But at the same time, we will notice its disadvantages , Including intrusion into the application itself 、 Because it depends on SDK Therefore, it is difficult to support multiple languages 、 The complexity of distributed management deployment .

The third stage is now Service grid technology . Through the ability to manage these services Sidecar turn , The ability of service governance can be understood and coupled with the application itself , It can better support multiple programming languages 、 At the same time these Sidecar Capabilities do not need to depend on a specific technical framework . these Sidecar Agents form a mesh data plane , Through this data plane, we can process and observe the traffic between all services . Control faces these Sidecar Agent for unified management . But it brings a certain degree of complexity .

The following figure shows the architecture of the service grid . Mentioned earlier , Under the service grid technology , Every application service instance is accompanied by Sidecar agent , Business code is not aware of Sidecar The existence of . This Sidecar The agent is responsible for intercepting the traffic of the application , And provide traffic management 、 Security 、 Three functions can be observed .

In the cloud native application model , An application may contain several services , Each service is composed of several instances , So these hundreds of applications Sidecar The agent forms a data surface , That is, the data plane layer in the figure .

And how to manage these Sidecar agent , This is the problem to be solved in the control plane part of the service grid . The control plane is the brain of the service grid , Responsible for data plane Sidecar The agent issues the configuration , How do the components of the management data plane execute , At the same time, it also provides a unified API, In order to easily manipulate grid management capabilities .

Generally speaking , After enabling the service grid , Developer 、 Operation and maintenance personnel and SRE The team will work in a unified manner 、 Solve the problem of application service management in a declarative way .

Cloud native application infrastructure supported by service grid

Service grid is a basic and core technology used to manage application service communication , It brings security to the calls between application services 、 reliable 、 Fast 、 Apply non aware traffic routing 、 Security 、 Observable ability .

You can see , The cloud native application infrastructure supported by the service grid brings important advantages , It is divided into six aspects .

One of the advantages ： Unified management of heterogeneous services

• Multi language and multi framework interoperability and governance 、 Dual mode architecture integrated with traditional micro service system

• Refined multi protocol flow control 、 Unified management of East-West and North-South flows

• Automatic service discovery of unified heterogeneous computing infrastructure

The second advantage ： End to end observable

• journal 、 Integrated intelligent operation and maintenance system integrating monitoring and tracking

• Intuitive and easy-to-use visual grid topology 、 Health recognition system based on color identification

• Built in best practices 、 Self service grid diagnosis

The third advantage ： Zero trust security

• End to end mTLS encryption 、 Property based access control (ABAC)

• OPA Declarative policy engine 、 Globally unique workload identity （Identity）

• Complete audit history and insight analysis with dashboard

The fourth advantage ： Optimize the combination of software and hardware

• The first one is based on Intel Multi-Buffer Technology upgrading TLS Encryption and decryption service grid platform

• NFD Automatically detect hardware features , Adaptive support such as AVX Instruction set 、QAT Acceleration and other characteristics

• The first batch passed the advanced certification of trusted cloud service grid platform and performance evaluation

The fifth advantage ：SLO Driven application flexibility

• Service level goals (SLO) Strategy

• Automatic elastic scaling of application services based on observable data

• Automatic switching and fault tolerance under multi cluster traffic burst

The sixth advantage ： Out of the box extensions & Ecological compatibility

• Out of the box EnvoyFilter Plug in market 、WebAssembly Plug in lifecycle management

• And Proxyless Unified integration of modes , Support SDK、 kernel eBPF The way

• compatible Istio The ecological system , Support Serverless/Knative, AI Serving/KServe

The following figure shows the service grid ASM The current architecture of the product . As the industry's first fully managed Istio Compatible service grid products ASM, From the beginning, we have maintained the relationship with the community in terms of architecture 、 Consistency of industry trends , The components of the control plane are hosted on the alicloud side , It is independent of the user cluster on the data side .ASM The product is open source based on the community Istio Custom implemented , The managed control surface provides component capabilities to support refined traffic management and security management . Through the hosting mode , Decoupled Istio Components and managed K8s Life cycle management of clusters , Make the architecture more flexible , It improves the scalability of the system .

Managed services grid ASM In the infrastructure of unified management of various heterogeneous types of computing services , It provides a unified traffic management capability 、 Unified service security capability 、 Unified service observability 、 And based on WebAssembly Realize unified agent scalability , So as to build enterprise level capabilities .

How to develop the next station of service grid technology

Sidecar Proxy And Proxyless The integration of patterns is summed up in one sentence , Namely The same control surface , Support different data surface forms . The same control surface refers to the use of ASM The managed side component serves as a unified standard form of control entry , This control surface runs on Alibaba cloud , Belong to hosted Hosting mode .

The data side supports Sidecar Proxy And Proxyless Integration of modes , Although the components of the data plane are not hosted Hosting mode , But also managed Pattern , In other words, the life cycle of these components is also determined by ASM To manage , Including distribution to the data side 、 upgrade 、 Unloading, etc .

say concretely , stay Sidecar Proxy In mode , In addition to the current standard Envoy Outside the agency , Our architecture can easily support other Sidecar, for example Dapr Sidecar, Current Microsoft OSM+Dapr It is this kind of double Sidecar Pattern .

stay Proxyless In mode , In order to improve QPS Reduce delay , have access to SDK The way , for example gRPC Has supported xDS Protocol client , our Dubbo The team is also on this road . I think we can make some breakthroughs at this point together this year .

Another one proxyless Pattern , Is refers to - kernel eBPF + Node level Proxy The way . This pattern is right sidecar A fundamental change in the pattern , There is only one node Proxy, And ability offload Go to the node . In this part, we will also launch some products this year .

Around service grid technology , There are a series of application centered ecosystems in the industry , among , Alibaba cloud managed services grid ASM It supports the following ecosystems . List the following ：

Modern software development life cycle management and DevOps innovation

The core principles of service grid （ Security 、 Reliability and observability ） It supports the life cycle management of modern software development and DevOps innovation , For how to design the architecture in the cloud computing environment 、 Development 、 Automated Deployment and operation and maintenance provide flexibility 、 Scalability and testability . thus it can be seen , Service grid provides a solid foundation for dealing with modern software development , Anything for Kubernetes Teams building and deploying applications should seriously consider implementing service grids .

DevOps One of the important components of is to create continuous integration and deployment (CI/CD), Deliver containerized application code to production systems faster and more reliably . stay CI/CD Pipeline Enabling Canary or blue-green deployment in can provide more powerful testing for new application versions in the production system , And adopt a safe rollback strategy . under these circumstances , The service grid helps Canary deployment in the production system . Current Alibaba cloud service grid ASM Supported and ArgoCD、Argo Rollout、KubeVela And cloud effect 、Flagger The integration of such systems realizes the blue-green or Canary release of the application , As follows ：

ArgoCD [1] The main responsibility is to monitor Git Changes in the application layout in the warehouse , And compare the real running state of the application in the cluster , Automatically / Manually desynchronize and pull the changes of application orchestration into the deployment cluster . How to use Alibaba cloud service grid ASM In the integration ArgoCD Publish the application 、 to update , Simplify operation and maintenance costs .

Argo Rollouts [2] Provides a more powerful blue-green 、 Canary deployment capability . In practice, the two can be combined to provide a service based on GitOps Incremental delivery capability .

KubeVela [3] It's an out of the box 、 Modern application delivery and management platform . Use service grid ASM combination KubeVela It can realize the progressive gray-scale publishing of applications , Achieve the purpose of gently upgrading the application .

Alibaba cloud cloud efficiency pipeline Flow [4] Alibaba cloud service grid ASM complete Kubernetes Blue and green release of applications .

Flagger [5] Is another progressive delivery tool , Can be automatically executed in Kubernetes The release process of the application running on . It passes while measuring indicators and running consistency tests , Gradually transfer traffic to the new version , Reduces the risk of introducing new software versions into production . Alicloud service grid ASM Has been supported through Flagger Achieve this progressive release capability .

Microservice framework compatible [6]

Support Spring Boot/Cloud Applications are seamlessly migrated to the service grid for unified management and governance , It provides the ability to solve typical problems in the integration process , Including how the services inside and outside the container cluster interact 、 Common scenarios such as how to interconnect different language services .

Serverless Container and automatic expansion and contraction based on flow mode [7]

Serverless and Service Mesh Are two popular cloud native technologies , Customers are exploring how to create value from it . As we explore these solutions with our customers , The problem often arises in the intersection between these two popular technologies and how they complement each other . Can we take advantage of Service Mesh To protect the 、 Observe and disclose our Knative Server less applications ？ In a managed service grid ASM Support on the technology platform based on Knative Of Serverless Containers , And automatic expansion and contraction capability based on flow mode , It can replace how to simplify the complexity of users' maintaining the underlying infrastructure through the managed service grid , Let users easily build their own Serverless platform .

AI Serving [8]

---

Kubeflow Serving It is a project based on Kubernetes Community projects that support machine learning , Its next generation name is changed to KServe, The purpose of this project is to support different machine learning frameworks in a cloud native way , Based on the service grid, we can realize the flow control and update and rollback of the model version .

Zero trust security and Policy As Code[9]

In the use of Kubernetes Network Policy Realize three-layer network security control , Service Grid ASM Provides capabilities including peer-to-peer identity and request identity authentication 、Istio Authorization policy and more refined management based on OPA(Open Policy Agent) Strategic control capability .

say concretely , Building a zero trust security capability system based on service grid includes the following aspects ：

• The foundation of zero trust ： Workload identity ; How to provide a unified identity for cloud native workloads ;ASM The product provides an easy-to-use identity definition for each workload under the service grid , It also provides a customized mechanism for extending the identity construction system according to specific scenarios , Also compatible with the community SPIFFE standard ;

• The carrier of zero trust ： A security certificate ,ASM The product provides how to issue certificates and manage the life cycle of certificates 、 Rotation and other mechanisms , adopt X509 TLS Certificates establish identity , Each agent uses this certificate . And provide certificate and private key rotation ;

• Zero trust engine ： Strategy execution , Policy based trust engine is the key core of building zero trust ,ASM In addition to supporting Istio RBAC Outside the authorization policy , It also provides a OPA Provide finer grained authorization policies ;

• Zero trust insight ： Visualization and analysis ,ASM The product provides an observable mechanism to monitor the logs and indicators of policy implementation , To judge the implementation of each strategy, etc ;

The transformation of cloud native applications brings a lot of business value , One of them is elastic expansion and contraction , It can better cope with peak and trough traffic , Achieve the purpose of reducing cost and improving efficiency . Service Grid ASM It provides a non-invasive ability to generate telemetry data for the communication between application services , The index acquisition does not need to modify the application logic itself .

According to the four gold indicator dimensions monitored （ Delay 、 Traffic 、 Error and saturation ）, Service Grid ASM Generate a series of indicators for managed services , Support multiple protocols , Include HTTP,HTTP/2,GRPC,TCP etc. .

Besides , The service grid has 20 Multiple monitoring tags , Support all Envoy Proxy indicator attribute definition 、 General expression language CEL, Support customization Istio Generated metrics .

meanwhile , We are also exploring new scenarios to broaden the service grid drive , Here is an example AI Serving An example of [1****0] .

This demand source also comes from our actual customers , The customer's use scenario is to run on the service grid technology KServe To achieve AI service .KServe Run smoothly on the service grid , Implement the blue of model service / Green and Canary deployment 、 The ability to distribute traffic between revisions . Support automatic scaling Serverless Reasoning workload deployment 、 Support high scalability 、 Intelligent load routing based on concurrency .

summary

As the industry's first fully managed Istio Compatible Alibaba cloud service grid products ASM, From the beginning, we have maintained the relationship with the community in terms of architecture 、 Consistency of industry trends , The components of the control plane are hosted on the alicloud side , It is independent of the user cluster on the data side .ASM The product is community-based Istio Custom implemented , The managed control surface provides component capabilities to support refined traffic management and security management . Through the hosting mode , Decoupled Istio Components and managed K8s Life cycle management of clusters , Make the architecture more flexible , It improves the scalability of the system .

from 2022 year 4 month 1 The date of , Alicloud service grid ASM Officially launched a commercial version , Provides richer capabilities 、 Larger scale support and better technical support , Better meet the different needs of customers .

Reference link ：

[1] ArgoCD：

https://developer.aliyun.com/article/971976

[2] Argo Rollouts：

https://developer.aliyun.com/article/971975

[3] KubeVela：

https://help.aliyun.com/document_detail/337899.html

[4] Alibaba cloud cloud efficiency pipeline Flow：

https://help.aliyun.com/document_detail/160071.html

[5] Flagger：

https://docs.flagger.app/install/flagger-install-on-alibaba-servicemesh

[6] Microservice framework compatible ：

https://developer.aliyun.com/article/974941

[7] Serverless Container and automatic expansion and contraction based on flow mode ：

https://developer.aliyun.com/article/975639

[8] AI Serving：

https://developer.aliyun.com/article/971974

[9] Zero trust security and Policy As Code：

https://developer.aliyun.com/article/787187

[10] AI Serving An example of ：

https://developer.aliyun.com/article/971974