Comparison and selection of kubernetes Devops CD Tools

One 、Flux

1.1  install

Flux Install and deploy others Pod In the same way , Support Helm Charts or Kustomize How to install . meanwhile fluxctl Command line is also provided （CLI） Tools .

Can pass fluxctl install Command to add parameters to monitor Git The warehouse URL.

1.2 GitOps Deploy

Flux Will pull the remote on a regular basis Git Warehouse , If there are new changes to the documents in the warehouse ,Flux Will GitOps Method to update the file to the target cluster . It can be deployment Application , Or maintenance Kubernetes Cluster configuration, etc .

In addition to the regular automatic mode , It can also be done through fluxctl sync Command manual trigger .

1.3  management

from K8S From the perspective of cluster management , Flux It's actually in the Pod A single binary file deployed in , After installation, there is almost no need to manage .

You can update yaml File to change the configuration or upgrade to a new version , Or use Kustomize or Helm Make changes .

besides , just so so fluxctl Command line tools for management , If synchronization is triggered immediately 、 Or query and display the relevant configurations and workloads deployed in the cluster .

1.4  multi-tenancy

because Flux In terms of architecture design, remote is not considered Git Warehouse and K8S The complexity of namespaces , Flux Multi tenancy mode is not considered .

although Flux There is no multi tenant mode , But it can be managed by the team Git Warehouse , And how to K8S Applications represented by namespaces in the cluster are associated with the target environment . In order to achieve isolation between different teams , You can use different Git Warehouse , And use different Flux Instance to monitor each Git Warehouse .

In the way of use , Can be in the same K8S Create multiple Flux example , Each instance monitors different Git Warehouse , And synchronize the updates of the warehouse to different K8S In the namespace . This approach requires different teams to manage their own Git Warehouse .

Although running multiple in the cluster Flux Instances may add some management overhead , But it allows teams to manage their own environmental repositories , And have appropriate permissions to submit changes and approvals PR. In addition, it can achieve a certain degree of isolation between namespaces , For different in the cluster Flux Each service account used by the instance provides more specific RBAC Set up .

1.5  Multi cluster

Multi tenant based usage scheme ,Flux It can also be used in multi cluster scenarios ： Different clusters have different Flux Instance to monitor Git Changes in the warehouse .

You can specify a directory to monitor in the warehouse , The team can flexibly choose the best in the case of multiple clusters Git Directory structure . A single repository can consist of two or more from different clusters Flux Instance to monitor , every last Flux Instance monitors a specific directory ; You can also use a separate version library to implement the corresponding functions .

1.6  Additional features

1.6.1  Automatically deploy new version container images

When a new version of the container image is available ,Flux You can choose to update the workload in the cluster . If you need to enable this function , Need to run fluxctl automate Or add comments in the deployment file of the workload , It will poll the image metadata in the image warehouse , If a new version of the specified image is available , It can use the new image to update the deployment .

When the new version image is automatically deployed ,Flux Will write a submission back to the original Git Warehouse , To update yaml The mirrored version used in the file , therefore Git It will still be consistent with the instances running in the cluster .

1.6.2

 

Reapply offset resources

No matter Git What are the new changes in the warehouse ,Flux Will synchronize the status of the workload for a period of time , And reapply its list . When the state of the application and configuration is GitOps When the workflow changes externally , This is very useful .

1.6.3  Garbage collection

When a resource is deleted from the environment repository ,Flux It can be removed from the cluster . This potentially dangerous operation needs to be tracked GitOps What resources have been created by the workflow .

When this function is enabled ,Flux According to the source （ Warehouse URL、 Branch and path combination specified during installation ）, Attach specific labels to all resources deployed in the cluster during the synchronization cycle . When it comes to garbage collection cycles ,Flux Will query APIServer, To retrieve all objects marked as coming from the source , And delete those objects that were not synchronized in the previous stage .

1.6.4  limitations

Flux The biggest limitation , It may be designed , And its biggest advantage . It only supports a single warehouse . This feature makes it a fairly simple tool , Easy to understand and troubleshoot . Considering that it runs on a lightweight deployment , This limitation can be overcome by multiple instances in the cluster , As in front of “ multi-tenancy ” As described in section .

Two 、ArgoCD

ArgoCD  Is a  Kubernetes Of 、 Declarative 、 be based on  GitOps  Continuous delivery of  （CD）  Tools . It focuses on the management of application deployment , With an excellent feature set , Covers multiple synchronization options 、 User access control 、 Status check, etc . It since 2018 Since Intuit Development , And in Apache 2.0 Under the license, in Github The open source .

Deployment from  ArgoCD  tracking  Git  Changes to the application manifest in the repository begin , Be similar to  Flux  How it works . however , What makes it shine is the ability to manage multi tenant and multi cluster deployments through fine-grained control . It can synchronize multiple applications from different teams to one or more  Kubernetes  colony . Besides , It also has a good modern Web UI, Users can view the status of their application deployment , Administrators can manage projects and user access .

2.1  Installation and management

ArgoCD It's all about Kubernetes Native installation and management . It's in Kubernetes Run in your own namespace on , All configurations are stored in ConfigMaps、Secrets and Custom Resources in . This allows us to use GitOps Workflow to manage ArgoCD In itself .

2.2  Supported manifest formats

ArgoCD stay GitOps The most extensive support for different formats is provided in the warehouse . According to the document , It can handle ：

• ·
   Kustomize Applications

• ·
   2.Helm Charts

• ·
   3.Ksonnet application

• ·
   4.YAML/JSON List of contents , contain Jsonnet

• ·
   5. Configuration management plug-in configuration of any custom configuration management tool

2.3  characteristic

ArgoCD Excellent in important functions such as multi tenancy , There are also many customization options .

2.3.1

 

multi-tenancy

One ArgoCD Instances can handle many applications from different teams . It uses the concept of a project to do this . Projects can accommodate multiple applications , And map to a team . Team members can only see the projects assigned to them , And only those applications in the project can be seen through the extension . This pattern is related to Kubernetes The resources in the namespace in are very similar .

2.3.2

 

Multi cluster

ArgoCD You can run it in Kubernetes Synchronize applications on the cluster , You can also manage external clusters . It can be configured to access only a limited set of namespaces .

Other clustered API The credentials of the server will be stored as secret in ArgoCD In the namespace of . As long as you can remotely expose other clusters API, This is a very useful function for managing all deployments from one place . Built in RBAC The mechanism provides some options , To control access to deployments in different environments by only specific users .

2.3.3

 

Configure offset detection

When the cluster operator fails  GitOps  workflow （ Submit to  Git） When changing resources ,Kubernetes  Resources may be stored in  Git  The configuration in is inconsistent . This is in GitOps China is a thorny and serious problem ： Because it's easy to make these changes and make Git The state in the repository is out of date . And Flux similar ,ArgoCD These changes can be detected and restored , So as to restore the state to Git Status defined in .

2.3.4  Garbage collection

Deleting resources is GitOps Another problem . When from Git When deleting resources such as deployment in ,kubectl apply It will be ignored （ Unless experimental --prune sign ）. This makes it impossible for developers to delete the resources they create , Because they are associated with Kubernetes The interface of is Git Warehouse .Helm It did solve the problem , but ArgoCD So it is with . It can delete obsolete resources in its own synchronization process , So you don't have to use Helm To achieve this function .

2.4  limitations

ArgoCD It feels like Flux Big brother of . It is a tool with rich functions , And Kubernetes Perfect integration . Its scope is also very clear ： It takes the list from Git The repository is synchronized to the cluster .

Basically, the entire configuration of the project to which the team belongs is stored ArgoCD Configured Git Files in the warehouse . It means , Every time you create or modify a new team , Must update this file . In a big company , The formation and dissolution of the development team are ongoing , This kind of management may become very cumbersome , It also limits ArgoCD Multi tenant settings provided .

3、 ... and 、Jenkins X

Jenkins X It's a circle GitOps Built completely CI/CD Solution , The bottom layer uses Tekton. First , In terms of name , We hope to see what everyone knows Jenkins The next generation of , Including its tasks and plug-in functions . actually ,Jenkins X Took different directions , And classic Jenkins Almost nothing in common .

It's gone Jenkins Of master-worker Node architecture , And become Kubernetes Native CI/CD The engine makes . It's in GitHub In order to Apache 2.0 Open source under license , from Cloudbees Development , The company also offers a commercial distribution .

It is worth noting that , In addition to based on GitOps Beyond the deployment function of ,Jenkins X It also covers a broader development cycle , Including from the typical CI Pipeline construction and testing stage , And the construction and storage of container images . So , It bundles and configures an impressive set of cloud native projects , So as to realize the modern development workflow .

Jenkins X Set Skaffold and Kaniko To build container images , And use Helm Chart to package Kubernetes detailed list . In the internal , It USES Tekton Run the assembly line , And use Lighthouse With your choice Git Provider Integration （ for example GitHub）, And in PR Enable rich interaction in threads .

4.1  install

Installation must use jx boot Command complete , And it assumes that the user has access to Kubernetes Extensive permissions of the cluster , Enough to create namespaces ,CRD, Set up sa And how to create GCP resources （ Such as Cloud Storage buckets）.

At the beginning of this process , Will clone the boot configuration library , And prompt the user to accept or change some default options , And provide  GitHub  User's credentials , These credentials will be in PR Used as a robot in threads GitHub User's credentials .

By default , stay GitHub Three... Are provided in Git Warehouse , To maintain the state of the environment in the cluster ：dev、staging and production.dev Environmental Science （ Cloned from the boot configuration repository ） Is where the tool runs .staging The environment and production Environment is the environment in which user applications run when deployed . Each environment has its own namespace and Git Warehouse .

4.2 GitOps Deploy

If the application warehouse is merged into master After that, the application was successfully built , The application will be created Helm chart New release of . To deploy this new version in one of these environments , Need to update the corresponding GitOps The repository . for example , To deploy to the production environment , Need to update the Git Warehouse .

follow GitOps principle , This change in the environmental repository triggers Tekton The deployment pipeline , Use Tekton Synchronize the content described in the warehouse with the content running in the cluster . The deployment process can also be achieved through jx promote Command to start , This command will update the environment repository , This triggers synchronization .

4.4  management

After installation , You can update dev The configuration file in the environment warehouse to adjust some internal configurations . When changes are submitted and pushed , Will trigger a pipeline ,Jenkins X It will read the configuration and update itself . Besides ,Jenkins X By itself GitOps Workflow to manage .

4.5  multi-tenancy

in consideration of Jenkins X There is no special isolation between applications from different teams in the cluster , Therefore, multi tenancy is not supported . Users and applications will share the same internal resources and components , And the same set of environment . Despite the concept of team , And by the jx team The subcommand supports , But this is a half baked solution , Basically, copy and paste the whole Jenkins X Deployment of .

4.6  Multi cluster

Application environment （ for example staging and production） It can run in other clusters . This is a Jenkins X A new feature in , Neglected for too long . The consequence of this neglect is , Now there are different solutions , There are different disadvantages , It seems that we will have to wait a while to find the final solution .

4.6.1  features Features

Jenkins X It has a lot of unique functions , Make the experience of developers more smooth and perfect than the other two tools .

4.6.2

 

Get started on new projects

For new projects ,jx quickstart The command can help create a new Git Warehouse , It's got Helm chart Application framework , And the use of Skaffold structure Docker Predefined build pipeline of image . It's still there. GitHub Configuration of the Git Warehouse , New submissions in branches and PR Then set a Webhook To trigger builds and other operations . have access to jx import Command to configure an existing project , This will help create Dockerfile and Helm chart（ If it doesn't exist ）, And set appropriate webhooks.

4.6.3

 

To configure Git Warehouse

Use the GitHub User credentials , During installation , Will be in... Of the selected organization GitHub Create an environment warehouse （dev,staging and production）, Or create as a user warehouse . Again , For the new quick start project mentioned above , There will also be a Git Warehouse and application framework .

4.6.4

 

Build pipeline

Jenkins X Support to trigger the task pipeline after an event occurs in the application repository . for example , Request for a new branch or pull , Build applications and run tests . In the so-called build package （buildpacks） There is a default build pipeline for different programming languages and frameworks . The application needs to have a jenkins-x.yaml file , And you can customize the pipeline .

4.6.5

 

ChatOps

ChatOps Is a term used to manage development and operational activities in chat threads , Usually, some automatic or requested operations are performed with the support of robot users . stay Jenkins X in , New created in the application or environment repository PR Will trigger the running pipeline . The result of the pipeline step is bot（ Configured during installation GitHub user ） Publish to PR In the thread . Users can interact with robots , To request other tasks , For example, rerun the test , Or approve and merge PR.

For integration Webhook, Triggers and events ,Jenkins X Use... Internally Prow, The tool was originally designed for GitHub Medium Kubernetes Built by the project operation . in addition ,Jenkins X The team is developing Lighthouse, To provide similar functions , Expand on different Git Distribution version （ Such as BitBucket and GitLab） Support for .

4.6.6

 

Preview environment

After a successful build , For example, in Jenkins X Newly created in the configured application repository PR in , A temporary preview deployment will be created for the current build . Through this preview , You can check manually as  PR  Current changes submitted （ for example , Open it in a browser  Web API  Applications ）, Then merge it into  master.

4.7  limitations

Although the scope of the other two tools is larger than Jenkins X Much smaller , But they can perfectly realize what they say . On the other hand ,Jenkins X It is a complex and comprehensive solution , This sets different expectations for people . The most significant missing feature is the multi tenant feature . When Jenkins X When installed on a cluster , We will expect it to serve all teams . Unfortunately ,Jenkins X The multi tenancy model of can be compared with Flux Compare the multi tenancy model of , Although the latter is a simple tool , but Jenkins X A large number of components are installed , Administrators should not want to install these components repeatedly for each team .

6、 ... and 、 Scheme comparison

Flux It's a small 、 Lightweight components , Can be implemented based on GitOps Deployment of , It may complement your current settings . It only needs to access one （ There is only one ）Git Warehouse , And in Limited RBAC Run under permission .

ArgoCD It can manage the deployment of multiple applications in different clusters . It runs within the cluster , But you can also manage the access rights and permissions of teams and projects . It has a smooth Web Interface to manage applications and projects , It also provides a set of good functions to manage the deployment .

Jenkins X Bundled with some controversial tools , To build a center around GitHub Warehouse development workflow （ Will soon support other suppliers ）. It runs CI Pipeline to build and run application tests , stay PR Provides rich interaction , And according to Git Change management deployment in the warehouse .