Cross cluster deployment of helm applications using karmada [cloud native open source]

author ： Huawei cloud's native open source team

background

By using Kubernetes Native API And provide advanced scheduling function ,Karmada It has implemented the... In the multi cluster scenario Kubernetes resources （ Include CRD） Distribution and management of . However, the current multi cluster application is often not a single resource form , Use Helm The use scenario of packaging applications is also very common .

With the help of Karmada Native API The ability to support ,Karmada Can use Flux Make it easy Helm Cross cluster deployment of applications .

Deploy Karmada

To deploy Karmada, You can refer to the community installation documentation （https://github.com/karmada-io/karmada/blob/master/docs/installation/installation.md）. If you want to experience it quickly Karmada, We propose to adopt hack/local-up-karmada.sh Construct a Karmada Development environment of .

Deploy Flux

stay Karmada In control surface , You need to install Flux Of CRD, But there is no need to install Flux The controller is based on CRD Created CR object , They are considered resource templates , Not specific resource instances . be based on Karmada Of work API, They will be encapsulated as a work Object to the member cluster , Finally, the members in the cluster Flux The controller is tuned .

kubectl apply -k github.com/fluxcd/flux2/manifests/crds?ref=main --kubeconfig ~/.kube/karmada.config

In a member cluster , You can install the complete... Based on the following command Flux Components .

flux install --kubeconfig ~/.kube/members.config --context member1flux install --kubeconfig ~/.kube/members.config --context member2

You can refer to the documentation here （https://fluxcd.io/docs/installation/） For more detailed installation Flux The details of the .

Tips ： If you want to manage all your clusters based on HelmRelease Application , You need to install... In all your member clusters Flux.

Helm release distribution

Ready for work , Next, we will use a podinfo As an example, it demonstrates how to complete Helm chart distribution .

1. stay Karmada Define a in the control plane Flux Of HelmRepository CR Object and a HelmRelease CR object . They will be treated as resource templates .

apiVersion: source.toolkit.fluxcd.io/v1beta2kind: HelmRepositorymetadata:name: podinfospec:interval: 1murl: https://stefanprodan.github.io/podinfo

---

apiVersion: helm.toolkit.fluxcd.io/v2beta1kind: HelmReleasemetadata:name: podinfospec:interval: 5mchart:spec:chart: podinfoversion: 5.0.3sourceRef:kind: HelmRepositoryname: podinfo

2. Define a Karmada Of PropagationPolicy Object to distribute their resource instances to the member cluster ：

apiVersion: policy.karmada.io/v1alpha1kind: PropagationPolicymetadata:name: helm-repospec:resourceSelectors:- apiVersion: source.toolkit.fluxcd.io/v1beta2kind: HelmRepositoryname: podinfoplacement:clusterAffinity:clusterNames:- member1- member2

---

apiVersion: policy.karmada.io/v1alpha1kind: PropagationPolicymetadata:name: helm-releasespec:resourceSelectors:- apiVersion: helm.toolkit.fluxcd.io/v2beta1kind: HelmReleasename: podinfoplacement:clusterAffinity:clusterNames:- member1- member2

The above configuration will make Flux The resource object of is distributed to the member cluster member1 and member2 in .

3. Submit the above objects to Karmada-apiserver：

kubectl apply -f ../helm/ --kubeconfig ~/.kube/karmada.config

You will get the following output ：

helmrelease.helm.toolkit.fluxcd.io/podinfo createdhelmrepository.source.toolkit.fluxcd.io/podinfo createdpropagationpolicy.policy.karmada.io/helm-release createdpropagationpolicy.policy.karmada.io/helm-repo created

4. Switch to the member cluster to verify whether the application is successfully distributed

helm --kubeconfig ~/.kube/members.config --kube-context member1 list

You will get the following output ：

be based on Karmada Of PropagationPolicy, You have the flexibility to Helm The application is published to the cluster you expect .

Customize for a specific cluster Helm application

The above example shows how the same Helm Application distribution to Karmada Multiple clusters in . Besides , You can still use it Karmada Of OverridePolicy Customize for a specific cluster Helm application . for example , The above applications include a Pod copy , If you just want to change member1 The applications in the cluster contain Pod replications , You can refer to the following OverridePolicy Strategy .

1. Define a Karmada Of OverridePolicy object .

apiVersion: policy.karmada.io/v1alpha1kind: OverridePolicymetadata:name: example-overridenamespace: defaultspec:resourceSelectors:- apiVersion: helm.toolkit.fluxcd.io/v2beta1kind: HelmReleasename: podinfooverrideRules:- targetCluster:clusterNames:- member1overriders:plaintext:- path: "/spec/values"operator: addvalue:replicaCount: 2

2. Submit the above objects to Karmada-apiserver：

kubectl apply -f example-override.yaml --kubeconfig ~/.kube/karmada.config

You will get the following output ：

overridepolicy.policy.karmada.io/example-override created

3. stay Karmada After applying the above strategy in the control plane , You'll find that member1 Members in the cluster Pod The number of instances has been changed to 2, but member2 Those in the cluster remain the same .

kubectl --kubeconfig ~/.kube/members.config --context member1 get po

You will get the following output ：

NAME READY STATUS RESTARTS AGEpodinfo-68979685bc-6wz6s 1/1 Running 0 6m28spodinfo-68979685bc-dz9f6 1/1 Running 0 7m42s

Reference documents ：

https://github.com/karmada-io/karmada/blob/master/docs/working-with-flux.md

attach ：Karmada Community technology exchange address

add to Karmada Community assistant wechat k8s2222 Enter the community communication group , and Maintainer Zero distance .

Project address

https://github.com/karmada-io/karmada

Slack Address ：

https://slack.cncf.io/