Cross cluster deployment of helm applications using karmada

Abstract ： With the help of Karmada Native API The ability to support ,Karmada Can use Flux Make it easy Helm Cross cluster deployment of applications .

This article is shared from Huawei cloud community 《  Use Karmada Realization Helm Cross cluster deployment of applications 【 Cloud native open source 】》, author ： Huawei cloud's native open source team .

background

By using Kubernetes Native API And provide advanced scheduling function ,Karmada It has implemented the... In the multi cluster scenario Kubernetes resources （ Include CRD） Distribution and management of . However, the current multi cluster application is often not a single resource form , Use Helm The use scenario of packaging applications is also very common .

With the help of Karmada Native API The ability to support ,Karmada Can use Flux Make it easy Helm Cross cluster deployment of applications .

Deploy Karmada

To deploy Karmada, You can refer to the community installation documentation （https://github.com/karmada-io/karmada/blob/master/docs/installation/installation.md）. If you want to experience it quickly Karmada, We propose to adopt hack/local-up-karmada.sh Construct a Karmada Development environment of .

Deploy Flux

stay Karmada In control surface , You need to install Flux Of CRD, But there is no need to install Flux The controller is based on CRD Created CR object , They are considered resource templates , Not specific resource instances . be based on Karmada Of work API, They will be encapsulated as a work Object to the member cluster , Finally, the members in the cluster Flux The controller is tuned .

kubectl apply -k github.com/fluxcd/flux2/manifests/crds?ref=main --kubeconfig ~/.kube/karmada.config

In a member cluster , You can install the complete... Based on the following command Flux Components .

flux install --kubeconfig ~/.kube/members.config --context member1flux install --kubeconfig ~/.kube/members.config --context member2

You can refer to the documentation here （https://fluxcd.io/docs/installation/） For more detailed installation Flux The details of the .

Tips ： If you want to manage all your clusters based on HelmRelease Application , You need to install... In all your member clusters Flux.

Helm release distribution

Ready for work , Next, we will use a podinfo As an example, it demonstrates how to complete Helm chart distribution .

1. stay Karmada Define a in the control plane Flux Of HelmRepository CR Object and a HelmRelease CR object . They will be treated as resource templates .

apiVersion: source.toolkit.fluxcd.io/v1beta2kind: HelmRepositorymetadata:  name: podinfospec:  interval: 1m  url: https://stefanprodan.github.io/podinfo  

---

apiVersion: helm.toolkit.fluxcd.io/v2beta1kind: HelmReleasemetadata:  name: podinfospec:  interval: 5m  chart:    spec:      chart: podinfo      version: 5.0.3      sourceRef:        kind: HelmRepository        name: podinfo

2. Define a Karmada Of PropagationPolicy Object to distribute their resource instances to the member cluster ：

apiVersion: policy.karmada.io/v1alpha1kind: PropagationPolicymetadata:  name: helm-repospec:  resourceSelectors:    - apiVersion: source.toolkit.fluxcd.io/v1beta2      kind: HelmRepository      name: podinfo  placement:    clusterAffinity:      clusterNames:        - member1        - member2

---

apiVersion: policy.karmada.io/v1alpha1kind: PropagationPolicymetadata:  name: helm-releasespec:  resourceSelectors:    - apiVersion: helm.toolkit.fluxcd.io/v2beta1      kind: HelmRelease      name: podinfo  placement:    clusterAffinity:      clusterNames:        - member1        - member2

The above configuration will make Flux The resource object of is distributed to the member cluster member1 and member2 in .

3. Submit the above objects to Karmada-apiserver：

kubectl apply -f ../helm/ --kubeconfig ~/.kube/karmada.config

You will get the following output ：

helmrelease.helm.toolkit.fluxcd.io/podinfo createdhelmrepository.source.toolkit.fluxcd.io/podinfo createdpropagationpolicy.policy.karmada.io/helm-release createdpropagationpolicy.policy.karmada.io/helm-repo created

4. Switch to the member cluster to verify whether the application is successfully distributed

helm --kubeconfig ~/.kube/members.config --kube-context member1 list

You will get the following output ：

be based on Karmada Of PropagationPolicy, You have the flexibility to Helm The application is published to the cluster you expect .

Customize for a specific cluster Helm application

The above example shows how the same Helm Application distribution to Karmada Multiple clusters in . Besides , You can still use it Karmada Of OverridePolicy Customize for a specific cluster Helm application . for example , The above applications include a Pod copy , If you just want to change member1 The applications in the cluster contain Pod replications , You can refer to the following OverridePolicy Strategy .

1. Define a Karmada Of OverridePolicy object .

apiVersion: policy.karmada.io/v1alpha1kind: OverridePolicymetadata:  name: example-override  namespace: defaultspec:  resourceSelectors:  - apiVersion: helm.toolkit.fluxcd.io/v2beta1    kind: HelmRelease    name: podinfo  overrideRules:  - targetCluster:      clusterNames:        - member1    overriders:      plaintext:        - path: "/spec/values"          operator: add          value:            replicaCount: 2

2. Submit the above objects to Karmada-apiserver：

kubectl apply -f example-override.yaml --kubeconfig ~/.kube/karmada.config

You will get the following output ：

overridepolicy.policy.karmada.io/example-override created

3. stay Karmada After applying the above strategy in the control plane , You'll find that member1 Members in the cluster Pod The number of instances has been changed to 2, but member2 Those in the cluster remain the same .

kubectl --kubeconfig ~/.kube/members.config --context member1 get po

You will get the following output ：

NAME                       READY   STATUS    RESTARTS   AGEpodinfo-68979685bc-6wz6s   1/1     Running   0          6m28spodinfo-68979685bc-dz9f6   1/1     Running   0          7m42s

Reference documents ：

https://github.com/karmada-io/karmada/blob/master/docs/working-with-flux.md

attach ：Karmada Community technology exchange address

add to Karmada Community assistant wechat k8s2222 Enter the community communication group , and Maintainer Zero distance .

Project address ：https://github.com/karmada-io/karmada

Slack Address ：https://slack.cncf.io/

 

 

Click to follow , The first time to learn about Huawei's new cloud technology ~