Cross cluster deployment of helm applications using karmada

This article is shared from Huawei cloud community 《 

Use Karmada Realization Helm Cross cluster deployment of applications 【 Cloud native open source 】

》, author ： Huawei cloud's native open source team .

background

By using  Kubernetes  Native  API  And provide advanced scheduling function ,Karmada It has implemented the... In the multi cluster scenario Kubernetes resources （ Include CRD） Distribution and management of . However, the current multi cluster application is often not a single resource form , Use Helm The use scenario of packaging applications is also very common .

With the help of Karmada Native API The ability to support ,Karmada Can use Flux Make it easy Helm Cross cluster deployment of applications .

Deploy Karmada

To deploy Karmada, You can refer to the community installation documentation （

https://github.com/karmada-io/karmada/blob/master/docs/installation/installation.md

）. If you want to experience it quickly Karmada, We propose to adopt hack/local-up-karmada.sh Construct a Karmada Development environment of .

Deploy Flux

stay Karmada In control surface , You need to install Flux Of CRD, But there is no need to install Flux The controller is based on CRD Created CR object , They are considered resource templates , Not specific resource instances . be based on Karmada Of work API, They will be encapsulated as a work Object to the member cluster , Finally, the members in the cluster Flux The controller is tuned .

kubectl apply -k github.com/fluxcd/flux2/manifests/crds?ref=main --kubeconfig ~/.kube/karmada.config

In a member cluster , You can install the complete... Based on the following command Flux Components .

flux install --kubeconfig ~/.kube/members.config --context member1

flux install --kubeconfig ~/.kube/members.config --context member2

You can refer to the documentation here （

https://fluxcd.io/docs/installation/

） For more detailed installation Flux The details of the .

Tips ： If you want to manage all your clusters based on HelmRelease Application , You need to install... In all your member clusters Flux.

Helm release distribution

Ready for work , Next, we will use a podinfo As an example, it demonstrates how to complete Helm chart distribution .

1. stay  Karmada  Define a in the control plane  Flux  Of HelmRepository CR Object and a  HelmRelease CR object . They will be treated as resource templates .

apiVersion: source.toolkit.fluxcd.io/v1beta2
kind: HelmRepository
metadata:
 name: podinfo
spec:
 interval: 1m
 url: https://stefanprodan.github.io/podinfo 

---

apiVersion: helm.toolkit.fluxcd.io/v2beta1
kind: HelmRelease
metadata:
 name: podinfo
spec:
 interval: 5m
 chart:
 spec:
 chart: podinfo
 version: 5.0.3
 sourceRef:
 kind: HelmRepository
 name: podinfo

2.  Define a  Karmada Of PropagationPolicy  Object to distribute their resource instances to the member cluster ：

apiVersion: policy.karmada.io/v1alpha1
kind: PropagationPolicy
metadata:
 name: helm-repo
spec:
 resourceSelectors:
 - apiVersion: source.toolkit.fluxcd.io/v1beta2
 kind: HelmRepository
 name: podinfo
 placement:
 clusterAffinity:
 clusterNames:
 - member1
 - member2

---

apiVersion: policy.karmada.io/v1alpha1
kind: PropagationPolicy
metadata:
 name: helm-release
spec:
 resourceSelectors:
 - apiVersion: helm.toolkit.fluxcd.io/v2beta1
 kind: HelmRelease
 name: podinfo
 placement:
 clusterAffinity:
 clusterNames:
 - member1
 - member2

The above configuration will make Flux The resource object of is distributed to the member cluster member1 and member2 in .

3.  Submit the above objects to Karmada-apiserver：

kubectl apply -f ../helm/ --kubeconfig ~/.kube/karmada.config

You will get the following output ：

helmrelease.helm.toolkit.fluxcd.io/podinfo created
helmrepository.source.toolkit.fluxcd.io/podinfo created
propagationpolicy.policy.karmada.io/helm-release created
propagationpolicy.policy.karmada.io/helm-repo created

4.  Switch to the member cluster to verify whether the application is successfully distributed

helm --kubeconfig ~/.kube/members.config --kube-context member1 list

You will get the following output ：

be based on  Karmada  Of  PropagationPolicy, You have the flexibility to  Helm The application is published to the cluster you expect .

Customize for a specific cluster  Helm  application

The above example shows how the same Helm Application distribution to  Karmada  Multiple clusters in . Besides , You can still use it  Karmada  Of  OverridePolicy  Customize for a specific cluster Helm application . for example , The above applications include a Pod copy , If you just want to change  member1 The applications in the cluster contain Pod replications , You can refer to the following  OverridePolicy Strategy .

1. Define a Karmada Of OverridePolicy object .

apiVersion: policy.karmada.io/v1alpha1
kind: OverridePolicy
metadata:
 name: example-override
 namespace: default
spec:
 resourceSelectors:
 - apiVersion: helm.toolkit.fluxcd.io/v2beta1
 kind: HelmRelease
 name: podinfo
 overrideRules:
 - targetCluster:
 clusterNames:
 - member1
 overriders:
 plaintext:
 - path: "/spec/values"
 operator: add
 value:
 replicaCount: 2

2.  Submit the above objects to Karmada-apiserver：

kubectl apply -f example-override.yaml --kubeconfig ~/.kube/karmada.config

You will get the following output ：

overridepolicy.policy.karmada.io/example-override created

3.  stay  Karmada  After applying the above strategy in the control plane , You'll find that  member1 Members in the cluster Pod The number of instances has been changed to  2, but  member2  Those in the cluster remain the same .

kubectl --kubeconfig ~/.kube/members.config --context member1 get po

You will get the following output ：

NAME READY STATUS RESTARTS AGE
podinfo-68979685bc-6wz6s 1/1 Running 0 6m28s
podinfo-68979685bc-dz9f6 1/1 Running 0 7m42s

Reference documents ：

https://github.com/karmada-io/karmada/blob/master/docs/working-with-flux.md

attach ：Karmada Community technology exchange address

add to Karmada Community assistant wechat k8s2222 Enter the community communication group , and Maintainer Zero distance .

Project address ：

https://github.com/karmada-io/karmada

Slack Address ：

https://slack.cncf.io/

Click to follow , The first time to learn about Huawei's new cloud technology ~