Cross cluster deployment of helm applications using karmada

Abstract ： With the help of Karmada Native API The ability to support ,Karmada Can use Flux Make it easy Helm Cross cluster deployment of applications .

This article is shared from Huawei cloud community 《  Use Karmada Realization Helm Cross cluster deployment of applications 【 Cloud native open source 】》, author ： Huawei cloud's native open source team .

background

By using Kubernetes Native API And provide advanced scheduling function ,Karmada It has implemented the... In the multi cluster scenario Kubernetes resources （ Include CRD） Distribution and management of . However, the current multi cluster application is often not a single resource form , Use Helm The use scenario of packaging applications is also very common .

With the help of Karmada Native API The ability to support ,Karmada Can use Flux Make it easy Helm Cross cluster deployment of applications .

Deploy Karmada

To deploy Karmada, You can refer to the community installation documentation （https://github.com/karmada-io/karmada/blob/master/docs/installation/installation.md）. If you want to experience it quickly Karmada, We propose to adopt hack/local-up-karmada.sh Construct a Karmada Development environment of .

Deploy Flux

stay Karmada In control surface , You need to install Flux Of CRD, But there is no need to install Flux The controller is based on CRD Created CR object , They are considered resource templates , Not specific resource instances . be based on Karmada Of work API, They will be encapsulated as a work Object to the member cluster , Finally, the members in the cluster Flux The controller is tuned .

kubectl apply -k github.com/fluxcd/flux2/manifests/crds?ref=main --kubeconfig ~/.kube/karmada.config

In a member cluster , You can install the complete... Based on the following command Flux Components .

flux install --kubeconfig ~/.kube/members.config --context member1

flux install --kubeconfig ~/.kube/members.config --context member2

You can refer to the documentation here （https://fluxcd.io/docs/installation/） For more detailed installation Flux The details of the .

Tips ： If you want to manage all your clusters based on HelmRelease Application , You need to install... In all your member clusters Flux.

Helm release distribution

Ready for work , Next, we will use a podinfo As an example, it demonstrates how to complete Helm chart distribution .

1. stay Karmada Define a in the control plane Flux Of HelmRepository CR Object and a HelmRelease CR object . They will be treated as resource templates .

apiVersion: source.toolkit.fluxcd.io/v1beta2
kind: HelmRepository
metadata:
  name: podinfo
spec:
  interval: 1m
  url: https://stefanprodan.github.io/podinfo  

---

apiVersion: helm.toolkit.fluxcd.io/v2beta1
kind: HelmRelease
metadata:
  name: podinfo
spec:
  interval: 5m
  chart:
    spec:
      chart: podinfo
      version: 5.0.3
      sourceRef:
        kind: HelmRepository
        name: podinfo

2. Define a Karmada Of PropagationPolicy Object to distribute their resource instances to the member cluster ：

apiVersion: policy.karmada.io/v1alpha1
kind: PropagationPolicy
metadata:
  name: helm-repo
spec:
  resourceSelectors:
    - apiVersion: source.toolkit.fluxcd.io/v1beta2
      kind: HelmRepository
      name: podinfo
  placement:
    clusterAffinity:
      clusterNames:
        - member1
        - member2

---

apiVersion: policy.karmada.io/v1alpha1
kind: PropagationPolicy
metadata:
  name: helm-release
spec:
  resourceSelectors:
    - apiVersion: helm.toolkit.fluxcd.io/v2beta1
      kind: HelmRelease
      name: podinfo
  placement:
    clusterAffinity:
      clusterNames:
        - member1
        - member2

The above configuration will make Flux The resource object of is distributed to the member cluster member1 and member2 in .

3. Submit the above objects to Karmada-apiserver：

kubectl apply -f ../helm/ --kubeconfig ~/.kube/karmada.config

You will get the following output ：

helmrelease.helm.toolkit.fluxcd.io/podinfo created
helmrepository.source.toolkit.fluxcd.io/podinfo created
propagationpolicy.policy.karmada.io/helm-release created
propagationpolicy.policy.karmada.io/helm-repo created

4. Switch to the member cluster to verify whether the application is successfully distributed

helm --kubeconfig ~/.kube/members.config --kube-context member1 list

You will get the following output ：

be based on Karmada Of PropagationPolicy, You have the flexibility to Helm The application is published to the cluster you expect .

Customize for a specific cluster Helm application

The above example shows how the same Helm Application distribution to Karmada Multiple clusters in . Besides , You can still use it Karmada Of OverridePolicy Customize for a specific cluster Helm application . for example , The above applications include a Pod copy , If you just want to change member1 The applications in the cluster contain Pod replications , You can refer to the following OverridePolicy Strategy .

1. Define a Karmada Of OverridePolicy object .

apiVersion: policy.karmada.io/v1alpha1
kind: OverridePolicy
metadata:
  name: example-override
  namespace: default
spec:
  resourceSelectors:
  - apiVersion: helm.toolkit.fluxcd.io/v2beta1
    kind: HelmRelease
    name: podinfo
  overrideRules:
  - targetCluster:
      clusterNames:
        - member1
    overriders:
      plaintext:
        - path: "/spec/values"
          operator: add
          value:
            replicaCount: 2

2. Submit the above objects to Karmada-apiserver：

kubectl apply -f example-override.yaml --kubeconfig ~/.kube/karmada.config

You will get the following output ：

overridepolicy.policy.karmada.io/example-override created

3. stay Karmada After applying the above strategy in the control plane , You'll find that member1 Members in the cluster Pod The number of instances has been changed to 2, but member2 Those in the cluster remain the same .

kubectl --kubeconfig ~/.kube/members.config --context member1 get po

You will get the following output ：

NAME                       READY   STATUS    RESTARTS   AGE
podinfo-68979685bc-6wz6s   1/1     Running   0          6m28s
podinfo-68979685bc-dz9f6   1/1     Running   0          7m42s

Reference documents ：

karmada/working-with-flux.md at master · karmada-io/karmada · GitHub

attach ：Karmada Community technology exchange address

add to Karmada Community assistant wechat k8s2222 Enter the community communication group , and Maintainer Zero distance .

Project address ：

GitHub - karmada-io/karmada: Open, Multi-Cloud, Multi-Cluster Kubernetes Orchestration

Slack Address ：https://slack.cncf.io/

Click to follow , The first time to learn about Huawei's new cloud technology ~