How to integrate Kata in kubernetes cluster

Kubernetes Cluster integration Kata

install Kubernetes colony

Use Kubeadm It is very convenient to install the cluster , You can refer to stay ubuntu Install in k8s colony .

You can also refer to official documents directly ：https://kubernetes.io/docs/setup/production-environment/tools/kubeadm/install-kubeadm/ .

• View installed Kubernetes edition

$ kubectl version

Client Version: version.Info{
    Major:"1", Minor:"17", GitVersion:"v1.17.9", GitCommit:"4fb7ed12476d57b8437ada90b4f93b17ffaeed99", GitTreeState:"clean", BuildDate:"2020-07-15T16:18:16Z", GoVersion:"go1.13.9", Compiler:"gc", Platform:"linux/amd64"}
Server Version: version.Info{
    Major:"1", Minor:"17", GitVersion:"v1.17.9", GitCommit:"4fb7ed12476d57b8437ada90b4f93b17ffaeed99", GitTreeState:"clean", BuildDate:"2020-07-15T16:10:45Z", GoVersion:"go1.13.9", Compiler:"gc", Platform:"linux/amd64"}

install Kata Command line tools

With CentOS Operating system as an example ：

$ source /etc/os-release
$ yum -y install yum-utils
$ ARCH=$(arch)
$ BRANCH="${BRANCH:-master}"
$ yum-config-manager --add-repo "http://download.opensuse.org/repositories/home:/katacontainers:/releases:/${ARCH}:/${BRANCH}/CentOS_${VERSION_ID}/home:katacontainers:releases:${ARCH}:${BRANCH}.repo"
$ yum -y install kata-runtime kata-proxy kata-shim

stay Ubuntu in （ Refer to official documentation ）：

$ ARCH=$(arch)
$ BRANCH="${BRANCH:-master}"
$ sudo sh -c "echo 'deb http://download.opensuse.org/repositories/home:/katacontainers:/releases:/${ARCH}:/${BRANCH}/xUbuntu_$(lsb_release -rs)/ /' > /etc/apt/sources.list.d/kata-containers.list"
$ curl -sL  http://download.opensuse.org/repositories/home:/katacontainers:/releases:/${
    ARCH}:/${
    BRANCH}/xUbuntu_$(lsb_release -rs)/Release.key | sudo apt-key add -
$ sudo -E apt-get update
$ sudo -E apt-get -y install kata-runtime kata-proxy kata-shim

Check if the hardware supports Kata

Kata The requirements for hardware need to meet any of the following conditions ：

• Intel VT-x technology.
• ARM Hyp mode (virtualization extension).
• IBM Power Systems.
• IBM Z mainframes.

installed kata-runtime after , Execute test command ：

$ kata-runtime kata-check

System is capable of running Kata Containers
System can currently create Kata Containers

The output here represents , The running environment supports Kata Containers

Configure and test Docker( Official documents )

• To configure kata-runtime Parameters

$ vim /etc/docker/daemon.json

Add the following , Default still used runc, But by specifying runtime Parameters can be used Kata .

{
    
  "runtimes": {
    "kata-runtime": {
    
      "path": "/usr/bin/kata-runtime"}
  }}

restart Docker service

$ systemctl daemon-reload
$ systemctl restart docker

test Kata Is the installation successful

$ docker run --runtime=kata-runtime  busybox uname -a

Linux 249a23f53475 5.4.60-65.1.container #1 SMP Thu Jan 1 00:00:00 UTC 1970 x86_64 GNU/Linux

$  docker run busybox uname -a

Linux b4812ed8990c 3.10.0-1127.el7.x86_64 #1 SMP Tue Mar 31 23:36:51 UTC 2020 x86_64 GNU/Linux

kata-runtime The container uses a different kernel version than the host , This means that kata-runtime Configuration worked .

To configure Kubelet

• New configuration file

$ mkdir -p  /etc/systemd/system/kubelet.service.d/
$ cat << EOF | sudo tee  /etc/systemd/system/kubelet.service.d/0-containerd.conf
[Service]                                                 
Environment="KUBELET_EXTRA_ARGS=--container-runtime=remote --runtime-request-timeout=15m --container-runtime-endpoint=unix:///run/containerd/containerd.sock"EOF

• Restart and take effect

$ systemctl daemon-reload
$ systemctl restart kubelet

What we use here is containerd . If you use CRI-O , The configuration will be different .

to Kubernetes Provide kata-runtime

By directly creating Container have access to kata-runtime . But in a cluster , How can we tell Kubernetes Which loads need to be used kata-runtime Well ？ According to different versions ,Kata Provides different ways .

First of all, you need to generate containerd The configuration file

$ containerd config default > /etc/containerd/config.toml

• RuntimeClass The way

This method requires relevant component versions ：

Kata Containers v1.5.0 or above (including 1.5.0-rc)
Containerd v1.2.0 or above
Kubernetes v1.12.0 or above

stay config.toml In profile , Add the following ：

    [plugins.cri.containerd]
      no_pivot = false[plugins.cri.containerd.runtimes]
      [plugins.cri.containerd.runtimes.runc]
         runtime_type = "io.containerd.runc.v1"
         [plugins.cri.containerd.runtimes.runc.options]
           NoPivotRoot = false
           NoNewKeyring = false
           ShimCgroup = ""
           IoUid = 0
           IoGid = 0
           BinaryName = "runc"
           Root = ""
           CriuPath = ""
           SystemdCgroup = false
      [plugins.cri.containerd.runtimes.kata]
         runtime_type = "io.containerd.kata.v2"
      [plugins.cri.containerd.runtimes.katacli]
         runtime_type = "io.containerd.runc.v1"
         [plugins.cri.containerd.runtimes.katacli.options]
           NoPivotRoot = false
           NoNewKeyring = false
           ShimCgroup = ""
           IoUid = 0
           IoGid = 0
           BinaryName = "/usr/bin/kata-runtime"
           Root = ""
           CriuPath = ""
           SystemdCgroup = false

here [plugins.cri.containerd.runtimes.kata] Medium kata Will be used as RuntimeClass handler keyword .

• Use untrusted_workload_runtime The way

For the environment that does not meet the requirements of the above version , You can use the previous method .

Add the following to the configuration file ：

    [plugins.cri.containerd.untrusted_workload_runtime]
      runtime_type = "io.containerd.runtime.v1.linux"
      runtime_engine = "/usr/bin/kata-runtime"

Last , Both need to be rebooted containerd.

$ containerd systemctl daemon-reload
$ systemctl restart containerd

And then I'll talk about kubelet Copy the configuration file to the user directory

sudo mkdir -p /home/xrw/.kube && sudo cp /etc/kubernetes/admin.conf /home/xrw/.kube/config

Use kata-runtime

RuntimeClass The way

• establish RuntimeClass

kata-runtime.yaml

kind: RuntimeClass
apiVersion: node.k8s.io/v1beta1
metadata:
  name: kata-containers
handler: kata

It can also be for runc establish RuntimeClass

$ kubectl get runtimeclass

NAME              CREATED AT
kata-containers   2020-08-30

Create a payload kata-pod.yaml

apiVersion: v1
kind: Pod
metadata:
  name: kata-nginx
spec:
  runtimeClassName: kata-containers
  containers:
  - name: nginx
    image: nginx
    ports:- containerPort: 80

$ kubectl apply -f kata-pod.yaml

Check whether the load is kata Running

#  obtain pod Medium Container ID 
$  sudo kubectl decribe pod kata-nginx
#  Check whether the container runtime of the container kata
$  sudo crictl inspect [Container ID]

untrusted_workload_runtime Use annotations tell Kubernetes Which loads of the cluster need to be used kata-runtime.

annotations:
  io.kubernetes.cri.untrusted-workload: "true"
 Here's an example  kata-pod-untrusted.yaml

apiVersion: v1
kind: Pod
metadata:
  name: kata-nginx-untrusted
  annotations:
    io.kubernetes.cri.untrusted-workload: "true"
spec:
  containers:
  - name: nginx
    image: nginx
    ports:- containerPort: 80

$ kubectl apply -f kata-pod-untrusted.yaml