当前位置:网站首页>Operator-1初识Operator

Operator-1初识Operator

2022-07-01 12:40:00 51CTO

背景:

接触kubernetes也好多年了,开始就各种听说Operator的,但是从来没有深入了解动手写过Operator
。开始体验一下简单的Operator

Operator初体验

什么是Operator?

参照:红帽官方文档 什么是 Kubernetes Operator?
**coreos2016年引入,**是一种封装、部署和管理 Kubernetes 应用的方法

  1. crd webhook controller

开发工具:

what is crd

**CRD **全称是 Custom Resource Definition, CRD是一种无需编码就可以扩展原生kubenetes API接口的方式。适合扩展kubernetes的自定义接口和功能。如果想更为灵活的添加逻辑就需要API Aggregation方式.

开始准备

常用的开发工具有一下几种:
Operator-1初识Operator_云原生
我的开发工具 包括goland kubebuilder kustomize,kubernetes1.23.6,工作环境rocky linux 8.5 go 1.17
注意:一定看一下go 版本 与开发工具对应版本,以及与kubernetes的版本

kubebuilder kustomize install

 https://github.com/kubernetes-sigs/kubebuilder/releases
Operator-1初识Operator_oprator_02

[[email protected] ~]# wget https://github.com/kubernetes-sigs/kubebuilder/releases/download/v3.5.0/kubebuilder_linux_amd64

     
  • 1.

Operator-1初识Operator_kubernetes_03

[[email protected] ~]# mv kubebuilder_linux_amd64 /usr/bin/kubebuilder
[[email protected] ~]# chmod +x /usr/bin/kubebuilder
[[email protected] ~]# kubebuilder version
Version: main.version{KubeBuilderVersion:"3.5.0", KubernetesVendor:"1.24.1", GitCommit:"26d12ab1134964dbbc3f68877ebe9cf6314e926a", BuildDate:"2022-06-24T12:17:52Z", GoOs:"linux", GoArch:"amd64"}

     
  • 1.
  • 2.
  • 3.
  • 4.
[email protected] ~]# wget https://github.com/kubernetes-sigs/kustomize/releases/download/kustomize%2Fv4.5.5/kustomize_v4.5.5_linux_amd64.tar.gz


     
  • 1.
  • 2.

Operator-1初识Operator_kubernetes_04

[[email protected] ~]# tar zxvf kustomize_v4.5.5_linux_amd64.tar.gz 
kustomize
[[email protected] ~]# chmod +x kustomize
[[email protected] ~]# mv kustomize /usr/bin/kustomize
[[email protected] ~]# kustomize version
{Version:kustomize/v4.5.5 GitCommit:daa3e5e2c2d3a4b8c94021a7384bfb06734bcd26 BuildDate:2022-05-20T20:25:40Z GoOs:linux GoArch:amd64}

     
  • 1.
  • 2.
  • 3.
  • 4.
  • 5.
  • 6.

Operator-1初识Operator_kubernetes_05

创建并初始化项目

goland创建名为kube-oprator1的项目:

Operator-1初识Operator_kubernetes_06
终端执行一下命令:

[[email protected] kube-oprator1]$ kubebuilder init --plugins go/v3 --domain zhangpeng.com --owner "zhang peng"


     
  • 1.
  • 2.

Operator-1初识Operator_云原生_07

貌似提示我go版本过低?(go版本 1.17.6我的是)
Operator-1初识Operator_云原生_08

升级一下go版本

注意:非必须,后面是降低了kubebuilder的版本。go版本就保持1.17版本了,

浏览器打开 https://golang.google.cn/dl/ go下载页面,选择1.17最新版本下载并替换本地GO版本!
Operator-1初识Operator_云原生_09

[[email protected] ~]# wget https://golang.google.cn/dl/go1.17.11.linux-amd64.tar.gz
[[email protected] ~]# tar zxvf go1.17.11.linux-amd64.tar.gz
[[email protected] ~]# which go
/usr/go/bin/go
[[email protected] ~]# cd go/
[[email protected] ~]# cp -Ra * /usr/go/
[[email protected] go]# go version go1.17.11 linux/amd64

     
  • 1.
  • 2.
  • 3.
  • 4.
  • 5.
  • 6.
  • 7.

。。。。貌似还是报错,仔细看了一眼 https://github.com/kubernetes-sigs/kubebuilder/releases我还是换个kubebuilder版本吧…
Operator-1初识Operator_oprator_10

kubebuilder 版本3.4.1

[[email protected] ~]# wget https://github.com/kubernetes-sigs/kubebuilder/releases/download/v3.4.1/kubebuilder_linux_amd64
[[email protected] ~]# mv kubebuilder_linux_amd64 /usr/bin/kubebuilder
mv:是否覆盖'/usr/bin/kubebuilder'? y
[[email protected] ~]# chmod +x /usr/bin/kubebuilder 
[[email protected] ~]# kubebuilder version
Version: main.version{KubeBuilderVersion:"3.4.1", KubernetesVendor:"1.23.5", GitCommit:"d59d7882ce95ce5de10238e135ddff31d8ede026", BuildDate:"2022-05-06T13:58:56Z", GoOs:"linux", GoArch:"amd64"}

     
  • 1.
  • 2.
  • 3.
  • 4.
  • 5.
  • 6.
kubebuilder init --plugins go/v3 --domain zhangpeng.com --owner "zhang peng"

     
  • 1.

Operator-1初识Operator_kubernetes_11
生成目录结构如下:
Operator-1初识Operator_kubernetes_12

重点关注一下config/default/ kustomization.yaml文件:
Operator-1初识Operator_kubernetes_13
现在能看懂的配置,命名空间!我这里就不修改了采用默认的配置!

 [[email protected] kube-oprator1]$ kubebuilder create api --group myapp1 --version v1 --kind Redis


     
  • 1.
  • 2.

Operator-1初识Operator_oprator_14
目录结构如下
Operator-1初识Operator_kubernetes_15

注意:关于 domain group version kind对应 :

apiVersion:myapp1.zhangpeng.com/v1
kind: Redis

     
  • 1.
  • 2.

Operator-1初识Operator_kubernetes_16

简单创建一个crd

api/v1/redis_type.go
Operator-1初识Operator_kubernetes_17
Operator-1初识Operator_oprator_18
随手演示删除Foo字段,添加一个Port字段,设置port字段为int类型!
Operator-1初识Operator_kubernetes_19

以test目录下yaml文件定制crd

test/redis.yaml

apiVersion: myapp1.zhangpeng.com/v1
kind: Redis
metadata:
  name: myapp
spec:
  port: 1011

     
  • 1.
  • 2.
  • 3.
  • 4.
  • 5.
  • 6.

make install创建crd

Operator-1初识Operator_oprator_20

[[email protected] kube-oprator1]$ kubectl get crd
No resources found
[[email protected] kube-oprator1]$ make install 
GOBIN=/home/zhangpeng/GolandProjects/kube-oprator1/bin go install sigs.k8s.io/controller-tools/cmd/[email protected]
/home/zhangpeng/GolandProjects/kube-oprator1/bin/controller-gen rbac:roleName=manager-role crd webhook paths="./..." output:crd:artifacts:config=config/crd/bases
/home/zhangpeng/GolandProjects/kube-oprator1/bin/kustomize build config/crd | kubectl apply -f -
customresourcedefinition.apiextensions.k8s.io/redis.myapp1.zhangpeng.com created
[[email protected] kube-oprator1]$ kubectl get crd
NAME                         CREATED AT
redis.myapp1.zhangpeng.com   2022-06-28T06:44:52Z


     
  • 1.
  • 2.
  • 3.
  • 4.
  • 5.
  • 6.
  • 7.
  • 8.
  • 9.
  • 10.
  • 11.

关于reconcile

controllers/redis_controller.go
关于reconcile就先不求甚解了
Operator-1初识Operator_云原生_21

func (r *RedisReconciler) Reconcile(ctx context.Context, req ctrl.Request) (ctrl.Result, error) {
	_ = log.FromContext(ctx)

	// TODO(user): your logic here
	redis := &myapp1v1.Redis{}
	if err := r.Get(ctx, req.NamespacedName, redis); err != nil {
		fmt.Println(err)
	} else {
		fmt.Println("object", redis)
	}

	return ctrl.Result{}, nil
}

     
  • 1.
  • 2.
  • 3.
  • 4.
  • 5.
  • 6.
  • 7.
  • 8.
  • 9.
  • 10.
  • 11.
  • 12.
  • 13.

Operator-1初识Operator_云原生_22

本地调试 make run

终端一运行

[[email protected] kube-oprator1]$ maker run  

     
  • 1.

终端2运行

 
 [[email protected] kube-oprator1]$ kubectl apply -f test/redis.yaml

     
  • 1.
  • 2.

观察终端1 得到如下输出:
Operator-1初识Operator_云原生_23

初步发布到kubernetes集群

注:我的环境安装了podman,关于podman自行百度,镜像仓库使用了腾讯云镜像仓库个人版

关于Podman

先修改docker构建命令为podman!
Operator-1初识Operator_kubernetes_24
podman login将密码记住…基本跟docker的使用方法差不多

[[email protected] kube-oprator1]$ podman login --username=xxxxx ccr.ccs.tencentyun.com


     
  • 1.
  • 2.

dockerhub加速

的特别强调一下dockerhub加速

[[email protected] kube-oprator1]$  vim /etc/containers/registries.conf

     
  • 1.

文件末尾添加了加速地址!

short-name-mode = "permissive"
[[registry]]
prefix="docker.io"
location="pvurwzu6.mirror.aliyuncs.com"

     
  • 1.
  • 2.
  • 3.
  • 4.

重启podman服务

[[email protected] kube-oprator1]$  systemctl restart podman

     
  • 1.

构建发布镜像

Dockerfile文件中添加GOPROXY

ENV GOPROXY=https://goproxy.io

     
  • 1.
[[email protected] kube-oprator1]$ make docker-build docker-push IMG=ccr.ccs.tencentyun.com/layatools/zpredis:v1

     
  • 1.

Operator-1初识Operator_云原生_25
顺便发现腾讯云个人仓库一个不显示OCI-Image大小的bug…
Operator-1初识Operator_oprator_26
Operator-1初识Operator_oprator_27

注:过程很曲折。中间有镜像下不动的科学上网了,自行脑部。如“gcr.io/distroless/static:nonroot镜像我的操作环境为rocky linux 8.5下载不动的时候我直接科学上网了…

发布方式:

[[email protected] kube-oprator1]$ make deploy IMG=ccr.ccs.tencentyun.com/layatools/zpredis:v1

     
  • 1.

make又失败了最终根据Makefile中deploy手动执行如下命令:

[[email protected] kube-oprator1]$ cd config/manager &&  kustomize edit set image controller=ccr.ccs.tencentyun.com/layatools/zpredis:v1
[[email protected] kube-oprator1]$ kustomize build config/default | kubectl apply -f -


     
  • 1.
  • 2.
  • 3.

注意:两条命令都是在kube-oprator1项目根目录下执行的!
Operator-1初识Operator_kubernetes_28
Operator-1初识Operator_kubernetes_29
默认命名空间没有修改查看kube-oprator1-system namespace命名空间下pod状态!

[[email protected] kube-oprator1]$ kubectl get ns
NAME                   STATUS   AGE
default                Active   61d
kube-node-lease        Active   61d
kube-oprator1-system   Active   25h
kube-public            Active   61d
kube-system            Active   61d
zhangpeng1             Active   8d
[[email protected] kube-oprator1]$ kubectl get pods -n kube-oprator1-system


     
  • 1.
  • 2.
  • 3.
  • 4.
  • 5.
  • 6.
  • 7.
  • 8.
  • 9.
  • 10.

理论上pod是没有部署成功的,原因如下:gcr.io/kubebuilder/kube-rbac-proxy:v0.11.0无法下载…我是用的苯方法,rocky开发机科学上网下载镜像上传到腾讯与镜像仓库,然后pull镜像到kuberntes机器work节点。当然了work节点我只有一个测试环境还好~
Operator-1初识Operator_kubernetes_30
等待pod running
Operator-1初识Operator_kubernetes_31

CRD自定义资源简单验证

以test/redis.yaml为例

apiVersion: myapp1.zhangpeng.com/v1
kind: Redis
metadata:
  name: myapp
spec:
  port: 1011

     
  • 1.
  • 2.
  • 3.
  • 4.
  • 5.
  • 6.

参照 https://book.kubebuilder.io/reference/markers/crd-validation.html 就设置一下port的范围!
Operator-1初识Operator_oprator_32

	// +kubebuilder:validation:Minimum:=1024
	// +kubebuilder:validation:Maximum:=10240

     
  • 1.
  • 2.

Operator-1初识Operator_oprator_33

make install 依然失败!还是手动命令了
注:这个地方我做错了好几次,原因是我以为make install =kustomize build config/crd | kubectl apply -f -,仔细看了一眼Makefile:
Operator-1初识Operator_oprator_34
包含manifests的步骤,尝试了一下果然如此!

[[email protected] kube-oprator1]$ ./bin/controller-gen rbac:roleName=manager-role crd webhook paths="./..." output:crd:artifacts:config=config/crd/bases

[[email protected] kube-oprator1]$ kustomize build config/crd | kubectl apply -f -

     
  • 1.
  • 2.
  • 3.

Operator-1初识Operator_云原生_35

[[email protected] kube-oprator1]$ kubectl get crd redis.myapp1.zhangpeng.com -o yaml


     
  • 1.
  • 2.

Operator-1初识Operator_kubernetes_36

[[email protected] kube-oprator1]$ kubectl delete -f test/redis.yaml
[[email protected] kube-oprator1]$ kubectl apply -f test/redis.yaml

     
  • 1.
  • 2.

得到如下输出:端口小于1024无法创建成功
Operator-1初识Operator_云原生_37
修改 test/redis.yaml port: 1024

apiVersion: myapp1.zhangpeng.com/v1
kind: Redis
metadata:
  name: myapp
spec:
  port: 1024

     
  • 1.
  • 2.
  • 3.
  • 4.
  • 5.
  • 6.

Operator-1初识Operator_kubernetes_38

[[email protected] cert]$ kubectl get pods -A|grep cert
cert-manager           cert-manager-677874db78-zcm6l                       1/1     Running            0                  14m
cert-manager           cert-manager-cainjector-6c5bf7b759-mf4gf            1/1     Running            0                  14m
cert-manager           cert-manager-webhook-5685fdbc4b-ncrxl               1/1     Running            0                  14m


     
  • 1.
  • 2.
  • 3.
  • 4.
  • 5.

webhook简单测试

简单准入控制器webhook create

[[email protected] kube-oprator1]$ kubebuilder create webhook --group myapp1 --version v1 --kind Redis --defaulting --programmatic-validation


     
  • 1.
  • 2.

Operator-1初识Operator_云原生_39
kube-oprator1 api/v1目录下增加了webhook的相关文件,做了一个简单的验证** name=zhangpeng**
Operator-1初识Operator_kubernetes_40

func (r *Redis) ValidateCreate() error {
	redislog.Info("validate create", "name", r.Name)
	if r.Name == "zhangpeng" {
		return errors.New("error name")
	}

	// TODO(user): fill in your validation logic upon object creation.
	return nil
}

     
  • 1.
  • 2.
  • 3.
  • 4.
  • 5.
  • 6.
  • 7.
  • 8.
  • 9.

证书管理cert-manager:

访问 https://github.com/cert-manager/cert-manager/releases下载页面,1.19.0是alpha版本 我用了1.18.2的版本!
Operator-1初识Operator_云原生_41

[[email protected] cert]$ pwd
/home/zhangpeng/cert
[[email protected] cert]$ wget https://github.com/cert-manager/cert-manager/releases/download/v1.8.2/cert-manager.yaml
[[email protected] cert]$ kubectl apply -f cert-manager.yaml
[[email protected] cert]$ kubectl get pods -A|grep cert

     
  • 1.
  • 2.
  • 3.
  • 4.
  • 5.

Operator-1初识Operator_oprator_42
Operator-1初识Operator_kubernetes_43
注意:镜像依然要科学上网下载

修改文件

config/default/kustomization.yaml下箭头标注部分解开注释
Operator-1初识Operator_kubernetes_44
Operator-1初识Operator_kubernetes_45
**config/manager/manager.yaml **
Operator-1初识Operator_kubernetes_46

删除crd

make uninstall可以不过我的make总是失败…直接删除了!

[[email protected] kube-oprator1]$kubectl delete crd redis.myapp1.zhangpeng.com


     
  • 1.
  • 2.

Operator-1初识Operator_云原生_47

打包镜像发布

打包发布镜像,其实最好应该修改一个镜像标签tag,这里就演示 就先这样了!make install 也不能用不知道那里有问题了 直接复制Makefile中的命令了!构建镜像并发布镜像!

[[email protected] kube-oprator1]$ ./bin/controller-gen rbac:roleName=manager-role crd webhook paths="./..." output:crd:artifacts:config=config/crd/bases
[[email protected] kube-oprator1]$ kustomize build config/crd  | kubectl apply -f -
customresourcedefinition.apiextensions.k8s.io/redis.myapp1.zhangpeng.com configured

[[email protected] kube-oprator1]$ make docker-build docker-push IMG=ccr.ccs.tencentyun.com/layatools/zpredis:v1
[[email protected] kube-oprator1]$ make deploy IMG=ccr.ccs.tencentyun.com/layatools/zpredis:v1

     
  • 1.
  • 2.
  • 3.
  • 4.
  • 5.
  • 6.

Operator-1初识Operator_云原生_48

Operator-1初识Operator_oprator_49

[[email protected] kube-oprator1]$ cd config/manager && kustomize edit set image controller=ccr.ccs.tencentyun.com/layatools/zpredis:v1
[[email protected] kube-oprator1]$ kustomize build config/default | kubectl apply -f -

     
  • 1.
  • 2.

Operator-1初识Operator_kubernetes_50
Operator-1初识Operator_kubernetes_51
恩修改成zhangpeng1
Operator-1初识Operator_kubernetes_52
创建成功了?看一下make run,but make run无法运行了?
Operator-1初识Operator_云原生_53
本地调试模式注释掉main.go SetupWebhookWithManager
Operator-1初识Operator_oprator_54
Operator-1初识Operator_云原生_55
Operator-1初识Operator_kubernetes_56

总结:

1.注意开发工具之间版本的匹配
2.make 失败时候看一下Makefile中相关命令可以手动运行一下
3.资源的清理,本地调试模式
4.接下来准备设计一个简单的Operator?

原网站

版权声明
本文为[51CTO]所创,转载请带上原文链接,感谢
https://blog.51cto.com/saynaihe/5432455