Environmental preparation
Dependent version
MAC M1
kubernetes： 1.18.3
go： 1.17.6
kubebuilder：3.1.0
Knowledge is necessary
Kubernetes Of Group、Version、Resource、Kind Shallow solution
Kubernetes It's a resource centric system ：Group、Version、Resource、Kind And other resource related data structures are particularly important .
Group
Group The resource group , stay kubernetes When grouping resources , The corresponding data structure is Group, Source path ：staging/src/k8s.io/apimachinery/pkg/apis/meta/v1/types.go
, as follows , so Group Has its own name and version ：
type APIGroup struct {
TypeMeta `json:",inline"`
Name string `json:"name" protobuf:"bytes,1,opt,name=name"`
Versions []GroupVersionForDiscovery `json:"versions" protobuf:"bytes,2,rep,name=versions"`
PreferredVersion GroupVersionForDiscovery `json:"preferredVersion,omitempty" protobuf:"bytes,3,opt,name=preferredVersion"`
ServerAddressByClientCIDRs []ServerAddressByClientCIDR `json:"serverAddressByClientCIDRs,omitempty" protobuf:"bytes,4,rep,name=serverAddressByClientCIDRs"`
}
stay kubernetes There are two resource groups in ： Resource group with group name and resource group without group name ( It's also called the core resource group Core Groups), They're all very common ：
Example ：deployment There's a group name ,pod There is no group name , Take both of them OpenAPI Put it together and compare it at a glance 
Version
Version Version , That's easy to understand ,kubernetes There are three versions of ：
Alpha： Internal test version , Such as v1alpha1
Beta： A relatively stable version that has gone through official and community testing , Such as v1beta1
Stable： Official release , Such as v1、v2
The red box is shown below , Resource group batch There is v1 and v2alpha1 Two versions , There are multiple resources in each version ：
Data structure source code is still staging/src/k8s.io/apimachinery/pkg/apis/meta/v1/types.go In file , as follows ：
type APIVersions struct {
TypeMeta `json:",inline"`
Versions []string `json:"versions" protobuf:"bytes,1,rep,name=versions"`
ServerAddressByClientCIDRs []ServerAddressByClientCIDR `json:"serverAddressByClientCIDRs" protobuf:"bytes,2,rep,name=serverAddressByClientCIDRs"`
}
Resource
Resource Resources in kubernetes It's self-evident that the importance of science and Technology , common pod、service、deployment These are resources .
stay kubernetes The resources that the environment is instantiated are resource objects (ResourceObject)
Resources are divided into persistence （Persistent Entity） And non persistent （Ephemeral Entity）, Persistence is like deployment, It will be created in etcd preservation , Non persistent, such as pod
kubernetes Ready for resources 8 Kind of operation ：create、delete、deletecollection、get、list、patch、update、watch, Each resource supports a part of it , It's in every resource API As you can see in the document
increase ：
create：Resource Object establish
Delete ：delete： Single Resource Object Delete
deletecollection： Multiple Resource Objects Delete
Change ：
patch：Resource Object Local field updateupdate：Resource Object Overall update
check ：
get： Single Resource Object obtain
list： Multiple Resource Objects obtain
watch：Resource Objects monitor
Resource support is based on namespace （namespace） In isolation
Resource object description file is often used in daily operation , It consists of five parts ：apiVersion、kind、metadata、spec、status, The picture below is official deployment Description file , Used to create 3 individual nginx pod, Just look at the red box and the text to understand the function of each part
The picture above doesn't have status, This part is used to reflect the state of the current resource object , Reflected in the data structure of resources , As shown below :
type Deployment struct {
metav1.TypeMeta `json:",inline"`
metav1.ObjectMeta `json:"metadata,omitempty" protobuf:"bytes,1,opt,name=metadata"`
Spec DeploymentSpec `json:"spec,omitempty" protobuf:"bytes,2,opt,name=spec"`
Status DeploymentStatus `json:"status,omitempty" protobuf:"bytes,3,opt,name=status"`
}
Data structure source code is still staging/src/k8s.io/apimachinery/pkg/apis/meta/v1/types.go In file , as follows ：
type APIResource struct {
Name string `json:"name" protobuf:"bytes,1,opt,name=name"`
SingularName string `json:"singularName" protobuf:"bytes,6,opt,name=singularName"`
Namespaced bool `json:"namespaced" protobuf:"varint,2,opt,name=namespaced"`
Group string `json:"group,omitempty" protobuf:"bytes,8,opt,name=group"`
Version string `json:"version,omitempty" protobuf:"bytes,9,opt,name=version"`
Kind string `json:"kind" protobuf:"bytes,3,opt,name=kind"`
Verbs Verbs `json:"verbs" protobuf:"bytes,4,opt,name=verbs"`
ShortNames []string `json:"shortNames,omitempty" protobuf:"bytes,5,rep,name=shortNames"`
Categories []string `json:"categories,omitempty" protobuf:"bytes,7,rep,name=categories"`
StorageVersionHash string `json:"storageVersionHash,omitempty" protobuf:"bytes,10,opt,name=storageVersionHash"`
}
Kind
Kind And Resource Belong to the same level of concept ,Kind Used to describe Resource The type of
Scheme
Each resource needs to have a corresponding Scheme,Scheme The structure contains gvkToType and typeToGVK Field mapping relationship ,APIServer according to Scheme To serialize and deserialize resources .
GV & GVK & GVR
GV: Api Group & Version
API Group It's related API Collection of functions
Every Group Have one or more Versions
GVK: Group Version Kind
Every GV Both contain N individual api type , be called Kinds, Different Version The same Kinds It may be different
GVR: Group Version Resource
Resource yes Kind Object ID of , Generally come Kind and Resource yes 1:1 Of , But sometimes there are 1:n The relationship between , But for the Operator It's all about 1:1 The relationship between
for instance , We are k8s Medium yaml The file has the following two lines ：
apiVersion: apps/v1 # This is GV,G yes apps,V yes v1
kind: Deployment # This is Kind
sepc: # Plus the decentralized spec Namely Resource 了
...
according to GVK K8s You can find out exactly what kind of resources you want to create , According to your definition Spec After the resources are created, they become Resource, That is to say GVR.GVK/GVR Namely K8s Resource coordinates , We created / Delete / modify / The basis of reading resources .
View all resources in the current environment , And its related properties
# ctl api-resources -o wide
View specific group The following resources , for example ：apps
# ctl api-resources --api-group apps -o wide
View the details of the specified resource , for example ：configmap
# ctl explain configmap
View all Group and Version The order of
# ctl api-versions
A quick look at official documents
In practical learning and development , When adding, deleting, modifying and querying the specified resources , Official documents are our most reliable reliance , Address ：https://kubernetes.io/docs/reference/kubernetes-api/
open deployment Documents , Here's the picture ：
And then there is API Documentation is also essential , The latest is 1.19 edition , Address ：https://v1-22.docs.kubernetes.io/docs/reference/generated/kubernetes-api/v1.22/
The picture below is deployment Of api Interface document , See the example 、path、 The request response parameters are described in detail ：
APIResources data structure
APIResource It's a common data structure , Can be used to describe resources , for example kubernetes/pkg/controller/resourcequota/resource_quota_controller_test.go There is a use for it in ：
func TestDiscoverySync(t *testing.T) {
serverResources := []*metav1.APIResourceList{
{
GroupVersion: "v1",
APIResources: []metav1.APIResource{
{Name: "pods", Namespaced: true, Kind: "Pod", Verbs: metav1.Verbs{"create", "delete", "list", "watch"}},
},
},
}
unsyncableServerResources := []*metav1.APIResourceList{
{
GroupVersion: "v1",
APIResources: []metav1.APIResource{
{Name: "pods", Namespaced: true, Kind: "Pod", Verbs: metav1.Verbs{"create", "delete", "list", "watch"}},
{Name: "secrets", Namespaced: true, Kind: "Secret", Verbs: metav1.Verbs{"create", "delete", "list", "watch"}},
},
},
}
How to Kubernetes Expand
Kubernetes It's a portable 、 Scalable open source platform , For managing containerized workloads and services , Promotes declarative configuration and Automation . Kubernetes With a huge and rapidly growing ecosystem .Kubernetes Service for 、 Support and tools are widely available .
Although now Kubernetes Is already the de facto standard for container choreography , Its own functions are also very rich and flexible , But it can't meet everyone's needs , In case of Kubernetes When the capabilities provided cannot meet our needs , We can use its powerful expansion ability to customize .
In practice , Yes kubernetes It is a common requirement for resources to perform various personalized configuration and control , For example, custom image pod How to control the number of copies 、 Master-slave relationship , And the control of various custom resources .
Kubernetes What are the extension points
As shown in the figure above , Run time from the client to the underlying container , Most places Kubernetes All reserved extension points for us , Let's look at it one by one from top to bottom
kubectl
kubectl It's what we usually do with Kubernetes The most commonly used client tools for interaction , Common operation and maintenance operations will pass kubectl To complete ,kubectl It provides us with a plug-in mechanism to facilitate extension .
kubectl The plug-in is actually based on kubectl- Any executable with prefix , perform kubectl When you plug-in, you can use kubectl The plugin name Parameters To run the plug-in .
It's like Ubuntu Use apt Management software ,mac have access to brew equally ,kubectl There are similar plug-in management tools krew , At the same time, we can learn from https://krew.sigs.Kubernetes.io/plugins/ Find out if the plug-in we need already exists
APIServer
Polymeric layer
from Kubernetes v1.7 After the version APIServer The function of aggregation layer is introduced , This feature enables every developer to aggregate API Services expose the interfaces they need , This process doesn't need to be recompiled Kubernetes Any code for .
If we submit the following resource to Kubernetes after , The user is visiting API Server's /apis/metrics.Kubernetes.io/v1beta1 When the path , Will be forwarded to the cluster metrics-server.kube-system.svc Service
apiVersion: apiregistration.Kubernetes.io/v1
kind: APIService
metadata:
name: v1beta1.metrics.Kubernetes.io
spec:
service:
name: metrics-server
namespace: kube-system
group: metrics.Kubernetes.io
version: v1beta1
insecureSkipTLSVerify: true
groupPriorityMinimum: 100
versionPriority: 100
Access control
Besides, whether from kubectl still client-go Requests from other clients will be sent to APIServer after authentication -> authentication -> Access control Steps for , Each of these steps can be extended , Among them, the most used is the expansion of access control .
Access control will go through , Change access control MutatingAdmissionWebhook, Then the access control will be verified ValidatingAdmissionWebhook, If any admission controller returns an error, the request will fail , For example, these two access controllers can do a lot of things , For example, inject sidecar, Verify resources , adjustment pod Quotas, etc .
Kubernetes resources
That we use a lot Deployment、Pod、Node Are all Kubernetes Official built-in resources , But sometimes the built-in resources can't meet our needs , You can use CR（CustomResource） That is, custom resources . Custom resources are often associated with Controller Used together , However, it should be noted that when using custom resources, you need to think about if only some configurations are possible ConfigMap It will be more suitable for , Don't abuse this feature .
Controller controller
Kubernetes The maintenance of resource status in is Controller To achieve ,Controller Will constantly try to adjust a resource to the state we describe , This is actually what we often call declarative api, declarative api The specific work behind it is Controller Dry .Controller Usually cooperate with CRD（Custom Resource Definition） Use it together
Schedule Scheduler
The scheduler is a special controller , Responsible for monitoring Pod Change and will Pod Assign to nodes , Schedulers can be replaced directly or multiple schedulers can be used , In addition, the official default scheduler also supports WebHook.
CNI The network plugin
CNI The network plugin , Full name Container Network Interface（ Container network interface ） Contains a set of plug-ins for development to configure Linux Interface and framework of network card in container . Generally, we don't do custom development of network plug-ins , But open source components , for example Flannel、Cilium, Use of cloud services Kubernetes You will also encounter some customized network plug-ins , For example, Alibaba cloud has Terway
CSI Storage plug-ins
CSI Storage plug-ins , Full name Container Storage Interface, Can pass CSI The interface supports different storage types
CRI Container runtime
CRI Container runtime , Full name Container Runtime Interface, Is a set of tools for managing container runtime and mirroring gRPC Interface , Using this interface, you can support docker、containerd When different containers run
kustomize
notice Kustomize My first reaction was this thing and helm What's the difference? ,Kustomize No template syntax , Only one binary command is needed to generate the corresponding yaml The file is very lightweight , and helm Support GoTemplate, There should also be more components , also helm adopt chart Package to release is still relatively heavyweight . Personally feel Kustomize More suitable gitops and helm It is more suitable for the distribution of application packages .
that kustomize What is it? , What's the usage? , How to use it? ？
brief introduction
kustomize It's a pass kustomization File customization kubernetes Tools for objects , It can generate some new resources through some resources , You can also customize the collection of different resources .
A typical scenario is an application , In different environments, such as production environment and test environment , its yaml The configuration is mostly the same , Only individual fields are different , It's time to take advantage of kustomize To solve ,kustomize It is also suitable for gitops workflow .
As shown in the figure above , There is one ldap Application ,/base The directory holds the basic configuration ,/overlays Configuration of different environments placed in , for example /dev、/staging,/prod These are the configurations of different environments ,/base There is one under the folder kustomization .yml file , Used for configuration .
perform kustomize build dir We can generate our final deployment yaml file , That is, we have reached the fourth step in the figure above , And then through kubectl apply -f Command to deploy .
Layout
├── base
│ ├── deployment.yaml
│ ├── kustomization.yaml
│ └── service.yaml
└── overlays
├── dev
│ ├── kustomization.yaml
│ └── patch.yaml
├── prod
│ ├── kustomization.yaml
│ └── patch.yaml
└── staging
├── kustomization.yaml
└── patch.yaml
A common project kustomize The project layout is shown above , You can see that there is one in each environment folder kustomization.yaml file , This file is similar to the configuration file , Specify the source file and some corresponding conversion files , for example patch etc.
kustomization.yml
A common kustomization.yml As shown below , Generally including apiVsersion and kind Two fixed fields
apiVersion: kustomize.config.k8s.io/v1beta1
kind: Kustomization
resources:
- manager.yaml
configMapGenerator:
- files:
- controller_manager_config.yaml
name: manager-config
kustomize Provides a rich selection of fields , In addition, you can also customize plug-ins , The following will outline the meaning of each field , When we need to use it, we know that we have such an ability , Then go again. Kustomize Official documents (https://kubectl.docs.kubernetes.io/zh/guides/) Find the corresponding API Just document
resources: Express k8s Location of resources , This can be a file , You can also point to a folder , When reading, it will be read in order , The path can be relative or absolute , If it's a relative path, it's relative to kustomization.yml The path of
crds: and resources similar , It's just crds It's our custom resource
namespace: Add... For all resources namespace
images: Change the name of the image 、tag or image digest , Without having to use patches
replicas: Modify the number of resource copies
namePrefix: Prefix the names of all resources and references
nameSuffix: Add suffixes to the names of all resources and references
patches: Add or overwrite fields on resources ,Kustomization Use patches Field to provide this function .
patchesJson6902: Each entry in the list should resolve to kubernetes Object and the... That will be applied to that object JSON patch.
patchesStrategicMerge: Use strategic merge patch standard Patch resources.
vars: Similar to specifying variables
commonAnnotations: Add... To all resources annotations If the corresponding key There is already a value , This value will be overwritten
commonAnnotations:
app.lailin.xyz/inject: agent
resources:
- deploy.yaml
commonLabels: Add... For all resources label and label selector Be careful ： This operation can be dangerous
apiVersion: kustomize.config.k8s.io/v1beta1
kind: Kustomization
commonLabels:
app: bingo
configMapGenerator Can generate config map, Each item in the list generates a configmap
secretGenerator Used to generate secret resources
generatorOptions Used to control the configMapGenerator and secretGenerator act
CRD
CRD： Custom resource definition ,Kubernetes Resource types in .
CR：Custom Resource, To use CRD The general name of the created custom resources
CRD It's used to extend Kubernetes The most common way , stay Service Mesh and Operator It is also widely used in . So if you want to be in Kubernetes For expansion and development on , It is very necessary to understand CRD Of .
Official documents ：https://kubernetes.io/docs/tasks/extend-kubernetes/custom-resources/custom-resource-definitions/
establish CRD（CustomResourceDefinition）
Create a new CustomResourceDefinition（CRD） when ,Kubernetes API Server A new version will be created for each version you specify RESTful Resource path .CRD Can be namespace , It can also be cluster wide , Can be in CRD scope Field . As with existing built-in objects , Deleting a namespace deletes all custom objects in that namespace .CustomResourceDefinition Itself is non namespace , Available for all namespaces .
Refer to the following CRD, Save its configuration in resourcedefinition.yaml In file ：
apiVersion: apiextensions.k8s.io/v1
kind: CustomResourceDefinition
metadata:
# The name must conform to the following format ：<plural>.<group>
name: crontabs.stable.example.com
spec:
# REST API Group name used ：/apis/<group>/<version>
group: stable.example.com
# REST API Version number used ：/apis/<group>/<version>
versions:
- name: v1
# Can pass served To switch every version
served: true
# There is and only one version Turn on storage
storage: true
schema:
openAPIV3Schema:
type: object
properties:
spec:
type: object
properties:
cronSpec:
type: string
image:
type: string
replicas:
type: integer
# Namespaced or Cluster
scope: Namespaced
names:
# URL Plural names used in : /apis/<group>/<version>/<plural>
plural: crontabs
# CLI Singular names used in
singular: crontab
# CamelCased The singular type of the form . Use... In the manifest file
kind: CronTab
# CLI The short name of the resource used in
shortNames:
- ct
Then create a new namespace in the following location RESTful API Endpoint ：
/apis/stable.example.com/v1/namespaces/*/crontabs/...
then , This endpoint URL Can be used to create and manage custom objects . above CRD The type defined in is CronTab.
It may take a few seconds to create an endpoint . Can monitor CustomResourceDefinition in Established When is the status of true, Or check out API Whether the resource is displayed in the resource discovery information .
Create custom objects
establish CustomResourceDefinition After the object , You can create custom objects . Custom objects can contain custom fields . These fields can contain any JSON. In the following example , cronSpec and image Custom fields are set in custom objects CronTab.CronTab The type comes from the... You created above CustomResourceDefinition Object specification .
If you will YAML Save to my-crontab.yaml：
apiVersion: "stable.example.com/v1"
kind: CronTab
metadata:
name: my-new-cron-object
spec:
cronSpec: "* * * * */5"
image: my-awesome-cron-image
And create it ：
kubectl create -f my-crontab.yaml
then , You can use kubectl management CronTab object . for example ：
kubectl get crontab
You should print such a list ：
NAME AGE
my-new-cron-object 6s
...
https://jimmysong.io/kubernetes-handbook/concepts/crd.html
https://liqiang.io/post/kubernetes-all-about-crd-part01-crd-introduction-fb14d399
Operator brief introduction
Kubernetes Is a highly scalable system , Although it has so many extension points , But generally speaking, we have more contact with Custom resources , controller , Access control , Some will be right kubectl and The scheduler makes some extensions , Most of the others can use mature open source components .
The official response to Operator Introduction to ：https://kubernetes.io/zh/docs/concepts/extend-kubernetes/operator/ ,Operator The execution process of the pattern is shown in the figure below ：
Operator follow Kubernetes Idea , It utilizes custom resource management applications and their components , Operator Patterns encapsulate the task automation code you write .
Operator Common uses include :
Deploy applications on demand
obtain / Restore backup of application state
Handle application code upgrades and related changes . for example , database schema Or additional configuration settings
Publish a service, The request does not support Kubernetes API It can also be found in applications
Simulate faults in the whole or part of the cluster to test its stability
In the absence of internal member election procedures , Select leader role for distributed application
from Operator Since the concept was put forward, there have been many tools that can help rapid and low-cost development , One of the most common is CoreOS Open source operator-sdk and k8s sig The team maintained kubebuilder.
In addition to their own development, they can also be used in https://operatorhub.io/ Find someone else's ready-made Operator To use
kubebuilder
kubebuilder First experience
# mkdir kubedev
# cd kubedev
# go mod init kubedev
go: creating new go.mod: module kubedev
# kubebuilder init --domain zise.feizhu
Error: failed to initialize project: unable to run pre-scaffold tasks of "base.go.kubebuilder.io/v3": go version 'go1.17.6' is incompatible because 'requires 1.13 <= version < 1.17'. You can skip this check using the --skip-go-version-check flag
In this case, you can add --skip-go-version-check Ignore this mistake , However, it is recommended to use the officially recommended version
# kubebuilder init --domain zise.feizhu --skip-go-version-check
Writing scaffold for you to edit...
Get controller runtime:
$ go get sigs.k8s.io/[email protected]
Update dependencies:
$ go mod tidy
Next: define a resource with:
$ kubebuilder create api
Add the following contents in the current directory , So this is a standard go module engineering ：
.
├── Dockerfile
├── Makefile # There are many script commands defined here , For example, running tests , Start execution, etc
├── PROJECT # Here is kubebuilder Some metadata information of
├── config # config Get the startup configuration under the directory
│ ├── default # Some default configurations
│ │ ├── kustomization.yaml
│ │ ├── manager_auth_proxy_patch.yaml
│ │ └── manager_config_patch.yaml
│ ├── manager # Deploy crd The required yaml
│ │ ├── controller_manager_config.yaml
│ │ ├── kustomization.yaml
│ │ └── manager.yaml
│ ├── prometheus # Monitoring indicator data collection configuration
│ │ ├── kustomization.yaml
│ │ └── monitor.yaml
│ └── rbac # Deploy the required rbac to grant authorization yaml
│ ├── auth_proxy_client_clusterrole.yaml
│ ├── auth_proxy_role.yaml
│ ├── auth_proxy_role_binding.yaml
│ ├── auth_proxy_service.yaml
│ ├── kustomization.yaml
│ ├── leader_election_role.yaml
│ ├── leader_election_role_binding.yaml
│ ├── role_binding.yaml
│ └── service_account.yaml
├── go.mod # A new Go modular , Match our project , Have basic dependencies
├── go.sum
├── hack # Script
│ └── boilerplate.go.txt
└── main.go
6 directories, 24 files
establish API(CRD and Controller)
# kubebuilder create api --group apps --version v1 --kind Application
Create Resource [y/n] # Whether to create resources
y
Create Controller [y/n] # Whether to create controller
y
After execution, we can find that some changes have taken place in the project structure
├── api
│ └── v1
│ ├── application_types.go # crd Definition
│ ├── groupversion_info.go # Main storage schema And our gvk The definition of
│ └── zz_generated.deepcopy.go # Serialize and deserialize our crd resources . This generally does not need attention
├── bin
│ └── controller-gen
├── config
│ ├── crd # Automatically generated crd file , There's no need to change here , Just modify v1 Medium go File after execution make generate that will do
│ │ ├── kustomization.yaml
│ │ ├── kustomizeconfig.yaml
│ │ └── patches
│ │ ├── cainjection_in_applications.yaml
│ │ └── webhook_in_applications.yaml
│ ├── rbac
│ │ ├── application_editor_role.yaml
│ │ ├── application_viewer_role.yaml
│ │ ├── auth_proxy_client_clusterrole.yaml
│ │ ├── auth_proxy_role.yaml
│ │ ├── auth_proxy_role_binding.yaml
│ │ ├── auth_proxy_service.yaml
│ │ ├── kustomization.yaml
│ │ ├── leader_election_role.yaml
│ │ ├── leader_election_role_binding.yaml
│ │ ├── role_binding.yaml
│ │ └── service_account.yaml
│ └── samples # Here is crd Sample file , Can be used to deploy to a cluster
│ └── apps_v1_application.yaml
├── controllers
│ ├── application_controller.go # The custom controller mainly realizes cr Of reconcile Method , By getting cr Resource information , Where our real business logic is stored
│ └── suite_test.go # Test components
13 directories, 37 files
Build and deploy CRD
kubebuilder Provided Makefile Greatly simplify the construction and deployment , Execute the following command to update the newly built CRD Deployed in kubernetes On
# make install
......
customresourcedefinition.apiextensions.k8s.io/applications.apps.zise.feizhu created
Compile and run controller
Mainly experience the basic process , Don't go deep into the source code , So make only a few changes to the code , It is used to verify whether it can take effect
main.go
if err = (&controllers.ApplicationReconciler{
Client: mgr.GetClient(),
Log: ctrl.Log.WithName("controllers").WithName("Application"), // add this line
Scheme: mgr.GetScheme(),
}).SetupWithManager(mgr); err != nil {
setupLog.Error(err, "unable to create controller", "controller", "Application")
os.Exit(1)
}
./controllers/application_controller.go
type ApplicationReconciler struct {
client.Client
Log logr.Logger
Scheme *runtime.Scheme
}
func (r *ApplicationReconciler) Reconcile(ctx context.Context, req ctrl.Request) (ctrl.Result, error) {
_ = log.FromContext(ctx)
_ = r.Log.WithValues("application",req.NamespacedName)
// your logic here
r.Log.Info("1. %v",req) // Print in parameters
r.Log.Info("2. %s",debug.Stack()) // Print stack
return ctrl.Result{}, nil
}
...
Execute the following command , Will compile and start the just modified controller：
# make run
/Users/zisefeizhu/linkun/goproject/kubedev/bin/controller-gen "crd:trivialVersions=true,preserveUnknownFields=false" rbac:roleName=manager-role webhook paths="./..." output:crd:artifacts:config=config/crd/bases
main.go:35:2: package kubedev/controllers imports github.com/go-logr/logr from implicitly required module; to add missing requirements, run:
go get github.com/go-logr/[email protected]
Error: not all generators ran successfully
run `controller-gen crd:trivialVersions=true,preserveUnknownFields=false rbac:roleName=manager-role webhook paths=./... output:crd:artifacts:config=config/crd/bases -w` to see all available markers, or `controller-gen crd:trivialVersions=true,preserveUnknownFields=false rbac:roleName=manager-role webhook paths=./... output:crd:artifacts:config=config/crd/bases -h` for usage
make: *** [manifests] Error 1
# go get github.com/go-logr/[email protected]
# make run
establish Application Examples of resources
Now? kubernetes It's deployed Application Type of CRD, And the corresponding controller It's already running , You can try to create Application Examples of types ( It's equivalent to having pod After the definition of , To create pod);
kubebuilder A deployment file of type has been automatically created ./config/samples/apps_v1_application.yaml , The contents are as follows , It's simple , Next, use this file to create Application example ：
apiVersion: apps.zise.feizhu/v1
kind: Application
metadata:
name: application-sample
spec:
# Add fields here
foo: bar
perform apply, Go to controller On the console , You can see that both new and modified operations have log output , All the new logs are in it , The code call stack is clear at a glance
make run
/Users/zisefeizhu/linkun/goproject/kubedev/bin/controller-gen "crd:trivialVersions=true,preserveUnknownFields=false" rbac:roleName=manager-role webhook paths="./..." output:crd:artifacts:config=config/crd/bases
/Users/zisefeizhu/linkun/goproject/kubedev/bin/controller-gen object:headerFile="hack/boilerplate.go.txt" paths="./..."
go fmt ./...
go vet ./...
go run ./main.go
I0210 15:26:27.993087 31662 request.go:655] Throttling request took 1.031351708s, request: GET:https://192.168.101.45:6443/apis/storage.k8s.io/v1beta1?timeout=32s
2022-02-10T15:26:29.262+0800 INFO controller-runtime.metrics metrics server is starting to listen {"addr": ":8080"}
2022-02-10T15:26:29.263+0800 INFO setup starting manager
2022-02-10T15:26:29.263+0800 INFO controller-runtime.manager starting metrics server {"path": "/metrics"}
2022-02-10T15:26:29.263+0800 INFO controller-runtime.manager.controller.application Starting EventSource {"reconciler group": "apps.zise.feizhu", "reconciler kind": "Application", "source": "kind source: /, Kind="}
2022-02-10T15:26:29.368+0800 INFO controller-runtime.manager.controller.application Starting Controller {"reconciler group": "apps.zise.feizhu", "reconciler kind": "Application"}
2022-02-10T15:26:29.368+0800 INFO controller-runtime.manager.controller.application Starting workers {"reconciler group": "apps.zise.feizhu", "reconciler kind": "Application", "worker count": 1}
2022-02-10T15:26:36.742+0800 INFO controllers.Application 1. default/application-sample
2022-02-10T15:26:36.743+0800 INFO controllers.Application 2. goroutine 420 [running]:
runtime/debug.Stack()
/Users/zisefeizhu/go/go1.17.6/src/runtime/debug/stack.go:24 +0x88
kubedev/controllers.(*ApplicationReconciler).Reconcile(0x140007443c0, {0x10399f3d8, 0x140006c5680}, {{{0x140003ac869, 0x7}, {0x1400071cc30, 0x12}}})
/Users/zisefeizhu/linkun/goproject/kubedev/controllers/application_controller.go:58 +0x184
sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).reconcileHandler(0x1400055e140, {0x10399f330, 0x14000632600}, {0x10383dfc0, 0x14000443f20})
/Users/zisefeizhu/go/pkg/mod/sigs.k8s.io/[email protected]/pkg/internal/controller/controller.go:298 +0x2ac
sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).processNextWorkItem(0x1400055e140, {0x10399f330, 0x14000632600})
/Users/zisefeizhu/go/pkg/mod/sigs.k8s.io/[email protected]/pkg/internal/controller/controller.go:253 +0x1d8
sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).Start.func1.2({0x10399f330, 0x14000632600})
/Users/zisefeizhu/go/pkg/mod/sigs.k8s.io/[email protected]/pkg/internal/controller/controller.go:216 +0x44
k8s.io/apimachinery/pkg/util/wait.JitterUntilWithContext.func1()
/Users/zisefeizhu/go/pkg/mod/k8s.io/[email protected]/pkg/util/wait/wait.go:185 +0x38
k8s.io/apimachinery/pkg/util/wait.BackoffUntil.func1(0x14000025748)
/Users/zisefeizhu/go/pkg/mod/k8s.io/[email protected]/pkg/util/wait/wait.go:155 +0x68
k8s.io/apimachinery/pkg/util/wait.BackoffUntil(0x1400052ff48, {0x10397a2e0, 0x140005585a0}, 0x1, 0x14000026300)
/Users/zisefeizhu/go/pkg/mod/k8s.io/[email protected]/pkg/util/wait/wait.go:156 +0x94
k8s.io/apimachinery/pkg/util/wait.JitterUntil(0x14000025748, 0x3b9aca00, 0x0, 0x1, 0x14000026300)
/Users/zisefeizhu/go/pkg/mod/k8s.io/[email protected]/pkg/util/wait/wait.go:133 +0x88
k8s.io/apimachinery/pkg/util/wait.JitterUntilWithContext({0x10399f330, 0x14000632600}, 0x14000720140, 0x3b9aca00, 0x0, 0x1)
/Users/zisefeizhu/go/pkg/mod/k8s.io/[email protected]/pkg/util/wait/wait.go:185 +0x88
k8s.io/apimachinery/pkg/util/wait.UntilWithContext({0x10399f330, 0x14000632600}, 0x14000720140, 0x3b9aca00)
/Users/zisefeizhu/go/pkg/mod/k8s.io/[email protected]/pkg/util/wait/wait.go:99 +0x50
created by sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).Start.func1
/Users/zisefeizhu/go/pkg/mod/sigs.k8s.io/[email protected]/pkg/internal/controller/controller.go:213 +0x308
Delete instance and stop controller
No longer need Application At instance time , Execute the following command to delete ：
# ctl delete -f config/samples/
No longer need controller When , Go to its console and use Ctrl+c Just interrupt
But in the actual production environment controller It usually runs on kubernetes In the environment , Like the one above running on kubernetes The other way is not suitable , Let's try making it docker The mirror image and then kubernetes Environment is running
take controller Made into docker Mirror image
Execute the following command to build docker Mirror and push to aliyun, The image is called
# docker login --username=zisefeizhu registry.cn-shenzhen.aliyuncs.com
Password:
Login Succeeded
# docker build -t registry.cn-shenzhen.aliyuncs.com/zisefeizhu-xm/application:v001 .
# docker push registry.cn-shenzhen.aliyuncs.com/zisefeizhu-xm/application:v001
stay kubernetes The deployment environment controller
# make deploy IMG=registry.cn-shenzhen.aliyuncs.com/zisefeizhu-xm/application:v001
/Users/zisefeizhu/linkun/goproject/kubedev/bin/controller-gen "crd:trivialVersions=true,preserveUnknownFields=false" rbac:roleName=manager-role webhook paths="./..." output:crd:artifacts:config=config/crd/bases
cd config/manager && /Users/zisefeizhu/linkun/goproject/kubedev/bin/kustomize edit set image controller=registry.cn-shenzhen.aliyuncs.com/zisefeizhu-xm/application:v001
/Users/zisefeizhu/linkun/goproject/kubedev/bin/kustomize build config/default | kubectl apply -f -
namespace/kubedev-system created
customresourcedefinition.apiextensions.k8s.io/applications.apps.zise.feizhu configured
serviceaccount/kubedev-controller-manager created
role.rbac.authorization.k8s.io/kubedev-leader-election-role created
clusterrole.rbac.authorization.k8s.io/kubedev-manager-role created
clusterrole.rbac.authorization.k8s.io/kubedev-metrics-reader created
clusterrole.rbac.authorization.k8s.io/kubedev-proxy-role created
rolebinding.rbac.authorization.k8s.io/kubedev-leader-election-rolebinding created
clusterrolebinding.rbac.authorization.k8s.io/kubedev-manager-rolebinding created
clusterrolebinding.rbac.authorization.k8s.io/kubedev-proxy-rolebinding created
configmap/kubedev-manager-config created
service/kubedev-controller-manager-metrics-service created
deployment.apps/kubedev-controller-manager created
# kubectl get pods -n kubedev-system -w
NAME READY STATUS RESTARTS AGE
kubedev-controller-manager-776899d98c-9xdfg 2/2 Running 0 9s
take kube-rbac-proxy Change to Alibaba cloud image address
Unload and clean up
Combine the previously created resources with CRD Clean it up , You can execute the following commands
# make uninstall
Simple implementation Controller
Definition CR
api/v1/application_types.go
// INSERT ADDITIONAL SPEC FIELDS - desired state of cluster
// Important: Run "make" to regenerate code after modifying this file
// Product The product to which the application belongs
Product string `json:"product,omitempty"`
}
After modification, execute make manifests generate You can find that the relevant fields have been generated , And the field annotation in the code is yaml Notes in the document
# make manifests generate
/Users/zisefeizhu/linkun/goproject/kubedev/bin/controller-gen "crd:trivialVersions=true,preserveUnknownFields=false" rbac:roleName=manager-role webhook paths="./..." output:crd:artifacts:config=config/crd/bases
/Users/zisefeizhu/linkun/goproject/kubedev/bin/controller-gen object:headerFile="hack/boilerplate.go.txt" paths="./..."
config/crd/bases/apps.lailin.xyz_applications.yaml
properties:
product:
description: Product The product to which the application belongs
type: string
Realization controller
kubebuilder It's done Operator Most of the logic required , So all you need to do is Reconcile Just implement business logic in
./controllers/application_controller.go
func (r *ApplicationReconciler) Reconcile(ctx context.Context, req ctrl.Request) (ctrl.Result, error) {
_ = log.FromContext(ctx)
_ = r.Log.WithValues("application", req.NamespacedName)
// your logic here
r.Log.Info("app changed", "ns", req.Namespace)
//r.Log.Info(fmt.Sprintf("1. %v", req)) // Print in parameters
//r.Log.Info(fmt.Sprintf("2. %s", debug.Stack())) // Print stack
return ctrl.Result{}, nil
}
After the logic is modified , Execute first make install install CRD, And then execute make run function controller
make run
/Users/zisefeizhu/linkun/goproject/kubedev/bin/controller-gen "crd:trivialVersions=true,preserveUnknownFields=false" rbac:roleName=manager-role webhook paths="./..." output:crd:artifacts:config=config/crd/bases
/Users/zisefeizhu/linkun/goproject/kubedev/bin/controller-gen object:headerFile="hack/boilerplate.go.txt" paths="./..."
go fmt ./...
go vet ./...
go run ./main.go
I0210 16:23:51.408872 35770 request.go:655] Throttling request took 1.029677459s, request: GET:https://192.168.101.45:6443/apis/storage.k8s.io/v1beta1?timeout=32s
2022-02-10T16:23:52.683+0800 INFO controller-runtime.metrics metrics server is starting to listen {"addr": ":8080"}
2022-02-10T16:23:52.684+0800 INFO setup starting manager
2022-02-10T16:23:52.684+0800 INFO controller-runtime.manager starting metrics server {"path": "/metrics"}
2022-02-10T16:23:52.684+0800 INFO controller-runtime.manager.controller.application Starting EventSource {"reconciler group": "apps.zise.feizhu", "reconciler kind": "Application", "source": "kind source: /, Kind="}
2022-02-10T16:23:52.786+0800 INFO controller-runtime.manager.controller.application Starting Controller {"reconciler group": "apps.zise.feizhu", "reconciler kind": "Application"}
2022-02-10T16:23:52.786+0800 INFO controller-runtime.manager.controller.application Starting workers {"reconciler group": "apps.zise.feizhu", "reconciler kind": "Application", "worker count": 1}
Deploy a test crd kubectl apply -f config/samples/apps_v1_application.yaml
apiVersion: apps.lailin.xyz/v1
kind: Application
metadata:
name: application-sample
spec:
# Add fields here
product: test
Then you can see that the log logic written before has been triggered
022-02-10T16:25:27.946+0800 INFO controllers.Application app changed {"ns": "default"}
Kubebuilder notes
In the generated code, we can see a lot //+kubebuilder:xxx Opening comment , Yes Go Familiar students should know that these comments are for the corresponding code generator , stay Go One of the more common routines in is to use go gennerate Generate corresponding go Code .
kubebuilder Use controller-gen（https://github.com/kubernetes-sigs/controller-tools/blob/master/cmd/controller-gen/main.go） Generate code and corresponding yaml file , This mainly includes CRD Generate 、 verification 、 Processing and WebHook Of RBAC Generation function of , Let me briefly introduce , The full version can be seen kubebuilder Official documents of （https://master.book.kubebuilder.io/quick-start.html）
CRD Generate
//+kubebuilder:subresource:status Turn on status Sub resources , After adding this comment, you can comment on status Update operation has been carried out
//+groupName=nodes.lailin.xyz Appoint groupname
//+kubebuilder:printcolumn by kubectl get xxx Add a column , This is very useful
……
CRD verification , Take advantage of this feature , Just add some comments , You can complete most of the functions that need to be verified
//+kubebuilder:default:= Set default values for fields
//+kubebuilder:validation:Pattern:=string Use regular validation fields
……
Webhook
//+kubebuilder:webhook Is used to specify the webhook How to generate , For example, we can specify to listen only Update The event webhook
RBAC Used to generate rbac Authority
//+kubebuilder:rbac
Reference documents ：
https://lailin.xyz/post/operator-01-overview.html
https://blog.csdn.net/boling_cavalry/article/details/113089414
operator The journey ( One ) More articles about
- vijosP1014 Travel agent simplified version
vijosP1014 Travel agent simplified version link :https://vijos.org/p/1014 [ Ideas ] double DP. set up ab,ab Go at the same time . use d[i][j] Express ab Node i.j, And define i>j, There is a turn ...
- 【C++ Journey of exploration 】 Part one lesson three ： first C++ Program
Content abstract 1. Part one lesson three : first C++ Program 2. Part one lesson four preview : Memory usage first C++ Program After two lessons , We already know what programming is , Programming language , The necessary software for programming ,C++ What is it? , We also installed the adapter ...
- Start Java8 The journey （ Four ） -- Four function interfaces
Preface Java8 There are many function interfaces in , There are probably dozens of them , I can't count the exact number , So at the beginning, I felt confused , But it's not that complicated at all , After all, there is no need to design a complex thing . A few words I'm learning ...
- 【Luogu1291】 Pepsi World Cup tour （ Dynamic programming , Mathematical expectation ）
[Luogu1291] Pepsi World Cup tour ( Dynamic programming , Mathematical expectation ) Topic Luogu Answer key set up \(f[i]\) It means that all have been collected \(i\) The expectation of a name Now there are two ways : Let me start with my own : \[f[i]=f[i-1]+1+ ...
- Luogu 【P1523】 The backpack of a travel agent （ Introduction to algorithms 15-1） Answer key
P1523 Travel agent simplified version Background Euclid travel agent \((Euclidean Traveling Salesman)\) The problem, that is, the problem of freight forwarders, has been perplexing mathematicians all over the world . The famous problem of computer scientists . There are no existing algorithms ...
- TiDB A journey of deep practice -- real “ Step on the pit ” experience
US group review TiDB A journey of deep practice (9000 Long writing / real “ Step on the pit ” experience ) 4 PingCAP · 154 Days ago, · 3956 Views This is a project created in 154 The theme of the day before , The information may have been sent out ...
- Bzoj3352 [ioi2009] Travel agent
Time Limit: 20 Sec Memory Limit: 128 MBSubmit: 89 Solved: 36 Description How to optimize the travel route is a very difficult calculation problem ...
- 【C++ Template tour 】 One time flexible use in the project C++ Templates (traits) The experience of - New notes
Problems and needs : Please read this article first ,[C++ Template tour ] One time flexible use in the project C++ Templates (traits) The experience of . For the questions raised in this article , I give you a new idea . talking is cheap,show me t ...
- Luogu P1291 [SHOI2002] Pepsi World Cup tour Problem solving report
P1291 [SHOI2002] Pepsi World Cup tour Title Description "-- stay 2002 year 6 The name of Pepsi Star will be on the bottle cap of any Pepsi beverage purchased before this month . Just put together the names of all Pepsi stars , You can take part in the Pepsi World Cup ...
- Cultural journey (dijstra)
2012_p4 Cultural journey (culture.cpp/c/pas) The time limit : 1 Sec Memory limit : 128 MB Submit : 43 solve : 16[ Submit ][ state ][ Discussion board ][ Propositional person : External import ] Title Description ...
Random recommendation
- [leetcode] Count Primes
Count Primes Description: Count the number of prime numbers less than a non-negative number, n click ...
- Windows The counter is used for performance monitoring （window server 2008 The server ）
Use Windows Counter One . Create a data collector set Two . Create a data collector 3、 ... and . Use the data collector 1. Modify the properties of the data collector 2. Manually enable . Manually stop the data collector set 3. Planning tasks 4. See... In the performance monitor One . Performance monitoring ...
- Jasper_chart_create a new stacked chart
How to make a stacked chart 1, prepare data Source here we will create a csv file. 2, import/configu ...
- CCF-201409-3- string matching
Problem description Question number : 201409-3 The title of the test question : string matching The time limit : 1.0s Memory limit : 256.0MB Problem description : Problem description Give a string and multiline text , Find the lines where the string appears in these words . you ...
- stay Mybatis-spring Implementation scheme of data source based on annotation on
One . The pain point Studying recently Spring-boot In the process , It involves operating the database . according to DOC introduce mybatis-spring-boot-starter, Then configure according to the routine application.properti ...
- hihocoder Image operator ( Gauss elimination ）
describe In the technology of image processing , Operators are often used to convolute images , So as to achieve the effect of smoothing the image or finding the boundary . Suppose the original drawing is H × W Matrix A, The operator matrix is D × D Matrix Op, Then the processed matrix B The size is (H-D+1) × ...
- About IOS Next click Solutions to the failure of event delegation
One . Because of some special circumstances , You need to use event delegation , For example, for dynamic creation DOM binding click event , Here you need an event delegate ( This is about : Target element and proxy element ) Target element : Dynamically created elements , Final click The event needs to be bound to the element ...
- Android Development -- Genymotion Simulator
Simulator installation http://blog.csdn.net/beiminglei/article/details/13776013 Connect ADB http://android3g.diandian.com/p ...
- POJ 2195 Going Home Minimum cost flow difficulty :1
Going Home Time Limit: 1000MS Memory Limit: 65536K Total Submissions: 17955 Accepted: 9145 Descr ...
- Swift1.2 And Xcode6.3 beta
Xcode6.3 and Swift1.2 Has been released . This release enhances Swift The compiler also gives Swift Added some new features . Details can be found here . Here we mainly focus on the more important content . Compiler improvements Swift1.2 The compiler is more ...
![[googolu] takeout rebate system - business domain name of KFC configuration applet](/img/54/d30b9c08e46760dac8c3405716941f.jpg)
![Cool background shadow effect [second with layered feeling] [picture hover style]](/img/ca/d68f2cf9f9af7b9346032b2a6e120b.jpg)
