Ebpf cilium practice (1) - team based network isolation

​ stay Rainbond In the cluster , Each team corresponds to the bottom Kubernetes One of the Namespace , Due to the underlying network previously used, it is impossible to Namespace Level network management , So in Rainbond Between different teams in the same cluster , So components can freely access each other , Users cannot make any restrictions on this , This also leads to the security hidden danger of the underlying network . Now by cilium Providing network services Kubernetes Clustering can solve this problem well , Users can use their own needs , Develop for each team 、 Network policy for each component , Strengthen the underlying network management , Realize the security control of network layer .

Use cilium As Kubernetes Network services

• When installing from the host , modify network.plugin The value is none

  

• install helm

wget https://goodrain-pkg.oss-cn-shanghai.aliyuncs.com/pkg/helm && chmod +x helm && mv helm /usr/local/bin/

• Deploy cilium

helm repo add cilium https://helm.cilium.io/helm install cilium cilium/cilium --version 1.11.2 --namespace kube-system --set operator.replicas=1kubectl get pods --all-namespaces -o custom-columns=NAMESPACE:.metadata.namespace,NAME:.metadata.name,HOSTNETWORK:.spec.hostNetwork --no-headers=true | grep '<none>' | awk '{print "-n "$1" "$2}' | xargs -L 1 -r kubectl delete pod

• verification cilium

download cilium Command line tools

curl -L --remote-name-all https://github.com/cilium/cilium-cli/releases/latest/download/cilium-linux-amd64.tar.gz{,.sha256sum}sha256sum --check cilium-linux-amd64.tar.gz.sha256sumsudo tar xzvfC cilium-linux-amd64.tar.gz /usr/local/binrm cilium-linux-amd64.tar.gz{,.sha256sum}

• Confirm the status

$ cilium status --wait/¯¯\/¯¯\__/¯¯\    Cilium:         OK\__/¯¯\__/    Operator:       OK/¯¯\__/¯¯\    Hubble:         disabled\__/¯¯\__/    ClusterMesh:    disabled\__/DaemonSet         cilium             Desired: 2, Ready: 2/2, Available: 2/2Deployment        cilium-operator    Desired: 2, Ready: 2/2, Available: 2/2Containers:       cilium-operator    Running: 2cilium             Running: 2Image versions    cilium             quay.io/cilium/cilium:v1.9.5: 2cilium-operator    quay.io/cilium/operator-generic:v1.9.5: 2

• Test network connectivity （ When testing domestic servers , Tests involving external networks may fail , Does not affect normal use ）

$ cilium connectivity test️  Monitor aggregation detected, will skip some flow validation steps [k8s-cluster] Creating namespace for connectivity check...(...)--------------------------------------------------------------------------------------------------------------------- Test Report--------------------------------------------------------------------------------------------------------------------- 69/69 tests successful (0 warnings)

​

Set up team network isolation

Cilium The network isolation strategy of follows the white list mechanism , Without creating a network policy , There are no restrictions on the network , Specify the type of pod After the collection creates the network policy , In addition to the access addresses allowed in the policy , All other requests will be rejected .

• Preparation

  • Create two development teams and test teams , The English name is set to dev and test
  • Create... Under the development team and test team nginx-dev and nginx-test Components , Open the internal port , The internal domain names are set to nginx-dev and nginx-test
  • Create client components under the development and test team

• No restrictions

  Without restrictions , All services between teams can communicate freely , Without any special restrictions

• Restrictions allow only components within the team to access each other , Isolate other teams from visiting

  In actual production , Development may be deployed simultaneously within a cluster 、 test 、 Production and other teams , For security reasons , Network isolation needs to be made for each team , Prohibit other teams from accessing them , Let's take the development team as an example to illustrate how to restrict the access of other teams .

• Cilium Network policy file (dev-ingress.yaml)

apiVersion: "cilium.io/v2"kind: CiliumNetworkPolicymetadata:name: "dev-namespace-ingress"spec:endpointSelector:matchLabels:"k8s:io.kubernetes.pod.namespace": devingress:- fromEndpoints:- matchLabels:"k8s:io.kubernetes.pod.namespace": dev

• Create a strategy

kubectl create -f dev-ingress.yaml -n dev

• Confirm strategy

$ kubectl get CiliumNetworkPolicy -ANAMESPACE   NAME                    AGEdev         dev-namespace-ingress   39s

• The test results

• Set the... Under the development team nginx-dev Components only allow components under the test team to access

  In some cases , Some components have more stringent safety requirements , It may only allow access to some components that meet the requirements within the team , Let's say nginx-dev As an example to illustrate how to restrict access to only some components .

• Cilium Network policy file (nginx-dev-ingress0.yaml)

apiVersion: "cilium.io/v2"kind: CiliumNetworkPolicymetadata:name: "nginx-dev-ingress"spec:endpointSelector:matchLabels:name: grc156cbingress:- fromEndpoints:- matchLabels:name: 

• Create a strategy

kubectl create -f nginx-dev-ingress0.yaml -n dev

• Confirm strategy

$ kubectl get CiliumNetworkPolicy -ANAMESPACE   NAME                    AGEdev         nginx-dev-ingress0       85s

• The test results

• Set the development team to allow the components under the team to access at the same time , Allow the development team to nginx-dev Components are accessed by any component in the test team

  When the team network isolation is set , Sometimes it is necessary to temporarily open some components to other teams for debugging , Let's say nginx-dev Component as an example to illustrate how to open the access rights of external teams when setting network isolation .

• Cilium Network policy file (nginx-dev-ingress1.yaml)

apiVersion: "cilium.io/v2"kind: CiliumNetworkPolicymetadata:name: "nginx-dev-ingress1"spec:endpointSelector:matchLabels:name: grc156cbingress:- fromEndpoints:- matchLabels:"k8s:io.kubernetes.pod.namespace": test

• Create a strategy

kubectl create -f dev-ingress.yaml -n devkubectl create -f nginx-dev-ingress.yaml -n dev

• Confirm strategy

$ kubectl get CiliumNetworkPolicy -ANAMESPACE   NAME                    AGEdev         dev-namespace-ingress   19sdev         nginx-dev-ingress1      12s

• The test results