Kubernetes 1.20.5 setting up Sentinel

background ：

The back-end application team prepares a wave springcloud framework , The configuration center uses Alibaba open source nacos. If there is no accident, it should be one higher sentinel Did the test ...... Make one demo Let's get started .

kubernetes The structure sentinel There are few cases . Take a look or springcloud More buckets for the whole family . There are still a few Alibaba open source . Baidu or Google Search for sentinel Most of them are redis Sentinel mode ... A little sad .

notes ： The construction method can refer to ：https://blog.csdn.net/fenglailea/article/details/92436337?utm_term=k8s%E9%83%A8%E7%BD%B2Sentinel&utm_medium=distribute.pc_aggpage_search_result.none-task-blog-2~all~sobaiduweb~default-0-92436337&spm=3001.4430.

One . build sentinel-dashboard:

1. Custom creation sentinel-dashboard image Mirror image

Um. Of course, I don't like it docker The term "mirror image" . Or use it image Well . In the blog cited above 1.6.1 The version of ？ He ran away and committed obsessive-compulsive disorder , The latest version is 1.8.1 according to foxiswho Modify the configuration file of the boss to mirror .

vim Dockerfile

FROM openjdk:11.0.3-jdk-stretch

MAINTAINER [email protected]

ARG version
ARG port

# sentinel version
ENV SENTINEL_VERSION ${version:-1.8.1}
#PORT
ENV PORT ${port:-8858}
ENV JAVA_OPT=""
#
ENV PROJECT_NAME sentinel-dashboard
ENV SERVER_HOST localhost
ENV SERVER_PORT 8858
ENV USERNAME sentinel
ENV PASSWORD sentinel


# sentinel home
ENV SENTINEL_HOME  /opt/
ENV SENTINEL_LOGS  /opt/logs

#tme zone
RUN rm -rf /etc/localtime \
&& ln -s /usr/share/zoneinfo/Asia/Shanghai /etc/localtime

# create logs
RUN mkdir -p ${SENTINEL_LOGS}

# get the version
#RUN cd /  \
# && wget https://github.com/alibaba/Sentinel/releases/download/${SENTINEL_VERSION}/sentinel-dashboard-${SENTINEL_VERSION}.jar -O sentinel-dashboard.jar \
# && mv sentinel-dashboard.jar ${SENTINEL_HOME} \
# && chmod -R +x ${SENTINEL_HOME}/*jar
# test file
COPY sentinel-dashboard.jar ${SENTINEL_HOME}

# add scripts
COPY scripts/* /usr/local/bin/
RUN chmod +x /usr/local/bin/docker-entrypoint.sh \
&& ln -s /usr/local/bin/docker-entrypoint.sh /opt/docker-entrypoint.sh

#
RUN chmod -R +x ${SENTINEL_HOME}/*jar

VOLUME ${SENTINEL_LOGS}

WORKDIR  ${SENTINEL_HOME}

EXPOSE ${PORT} 8719


CMD java ${JAVA_OPT} -jar sentinel-dashboard.jar

ENTRYPOINT ["docker-entrypoint.sh"]

notes ： The bosses Dockerfile in The exposed port is 8200, Because of seeing sentinel The exposed ports are 8518 I just put dockerfile Revised . And then put https://github.com/alibaba/Sentinel/releases Downloaded 1.8.1 Version of jar The package was renamed sentinel-dashboard.jar Put it in the current directory

image.png

image.png

dc Directory can be ignored , Original project copy since https://github.com/foxiswho/docker-sentinel

cat scripts/docker-entrypoint.sh

#!/bin/bash

# Licensed to the Apache Software Foundation (ASF) under one or more
# contributor license agreements.  See the NOTICE file distributed with
# this work for additional information regarding copyright ownership.
# The ASF licenses this file to You under the Apache License, Version 2.0
# (the "License"); you may not use this file except in compliance with
# the License.  You may obtain a copy of the License at
#
#     http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.

#===========================================================================================
# Java Environment Setting
#===========================================================================================
error_exit ()
{
    echo "ERROR: $1 !!"
    exit 1
}

[ ! -e "$JAVA_HOME/bin/java" ] && JAVA_HOME=$HOME/jdk/java
[ ! -e "$JAVA_HOME/bin/java" ] && JAVA_HOME=/usr/java
[ ! -e "$JAVA_HOME/bin/java" ] && error_exit "Please set the JAVA_HOME variable in your environment, We need java(x64)!"

export JAVA_HOME
export JAVA="$JAVA_HOME/bin/java"
export BASE_DIR=$(dirname $0)/..
export CLASSPATH=.:${BASE_DIR}/conf:${CLASSPATH}

#===========================================================================================
# JVM Configuration
#===========================================================================================
# Get the max heap used by a jvm, which used all the ram available to the container.
if [ -z "$MAX_POSSIBLE_HEAP" ]
then
	MAX_POSSIBLE_RAM_STR=$(java -XX:+UnlockExperimentalVMOptions -XX:MaxRAMFraction=1 -XshowSettings:vm -version 2>&1 | awk '/Max\. Heap Size \(Estimated\): [0-9KMG]+/{ print $5}')
	MAX_POSSIBLE_RAM=$MAX_POSSIBLE_RAM_STR
	CAL_UNIT=${MAX_POSSIBLE_RAM_STR: -1}
	if [ "$CAL_UNIT" == "G" -o "$CAL_UNIT" == "g" ]; then
		MAX_POSSIBLE_RAM=$(echo ${MAX_POSSIBLE_RAM_STR:0:${#MAX_POSSIBLE_RAM_STR}-1} `expr 1 \* 1024 \* 1024 \* 1024` | awk '{printf "%d",$1*$2}')
	elif [ "$CAL_UNIT" == "M" -o "$CAL_UNIT" == "m" ]; then
		MAX_POSSIBLE_RAM=$(echo ${MAX_POSSIBLE_RAM_STR:0:${#MAX_POSSIBLE_RAM_STR}-1} `expr 1 \* 1024 \* 1024` | awk '{printf "%d",$1*$2}')
	elif [ "$CAL_UNIT" == "K" -o "$CAL_UNIT" == "k" ]; then
		MAX_POSSIBLE_RAM=$(echo ${MAX_POSSIBLE_RAM_STR:0:${#MAX_POSSIBLE_RAM_STR}-1} `expr 1 \* 1024` | awk '{printf "%d",$1*$2}')
	fi
	MAX_POSSIBLE_HEAP=$[MAX_POSSIBLE_RAM/4]
fi

# Dynamically calculate parameters, for reference.
Xms=$MAX_POSSIBLE_HEAP
Xmx=$MAX_POSSIBLE_HEAP
Xmn=$[MAX_POSSIBLE_HEAP/2]
# Set for `JAVA_OPT`.
JAVA_OPT="${JAVA_OPT} -server "
if [ x"${MAX_POSSIBLE_HEAP_AUTO}" = x"auto" ];then
    JAVA_OPT="${JAVA_OPT} -Xms${Xms} -Xmx${Xmx} -Xmn${Xmn}"
fi
#-XX:+UseCMSCompactAtFullCollection
#JAVA_OPT="${JAVA_OPT} -XX:+UseConcMarkSweepGC -XX:CMSInitiatingOccupancyFraction=70 -XX:+CMSParallelRemarkEnabled -XX:SoftRefLRUPolicyMSPerMB=0 -XX:+CMSClassUnloadingEnabled -XX:SurvivorRatio=8 "
#JAVA_OPT="${JAVA_OPT} -verbose:gc -Xloggc:/dev/shm/rmq_srv_gc.log -XX:+PrintGCDetails"
#JAVA_OPT="${JAVA_OPT} -XX:-OmitStackTraceInFastThrow"
#JAVA_OPT="${JAVA_OPT}  -XX:-UseLargePages"
#JAVA_OPT="${JAVA_OPT} -Djava.ext.dirs=${JAVA_HOME}/jre/lib/ext:${BASE_DIR}/lib"
#JAVA_OPT="${JAVA_OPT} -Xdebug -Xrunjdwp:transport=dt_socket,address=9555,server=y,suspend=n"
JAVA_OPT="${JAVA_OPT} -Dserver.port=${PORT} "
JAVA_OPT="${JAVA_OPT} -Dcsp.sentinel.log.dir=${SENTINEL_LOGS} "
JAVA_OPT="${JAVA_OPT} -Djava.security.egd=file:/dev/./urandom"
JAVA_OPT="${JAVA_OPT} -Dproject.name=${PROJECT_NAME} "
JAVA_OPT="${JAVA_OPT} -Dcsp.sentinel.app.type=1 "
JAVA_OPT="${JAVA_OPT} -Dsentinel.dashboard.auth.username=${USERNAME} "
JAVA_OPT="${JAVA_OPT} -Dsentinel.dashboard.auth.password=${PASSWORD} "
JAVA_OPT="${JAVA_OPT} -Dcsp.sentinel.dashboard.server=${SERVER_HOST:-localhost}:${SERVER_PORT:-8558} "
JAVA_OPT="${JAVA_OPT} ${JAVA_OPT_EXT}"
JAVA_OPT="${JAVA_OPT} -jar sentinel-dashboard.jar "
JAVA_OPT="${JAVA_OPT} -cp ${CLASSPATH}"
echo "JAVA_OPT============"
echo "JAVA_OPT============"
echo "JAVA_OPT============"
echo $JAVA_OPT

$JAVA ${JAVA_OPT} [email protected]

Still copy the startup file of the boss . But notice ... The boss also wrote the port here 8200.... Remember to revise

Um. Start build Mirror image

docker build -t ccr.ccs.tencentyun.com/xxxx/sentinel:1.8.1 .
docker push ccr.ccs.tencentyun.com/xxxx/sentinel:1.8.1

image.png

by the way Can I use crictl Command operation ？crictl ctr I won't support it build........ Can you consider using

buildkit Build a mirror image ？

2. stay kubernetes Deployment in cluster sentinel

stay Kubernetes 1.20.5 build nacos We have established nacos namespace. sentinel It's also deployed in sentinel The namespace has not done too much complex configuration . It's easy to run here demo, Go through the process first

1. Deploy configmap

cat config.yaml

apiVersion: v1
kind: ConfigMap
metadata:
  name: sentinel-cm
data:
  sentinel.server.host: "sentinel"
  sentinel.server.port: "8858"
  sentinel.dashboard.auth.username: "sentinel111111"
  sentinel.dashboard.auth.password: "W3$ti$aifffdfGEqjf.xOkZ"

notes ： there sentinel.server.host What I write here is the service name , There is no abnormal startup yet . Is it normal to enter a fqdn? sentinel.nacos.svc.cluster.local So ？（ Yes, of course my domain No cluster.local）.

kubectl apply -f config.yaml -n nacos

2 Deploy sentinel statefulset

cat pod.yaml

apiVersion: apps/v1
kind: StatefulSet
metadata:

  name: sentinel
  labels:
    app: sentinel
spec:
  serviceName: sentinel
  replicas: 1
  selector:
    matchLabels:
      app: sentinel
  template:
    metadata:
      labels:
        app: sentinel
    spec:
      containers:
        - name: sentinel
          image: ccr.ccs.tencentyun.com/XXXX/sentinel:1.8.1
          imagePullPolicy: IfNotPresent
          resources:
            limits:
              cpu: 450m
              memory: 1024Mi
            requests:
              cpu: 400m
              memory: 1024Mi
          env:
            - name: TZ
              value: Asia/Shanghai
            - name: JAVA_OPT_EXT
              value: "-Dserver.servlet.session.timeout=7200 "
            - name: SERVER_HOST
              valueFrom:
                configMapKeyRef:
                  name: sentinel-cm
                  key: sentinel.server.host
            - name: SERVER_PORT
              valueFrom:
                configMapKeyRef:
                  name: sentinel-cm
                  key: sentinel.server.port
            - name: USERNAME
              valueFrom:
                  configMapKeyRef:
                    name: sentinel-cm
                    key: sentinel.dashboard.auth.username
            - name: PASSWORD
              valueFrom:
                  configMapKeyRef:
                    name: sentinel-cm
                    key: sentinel.dashboard.auth.password
          ports:
            - containerPort: 8858
            - containerPort: 8719
          volumeMounts:
            - name: vol-log
              mountPath: /opt/logs
      volumes:
        - name: vol-log
          hostPath:
            path: /www/k8s/foxdev/sentinel/logs
            type: Directory

kubectl  apply -f pod.yaml -n nacos

Be careful ： Be lazy volumes I don't want to mount. It's a test In three work The nodes are all gone /www/k8s/foxdev/sentinel/logs Catalog . Direct copy foxiswho The configuration of the basic .

3. Deploy service service

cat svc.yaml

apiVersion: v1
kind: Service
metadata:

  name: sentinel
  labels:
    app: sentinel
spec:
  type: ClusterIP
  ports:
    - port: 8858
      targetPort: 8858
      name: web
    - port: 8719
      targetPort: 8719
      name: api
  selector:
    app: sentinel

kubectl apply -f svc -n nacos

4. Verify whether the service is normal

kubectl get pod,svc -n nacos
kubectl logs -f sentinel-0 -n nacos

image.png

5. ingress External exposure sentinel dashboard

cat ingress.yaml

apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
  name: sentinel-http
  namespace: nacos
  annotations:
    kubernetes.io/ingress.class: traefik  
    traefik.ingress.kubernetes.io/router.entrypoints: web
spec:
  rules:
  - host: sentinel.saynaihe.com 
    http:
      paths:
      - pathType: Prefix
        path: /
        backend:
          service:
            name: sentinel
            port:
              number: 8858

Input configmap User name and password set in

image.png

Access control ：

image.png

Real-time monitoring , Request link The terms "flow control rule" and "downgrade rule" are personal favorites ..... We will study and use it later .