Kubernets traifik proxy WS WSS application

background ：

The team will release a set of applications ,springboot Developed ws application . Then you need to go outside . Support ws wss agreement .jenkins finish writing sth. pipeline Release task . I remember that Tencent cloud was used when there was no container in the past cls Pending certificate mapping cvm port . My current network environment is like this ：Kubernetes 1.20.5 install traefik Practice under Tencent cloud （ Of course, the environment this time is running in tke1.20.6 above , All are built according to the above examples --- Except I built a new one namespace traefik, And will traefik Applications are installed in this namespace ！ And the reason for that is tke Of kebe-system Under the pod That's too much ！ I have ocd ）

Deployment and analysis process ：

1. About my app ：

The deployment mode of the application is statefulset, as follows ：

cat <<EOF >  xxx-gateway.yaml
apiVersion: apps/v1
kind: StatefulSet
metadata:
  name: xxx-gateway
spec:
  serviceName: xxx-gateway
  replicas: 1
  selector:
    matchLabels:
      app: xxx-gateway
  template:
    metadata:
      labels:
        app: xxx-gateway
    spec:
      containers:
        - name: xxx-gateway
          image: ccr.ccs.tencentyun.com/xxx-master/xxx-gateway:202107151002
          env:
          - name: SPRING_PROFILES_ACTIVE
            value: "official"
          - name: SPRING_APPLICATION_JSON
            valueFrom:
             configMapKeyRef:
              name: spring-config
              key: dev-config.json
          ports:
            - containerPort: 8443
          resources:
            requests:
              memory: "512M"
              cpu: "500m"
            limits:
              memory: "512M"
              cpu: "500m" 
      imagePullSecrets:                                              
        - name: tencent
---

apiVersion: v1
kind: Service
metadata:
  name: xxx-gateway
  labels:
    app: xxx-gateway
spec:
  ports:
  - port: 8443
  selector:
    app: xxx-gateway
  clusterIP: None
EOF

kubectl apply -f xxx-gateway.yaml -n official

Steal a lazy and direct copy For another application ingress yaml Modified it , as follows ：

cat <<EOF >  gateway-0-ingress.yaml
apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
  name: layaverse-gateway-0-http
  namespace: official
  annotations:
    kubernetes.io/ingress.class: traefik  
    traefik.ingress.kubernetes.io/router.entrypoints: web
spec:
  rules:
  - host: xxx-gateway-0.xxx.com
    http:
      paths:
      - pathType: Prefix
        path: /
        backend:
          service:
            name: xxx-gateway 
            port: 
              number: 8443
EOF

Deploy ingress

kubectl apply -f gateway-0-ingress.yaml

see ingress Deployment status

kubectl get ingress -n official

image.png

Um. And test it out wss（wss I use... Directly 443 Port . Certificate mount slb Layer of -- That's what I understand ! Refer to me for details traefik Configuration of ）, Let's emphasize here wscat This tool . Anyway, I looked at our back-end partner test ws Applications are online ws Tools :

image.png

That's it . Then I happened to see wscat Just install it ：

sudo apt install npm
sudo npm install -g wscat 
wscat -c wss://xxx-gateway-0.xxx.com:443/ws

image.png

Uh huh Basically, it can be confirmed that the external application is successful ？

---

Of course, the above is just my smooth assumption ！

In fact, it is connected to the back end after the proxy ws There are still various problems with the service （ At first I suspected that traefik The problem of ）, Still can't connect ！ I'm rude xxx-gateway The exposure mode of is modified to NodePort Then mount to slb layer （ stay scl Directly added ssl certificate ）, The test is OK, so it can be used directly . Let the app run first , Then study how to deal with .

2. About ws and http:

Don't worry about so much , First understand and realize my traefik How to implement agent ws Well ？

image.png

The content in the picture is taken from ：https://blog.csdn.net/fmm_sunshine/article/details/77918477

---

3. Find out whose pot it is

1. Build a simple ws application

Since I don't understand the back-end code , Then I'll find a simple ws Then use traefik Agent test ！

dockerhub Found one nodejs Of websocket Mirror image ：https://hub.docker.com/r/ksdn117/web-socket-test

Deploy it ：

cat <<EOF >  web-socket-test.yaml
apiVersion: apps/v1
kind: StatefulSet
metadata:
  name: web-socket-test
spec:
  serviceName: web-socket-test
  replicas: 1
  selector:
    matchLabels:
      app: web-socket-test
  template:
    metadata:
      labels:
        app: web-socket-test
    spec:
      containers:
        - name: web-socket-test
          image: ksdn117/web-socket-test
          ports:
            - containerPort: 8010
              name: web
            - containerPort: 8443
              name: ssl
          resources:
            requests:
              memory: "512M"
              cpu: "500m"
            limits:
              memory: "512M"
              cpu: "500m"
---

apiVersion: v1
kind: Service
metadata:
  name: web-socket-test
  labels:
    app: web-socket-test
spec:
  type: NodePort
  ports:
  - port: 8010
    targetPort: 8010
    protocol: TCP
    name: web
  - port: 8443
    targetPort: 8443
    name: ssl
    protocol: TCP
  selector:
    app: web-socket-test
EOF

notes ： I added... To the configuration file here type:NodePort

kubectl  apply -f web-socket-test.yaml
kubectl get pods 
kubectl get svc 

image.png

2. Inside wscat test ws Is the service connected

Connect internally first container pod ip Test the service ：

image.png

wscat --connect ws://172.22.0.230:8010

kubectl logs -f web-socket-test-0

image.png

3.traefik External agency ws Apply and test

traefik Normal external exposure services can use ingress There are also ways to ingressroute I'll try it ：

1. ingressroute The way

cat <<EOF >  web-socket-ingressroute.yaml
apiVersion: traefik.containo.us/v1alpha1
kind: IngressRoute
metadata:
  name: web-socket-test-http
  namespaces: default
spec:
  entryPoints:
  - web
  routes:
  - match: Host(`web-socket-test.xxx.com`)
    kind: Rule
    services:
      - name: web-socket-test
        port: 8010
 EOF
 kubectl apply -f web-socket-ingressroute.yaml

image.png

wscat Test the connection :

image.png

There is no problem with this measurement ？

Delete ingress

 kubectl delete -f web-socket-ingressroute.yaml

2. ingress The way

Straighten it up ingress The way ：

cat <<EOF >  web-socket-ingress.yaml
apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
  name: web-socket-test
  namespace: default
  annotations:
    kubernetes.io/ingress.class: traefik  
    traefik.ingress.kubernetes.io/router.entrypoints: web
spec:
  rules:
  - host: web-socket-test.layame.com
    http:
      paths:
      - pathType: Prefix
        path: /
        backend:
          service:
            name: web-socket-test 
            port: 
              number: 8010
 EOF
 kubectl apply -f web-socket-ingress.yaml

image.png

wscat --connect wss://web-socket-test.xxx.com:443

image.png

The basic completion of pot throwing is not the problem of my infrastructure at least ..... Let the backend partners test to see what's wrong . From my agent level, there is no problem ！

About other ：

Of course, read some blogs and add passHostHeader: true The configuration of

1. ingressroute：

image.png

2. ingress

ingress:traefik.ingress.kubernetes.io/service.passhostheader: "true"

If there are questions You can try the above way ！