OpenShift 4 - Deploy Redis Cluster with Operator

《OpenShift 4.x HOL教程汇总》
说明：本文已经在OpenShift 4.10 环境中验证

在 OpenShift 中提供了丰富的 PaaS 软件,One of them is the Enterprise Edition Redis 集群软件.企业版 Redis Cluster software is passed OpenShift 的 OperatorHub 安装部署的,本文介绍如何安装 Redis Enterprise Operator,and use its configuration Redis 集群环境.

Redis 企业集群

Redis An enterprise cluster is a group with shared nothing/Nodes with symmetric architecture.“不共享”Refers to the various running inside a node Redis 分片（Redis 实例）互不感知,相互隔离,防止相互影响. “对称”是指 Redis Enterprise Cluster nodes all know the same thing：They each have all the information they need to take over the role of the cluster,This means that the cluster can survive node shutdown or replacement.更重要的是,It can be scaled linearly simply by adding more nodes.

Redis Enterprise clusters provide management API、图形 UI and proxy functions,Provides a single endpoint for applications、Sharded database transparent to developers and applications.It is also responsible for data replication、Shard placement and shard failover in a cluster.

在 Redis Enterprise Operator Contains the following two CRD 对象：

• Redis 企业集群（REC）：一个用于创建 Redis enterprise cluster API.每个 Operator Deployment supports only one cluster.
• Redis 企业数据库（REDB）：One for creating in Redis running on an enterprise cluster Redis 数据库的 API.

使用 Redis Enterprise Operator 创建 Redis Cluster 和 DB 过程如下：

1.一个 Redis Enterprise Cluster 自定义资源（“CR”）在 Operator 中被创建.
2. 集群 StatefulSet、Service Rigger、集群管理 Secret 、RS/UI 服务被创建.
3. 一个 Redis Enterprise DB 的 CR 在 Operator 中被创建.
4. Redis Enterprise DB created on the cluster,Database access credentials are stored in Kubernetes Secret 对象中.
5. Service Rigger Discover new databases and configure them Kubernetes 服务.
6. Application workloads use databases Secret and services to access Redis 数据.

安装 Redis Operator 并创建 Redis 集群

集群要求

为了在 OpenShift Deploy and run in the cluster Redis 集群,OpenShift The cluster needs to have at least 3 个 Worker 节点.At least there is 3GB Available memory space and 10GB 存储空间.

准备环境

1. 创建项目.

$ REDIS_PROJ=my-redis
$ oc new-project $REDIS_PROJ

2. 创建 SecurityContextConstraints 对象.

$ cat << EOF | oc -n my-redis apply -f -
kind: SecurityContextConstraints
apiVersion: security.openshift.io/v1
metadata:
  name: redis-enterprise-scc
allowPrivilegedContainer: false
allowedCapabilities:
  - SYS_RESOURCE
runAsUser:
  type: MustRunAs
  uid: 1001
FSGroup:
  type: MustRunAs
  ranges: 1001,1001
seLinuxContext:
  type: RunAsAny
EOF

3. 执行命令,为 ServiceAccount 提供 redis-enterprise-scc 权限.

 $ oc adm policy add-scc-to-user redis-enterprise-scc system:serviceaccount:$REDIS_PROJ

安装 Redis Enterprise Operator,创建 Redis 集群

1. 在 OperatorHub 中找到 Redis Enterprise Operator ,然后将其安装到 my-redis 命名空间下.
   
2. 在安装好的 Redis Enterprise Operator Create one with the following configuration RedisEnterpriseCluster 实例.注意：将 uiServiceType 设为 LoadBalancer .

apiVersion: app.redislabs.com/v1
kind: RedisEnterpriseCluster
metadata:
  name: rec
  namespace: my-redis
spec:
  redisEnterpriseNodeResources:
    limits:
      cpu: 2000m
      memory: 2Gi
    requests:
      cpu: 2000m
      memory: 2Gi
  bootstrapperImageSpec:
    repository: registry.connect.redhat.com/redislabs/redis-enterprise-operator
  persistentSpec:
    enabled: true
  redisEnterpriseServicesRiggerImageSpec:
    repository: registry.connect.redhat.com/redislabs/services-manager
  redisEnterpriseImageSpec:
    imagePullPolicy: IfNotPresent
    repository: registry.connect.redhat.com/redislabs/redis-enterprise
  nodes: 3
  uiServiceType: ClusterIP
  username: [email protected]

3. 完成后可以在 OpenShift 的 “拓扑” to see deployment resources and status.
   
4. 查看 Service.

$ oc get svc
NAME                                TYPE        CLUSTER-IP       EXTERNAL-IP   PORT(S)             AGE
rec                                 ClusterIP   None             <none>        9443/TCP,8001/TCP   16m
rec-prom                            ClusterIP   None             <none>        8070/TCP            16m
rec-ui                              ClusterIP   172.30.210.112   <none>        8443/TCP            16m
redis-enterprise-operator-service   ClusterIP   172.30.189.212   <none>        8443/TCP            17m

5. (可选) 根据 rec-ui 服务创建 Route,然后获取 Route 的访问地址.There is also login Redis Username and password for the console.

$ oc create route passthrough rec-ui --service=rec-ui --insecure-policy=Redirect
$ oc get route rec-ui -o jsonpath=https://'{.spec.host}'; echo
$ oc get secret rec -o jsonpath="{.data.username}" | base64 --decode; echo
$ oc get secret rec -o jsonpath="{.data.password}" | base64 --decode; echo

6. (可选) 登录 Redis 控制台,可以看到以下 create new database 界面.
   
   Also accessible cluster 和 nodes 查看.
   
7. Create the following configuration RedisEnterpriseDatabase 实例

apiVersion: app.redislabs.com/v1alpha1
kind: RedisEnterpriseDatabase
metadata:
  name: redb
  namespace: my-redis
spec:
  redisEnterpriseCluster:
    name: rec
  tlsMode: disable

8. 完成后可以在 “所有实例” 中查看状态.
   
   此时在 Redis The console is also available database 中看到 redb .
   
9. 查看 Service 和 Pod,Confirm name rec-0、rec-1、rec-2 的 Pod 中的 2 个容器都运行正常.

$ oc get svc
NAME                                TYPE        CLUSTER-IP       EXTERNAL-IP   PORT(S)             AGE
rec                                 ClusterIP   None             <none>        9443/TCP,8001/TCP   76m
rec-prom                            ClusterIP   None             <none>        8070/TCP            76m
rec-ui                              ClusterIP   172.30.210.112   <none>        8443/TCP            76m
redb                                ClusterIP   172.30.105.156   <none>        17933/TCP           60m
redb-headless                       ClusterIP   None             <none>        17933/TCP           60m
redis-enterprise-operator-service   ClusterIP   172.30.189.212   <none>        8443/TCP            76m
 
$ oc get pod
NAME                                         READY   STATUS    RESTARTS   AGE
rec-0                                        2/2     Running   0          65m
rec-1                                        2/2     Running   0          64m
rec-2                                        2/2     Running   0          58m
rec-services-rigger-fdff5f664-z2k68          1/1     Running   0          65m
redis-enterprise-operator-767d5f6b67-lxg8r   2/2     Running   0          75m

10. 获取访问 redb 的端口和密码

$ oc get secret redb-redb -o jsonpath="{.data.service_names}" | base64 --decode
redb, redb-headless
$ oc get secret redb-redb -o jsonpath="{.data.port}" | base64 --decode
17933
$ oc get secret redb-redb -o jsonpath="{.data.password}" | base64 --decode
PW7WIEKQ

11. 进入名为 rec-0 的 pod,然后使用 redis-cli command and corresponding port and password access redb,Finally set and get name.

$ oc debug pod/rec-0
Defaulting container name to redis-enterprise-node.
Use 'oc describe pod/rec-0-debug -n redis' to see all of the containers in this pod.

Starting pod/rec-0-debug ...
Pod IP: 10.130.0.19
If you don't see a command prompt, try pressing enter.
sh-4.4$ redis-cli -h redb -p 17933
redb:17933> auth PW7WIEKQ

OK
redb:17933> set name lxy
OK
redb:17933> get name
"lxy"

参考

https://medium.com/@moreal70/redis-on-openshift-at-ibm-cloud-2c9a34f46b2b
https://community.intel.com/t5/Blogs/Thought-Leadership/Big-Ideas/Validating-Redis-Enterprise-Operator-on-Bare-Metal-with-Red-Hat/post/1386198
https://developer.redis.com/create/openshift/
https://darshanadinushal.blogspot.com/2021/02/redis-enterprise-on-openshift-to-manage.html
https://developer.redis.com/create/kubernetes/kubernetes-operator/