docker-compose启动redis集群

1. 伪集群安装

docker-compose.yml

创建redis-cluster目录，目录下创建docker-compose.yml文件如下

version: '2.2'

x-image:
 &default-image
 bitnami/redis-cluster:7.0
x-restart:
 &default-restart
 always

services:
  redis-node-0:
    image: *default-image
    restart: *default-restart
    container_name: redis-node-0
    ports:
      - 6379:6379
    volumes:
      - ./redis-cluster_data-0:/bitnami/redis/data
    environment:
      - '[email protected]'
      - 'REDIS_NODES=redis-node-0 redis-node-1 redis-node-2 redis-node-3 redis-node-4 redis-node-5'

  redis-node-1:
    image: *default-image
    restart: *default-restart
    container_name: redis-node-1
    ports:
      - 16379:6379
    volumes:
      - ./redis-cluster_data-1:/bitnami/redis/data
    environment:
      - '[email protected]'
      - 'REDIS_NODES=redis-node-0 redis-node-1 redis-node-2 redis-node-3 redis-node-4 redis-node-5'

  redis-node-2:
    image: *default-image
    restart: *default-restart
    container_name: redis-node-2
    ports:
      - 26379:6379
    volumes:
      - ./redis-cluster_data-2:/bitnami/redis/data
    environment:
      - '[email protected]'
      - 'REDIS_NODES=redis-node-0 redis-node-1 redis-node-2 redis-node-3 redis-node-4 redis-node-5'

  redis-node-3:
    image: *default-image
    restart: *default-restart
    container_name: redis-node-3
    ports:
      - 36379:6379
    volumes:
      - ./redis-cluster_data-3:/bitnami/redis/data
    environment:
      - '[email protected]'
      - 'REDIS_NODES=redis-node-0 redis-node-1 redis-node-2 redis-node-3 redis-node-4 redis-node-5'

  redis-node-4:
    image: *default-image
    restart: *default-restart
    container_name: redis-node-4
    ports:
      - 46379:6379
    volumes:
      - ./redis-cluster_data-4:/bitnami/redis/data
    environment:
      - '[email protected]'
      - 'REDIS_NODES=redis-node-0 redis-node-1 redis-node-2 redis-node-3 redis-node-4 redis-node-5'
  redis-node-5:
    image: *default-image
    restart: *default-restart
    container_name: redis-node-5
    ports:
      - 56379:6379
    volumes:
      - ./redis-cluster_data-5:/bitnami/redis/data
    depends_on:
      - redis-node-0
      - redis-node-1
      - redis-node-2
      - redis-node-3
      - redis-node-4
    environment:
      - '[email protected]'
      - 'REDIS_NODES=redis-node-0 redis-node-1 redis-node-2 redis-node-3 redis-node-4 redis-node-5'

持久化存储

我们直接启动这些目录是可以创建的，但是容器中没有使用root用户，一般服务器都不会有写权限。因此

• 我们手工创建这些目录，先给读写权限。
• 建议等启动后，看到是对应宿主机哪个用户了，我们再将目录属主改为哪个用户。

• 创建存储目录

mkdir redis-cluster_data-{
    0..5}
chmod 777 redis-cluster-data-*

• 结果如下

[[email protected] redis-cluster]# ll
总用量 7
-rw-r----- 1 root root 2481 7月   6 10:21 docker-compose.yml
drwxrwxrwx 3 root root   61 7月   6 10:43 redis-cluster_data-0
drwxrwxrwx 3 root root   61 7月   6 10:43 redis-cluster_data-1
drwxrwxrwx 3 root root   61 7月   6 10:43 redis-cluster_data-2
drwxrwxrwx 3 root root   77 7月   6 10:51 redis-cluster_data-3
drwxrwxrwx 3 root root   77 7月   6 10:51 redis-cluster_data-4
drwxrwxrwx 3 root root   77 7月   6 10:51 redis-cluster_data-5

启动集群

• 启动集群

# docker-compose up -d 

• 查看结果

[[email protected] redis-cluster]# docker-compose ps
    Name                  Command               State            Ports
-------------------------------------------------------------------------------
redis-node-0   /opt/bitnami/scripts/redis ...   Up      0.0.0.0:6379->6379/tcp
redis-node-1   /opt/bitnami/scripts/redis ...   Up      0.0.0.0:16379->6379/tcp
redis-node-2   /opt/bitnami/scripts/redis ...   Up      0.0.0.0:26379->6379/tcp
redis-node-3   /opt/bitnami/scripts/redis ...   Up      0.0.0.0:36379->6379/tcp
redis-node-4   /opt/bitnami/scripts/redis ...   Up      0.0.0.0:46379->6379/tcp
redis-node-5   /opt/bitnami/scripts/redis ...   Up      0.0.0.0:56379->6379/tcp

集群初始化

• 初始化集群

进入其中一个容器，执行命令如下：

I have no name!@d4a8a3ef35ce:/$ redis-cli -a [email protected] --cluster create redis-node-0:6379 redis-node-1:6379 redis-node-2:6379 redis-node-3:6379 redis-node-4:6379 redis-node-5:6379 --cluster-replicas 1

输出如下：

Warning: Using a password with '-a' or '-u' option on the command line interface may not be safe.
>>> Performing hash slots allocation on 6 nodes...
Master[0] -> Slots 0 - 5460
Master[1] -> Slots 5461 - 10922
Master[2] -> Slots 10923 - 16383
Adding replica redis-node-4:6379 to redis-node-0:6379
Adding replica redis-node-5:6379 to redis-node-1:6379
Adding replica redis-node-3:6379 to redis-node-2:6379
M: 4cdf0f24f67eadb3b7a610fd9b5409a571cce8d3 redis-node-0:6379
   slots:[0-5460] (5461 slots) master
M: ee85d84a95793ee031a4b45fe3600ef81ecef7d1 redis-node-1:6379
   slots:[5461-10922] (5462 slots) master
M: f48ab32421dfe4405b73129d88f64a4ce4d076e3 redis-node-2:6379
   slots:[10923-16383] (5461 slots) master
S: 0315d864aec40531c9630d5e21959aea6837236e redis-node-3:6379
   replicates f48ab32421dfe4405b73129d88f64a4ce4d076e3
S: ec43dee472ce9f1531ccdbd0853cd672519ec2fe redis-node-4:6379
   replicates 4cdf0f24f67eadb3b7a610fd9b5409a571cce8d3
S: c834ba17bfdf0d498533355022e548b040083ed9 redis-node-5:6379
   replicates ee85d84a95793ee031a4b45fe3600ef81ecef7d1
Can I set the above configuration? (type 'yes' to accept): yes
>>> Nodes configuration updated
>>> Assign a different config epoch to each node
>>> Sending CLUSTER MEET messages to join the cluster
Waiting for the cluster to join
..
>>> Performing Cluster Check (using node redis-node-0:6379)
M: 4cdf0f24f67eadb3b7a610fd9b5409a571cce8d3 redis-node-0:6379
   slots:[0-5460] (5461 slots) master
   1 additional replica(s)
S: ec43dee472ce9f1531ccdbd0853cd672519ec2fe 172.29.0.2:6379
   slots: (0 slots) slave
   replicates 4cdf0f24f67eadb3b7a610fd9b5409a571cce8d3
M: f48ab32421dfe4405b73129d88f64a4ce4d076e3 172.29.0.4:6379
   slots:[10923-16383] (5461 slots) master
   1 additional replica(s)
M: ee85d84a95793ee031a4b45fe3600ef81ecef7d1 172.29.0.5:6379
   slots:[5461-10922] (5462 slots) master
   1 additional replica(s)
S: 0315d864aec40531c9630d5e21959aea6837236e 172.29.0.6:6379
   slots: (0 slots) slave
   replicates f48ab32421dfe4405b73129d88f64a4ce4d076e3
S: c834ba17bfdf0d498533355022e548b040083ed9 172.29.0.7:6379
   slots: (0 slots) slave
   replicates ee85d84a95793ee031a4b45fe3600ef81ecef7d1
[OK] All nodes agree about slots configuration.
>>> Check for open slots...
>>> Check slots coverage...
[OK] All 16384 slots covered.

• 查看集群状态

redis-cli -h redis-node-0 -a [email protected] cluster info

输出信息如下：

Warning: Using a password with '-a' or '-u' option on the command line interface may not be safe.
cluster_state:ok
cluster_slots_assigned:16384
cluster_slots_ok:16384
cluster_slots_pfail:0
cluster_slots_fail:0
cluster_known_nodes:6
cluster_size:3
cluster_current_epoch:6
cluster_my_epoch:1
cluster_stats_messages_ping_sent:174
cluster_stats_messages_pong_sent:203
cluster_stats_messages_sent:377
cluster_stats_messages_ping_received:198
cluster_stats_messages_pong_received:174
cluster_stats_messages_meet_received:5
cluster_stats_messages_received:377
total_cluster_links_buffer_limit_exceeded:0
I have no name!@c193f9814adb:/$

• 查看集群节点状态

redis-cli -h redis-node-0 -a [email protected] cluster nodes

输出

Warning: Using a password with '-a' or '-u' option on the command line interface may not be safe.
921dd4fc8977562273e8f0f297d7809f5d785a96 192.168.0.2:[email protected] myself,master - 0 1657079063000 1 connected 0-5460
e3be1ac5c4b5a8ea5789dd62aaa550cece718504 192.168.0.5:[email protected] slave 921dd4fc8977562273e8f0f297d7809f5d785a96 0 1657079062000 1 connected
d45ea5232290aefd17ec87229b2b68be76061b84 192.168.0.6:[email protected] slave 505a1c9a1e3bc8b7685a6cdf6aa1d82bd48cc95f 0 1657079063278 3 connected
88f0d7a2377bf876105925e330793091f2390a20 192.168.0.4:[email protected] slave 477f835648ca3ddffa48af99a2e162e541277186 0 1657079062274 2 connected
505a1c9a1e3bc8b7685a6cdf6aa1d82bd48cc95f 192.168.0.7:[email protected] master - 0 1657079064281 3 connected 10923-16383
477f835648ca3ddffa48af99a2e162e541277186 192.168.0.3:[email protected] master - 0 1657079065285 2 connected 5461-10922

2. 测试

读写测试

• 以集群模式登录

redis-cli -h redis-nod-1  -a [email protected] -c

• 写入数据

redis-node-1:6379> set name liubei
OK

• 在另一个节点上读

I have no name!@ab2a18399901:/$ redis-cli -h redis-node-5 -a [email protected] -c
Warning: Using a password with '-a' or '-u' option on the command line interface may not be safe.
redis-node-5:6379> get name
-> Redirected to slot [5798] located at 192.168.0.3:6379
"liubei"
192.168.0.3:6379>

我们可以看到，连接会跳到存储这个key的节点上（192.168.0.3）。

key所在节点

• 登录其中一个节点

redis-cli -h redis-node-0 -a [email protected] -c

• 查看节点哈希槽编号的范围

redis-node-0:6379> cluster slots
1) 1) (integer) 0
   2) (integer) 5460
   3) 1) "192.168.0.2"
      2) (integer) 6379
      3) "921dd4fc8977562273e8f0f297d7809f5d785a96"
      4) (empty array)
   4) 1) "192.168.0.5"
      2) (integer) 6379
      3) "e3be1ac5c4b5a8ea5789dd62aaa550cece718504"
      4) (empty array)
2) 1) (integer) 5461
   2) (integer) 10922
   3) 1) "192.168.0.3"
      2) (integer) 6379
      3) "477f835648ca3ddffa48af99a2e162e541277186"
      4) (empty array)
   4) 1) "192.168.0.4"
      2) (integer) 6379
      3) "88f0d7a2377bf876105925e330793091f2390a20"
      4) (empty array)
3) 1) (integer) 10923
   2) (integer) 16383
   3) 1) "192.168.0.7"
      2) (integer) 6379
      3) "505a1c9a1e3bc8b7685a6cdf6aa1d82bd48cc95f"
      4) (empty array)
   4) 1) "192.168.0.6"
      2) (integer) 6379
      3) "d45ea5232290aefd17ec87229b2b68be76061b84"
      4) (empty array)

• 查看name的哈希槽编号

redis-node-0:6379> cluster keyslot name
(integer) 5798

通过第一步查看节点哈希槽编号的范围我们可以看到5798 这个编号在 192.168.0.3 上

如果我们不用集群模式登录，只能在该节点上读取到这个key值。

• 集群模式验证是否准确

redis-node-0:6379> get name
-> Redirected to slot [5798] located at 192.168.0.3:6379
"liubei"
192.168.0.3:6379>

集群模式登录，查找这个key，我们的连接跳到了 192.168.0.3上。

• 非集群模式登录查看

登录其他节点，会提示我们这个key存在了192.168.0.3上

I have no name!@ab2a18399901:/$ redis-cli -h 192.168.0.5 -a [email protected]
Warning: Using a password with '-a' or '-u' option on the command line interface may not be safe.
192.168.0.5:6379> get name
(error) MOVED 5798 192.168.0.3:6379

只有登录这个节点才能查询到值

I have no name!@ab2a18399901:/$ redis-cli -h 192.168.0.3 -a [email protected]
Warning: Using a password with '-a' or '-u' option on the command line interface may not be safe.
192.168.0.3:6379> get name
"liubei"