Redis3.0官方集群配置

时间:2023-01-06 20:34:12

简单介绍

1.集群主要做两件事情:
- 负载均衡,多个redis实例分担负荷;
- 主备容灾,为高可用服务。

下面的操作都是根据这个博文进行的:http://www.redis.cn/topics/cluster-tutorial.html

下面我使用的是redis-3.2.8版本进行的配置。根据官网教程操作。目前官网的集群方案,还没有通过大量的实践验证它的稳定性,用不用,其实看项目组的取舍。

实验目标

配置6个redis实例,3个主,3个从。
Redis3.0官方集群配置

配置过程

1.在~/redis文件夹下面,放置编译好的redis-server, redis-cli程序;
2.分别建立文件夹7001,7002,7003,9001,9002,9003
4.每个文件夹下面放置redis.conf,主要的配置,其它默认即可:

port 7001
bind 127.0.0.1
daemonize yes
pidfile ./pid
logfile ./redis-server.log
dir ./
cluster-enabled yes
cluster-config-file nodes.conf
cluster-node-timeout 15000

配置完之后,得到下面的目录
Redis3.0官方集群配置

搭建集群
官网的源代码编译之后,在src文件夹,有ruby脚本可以自动搭建集群,也有shell脚本,基本是一样的操作,用ruby前需要安装ruby的运行环境,还有redis的库:

sudo apt-get install ruby2.0
sudo gem install redis

然后在src文件夹下面,执行命令:

./redis-trib.rb create --replicas 1 127.0.0.1:7001 127.0.0.1:7002 127.0.0.1:7003 127.0.0.1:9001 127.0.0.1:9002 127.0.0.1:9003

脚本会把上面的6个实例,自动分配主从。分配完后,输入yes即可:

ubuntu@VM-162-193-ubuntu:~/redis$ ./redis-trib.rb create --replicas 1 127.0.0.1:7001 127.0.0.1:7002 127.0.0.1:7003 127.0.0.1:9001 127.0.0.1:9002 127.0.0.1:9003
>>> Creating cluster
>>> Performing hash slots allocation on 6 nodes...
Using 3 masters:
127.0.0.1:7001
127.0.0.1:7002
127.0.0.1:7003
Adding replica 127.0.0.1:9001 to 127.0.0.1:7001
Adding replica 127.0.0.1:9002 to 127.0.0.1:7002
Adding replica 127.0.0.1:9003 to 127.0.0.1:7003
M: f8413b9565bf6bf3fb07169d9a1ad86eef779830 127.0.0.1:7001
slots:0-5460 (5461 slots) master
M: f843acf1dc5ecd9a81595ef22ca2630d2dad60f5 127.0.0.1:7002
slots:5461-10922 (5462 slots) master
M: 6be993ddf3a047abb564f1490467acfe7e0acb2f 127.0.0.1:7003
slots:10923-16383 (5461 slots) master
S: a7c08dd6a628833495380bba7abfb331c4553733 127.0.0.1:9001
replicates f8413b9565bf6bf3fb07169d9a1ad86eef779830
S: ffc808b37a178b62e4a8317ccddf43991d610dca 127.0.0.1:9002
replicates f843acf1dc5ecd9a81595ef22ca2630d2dad60f5
S: 4c1861c9830fa5e7705889a2f173a10d5b9c5f30 127.0.0.1:9003
replicates 6be993ddf3a047abb564f1490467acfe7e0acb2f
Can I set the above configuration? (type 'yes' to accept): yes
>>> Nodes configuration updated
>>> Assign a different config epoch to each node
>>> Sending CLUSTER MEET messages to join the cluster
Waiting for the cluster to join....
>>> Performing Cluster Check (using node 127.0.0.1:7001)
M: f8413b9565bf6bf3fb07169d9a1ad86eef779830 127.0.0.1:7001
slots:0-5460 (5461 slots) master
1 additional replica(s)
S: 4c1861c9830fa5e7705889a2f173a10d5b9c5f30 127.0.0.1:9003
slots: (0 slots) slave
replicates 6be993ddf3a047abb564f1490467acfe7e0acb2f
M: 6be993ddf3a047abb564f1490467acfe7e0acb2f 127.0.0.1:7003
slots:10923-16383 (5461 slots) master
1 additional replica(s)
S: ffc808b37a178b62e4a8317ccddf43991d610dca 127.0.0.1:9002
slots: (0 slots) slave
replicates f843acf1dc5ecd9a81595ef22ca2630d2dad60f5
S: a7c08dd6a628833495380bba7abfb331c4553733 127.0.0.1:9001
slots: (0 slots) slave
replicates f8413b9565bf6bf3fb07169d9a1ad86eef779830
M: f843acf1dc5ecd9a81595ef22ca2630d2dad60f5 127.0.0.1:7002
slots:5461-10922 (5462 slots) master
1 additional replica(s)
[OK] All nodes agree about slots configuration.
>>> Check for open slots...
>>> Check slots coverage...
[OK] All 16384 slots covered.

从上面的信息可以看到,redis已经创建了16384个slot,然后各自分布到不同的redis实例上。

使用集群

-c参数启动时,redis-cli程序对集群有一些基本的支持。在7003的实例创建key-2:

ubuntu@VM-162-193-ubuntu:~/redis$ ./redis-cli -c -p 7003
127.0.0.1:7003> set key-2 2
OK
127.0.0.1:7003>

在7001机器获取:

ubuntu@VM-162-193-ubuntu:~/redis$ ./redis-cli -c -p 7001
127.0.0.1:7001> get key-2
-> Redirected to slot [12422] located at 127.0.0.1:7003
"2"
127.0.0.1:7003>

这时候,集群检测到这个key所在的slot是在7003机器上的,redis-cli跳转到7003机器,继续敲打命令,都是在7003机器上操作。

现在的情况就是,每一个运行的实例都维护了集群的信息,只要知道一个有效的实例,那么我们就可以根据这个实例,找到同一个集群的其它机器。

添加一个新的主节点:

//127.0.0.1:7004 代表新的redis实例
//后面的127.0.0.1:7002代表一个可以已经在集群里的redis实例
./redis-trib.rb add-node 127.0.0.1:7004 127.0.0.1:7002

上面的命令是给集群添加了节点,但是这个节点并没有什么作用,因为集群的slot没有一个是分配到这个节点上的。所以,需要对集群的slot重新分配,也就是要启用分片的过程:

//127.0.0.1:7002代表一个可以已经在集群里的redis实例
./redis-trib.rb reshard 127.0.0.1:7002

然后命令会提示你要分片多少slot,要把这些slot迁移到哪个节点id,从哪些redis实例获取迁移这些slot。

ubuntu@VM-162-193-ubuntu:~/redis$ ./redis-trib.rb reshard 127.0.0.1:7002
>>> Performing Cluster Check (using node 127.0.0.1:7001)
M: f8413b9565bf6bf3fb07169d9a1ad86eef779830 127.0.0.1:7001
slots:0-5460 (5461 slots) master
1 additional replica(s)
S: 4c1861c9830fa5e7705889a2f173a10d5b9c5f30 127.0.0.1:9003
slots: (0 slots) slave
replicates 6be993ddf3a047abb564f1490467acfe7e0acb2f
M: 6be993ddf3a047abb564f1490467acfe7e0acb2f 127.0.0.1:7003
slots:10923-16383 (5461 slots) master
1 additional replica(s)
S: ffc808b37a178b62e4a8317ccddf43991d610dca 127.0.0.1:9002
slots: (0 slots) slave
replicates f843acf1dc5ecd9a81595ef22ca2630d2dad60f5
S: a7c08dd6a628833495380bba7abfb331c4553733 127.0.0.1:9001
slots: (0 slots) slave
replicates f8413b9565bf6bf3fb07169d9a1ad86eef779830
M: f843acf1dc5ecd9a81595ef22ca2630d2dad60f5 127.0.0.1:7002
slots:5461-10922 (5462 slots) master
1 additional replica(s)
[OK] All nodes agree about slots configuration.
>>> Check for open slots...
>>> Check slots coverage...
[OK] All 16384 slots covered.
How many slots do you want to move (from 1 to 16384)? 100
What is the receiving node ID? f8413b9565bf6bf3fb07169d9a1ad86eef779830
Please enter all the source node IDs.
Type 'all' to use all the nodes as source nodes for the hash slots.
Type 'done' once you entered all the source nodes IDs.
Source node #1:all

分片完了,可以用下面的命令检查集群里实例分配的slot情况:

./redis-trib.rb check 127.0.0.1:7002

主从的故障恢复

使7001的redis实例故障:

./redis-cli -p 7001 debug segfault

然后检查一个状态:

ubuntu@VM-162-193-ubuntu:~/redis$ ./redis-trib.rb check 127.0.0.1:7002
>>> Performing Cluster Check (using node 127.0.0.1:7002)
M: f843acf1dc5ecd9a81595ef22ca2630d2dad60f5 127.0.0.1:7002
slots:5512-10922 (5411 slots) master
1 additional replica(s)
S: 4c1861c9830fa5e7705889a2f173a10d5b9c5f30 127.0.0.1:9003
slots: (0 slots) slave
replicates 6be993ddf3a047abb564f1490467acfe7e0acb2f
S: ffc808b37a178b62e4a8317ccddf43991d610dca 127.0.0.1:9002
slots: (0 slots) slave
replicates f843acf1dc5ecd9a81595ef22ca2630d2dad60f5
M: a7c08dd6a628833495380bba7abfb331c4553733 127.0.0.1:9001
slots:0-5511,10923-10971 (5561 slots) master
0 additional replica(s)
M: 6be993ddf3a047abb564f1490467acfe7e0acb2f 127.0.0.1:7003
slots:10972-16383 (5412 slots) master
1 additional replica(s)
[OK] All nodes agree about slots configuration.

原来7001的从redis——9001升级为master了,并且把之前7001的slot继承过来。

移除一个节点

移除一个节点的前提是,这个节点是空的,所以在移除节点之前,要手工进行分片操作,不然会导致数据丢失。
移除节点的命令:

./redis-trib del-node 127.0.0.1:7002 `<node-id>`