Redis 集群简介

Redis 是一个开源的 key-value 存储系统,由于出众的性能,大部分互联网企业都用来做服务器端缓存。Redis 在3.0版本前只支持单实例模式,虽然支持主从模式、哨兵模式部署来解决单点故障,但是现在互联网企业动辄大几百G的数据,可完全是没法满足业务的需求,所以,Redis 在 3.0 版本以后就推出了集群模式。

Redis 从3.0.0正式版开始官方支持集群, Redis 集群采用了P2P的模式,完全去中心化。Redis 把所有的 Key 分成了 16384 个 slot,每个 Redis 实例负责其中一部分 slot 。集群中的所有信息(节点、端口、slot等),都通过节点之间定期的数据交换而更新。

Redis 客户端可以在任意一个 Redis 实例发出请求,如果所需数据不在该实例中,通过重定向命令引导客户端访问所需的实例。

随随便便搭建一个集群

安装部署任何一个应用其实都很简单,只要安装步骤一步一步来就行了。下面说一下 Redis 集群搭建规划,由于集群至少需要6个节点(3主3从模式),所以,没有这么多机器给我玩,现在计划是在一台机器上模拟一个集群,当然,这和生产环境的集群搭建没本质区别。

1.获取软件包

cd /tmp/ && wget http://download.redis.io/releases/redis-4.0.11.tar.gz
mkdir /usr/local/redis && tar -xr /tmp/redis-4.0.11.tar.gz -C /usr/local/redis
cd /usr/local/redis
./configure
make -j 4
make install

2.规划创建文件目录

我们计划集群中 Redis 节点的端口号为 9001-9006 ,端口号即集群下各实例文件夹。数据存放在 端口号/data 文件夹中。

mkdir /usr/local/redis-cluster/
cd /usr/local/redis-cluster/

for i in {1..6};do
    mkdir 900${i}/data
done

3. 复制执行脚本

在 /usr/local/redis-cluster 下创建 bin 文件夹,用来存放集群运行脚本,并把安装好的 Redis 的 src 路径下的运行脚本拷贝过来。看命令:

mkdir /usr/local/redis-cluster/bin
cd /usr/local/redis/src # 下载后解压编译后的src目录
cp mkreleasehdr.sh redis-benchmark redis-check-aof redis-check-dump redis-cli redis-server redis-trib.rb /usr/local/redis-cluster/bin

4.复制一个新 Redis 实例

我们现在从已安装好的 Redis 中复制一个新的实例到 9001 文件夹,并修改 redis.conf 配置。

cp /usr/local/redis/* /usr/local/redis-cluster/9001


port 9001(每个节点的端口号)
daemonize yes
bind 192.168.56.127(绑定当前机器 IP)
dir /usr/local/redis-cluster/9001/data/(数据文件存放位置)
pidfile /var/run/redis_9001.pid(pid 9001和port要对应)
cluster-enabled yes(启动集群模式)
cluster-config-file nodes9001.conf(9001和port要对应)
cluster-node-timeout 15000
appendonly yes

5.再复制出五个新 Redis 实例

我们已经完成了一个节点了,其实接下来就是机械化的再完成另外五个节点,其实可以这么做:把 9001 实例 复制到另外五个文件夹中,唯一要修改的就是 redis.conf 中的所有和端口的相关的信息即可,其实就那么四个位置。开始操作,看命令:

\cp -rf /usr/local/redis-cluster/9001/* /usr/local/redis-cluster/9002
\cp -rf /usr/local/redis-cluster/9001/* /usr/local/redis-cluster/9003
\cp -rf /usr/local/redis-cluster/9001/* /usr/local/redis-cluster/9004
\cp -rf /usr/local/redis-cluster/9001/* /usr/local/redis-cluster/9005
\cp -rf /usr/local/redis-cluster/9001/* /usr/local/redis-cluster/9006

这里我们把配置复制过去之后需要修改对应文件夹中的redis.conf 用sed或者vim全局替换即可,主要是端口

下面,下面我们来启动它们

for i in {1..6};do
    /usr/local/bin/redis-server /usr/local/redis-cluster/900${i}/redis.conf 
done


[root@localhost redis-cluster]# ps aux | grep redis | grep -v grep
root       3995  0.5  0.6 151404 11864 ?        Ssl  10:51   0:10 /usr/local/redis-cluster/bin/redis-server 192.168.56.127:9001 [cluster]
root       4000  0.4  0.6 151404 11856 ?        Ssl  10:51   0:09 /usr/local/redis-cluster/bin/redis-server 192.168.56.127:9002 [cluster]
root       4005  0.4  0.6 151404 11856 ?        Ssl  10:51   0:09 /usr/local/redis-cluster/bin/redis-server 192.168.56.127:9003 [cluster]
root       4010  0.1  0.6 151404 11872 ?        Ssl  10:51   0:02 /usr/local/redis-cluster/bin/redis-server 192.168.56.127:9004 [cluster]
root       4015  0.0  0.6 151404 11876 ?        Ssl  10:51   0:01 /usr/local/redis-cluster/bin/redis-server 192.168.56.127:9005 [cluster]
root       4020  0.0  0.6 151404 11872 ?        Ssl  10:51   0:01 /usr/local/redis-cluster/bin/redis-server 192.168.56.127:9006 [cluster]
root       4509  0.8  0.6 149356 11880 ?        Ssl  10:55   0:14 /usr/local/redis-cluster/bin/redis-server 
[root@localhost redis-cluster]#

以上 可以看出我们的6个实例已经启动了
测试一下?

随便找个节点测试一下:

/usr/local/redis-cluster/bin/redis-cli -h 192.168.56.127 -p 9001

192.168.56.127:9001> set name guomaoqiu
(error) CLUSTERDOWN Hash slot not served
192.168.56.127:9001>

连接成功了,但好像报错了

这是因为虽然我们配置并启动了 Redis 集群服务,但是他们暂时还并不在一个集群中,互相直接发现不了,而且还没有可存储的位置,就是所谓的slot(槽)。

6.安装集群所需软件

由于 Redis 集群需要使用 ruby 命令,所以我们需要安装 ruby 和相关接口。

安装redis需要ruby版本最低是2.2.2,而centos yum库中ruby版本支持到2.0.0。所以,无法满足需求。
解决方案:

1.安装curl
sudo yum install curl
2.安装RVM
curl -L get.rvm.io | bash -s stable
source /usr/local/rvm/scripts/rvm
3.查看rvm库中已知的ruby版本
rvm list known
4.安装一个ruby版本
rvm install 2.3.3
5.设置默认版本
rvm use 2.3.3
6.卸载一个已知版本
rvm remove 2.0.0
7.查看ruby版本
ruby –version
8.再安装redis就可以了
gem install redis

7. 利用官方提供的命令创建集群

/usr/local/redis-cluster/bin/redis-trib.rb create --replicas 1 192.168.56.127:9001 192.168.56.127:9002 192.168.56.127:9003 192.168.56.127:9004 192.168.56.127:9005 192.168.56.127:9006

简单解释一下这个命令:调用 ruby 命令来进行创建集群,–replicas 1 表示主从复制比例为 1:1,即一个主节点对应一个从节点;然后,默认给我们分配好了每个主节点和对应从节点服务,以及 solt 的大小,因为在 Redis 集群中有且仅有 16383 个 solt ,默认情况会给我们平均分配,当然你可以指定,后续的增减节点也可以重新分配。

[root@localhost redis-cluster]# /usr/local/redis-cluster/bin/redis-trib.rb create --replicas 1 192.168.56.127:9001 192.168.56.127:9002 192.168.56.127:9003 192.168.56.127:9004 192.168.56.127:9005 192.168.56.127:9006
>>> Creating cluster
>>> Performing hash slots allocation on 6 nodes...
Using 3 masters:
# 三个主节点
192.168.56.127:9001
192.168.56.127:9002
192.168.56.127:9003
# 三个主节点对应三个从节点
Adding replica 192.168.56.127:9005 to 192.168.56.127:9001
Adding replica 192.168.56.127:9006 to 192.168.56.127:9002
Adding replica 192.168.56.127:9004 to 192.168.56.127:9003
>>> Trying to optimize slaves allocation for anti-affinity
[WARNING] Some slaves are in the same host as their master
M: 0fcbaa301451baf87284546513003568e47b5daa 192.168.56.127:9001
   slots:0-5460 (5461 slots) master
M: 474fd53f7199ad70cce7f18bd68f5445b559509a 192.168.56.127:9002
   slots:5461-10922 (5462 slots) master
M: e070e76789352d2492cfc9800850e943e0c4b72d 192.168.56.127:9003
   slots:10923-16383 (5461 slots) master
S: 950c399165439660efb6ff2f907f938e213de530 192.168.56.127:9004
   replicates 474fd53f7199ad70cce7f18bd68f5445b559509a
S: 17db098db9fd222df002e649506f88efcdc83633 192.168.56.127:9005
   replicates e070e76789352d2492cfc9800850e943e0c4b72d
S: bd278fcea63862199e964a6aab2e786710cf5575 192.168.56.127:9006
   replicates 0fcbaa301451baf87284546513003568e47b5daa
Can I set the above configuration? (type 'yes' to accept): yes
>>> Nodes configuration updated
>>> Assign a different config epoch to each node
>>> Sending CLUSTER MEET messages to join the cluster
Waiting for the cluster to join...
>>> Performing Cluster Check (using node 192.168.56.127:9001)
M: 0fcbaa301451baf87284546513003568e47b5daa 192.168.56.127:9001
   slots:0-5460 (5461 slots) master
   1 additional replica(s)
M: 474fd53f7199ad70cce7f18bd68f5445b559509a 192.168.56.127:9002
   slots:5461-10922 (5462 slots) master
   1 additional replica(s)
M: e070e76789352d2492cfc9800850e943e0c4b72d 192.168.56.127:9003
   slots:10923-16383 (5461 slots) master
   1 additional replica(s)
S: bd278fcea63862199e964a6aab2e786710cf5575 192.168.56.127:9006
   slots: (0 slots) slave
   replicates 0fcbaa301451baf87284546513003568e47b5daa
S: 950c399165439660efb6ff2f907f938e213de530 192.168.56.127:9004
   slots: (0 slots) slave
   replicates 474fd53f7199ad70cce7f18bd68f5445b559509a
S: 17db098db9fd222df002e649506f88efcdc83633 192.168.56.127:9005
   slots: (0 slots) slave
   replicates e070e76789352d2492cfc9800850e943e0c4b72d
[OK] All nodes agree about slots configuration.
>>> Check for open slots...
>>> Check slots coverage...
[OK] All 16384 slots covered.

以上执行结果代表集群搭建成功啦!!!

验证:

/usr/local/redis-cluster/bin/redis-cli -c -h 192.168.56.127 -p 9001
192.168.56.127:9001> cluster info
cluster_state:ok
cluster_slots_assigned:16384
cluster_slots_ok:16384
cluster_slots_pfail:0
cluster_slots_fail:0
cluster_known_nodes:6
cluster_size:3
cluster_current_epoch:6
cluster_my_epoch:1
cluster_stats_messages_ping_sent:167
cluster_stats_messages_pong_sent:172
cluster_stats_messages_sent:339
cluster_stats_messages_ping_received:167
cluster_stats_messages_pong_received:167
cluster_stats_messages_meet_received:5
cluster_stats_messages_received:339
192.168.56.127:9001> cluster nodes
474fd53f7199ad70cce7f18bd68f5445b559509a 192.168.56.127:9002@19002 master - 0 1536205294229 2 connected 5461-10922
e070e76789352d2492cfc9800850e943e0c4b72d 192.168.56.127:9003@19003 master - 0 1536205293000 3 connected 10923-16383
bd278fcea63862199e964a6aab2e786710cf5575 192.168.56.127:9006@19006 slave 0fcbaa301451baf87284546513003568e47b5daa 0 1536205295234 6 connected
0fcbaa301451baf87284546513003568e47b5daa 192.168.56.127:9001@19001 myself,master - 0 1536205294000 1 connected 0-5460
950c399165439660efb6ff2f907f938e213de530 192.168.56.127:9004@19004 slave 474fd53f7199ad70cce7f18bd68f5445b559509a 0 1536205293217 4 connected
17db098db9fd222df002e649506f88efcdc83633 192.168.56.127:9005@19005 slave e070e76789352d2492cfc9800850e943e0c4b72d 0 1536205292212 5 connected
192.168.56.127:9001> set name guomaoqiu
-> Redirected to slot [5798] located at 192.168.56.127:9002
OK
192.168.56.127:9002> get name
"guomaoqiu"
192.168.56.127:9002>

通过命令,可以详细的看出集群信息和各个节点状态,主从信息以及连接数、槽信息等。这么看到,我们已经真的把 Redis 集群搭建部署成功啦!

如何新增节点?

新增节点无非就是复制配置,修改配置,然后启动;我这里还是创建了一对一主一从的配置,端口分别对应的是9007、9008

1.启动两个实例(假设我这里已经复制配置,修改完配置了)

for i in {7..8};do
    /usr/local/redis-cluster/bin/redis-server /usr/local/redis-cluster/900${i}/redis.conf 
done

6002:C 06 Sep 11:50:53.048 # oO0OoO0OoO0Oo Redis is starting oO0OoO0OoO0Oo
6002:C 06 Sep 11:50:53.048 # Redis version=4.0.11, bits=64, commit=00000000, modified=0, pid=6002, just started
6002:C 06 Sep 11:50:53.048 # Configuration loaded
6004:C 06 Sep 11:50:53.060 # oO0OoO0OoO0Oo Redis is starting oO0OoO0OoO0Oo
6004:C 06 Sep 11:50:53.060 # Redis version=4.0.11, bits=64, commit=00000000, modified=0, pid=6004, just started
6004:C 06 Sep 11:50:53.060 # Configuration loaded

ps aux | grep redis | grep -v grep
redis      1180  0.0  0.3 142900  5828 ?        Ssl  11:28   0:01 /usr/bin/redis-server 0.0.0.0:6379
root       4257  0.0  0.6 149356 11880 ?        Ssl  11:32   0:00 /usr/local/redis-cluster/bin/redis-server 192.168.56.127:9001 [cluster]
root       4262  0.0  0.6 149356 11880 ?        Ssl  11:32   0:00 /usr/local/redis-cluster/bin/redis-server 192.168.56.127:9002 [cluster]
root       4267  0.0  0.6 149356 11876 ?        Ssl  11:32   0:00 /usr/local/redis-cluster/bin/redis-server 192.168.56.127:9003 [cluster]
root       4272  0.0  0.6 149356 11908 ?        Ssl  11:32   0:00 /usr/local/redis-cluster/bin/redis-server 192.168.56.127:9004 [cluster]
root       4277  0.0  0.6 149356 11920 ?        Ssl  11:32   0:00 /usr/local/redis-cluster/bin/redis-server 192.168.56.127:9005 [cluster]
root       4282  0.0  0.6 149356 11912 ?        Ssl  11:32   0:00 /usr/local/redis-cluster/bin/redis-server 192.168.56.127:9006 [cluster]
root       6003  0.1  0.5 147308  9616 ?        Ssl  11:50   0:00 /usr/local/redis-cluster/bin/redis-server 192.168.56.127:9007 [cluster]
root       6008  0.0  0.5 147308  9616 ?        Ssl  11:50   0:00 /usr/local/redis-cluster/bin/redis-server 192.168.56.127:9008 [cluster]

以上 可以看到我们启动了后面两个实例,并且进程中标注了是以集群方式启动, 但是此时还并没有将实例加入到集群当中。

2.添加实例到集群

[root@locahost redis-cluster]# /usr/local/redis-cluster/bin/redis-trib.rb add-node 192.168.56.127:9007 192.168.56.127:9001
>>> Adding node 192.168.56.127:9007 to cluster 192.168.56.127:9001
>>> Performing Cluster Check (using node 192.168.56.127:9001)
M: 0fcbaa301451baf87284546513003568e47b5daa 192.168.56.127:9001
   slots:0-5460 (5461 slots) master
   1 additional replica(s)
M: 474fd53f7199ad70cce7f18bd68f5445b559509a 192.168.56.127:9002
   slots:5461-10922 (5462 slots) master
   1 additional replica(s)
M: e070e76789352d2492cfc9800850e943e0c4b72d 192.168.56.127:9003
   slots:10923-16383 (5461 slots) master
   1 additional replica(s)
S: bd278fcea63862199e964a6aab2e786710cf5575 192.168.56.127:9006
   slots: (0 slots) slave
   replicates 0fcbaa301451baf87284546513003568e47b5daa
S: 950c399165439660efb6ff2f907f938e213de530 192.168.56.127:9004
   slots: (0 slots) slave
   replicates 474fd53f7199ad70cce7f18bd68f5445b559509a
S: 17db098db9fd222df002e649506f88efcdc83633 192.168.56.127:9005
   slots: (0 slots) slave
   replicates e070e76789352d2492cfc9800850e943e0c4b72d
[OK] All nodes agree about slots configuration.
>>> Check for open slots...
>>> Check slots coverage...
[OK] All 16384 slots covered.
>>> Send CLUSTER MEET to node 192.168.56.127:9007 to make it join the cluster.
[OK] New node added correctly.
[root@localhost redis-cluster]# /usr/local/redis-cluster/bin/redis-cli -c -h 192.168.56.127 -p 9001 cluster nodes
474fd53f7199ad70cce7f18bd68f5445b559509a 192.168.56.127:9002@19002 master - 0 1536215194000 2 connected 5461-10922
e070e76789352d2492cfc9800850e943e0c4b72d 192.168.56.127:9003@19003 master - 0 1536215194000 3 connected 10923-16383
bd278fcea63862199e964a6aab2e786710cf5575 192.168.56.127:9006@19006 slave 0fcbaa301451baf87284546513003568e47b5daa 0 1536215193000 6 connected
0fcbaa301451baf87284546513003568e47b5daa 192.168.56.127:9001@19001 myself,master - 0 1536215192000 1 connected 0-5460
950c399165439660efb6ff2f907f938e213de530 192.168.56.127:9004@19004 slave 474fd53f7199ad70cce7f18bd68f5445b559509a 0 1536215195858 4 connected
1714784542b3afc9e2a8b2c93fff49d095e7d72b 192.168.56.127:9007@19007 master - 0 1536215195000 0 connected
17db098db9fd222df002e649506f88efcdc83633 192.168.56.127:9005@19005 slave e070e76789352d2492cfc9800850e943e0c4b72d 0 1536215194855 5 connected

以上可以看到我们通过 add-node 参数添加了 9007这个实例;命令最后一个参数是已经成为集群的集群成员host:port
而且实例9007获得了一个集群ID:1714784542b3afc9e2a8b2c93fff49d095e7d72b

5. 增加实例的从节点到集群

以上我们将实例9007作为master增加到了集群中,但是按照前面的规划,每个master对应一个slave所以我们需要将实例9008加入集群并将其称为9007的slave,命令如下

[root@localhost redis-cluster]# /usr/local/redis-cluster/bin/redis-trib.rb add-node --slave --master-id 1714784542b3afc9e2a8b2c93fff49d095e7d72b 192.168.56.127:9008 192.168.56.127:9001
>>> Adding node 192.168.56.127:9008 to cluster 192.168.56.127:9001
>>> Performing Cluster Check (using node 192.168.56.127:9001)
M: 0fcbaa301451baf87284546513003568e47b5daa 192.168.56.127:9001
   slots:0-5460 (5461 slots) master
   1 additional replica(s)
M: 474fd53f7199ad70cce7f18bd68f5445b559509a 192.168.56.127:9002
   slots:5461-10922 (5462 slots) master
   1 additional replica(s)
M: e070e76789352d2492cfc9800850e943e0c4b72d 192.168.56.127:9003
   slots:10923-16383 (5461 slots) master
   1 additional replica(s)
S: bd278fcea63862199e964a6aab2e786710cf5575 192.168.56.127:9006
   slots: (0 slots) slave
   replicates 0fcbaa301451baf87284546513003568e47b5daa
S: 950c399165439660efb6ff2f907f938e213de530 192.168.56.127:9004
   slots: (0 slots) slave
   replicates 474fd53f7199ad70cce7f18bd68f5445b559509a
M: 1714784542b3afc9e2a8b2c93fff49d095e7d72b 192.168.56.127:9007
   slots: (0 slots) master
   0 additional replica(s)
S: 17db098db9fd222df002e649506f88efcdc83633 192.168.56.127:9005
   slots: (0 slots) slave
   replicates e070e76789352d2492cfc9800850e943e0c4b72d
[OK] All nodes agree about slots configuration.
>>> Check for open slots...
>>> Check slots coverage...
[OK] All 16384 slots covered.
>>> Send CLUSTER MEET to node 192.168.56.127:9008 to make it join the cluster.
Waiting for the cluster to join.
>>> Configure node as replica of 192.168.56.127:9007.
[OK] New node added correctly.

[root@localhost redis-cluster]# /usr/local/redis-cluster/bin/redis-trib.rb check 192.168.56.127:9001
>>> Performing Cluster Check (using node 192.168.56.127:9001)
M: 0fcbaa301451baf87284546513003568e47b5daa 192.168.56.127:9001
   slots:0-5460 (5461 slots) master
   1 additional replica(s)
M: 474fd53f7199ad70cce7f18bd68f5445b559509a 192.168.56.127:9002
   slots:5461-10922 (5462 slots) master
   1 additional replica(s)
M: e070e76789352d2492cfc9800850e943e0c4b72d 192.168.56.127:9003
   slots:10923-16383 (5461 slots) master
   1 additional replica(s)
S: 5ed978b695e6a222e67d13c68e55f6c4e74b7ba6 192.168.56.127:9008
   slots: (0 slots) slave
   replicates 1714784542b3afc9e2a8b2c93fff49d095e7d72b
S: bd278fcea63862199e964a6aab2e786710cf5575 192.168.56.127:9006
   slots: (0 slots) slave
   replicates 0fcbaa301451baf87284546513003568e47b5daa
S: 950c399165439660efb6ff2f907f938e213de530 192.168.56.127:9004
   slots: (0 slots) slave
   replicates 474fd53f7199ad70cce7f18bd68f5445b559509a
M: 1714784542b3afc9e2a8b2c93fff49d095e7d72b 192.168.56.127:9007
   slots: (0 slots) master
   1 additional replica(s)
S: 17db098db9fd222df002e649506f88efcdc83633 192.168.56.127:9005
   slots: (0 slots) slave
   replicates e070e76789352d2492cfc9800850e943e0c4b72d
[OK] All nodes agree about slots configuration.
>>> Check for open slots...
>>> Check slots coverage...
[OK] All 16384 slots covered.

以上我们将两个节点加入到了集群中,但是细心的同学会发现我们的实例9007所拥有的slot数量为0,那此时就需要将重新调整哈希槽啦~
再次之前我们三个master 平均将16384 分成了三等分,那我们新增加了一个master ,为了平衡我们可以计算出这个新的master需要多少哈希槽,即 16384/4 = 4096 我们需要移动4096个槽点到9007上。

[root@localhost redis-cluster]# /usr/local/redis-cluster/bin/redis-trib.rb reshard 192.168.56.127:9001
>>> Performing Cluster Check (using node 192.168.56.127:9001)
M: 0fcbaa301451baf87284546513003568e47b5daa 192.168.56.127:9001
   slots:0-5460 (5461 slots) master
   1 additional replica(s)
S: 5ed978b695e6a222e67d13c68e55f6c4e74b7ba6 192.168.56.127:9008
   slots: (0 slots) slave
   replicates 1714784542b3afc9e2a8b2c93fff49d095e7d72b
M: 1714784542b3afc9e2a8b2c93fff49d095e7d72b 192.168.56.127:9007
   slots:5461-5798 (0 slots) master
   1 additional replica(s)
S: 17db098db9fd222df002e649506f88efcdc83633 192.168.56.127:9005
   slots: (0 slots) slave
   replicates e070e76789352d2492cfc9800850e943e0c4b72d
M: 474fd53f7199ad70cce7f18bd68f5445b559509a 192.168.56.127:9002
   slots:5799-10922 (5462 slots) master
   1 additional replica(s)
M: e070e76789352d2492cfc9800850e943e0c4b72d 192.168.56.127:9003
   slots:10923-16383 (5461 slots) master
   1 additional replica(s)
S: bd278fcea63862199e964a6aab2e786710cf5575 192.168.56.127:9006
   slots: (0 slots) slave
   replicates 0fcbaa301451baf87284546513003568e47b5daa
S: 950c399165439660efb6ff2f907f938e213de530 192.168.56.127:9004
   slots: (0 slots) slave
   replicates 474fd53f7199ad70cce7f18bd68f5445b559509a
[OK] All nodes agree about slots configuration.
>>> Check for open slots...
>>> Check slots coverage...
[OK] All 16384 slots covered.
# 这里需要输入我们所需要的哈希槽
How many slots do you want to move (from 1 to 16384)? 4097
# 这里需要输入我们实例9007的ID
What is the receiving node ID? 1714784542b3afc9e2a8b2c93fff49d095e7d72b
Please enter all the source node IDs.
  Type 'all' to use all the nodes as source nodes for the hash slots.
  Type 'done' once you entered all the source nodes IDs.
Source node #1:all
#输入all 表示从所有的主节点中随机转移,凑够1000个哈希槽
#然后再输入yes,redis集群就开始分配哈希槽了。

以上:
redis-trib 会向你询问重新分片的源节点(source node),即,要从特点的哪个节点中取出 4096 个哈希槽,还是从全部节点提取4096个哈希槽, 并将这些槽移动到9007节点上面。

如果我们不打算从特定的节点上取出指定数量的哈希槽,那么可以向redis-trib输入 all,这样的话, 集群中的所有主节点都会成为源节点,redis-trib从各个源节点中各取出一部分哈希槽,凑够4096个,然后移动到9007节点上:

验证:

[root@localhost redis-cluster]# /usr/local/redis-cluster/bin/redis-trib.rb check 192.168.56.127:9001
>>> Performing Cluster Check (using node 192.168.56.127:9001)
M: 0fcbaa301451baf87284546513003568e47b5daa 192.168.56.127:9001
   slots:1280-5460 (4096 slots) master
   1 additional replica(s)
S: 5ed978b695e6a222e67d13c68e55f6c4e74b7ba6 192.168.56.127:9008
   slots: (0 slots) slave
   replicates 1714784542b3afc9e2a8b2c93fff49d095e7d72b
M: 1714784542b3afc9e2a8b2c93fff49d095e7d72b 192.168.56.127:9007
   slots:0-1279,5461-6998,10923-12201 (4096 slots) master
   1 additional replica(s)
S: 17db098db9fd222df002e649506f88efcdc83633 192.168.56.127:9005
   slots: (0 slots) slave
   replicates e070e76789352d2492cfc9800850e943e0c4b72d
M: 474fd53f7199ad70cce7f18bd68f5445b559509a 192.168.56.127:9002
   slots:6999-10922 (4096 slots) master
   1 additional replica(s)
M: e070e76789352d2492cfc9800850e943e0c4b72d 192.168.56.127:9003
   slots:12202-16383 (4096 slots) master
   1 additional replica(s)
S: bd278fcea63862199e964a6aab2e786710cf5575 192.168.56.127:9006
   slots: (0 slots) slave
   replicates 0fcbaa301451baf87284546513003568e47b5daa
S: 950c399165439660efb6ff2f907f938e213de530 192.168.56.127:9004
   slots: (0 slots) slave
   replicates 474fd53f7199ad70cce7f18bd68f5445b559509a
[OK] All nodes agree about slots configuration.
>>> Check for open slots...
>>> Check slots coverage...
[OK] All 16384 slots covered.

以上我们的集群搭建以及新增节点就完成了;

6. 从集群中删除节点:

和节点添加一样,移除节点也有移除主节点,从节点

1、移除主节点

移除节点使用redis-trib的del-node命令

/usr/local/redis-cluster/bin/redis-trib.rb del-node 192.168.56.127:9001 ${node-id}

192.168.56.127:9001其中一个集群节点,node-id为要删除的主节点,这里和添加节点不同,移除节点node-id是必需的,测试删除9003主节点

[root@localhost redis-cluster]# /usr/local/redis-cluster/bin/redis-trib.rb del-node 192.168.56.127:9001 e070e76789352d2492cfc9800850e943e0c4b72d
>>> Removing node e070e76789352d2492cfc9800850e943e0c4b72d from cluster 192.168.56.127:9001
[ERR] Node 192.168.56.127:9003 is not empty! Reshard data away and try again.

redis cluster提示9003已经有数据了,不能够被删除,需要将他的数据转移出去,也就是和新增主节点一样需重新分片。

/usr/local/redis-cluster/bin/redis-trib.rb reshard 192.168.56.127:9001

执行以后会提示我们移除的大小,因为9003占用了4096个槽点

>>> Check for open slots...
>>> Check slots coverage...
[OK] All 16384 slots covered.
How many slots do you want to move (from 1 to 16384)?

输入4096
提示移动的node id,填写9007的node id。

>>> Check for open slots...
>>> Check slots coverage...
[OK] All 16384 slots covered.
How many slots do you want to move (from 1 to 16384)? 4182
What is the receiving node ID? 1714784542b3afc9e2a8b2c93fff49d095e7d72b

需要移动到全部主节点上还是单个主节点

Please enter all the source node IDs.
  Type 'all' to use all the nodes as source nodes for the hash slots.
  Type 'done' once you entered all the source nodes IDs.
Source node #1:

将4096个槽点移动到9007上,填写9003的node id :e070e76789352d2492cfc9800850e943e0c4b72d

Source node #1:e070e76789352d2492cfc9800850e943e0c4b72d
Source node #2:done

确认之后会一个一个将9003的卡槽移到到9007上。

[root@localhost redis-cluster]# /usr/local/redis-cluster/bin/redis-trib.rb check 192.168.56.127:9001
>>> Performing Cluster Check (using node 192.168.56.127:9001)
M: 0fcbaa301451baf87284546513003568e47b5daa 192.168.56.127:9001
   slots:1280-5460 (4096 slots) master
   1 additional replica(s)
S: 5ed978b695e6a222e67d13c68e55f6c4e74b7ba6 192.168.56.127:9008
   slots: (0 slots) slave
   replicates 1714784542b3afc9e2a8b2c93fff49d095e7d72b
M: 1714784542b3afc9e2a8b2c93fff49d095e7d72b 192.168.56.127:9007
   slots:0-1279,5461-6998,10923-16383 (8192 slots) master
   2 additional replica(s)
S: 17db098db9fd222df002e649506f88efcdc83633 192.168.56.127:9005
   slots: (0 slots) slave
   replicates 1714784542b3afc9e2a8b2c93fff49d095e7d72b
M: 474fd53f7199ad70cce7f18bd68f5445b559509a 192.168.56.127:9002
   slots:6999-10922 (4096 slots) master
   1 additional replica(s)
M: e070e76789352d2492cfc9800850e943e0c4b72d 192.168.56.127:9003
   slots: (0 slots) master
   0 additional replica(s)
S: bd278fcea63862199e964a6aab2e786710cf5575 192.168.56.127:9006
   slots: (0 slots) slave
   replicates 0fcbaa301451baf87284546513003568e47b5daa
S: 950c399165439660efb6ff2f907f938e213de530 192.168.56.127:9004
   slots: (0 slots) slave
   replicates 474fd53f7199ad70cce7f18bd68f5445b559509a
[OK] All nodes agree about slots configuration.
>>> Check for open slots...
>>> Check slots coverage...
[OK] All 16384 slots covered.

以上我们将实例9003的哈希槽全部转移到了9007,已经为0 ,此时再来删除我们的节点就行了~

[root@localhost redis-cluster]# /usr/local/redis-cluster/bin/redis-trib.rb del-node 192.168.56.127:9001 e070e76789352d2492cfc9800850e943e0c4b72d
>>> Removing node e070e76789352d2492cfc9800850e943e0c4b72d from cluster 192.168.56.127:9001
>>> Sending CLUSTER FORGET messages to the cluster...
>>> SHUTDOWN the node.

# 看下面结果 执行完del-node 9003节点就已经从集群中剔除啦~😁
>>> Performing Cluster Check (using node 192.168.56.127:9001)
M: 0fcbaa301451baf87284546513003568e47b5daa 192.168.56.127:9001
   slots:1280-5460 (4096 slots) master
   1 additional replica(s)
S: 5ed978b695e6a222e67d13c68e55f6c4e74b7ba6 192.168.56.127:9008
   slots: (0 slots) slave
   replicates 1714784542b3afc9e2a8b2c93fff49d095e7d72b
M: 1714784542b3afc9e2a8b2c93fff49d095e7d72b 192.168.56.127:9007
   slots:0-1279,5461-6998,10923-16383 (8192 slots) master
   2 additional replica(s)
S: 17db098db9fd222df002e649506f88efcdc83633 192.168.56.127:9005
   slots: (0 slots) slave
   replicates 1714784542b3afc9e2a8b2c93fff49d095e7d72b
M: 474fd53f7199ad70cce7f18bd68f5445b559509a 192.168.56.127:9002
   slots:6999-10922 (4096 slots) master
   1 additional replica(s)
S: bd278fcea63862199e964a6aab2e786710cf5575 192.168.56.127:9006
   slots: (0 slots) slave
   replicates 0fcbaa301451baf87284546513003568e47b5daa
S: 950c399165439660efb6ff2f907f938e213de530 192.168.56.127:9004
   slots: (0 slots) slave
   replicates 474fd53f7199ad70cce7f18bd68f5445b559509a
[OK] All nodes agree about slots configuration.
>>> Check for open slots...
>>> Check slots coverage...
[OK] All 16384 slots covered.

最后

当然我们集群搭建已经完成,那集群的作用是为了解决分片的问题,如果需要搭建高可用集群;也就是主的失败后从的能够切换角色,
所以可以使用哨兵机制来解决HA.
sentinel和cluster主要区别,sentinel用来解决HA(高可用)问题,而cluster主要解决sharding(分片)问题。两个经常结合使用搭建高可用集群。

关于哈希槽的概念:

Redis 集群中内置了 16384 个哈希槽,当需要在 Redis 集群中放置一个 key-value时,redis 先对 key 使用 crc16 算法算出一个结果,然后把结果对 16384 求余数,这样每个 key 都会对应一个编号在 0-16383 之间的哈希槽,redis 会根据节点数量大致均等的将哈希槽映射到不同的节点。

Redis 集群没有使用一致性hash, 而是引入了哈希槽的概念。

Redis 集群有16384个哈希槽,每个key通过CRC16校验后对16384取模来决定放置哪个槽.集群的每个节点负责一部分hash槽。这种结构很容易添加或者删除节点,并且无论是添加删除或者修改某一个节点,都不会造成集群不可用的状态。

使用哈希槽的好处就在于可以方便的添加或移除节点。

当需要增加节点时,只需要把其他节点的某些哈希槽挪到新节点就可以了;

当需要移除节点时,只需要把移除节点上的哈希槽挪到其他节点就行了;

在这一点上,我们以后新增或移除节点的时候不用先停掉所有的 redis 服务。

“用了哈希槽的概念,而没有用一致性哈希算法,不都是哈希么?这样做的原因是为什么呢?”
Redis Cluster是自己做的crc16的简单hash算法,没有用一致性hash。Redis的作者认为它的crc16(key) mod 16384的效果已经不错了,虽然没有一致性hash灵活,但实现很简单,节点增删时处理起来也很方便。

“为了动态增删节点的时候,不至于丢失数据么?”
节点增删时不丢失数据和hash算法没什么关系,不丢失数据要求的是一份数据有多个副本。

“还有集群总共有2的14次方,16384个哈希槽,那么每一个哈希槽中存的key 和 value是什么?”
当你往Redis Cluster中加入一个Key时,会根据crc16(key) mod 16384计算这个key应该分布到哪个hash slot中,一个hash slot中会有很多key和value。你可以理解成表的分区,使用单节点时的redis时只有一个表,所有的key都放在这个表里;改用Redis Cluster以后会自动为你生成16384个分区表,你insert数据时会根据上面的简单算法来决定你的key应该存在哪个分区,每个分区里有很多key。