redis-cluster集群的概念和工作原理

<code>在说明使用和安装之前，先大概介绍下，Redis 集群是一个可以在多个 Redis 节点之间进行数据共享的设施。通过分区（partition）来提供一定程度的可用性（availability）,即使集群中有一部分节点失效或者无法进行通讯,集群也可以继续处理命令请求。 

/<code>

• 架构图

• 工作原理

<code>1、所有的redis节点彼此互联(PING-PONG机制),内部使用二进制协议优化传输速度和带宽.
2、节点的fail是通过集群中超过半数的节点检测失效时才生效
3、客户端与Redis节点直连,不需要中间Proxy层，直接连接任意一个Master节点
4、redis-cluster把所有的物理节点映射到[0-16383]slot上，根据公式HASH_SLOT=CRC16(key) mod 16384，计算出映射到哪个分片上，然后Redis会去相应的节点进行操作
/<code>

• 优点和缺点

<code>1、优点:
(1)无需Sentinel哨兵监控，如果Master挂了，Redis Cluster内部自动将Slave切换Master
(2)可以进行水平扩容
(3)支持自动化迁移，当出现某个Slave宕机了，那么就只有Master了，这时候的高可用性就无法很好的保证了，万一master也宕机了，咋办呢？ 针对这种情况，如果说其他Master有多余的Slave ，集群自动把多余的Slave迁移到没有Slave的Master 中。
2、缺点:
(1)批量操作是个坑 

(2)资源隔离性较差，容易出现相互影响的情况。
/<code>

前期准备工作和说明

• 1、主机清单

<code>10.0.0.221 cyq-redis-01
10.0.0.224 cyq-redis-02
/<code>

• 2、前提2台机器都安装redis

<code>mkdir -p /data/redis
chmod  755 /data/redis
mkdir -p /data/logs/redis
chmod 777 /data/logs/redis
wget http://download.redis.io/releases/redis-3.2.11.tar.gz
tar -zxvf redis-3.2.11.tar.gz
mv redis-3.2.10 /usr/local/redis
cd /usr/local/redis
make 
cd src
make PREFIX=/usr/local/redis install
/<code>

一、集群环境开始安装

• 1、在cyq-redis-01 机器上执行

<code>mkdir /usr/local/redis-cluster
cd /usr/local/redis-cluster
mkdir 7001 7002 7003
cp /usr/local/redis/ /usr/local/redis-cluster/7001/ -r
cp /usr/local/redis/ /usr/local/redis-cluster/7002/ -r
cp /usr/local/redis/ /usr/local/redis-cluster/7003/ -r
/<code>

<code>增加数据路径
mkdir -p /data/redis/7001
mkdir -p /data/redis/7002
mkdir -p /data/redis/7003
/<code>

• 2、在cyq-redis-02机器上执行

<code>mkdir /usr/local/redis-cluster
cd /usr/local/redis-cluster
mkdir 7004 7005 7006
cp /usr/local/redis/ /usr/local/redis-cluster/7004/ -r
cp /usr/local/redis/ /usr/local/redis-cluster/7005/ -r
cp /usr/local/redis/ /usr/local/redis-cluster/7006/ -r
/<code>

<code>mkdir -p /data/redis/7004
mkdir -p /data/redis/7005
mkdir -p /data/redis/7006
/<code>

• 3、修改配置文件redis.conf（修改参数如下）

<code>port 7001  # 端口
bind 172.16.0.15  #绑定IP地址 
daemonize yes   #开启后台运行
pidfile /var/run/redis_7001.pid  #pid文件
logfile "/data/logs/redis/redis7001.log"  #日志文件
dir /data/redis/7001  #数据目录
cluster-enabled yes #启用集群模式
cluster-config-file nodes.conf  #生成的集群配置文件名称，集群搭建成功后会自动生成，在工作目录下
cluster-node-timeout 5000 #节点宕机发现时间，可以理解为主节点宕机后从节点升级为主节点时间
appendonly yes   #开启AOF模式
masterauth atx@7wixs 

requirepass atx@7wixs
/<code>

<code>6台redis都需要修改配置
sed -i "s#7001#7002#g" redis.conf

如果需要密码认证，提前加入账号：
masterauth atx@7wixs
requirepass atx@7wixs
/<code>

• 4、启动
• 4-1、在cyq-redis-01执行

<code>/usr/local/redis-cluster/7001/bin/redis-server  /usr/local/redis-cluster/7001/redis.conf
/usr/local/redis-cluster/7002/bin/redis-server  /usr/local/redis-cluster/7002/redis.conf
/usr/local/redis-cluster/7003/bin/redis-server  /usr/local/redis-cluster/7003/redis.conf
/<code>

• 4-2、在cyq-redis-02执行

<code>/usr/local/redis-cluster/7004/bin/redis-server  /usr/local/redis-cluster/7004/redis.conf
/usr/local/redis-cluster/7005/bin/redis-server  /usr/local/redis-cluster/7005/redis.conf
/usr/local/redis-cluster/7006/bin/redis-server  /usr/local/redis-cluster/7006/redis.conf
/<code>

• 5、查看进程状态

<code>[root@cyq-redis-02 ~]# ps -ef |grep redis
root     19755     1  0 16:32 ?        00:00:00 /usr/local/redis-cluster/7004/bin/redis-server 10.0.0.224:7004 [cluster]
root     19757     1  0 16:32 ?        00:00:00 /usr/local/redis-cluster/7005/bin/redis-server 10.0.0.224:7005 [cluster]
root     19763     1  0 16:32 ?        00:00:00 /usr/local/redis-cluster/7006/bin/redis-server 10.0.0.224:7006 [cluster]
/<code>

二、创建集群需要用到redis-trib这个命令，它依赖Ruby和RubyGems，因此我们要先安装一下

• 1、安装ruby环境和库支持

<code>yum install ruby rubygems -y
gem install redis   或者gem install redis --version 3.3.3
/<code>

• 2、如果安装过程中，提示ruby版本过低，可以安装rvm管理ruby.

<code>#!/bin/bash
curl -sSL https://get.rvm.io | bash
source /usr/local/rvm/scripts/rvm
rvm list known
#rvm install 2.0.0-p247  （安装指定版本）
rvm install 2.2.10
rvm list
rvm use
gem install redis
/<code>

• 3、设置密码之后如果需要使用redis-trib.rb的各种命令

<code>如：./redis-trib.rb check 127.0.0.1:7000，则会报错ERR] Sorry, can’t connect to node 127.0.0.1:7000 
vim /usr/local/rvm/gems/ruby-2.2.10/gems/redis-4.1.0/lib/redis/client.rb   修改密码password
require_relative "errors"
require "socket"
require "cgi"

class Redis
  class Client

    DEFAULTS = {
      :url => lambda { ENV["REDIS_URL"] },
      :scheme => "redis",
      :host => "127.0.0.1",
      :port => 6379,
      :path => nil,
      :timeout => 5.0, 

      :password => "atx@7wixs",
      :db => 0,
      :driver => nil,
      :id => nil,
      :tcp_keepalive => 0,
      :reconnect_attempts => 1,
      :reconnect_delay => 0,
      :reconnect_delay_max => 0.5,
      :inherit_socket => false
    }
/<code>

• 4、复制redis-trib.rb命令开始创建集群 cp /usr/local/redis/src/redis-trib.rb /usr/local/bin/redis-trib

<code>创建集群：
redis-trib create --replicas 1  10.0.0.221:7001 10.0.0.221:7002 10.0.0.221:7003 10.0.0.224:7004 10.0.0.224:7005 10.0.0.224:7006
命令的意义如下：
给定 redis-trib.rb 程序的命令是 create ， 这表示我们希望创建一个新的集群。
选项 --replicas 1 表示我们希望为集群中的每个主节点创建一个从节点。
之后跟着的其他参数则是实例的地址列表， 我们希望程序使用这些地址所指示的实例来创建新集群。
简单来说， 以上命令的意思就是让 redis-trib 程序创建一个包含三个主节点和三个从节点的集群。

接着， redis-trib 会打印出一份预想中的配置给你看， 如果你觉得没问题的话， 就可以输入 yes ， redis-trib 就会将这份配置应用到集群当中：
/<code>

<code>[root@cyq-redis-01 ~]# redis-trib create --replicas 1  10.0.0.221:7001 10.0.0.221:7002 10.0.0.221:7003 10.0.0.224:7004 10.0.0.224:7005 10.0.0.224:7006
>>> Creating cluster
>>> Performing hash slots allocation on 6 nodes... 

Using 3 masters:
10.0.0.221:7001
10.0.0.224:7004
10.0.0.221:7002
Adding replica 10.0.0.224:7005 to 10.0.0.221:7001
Adding replica 10.0.0.221:7003 to 10.0.0.224:7004
Adding replica 10.0.0.224:7006 to 10.0.0.221:7002
M: 9d1ae0b77bdc3604bb604c64a957b75137517e26 10.0.0.221:7001
   slots:0-5460 (5461 slots) master
M: bd79d80d1cb121094f718b7f53e8a8dfd4f2b7e1 10.0.0.221:7002
   slots:10923-16383 (5461 slots) master
S: 6a1688eb39211c014d180bbbac67012a0097359a 10.0.0.221:7003
   replicates bd46eddd2a3022723df19b3442c6500f51c7bbb3
M: bd46eddd2a3022723df19b3442c6500f51c7bbb3 10.0.0.224:7004
   slots:5461-10922 (5462 slots) master
S: f3f2686e8240547c0c1d9c60c20cdd5b01a791b0 10.0.0.224:7005
   replicates 9d1ae0b77bdc3604bb604c64a957b75137517e26
S: e5d4081e0c6e676bdfd93f5e7560ffd0cb03e5f9 10.0.0.224:7006
   replicates bd79d80d1cb121094f718b7f53e8a8dfd4f2b7e1
Can I set the above configuration? (type 'yes' to accept): yes
>>> Nodes configuration updated
>>> Assign a different config epoch to each node
>>> Sending CLUSTER MEET messages to join the cluster
Waiting for the cluster to join...
>>> Performing Cluster Check (using node 10.0.0.221:7001)
M: 9d1ae0b77bdc3604bb604c64a957b75137517e26 10.0.0.221:7001
   slots:0-5460 (5461 slots) master
   1 additional replica(s)
M: bd79d80d1cb121094f718b7f53e8a8dfd4f2b7e1 10.0.0.221:7002
   slots:10923-16383 (5461 slots) master
   1 additional replica(s)
S: f3f2686e8240547c0c1d9c60c20cdd5b01a791b0 10.0.0.224:7005
   slots: (0 slots) slave
   replicates 9d1ae0b77bdc3604bb604c64a957b75137517e26
S: 6a1688eb39211c014d180bbbac67012a0097359a 10.0.0.221:7003
   slots: (0 slots) slave
   replicates bd46eddd2a3022723df19b3442c6500f51c7bbb3
M: bd46eddd2a3022723df19b3442c6500f51c7bbb3 10.0.0.224:7004
   slots:5461-10922 (5462 slots) master
   1 additional replica(s)
S: e5d4081e0c6e676bdfd93f5e7560ffd0cb03e5f9 10.0.0.224:7006
   slots: (0 slots) slave
   replicates bd79d80d1cb121094f718b7f53e8a8dfd4f2b7e1
[OK] All nodes agree about slots configuration.
>>> Check for open slots...
>>> Check slots coverage...
[OK] All 16384 slots covered.
/<code>

• 5、校验命令

<code>[root@cyq-redis-01 ~]# redis-cli -h 10.0.0.221 -c -p 7001 -a atx@7wixs
10.0.0.221:7001> set aa "123456"
OK
10.0.0.221:7001> get aa
"123456"
10.0.0.221:7001> 

[root@cyq-redis-02 ~]# redis-cli -h 10.0.0.224 -c -p 7004 -a atx@7wixs
10.0.0.224:7004> get aa
-> Redirected to slot [1180] located at 10.0.0.221:7001
"123456"
10.0.0.221:7001> 

[root@cyq-redis-01 ~]# redis-cli -h 10.0.0.221 -c -p 7002 -a atx@7wixs
10.0.0.221:7002> getaa
(error) ERR unknown command 'getaa'
10.0.0.221:7002> get aa
-> Redirected to slot [1180] located at 10.0.0.221:7001
"123456"
10.0.0.221:7001> 
/<code>

• 6、集群状态

<code>[root@cyq-redis-01 ~]# redis-cli -h 10.0.0.221  -p 7001 -a atx@7wixs cluster nodes 
bd79d80d1cb121094f718b7f53e8a8dfd4f2b7e1 10.0.0.221:7002 master - 0 1550657022706 2 connected 10923-16383
9d1ae0b77bdc3604bb604c64a957b75137517e26 10.0.0.221:7001 myself,master - 0 0 1 connected 0-5460
f3f2686e8240547c0c1d9c60c20cdd5b01a791b0 10.0.0.224:7005 slave 9d1ae0b77bdc3604bb604c64a957b75137517e26 0 1550657021705 5 connected
6a1688eb39211c014d180bbbac67012a0097359a 10.0.0.221:7003 slave bd46eddd2a3022723df19b3442c6500f51c7bbb3 0 1550657021204 4 connected
bd46eddd2a3022723df19b3442c6500f51c7bbb3 10.0.0.224:7004 master - 0 1550657021705 4 connected 5461-10922
e5d4081e0c6e676bdfd93f5e7560ffd0cb03e5f9 10.0.0.224:7006 slave bd79d80d1cb121094f718b7f53e8a8dfd4f2b7e1 0 1550657021204 6 connected
/<code>

• 7 redis 集群启动和关闭脚本

<code>redis-cluster-start.sh
#!/bin/bash
source /etc/profile
/usr/local/redis-cluster/7001/bin/redis-server  /usr/local/redis-cluster/7001/redis.conf
/usr/local/redis-cluster/7002/bin/redis-server  /usr/local/redis-cluster/7002/redis.conf
/usr/local/redis-cluster/7003/bin/redis-server  /usr/local/redis-cluster/7003/redis.conf
 

/usr/local/redis-cluster/7004/bin/redis-server  /usr/local/redis-cluster/7004/redis.conf
/usr/local/redis-cluster/7005/bin/redis-server  /usr/local/redis-cluster/7005/redis.conf
/usr/local/redis-cluster/7006/bin/redis-server  /usr/local/redis-cluster/7006/redis.conf
/<code>

<code>redis-cluster-stop.sh
#!/bin/bash
source /etc/profile
/usr/local/redis-cluster/7001/bin/redis-cli -h 10.0.0.221 -p 7001 -a atx@7wixs shutdown
/usr/local/redis-cluster/7002/bin/redis-cli -h 10.0.0.221 -p 7002 -a atx@7wixs shutdown
/usr/local/redis-cluster/7003/bin/redis-cli -h 10.0.0.221 -p 7003 -a atx@7wixs shutdown
/usr/local/redis-cluster/7004/bin/redis-cli -h 10.0.0.224 -p 7004 -a atx@7wixs shutdown
/usr/local/redis-cluster/7005/bin/redis-cli -h 10.0.0.224 -p 7005 -a atx@7wixs shutdown
/usr/local/redis-cluster/7006/bin/redis-cli -h 10.0.0.224 -p 7006 -a atx@7wixs shutdown/<code>

閱讀更多 運維攻城獅 的文章

關鍵字: Redis 部署 使用