redis6数据库

发表于 分类于 ，
Redis6数据库学习笔记：

redis6数据库

redis6的编译安装

软件版本
redis6.2.1
节点IP系统功能CPU内存硬盘
node110.80.10.1centos7.9redis4核心8GB20GB

node1

redis简介：

  • redis是一种内存型的nosql数据库。

  • redis存储数据的方法是以key-value的形式。

  • 类型支持字符串、列表、哈希等多种类型。

node1

安装redis环境：

1
# yum install -y gcc gcc-c++ openssl openssl-devel cmake

下载安装redis：

下载地址：https://download.redis.io/releases/

1
2
3
4
5
# cd /usr/local/src/
# wget https://download.redis.io/releases/redis-6.2.1.tar.gz
# tar -zxvf redis-6.2.1.tar.gz
# cd redis-6.2.1
# make -j 4 && make -j 4 install

验证安装：

1
2
# redis-server -v
Redis server v=6.2.1 sha=00000000:0 malloc=jemalloc-5.1.0 bits=64 build=6561781fbc748ebe

其他服务器可以直接拷贝redis命令进行安装：

1
2
# ls /usr/local/bin/redis-*
/usr/local/bin/redis-benchmark  /usr/local/bin/redis-check-aof  /usr/local/bin/redis-check-rdb  /usr/local/bin/redis-cli  /usr/local/bin/redis-sentinel  /usr/local/bin/redis-server

redis6的配置启动和参数调优

node1

拷贝配置文件：

1
2
3
# cp /usr/local/src/redis-6.2.1/redis.conf /etc/
# cp /etc/redis.conf /etc/redis.conf.bak
# sed -i '/^#/d; /^$/d' /etc/redis.conf

修改配置：

1
2
3
4
5
6
7
8
9
10
11
12
13
# vim /etc/redis.conf
# 1行，修改配置
bind 0.0.0.0
# 20行，修改配置
dir /data/redis
# 8行，修改配置
pidfile "redis.pid"
# 10行，修改配置
logfile "redis.log"
# 7行，修改配置
daemonize yes
# 尾行，添加配置
requirepass 123456
1
# cp /etc/redis.conf /etc/redis.conf.bak2

创建数据目录：

1
# mkdir -p /data/redis

systemctl管理redis：

1
2
3
4
5
6
7
8
9
10
11
# vim /usr/lib/systemd/system/redis.service
[Unit]
Description=redis
After=network.target

[Service]
Type=forking
ExecStart=/usr/local/bin/redis-server /etc/redis.conf 

[Install]
WantedBy=multi-user.target

启动redis：

1
2
3
# systemctl start redis
# systemctl enable redis
# systemctl status redis

查看端口和进程：

1
2
3
4
5
# ps aux | grep redis
root      18480  0.1  0.1 162492  9832 ?        Ssl  21:10   0:00 /usr/local/bin/redis-server 0.0.0.0:6379
root      18846  0.0  0.0 112824   984 pts/0    S+   21:11   0:00 grep --color=auto redis
# netstat -tlunp | grep redis
tcp        0      0 0.0.0.0:6379            0.0.0.0:*               LISTEN      18480/redis-server

查看日志，修改系统配置：

1
2
3
4
5
6
7
8
9
10
11
# cat /data/redis/redis.log
18480:C 16 Dec 2023 21:10:51.127 # oO0OoO0OoO0Oo Redis is starting oO0OoO0OoO0Oo
18480:C 16 Dec 2023 21:10:51.127 # Redis version=6.2.1, bits=64, commit=00000000, modified=0, pid=18480, just started
18480:C 16 Dec 2023 21:10:51.127 # Configuration loaded
18480:M 16 Dec 2023 21:10:51.128 * Increased maximum number of open files to 10032 (it was originally set to 1024).
18480:M 16 Dec 2023 21:10:51.128 * monotonic clock: POSIX clock_gettime
18480:M 16 Dec 2023 21:10:51.128 * Running mode=standalone, port=6379.
18480:M 16 Dec 2023 21:10:51.128 # WARNING: The TCP backlog setting of 511 cannot be enforced because /proc/sys/net/core/somaxconn is set to the lower value of 128.
18480:M 16 Dec 2023 21:10:51.128 # Server initialized
18480:M 16 Dec 2023 21:10:51.128 # WARNING overcommit_memory is set to 0! Background save may fail under low memory condition. To fix this issue add 'vm.overcommit_memory = 1' to /etc/sysctl.conf and then reboot or run the command 'sysctl vm.overcommit_memory=1' for this to take effect.
18480:M 16 Dec 2023 21:10:51.128 * Ready to accept connections

修改最大文件打开数：

1
2
3
# vim /etc/security/limits.conf
# 尾行，添加配置
* - nofile 102400

重新登陆生效：

1
2
# ulimit -n
102400

修改内核参数：

1
2
3
4
# vim /etc/sysctl.conf
# 尾行，添加配置
net.core.somaxconn = 10240
vm.overcommit_memory = 1
1
2
3
# sysctl -p
net.core.somaxconn = 10240
vm.overcommit_memory = 1

配置内存设置，永久生效：

1
2
3
4
# echo madvise > /sys/kernel/mm/transparent_hugepage/enabled
# vim /etc/rc.local
# 尾行，添加配置
echo madvise > /sys/kernel/mm/transparent_hugepage/enabled
1
# chmod +x /etc/rc.d/rc.local

清空日志：

1
# :> /data/redis/redis.log

重启redis：

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
# systemctl restart redis
# cat /data/redis/redis.log
18480:signal-handler (1702732346) Received SIGTERM scheduling shutdown...
18480:M 16 Dec 2023 21:12:26.562 # User requested shutdown...
18480:M 16 Dec 2023 21:12:26.562 * Saving the final RDB snapshot before exiting.
18480:M 16 Dec 2023 21:12:26.563 * DB saved on disk
18480:M 16 Dec 2023 21:12:26.563 * Removing the pid file.
18480:M 16 Dec 2023 21:12:26.563 # Redis is now ready to exit, bye bye...
20400:C 16 Dec 2023 21:12:26.566 # oO0OoO0OoO0Oo Redis is starting oO0OoO0OoO0Oo
20400:C 16 Dec 2023 21:12:26.566 # Redis version=6.2.1, bits=64, commit=00000000, modified=0, pid=20400, just started
20400:C 16 Dec 2023 21:12:26.566 # Configuration loaded
20400:M 16 Dec 2023 21:12:26.567 * Increased maximum number of open files to 10032 (it was originally set to 1024).
20400:M 16 Dec 2023 21:12:26.567 * monotonic clock: POSIX clock_gettime
20400:M 16 Dec 2023 21:12:26.567 * Running mode=standalone, port=6379.
20400:M 16 Dec 2023 21:12:26.567 # Server initialized
20400:M 16 Dec 2023 21:12:26.567 * Loading RDB produced by version 6.2.1
20400:M 16 Dec 2023 21:12:26.567 * RDB age 0 seconds
20400:M 16 Dec 2023 21:12:26.567 * RDB memory usage when created 0.77 Mb
20400:M 16 Dec 2023 21:12:26.567 * DB loaded from disk: 0.000 seconds
20400:M 16 Dec 2023 21:12:26.567 * Ready to accept connections

redis字符串类型基本操作

node1

redis-cli登陆redis：

1
# redis-cli

  • -h：指定ip。

  • -p：指定端口。

  • -a：认证。

密码认证：

1
2
127.0.0.1:6379> auth 123456
OK

查看信息：

1
127.0.0.1:6379> info

字符串操作：

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
127.0.0.1:6379> set name student
OK

127.0.0.1:6379> get name
"student"

127.0.0.1:6379> set name teacher
OK

127.0.0.1:6379> get name
"teacher"

127.0.0.1:6379> del name
(integer) 1

127.0.0.1:6379> get name
(nil)

退出redis：

1
127.0.0.1:6379> quit

命令区不分大小写，key-value区分大小写。

非交互式操作redis，将warnning报错重定向：

1
2
3
4
5
6
7
# redis-cli -a 123456 info
# redis-cli -a 123456 set name classmate 2> /dev/null
OK
# redis-cli -a 123456 get name 2> /dev/null
"classmate"
# redis-cli -a 123456 del name 2> /dev/null
(integer) 1

shell批量写入数据：

1
2
3
4
# for i in $(seq -w 10);do
    redis-cli -a 123456 set name${i} student${i}
    redis-cli -a 123456 get name${i}
  done 2> /dev/null

reids列表和集合的基本操作

node1

redis列表：有顺序、数据可重复。

rpush右添加：

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
# redis-cli -a 123456

127.0.0.1:6379> rpush names student1
(integer) 1

127.0.0.1:6379> rpush names student2
(integer) 2

127.0.0.1:6379> rpush names student3
(integer) 3

127.0.0.1:6379> rpush names student4
(integer) 4

127.0.0.1:6379> rpush names student5
(integer) 5

127.0.0.1:6379> rpush names student1
(integer) 6

127.0.0.1:6379> rpush names student2
(integer) 7

查看names列表所有数据：

1
2
3
4
5
6
7
8
127.0.0.1:6379> lrange names 0 -1
1) "student1"
2) "student2"
3) "student3"
4) "student4"
5) "student5"
6) "student1"
7) "student2"

lpush左添加：

1
2
3
4
5
6
7
8
9
10
11
12
13
14
127.0.0.1:6379> lpush names2 teacher1
(integer) 1

127.0.0.1:6379> lpush names2 teacher2
(integer) 2

127.0.0.1:6379> lpush names2 teacher3
(integer) 3

127.0.0.1:6379> lpush names2 teacher1
(integer) 4

127.0.0.1:6379> lpush names2 teacher2
(integer) 5

查看names列表所有数据：

1
2
3
4
5
6
127.0.0.1:6379> lrange names2 0 -1
1) "teacher2"
2) "teacher1"
3) "teacher3"
4) "teacher2"
5) "teacher1"

移除一个student1元素，默认移除最早添加的：

1
2
3
4
5
6
7
8
9
10
127.0.0.1:6379> lrem names 1 student1
(integer) 1

127.0.0.1:6379> lrange names 0 -1
1) "student2"
2) "student3"
3) "student4"
4) "student5"
5) "student1"
6) "student2"

移除所有student2元素：

1
2
3
4
5
6
7
8
127.0.0.1:6379> lrem names 0 student2
(integer) 2

127.0.0.1:6379> lrange names 0 -1
1) "student3"
2) "student4"
3) "student5"
4) "student1"

删除重新写入数据：

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
127.0.0.1:6379> del names
(integer) 1

127.0.0.1:6379> rpush names student1
(integer) 1

127.0.0.1:6379> rpush names student2
(integer) 2

127.0.0.1:6379> rpush names student3
(integer) 3

127.0.0.1:6379> rpush names student4
(integer) 4

127.0.0.1:6379> rpush names student5
(integer) 5

127.0.0.1:6379> rpush names student6
(integer) 6

127.0.0.1:6379> lrange names 0 -1
1) "student1"
2) "student2"
3) "student3"
4) "student4"
5) "student5"
6) "student6"

pop左弹出，弹出第一个元素：

1
2
3
4
5
6
7
8
9
127.0.0.1:6379> lpop names
"student1"

127.0.0.1:6379> lrange names 0 -1
1) "student2"
2) "student3"
3) "student4"
4) "student5"
5) "student6"

rpop右弹出，弹出最后一个元素：

1
2
3
4
5
6
7
8
127.0.0.1:6379> rpop names
"student6"

127.0.0.1:6379> lrange names 0 -1
1) "student2"
2) "student3"
3) "student4"
4) "student5"

rpush和lpop，先进先出。rpush和rpop，后进先出：

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
127.0.0.1:6379> del names
(integer) 1

127.0.0.1:6379> rpush names student1
(integer) 1

127.0.0.1:6379> rpush names student2
(integer) 2

127.0.0.1:6379> rpush names student3
(integer) 3

127.0.0.1:6379> rpush names student4
(integer) 4

127.0.0.1:6379> rpush names student5
(integer) 5

127.0.0.1:6379> rpush names student6
(integer) 6

127.0.0.1:6379> lrange names 0 -1
1) "student1"
2) "student2"
3) "student3"
4) "student4"
5) "student5"
6) "student6"
1
2
3
4
5
6
7
8
9
10
11
127.0.0.1:6379> lpop names
"student1"

127.0.0.1:6379> rpop names
"student6"

127.0.0.1:6379> lrange names 0 -1
1) "student2"
2) "student3"
3) "student4"
4) "student5"

redis集合：无顺序的，不重复的。

sadd添加：

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
127.0.0.1:6379> del names
(integer) 1

127.0.0.1:6379> sadd names student1
(integer) 1

127.0.0.1:6379> sadd names student2
(integer) 1

127.0.0.1:6379> sadd names student3
(integer) 1

127.0.0.1:6379> sadd names student4
(integer) 1

127.0.0.1:6379> sadd names student5
(integer) 1

127.0.0.1:6379> sadd names student1
(integer) 0

127.0.0.1:6379> sadd names student2
(integer) 0

127.0.0.1:6379> smembers names
1) "student3"
2) "student1"
3) "student5"
4) "student4"
5) "student2"

删除student1元素：

1
2
3
4
5
6
7
8
127.0.0.1:6379> srem names student1
(integer) 1

127.0.0.1:6379> smembers names
1) "student5"
2) "student4"
3) "student2"
4) "student3"

随机删除元素：

1
2
3
4
5
6
7
127.0.0.1:6379> spop names
"student3"

127.0.0.1:6379> smembers names
1) "student5"
2) "student2"
3) "student4"

判断元素是否存在，1在0不在：

1
2
127.0.0.1:6379> sismember names student3
(integer) 0

redis哈希和订阅类型基本操作

node1

hash写入：

1
2
3
4
5
6
7
8
127.0.0.1:6379> hset info name student
(integer) 1

127.0.0.1:6379> hset info location china
(integer) 1

127.0.0.1:6379> hset info age 100
(integer) 1

获取全部数据：

1
2
3
4
5
6
7
127.0.0.1:6379> hgetall info
1) "name"
2) "student"
3) "location"
4) "china"
5) "age"
6) "100"

获取单个数据：

1
2
3
4
5
6
7
8
127.0.0.1:6379> hget info name
"student"

127.0.0.1:6379> hget info location
"china"

127.0.0.1:6379> hget info age
"100"

删除单个数据：

1
2
3
4
5
6
7
8
127.0.0.1:6379> hdel info age
(integer) 1

127.0.0.1:6379> hgetall info
1) "name"
2) "student"
3) "location"
4) "china"

删除所有数据：

1
2
3
4
5
127.0.0.1:6379> del info
(integer) 1

127.0.0.1:6379> hgetall info
(empty array)

批量添加：

1
2
3
4
5
6
7
8
9
10
127.0.0.1:6379> hmset info name teacher age 1000 location china
OK

127.0.0.1:6379> hgetall info
1) "name"
2) "teacher"
3) "age"
4) "1000"
5) "location"
6) "china"

redis的发布订阅：

1
2
3
4
5
127.0.0.1:6379> subscribe newpro
Reading messages... (press Ctrl-C to quit)
1) "subscribe"
2) "newpro"
3) (integer) 1

新建终端，发布订阅：

1
2
3
4
5
6
7
# redis-cli -a 123456

127.0.0.1:6379> subscribe newpro
Reading messages... (press Ctrl-C to quit)
1) "subscribe"
2) "newpro"
3) (integer) 1

再次新建终端进行发布，其他订阅的终端都会收到：

1
2
3
4
5
6
# redis-cli -a 123456

127.0.0.1:6379> publish newpro "name teacher, age 100"
(integer) 2

127.0.0.1:6379> quit

python操作redis单例实战

node1

安装python环境：

1
2
# yum install -y python36
# pip3 install redis==3.5.3 -i https://mirrors.aliyun.com/pypi/simple/

操作redis字符串：

1
2
3
4
5
6
7
8
# cd
# vim testredis.py
import redis

r = redis.Redis(host='127.0.0.1', port=6379, password='123456', db=0)
r.set('pyname','testvalue')
value = r.get('pyname')
print(value)
1
2
# python3 testredis.py
b'testvalue'

操作redis列表：

1
2
3
4
5
6
7
8
9
10
11
# vim redislist.py
import redis

r = redis.Redis(host='127.0.0.1', port=6379, password='123456', db=0)
r.rpush('pynames','pylist1')
r.rpush('pynames','pylist2')
r.rpush('pynames','pylist3')
result = r.lpop('pynames')
print(result)
result = r.lrange('pynames',0,-1)
print(result)
1
2
3
# python3 redislist.py
b'pylist1'
[b'pylist2', b'pylist3']

操作redis hash：

1
2
3
4
5
6
7
8
9
# vim redishash.py
import redis

r = redis.Redis(host='127.0.0.1', port=6379, password='123456', db=0)
r.hset('pyinfo','name','teacher')
r.hset('pyinfo','age',100)
r.hset('pyinfo','city','china')
result = r.hgetall('pyinfo')
print(result)
1
2
# python3 redishash.py
{b'name': b'teacher', b'age': b'100', b'city': b'china'}

redis多数据库切换

node1

redis默认16个库：

1
2
# cat /etc/redis.conf | grep databases
databases 16

默认使用0库：

1
2
3
4
5
# redis-cli -a 123456

127.0.0.1:6379> info keyspace
# Keyspace
db0:keys=16,expires=0,avg_ttl=0

切换1库：

1
2
3
4
5
6
7
8
9
10
127.0.0.1:6379> select 1
OK

127.0.0.1:6379[1]> set name 123
OK

127.0.0.1:6379[1]> info keyspace
# Keyspace
db0:keys=16,expires=0,avg_ttl=0
db1:keys=1,expires=0,avg_ttl=0

不同库数据不影响：

1
2
3
4
5
6
7
8
9
10
127.0.0.1:6379[1]> select 0
OK

127.0.0.1:6379> get name
(nil)

127.0.0.1:6379> info keyspace
# Keyspace
db0:keys=16,expires=0,avg_ttl=0
db1:keys=1,expires=0,avg_ttl=0

redis运维监控命令

node1

随机返回key：

1
2
3
4
5
6
7
8
127.0.0.1:6379> randomkey
"name02"

127.0.0.1:6379> randomkey
"name09"

127.0.0.1:6379> randomkey
"name06"

返回所有key：

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
127.0.0.1:6379> info keyspace
# Keyspace
db0:keys=16,expires=0,avg_ttl=0
db1:keys=1,expires=0,avg_ttl=0

127.0.0.1:6379> keys *
 1) "name08"
 2) "pyname"
 3) "name09"
 4) "name01"
 5) "name10"
 6) "name04"
 7) "pynames"
 8) "name07"
 9) "names2"
10) "names"
11) "pyinfo"
12) "name05"
13) "info"
14) "name06"
15) "name03"
16) "name02"

新建终端，添加数据：

1
2
3
# for i in $(seq -w 30); do
    redis-cli -a 123456 set name${i} student${i}
  done 2> /dev/null

返回旧终端，每次获取十个keys，直到再次返回0：

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
127.0.0.1:6379> scan 0
1) "60"
2)  1) "name05"
    2) "name02"
    3) "name28"
    4) "name09"
    5) "pynames"
    6) "name10"
    7) "name19"
    8) "name14"
    9) "name16"
   10) "pyinfo"
   11) "name24"

127.0.0.1:6379> scan 60
1) "49"
2)  1) "name22"
    2) "name13"
    3) "names2"
    4) "info"
    5) "name06"
    6) "name12"
    7) "name17"
    8) "name21"
    9) "names"
   10) "name08"
   11) "name18"

127.0.0.1:6379> scan 49
1) "3"
2)  1) "name07"
    2) "name15"
    3) "name29"
    4) "name03"
    5) "name25"
    6) "name11"
    7) "pyname"
    8) "name04"
    9) "name30"
   10) "name01"

127.0.0.1:6379> scan 3
1) "0"
2) 1) "name26"
   2) "name23"
   3) "name20"
   4) "name27"

127.0.0.1:6379> quit

scan扫描redis数据：

1
2
3
4
5
6
7
8
9
10
# vim redisscan.py
import redis, time

r = redis.Redis(host='127.0.0.1', port=6379, password='123456', db=0) 
result = r.scan(0)
print(result)
while result[0] != 0:
  result = r.scan(result[0])
  print(result)
  time.sleep(0.2)
1
2
3
4
5
# python3 redisscan.py
(60, [b'name05', b'name02', b'name28', b'name09', b'pynames', b'name10', b'name19', b'name14', b'name16', b'pyinfo', b'name24'])
(49, [b'name22', b'name13', b'names2', b'info', b'name06', b'name12', b'name17', b'name21', b'names', b'name08', b'name18'])
(3, [b'name07', b'name15', b'name29', b'name03', b'name25', b'name11', b'pyname', b'name04', b'name30', b'name01'])
(0, [b'name26', b'name23', b'name20', b'name27'])

实时监控redis状态：

1
# redis-cli -a 123456 --stat

实时监控redis操作：

1
# redis-cli -a 123456 monitor

新建终端，操作redis数据库，会被记录操作

1
2
3
4
5
6
7
8
9
10
11
12
13
# redis-cli -a 123456

127.0.0.1:6379> info keyspace

127.0.0.1:6379> set a b
OK

127.0.0.1:6379> set name teacher
OK

127.0.0.1:6379> keys *

127.0.0.1:6379> quit

查看redis信息：

1
# redis-cli -a 123456 info
1
2
3
4
5
6
7
8
9
10
11
12
Server：服务器信息
Memory：Redis内存占用
  used_memory：数据占用内存
  used_memory_rss：实际使用内存
  mem_fragmentation_ratio：内存碎片化
  maxmemory：最大内存限制
Persistence：持久化信息
Stats：状态信息：监控重点
Replication：主从状态监控
CPU：sys、user占用
Cluster：Redis集群信息
Keyspace：键的分布信息
1
2
# redis-cli -a 123456 info clients
# redis-cli -a 123456 info memory

redis配置动态更新和写入

node1

修改密码：

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
# redis-cli -a 123456

127.0.0.1:6379> config get requirepass
1) "requirepass"
2) "123456"

127.0.0.1:6379> config set requirepass 123456789
OK

127.0.0.1:6379> config get requirepass
1) "requirepass"
2) "123456789"

127.0.0.1:6379> config set requirepass 123456
OK

127.0.0.1:6379> config rewrite
OK
  • config rewrite：写入配置文件。

修改最大连接数：

1
2
3
4
5
6
7
8
9
10
11
12
13
127.0.0.1:6379> config get maxclients
1) "maxclients"
2) "10000"

127.0.0.1:6379> config set maxclients 20000
OK

127.0.0.1:6379> config rewrite
OK

127.0.0.1:6379> config get maxclients
1) "maxclients"
2) "20000"

redis6多用户管理

老版本没有用户名，只有密码，redis6已经有用户名，默认用户名default。

node1

查看用户：

1
2
127.0.0.1:6379> acl list
1) "user default on #8d969eef6ecad3c29a3a629280e686cf0c3f5d5a86aff3ca12020c923adc6c92 ~* &* +@all"

查看default用户权限：

1
2
3
4
5
6
7
8
9
10
11
12
13
14
127.0.0.1:6379> acl getuser default
 1) "flags"
 2) 1) "on"
    2) "allkeys"
    3) "allchannels"
    4) "allcommands"
 3) "passwords"
 4) 1) "8d969eef6ecad3c29a3a629280e686cf0c3f5d5a86aff3ca12020c923adc6c92"
 5) "commands"
 6) "+@all"
 7) "keys"
 8) 1) "*"
 9) "channels"
10) 1) "*"

default对所有key，所有命令均有权限，最高权限。默认用户，使用auth进行认证。

创建student用户：

1
2
3
4
5
6
127.0.0.1:6379> acl setuser student on >123456 ~name* +get
OK

127.0.0.1:6379> acl list
1) "user default on #8d969eef6ecad3c29a3a629280e686cf0c3f5d5a86aff3ca12020c923adc6c92 ~* &* +@all"
2) "user student on #8d969eef6ecad3c29a3a629280e686cf0c3f5d5a86aff3ca12020c923adc6c92 ~name* &* -@all +get"

新建终端，student用户登录有两种方式，student用户默认读权限可以读取有value的key：

1
2
3
4
5
6
7
8
9
10
11
12
13
14
# redis-cli

127.0.0.1:6379> auth student 123456
OK
127.0.0.1:6379> get name10
"student10"

127.0.0.1:6379> get abc
(error) NOPERM this user has no permissions to access one of the keys used as arguments

127.0.0.1:6379> set a b
(error) NOPERM this user has no permissions to run the 'set' command or its subcommand

127.0.0.1:6379> quit
1
2
3
4
5
6
7
8
9
# redis-cli --user student --pass 123456

127.0.0.1:6379> get name10
"student10"

127.0.0.1:6379> get abc
(error) NOPERM this user has no permissions to access one of the keys used as arguments

127.0.0.1:6379> quit

返回旧终端，查看权限类型：

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
127.0.0.1:6379> acl cat
 1) "keyspace"
 2) "read"
 3) "write"
 4) "set"
 5) "sortedset"
 6) "list"
 7) "hash"
 8) "string"
 9) "bitmap"
10) "hyperloglog"
11) "geo"
12) "stream"
13) "pubsub"
14) "admin"
15) "fast"
16) "slow"
17) "blocking"
18) "dangerous"
19) "connection"
20) "transaction"
21) "scripting"

授权student用户read权限，可以读取所有key：

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
127.0.0.1:6379> acl setuser student on >123456 ~* +@read
OK

127.0.0.1:6379> acl getuser student
 1) "flags"
 2) 1) "on"
    2) "allkeys"
    3) "allchannels"
 3) "passwords"
 4) 1) "8d969eef6ecad3c29a3a629280e686cf0c3f5d5a86aff3ca12020c923adc6c92"
 5) "commands"
 6) "-@all +@read"
 7) "keys"
 8) 1) "*"
 9) "channels"
10) 1) "*"

授权monitor用户所有权限：

1
2
127.0.0.1:6379> acl setuser monitor on >123456 ~* +@all
OK

去除student用户config权限：

1
2
127.0.0.1:6379> acl setuser student on >123456 ~* +@all -config
OK

授权monitor用户监控权限：

1
2
127.0.0.1:6379> acl setuser monitor on >123456 ~* +info +monitor
OK

使用acl创建用户写入配置文件，重启redis用户信息才不会丢失

1
2
3
4
127.0.0.1:6379> config rewrite
OK

127.0.0.1:6379> quit

redis弱密码入侵及慢日志实战

node1

入侵无密码或弱密码的redis服务器，能够更改系统文件，模拟备入侵：

1
2
3
4
5
6
7
8
9
10
11
12
13
14
# redis-cli -a 123456

127.0.0.1:6379> config get dir
1) "dir"
2) "/data/redis"

127.0.0.1:6379> config set dir '/etc/'
OK

127.0.0.1:6379> config set dbfilename 'crontab'
OK

127.0.0.1:6379> bgsave
Background saving started

新建终端，crontab被篡改：

1
2
# cat /etc/crontab
# :> /etc/crontab

返回旧终端，还原配置：

1
2
3
4
5
127.0.0.1:6379> config set dir '/data/redis'
OK

127.0.0.1:6379> config set dbfilename 'dump.rdb'
OK

另外被入侵后，也会被随意更改数据，随意删除数据等 。

新建终端，批量写入数据：

1
2
3
4
# for i in $(seq -w 10000); do
    redis-cli -a 123456 set name$i student${i}
    redis-cli -a 123456 get name$i
  done 2> /dev/null

返回旧终端，查看慢日志时间：

1
2
3
4
5
6
7
8
9
10
127.0.0.1:6379> config get slow*
1) "slowlog-max-len"
2) "128"
3) "slowlog-log-slower-than"
4) "10000"

127.0.0.1:6379> keys *

127.0.0.1:6379> slowlog get
(empty array)

修改慢日志时间1毫秒，查看慢日志：

1
2
3
4
5
6
7
8
9
10
11
12
13
127.0.0.1:6379> config set slowlog-log-slower-than 1000
OK

127.0.0.1:6379> keys *

127.0.0.1:6379> slowlog get
1) 1) (integer) 0
   2) (integer) 1702737493
   3) (integer) 1559
   4) 1) "keys"
      2) "*"
   5) "127.0.0.1:36914"
   6) ""

多查询几次，产生慢日志：

1
2
3
4
5
6
7
8
9
127.0.0.1:6379> keys *
127.0.0.1:6379> keys *
127.0.0.1:6379> keys *
127.0.0.1:6379> keys *
127.0.0.1:6379> keys *
127.0.0.1:6379> keys *
127.0.0.1:6379> keys *
127.0.0.1:6379> keys *
127.0.0.1:6379> keys *

查看慢日志总数：

1
2
127.0.0.1:6379> slowlog len
(integer) 10

获取5条慢日志，默认获取10条：

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
127.0.0.1:6379> slowlog get 5
1) 1) (integer) 9
   2) (integer) 1702737526
   3) (integer) 1576
   4) 1) "keys"
      2) "*"
   5) "127.0.0.1:36914"
   6) ""
2) 1) (integer) 8
   2) (integer) 1702737524
   3) (integer) 1495
   4) 1) "keys"
      2) "*"
   5) "127.0.0.1:36914"
   6) ""
3) 1) (integer) 7
   2) (integer) 1702737523
   3) (integer) 1500
   4) 1) "keys"
      2) "*"
   5) "127.0.0.1:36914"
   6) ""
4) 1) (integer) 6
   2) (integer) 1702737522
   3) (integer) 1406
   4) 1) "keys"
      2) "*"
   5) "127.0.0.1:36914"
   6) ""
5) 1) (integer) 5
   2) (integer) 1702737520
   3) (integer) 1443
   4) 1) "keys"
      2) "*"
   5) "127.0.0.1:36914"
   6) ""

slowlog各字段意思：

1
2
3
4
5
6
7
8
127.0.0.1:6379> slowlog get
 1) 1) (integer) 9               --->id
    2) (integer) 1702737526      --->time
    3) (integer) 1576            --->run time
    4) 1) "keys"
       2) "*"
    5) "127.0.0.1:36914"
    6) ""

时间戳转换时间：

1
2
# date -d @1702737526
Sat Dec 16 22:38:46 CST 2023

清空慢日志：

1
2
3
4
5
127.0.0.1:6379> slowlog reset
OK

127.0.0.1:6379> slowlog get
(empty array)

redis的key有效期及最大内存

node1

给redis的key设置有效期：

1
2
3
4
5
127.0.0.1:6379> set name friend
OK

127.0.0.1:6379> ttl name
(integer) -1

  • -1：表示永久。

  • -2：代表被redis回收了。

设置有效期，20秒后失效：

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
127.0.0.1:6379> expire name 20
(integer) 1

127.0.0.1:6379> ttl name
(integer) 17

127.0.0.1:6379> get name
"friend"

127.0.0.1:6379> ttl name
(integer) 3

127.0.0.1:6379> get name
(nil)

127.0.0.1:6379> ttl name
(integer) -2
1
2
3
4
5
6
7
8
127.0.0.1:6379> set name friend ex 20
OK

127.0.0.1:6379> ttl name
(integer) 16

127.0.0.1:6379> get name
"friend"

redis限制内存使用：

  • maxmemory设置最大内存。

  • maxmemory超过最大内存可设置删除算法。

设置redis的最大内存使用2m，模拟超出最大内存：

1
2
3
4
5
6
7
8
9
10
11
12
127.0.0.1:6379> config get maxmemory*
1) "maxmemory-policy"
2) "noeviction"
3) "maxmemory-samples"
4) "5"
5) "maxmemory-eviction-tenacity"
6) "10"
7) "maxmemory"
8) "0"

127.0.0.1:6379> config set maxmemory 2m
OK

达到最大内存的删除算法：

  • noeviction：直接返回错误。

  • volatile-lru设置过期的key：使用lru算法删除。

  • volatile-random设置过期的key: 随机删除。

  • volatile-ttl设置过期的key：删除ttl小的。

  • allkeys-lru：所有key使用lru算法删除。

  • allkeys-random：所有key随机删除。

新建终端，模拟超过最大内存，直接返回错误：

1
2
3
4
# for line in $(seq -w 10000); do 
    redis-cli -a 123456 set name_${line} teacher_${line} 
    redis-cli -a 123456 get name_${line} 
  done 2> /dev/null

返回旧终端，查看设置：

1
2
3
4
5
6
7
8
9
127.0.0.1:6379> config get maxmemory*
1) "maxmemory-policy"
2) "noeviction"
3) "maxmemory-samples"
4) "5"
5) "maxmemory-eviction-tenacity"
6) "10"
7) "maxmemory"
8) "2000000"

设置随机删除算法：

1
2
3
4
5
6
7
127.0.0.1:6379> config set maxmemory-policy volatile-random
OK

127.0.0.1:6379> flushall
OK

127.0.0.1:6379> quit

模拟超过最大内存，设置过期可以一直添加，不设置过期时间因为无法删除过期key报错：

1
2
3
4
# for line in $(seq -w 10000); do
    redis-cli -a 123456 set name_${line} teacher_${line} ex 3600
    redis-cli -a 123456 get name_${line}
  done 2> /dev/null

redis禁用危险命令和压测

redis禁用危险命令：

  • flushall会清空redis所有数据。

  • flushdb会清除当前db所有数据。

  • keys在键过多的时候使用会阻塞业务请求。

  • config命令能更改系统文件。

node1

redis禁用危险命令，需要重启：

1
2
3
4
5
# vim /etc/redis.conf
# 尾行，添加配置
rename-command FLUSHALL ""
rename-command FLUSHDB ""
rename-command KEYS "" 
1
# systemctl restart redis

测试命令：

1
2
3
4
5
6
7
8
9
10
11
12
# redis-cli -a 123456

127.0.0.1:6379> keys *
(error) ERR unknown command `keys`, with args beginning with: `*`, 

127.0.0.1:6379> flushdb
(error) ERR unknown command `flushdb`, with args beginning with: 

127.0.0.1:6379> flushall
(error) ERR unknown command `flushall`, with args beginning with: 

127.0.0.1:6379> quit

redis压力测试：

1
2
# redis-benchmark --help
# redis-benchmark -a 123456 | tee /tmp/new.log

指定10个请求：

1
# redis-benchmark -a 123456 -n 10

redis的rdb存储实战

redis的运行方式：

  • redis提供持久化存储的功能。

  • redis如果提供缓存的服务，可以关闭所有存储功能。

redis支持的存储方式：

  • rdb存储，rdb存储是redis实现的一种存储机制，默认是开启的。

  • aof存储，直接把操作的命令记录下来。

node1

查看rdb文件位置：

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
# redis-cli -a 123456

127.0.0.1:6379> config get dir
1) "dir"
2) "/data/redis"

127.0.0.1:6379> config get dbfilename
1) "dbfilename"
2) "dump.rdb"

127.0.0.1:6379> info keyspace
# Keyspace
db0:keys=4667,expires=4663,avg_ttl=3024079

127.0.0.1:6379> quit

使用dump.rdb存储数据，redis重启数据不会消失：

1
2
3
4
# systemctl restart redis
# redis-cli -a 123456 info keyspace 2> /dev/null
# Keyspace
db0:keys=4666,expires=4662,avg_ttl=3437344

模拟删除数据：

1
2
3
4
5
# systemctl stop redis
# mv /data/redis/dump.rdb /data/redis/dump.rdb.bak
# systemctl restart redis
# redis-cli -a 123456 info keyspace 2> /dev/null
# Keyspace

查看rdb保存时间：

1
2
3
4
5
# redis-cli -a 123456

127.0.0.1:6379> config get save
1) "save"
2) "3600 1 300 100 60 10000"

  • 更新得越频繁，保存得越频繁。

  • 关闭redis服务器会立马触发rdb存储。

设置关闭rdb：

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
127.0.0.1:6379> config set save ""
OK

127.0.0.1:6379> config get save
1) "save"
2) ""

127.0.0.1:6379> config rewrite
OK

127.0.0.1:6379> config get rdb*
 1) "rdbchecksum"
 2) "yes"
 3) "rdbcompression"
 4) "yes"
 5) "rdb-del-sync-files"
 6) "no"
 7) "rdb-save-incremental-fsync"
 8) "yes"
 9) "rdb-key-save-delay"
10) "0"
  • config set rdbcompression no：压缩，暂时不关。

设置rdb存储：

1
2
3
4
5
127.0.0.1:6379> config set save "3600 1 300 100 60 1000"
OK

127.0.0.1:6379> config rewrite
OK

新建终端，写入数据，观察rdb存储：

1
2
3
# for line in $(seq -w 1000); do 
    redis-cli -a 123456 set name_${line} rdb_redis_${line}
  done 2> /dev/null
1
2
# ll -h /data/redis/dump.rdb
-rw-r--r-- 1 root root 24K Dec 16 23:09 /data/redis/dump.rdb
1
2
3
4
5
6
7
8
9
# stat /data/redis/dump.rdb
  File: ‘/data/redis/dump.rdb’
  Size: 24100           Blocks: 48         IO Block: 4096   regular file
Device: fd00h/64768d    Inode: 1175331     Links: 1
Access: (0644/-rw-r--r--)  Uid: (    0/    root)   Gid: (    0/    root)
Access: 2023-12-16 23:08:30.056063123 +0800
Modify: 2023-12-16 23:08:30.056063123 +0800
Change: 2023-12-16 23:08:30.057063128 +0800
 Birth: -

再次添加数据：

1
2
3
# for line in $(seq -w 999); do 
    redis-cli -a 123456 set name_${line} rdb_redis_${line}
  done 2> /dev/null
1
2
# ll -h /data/redis/dump.rdb
-rw-r--r-- 1 root root 24K Dec 16 23:09 /data/redis/dump.rdb

返回redis，手动触发rdb存储：

1
2
3
4
127.0.0.1:6379> bgsave
Background saving started

127.0.0.1:6379> exit
1
2
# ll -h /data/redis/dump.rdb
-rw-r--r-- 1 root root 24K Dec 16 23:10 /data/redis/dump.rdb

可以重启，但不建议使用kill -9。

redis的aof存储实战

redis的aof存储方式：

  • aof存储默认是关闭的。

  • aof存储是把命令直接写入到文件中，文件会不断扩大。

node1

开启aof前先关闭rdb：

1
2
3
4
5
6
7
# redis-cli -a 123456

127.0.0.1:6379> config set save ""
OK

127.0.0.1:6379> config rewrite
OK

开启aof存储方式，默认是关闭的：

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
127.0.0.1:6379> config get append*
1) "appendonly"
2) "no"
3) "appendfilename"
4) "appendonly.aof"
5) "appendfsync"
6) "everysec"

127.0.0.1:6379> config set appendonly yes
OK

127.0.0.1:6379> config rewrite
OK

127.0.0.1:6379> quit
1
2
# ll -h /data/redis/appendonly.aof 
-rw-r--r-- 1 root root 24K Dec 16 23:17 /data/redis/appendonly.aof

新建终端，多次写入数据aof会不断扩大：

1
2
3
4
5
# for line in $(seq -w 1000); do
    redis-cli -a 123456 set name_${line} aof_redis_${line}
  done 2> /dev/null
# ll -h /data/redis/appendonly.aof
-rw-r--r-- 1 root root 72K Dec 16 23:17 /data/redis/appendonly.aof

手动重写aof减少aof文件大小：

1
2
3
4
5
6
# redis-cli -a 123456

127.0.0.1:6379> bgrewriteaof
Background append only file rewriting started

127.0.0.1:6379> quit
1
2
# ll -h /data/redis/appendonly.aof 
-rw-r--r-- 1 root root 49K Dec 16 23:17 /data/redis/appendonly.aof

自动重写aof：

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
# redis-cli -a 123456

127.0.0.1:6379> config get *aof*
 1) "aof-rewrite-incremental-fsync"
 2) "yes"
 3) "aof-load-truncated"
 4) "yes"
 5) "aof-use-rdb-preamble"
 6) "yes"
 7) "aof_rewrite_cpulist"
 8) ""
 9) "auto-aof-rewrite-percentage"
10) "100"
11) "auto-aof-rewrite-min-size"
12) "67108864"
13) "replicaof"
14) ""

127.0.0.1:6379> quit

重写条件是aof文件大小大于67M（aof文件增大了一倍）。

自动触发重写在业务高峰压力大，可选择在压力小的时候脚本运行bgrewriteaof。

rdb工具分析redis各key的大小

node1

查看大key：

1
# redis-cli -a 123456 --bigkeys

模拟大key：

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
# redis-cli -a 123456

127.0.0.1:6379> set name xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx
OK

127.0.0.1:6379> lpush names keynum
(integer) 1

127.0.0.1:6379> lpush names keynum
(integer) 2

127.0.0.1:6379> lpush names keynum
(integer) 3

127.0.0.1:6379> lpush names keynum
(integer) 4

127.0.0.1:6379> lpush names keynum
(integer) 5

127.0.0.1:6379> lpush names keynum
(integer) 6

127.0.0.1:6379> lpush names keynum
(integer) 7

127.0.0.1:6379> lpush names keynum
(integer) 8

127.0.0.1:6379> lpush names keynum
(integer) 9

127.0.0.1:6379> lpush names keynum
(integer) 10

127.0.0.1:6379> lpush names keynum
(integer) 11

127.0.0.1:6379> lpush names keynum
(integer) 12

127.0.0.1:6379> lpush names keynum
(integer) 13

127.0.0.1:6379> lpush names keynum
(integer) 14

127.0.0.1:6379> lpush names keynum
(integer) 15

127.0.0.1:6379> lpush names keynum
(integer) 16

127.0.0.1:6379> llen names
(integer) 16

新建终端，登录monitor：

1
# redis-cli -a 123456 monitor

返回旧终端，查看大key：

1
2
3
127.0.0.1:6379> quit

# redis-cli -a 123456 --bigkeys

查看数据大小，删除mylist，减少redis内存使用：

1
2
3
# redis-cli -a 123456 type mylist
# redis-cli -a 123456 llen mylist
# redis-cli -a 123456 del mylist

rdb分析所有key大小，建议拷贝到闲置的服务器进行分析：

1
2
3
4
5
# yum install -y python36
# pip3 install rdbtools==0.1.15 -i https://mirrors.aliyun.com/pypi/simple/
# redis-cli -a 123456 bgsave
# cd /data/redis/
# rdb -c memory dump.rdb > /tmp/student.csv

大小排序：

1
# cat /tmp/student.csv | sort -nrk 4 -t ','

redis主从复制实战

软件版本
redis6.2.1
节点IP系统功能CPU内存硬盘
node110.80.10.1centos7.9redis4核心8GB20GB
node210.80.10.2centos7.9redis4核心8GB20GB

node2

安装同步命令：

1
# yum install -y rsync

node1

还原配置：

1
2
# \cp /etc/redis.conf.bak2 /etc/redis.conf
# systemctl restart redis

同步redis命令：

1
2
3
# yum install -y rsync
# rsync -av /usr/local/bin/redis-* 10.80.10.2:/usr/local/bin/
# scp /etc/redis.conf 10.80.10.2:/etc/

清空数据库：

1
# redis-cli -a 123456 flushall

node2

创建数据目录：

1
# mkdir -p /data/redis

systemctl管理redis：

1
2
3
4
5
6
7
8
9
10
11
# vim /usr/lib/systemd/system/redis.service
[Unit]
Description=redis
After=network.target

[Service]
Type=forking
ExecStart=/usr/local/bin/redis-server /etc/redis.conf 

[Install]
WantedBy=multi-user.target

启动redis：

1
2
3
# systemctl start redis
# systemctl enable redis
# systemctl status redis

查看端口和进程：

1
2
3
4
5
# ps aux | grep redis
root     112020  0.0  0.1 162492  9820 ?        Ssl  23:25   0:00 /usr/local/bin/redis-server 0.0.0.0:6379
root     112194  0.0  0.0 112824   980 pts/0    S+   23:25   0:00 grep --color=auto redis
# netstat -tlunp | grep redis
tcp        0      0 0.0.0.0:6379            0.0.0.0:*               LISTEN      112020/redis-server

配置从库，需要配置认证：

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
# redis-cli -a 123456

127.0.0.1:6379> config set masterauth 123456
OK

127.0.0.1:6379> slaveof 10.80.10.1 6379
OK

127.0.0.1:6379> config rewrite
OK

127.0.0.1:6379> info replication
# Replication
role:slave
master_host:10.80.10.1
master_port:6379
master_link_status:up
master_last_io_seconds_ago:5
master_sync_in_progress:0
slave_repl_offset:0
slave_priority:100
slave_read_only:1
connected_slaves:0
master_failover_state:no-failover
master_replid:a41e1695c95a3a30f582880a8a42540b73bd8977
master_replid2:0000000000000000000000000000000000000000
master_repl_offset:0
second_repl_offset:-1
repl_backlog_active:1
repl_backlog_size:1048576
repl_backlog_first_byte_offset:1
repl_backlog_histlen:0

node1

查看主库状况：

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
# redis-cli -a 123456

127.0.0.1:6379> info replication
# Replication
role:master
connected_slaves:1
slave0:ip=10.80.10.2,port=6379,state=online,offset=28,lag=1
master_failover_state:no-failover
master_replid:a41e1695c95a3a30f582880a8a42540b73bd8977
master_replid2:0000000000000000000000000000000000000000
master_repl_offset:28
second_repl_offset:-1
repl_backlog_active:1
repl_backlog_size:1048576
repl_backlog_first_byte_offset:1
repl_backlog_histlen:28

测试主从：

1
2
3
4
5
127.0.0.1:6379> set name newstudent ex 3600
OK

127.0.0.1:6379> get name
"newstudent"

node2

测试主从：

1
2
3
4
5
127.0.0.1:6379> get name
"newstudent"

127.0.0.1:6379> ttl name
(integer) 3583

从库无写权限，slave-read-only：

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
127.0.0.1:6379> config get slave*
 1) "slave-lazy-flush"
 2) "no"
 3) "slave-serve-stale-data"
 4) "yes"
 5) "slave-read-only"
 6) "yes"
 7) "slave-ignore-maxmemory"
 8) "yes"
 9) "slave-announce-ip"
10) ""
11) "slave-priority"
12) "100"
13) "slave-announce-port"
14) "0"
15) "slaveof"
16) "192.168.80.71 6379"

从库取消主从同步：

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
127.0.0.1:6379> slaveof no one
OK

127.0.0.1:6379> config rewrite
OK

127.0.0.1:6379> info replication
# Replication
role:master
connected_slaves:0
master_failover_state:no-failover
master_replid:c77fd07798927beb8e8c2e0dfeb704828f9fac2d
master_replid2:a41e1695c95a3a30f582880a8a42540b73bd8977
master_repl_offset:140
second_repl_offset:141
repl_backlog_active:1
repl_backlog_size:1048576
repl_backlog_first_byte_offset:1
repl_backlog_histlen:140

开启从库写权限，容易数据不同步，暂不开启：

1
127.0.0.1:6379> config set slave-read-only no

redis主从复制之独立用户实战

node2

配置从库用户同步认证为空

1
2
127.0.0.1:6379> config set masterauth ""
OK

node1

默认是使用default用户同步：

1
2
3
4
5
6
7
8
9
10
11
12
13
14
127.0.0.1:6379> acl getuser default
 1) "flags"
 2) 1) "on"
    2) "allkeys"
    3) "allchannels"
    4) "allcommands"
 3) "passwords"
 4) 1) "8d969eef6ecad3c29a3a629280e686cf0c3f5d5a86aff3ca12020c923adc6c92"
 5) "commands"
 6) "+@all"
 7) "keys"
 8) 1) "*"
 9) "channels"
10) 1) "*"

增加同步用户：

1
2
3
4
5
6
127.0.0.1:6379> acl setuser slave on >slavepassword ~* +@all
OK

127.0.0.1:6379> acl list
1) "user default on #8d969eef6ecad3c29a3a629280e686cf0c3f5d5a86aff3ca12020c923adc6c92 ~* &* +@all"
2) "user slave on #f393e3ec722f8f1cc694326cfb371050512a5c59f8f4e6daf2b5123a53952a42 ~* &* +@all"

node2

同步命令：

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
127.0.0.1:6379> config set masteruser slave
OK

127.0.0.1:6379> config set masterauth slavepassword
OK

127.0.0.1:6379> slaveof 10.80.10.1 6379
OK

127.0.0.1:6379> info replication
# Replication
role:slave
master_host:10.80.10.1
master_port:6379
master_link_status:up
master_last_io_seconds_ago:4
master_sync_in_progress:0
slave_repl_offset:154
slave_priority:100
slave_read_only:1
connected_slaves:0
master_failover_state:no-failover
master_replid:a41e1695c95a3a30f582880a8a42540b73bd8977
master_replid2:0000000000000000000000000000000000000000
master_repl_offset:154
second_repl_offset:-1
repl_backlog_active:1
repl_backlog_size:1048576
repl_backlog_first_byte_offset:141
repl_backlog_histlen:14

127.0.0.1:6379> quit

node1

查看状况：

1
2
3
4
5
6
7
8
9
10
11
12
13
14
127.0.0.1:6379> info replication
# Replication
role:master
connected_slaves:1
slave0:ip=10.80.10.2,port=6379,state=online,offset=168,lag=0
master_failover_state:no-failover
master_replid:a41e1695c95a3a30f582880a8a42540b73bd8977
master_replid2:0000000000000000000000000000000000000000
master_repl_offset:168
second_repl_offset:-1
repl_backlog_active:1
repl_backlog_size:1048576
repl_backlog_first_byte_offset:1
repl_backlog_histlen:168

redis多主从复制架构

多从复制架构：

  • 主–>多从。

  • 主—>从（主）—>从。

node2

重启redis：

1
# systemctl restart redis

搭建node1的6379和node2的6379主从，使用default用户：

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
# redis-cli -a 123456

127.0.0.1:6379> config set masterauth 123456
OK

127.0.0.1:6379> slaveof 10.80.10.1 6379
OK

127.0.0.1:6379> config rewrite
OK

127.0.0.1:6379> info replication
# Replication
role:slave
master_host:10.80.10.1
master_port:6379
master_link_status:up
master_last_io_seconds_ago:4
master_sync_in_progress:0
slave_repl_offset:560
slave_priority:100
slave_read_only:1
connected_slaves:0
master_failover_state:no-failover
master_replid:a41e1695c95a3a30f582880a8a42540b73bd8977
master_replid2:0000000000000000000000000000000000000000
master_repl_offset:560
second_repl_offset:-1
repl_backlog_active:1
repl_backlog_size:1048576
repl_backlog_first_byte_offset:547
repl_backlog_histlen:14

127.0.0.1:6379> quit

配置node1的6379和node2的6380主从：

1
2
# mkdir -p /data/redis6380
# cp /etc/redis.conf /data/redis6380/

修改redis配置：

1
2
3
4
5
6
7
8
9
# vim /data/redis6380/redis.conf
# 3行，修改配置
port 6380
# 20行，修改配置
dir "/data/redis6380"
# 72行，删除配置
masterauth "123456"
# 倒数3行，删除配置
user default on sanitize-payload #8d969eef6ecad3c29a3a629280e686cf0c3f5d5a86aff3ca12020c923adc6c92 ~* &* +@all

启动redis：

1
2
3
4
5
6
7
8
# redis-server /data/redis6380/redis.conf
# ps aux | grep redis
root     121034  0.1  0.1 240832 10116 ?        Ssl  23:33   0:00 /usr/local/bin/redis-server 0.0.0.0:6379
root     122832 11.6  0.1 238272  9956 ?        Ssl  23:34   0:00 redis-server 0.0.0.0:6380
root     122898  0.0  0.0 112824   980 pts/0    S+   23:34   0:00 grep --color=auto redis
# netstat -tlunp | grep redis
tcp        0      0 0.0.0.0:6379            0.0.0.0:*               LISTEN      121034/redis-server 
tcp        0      0 0.0.0.0:6380            0.0.0.0:*               LISTEN      122832/redis-server

多从复制架构：

node1（主）—>node2：6379（从/主）—>node2：6380（从）

搭建6380主从：

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
# redis-cli -a 123456 -p 6380

127.0.0.1:6380> config set masterauth 123456
OK

127.0.0.1:6380> slaveof 10.80.10.1 6379
OK

127.0.0.1:6380> config rewrite
OK

127.0.0.1:6380> info replication
# Replication
role:slave
master_host:10.80.10.1
master_port:6379
master_link_status:up
master_last_io_seconds_ago:1
master_sync_in_progress:0
slave_repl_offset:882
slave_priority:100
slave_read_only:1
connected_slaves:0
master_failover_state:no-failover
master_replid:a41e1695c95a3a30f582880a8a42540b73bd8977
master_replid2:0000000000000000000000000000000000000000
master_repl_offset:882
second_repl_offset:-1
repl_backlog_active:1
repl_backlog_size:1048576
repl_backlog_first_byte_offset:729
repl_backlog_histlen:154

127.0.0.1:6380> quit

node1

验证主从：

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
127.0.0.1:6379> info replication
# Replication
role:master
connected_slaves:2
slave0:ip=10.80.10.2,port=6379,state=online,offset=896,lag=0
slave1:ip=10.80.10.2,port=6380,state=online,offset=896,lag=0
master_failover_state:no-failover
master_replid:a41e1695c95a3a30f582880a8a42540b73bd8977
master_replid2:0000000000000000000000000000000000000000
master_repl_offset:896
second_repl_offset:-1
repl_backlog_active:1
repl_backlog_size:1048576
repl_backlog_first_byte_offset:1
repl_backlog_histlen:896

127.0.0.1:6379> lpush test abc1
(integer) 1

127.0.0.1:6379> lpush test abc2
(integer) 2

127.0.0.1:6379> lpush test abc3
(integer) 3

127.0.0.1:6379> lrange test 0 -1
1) "abc3"
2) "abc2"
3) "abc1"

127.0.0.1:6379> quit

node2

验证主从：

1
2
3
4
5
6
7
8
# redis-cli -a 123456 -p 6379 lrange test 0 -1 2> /dev/null
1) "abc3"
2) "abc2"
3) "abc1"
# redis-cli -a 123456 -p 6380 lrange test 0 -1 2> /dev/null
1) "abc3"
2) "abc2"
3) "abc1"

也可以配置node2的6379和node2的6380主从：

node1（主）—>node2：6379（从）—>node2：6380（从）

哨兵sentinel实现redis主从自动切换

主动自动切换：

  • redis主从配置后，当主挂掉后，业务会有异常。

  • redis提供sentinel工具实现主从自动切换。

node2

关闭6380的redis：

1
# redis-cli -a 123456 -p 6380 shutdown

node1

配置sentinel：

1
2
3
4
5
6
7
8
9
10
# vim /etc/sentinel.conf
bind 0.0.0.0
daemonize yes
port 26379
dir "/tmp"
logfile "sentinel.log"
sentinel monitor studentmaster 10.80.10.1 6379 2
sentinel auth-pass studentmaster 123456
sentinel down-after-milliseconds studentmaster 5000
sentinel failover-timeout studentmaster 18000

配置说明：

  • 2：代表有2个sentinel认为master有问题，才会切换。

  • 5000：5秒没响应认为主挂了。

  • 18000：从提升为主的超时时间18s（可适当调大）。

启动sentinel：

1
2
# redis-sentinel /etc/sentinel.conf
# tail -f /tmp/sentinel.log

node2

配置sentinel：

1
2
3
4
5
6
7
8
9
10
# vim /etc/sentinel.conf
bind 0.0.0.0
daemonize yes
port 26379
dir "/tmp"
logfile "sentinel.log"
sentinel monitor studentmaster 10.80.10.1 6379 2
sentinel auth-pass studentmaster 123456
sentinel down-after-milliseconds studentmaster 5000
sentinel failover-timeout studentmaster 18000

启动观察sentinel：

1
2
# redis-sentinel /etc/sentinel.conf
# tail -f /tmp/sentinel.log

node1

配置密码，否则重启后无法连接到主库：

1
2
3
4
5
6
7
# redis-cli -a 123456

127.0.0.1:6379> config set masterauth 123456
OK

127.0.0.1:6379> config rewrite
OK

主库挂掉后，从库自动提升为主库：

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
127.0.0.1:6379> info replication
# Replication
role:master
connected_slaves:1
slave0:ip=10.80.10.2,port=6379,state=online,offset=5273,lag=1
master_failover_state:no-failover
master_replid:a41e1695c95a3a30f582880a8a42540b73bd8977
master_replid2:0000000000000000000000000000000000000000
master_repl_offset:5287
second_repl_offset:-1
repl_backlog_active:1
repl_backlog_size:1048576
repl_backlog_first_byte_offset:1
repl_backlog_histlen:5287

127.0.0.1:6379> shutdown

not connected> quit

查看sentinel日志：

1
2
3
4
5
6
7
8
9
10
11
# tail /tmp/sentinel.log
122353:X 16 Dec 2023 23:43:05.886 # +elected-leader master studentmaster 10.80.10.1 6379
122353:X 16 Dec 2023 23:43:05.887 # +failover-state-select-slave master studentmaster 10.80.10.1 6379
122353:X 16 Dec 2023 23:43:05.958 # +selected-slave slave 10.80.10.2:6379 10.80.10.2 6379 @ studentmaster 10.80.10.1 6379
122353:X 16 Dec 2023 23:43:05.958 * +failover-state-send-slaveof-noone slave 10.80.10.2:6379 10.80.10.2 6379 @ studentmaster 10.80.10.1 6379
122353:X 16 Dec 2023 23:43:06.025 * +failover-state-wait-promotion slave 10.80.10.2:6379 10.80.10.2 6379 @ studentmaster 10.80.10.1 6379
122353:X 16 Dec 2023 23:43:06.877 # +promoted-slave slave 10.80.10.2:6379 10.80.10.2 6379 @ studentmaster 10.80.10.1 6379
122353:X 16 Dec 2023 23:43:06.877 # +failover-state-reconf-slaves master studentmaster 10.80.10.1 6379
122353:X 16 Dec 2023 23:43:06.937 # +failover-end master studentmaster 10.80.10.1 6379
122353:X 16 Dec 2023 23:43:06.938 # +switch-master studentmaster 10.80.10.1 6379 10.80.10.2 6379
122353:X 16 Dec 2023 23:43:06.938 * +slave slave 10.80.10.1:6379 10.80.10.1 6379 @ studentmaster 10.80.10.2 6379

node2

从库变为主库：

1
2
3
4
5
6
7
8
9
10
11
12
13
14
# redis-cli -a 123456 info replication
Warning: Using a password with '-a' or '-u' option on the command line interface may not be safe.
# Replication
role:master
connected_slaves:0
master_failover_state:no-failover
master_replid:97553861535158f556163c7c618c2556361d6e36
master_replid2:a41e1695c95a3a30f582880a8a42540b73bd8977
master_repl_offset:8964
second_repl_offset:6702
repl_backlog_active:1
repl_backlog_size:1048576
repl_backlog_first_byte_offset:547
repl_backlog_histlen:8418

主恢复后，如果自动转为从库。要求主从都需要设置masterauth

实战环境sentinel最好跟redis-server在不同服务器上。

node1

再次启动redis：

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
# systemctl start redis
# redis-cli -a 123456 info replication
Warning: Using a password with '-a' or '-u' option on the command line interface may not be safe.
# Replication
role:slave
master_host:10.80.10.2
master_port:6379
master_link_status:up
master_last_io_seconds_ago:1
master_sync_in_progress:0
slave_repl_offset:12627
slave_priority:100
slave_read_only:1
connected_slaves:0
master_failover_state:no-failover
master_replid:97553861535158f556163c7c618c2556361d6e36
master_replid2:0000000000000000000000000000000000000000
master_repl_offset:12627
second_repl_offset:-1
repl_backlog_active:1
repl_backlog_size:1048576
repl_backlog_first_byte_offset:12325
repl_backlog_histlen:303

python操作redis哨兵集群

普通连接Redis和哨兵连接说明：

  • 普通连接Redis指定地址，切换后程序会有异常。

  • 程序如果连接哨兵的话，会自动切换程序正常运行。

node1

安装扩展：

1
2
# yum install python36
# pip3 install redis==3.5.3 -i https://mirrors.aliyun.com/pypi/simple/

程序查看主从：

1
2
3
4
5
6
7
8
9
10
# cd
# vim test.py
import redis

from redis.sentinel import Sentinel
sentinel = Sentinel([('10.80.10.1',26379),('10.80.10.2',26379)],socket_timeout=3)
master = sentinel.discover_master('studentmaster')
print(master)
slave = sentinel.discover_slaves('studentmaster')
print(slave)
1
2
3
# python3 test.py
('10.80.10.2', 6379)
[('10.80.10.1', 6379)]

关闭从库进行测试：

1
2
3
4
5
# systemctl stop redis
# python3 test.py
('10.80.10.2', 6379)
[]
# systemctl restart redis

程序操作redis，能够跟随redis哨兵自动切换 ：

1
2
3
4
5
6
7
8
9
10
11
# vim test.py
import redis

from redis.sentinel import Sentinel
sentinel = Sentinel([('10.80.10.1',26379),('10.80.10.2',26379)],socket_timeout=3)
master = sentinel.master_for('studentmaster',socket_timeout=3,password='123456',db=0) 
result = master.set('name','studentsentinel') 
print(result) 
slave = sentinel.slave_for('studentmaster',socket_timeout=3,password='123456',db=0) 
result = slave.get('name') 
print(result) 
1
2
3
# python3 test.py
True
b'studentsentinel'

node2

关闭主库，redis主从切换操作无影响：

1
# systemctl stop redis

node1

可能需要等待几秒：

1
2
3
# python3 test.py
True
b'studentsentinel'

redis cluster分片集群实战

节点IP系统端口功能端口功能CPU内存硬盘
node110.80.10.1centos7.97000redis主8000redis从（7002从）4核心8GB20GB
node210.80.10.2centos7.97001redis主8001redis从（7000从）4核心8GB20GB
node310.80.10.3centos7.97002redis主8002redis从（7001从）4核心8GB20GB
node310.80.10.3centos7.97003redis主8003redis从（7001从）4核心8GB20GB

主从和分片集群区别：

  • 主从所有Redis数据一致，key过多会影响性能。

  • 分片集群数据分散到多个redis，数据分片存储。

redis cluster：

  • 数据分片。

  • 多个入口。

  • 故障自动切换。

redis自带集群的搭建：

  • redis集群至少需要三主三从。

  • 三从保证主有问题能够切换。

redis自带集群搭建：

  • 目录：/data/cluster

  • 三主：7000、7001、7002

  • 三从：8000、8001、8002

node3

安装同步命令：

1
# yum install -y rsync

node1

同步redis命令：

1
2
# rsync -av /usr/local/bin/redis-* 10.80.10.3:/usr/local/bin/
# scp /etc/redis.conf 10.80.10.2:/etc/

node1、node2

关闭redis：

1
# pkill redis

node1、node2、node3

创建数据目录：

1
2
3
4
5
6
# node1
# mkdir -p /data/cluster/{7000,8000}
# node2
# mkdir -p /data/cluster/{7001,8001}
# node3
# mkdir -p /data/cluster/{7002,8002}

node1

配置7000：

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
# vim /data/cluster/7000/redis.conf
cluster-enabled yes
port 7000
dir "/data/cluster/7000"
logfile "redis.log"
pidfile "redis.pid"
daemonize yes
bind 0.0.0.0
requirepass studentpassword
masterauth studentpassword
tcp-backlog 1024
tcp-keepalive 0
loglevel notice
stop-writes-on-bgsave-error yes
rdbcompression yes
rdbchecksum yes
save 900 1
save 300 10
save 60 10000
dbfilename "dump.rdb"
slave-serve-stale-data yes
slave-read-only yes
repl-diskless-sync no
repl-diskless-sync-delay 5
repl-disable-tcp-nodelay no
slave-priority 100
appendonly no
appendfilename "appendonly.aof"
appendfsync everysec
no-appendfsync-on-rewrite yes
auto-aof-rewrite-percentage 100
auto-aof-rewrite-min-size 64mb
aof-load-truncated yes
lua-time-limit 5000
slowlog-log-slower-than 10000
slowlog-max-len 128
latency-monitor-threshold 0
notify-keyspace-events ""
hash-max-ziplist-entries 512
hash-max-ziplist-value 64
list-max-ziplist-entries 512
list-max-ziplist-value 64
set-max-intset-entries 512
zset-max-ziplist-entries 128
zset-max-ziplist-value 64
hll-sparse-max-bytes 3000
activerehashing yes
client-output-buffer-limit normal 0 0 0
client-output-buffer-limit slave 256mb 64mb 60
client-output-buffer-limit pubsub 32mb 8mb 60
hz 10
aof-rewrite-incremental-fsync yes

拷贝到其他节点：

1
2
# scp /data/cluster/7000/redis.conf 10.80.10.2:/data/cluster/7001/redis.conf
# scp /data/cluster/7000/redis.conf 10.80.10.3:/data/cluster/7002/redis.conf

node2

修改端口：

1
# sed -i 's/7000/7001/g' /data/cluster/7001/redis.conf

node3

修改端口：

1
# sed -i 's/7000/7002/g' /data/cluster/7002/redis.conf

node1、node2、node3

启动redis：

1
2
3
4
5
6
7
8
9
# node1
# redis-server /data/cluster/7000/redis.conf
# redis-cli -a studentpassword -p 7000 info
# node2
# redis-server /data/cluster/7001/redis.conf
# redis-cli -a studentpassword -p 7001 info
# node3
# redis-server /data/cluster/7002/redis.conf
# redis-cli -a studentpassword -p 7002 info

node1

启动主集群：

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
# redis-cli -a studentpassword --cluster create 10.80.10.1:7000 10.80.10.2:7001 10.80.10.3:7002
yes
# redis-cli -a studentpassword -p 7000 cluster info
Warning: Using a password with '-a' or '-u' option on the command line interface may not be safe.
cluster_state:fail
cluster_slots_assigned:16384
cluster_slots_ok:16384
cluster_slots_pfail:0
cluster_slots_fail:0
cluster_known_nodes:3
cluster_size:3
cluster_current_epoch:3
cluster_my_epoch:1
cluster_stats_messages_ping_sent:6
cluster_stats_messages_pong_sent:9
cluster_stats_messages_sent:15
cluster_stats_messages_ping_received:7
cluster_stats_messages_pong_received:6
cluster_stats_messages_meet_received:2
cluster_stats_messages_received:15

查看节点状态：

1
2
3
4
5
# redis-cli -a studentpassword -p 7000 cluster nodes
Warning: Using a password with '-a' or '-u' option on the command line interface may not be safe.
c35f61612453dd267f7d7519e3badcac70e87b8b 10.80.10.3:7002@17002 master - 0 1702742249749 3 connected 10923-16383
13bc03bced79aebb410df9aa291402d60b0f8811 10.80.10.2:7001@17001 master - 0 1702742250752 2 connected 5461-10922
c85afaff1dfbca5287a25eec868c58042826c2a8 10.80.10.1:7000@17000 myself,master - 0 1702742250000 1 connected 0-5460
  • 一共16384个插槽。

登录继续需要添加-c参数：

1
2
3
4
5
6
7
# redis-cli -p 7000 -a studentpassword -c

127.0.0.1:7000> set name teacher
-> Redirected to slot [5798] located at 10.80.10.2:7001
OK

10.80.10.2:7001> exit

node2

10.80.10.2:7001有数据产生：

1
2
3
4
# redis-cli -a studentpassword -p 7001 info keyspace
Warning: Using a password with '-a' or '-u' option on the command line interface may not be safe.
# Keyspace
db0:keys=1,expires=0,avg_ttl=0

node1

批量写入数据：

1
2
3
# for line in $(seq -w 1000); do
  redis-cli -a studentpassword -p 7000 -c set name_${line} teacher_${line}
done 2> /dev/null

node1、node2、node3

查看数据分布：

1
2
3
4
5
6
# node1
# redis-cli -a studentpassword -p 7000 info keyspace
# node2
# redis-cli -a studentpassword -p 7001 info keyspace
# node3
# redis-cli -a studentpassword -p 7002 info keyspace

node1

获取数据：

1
2
3
# for line in $(seq -w 1000); do
  redis-cli -a studentpassword -p 7000 -c get name_${line}
done 2> /dev/null

redis分片集群高可用性

node1

配置8000：

1
2
# cp /data/cluster/7000/redis.conf /data/cluster/8000/redis.conf
# sed -i 's/7000/8000/g' /data/cluster/8000/redis.conf

node2

配置8001：

1
2
# cp /data/cluster/7001/redis.conf /data/cluster/8001/redis.conf
# sed -i 's/7001/8001/g' /data/cluster/8001/redis.conf

node3

配置8002：

1
2
# cp /data/cluster/7002/redis.conf /data/cluster/8002/redis.conf
# sed -i 's/7002/8002/g' /data/cluster/8002/redis.conf

node1、node2、node3

启动redis：

1
2
3
4
5
6
7
8
9
# node1
# redis-server /data/cluster/8000/redis.conf
# redis-cli -a studentpassword -p 8000 info
# node2
# redis-server /data/cluster/8001/redis.conf
# redis-cli -a studentpassword -p 8001 info
# node3
# redis-server /data/cluster/8002/redis.conf
# redis-cli -a studentpassword -p 8002 info

node1

获取集群状态：

1
2
3
4
5
# redis-cli -a studentpassword -p 7000 cluster nodes
Warning: Using a password with '-a' or '-u' option on the command line interface may not be safe.
c35f61612453dd267f7d7519e3badcac70e87b8b 10.80.10.3:7002@17002 master - 0 1702742472456 3 connected 10923-16383
13bc03bced79aebb410df9aa291402d60b0f8811 10.80.10.2:7001@17001 master - 0 1702742473460 2 connected 5461-10922
c85afaff1dfbca5287a25eec868c58042826c2a8 10.80.10.1:7000@17000 myself,master - 0 1702742473000 1 connected 0-5460

redis集群添加从库：

1
2
3
# redis-cli -a studentpassword --cluster add-node --cluster-slave --cluster-master-id c85afaff1dfbca5287a25eec868c58042826c2a8 10.80.10.2:8001 10.80.10.1:7000
# redis-cli -a studentpassword --cluster add-node --cluster-slave --cluster-master-id 13bc03bced79aebb410df9aa291402d60b0f8811 10.80.10.3:8002 10.80.10.2:7001
# redis-cli -a studentpassword --cluster add-node --cluster-slave --cluster-master-id c35f61612453dd267f7d7519e3badcac70e87b8b 10.80.10.1:8000 10.80.10.3:7002

查看节点状态：

1
2
3
4
5
6
7
8
# redis-cli -a studentpassword -p 7000 cluster nodes
Warning: Using a password with '-a' or '-u' option on the command line interface may not be safe.
a4341e2782b688779c8b0ac6ccb883e887b633e8 10.80.10.2:8001@18001 slave c85afaff1dfbca5287a25eec868c58042826c2a8 0 1702742958915 1 connected
88d8ad9d3e32d13d335e3eb68d5a4b6635476ef0 10.80.10.3:8002@18002 slave 13bc03bced79aebb410df9aa291402d60b0f8811 0 1702742957912 2 connected
13bc03bced79aebb410df9aa291402d60b0f8811 10.80.10.2:7001@17001 master - 0 1702742958000 2 connected 5461-10922
c35f61612453dd267f7d7519e3badcac70e87b8b 10.80.10.3:7002@17002 master - 0 1702742956910 3 connected 10923-16383
91c8c4aad68d0b70d8d17750dca755a7f29b8673 10.80.10.1:8000@18000 slave c35f61612453dd267f7d7519e3badcac70e87b8b 0 1702742956000 3 connected
c85afaff1dfbca5287a25eec868c58042826c2a8 10.80.10.1:7000@17000 myself,master - 0 1702742955000 1 connected 0-5460

node1、node2、node3

查看节点数据，已经同步过去了：

1
2
3
4
5
6
# node1
# redis-cli -a studentpassword -p 8000 info keyspace
# node2
# redis-cli -a studentpassword -p 8001 info keyspace
# node3
# redis-cli -a studentpassword -p 8002 info keyspace

node1

获取数据正常：

1
2
3
# for line in $(seq -w 1000); do
    redis-cli -a studentpassword -p 7000 -c get name_${line}
  done 2> /dev/null

node2

主库有问题时从库会自动切换为主库：

1
# redis-cli -a studentpassword -p 7001

node3

8002变为主库，需要等待一段时间：

1
2
3
4
5
6
7
8
9
10
11
12
13
14
# redis-cli -a studentpassword -p 8002 info replication
Warning: Using a password with '-a' or '-u' option on the command line interface may not be safe.
# Replication
role:master
connected_slaves:0
master_failover_state:no-failover
master_replid:7a63c0d592537fb316f20c9385a0b41c3f2bf299
master_replid2:6d6ab082dc462eafbfc087c71a41690badb66666
master_repl_offset:252
second_repl_offset:253
repl_backlog_active:1
repl_backlog_size:1048576
repl_backlog_first_byte_offset:1
repl_backlog_histlen:252

node1

获取数据正常：

1
2
3
# for line in $(seq -w 1000); do
    redis-cli -a studentpassword -p 7000 -c get name_${line}
  done 2> /dev/null

写入数据正常：

1
# for line in $(seq -w 1000); do redis-cli -a studentpassword -p 7000 -c set name_${line} teacher_${line}; done 2> /dev/null

node2

主库再启动，会自动变成从库：

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
# redis-server /data/cluster/7001/redis.conf
# redis-cli -a studentpassword -p 7001 info replication
Warning: Using a password with '-a' or '-u' option on the command line interface may not be safe.
# Replication
role:slave
master_host:10.80.10.3
master_port:8002
master_link_status:up
master_last_io_seconds_ago:2
master_sync_in_progress:0
slave_repl_offset:15799
slave_priority:100
slave_read_only:1
connected_slaves:0
master_failover_state:no-failover
master_replid:7a63c0d592537fb316f20c9385a0b41c3f2bf299
master_replid2:0000000000000000000000000000000000000000
master_repl_offset:15799
second_repl_offset:-1
repl_backlog_active:1
repl_backlog_size:1048576
repl_backlog_first_byte_offset:15786
repl_backlog_histlen:14

从库手动提升为主库的方法：

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
# redis-cli -a studentpassword -p 7001 cluster failover
Warning: Using a password with '-a' or '-u' option on the command line interface may not be safe.
OK
# redis-cli -a studentpassword -p 7001 info replication
Warning: Using a password with '-a' or '-u' option on the command line interface may not be safe.
# Replication
role:master
connected_slaves:1
slave0:ip=10.80.10.3,port=8002,state=online,offset=15813,lag=1
master_failover_state:no-failover
master_replid:dde4d3d7ffdde2ca1f6f6c5880a6f66fc9bf8c53
master_replid2:7a63c0d592537fb316f20c9385a0b41c3f2bf299
master_repl_offset:15813
second_repl_offset:15814
repl_backlog_active:1
repl_backlog_size:1048576
repl_backlog_first_byte_offset:15786
repl_backlog_histlen:28

redis集群移除从库，主库ip端口+从库id，暂不移除：

1
redis-cli -a sjgpwd --cluster del-node xxx:xxx xxxxxxxxxxxxxxxxxxxxxxxxxxxxxx

移除从库会关闭从库，清空数据。

redis集群添加节点和删除节点

redis集群：

  • 去中心化，一般程序写三个入口。

  • 集群扩容和缩减也比较方便，但数据量大影响比较大，上线前需要预估好量。

node3

创建新节点配置：

1
2
3
# mkdir -p /data/cluster/7003
# cp /data/cluster/7002/redis.conf /data/cluster/7003/redis.conf
# sed -i 's/7002/7003/g' /data/cluster/7003/redis.conf

启动redis：

1
2
# redis-server /data/cluster/7003/redis.conf
# redis-cli -a studentpassword -p 7003 info

node1

redis添加主节点：

1
# redis-cli -a studentpassword --cluster add-node 10.80.10.3:7003 10.80.10.1:7000

添加完节点默认没有分配slot：

1
2
3
4
5
6
7
8
9
# redis-cli -a studentpassword -p 7000 cluster nodes
Warning: Using a password with '-a' or '-u' option on the command line interface may not be safe.
a4341e2782b688779c8b0ac6ccb883e887b633e8 10.80.10.2:8001@18001 slave c85afaff1dfbca5287a25eec868c58042826c2a8 0 1702743267685 1 connected
88d8ad9d3e32d13d335e3eb68d5a4b6635476ef0 10.80.10.3:8002@18002 slave 13bc03bced79aebb410df9aa291402d60b0f8811 0 1702743268689 5 connected
13bc03bced79aebb410df9aa291402d60b0f8811 10.80.10.2:7001@17001 master - 0 1702743267000 5 connected 5461-10922
21bb510a27566ae6385d64bfa942208bb7e5c4e7 10.80.10.3:7003@17003 master - 0 1702743266682 0 connected
c35f61612453dd267f7d7519e3badcac70e87b8b 10.80.10.3:7002@17002 master - 0 1702743265679 3 connected 10923-16383
91c8c4aad68d0b70d8d17750dca755a7f29b8673 10.80.10.1:8000@18000 slave c35f61612453dd267f7d7519e3badcac70e87b8b 0 1702743263000 3 connected
c85afaff1dfbca5287a25eec868c58042826c2a8 10.80.10.1:7000@17000 myself,master - 0 1702743263000 1 connected 0-5460

从7000分配2000个slot给7003：

1
2
3
4
5
6
# redis-cli -a studentpassword --cluster reshard 10.80.10.1:7000
2000
7003的id（目的）
7000的id（源）
done
yes

node3

查看新节点数据：

1
# redis-cli -a studentpassword -p 7003 info

node1

查看节点：

1
2
3
4
5
6
7
8
9
# redis-cli -a studentpassword -p 7000 cluster nodes
Warning: Using a password with '-a' or '-u' option on the command line interface may not be safe.
a4341e2782b688779c8b0ac6ccb883e887b633e8 10.80.10.2:8001@18001 slave c85afaff1dfbca5287a25eec868c58042826c2a8 0 1702743453000 7 connected
88d8ad9d3e32d13d335e3eb68d5a4b6635476ef0 10.80.10.3:8002@18002 slave 13bc03bced79aebb410df9aa291402d60b0f8811 0 1702743455219 5 connected
13bc03bced79aebb410df9aa291402d60b0f8811 10.80.10.2:7001@17001 master - 0 1702743454216 5 connected 5461-10922
21bb510a27566ae6385d64bfa942208bb7e5c4e7 10.80.10.3:7003@17003 master - 0 1681486118675 6 connected 0-1999
c35f61612453dd267f7d7519e3badcac70e87b8b 10.80.10.3:7002@17002 master - 0 1702743452000 3 connected 10923-16383
91c8c4aad68d0b70d8d17750dca755a7f29b8673 10.80.10.1:8000@18000 slave c35f61612453dd267f7d7519e3badcac70e87b8b 0 1702743451000 3 connected
c85afaff1dfbca5287a25eec868c58042826c2a8 10.80.10.1:7000@17000 myself,master - 0 1702743450000 1 connected 2000-5460

删除节点，需要先移除数据才能删除节点：

1
2
3
4
5
6
# redis-cli -a studentpassword --cluster reshard 10.80.10.1:7000
2000
7000的id（目的）
7003的id（源）
done
yes

任意主节点ip端口+7003id

1
# redis-cli -a studentpassword --cluster del-node 10.80.10.1:7000 eeb09f8a0d7058dc8020b7e91246a567fbbf80fe

node3

查看新节点数据：

1
# redis-cli -a studentpassword -p 7003 info

node1

集群获取数据正常：

1
# for line in $(seq -w 2000); do redis-cli -a studentpassword -p 7000 -c get name_${line}; done 2> /dev/null

python操作redis分片集群

node1

安装扩展：

1
2
3
# yum install python36
# pip3 install redis==3.5.3 -i https://mirrors.aliyun.com/pypi/simple/
# pip3 install redis-py-cluster==2.1.0 -i https://mirrors.aliyun.com/pypi/simple/

操作代码：

1
2
3
4
5
6
7
8
9
10
11
12
# vim test.py
from rediscluster import RedisCluster

nodes=[{"host":"10.80.10.1","port":7000},
{"host":"10.80.10.2","port":7001},
{"host":"10.80.10.3","port":7002}
]
rc = RedisCluster(startup_nodes=nodes,password='studentpassword')
for i in range(15):
  result=rc.set('studentkey{0}'.format(i),'studentclustervalue{0}'.format(i))
  result=rc.get('studentkey{0}'.format(i))
  print(result)
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
# python3 test.py
b'studentclustervalue0'
b'studentclustervalue1'
b'studentclustervalue2'
b'studentclustervalue3'
b'studentclustervalue4'
b'studentclustervalue5'
b'studentclustervalue6'
b'studentclustervalue7'
b'studentclustervalue8'
b'studentclustervalue9'
b'studentclustervalue10'
b'studentclustervalue11'
b'studentclustervalue12'
b'studentclustervalue13'
b'studentclustervalue14'

挂掉其中一个节点，不影响程序操作：

1
# redis-cli -p 7000 -a studentpassword shutdown

集群切换需要等待一会：

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
# python3 test.py
b'studentclustervalue0'
b'studentclustervalue1'
b'studentclustervalue2'
b'studentclustervalue3'
b'studentclustervalue4'
b'studentclustervalue5'
b'studentclustervalue6'
b'studentclustervalue7'
b'studentclustervalue8'
b'studentclustervalue9'
b'studentclustervalue10'
b'studentclustervalue11'
b'studentclustervalue12'
b'studentclustervalue13'
b'studentclustervalue14'