Redis的持久化

Redis的持久化

Redis的持久化

一、RDB

1、RDB

是指在指定的时间间隔内,将内存中的数据集快照写入磁盘。也就是Snapshot快照,它恢复时是将快照文件直接读到内存中。

2、Redis

会单独创建(Fork)一个子进程来进行持久化,会先将数据写到一个临时文件中,待持久化的过程结束了,再用这个临时文件替换上次持久化好的文件。整个过程中,主进程不进行任何IO操作,这就确保了极高的性能。如果需要进行大规模数据的恢复,且对于数据的完整性不是十分的敏感,那RDB方式要比AOF方式更加的高效。RDB的缺点是最后一次持久化的数据可能丢失。

3、Fork

的作用复制一个与当前进程一样的进程。新进程的所有数据(变量、环境变量、程序计数器等)数值和原进程一致,但是是一个全新的进程,并作为原进程的子进程。

4、RDB保存的是dump.rdb文件。

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
################################ SNAPSHOTTING  ################################
#
# Save the DB on disk:
#
# save <seconds> <changes>
#
# Will save the DB if both the given number of seconds and the given
# number of write operations against the DB occurred.
#
# In the example below the behaviour will be to save:
# after 900 sec (15 min) if at least 1 key changed
# after 300 sec (5 min) if at least 10 keys changed
# after 60 sec if at least 10000 keys changed
#
# Note: you can disable saving completely by commenting out all "save" lines.
#
# It is also possible to remove all the previously configured save
# points by adding a save directive with a single empty string argument
# like in the following example:
#
# save ""
#表示900秒之内修改了一次就会记录一下快照
save 900 1
#300秒改动10次
save 300 10
6010000
save 60 10000

# By default Redis will stop accepting writes if RDB snapshots are enabled
# (at least one save point) and the latest background save failed.
# This will make the user aware (in a hard way) that data is not persisting
# on disk properly, otherwise chances are that no one will notice and some
# disaster will happen.
#
# If the background saving process will start working again Redis will
# automatically allow writes again.

如果发生宕机等行为,下次重启,redis会自动由dump.drb自动恢复

#表示900秒之内修改了一次就会记录一下快照

save 900 1

#300秒改动10次

save 300 10

60秒10000次

save 60 10000

5、save命令:即刻备份。

bgsave:reids会在后台异步进行快照操作,快照同时还可以响应客户端请求。可以通过lastsave命令获取最后一次成功执行快照的时间。

6、stop-writes-on-bgsave-error yes

1
2
3
4
5
6
7
8
9
# If the background saving process will start working again Redis will
# automatically allow writes again.
#
# However if you have setup your proper monitoring of the Redis server
# and persistence, you may want to disable this feature so that Redis will
# continue to work as usual even if there are problems with disk,
# permissions, and so forth.
stop-writes-on-bgsave-error yes
//后台保存出错前台停止写入

如果snapshot过程中出现错误,即数据持久化失败,是否终止所有的客户端write请求。这个选项很让人为难,”yes”表示终止,一旦snapshot故障,那么此server为只读服务;

如果为”no”,那么此次snapshot将失败,但下一次snapshot不会受到影响,不过如果出现故障,数据只能恢复到”最近一个成功点”。
如果配置成no表示不在乎数据的不一致性或者有其他的手段发现和控制

7、rdbcompression yes

、是否启用rdb文件压缩手段,默认为yes.压缩可能需要额外的cpu开支,不过这能够有效的减小rdb文件的大小,有利于存储/备份/传输/数据恢复。

1
2
3
4
5
# Compress string objects using LZF when dump .rdb databases?
# For default that's set to 'yes' as it's almost always a win.
# If you want to save some CPU in the saving child set it to 'no' but
# the dataset will likely be bigger if you have compressible values or keys.
rdbcompression yes

7、rdbchecksum yes

是否对rdb文件使用CRC64校验和,默认为”yes”,那么每个rdb文件内容的末尾都会追加CRC校验和。对于其他第三方校验工具,可以很方便的检测文件的完整性。

1
2
3
# RDB files created with checksum disabled have a checksum of zero that will
# tell the loading code to skip the check.
rdbchecksum yes

8、dbfilename dump.rdb

指定rdb文件的名称

1
2
# The filename where to dump the DB
dbfilename dump.rdb

9、dir ./

指定rdb/AOF文件的目录位置

1
2
3
4
5
6
7
8
9
# The working directory.
#
# The DB will be written inside this directory, with the filename specified
# above using the 'dbfilename' configuration directive.
#
# The Append Only File will also be created inside this directory.
#
# Note that you must specify a directory here, not a file name.
dir ./

优势:适合大规模数据的恢复,对数据的一致性要求不高。

劣势:在一定的间隔时间做一次备份,所以如果redis意外down掉,就会丢失最后一次快照的所有修改。
fork的时候,内存中的数据被克隆了一份,大致2
倍的膨胀性能需要考虑。

10、如何停止RDB:

动态停止RDB保存规则的方法:redis-cli config set save “”

二、AOF

1、介绍

aop是以日志中的形式来记录每个写操作,将redis执行过的所有写指令记录下来(读操作不记录),只许追加文件但不可以改写文件,redis启动之初会读取该文件重新构建数据,换言之,redis重启的话就会根据日志文件的内容将写指令从前到后执行一次以完成数据的恢复工作。

aof保存的是appendonly.aof文件

2、配置文件

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
############################## APPEND ONLY MODE ###############################

# By default Redis asynchronously dumps the dataset on disk. This mode is
# good enough in many applications, but an issue with the Redis process or
# a power outage may result into a few minutes of writes lost (depending on
# the configured save points).
#
# The Append Only File is an alternative persistence mode that provides
# much better durability. For instance using the default data fsync policy
# (see later in the config file) Redis can lose just one second of writes in a
# dramatic event like a server power outage, or a single write if something
# wrong with the Redis process itself happens, but the operating system is
# still running correctly.
#
# AOF and RDB persistence can be enabled at the same time without problems.
# If the AOF is enabled on startup Redis will load the AOF, that is the file
# with the better durability guarantees.
#
# Please check http://redis.io/topics/persistence for more information.

//默认关闭
appendonly no

# The name of the append only file (default: "appendonly.aof")

appendfilename "appendonly.aof"

# The fsync() call tells the Operating System to actually write data on disk
# instead of waiting for more data in the output buffer. Some OS will really flush
# data on disk, some other OS will just try to do it ASAP.
#
# Redis supports three different modes:
#
# no: don't fsync, just let the OS flush the data when it wants. Faster.
# always: fsync after every write to the append only log. Slow, Safest.
# everysec: fsync only one time every second. Compromise.
#
# The default is "everysec", as that's usually the right compromise between

使用aof需要设置:appendonly yes

3、appendfilename “appendonly.aof”

1
2
3
 The name of the append only file (default: "appendonly.aof")

appendfilename "appendonly.aof"

4、appendfsync everysec

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
# Redis supports three different modes:
#
# no: don't fsync, just let the OS flush the data when it wants. Faster.
# always: fsync after every write to the append only log. Slow, Safest.
# everysec: fsync only one time every second. Compromise.
#
# The default is "everysec", as that's usually the right compromise between
# speed and data safety. It's up to you to understand if you can relax this to
# "no" that will let the operating system flush the output buffer when
# it wants, for better performances (but if you can live with the idea of
# some data loss consider the default persistence mode that's snapshotting),
# or on the contrary, use "always" that's very slow but a bit safer than
# everysec.
#
# More details please check the following article:
# http://antirez.com/post/redis-persistence-demystified.html
#
# If unsure, use "everysec".

# appendfsync always
appendfsync everysec
# appendfsync no

说明:Always:同步持久化,每次发生数据变更会被立即记录到磁盘,性能较差但数据完整性较好。
Everysec:出厂默认推荐,异步操作,每秒记录,如果一秒内宕机,有数据丢失;
No:

5、启动:

设置:appendonly yes

6、恢复:

dump.rdb 和appendonly.aof两个文件可以共存,但是会默认有aof启动恢复。如果aof出错,此时不能启动redis,如:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
SELECT
$1
0
*3
$3
set
$2
k1
$2
v1
*3
$3
set
$2
k2
$2
v2
*3
$3
set
$2
k3
$2
v3
*2
$6
SELECT
$1
0
qwddcdsdffczcsdvxvvxzvdsczccxfdsxcxz,cx cxzcmz,xzcvsdfcc
dsDcsdczcz
~

此时可以通过redis-check-aof –fix appendonly.aof来修复文件。

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
SELECT
$1
0
*3
$3
set
$2
k1
$2
v1
*3
$3
set
$2
k2
$2
v2
*3
$3
set
$2
k3
$2
v3
*2
$6
SELECT
$1
0
qwddcdsdffczcsdvxvvxzvdsczccxfdsxcxz,cx cxzcmz,xzcvsdfcc
dsDcsdczcz
~

7、rewrite

AOF采用文件追加,文件会越来越大,为了避免出现此种情况,新增了重写机制。当AOF文件的大小超过了所设定的阈值时,Redis就会启动AOF文件的内容压缩,只保留可以恢复数据的最小指令集,可以使用命令:bgrewriteaof。

原理:AOF文件持续增大而过大时,会fork出一条新进程来将文件重写(也是先写临时文件最后再rename),遍历新进程的内存中的数据,每条记录有一条set语句。重写AOF文件的操作,并没有读取旧的aof文件,而是将整个内存中的数据库内容用命令的方式重写了一个新的aof文件,这点和快照有点类似。

触发机制:redis会记录上次重写时的AOF大小,默认配置是AOF文件大小是上次rewrite后大小的一倍且文件大于64M时出发。

//100即上次的一倍

auto-aof-rewrite-percentage 100

//阈值是64M,超过64M时启动

auto-aof-rewrite-min-size 64mb

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
# Automatic rewrite of the append only file.
# Redis is able to automatically rewrite the log file implicitly calling
# BGREWRITEAOF when the AOF log size grows by the specified percentage.
#
# This is how it works: Redis remembers the size of the AOF file after the
# latest rewrite (if no rewrite has happened since the restart, the size of
# the AOF at startup is used).
#
# This base size is compared to the current size. If the current size is
# bigger than the specified percentage, the rewrite is triggered. Also
# you need to specify a minimal size for the AOF file to be rewritten, this
# is useful to avoid rewriting the AOF file even if the percentage increase
# is reached but it is still pretty small.
#
# Specify a percentage of zero in order to disable the automatic AOF
# rewrite feature.

//100即上次的一倍
auto-aof-rewrite-percentage 100

//阈值是64M,超过64M时启动
auto-aof-rewrite-min-size 64mb

8、no-appendfsync-on-rewrite no

重写时是否可以运用(4)中Appendsync,默认no即可,保证数据的安全性。

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
# When the AOF fsync policy is set to always or everysec, and a background
# saving process (a background save or AOF log background rewriting) is
# performing a lot of I/O against the disk, in some Linux configurations
# Redis may block too long on the fsync() call. Note that there is no fix for
# this currently, as even performing fsync in a different thread will block
# our synchronous write(2) call.
#
# In order to mitigate this problem it's possible to use the following option
# that will prevent fsync() from being called in the main process while a
# BGSAVE or BGREWRITEAOF is in progress.
#
# This means that while another child is saving, the durability of Redis is
# the same as "appendfsync none". In practical terms, this means that it is
# possible to lose up to 30 seconds of log in the worst scenario (with the
# default Linux settings).
#
# If you have latency problems turn this to "yes". Otherwise leave it as
# "no" that is the safest pick from the point of view of durability.

no-appendfsync-on-rewrite no

三、rdb和aof的比较

rdb和aof策略可以同时存在

AOF优势:

以有不同的策略,即:

appendfsync always

appendfsync everysec

appendfsync no:从不同步

AOF劣势:

相同数据集的数据而言aof文件要远大于rdb,恢复慢于rdb

aof运行效率慢于rdb,每秒同步效率好,不同步效率与rdb相同。

官网建议:

RDB:持久化方式能够再指定的时间间隔能对你的数据进行快照

AOF:持久化记录每次对服务器的写操作,当服务器重启的时候会执行这些命令来恢复原始数据,AOF命令以redis协议追加保存每次写的操作到文件末尾。redis还能对AOF文件进行后台重写,使得AOF文件的体积不至于过大。

只做缓存:如果只希望数据在服务器运行的时候存在,也可以不适用任何持久化方式。

`

文章目录
  1. 1. Redis的持久化
  2. 2. 一、RDB
    1. 2.1. 1、RDB
    2. 2.2. 2、Redis
    3. 2.3. 3、Fork
    4. 2.4. 4、RDB保存的是dump.rdb文件。
      1. 2.4.0.0.0.1. #表示900秒之内修改了一次就会记录一下快照
      2. 2.4.0.0.0.2. #300秒改动10次
      3. 2.4.0.0.0.3. 60秒10000次
  • 2.5. 5、save命令:即刻备份。
  • 2.6. 6、stop-writes-on-bgsave-error yes
  • 2.7. 7、rdbcompression yes
  • 2.8. 7、rdbchecksum yes
  • 2.9. 8、dbfilename dump.rdb
  • 2.10. 9、dir ./
  • 2.11. 10、如何停止RDB:
  • 3. 二、AOF
    1. 3.1. 1、介绍
    2. 3.2. 2、配置文件
    3. 3.3. 3、appendfilename “appendonly.aof”
    4. 3.4. 4、appendfsync everysec
    5. 3.5. 5、启动:
    6. 3.6. 6、恢复:
    7. 3.7. 7、rewrite
      1. 3.7.0.1. auto-aof-rewrite-percentage 100
      2. 3.7.0.2. auto-aof-rewrite-min-size 64mb
  • 3.8. 8、no-appendfsync-on-rewrite no
  • 4. 三、rdb和aof的比较
    1. 4.0.0.0.1. AOF优势:
    2. 4.0.0.0.2. AOF劣势:
    3. 4.0.0.0.3. 官网建议: