Redis Persistence mechanism

Redis There are two ways to persist ：RDB and AOF

RDB

RDB The principle is that Redis Timing of data recording in memory dump To disk .

1> The snapshot file is called RDB file , By default, it is saved in the current running Directory , Also available at redis.conf Middle configuration

# Is it compressed? , It is not recommended to open , Compression also consumes cpu, Disks are worthless 
rdbcompression yes
#RDB File name 
dbfilename dump.rdb
# The path and directory where the file is saved 
dir ./

2> RDB When will it be executed ?

Automatic triggering :

• The default is when the service stops :Redis During shutdown, it will be executed once RDB, When Redis After instance failure and restart , Read snapshot file from disk , Restore data .

• save m n : stay m Seconds , If there is n The keys change , Will trigger persistence automatically , adopt bgsave perform , If you set multiple 、 As long as its — It will trigger , The configuration file has a default configuration ( You can comment out )

  save 900 1 #900 Seconds , If at least 1 individual key Be modified , execute bgsave
  save 300 10 #300 Seconds , If at least 10 individual key Be modified , execute bgsave
  save “” # Ban RDB

• flushall: For emptying redis All databases ,flushdb To empty the current redis Data in the library ( The default is 0 The database ), It will be emptied RDB file , It also generates dump.rdb、 Content is empty

• Master slave synchronization : It will be triggered automatically during full synchronization bgsave command , Generate rdb Send to slave

Manual trigger ：

[[email protected] ~]# redis-cli
127.0.0.1:6379> save
127.0.0.1:6379> bgsave
Background saving started

• save command , send Redis In a blocking state , until RDB Persistence complete , Will respond to commands from other clients .
• bgsave command ,fork Create a child process to perform persistence , The main process is only in fork There is a brief blockage in the process , After the child process is created , The main process can respond to client requests . The subprocess writes the data set to the temporary file first , After writing successfully , Replace the previous file , Compress storage with binary .

fork Underlying implementation ：

The data of process operation is stored in physical memory , But all processes cannot directly manipulate physical memory , Instead, the operating system allocates virtual memory to each process , Virtual memory maps to physical memory , Processes can only manipulate virtual memory , The operating system maintains virtual memory through page tables , The process reads and writes data to physical memory based on the page table .fork The process is not to copy a copy of the memory data to the child process , This slow speed affects the efficiency of the main process , Instead, copy the page table of the main process to the child process , And set the data to read-only , In this way, the child process can also operate the same piece of physical memory as the main process , So as to realize the space sharing with the main process . There is no need to copy all data to child processes , Fast , The main process has a short blocking time . Then the subprocess can read the data in memory according to the page table and write it to disk , Replace the old backup file .

When the sub process is backed up, the main process changes the data process , It's using copy-on-write technology ：

• When the main process performs a read operation , Access shared memory
• When the main process performs a write operation , Will involve the memory page of the write operation , Write on the newly copied memory page

RDB The way bgsave Basic flow ：

• fork The main process gets a child process , Shared memory space
• The subprocess reads the memory data and writes new data RDB file
• New use RDB Replace the old file with RDB file .

RDB The shortcomings of ?

• because RDB Long execution interval , two RDB In case of downtime when writing data between , There is a risk of data loss

• fork Subprocesses 、 Compress 、 Write RDB The files are time-consuming

AOF

AOF Its full name is Append Only File( Additional documents ).Redis Each write command will be recorded in an additional way AOF file

1> AOF The default is off , Need modification redis.conf Configuration file to open AOF:

# Open or not AOF function , The default is no
appendonly yes
#AOF The name of the document 
appendfilename "appendonly.aof"

2> AOF Synchronization strategies , It can be done by redis.conf File to configure ：

# Indicates that every time a write command is executed , Immediately record to AOF file , Lose at most — strip 
appendfsync always
# After the write command is executed, put it first AOF buffer , Then it means every 1 Seconds to write buffer data to AOF file , Is the default scheme , Once the system goes down , Then the modified data will be lost in one second 
appendfsync everysec
# After the write command is executed, put it first AOF buffer , It's up to the operating system to decide when to write the contents of the buffer back to disk , More data may be lost 
appendfsync no

Configuration item	When to brush the disk	advantage	shortcoming
Always	Synchronous brush set	High reliability , Hardly lose data	High performance impact
everysec	Swipe the disk every second	Moderate performance	At most 1S data
no	Operating system control	Best performance	Poor reliability , A large amount of data may be lost

3> AOF rewrite ：AOF Because it's a record command , File than RDB Much bigger . and AOF Will record for the same key Multiple write operations , But only the last write operation makes sense . Through execution bgrewriteaof command , It can make AOF The file performs the rewriting function , Achieve the same effect with the least number of commands .

Redis It will also be automatically rewritten when the threshold is triggered AOF file . The threshold can also be in redis.conf Middle configuration ：

#AOF If the file grows more than the last time, the rewriting will be triggered 
auto-aof-rewrite-percentage 100
#AOF What is the minimum size of the file to trigger rewriting 
auto-aof-rewrite-min-size 64mb

Mixed mode

AOF | RDB contrast ：

	RDB	AOF
Persistence mode	Snapshot the whole memory regularly	Record every command executed
Data integrity	Incomplete , Lost between backups	Relatively complete , Depending on the disk brushing strategy
file size	There will be compression , Small file size	Record orders , Large file size
Downtime recovery speed	Soon	slow
Data recovery priority	low , Data integrity is not as good as AOF	high , Higher data integrity
System resource usage	high , A lot of CPU And memory consumption .fork Subprocesses 、 Compress 、 Write RDB The files are time-consuming , When the data set is large , It may cause the whole server to stop serving for hundreds of milliseconds , even to the extent that 1 Second .	low , Mainly disk IO resources , but AOF Rewriting takes up a lot of CPU And memory resources
Use scenarios	Can tolerate several minutes of data loss , Pursue faster startup speed	Scenarios that require high data security

RDB and AOF Each has its own advantages and disadvantages , Use RDB Persistence carries the risk of data loss , But the recovery speed is fast , While using AOF Persistence ensures data integrity , But it will be slow to recover data . So from the redis4 After that, a new hybrid AOF and RDB The pattern of ：RDB Make a full backup , stay RDB Write operations during backup are recorded to AOF In file , namely AOF Do incremental persistence .

When the rewriting policy is satisfied or the rewriting is triggered manually ,fork A child thread stores memory data in RDB Write in binary format AOF File header , Those that are executed after the rewrite operation is executed redis command , with AOF The persistence method is appended to AOF End of file .

In this case , When the service is restarted, it will pass RDB File to rebuild content , Reuse AOF To re execute the recent write instruction , The data integrity is guaranteed , It also improves the performance of recovery .

aof-rdb The persistence mechanism is not enabled by default , Need to be in redis.conf Configuration is enabled in the configuration file , And open AOF Persistence .

#  Hybrid persistence switch 
aof-use-rdb-preamble yes