The persistence mode of redis - RDB and AOF persistence mechanisms

redis Persistence of -RDB and AOF Two persistence mechanisms

Redis It is a memory based non relational K-V database , Since it's memory based , If Redis The server is down , The data is lost . To avoid data loss ,Redis Provides Persistence , That is, save the data to disk .

The loading process of persistent files is as follows :

1.RDB—Redis Default persistence method

Write the data set snapshot in memory to disk within the specified time interval

RDB There are two kinds of ：

• One is synchronous , call save The command triggers redis Conduct RDB File generation backup , But this is a synchronization command , Before the backup is complete ,redis The server does not respond to any client requests .
• The other is asynchronous , call bgsave command ,redis The server fork A subprocess carries out RDB File backup generation , meanwhile , The main process can still respond to client requests .

Two trigger modes ： Manual trigger （save command ）, Automatic triggering （bgsave command , recommend ）
save command ： Block the current redis until rdb Persistence complete . If the memory instance is large , It will cause long-term congestion , Online environment is not recommended .
bgsave command ：redis Process execution fork Command to create subprocesses to complete persistence , Short blocking time （ microsecond ）,save Command optimization .
bgsave Trigger conditions ：
1. stay redis-cli In the implementation of shutdown close redis The service , If it's not on aof Persistence , Automatic execution bgsave command
2.redis.conf in save m n if redis stay m In seconds n The second command execution triggers
3. When the slave node is just online , Trigger master node bgsave command , And then what will be generated rdb Send the file to the slave node to complete the full copy

fork yes linux System call ： In the current process ,fork A subprocess , The child process initially shares a memory area with the main process . Because the main process keeps writing data , There is a concurrency conflict with the child process . here ,redis use Copy on write technology （cow）：
When the main process writes , First, a copy of the memory page that will involve the write operation will be copied . Then the main process writes on the newly copied memory page , The original memory pages continue to be persisted by child processes

Related configuration

1.save Trigger bgsave

save 900 1           # 900 Once per second set operation   Then it is persistent 1 Time  
save 300 10         # 300 Within seconds 10 Time set operation , Then it is persistent 1 Time 
save 60 10000     # 60 Within seconds 10000 Time set operation , Then it is persistent 1 Time 

2. stay redis.conf There are settings in the file The default is file name and save path （ The default is Under the current startup directory Is the directory where the startup command is executed ）

3. stop -writes -on-bgsave-er ror When Redis If you can't write to disk ( The disk is full ), Turn it off Redis Write operations for . recommend yes.

4.rdbcompression

For snapshots stored on disk , You can set whether to compress storage . If so ,redis Will be used LZF Algorithm for compression .

If you don't want to consume CPU To compress , It can be set to turn off this function . recommend yes.

5.rdbchecksum Check for integrity

After storing the snapshot , Can also let redis Use CRC64 Algorithm to check data , check RDB Whether the file is damaged

But doing so will add about 10% Performance consumption of , If you want to get the maximum performance improvement , This function can be turned off. It is recommended that yes.

Advantages and disadvantages

• advantage ：

l Flexible setting of backup frequency and cycle .

l Suitable for large-scale data recovery , Perfect for cold backup , For disaster recovery ,RDB It's a very good choice . Save on other cloud storage

l Data integrity and consistency requirements are not high, more suitable for use

l Save disk space

l Maximize performance , It can make Redis Maintain high performance .

l Fast recovery , Compared with AOF Mechanism ,RDB Faster recovery of , Better for data recovery , Especially when the data set is very large .

• shortcoming ：

l Fork When , The data in memory is cloned , roughly 2 Double expansion needs to be considered

l Make a backup at a certain interval in the backup cycle , So if Redis accident down If you drop it , All changes since the last snapshot will be lost

l If the dataset is ⼤,fork() It can be time consuming , And if the dataset is ⼤ And CPU Poor performance , May lead to Redis stop ⽌ Be a guest ⼾ End service ⼏ Millisecond is very ⾄⼀ Second .

​ therefore ,RDB It is recommended to implement when the business is low , For example, in the middle of the night .

2.AOF（Append Only File)

Log every write operation in the form of a log （ Incremental save ）, take Redis All written instructions executed are recorded ( Reading operation does not record ), Only files can be added but not rewritten .

Not on by default , Change it to yes open . The saved path is the same as rdb equally .

AOF and RDB At the same time open , The system defaults to AOF The data of （ There is no loss of data ）

Data backup and recovery rdb equally : Save a file , When something goes wrong ,copy To come over , restart redis

Abnormal recovery

Modify the default appendonly no, Change it to yes

If encountered AOF File corruption , adopt /usr/local/bin/redis-check-aof–fix appendonly.aof Resume

Backup is bad AOF file

recovery ： restart redis, Then reload

Related configuration

AOF Synchronous frequency setting

appendfsync always

Always sync , Every time Redis All writes are immediately logged ; Poor performance but good data integrity

appendfsync everysec

A second synchronous , Log once per second , If it goes down , This second's data may be lost .

appendfsync no

redis Don't take the initiative to synchronize , Give the timing of synchronization to the operating system .

Rewrite Compress

Rewrite the compression mechanism ,AOF When the file size exceeds the set threshold ,Redis Will start AOF The content of the file is compressed , Keep only the smallest instruction set that can recover data

Redis It will record the last time it was rewritten AOF size , The default configuration is when AOF The file size is last rewrite Double the size and the file is larger than 64M Trigger when

If Redis Of AOF The current size >= base_size +base_size*100% ( Default ) And the current size >=64mb( Default ) Under the circumstances ,Redis Would be right AOF Rewrite .

AOF Persistence process

（1） The client request write command will be append Append to AOF In the buffer ;

（2）AOF Buffer based on AOF Persistence strategy [always,everysec,no] Will operate sync Synced to disk AOF In file ;（ The default is everysec）

（3）AOF When the file size exceeds the rewrite policy or manual rewrite , Would be right AOF file rewrite rewrite , Compress AOF File capacity ;

（4）Redis When the service is restarted , Will return load load AOF The write operation in the file achieves the purpose of data recovery ;

Advantages and disadvantages

optimal

• The backup mechanism is more robust , Lower probability of data loss .

• Readable log text , By manipulating the AOF steady , It can handle misoperation .

Lack of

• Compared with RDB Take up more disk space .

• Recovery of backup is slower .

• If every read and write is synchronized , There is a certain performance pressure .

• There are individual problems Bug, It can't recover .

The official recommendation is to use both .

If you're not sensitive to data , You can use the menu alone RDB.

It is not recommended to use it alone AOF, Because there may be Bug.

If it's just a pure memory cache , You don't have to .

When on at the same time

3.RDB-AOF Mix persistence

# When rewriting the AOF file, Redis is able to use an RDB preamble in the
# AOF file for faster rewrites and recoveries. When this option is turned
# on the rewritten AOF file is composed of two different stanzas:
#
#   [RDB file][AOF tail]
#
# When loading Redis recognizes that the AOF file starts with the "REDIS"
# string and loads the prefixed RDB file, and continues loading the AOF
# tail.
aof-use-rdb-preamble yes

After opening

Redis The server In execution AOF When overriding the operation , It'll be like executing BGSAVE Order that , According to the current state of the database To produce Corresponding RDB data , And put these data write in New AOF In file , As for those stay AOF After rewriting starts Executive Redis command , Will continue to be in the form of agreement text Append to new AOF End of file , That is, the existing RDB Behind the data

• In this mode ,AOF The resulting file will also contain RDB Content and AOF Content of format , The first half of the document is RDB Full data in format , The second half is Redis Incremental data in command format

When a supporter RDB-AOF Mixed persistence mode Redis The server starts and loads AOF When you file , It will check. AOF Whether the beginning of the file contains RDB Content of format