Redis persistence operation (RDB, AOF)

Redis Persistence operation

Redis Two persistence strategies are provided ：

• RDB (Redis DataBase)
• AOF (Append Only File)

1. RDB (Redis DataBase)

RDB Persistence is mainly through SAVE and BGSAVE Two orders, right Redis The current data in the database do snapshot ( snapshot ) And generate rdb File to achieve . stay Redis It detects when it starts rdb file , Then load rdb The unexpired data in the file is sent to the server .

• SAVE Just keep it , Nothing else , All blocked . Save manually .
• BGSAVE It's non blocking , adopt fork It's done by a subprocess .

SAVE and BGSAVE The difference between ：

• SAVE Call directly rdbSave function , Blocking Redis The main process , Until the save is complete . During the main process block , The server cannot process any requests from the client .
• BGSAVE be fork Make a sub process , The subprocess is responsible for calling rdbSave function , And send a signal to the main process after saving , Notification save completed . Redis The server BGSAVE The client's request can still be processed during execution .

RDB It can be saved dynamically by providing configuration information to the server . For example, by default, the server meets the following requirements 3 Any one of these conditions will trigger BGSAVE command .

save 900 1 //  The server 900 In seconds , The database is at least 1 Time modification 
save 300 10 //  The server 300 In seconds , The database is at least 10 Time modification 
save 60 10000 //  The server 60 In seconds , The database is at least 10000 Time modification 

advantage ：

• Suitable for large-scale data recovery
• Data integrity and consistency requirements are not high, more suitable for use
• Save disk space
• Fast recovery

Inferiority ：

• Fork When , The data in memory is cloned , roughly 2 Double expansion needs to be considered
• although Redis stay fork When used Copy on write technology , But if the data is huge, it still consumes performance .
• Make a backup at a certain interval in the backup cycle , So if Redis accident down If you drop it , All changes since the last snapshot will be lost .

2. AOF (Append Only File)

With journal To record each write operation （ Incremental save ）, take Redis All written instructions executed are recorded ( Reading operation does not record ), Only files can be added but not rewritten ,redis At the beginning of startup, it will read the file and rebuild the data , That is to say redis In case of restart, execute the write instruction from the front to the back according to the contents of the log file to complete the data recovery .AOF Divided into Order to append （append）、 File is written to 、* File synchronization （sync）* Three steps .AOF It is not enabled by default , It can be opened after modifying the configuration file AOF, By default, the file used to store instructions is called ：appendonly.aof

Order to append :

When there is a change 、 Delete operation , The server will append the executed write command to the server state in protocol format after execution aof_buf End of buffer .

File is written to :

Redis The service process of is an event loop , The file events in this loop are responsible for receiving command requests from clients . The server may execute write commands when handling file events , At the same time, it will be added to aof_buf buffer , So before each end of the cycle , Will be called flushAppendOnlyFile function , take aof_buf The data in the buffer is written to AOF In the document .

File synchronization :

flushAppendOnlyFile Functions are configured through the server appendfsync Before the end of each loop aof_buf The data in the buffer is written to AOF After the document , Will sync to in some way AOF In the document , for example ： Sync every time .

AOF rewrite (Rewrite)

AOF When the mode is persistent, write commands are recorded one by one , As the server runs longer and longer ,AOF The file will get bigger and bigger ,AOF Rewriting is to solve this problem .

AOF When files continue to grow and become too large , Meeting fork A new process to rewrite the file ( Also write temporary documents first and then rename),redis4.0 Post version rewriting , It means to put up rdb Snapshot , In the form of a two-tier system attached to the new aof Head , As historical data available , Replace the original daily account operation .

no-appendfsync-on-rewrite ：

• If no-appendfsync-on-rewrite=yes , Don't write aof Files are only written to the cache , User requests don't block , But in this period of time, if you lose the cache data in this period of time .（ Reduce data security , Improve performance ）
• If no-appendfsync-on-rewrite=no, I'll still brush the data to the disk , But we're encountering an override operation , There could be a blockage .（ Data security , But the performance degrades ）

Trigger mechanism , When to rewrite :

Redis It will record the last time it was rewritten AOF size , The default configuration is when AOF The file size is last rewrite Double the size and the file is larger than 64M Trigger when .

Rewriting can save a lot of disk space , Reduce recovery time . But there's a burden to rewrite every time , So set Redis You have to meet certain conditions to rewrite .

• auto-aof-rewrite-percentage： Set the overridden benchmark , The document reaches 100% Start rewriting （ The file is the original rewritten file 2 It's time to trigger ）.
• auto-aof-rewrite-min-size： Set the overridden benchmark , The smallest file 64MB. Reach this value and start rewriting .

If Redis Of AOF The current size >= base_size +base_size*100% ( Default ) And the current size >=64mb( Default ) Under the circumstances ,Redis Would be right AOF Rewrite .

Rewrite process ：

1. bgrewriteaof Trigger override , Judge whether there is bgsave or bgrewriteaof Running , If there is , Then wait for the end of the command to continue .
2. The main process fork Out of the child process to perform the rewrite operation , Make sure the main process doesn't block .
3. Subprocess traversal redis Data in memory to temporary file , The client's write request is written at the same time aof_buf Buffers and aof_rewrite_buf Rewriting the buffer ensures that the original AOF The file is complete and new AOF New data modification actions during file generation will not be lost .
4. (1) The subprocess writes a new AOF After the document , Signal the main process , Parent process update statistics .(2) The main process puts aof_rewrite_buf Write the data in to the new AOF file .
5. Use the new AOF The file covers the old AOF file , complete AOF rewrite .

advantage ：

• The backup mechanism is more robust , Lower probability of data loss .
• Readable log text , By manipulating the AOF steady , It can handle misoperation .

Inferiority ：

• Compared with RDB Take up more disk space .
• Recovery of backup is slower .
• If every read and write is synchronized , There is a certain performance pressure .
• There are individual problems Bug, It can't recover .

3. summary

The two persistence strategies are good ？

• The official recommendation is to use both .
• If you're not sensitive to data , You can use the menu alone RDB.
• It is not recommended to use it alone AOF, Because there may be Bug.
• If it's just a pure memory cache , You don't have to .

Reference article ：Redis Data persistence RDB and AOF The difference between