Redis (II) -- persistence

Redis（ Two ）—— Persistence

1、 brief introduction

What is persistence ？

utilize Permanence The storage medium saves the data , The working mechanism of restoring the saved data at a specific time is called persistence .

Why persistence

prevent Unexpected data The loss of , Ensure data Security

What does persistence save

• Will the current Data status Preservation , snapshot form , Store data results , The storage format is simple , The focus is data
• Transfer data to The operation process Preservation , journal form , Store operation procedures , The storage format is complex , The focus is on the operation of data The process

2、RDB

RDB Starting mode ——save

• command

  saveCopy
  

• effect

  Manually perform a save operation

RDB Configure related commands

• dbfilename dump.rdb
  • explain ： Set the local database file name , The default value is dump.rdb
  • Experience ： Usually set to dump- Port number .rdb
• dir
  • explain ： Set up storage .rdb Path to file
  • Experience ： It is usually set to a directory with large storage space , Directory name data
• rdbcompression yes
  • explain ： Set whether to compress data when storing to local database , The default is yes, use LZF Compress
  • Experience ： It is usually on by default , If set to no, Can save CPU The elapsed time , But it makes the stored files bigger （ huge ）
• rdbchecksum yes
  • explain ： Set whether to RDB File format verification , The verification process is carried out in the process of writing and reading files
  • Experience ： It is usually on by default , If set to no, It can save the reading and writing process about 10% Time consuming , However, there is a certain risk of data corruption

RDB Starting mode ——save How commands work

Be careful ：save Instructions Executive meeting Blocking At present Redis The server , Until now RDB Until the process is complete , It could cause Long time blocking , The online environment Not recommended .

RDB Starting mode ——bgsave

• command

  bgsaveCopy
  

• effect

  Start the background save operation manually , but Not immediately

RDB Starting mode —— bgsave How commands work

Be careful ： bgsave command Is aimed at save Blocking problems do Optimize .Redis It's all about RDB All operations use bgsave The way ,save The command can be discarded , Recommended bgsave

bgsave The save operation of can be performed through redis Log view of

docker logs myredisCopy

RDB Starting mode ——save To configure

• To configure

  save second changesCopy
  

• effect

  Satisfy Time limit Within the scope of key The number of changes reached Specify the quantity That is, persistence

• Parameters

  • second： Monitor the time frame
  • changes： monitor key Amount of change

• Configuration location

  stay conf file To configure

RDB Starting mode ——save Configuration principle

Be careful ：

• save The configuration should be set according to the actual business situation , If the frequency is too high or too low, performance problems will occur , The results can be catastrophic
• save In the configuration, for second And changes Settings usually have Complementary correspondence Relationship （ One big and one small ）, Try not to set up an inclusive relationship
• save After the configuration is started, the execution is bgsave operation

RDB Comparison of startup modes

RDB Advantages and disadvantages

• advantage
  • RDB Is a compact binary , High storage efficiency
  • RDB What's stored inside is redis A snapshot of data at a point in time , Very suitable for The data backup , Copy in full Such as the scene
  • RDB Recovery of data Speed than AOF fast quite a lot
  • application ： Every X Hours to perform bgsave Backup , And will RDB Copy the file to the remote machine , For disaster recovery
• shortcoming
  • RDB Whether it's executing instructions or using configuration , Unable to achieve real-time persistence , There is a high probability of data loss
  • bgsave The instructions are executed every time they run fork operation Create child process , want sacrifice Drop some performance
  • Redis In many versions of RDB Unified version of file format , There may be data formats between versions of services Not compatible The phenomenon

3、AOF

AOF Concept

• AOF(append only file) Persistence ： Record in a separate log Every time Write orders , Reexecute on reboot AOF Command in file , To recover data . And RDB In contrast, it can simply describe the process of changing recorded data to recorded data generation
• AOF Its main function is to solve the real-time problem of data persistence , So far Redis persistent Main stream The way

AOF The process of writing data

AOF Three strategies for writing data (appendfsync)

• always
  • Each write operation is synchronized to AOF In file , Data zero error , Low performance , Not recommended
• everysec
  • Synchronizes the instructions in the buffer to per second AOF In file , The accuracy of the data is high , Higher performance , It is recommended to use , Is also the default configuration
  • Lost in the event of a sudden system outage 1 Seconds of data
• no
  • The operating system controls each synchronization to AOF The period of the document , The whole process Out of control

AOF Function of open

• To configure

  appendonly yes|noCopy
  

  • effect
    • Open or not AOF Persistence function , The default is not on

• To configure

  appendfsync always|everysec|noCopy
  

  • effect
    • AOF Write data strategy

AOF rewrite

effect

• Reduce disk usage , Improve disk utilization
• Improve persistence efficiency , Reduce persistent write time , Improve IO performance
• Reduce data recovery time , Improve data recovery efficiency

The rules

• Data that has timed out in the process is no longer written to the file

• Ignore invalid instructions , In process data generation is used when rewriting , So new AOF Write command to keep only the final data in the file

  • Such as del key1、 hdel key2、srem key3、set key4 111、set key4 222 etc.

• Multiple write commands for the same data are combined into one command

  • Such as lpush list1 a、lpush list1 b、 lpush list1 c Can be converted to ：lpush list1 a b c
  • In order to prevent client buffer overflow caused by excessive data , Yes list、set、hash、zset Other types , Each instruction can write at most 64 Elements

How to use

• Manual rewriting

  bgrewriteaofCopy
  

• Automatic override

  auto-aof-rewrite-min-size size 
  auto-aof-rewrite-percentage percentageCopy
  

working principle

AOF Automatic override

• Automatically override trigger condition settings

  // The minimum size of the trigger override 
  auto-aof-rewrite-min-size size 
  // The minimum percentage that must be reached to trigger rewriting 
  auto-aof-rewrite-percentage percentCopy
  

• Automatically rewrite trigger comparison parameters （ Operation instruction info Persistence Get specific information ）

  // At present .aof File size 
  aof_current_size 
  // Base file size 
  aof_base_sizeCopy
  

• Automatically override trigger conditions

  

working principle

Buffer strategy

AOF Buffer synchronization file policy , By the parameter appendfsync control

• write Operation will trigger delay write （delayed write） Mechanism ,Linux Provide a page buffer in the kernel To improve the hard disk IO performance .write The operation directly returns... After writing to the system buffer . Synchronize hard disk operation It depends on the system scheduling mechanism , Columns such as ： Buffer page space is full or reaches a specific time period . Synchronize files front , If the system fails at this time , Data in the buffer will be lost .
• fsync For single file operations （ such as AOF file ）, Do force hard disk synchronization ,fsync Will know After writing to the hard disk, return to , Data persistence is guaranteed .

4、RDB VS AOF

RDB And AOF The choice of

• Very sensitive to data , The default... Is recommended AOF Persistence scheme

  • AOF Persistence strategy uses everysecond, Every second fsync once . The strategy redis It can still maintain good processing performance , When something goes wrong , At most 0-1 Seconds of data .
  • Be careful ： because AOF file Large storage volume , And Slow recovery

• Data presentation phase validity , It is recommended to use RDB Persistence scheme

  • Data can be well done in the phase without loss （ This stage is manually maintained by developers or operation and maintenance personnel ）, And Faster recovery , Stage Point data recovery usually uses RDB programme
  • Be careful ： utilize RDB Implementing compact data persistence makes Redis It's very low

• Comprehensive comparison

  • RDB And AOF Your choice is actually a trade-off , Each has its advantages and disadvantages
  • If you can't afford to lose data within a few minutes , It is very important for business data sensitive , choose AOF
  • If it can withstand data loss within a few minutes , And the pursuit of large data sets Recovery speed , choose RDB
  • Disaster recovery options RDB
  • Double insurance strategy , At the same time open RDB and AOF, After restart ,Redis priority of use AOF To recover data , Reduce lost data