MySQL three logs

Catalog

One 、binlog

1.1、binlog Format

1.2、binlog Storage mechanism

1.3、binlog application

Two 、redo log

2.1、 Why redo log

2.2、redo log Basic concepts

2.3、redo log Record form

3、 ... and 、undo log

summary

One 、binlog

binlog  Used to record the execution of the database Write operations ( Exclude queries ) Information , Stored on disk in binary form .binlog  yes  mysql Of Logic log , And by the  Server  Layer to record , Using any storage engine  mysql  The database records  binlog  journal

binlog  It is written by appending , Can pass max_binlog_size  Parameter settings for each  binlog File size , When the file size reaches the given value , A new file will be generated to save the log .

1.1、binlog Format

binlog  There are three formats for logs , Respectively  STATMENT 、 ROW  and  MIXED.

stay  MySQL 5.7.7  Before , The default format is  STATEMENT , MySQL 5.7.7  after , The default value is  ROW. The log format is through  binlog-format  Appoint

• STATMENT： be based on SQL  Copy of sentences , Each one will modify the data sql The statement will record binlog  in , Yes, remember sql sentence  

  • advantage ： You don't need to record every line change , Less binlog Log volume , Economize IO  , This improves performance ;

  • shortcoming ： In some cases, the master-slave data is inconsistent , Such as execution sysdate() 、  slepp()   etc. .

• ROW： Line based replication , Don't record every sql The context information of the statement , Just record which data has been modified ,row The format records the contents of the line , Make two notes , Before and after the update .

  • advantage ： There will be no stored procedures under certain circumstances 、 or function、 or trigger The call and trigger of cannot be copied correctly ;

  • shortcoming ： There will be a lot of logs , In especial alter table  When it's time to make the journal skyrocket

• MIXED： be based on STATMENT  and  ROW  Mixed replication of the two modes , General replication uses STATEMENT  Mode save  binlog , about  STATEMENT  Mode can not be copied by operation using  ROW  Mode save  binlog

1.2、binlog Storage mechanism

about  InnoDB  For storage engines , Only in Transaction submission Only when biglog , At this point, the record is still Memory in , that  biglog When did it arrive disk What about the middle ？

mysql  adopt  sync_binlog  Parameter control  biglog  When to brush the disk , The value range is  0-N：

• 0： Don't force it , It is up to the system to determine when to write to the disk ;

• 1： Every time  commit  All the time, we have to  binlog  Write to disk ;

• N： Every time N One transaction , Will be  binlog  Write to disk .

As can be seen from the above , sync_binlog  most Security Yes setting yes  1 , This is also MySQL 5.7.7 The default value for later versions . But setting a larger value can improve database performance , Therefore, in practice, you can also increase the value appropriately , Sacrifice some consistency for better performance .

1.3、binlog application

1. Master slave copy  ： stay  Master  End open  binlog , And then  binlog Send to each  Slave  End , Slave  End replay  binlog  So as to achieve the consistency of master-slave data , For example, database Master slave copy 、 Alibaba Middleware canal It's all through binlog Realization

2. Data recovery  ： By using  mysqlbinlog  Tools to recover data .

Two 、redo log

2.1、 Why redo log

We all know , One of the four characteristics of a transaction is   persistence  , To be specific As long as the transaction is committed successfully , Then the changes to the database will be permanently saved , It's impossible to return to the original state for any reason  .redo log yes innodb Layers produce , After the record on the log has been modified , Logs will be covered , Cannot be used for data rollback / Data recovery and other operations .

The easiest way to do this is to do it every time a transaction is committed , The transaction involves the modified Data pages Refresh all to disk . But there are serious performance problems with doing this , It is mainly reflected in two aspects ：

1. because  Innodb  In order to   page   Disk interaction for units , A transaction is likely to modify only a few bytes in a data page , At this time, the complete data page will be flashed to the disk , It's a waste of resources ！

2. A transaction may involve modification Multiple data pages , And these Data pages are not physically continuous , Use random IO Poor write performance ！

therefore  mysql  Designed  redo log ,  Specifically, it only records the changes made by transactions to the data page , This will solve the performance problem perfectly

2.2、redo log Basic concepts

redo log  It consists of two parts ： One is in memory The log buffer ( redo log buffer ), The other is... On disk Log files ( redo logfile).

mysql  Every time you execute one  DML  sentence , First write the record to  redo log buffer, At a later time point, multiple operation records will be written to  redo log file. such   Write the log , Write the disk again   The technology is  MySQL It's often said in  WAL(Write-Ahead Logging)  technology .

In a computer operating system , User space ( user space ) In general, the buffer data under the following conditions cannot be directly written to the disk , The middle must pass through the operating system kernel space ( kernel space ) buffer ( OS Buffer ).

therefore , redo log buffer  write in  redo logfile  It's actually Write... First  OS Buffer , And then through the system call  fsync()  Brush it to  redo log file in .

MySQL Support user-defined in commit How to make log buffer Log brush in log file in . This control is through variables  innodb_flush_log_at_trx_commit  The value of . This variable has 3 Seed value ：0、1、2, The default is 1. But pay attention to , This variable just controls commit Whether the action refreshes log buffer To disk .

• When set to 1 When , Every time a transaction commits, it will log buffer Log writing in os buffer And call fsync() Brush to log file on disk in . In this way, even if the system crashes, it won't lose any data , But because every commit is written to disk ,IO Poor performance .
• When set to 0 When , The transaction will not commit log buffer The log is written to os buffer, It's written every second os buffer And call fsync() Write to log file on disk in . That is to say, it is set to 0 When is ( about ) Refresh writes to disk per second , When the system crashes , Will lose 1 Seconds of data .
• When set to 2 When , Each submission is only written to os buffer, And then it's called every second fsync() take os buffer The log in is written to log file on disk.

2.3、redo log Record form

In the same transaction , Whenever the database changes data , After updating the modification results to memory , Will be in redo log Add a row of records “ What changes need to be made on which data page ”, And set the record status to prepare, wait until commit After committing the transaction , We will put this transaction in redo log The status of the added records is set to commit state , After that, we will modify the falling disc , Will redo log The state is commit All the changes of records are written to disk . The process is as follows

redolog The size of is fixed , stay mysql By modifying the configuration parameters innodb_log_files_in_group and innodb_log_file_size Configure the number of log files and the size of each log file ,redolog Write in a circular way , When it comes to the end , You'll go back to the beginning and cycle through the log . Here's the picture

write pos Indicates the current record position of the log , When ib_logfile_4 After full , From ib_logfile_1 Record from the beginning ;check point Write changes to the log to disk , Complete the data input , After the data is on the disk checkpoint Will erase the relevant records from the log , namely write pos->checkpoint The part between is redo log The empty part , Used to record new records ,checkpoint->write pos Between redo log Data modification record to be set down . When writepos Catch up checkpoint when , Stop recording first , Push first checkpoint Move forward , Vacate the location to record a new log .
With redo log, When the database is down and restarted , It can be done by redo log Recover the data that has not been dropped , That is to ensure that the submitted transaction records will not be lost .

3、 ... and 、undo log

One of the four features of database transaction is   Atomicity  , To be specific   Atomicity refers to a series of operations on a database , All or nothing , All or nothing , Partial success is unlikely .

actually ,  Atomicity   The bottom floor is through  undo log  Realized .undo log It mainly records the logical changes of data , Like one  INSERT  sentence , There's a corresponding one DELETE  Of  undo log , For each  UPDATE  sentence , It's the opposite  UPDATE  Of  undo log , So when something goes wrong , You can roll back to the data state before the transaction .