Mysql's redo log detailed explanation

1. Introduction

The redo log is generated after the transaction is committed. If the service is down at this time, the redo log can be used to restore the data in the later restart.Guarantee the durability of the transaction (the transaction will take effect permanently after the transaction is committed).

2, redo log generation process

Step 1: First read the original data from disk into memory, modify the memory copy of the data
Step 2: Generate a redoThe log is written to the redo log buffer, which records the modified value of the data.
Step 3: When the transaction commits, flush the contents of the redo log buffer to the redo log file.The redo log file adopts the method of append writing
Step 4: Regularly flush the modified data in memory to the disk

3, redo log memory level

When the server starts, it applies to the operating system for a large piece of continuous memory space called redo log buffer.In Chinese, it is the redo log buffer.This memory space is divided into several consecutive redo log blocks.A redo log block occupies a size of 512 Nin knots.

The default is 16M, the range can be modified: 4096M~ 1M

View: show variables like '%innodb_log_buffer_size%'

4. Redo log disk level

Storage location (default): /var/lib/mysql

The size of the space is also opened up by default and will not increase over time.

5. Brush strategy

The writing of the redo log is not directly written to the disk. The InnoDB engine will first write the redo log buffer when writing the redo log, and then flush it to the real redo log file at a certain frequency.What do you think of a certain frequency here? This is the brushing strategy we are going to talk about.

Note that the process of flushing the redo log buffer to the redo log file is not really flushing to the disk, but just flushing it to the file system cache (pagecache) (this is a modern operating system to improve the efficiency of file writing)An optimization to do), the real write will be decided by the system itself (for example, if the page cache is large enough). Then there is a problem for InnoDB, if it is handed over to the system for synchronization, also if the system goes down,Then the data is also lost (although the probability of the entire system being down is still relatively small).
In response to this situation, InnoDB provides the innodb_flush_log_at_trx_commit parameter, which controls how the commit commits the transaction.The log in the redo log buffer is flushed to the redo log file. It supports three strategies:

6. Strategy summary

0: Indicates that no flush operation is performed each time a transaction is committed.(The system defaults the master thread to synchronize the redo log every 1s)
1: Indicates that the synchronization will be performed each time a transaction is committed, and the disk brush operation (default value)
2: Indicates that each transaction is committedAll only write the contents of the redo log buffer to the page cache without synchronization.It is up to the OS itself to decide when to sync to disk files.

1) When innodb_flush_log_at_trx_commit=1
is 1, as long as the transaction is successfully submitted, the redo log record must be in the hard disk, and there will be no data loss.
If MysQL hangs or crashes during transaction execution, this part of the log is lost, but the transaction is not committed, so there will be no loss if the log is lost.D of ACID can be guaranteed, data will never be lost, but the efficiency is the worst.
It is recommended to use the default value. Although the probability of operating system downtime is smaller than the probability of database downtime, since transactions are generally used, data security is relatively more important.

2) When innodb_flush_log_at_trx_commit=2
is 2, as long as the transaction commits successfully, the content in the redo log buffer is only written to the file system cache (page cache).
If only MySQL hangs, there will be no data loss, but if the operating system is down, there may be data loss for 1 second. In this case, D in ACID cannot be satisfied.But a value of 2 is definitely the most efficient.

3) When innodb_flush_log_at_trx_commit=o
is 0, the fsync operation of the redo log is performed every 1 second in the master thread, so the instance crash loses transactions within 1 second at most.(The master thread is responsible for asynchronously flushing the data in the buffer pool to the disk to ensure data consistency)
If the value is o, it is a compromise. Its IO efficiency theory is higher than 1, lower than2, this strategy also has the risk of losing data, and there is no guarantee D.

7. Several important configurations

1) innodb_log_files_in_group: Indicate the number of redo log files, named such as: ib_logfile0, iblogfile....iblogfilen.The default is 2, and the maximum is 100.

2) innodb_log_file_size: Set the size of a single redo log file, the default value is 48M.The maximum value is 512G. Note that the maximum value refers to the sum of the entire redo log. series of files, that is, (innodb_log_files_in_group * innodb_log_file_size) cannot be greater than the maximum value of 512G.

8. Redo log flushing process

Every time the redo log is flushed and recorded in the log file group, the write pos position will be moved back and updated.Every time MySQL loads the log file group to restore data, it will clear the loaded redo log records and move the checkpoint back to update.The empty space between write pos and checkpoint can be used to write new redo log records.

If the write pos catches up with the checkpoint, it means that the log file group is full. At this time, no new redo log records can be written. MysQL has to stop, clear some records, and advance the checkpoint.