MySQL - Multi version concurrency control (mvcc)

Multi version concurrency control （MVCC）

1. What is? MVCC

MVCC (Multiversion Concurrency Control), Multi version concurrency control . seeing the name of a thing one thinks of its function ,MVCC Multiple versions of the database are used to manage the data concurrency control . This technology makes it possible to InnoDB Execute at the transaction isolation level of Read consistency . The operation is guaranteed . In other words , To query some rows that are being updated by another transaction , And you can see their values before they are updated , In this way, when doing the query, you don't have to wait for another transaction to release the lock .

MVCC There is no formal standard , In different DBMS in MVCC The implementation of may be different , Nor is it commonly used ( You can refer to the relevant DBMS file ). Here's how InnoDB in MVCC Implementation mechanism (MySQL Other storage engines don't support it )

2 Snapshot read and current read

MVCC stay MySQL InnoDB In order to improve the database concurrency performance , Deal with it in a better way read - Write conflict , Even if there is a conflict between reading and writing , Can also do No locks , Non blocking concurrent read , And this reading refers to Read the snapshot , Instead of The current reading . The current reading is actually a lock operation , It's the realization of pessimistic lock . and MVCC The essence is a way of adopting optimistic lock thought .

2.1 Read the snapshot
Snapshot reading is also called consistent reading , Read snapshot data . Simple without lock SELECT All belong to snapshot reading , Non blocking read without lock ; Such as this :

select * from player where ...

The reason why snapshot reading occurs , It is based on the consideration of improving concurrent performance , The implementation of snapshot reading is based on MVCC, In many cases , Avoid lock operation , Lower the cost .

Since it's based on multiple versions , What the snapshot may read is not necessarily the latest version of the data , It could be the previous version of history .

The premise of snapshot read is that isolation level is not serial level , Snapshot reads at the serial level degrade to current reads .

2.2 The current reading
The latest version of the record is currently being read （ The latest data , Instead of historical versions of the data ), When reading, it is also necessary to ensure that other concurrent transactions cannot modify the current record , Will lock the read record . The lock SELECT, Or add, delete and modify the data, which will be read currently . such as :

3. review

3.1 Let's talk about the isolation level again
We know that business has 4 Isolation levels , There are three possible concurrency problems ：

stay MysQL in , The default isolation level is repeatable , It can solve the problem of dirty reading and non repeatable reading , If only from the perspective of definition , Reading it doesn't solve the problem . If we want to solve the problem of unreal reading , We need to use serialization , That is to raise the isolation level to the highest level , But this will greatly reduce the transaction concurrency of the database .

MVCC You don't have to use a lock mechanism , But through optimistic locking to solve the problem of non repeatable reading and unreal reading ! It can replace row level locks in most cases , Reduce system overhead .

3.2 Hide fields 、Undo Log Version chain

Take a look back. undo The version chain of the log , For the use of InnoDB For the table of the storage engine , Its clustered index records contain two necessary hidden columns .

• trx_id: Every time a transaction changes a clustered index record , Will put the affairs of the business id Assign a value to trx_id Hide columns .
• roll_pointer: Every time a cluster index record is changed , Will write the old version to undo In the log , Then the hidden column is equivalent to a pointer , It can be used to find the information before the modification of the record .

4、MVCC The realization principle is ReadView

MVCC The realization of depends on ： Hide fields 、Undo Log、Read View

4.1 What is? ReadView
stay MVCC In mechanism , When multiple transactions update the same row record, multiple historical snapshots will be generated , These historical snapshots are kept in Undo Log in . If a transaction wants to query the row record , Which version of the row record need to be read ? It needs to be used ReadView 了 , It helps us solve the visibility problem of rows .

ReadView Is that transactions are using MVCC The read view generated when the snapshot read operation is performed by the mechanism . When a transaction starts , A snapshot of the current database system will be generated ,InnoDB An array is constructed for each transaction , Used to record and maintain the current active transactions of the system lD(“" active " Refers to , Launched but not yet submitted ）.

4.2 Design thinking
Use READ UNCOMMITTED Isolation level transactions , Because you can read the modified records of uncommitted transactions , So just read the latest version of the record .

Use SERIALIZABLE Isolation level transactions ,InnoDB Specify the use of locks to access records .

Use READ COMMITTED and REPEATABLE READ Isolation level transactions , Must be sure to read Transactions that have been committed Modified records . If another transaction has modified the record but has not yet committed , You can't read the latest version of the record directly , The core problem is to judge which version in the version chain is visible to the current transaction , This is a ReadView The main problem to be solved .

This ReadView It mainly includes 4 A more important content , They are as follows :

4.3 ReadView The rules of

With this ReadView, So when you access a record , Just follow the steps below to determine if a version of the record is visible .

• If the accessed version of trx_id Property value and ReadView Medium creator_trx_id Same value , It means that the current transaction is accessing its own modified records , So this version can be accessed by the current transaction .
• If the accessed version of trx_id Attribute value less than ReadView Medium up_limit_id value , Indicates that the transaction generating this version generates ReadView Submitted before , So this version can be accessed by the current transaction .
• If the accessed version of trx_id Property value is greater than or equal to ReadView Medium low_limit_id value , Indicates that the transaction generating this version generates ReadView It's only opened after , So this version cannot be accessed by the current transaction .
• If the accessed version of trx_id The attribute value is ReadView Of up_limit_id and low_limit_id Between , Then we need to judge trx_id Is the attribute value in trx_ids In the list .
  If in , Description creation ReadView The transaction that generated this version is still active , This version is not accessible .
  If not , Description creation ReadView The transaction that generated this version has been committed , This version can be accessed .

4.4 MVCC The overall operation process

After understanding these concepts , Let's take a look at when querying a record , How the system passes through MVCC Find it ：
1. First get the version number of the transaction itself , That's business ID;
2． obtain ReadView;
3． The data from the query , Then with ReadView Compare the transaction version number in the ;
4. If it doesn't meet Readview The rules , You need from Undo Log Take a historical snapshot in ;
5． Finally, the data that meets the rules is returned .

If a version of the data is not visible to the current transaction , Then follow the version chain to find the data of the next version , Continue to follow the steps above to determine visibility , And so on , Until the last version in the version chain . If the last version is not visible , This means that the record is completely invisible to the transaction , The query result does not contain the record .

InnoDB in ,MVCC It's through Undo Log + Read View Read the data ,Undo Log Saved a historical snapshot , and Read View Rules help us determine whether the current version of the data is visible .

At the isolation level, read committed （Read Committed） when , Every time in a transaction select The query will be retrieved again Read View.

When the isolation level is readable , You can't avoid repetition , This is because a transaction is only the first time select You'll get it once Read View, And all the rest select Will reuse this Read View, As shown below ：

5. Illustrate with examples

Suppose now student Only one item in the table consists of Business id by 8 Insert a record for the transaction of ：

MVCC Only in READ COMMITTED and REPEATABLE READ Working at two isolation levels . Let's take a look READ COMMITTED and REPEATABLE READ The so-called generation Readview What is the difference between the different timing of .

5.1 READ COMMITTED Under isolation level ：
READ COMMITTED： Generate a... Before each read ReadView
Now there are two transactions id Respectively 10、20 Your transaction is executing ：

explain : During transaction execution , Only the first time a record is actually modified （ For example, use INSERT、DELETE、UPDATE sentence ), Will be assigned a separate transaction id, This business id Is increasing . That's why we're doing business 2 Update the records of some other tables in , The purpose is to let it allocate transactions id.

At the moment , surface student in id by 1 The linked list of versions obtained from the records is as follows ：

Let's say there's a use now READ COMMITED Isolation level transactions start executing ：

This SELECT1 The implementation process is as follows :

step 1∶ In execution SELECT I'll make a speech ReadView ,ReadView Of trx_ids The content of the list is [10,20],up_limit_id by 10, low_limit_id by 21 , creator_trx_id by 0.
step 2: Pick visible records from the version chain , As you can see from the picture , The latest version of columns name The content is ' Wang Wu ', The version trx_id The value is 10, stay trx_ids In the list , So it doesn't meet the visibility requirements , according to roll_pointer Jump to the next version .
step 3: The next version of the column name The content is ' Li Si ', The version trx_id Value for 10, Also in the trx_ids In the list , So it doesn't meet the requirements , Continue to the next version .
step 4: The next version of the column name The content is ‘ Zhang San ', The version trx_id The value is 8, Less than ReadView Medium up_limit_id value 10, So this version meets the requirements , The last version returned to the user is this column name by ‘ Zhang San ’ The record of .

after , Let's do business id by 10 I'd like to submit my business :

And then to Business id by 20 Update the table in the transaction student in id by 1 The record of :

At the moment , surface student in id by 1 The version chain of records is so long ：

And then use it just now READ COMMITTED Continue to find... In isolation level transactions id by 1 The record of , as follows ：

This SELECT2 The implementation process is as follows :

step 1∶ In execution SELECT The statement will generate another ReadView, The ReadView Of trx_ids The content of the list is [20],up_limit_id by 20,low_limit_id by 21, creator_trx_id by 0.
step 2: Pick visible records from the version chain , As you can see from the picture , The latest version of columns name The content is ‘ Song Ba ’, The version tr×_id The value is 20, stay trx_ids In the list , So it doesn't meet the visibility requirements , according to roll.pointer Jump to the next version .
step 3∶ The next version of the column name The content is ’ Qian Qi ’, The version trx_id The value is 20, Also in the trx_ids In the list , So it doesn't meet the requirements , Continue to the next version .
step 4∶ The next version of the column name The content is ’ Wang Wu ’, The version trx_id The value is 10, Less than ReadView Medium up_limit.id value 20, So this version meets the requirements , The last version returned to the user is this column name by ’ Wang Wu ’ The record of .
And so on , If later transactions id by 20 My records were also submitted , Use again READ CONMMITTED Query table in isolation level transaction student in id The value is 1 When recording , The result is ‘ Song Ba ’ 了 , We won't analyze the specific process .

5.2 REPEATABLE READ Under isolation level
Use REPEATABLE READ For isolation level transactions , Only one... Is generated when the query statement is executed for the first time ReadView, After that, the query will not be generated repeatedly .

such as , There are two transactions in the system id Respectively 10、20 Your transaction is executing :

At the moment , surface student in id by 1 The linked list of versions obtained from the records is as follows :

Let's say there's a use now REPEATABLE READ Isolation level transactions start executing :

At this point, the execution process is the same as read committed identical

And then use it just now REPEATABLE READ Continue to find... In isolation level transactions id by 1 The record of , as follows ：

This SELECT2 The implementation process is as follows :

step 1: Because the isolation level of the current transaction is REPEATABLE READ, And before the execution SELECT1 Has been generated by ReadView 了 , So at this time, directly reuse the previous ReadView, Previous ReadView Of trx_ids The content of the list is [10,20],up_limit_id by 10, low_limit_id by 21 , creator_trx_id by 0.
step 2: Then select the visible records from the version chain , As you can see from the diagram , The latest version of columns name The content is ’ Song Ba ’trx_id The value is 20, stay trx_ids In the list , So it doesn't meet the visibility requirements , according to roll_pointer Jump to the next version .
step 3:∶ The next version of the column name The content is ’ Qian Qi ’, The version trx_id The value is 20, Also in the trx_ids Requirements in the list , Continue to the next version .
step 4: The next version of the column name The content is ’ Wang Wu ’, The version trx_id The value is 10, and trx_ids The list contains values of 10 The business of id Of , So this version does not meet the requirements , Similarly, the next column name The content is ’ Li Si ’ The version of does not meet the requirements . Continue to skip to the next version .
Step by step 5∶ The next version of the column name The content is ’ Zhang San ’, The version trx_id The value is 80, Less than Readview Medium up_limit_id value 10, So this version meets the requirements , The last version returned to the user is this column c by ‘ Zhang San ’ The record of .
two SELECT The results of the query are duplicate , Columns of records c Values are ’ Zhang San ’, This is what repeatable means . If we do this later id by 20 Your records were submitted , And then use it just now REPEATABLE READ Continue to find this in the transaction of isolation level brush id by 1 The record of , The result is still ’ Zhang San ’, You can analyze the specific implementation process yourself .

5.3 How to solve unreal reading
Let's say that now the table student There's only one piece of data , In the data content , Primary key id=1, Hidden trx_id=10, its undo log As shown in the figure below .

Suppose there is a transaction now A And transaction B Concurrent execution , Business A The business of id by 20, Business B The business of id by 30.
step 1: Business A Start querying data for the first time , Of the query SQL The statement is as follows .

select * from student where id > 1;

Before starting the query ,MySQL Will be for business A Produce a ReadView, here ReadView Is as follows : trx_ids=[20, 30 ] ,up_limit_id=20 , low_limit_id=31 , creator_trx_id=20.

At this time, due to student There's only one piece of data , And consistent with where id>=1 Conditions , So it will find out . And then according to ReadView Mechanism , Find the data in this row trx_id=10, Less than transaction A Of ReadView in up_limit_id, This means that this data is a transaction A Before opening , Data already submitted by other transactions , So the business A It can be read .

Conclusion ： Business A Your first query , Can read a piece of data ,id=1.

step 2∶ Then business B(trx_id=30), To the table student Insert two new pieces of data , And commit the transaction .

insert into student(id,name) values(2,' Li Si ');
insert into student(id,name) values(3,' Wang Wu ');

At this point, the table student There are three pieces of data in , Corresponding undo As shown in the figure below ：

step 3∶ Then business A Open the second query , According to the rules of repeatable read isolation level , At this point, the transaction A It won't regenerate ReadView. At this point, the table student Medium 3 All data meet where id>=1 Conditions , So we'll find out first . And then according to ReadView Mechanism , Determine whether each piece of data can be transacted A notice .
1） First id=1 This data of , I've already said that , Can be used by transactions A notice .
2） And then there was id=2 The data of , its trx_id=30, At this point, the transaction A Find out , This value is in up_limit_id and low_limit_id Between , So we need to judge again 30 Is it in trx_ids In array . Due to transaction A Of trx_ids=[20,30], So in the array , This means id=2 This data is related to the transaction A Committed by other transactions started at the same time , So this data cannot make transactions A notice .
3) Empathy ,id=3 This data of , trx_id Also for the 30, Therefore, it cannot be used by transactions A See .

Conclusion : Final transaction A The second query , Only query id=1 This data of . This and business A The results of the first query are the same , Therefore, there is no illusory reading phenomenon , So in the MySQL Under the repeatable continuous isolation level , There is no magic reading problem .

6. summary

Here is the introduction of MVCC stay READ COMNNITTD、REPEATABLE READ These two isolation level transactions access the version chain of records when performing snapshot read operations . This makes different transactions read - Write 、 Write - read Operations are executed concurrently , So as to improve the system performance .

The core point is ReadView Principle ,READ CONMITTD、REPEATABLE READ One big difference between these two isolation levels is generation ReadView The timing is different :

• READ COMMITTD In every ordinary SELECT Before operation, a ReadView
• REPEATABLE READ Only for the first time SELECT Generate a before operation ReadView, This is repeated in subsequent query operations Readview Just fine .

adopt MVCC We can solve :
1. The problem of blocking between reading and writing . adopt MVcc It can make reading and writing not block each other , That is, reading does not block writing , Writing doesn't block reading , In this way, the ability of concurrent transaction processing can be improved .
2. Reduced the probability of deadlock . This is because MVCC Adopted the optimistic lock way , There is no need to lock when reading data , For write operations , Lock only the necessary lines .
3. Solve the problem of snapshot reading . When we query a snapshot of a database at a certain point in time , You can only see the result of the transaction commit update before this point in time , You can't see the update result of transaction commit after this point in time .