MySQL 45 lecture - learn the actual combat notes of MySQL in Geek time 45 lecture - 06 | global lock and table lock_ Why are there so many obstacles in adding a field to the table

06 | Global lock and table lock _ How can adding a field to a table be so much of a hindrance

This article is about MySQL actual combat 45 speak (geekbang.org) Learning notes of

Please support the author greatly , That's great ~

Here are just Xiao Fu's notes on the article ~

You can also learn five bars for free ~

MySQL lock

The original intention of database lock design is to deal with concurrency as a resource shared by multiple users , When concurrent access occurs , The database needs to reasonably control the access rules of resources . Lock is an important data structure to implement these access rules .

According to the range of locking ,MySQL The locks inside can be roughly divided into global locks 、 Table level lock and row lock .

Today's article , I will share with you Global locks and table level locks .

Global lock

What is a global lock ？

Global lock is to lock the whole database instance .

How to lock globally ？

MySQL It provides a way to add global read lock , The order is Flush tables with read lock (FTWRL). When you need When the entire library is read-only , You can use this command , After that, the following statements of other threads will be blocked ： Data update statement DML（ Data addition, deletion and modification ）、 Data definition statement DDL（ Including building tables 、 Modify table structure, etc ） Commit statements for and update class transactions .

When will global locks be used ？

A typical use scenario for global locks is —— Make a full library logical backup .

That is to put every table in the whole library select Come out and save it as text .

There used to be a way , It's through FTWRL Make sure that no other threads update the database , Then back up the entire library . Be careful , During the backup process, the entire library is completely read-only .

The harm of global lock

1️⃣ If you are in the Main library Perform global lock backup on , Then updates cannot be performed during backup , Business basically has to stop .

2️⃣ If you are in the Slave Library Perform global lock backup on , During the backup period, the master database cannot be synchronized from the slave database binlog（ Archive log ） Will cause master-slave delay .

Why should backup be locked ？ What's wrong with not locking ？

Take up a

Suppose you want to maintain “ Geek time ” The purchase system of , Focus on the user account balance table and user curriculum .

Now initiate a logical backup . Assuming the backup period , There's a user , He bought a course , His balance will be deducted from the business logic , Then add a course to the purchased course .

If the time sequence is to back up the account balance table first (u_account), And then the user buys , Then back up the user curriculum (u_course), What will happen ？ You can take a look at this picture ：

You can see , In this backup result , user A The data status of is “ The account balance is not deducted , however There is already one more course in the user's curriculum ”. If you use this backup to recover data later , user A Found , I earned .

Of course, everything has opposites , If you back up the user curriculum first （u_course） Then you will find it miraculously —— The money is deducted , Classes are not included in your schedule .

in other words , If you don't lock it , The library that the backup system backs up is not a logical point in time , This Views are logically inconsistent .

At this point, the data is inconsistent .

When it comes to views, you must remember , When we talked about transaction isolation , In fact, there is a way to get a consistent view , Right ？

Yes , Is in the Repeatable read isolation level （RR） Next open a transaction .

The official logic backup tool is mysqldump. When mysqldump Using parameter –single-transaction When , A transaction will be started before importing data , Come on Make sure you get a consistent view . and because MVCC Support for , During this process, the data can be updated normally .

interviewer ： substitutions , How can he even MVCC I don't know the relationship with the transaction isolation level ？

At this time, you will have doubts —— With MVCC Support for , Why do we still need FTWRL Well ？

Consistent reading is good , But only if the engine supports this isolation level

such as , about MyISAM This engine does not support transactions , If there is an update during the backup , Always only get the latest data , that It destroys the consistency of backup . At this time , We need to use it FTWRL The command .

Be careful ：

1️⃣ single-transaction Method only applies to all tables using the transaction engine's Library

2️⃣ If Some tables use engines that do not support transactions , that Backup can only be done through FTWRL Method . This is often DBA Require business developers Use InnoDB replace MyISAM One of the reasons .

Since the whole library is read-only , Why not use set global readonly=true What is the way? ？

exactly readonly Mode can also make the entire library read-only , But I'll suggest you use FTWRL The way , There are two main reasons ：

1️⃣ In some systems ,readonly The value of will be used to do other logic , For example, it is used to determine whether a database is a primary or a standby database . therefore , modify global The way the variables affect the surface is larger , I don't recommend that you use .

2️⃣ There are differences in exception handling mechanisms . If you execute FTWRL After the command, due to the abnormal disconnection of the client , that MySQL It will be released automatically This global lock , The whole library Return to the status that can be updated normally . And will be The entire library is set to readonly after , If the client has an exception , Then the database will remain readonly state , This will cause the entire library to be in a non writable state for a long time , High risk .

Business update is not just about adding, deleting and changing data （DML), It is also possible to add fields and other operations to modify the table structure （DDL）. Either way , When a library is locked globally , You need to add fields to any of the tables , It's all locked up .

Table lock

MySQL There are two types of lock at the inner table level ： One is watch lock , One is metadata lock （meta data lock,MDL).

Table locks

The syntax of table lock is lock tables … read/write.

And FTWRL similar , It can be used unlock tables Active release lock , It can also be released automatically when the client is disconnected .

Be careful ：

1️⃣ lock tables The syntax will restrict the reading and writing of other threads , It also defines the next operation objects of this thread .

Take up a

If in a thread A In the implementation of lock tables t1 read, t2 write; This statement , Other threads write t1、 Reading and writing t2 All of the statements will be blocked .

meanwhile , Threads A In execution unlock tables Before , It can only be read t1、 Reading and writing t2 The operation of . Linking t1 Not allowed , Naturally, you can't access other tables

Summary ： Writing is an exclusive lock , A write lock means that other threads cannot read or write . Read lock is a shared lock , After adding, other locks can only be read but not written , This thread cannot write

Before a finer grained lock appears , Table locking is the most common way to handle concurrency . and about InnoDB This kind of engine supports row lock , Generally not used lock tables Command to control concurrency , After all, the effect of locking the whole watch is still too great .

MDL（ Metadata lock ）

MDL You don't need to explicitly use , When accessing a table, it will be automatically added .MDL The role of is , Ensure the correctness of reading and writing . You can imagine , If a query is traversing data in a table , During execution, another thread changes the table structure , Delete a list , Then the result obtained by the query thread does not match the table structure , Surely not .

• stay MySQL 5.5 The version introduces MDL, When When adding, deleting, modifying, and querying a table , Add MDL Read the lock ; Be right When the table changes its structure , Add MDL Write lock .

1️⃣ Read locks are not mutually exclusive , So you Multiple threads can add, delete, modify and query a table at the same time .

2️⃣ Read-write lock 、 Write locks are mutually exclusive , To ensure the security of the operation to change the structure of the table . therefore , If there is Two threads should add fields to a table at the same time , One of them can't be executed until the other has finished .

although MDL Locks are added by default , But it's a mechanism you can't ignore . Here's an example , I often see people fall into this pit ： Add a field to a small table , Cause the whole library to hang .

Take up a

Add fields to a table , Or modify the fields , Or the Caucasian , Need to scan the data of the whole table .

The index should be created according to the record value of each row in the table , So you need a full table scan ; Add fields or modify fields , Also modify the data of the corresponding column in each row of records , So we also need to scan the whole table

** Be careful ：** actually , Even a small watch , Careless operation can also cause problems . Let's take a look at the following sequence of operations , Hypothesis table t It's a little watch .

The normal process is as follows ：

mysql MDL The read-write lock is blocked , as well as online ddl Caused by the “ Jump the queue ” The phenomenon _

A( Read the lock ) - > B( Read the lock ) - > C( Apply for write lock ( Write lock wait ) block) - > D( Apply for read lock , Because write lock wait , Blocking ) - > A,B(Commit) - > C( Get the write lock ) - > D( Apply for read lock block) - > C(ddl online Write lock degenerates into read lock ) - > D( Get read lock ) - > C( Upgrade write lock block By D) - > D(commit, Read lock release ) - > C( complete ddl , Release the write lock )

ps: apply MDL Lock operations form a queue , Write lock application Priority over Read the lock

1️⃣ We can see session A Start... First , I'll check my watch t Add one more MDL Read the lock . because session B What is needed is MDL Read the lock , So it can be executed normally .

2️⃣ after session C Will be blocked, Because session A Of MDL The lock has not been released , and session C need MDL Write lock , So it can only be blocked .

3️⃣ If only session C It doesn't matter that I'm blocked , But after that, it's all on the watch t New application MDL The request to read the lock will also be session C Blocking . We said that before , All operations of adding, deleting, modifying and querying tables need to be applied for first MDL Read the lock , They're all locked , It means that it's completely unreadable .

4️⃣ If the query statements on a table are frequent , And the client has a retry mechanism , That is to say, there will be a new one after the timeout session If you ask again , The threads in this library will soon be full . You should know by now , In the transaction MDL lock , Apply at the beginning of statement execution , However, the statement is not released immediately after completion , It will wait until the whole transaction is committed before releasing .

How to add fields to a small table safely ？

1️⃣ First of all, we have to deal with long affairs , The transaction does not commit , Will always occupy MDL lock . stay MySQL Of information_schema Library innodb_trx In the table , You can find the current transaction in progress . If you want to do DDL The changed table happens to have a long transaction running , Consider suspending DDL, perhaps kill Drop this long business .

If the table you want to change is a hotspot table , Although the amount of data is small , But the requests are frequent , And you have to add a field , What should you do ？

stay alter table Statement inside Set the waiting time , If ** Within the specified waiting time, you can get MDL It's better to write lock , Don't block the following business statements if you can't get them , Give up first .** Then the developer or DBA Repeat the process by retrying the command .

ALTER TABLE tbl_name NOWAIT add column ...
ALTER TABLE tbl_name WAIT N add column ... 

Summary

1️⃣ Global lock is mainly used in the process of logical backup . For all of them InnoDB Engine library , I suggest you choose to use –single-transaction Parameters , It's more app friendly .

2️⃣ ** Table locks are generally used when the database engine does not support row locks .** If you find that your app has lock tables Such a statement , You need to trace it , The more likely scenario is ：

• Either your system is still working MyISAM This type of engine does not support transactions , Then we should arrange to upgrade and change the engine ;

• Either your engine has been upgraded , But the code hasn't been upgraded . I've seen this happen , Finally, business development is to put lock tables and unlock tables Change to begin and commit, The problem is solved .

Be careful ：

1️⃣ The scenarios that originally need table locks can be directly replaced by transactions , Because transactions are automatically added MDL lock ,MDL Locks can distinguish between the modification of table structure and ordinary addition, deletion, modification and query , Finer particle size .

After class questions

Backup is usually performed on the standby database , You are using –single-transaction Method in the process of logical backup , If a small table on the main database makes a DDL, Like adding a column to a table . Now , What can be seen from the standby database ？

1️⃣ situation 1： master On the small table t1 Of DDL Transferred to the slave Time to apply ,mysqldump The backup is finished t1 The data table , here slave Synchronization ok , There will be no problem .

2️⃣ situation 2：master On the small table t1 Of DDL Transferred to the slave Time to apply ,mysqldump Backup now t1 The data table , This happens MDL lock , From the library t1 All operations of the table will block . Master-slave delay , until Q6 Execution completed .

3️⃣ situation 3： master On the small table t1 Of DDL Transferred to the slave Time to apply ,mysqldump Not yet t1 Tables are backed up , The DDL Will be in slave Of t1 Table applied successfully , But when exporting to t1 When I watch, I will report “ERROR 1412 (HY000): Table definition has changed, please retry transaction” error , Causes the export to fail ！

This section summarizes

summary ： According to the lock range ：MySQL The lock inside can be divided into ： Global lock 、 Table lock 、 Row-level locks 、

---

One 、 Global lock ：

Lock the whole database instance .

MySQL Provides a way to add a global read lock ：Flush tables with read lock(FTWRL) This command makes the entire library read-only . After using the command , Data update statement 、 Data definition statements and commit statements of update class transactions are blocked .

Use scenarios ： Full library logical backup .

risk ：

1. If it is backed up in the main database , Cannot update during backup , Business suspension

2. If you are backing up from the database , Cannot perform primary database synchronization during backup binlog, Cause master-slave delay

The official logic backup tool mysqldump, When mysqldump Using parameter –single-transaction When , Will start a transaction , Make sure you get a consistent view . And because the MVCC Support for , During this process, the data can be updated normally .

Consistent reading is good , But only if the engine supports this isolation level .

If you want the full library to be read-only , Why not use set global readonly=true The way ？

1. In some systems ,readonly The value of will be used to do other logic , For example, judge the primary and standby databases . So modify global The way variables affect too much .

2. There are differences in exception handling mechanisms . If you execute FTWRL After the command, due to the abnormal disconnection of the client , that MySQL This global lock will be released automatically , The whole library can be updated normally . Instead, set the entire library to readonly after , If the client has an exception , Then the database will remain readonly state , This will cause the entire library to be in a non writable state for a long time , High risk .

---

Two 、 Table lock

MySQL There are two kinds of table lock , One is watch lock , One is the metadata Institute (meta data lock,MDL)

The syntax of table lock is :lock tables … read/write

It can be used unlock tables Active release lock , It can also be released automatically when the client is disconnected .

lock tables The syntax will restrict the reading and writing of other threads , It also defines the next operation objects of this thread .

about InnoDB This kind of engine supports row lock , Generally not used lock tables Command to control concurrency , After all, the effect of locking the whole watch is still too great .

MDL： You don't need to explicitly use , When accessing a table, it will be automatically added .

MDL The role of ： Ensure the correctness of reading and writing .

When adding, deleting, modifying and querying a table , Add MDL Read the lock ; When you want to make structural changes to a table , Add MDL Write lock .

Read locks are not mutually exclusive . Read-write lock , Write locks are mutually exclusive , To ensure the security of the operation to change the structure of the table .

MDL Will not be released until the transaction is committed , When making table structure changes , Be careful not to lock online queries and updates .

---