06 _ Global lock and table lock: Why are there so many obstacles to adding a field to a table?

I want to talk to you today MySQL Lock of . The original intention of database lock design is to deal with concurrent problems . Shared resources as multiple users , When concurrent access occurs , Database needs to control access rules of resources reasonably . Lock is an important data structure to implement these access rules .

According to the range of locking ,MySQL The locks inside can be roughly divided into global locks 、 Table level lock and row lock . Today's article , I will share global locks and table level locks with you . And about row locks , I'll keep it for you in the next article .

Here's the thing to note , The design of the lock is more complicated , These two articles will not cover the specific implementation details of locks , This paper mainly introduces the phenomenon when encountering lock and the principle behind it .

Global lock

seeing the name of a thing one thinks of its function , Global lock is to lock the whole database instance .MySQL It provides a way to add global read lock , The order is Flush tables with read lock (FTWRL). When you need to make the entire library read-only , You can use this command , After that, the following statements of other threads will be blocked ： Data update statement （ Data addition, deletion and modification ）、 Data definition statement （ Including building tables 、 Modify table structure, etc ） Commit statements for and update class transactions .

A typical use scenario for global locks is , Make a full library logical backup . That is to say, every table in the whole library select Come out and save it as text .

There used to be a way , It's through FTWRL Make sure that no other threads update the database , Then back up the entire library . Be careful , During the backup process, the entire library is completely read-only .

But make the entire library read-only , It sounds dangerous ：

• If you back up on the main database , The update cannot be performed during the backup , Business basically has to stop ;
• If you are backing up from a library , During the backup period, the master database cannot be synchronized from the slave database binlog, Will cause master-slave delay .

It seems that it's not good to add a global lock . But think about it , Why is backup locked ？ Let's take a look at what happens when you don't lock it .

Suppose you want to maintain “ Geek time ” The purchase system of , Focus on the user account balance table and user curriculum .

Now initiate a logical backup . Assuming the backup period , There's a user , He bought a course , His balance will be deducted from the business logic , Then add a course to the purchased course .

If the time sequence is to back up the account balance table first (u_account), And then the user buys , Then back up the user curriculum (u_course), What will happen ？ You can take a look at this picture ：

chart 1 Business and backup state diagram

You can see , In this backup result , user A The data status of is “ The account balance is not deducted , But there is one more course in the user's curriculum ”. If you use this backup to recover data later , user A Found , I earned .

As a user, don't think it's so nice , You can imagine ： If the order of the backup tables is reversed , First back up the user curriculum and then the account balance table , And what could happen ？

in other words , If you don't lock it , The library that the backup system backs up is not a logical point in time , This view is logically inconsistent .

When it comes to views, you must remember , When we talked about transaction isolation , In fact, there is a way to get a consistent view , Right ？

Yes , This is to start a transaction at the repeatable read isolation level .

The official logic backup tool is mysqldump. When mysqldump Using parameter –single-transaction When , A transaction will be started before importing data , To ensure a consistent view . And because the MVCC Support for , During this process, the data can be updated normally .

You must be wondering , With this function , Why do we still need FTWRL Well ？ Consistent reading is good , But only if the engine supports this isolation level . such as , about MyISAM This engine does not support transactions , If there is an update during the backup , Always only get the latest data , Then it destroys the consistency of the backup . At this time , We need to use it FTWRL The command .

therefore ,single-transaction Method only applies to all tables using the transaction engine's Library . If some tables use an engine that does not support transactions , Then backup can only be through FTWRL Method . This is often DBA Ask business developers to use InnoDB replace MyISAM One of the reasons .

You may ask , Since the whole library is read-only , Why not use set global readonly=true What is the way? ？ exactly readonly Mode can also make the entire library read-only , But I'll suggest you use FTWRL The way , There are two main reasons ：

• One is , In some systems ,readonly The value of will be used to do other logic , For example, it is used to determine whether a database is a primary or a standby database . therefore , modify global The way the variables affect the surface is larger , I don't recommend that you use .
• Two is , There are differences in exception handling mechanisms . If you execute FTWRL After the command, due to the abnormal disconnection of the client , that MySQL This global lock will be released automatically , The whole library can be updated normally . Instead, set the entire library to readonly after , If the client has an exception , Then the database will remain readonly state , This will cause the entire library to be in a non writable state for a long time , High risk .

Business update is not just about adding, deleting and changing data （DML), It is also possible to add fields and other operations to modify the table structure （DDL）. Either way , When a library is locked globally , You need to add fields to any of the tables , It's all locked up .

however , Even if it is not locked by the whole situation , Adding fields doesn't make it easy , Because you will encounter the table level lock that we will introduce next .

Table lock

MySQL There are two types of lock at the inner table level ： One is watch lock , One is metadata lock （meta data lock,MDL).

The syntax of table lock is lock tables … read/write. And FTWRL similar , It can be used unlock tables Active release lock , It can also be released automatically when the client is disconnected . We need to pay attention to ,lock tables The syntax will restrict the reading and writing of other threads , It also defines the next operation objects of this thread .

for instance , If in a thread A In the implementation of lock tables t1 read, t2 write; This statement , Other threads write t1、 Reading and writing t2 All of the statements will be blocked . meanwhile , Threads A In execution unlock tables Before , It can only be read t1、 Reading and writing t2 The operation of . Linking t1 Not allowed , Naturally, you can't access other tables .

Before a finer grained lock appears , Table locking is the most common way to handle concurrency . And for InnoDB This kind of engine supports row lock , Generally not used lock tables Command to control concurrency , After all, the effect of locking the whole watch is still too great .

Another type of table level lock is MDL（metadata lock).MDL You don't need to explicitly use , When accessing a table, it will be automatically added .MDL The role of is , Ensure the correctness of reading and writing . You can imagine , If a query is traversing data in a table , During execution, another thread changes the table structure , Delete a list , Then the result obtained by the query thread does not match the table structure , Surely not .

therefore , stay MySQL 5.5 The version introduces MDL, When adding, deleting, modifying and querying a table , Add MDL Read the lock ; When you want to make structural changes to a table , Add MDL Write lock .

• Read locks are not mutually exclusive , So you can have multiple threads to add, delete, modify and query a table at the same time .

• Read-write lock 、 Write locks are mutually exclusive , To ensure the security of the operation to change the structure of the table . therefore , If two threads want to add fields to a table at the same time , One of them can't be executed until the other has finished .

although MDL Locks are added by default , But it's a mechanism you can't ignore . Here's an example , I often see people fall into this pit ： Add a field to a small table , Cause the whole library to hang .

You must know , Add fields to a table , Or modify the fields , Or the Caucasian , Need to scan the data of the whole table . When working on a large watch , You must be very careful , In order to avoid the impact on online services . But in fact , Even a small watch , Careless operation can also cause problems . Let's take a look at the following sequence of operations , Hypothesis table t It's a little watch .

remarks ： The experimental environment here is MySQL 5.6.

We can see session A Start... First , I'll check my watch t Add one more MDL Read the lock . because session B What is needed is MDL Read the lock , So it can be executed normally .

after session C Will be blocked, Because session A Of MDL The lock has not been released , and session C need MDL Write lock , So it can only be blocked .

If only session C It doesn't matter that I'm blocked , But after that, it's all on the watch t New application MDL The request to read the lock will also be session C Blocking . We said that before , All operations of adding, deleting, modifying and querying tables need to be applied for first MDL Read the lock , They're all locked , It means that it's completely unreadable .

If the query statements on a table are frequent , And the client has a retry mechanism , That is to say, there will be a new one after the timeout session If you ask again , The threads in this library will soon be full .

You should know by now , In the transaction MDL lock , Apply at the beginning of statement execution , However, the statement is not released immediately after completion , It will wait until the whole transaction is committed before releasing .

Based on the above analysis , Let's talk about a problem , How to add fields to a small table safely ？

First of all, we have to deal with long affairs , The transaction does not commit , Will always occupy MDL lock . stay MySQL Of information_schema Library innodb_trx In the table , You can find the current transaction in progress . If you want to do DDL The changed table happens to have a long transaction running , Consider suspending DDL, perhaps kill Drop this long business .

But think about this scenario . If the table you want to change is a hotspot table , Although the amount of data is small , But the requests are frequent , And you have to add a field , What should you do ？

Now kill It may not work , Because new requests are coming soon . The ideal mechanism is , stay alter table Set the waiting time in the statement , If you can get it within the specified waiting time MDL It's better to write lock , Don't block the following business statements if you can't get them , Give up first . Then the developer or DBA Repeat the process by retrying the command .

MariaDB Has merged AliSQL This function of , So these two open source branches currently support DDL NOWAIT/WAIT n This grammar .

ALTER TABLE tbl_name NOWAIT add column ...
ALTER TABLE tbl_name WAIT N add column ... 

Summary

today , I introduced you to MySQL Global locks and table level locks .

Global lock is mainly used in the process of logical backup . For all of them InnoDB Engine library , I suggest you choose to use –single-transaction Parameters , It's more app friendly .

Table locks are generally used when the database engine does not support row locks . If you find that your app has lock tables Such a statement , You need to trace it , The more likely scenario is ：

• Either your system is still working MyISAM This type of engine does not support transactions , Then we should arrange to upgrade and change the engine ;
• Either your engine has been upgraded , But the code hasn't been upgraded . I've seen this happen , Finally, business development is to put lock tables and unlock tables Change to begin and commit, The problem is solved .

MDL Will not be released until the transaction is committed , When making table structure changes , You must be careful not to lock up online queries and updates .