MySQL transaction

MySQL Business

MySQL Transaction mechanism is mainly used to deal with large amount of operations 、 Data with high complexity

• stay MySQL Only use Innodb Only the data tables and databases of the database engine support transactions

• Transaction processing can be used to maintain data integrity , Guarantee multiple SQL Statement or all execution , Or none at all

• Transactions for management insert、update and delete And so on. DML sentence ,[select sentence ], Other types of SQL Statement has no concept of transaction

Overview transactions

The transaction must satisfy ACID4 Conditions ：A Atomicity 、C Uniformity 、I Isolation, 、D persistence

• Atomicity ： All operations in a transaction are either completed 、 Or don't do any , It doesn't end in the middle

• Uniformity ： After the transaction is completed, the integrity of the database is not damaged

• Isolation, ： The database allows multiple concurrent transactions to read and write data in the database , Isolation can prevent data inconsistency caused by concurrent execution of multiple transactions . According to the isolation level, transactions can be divided into 4 level ： Read uncommitted 、 Read submitted 、 Repeatable reading and serialization

• persistence ： After the transaction is completed, the modification of data is permanent

stay MySQL The command line defaults to , Transactions are automatically committed . If you need to use transactions, you need to explicitly open transactions start transaction perhaps begin, Or execute orders set autocommit=0, Used to disable automatic submission of the current session

Transaction and database underlying data

In the course of the transaction , Before it's over ,DML Statement does not directly change the underlying data , Just record the historical operation , Record in memory . Only at the end of the transaction , And it should be at the end of success , Will modify the data in the underlying hard disk file

• The atomicity of a transaction is through undo log To achieve

• Transaction persistence is achieved by redo log To achieve

• Transaction isolation is through 【 Read-write lock +MVCC Multi version concurrency control 】 To achieve

• The consistency of transactions is through atomicity 、 Persistence and isolation

Transaction control statement

• begin perhaps start transaction You can explicitly open a transaction , End transactions include commit and rollback 2 Ways of planting

• commit Commit transaction , And make all the modifications to the database that have been executed become permanent modifications

• rollback Rollback ends the transaction , Undo the uncommitted modification that has been performed

• savepoint Logo name Used to create a savepoint during a transaction , This supports partial rollback . Multiple savepoints can be added to a transaction

• release savepoint ID name The savepoint used to delete a transaction , If the corresponding name does not exist when it is saved, an exception is thrown

• rollback to ID name Roll back the transaction to the specified savepoint , Perform all operations from the savepoint of the name to the current location, and undo , However, the operation before the savepoint is still retained , Wait for the transaction to end

• set transaction isolation level Used to set transaction isolation ,innodb The isolation provided by the storage engine is read uncommitted read uncommitted、 Read submitted read committed、 Repeatable repeatable read and serializable Serialization , The default isolation level of the system is repeatable

Transaction processing

begin Open transaction rollback Transaction rollback commit Transaction submission

You can also use set change MySQL The auto submit mode of

• set autocommit=0 Disable auto submit

• set autocommit=1 Turn on auto submit

Basic test

stay navicat Or open two windows in the command line to simulate two processes that modify the database concurrently

1、 Create database create database test1;

2、 Switch the current library use test1;

3、 Create a table for the test create table t1(id int primary key,name varchar(32))

engine=innodb;

MySQL8 The storage engine of the default database is innodb

4、 Open transaction ： begin;

5、 The insert ：insert into t1 values(1,'zhangsan');, Execute the query in another window , Then you can't see the inserted data , Because the default isolation level of transactions is repeatable

6、 Commit transaction commit; The amendment takes effect , Data can be queried in another window

7、 If not submitted, you can also use rollback Undo the modification

Multipoint rollback

begin;

update t1 set name='modify1' where id=1;
select * from t1; 

savepoint s1; 

delete from t1; 

rollback to s1; 
select * from t1;

commit;

Related log problems

• The atomicity of a transaction is through undo log To achieve

• Transaction persistence is achieved by redo log To achieve

redo log

If you need a disk every time you read and write data IO, It's going to be inefficient .innodb Provides caching buffer pool As a cache for accessing the database , Both read and modify operations will involve cache operations , The cache is periodically flushed to disk , However, the data written to the cache will be lost when the system goes down , The persistence of transactions cannot be guaranteed . Every time I read and write hard disk data IO The cost is too high , To solve this problem , Introduced redo log To improve the execution efficiency of updating data .

When the transaction commits , First the redo log buffer write in redo log File persistence , To be done commit Only when the operation is finished is it finished . This behavior becomes pre log persistence write-ahead log. Before persisting a data page , First, persist the corresponding log page in memory . When a piece of data needs to be updated ,innodb Records will be written to redo log in , And update memory , The update is complete .innodb At the right time , For example, when the system is idle , To really update the operation record to disk . If the system goes down before the data is dropped , After the database restarts , The integrity of data can be guaranteed through logs .

undo log

undo log Provides two functions ： Provides rollback and multi version control MVCC

When data is modified, not only redo, The corresponding undo.undo log The main record is the logical change of data , In order to roll back all previous operations when an error occurs .

undo Logs are used to restore the database logic to its original state , So the actual recording is the opposite work . for example insert The corresponding is delete.undo Logs are used for rollback operations of transactions , Thus ensuring the atomicity of the transaction

Isolation level

The important function of database is to realize data sharing , For multiple transactions running simultaneously , When multiple transactions access the same data in the database at the same time , If necessary isolation mechanism is not adopted , It will lead to various concurrency problems .

The essence of the problem is the thread safety problem of sharing data

common problem

1、 The first type of lost updates ：A When the transaction is rolled back, the committed B The data updated by the transaction is over written . The solution is a lock mechanism

2、 Dirty reading ：A Transaction read to B Data updated by transactions but not yet provided , If B Rollback undo , be A The data read is temporary and invalid data .

3、 It can't be read repeatedly ：A The transaction read a field value , however B Update and submit the modification of this field ,A Read the same field value again , But the contents read twice are inconsistent

4、 Fantasy reading ：A The transaction reads multiple rows of data from a table , however B The transaction inserts or deletes some new rows , If A Read again , Then it is found that there will be more or less rows in the data

5、 The second kind of lost updates ：A Transaction modification record , meanwhile B Transaction modification record ,B Use... After submitting data B The modified result of overwrites the transaction A Modification results of

Transaction isolation

The database system must have the ability to isolate concurrent transactions , So that they will not affect each other , Avoid all kinds of concurrent problems . The degree to which a transaction is isolated from other transactions becomes the isolation level . Multiple transaction isolation levels are specified in the database . Different isolation levels correspond to different interference levels , The higher the isolation level is , The more consistent the data is , But the worse the concurrency

MySQL Database support 4 Kind of isolation level , The default is repeatable

Range of isolation levels

The scope of isolation level can be divided into global level and session level . The global level is valid for all sessions , Session level is only valid for the current session

Set the global isolation level set global transaction isolation level read committed;

Set session level isolation level set session transaction isolation level read uncommitted;

Summary isolation level

Isolation level	describe
Read-Uncommitted	Allow transactions to read data not committed by other transactions , Dirty reading 、 Both unrepeatable reading and unreal reading problems will occur
Read-Committed	Only transactions are allowed to read data committed by other transactions , Avoid dirty reading , But both non repeatable reading and unreal reading problems will appear
Repeatable-Read	It can ensure that the same data can be read from a field multiple times , It can be considered that the existing data will be snapshot automatically when the transaction is started , Other transaction modifications whether or not committed , Snapshot data read by the current transaction , It can avoid dirty reading and non repeatable reading , But unreal reading problems arise . A snapshot is MVCC Multi version concurrency control
Serializable	It can ensure that the transaction is executed serially , All concurrency problems can be avoided , But due to low performance , Generally not used

In specific application development , It is generally recommended to use the default isolation level of the database management system , At the same time, optimistic lock is introduced into programming

Examples ： Two transactions operate at the same time tb_users In the table age value

Read uncommitted

MySQL In fact, the isolation of transactions in the database is realized by locking mechanism , But locking will bring performance loss . The read uncommitted isolation level is unlocked , So the best performance , But because there are basically no restrictions , So the dirty reading problem can't be solved .

Read submitted

The way to solve the dirty reading problem is to only allow reading the data committed by other transactions , Other transactions have not committed data. The current transaction cannot read . for example oracle The default transaction isolation level is read committed . Because only submitted data can be read , Therefore, the data read twice may be inconsistent

Repeatable

Aiming at the problem of non repeatable reading, the isolation level of repeatable reading is proposed , For the query MVCC Multi version concurrency control introduces snapshot mechanism , Each transaction has its own data snapshot , Even if other transactions commit data , It also does not affect the data snapshot of relevant rows of the current transaction . Unreal reading will still appear

In order to solve the problem of non repetition ,MySQL Adopted MVCC Multi version concurrency control . A row of records in the database actually has multiple versions , In addition to the data in each version , There is also a field identifying the version row trx_id, This field is the corresponding transaction generated id, Apply to the transaction system at the beginning of the transaction , In chronological order

[ Failed to transfer the external chain picture , The origin station may have anti-theft chain mechanism , It is suggested to save the pictures and upload them directly (img-NnIcz5mu-1654399757230)(C:\Users\Administrator\AppData\Roaming\Typora\typora-user-images\image-20220605111216702.png)]

A line of records now has 3 A different version , The transactions that make it happen are recorded in each version id, The backup of each data store is a snapshot 【 Consistency view 】, Rereadable is to generate a global snapshot of the current transaction at the beginning of the transaction , But read commit is to regenerate a snapshot every time a statement is executed .

Rules for reading snapshot data ：

• The version is not submitted and cannot be read

• Version has been submitted , But it was committed after the snapshot was created , Cannot read

• Version has been submitted , It was committed before the snapshot was created , Can be read

The main difference between repeatable read and read committed is the creation of snapshots , Repeatable reads are created only once at the beginning of a transaction , And the read has been committed every time sql Statements are created once

Serialization

The isolation level is the highest , The isolation effect is the best , Can solve dirty reading 、 Nonrepeatability and unreal reading , At that time, the concurrency was the worst . Convert the concurrent execution of transactions to sequential execution , The execution of the latter transaction must wait for the end of the previous transaction

Concurrent write problems

Business A perform update operation ,update Lock the modified data row when , The transaction of this row can be released only after it is committed , And in business A Before submitting , If the transaction B I also hope to modify this line of data , You must apply for row lock first , But because of A The row lock has been occupied , therefore B The application did not arrive , here B Will always be waiting , until A After submitting the release lock ,B To perform

update tb_users set age=10 where id=1;

id It's the primary key PK, It's indexed , that MySQL This row of data was found in the index tree , Then add the uplink lock

update tb_users set age=20 where age=10;

Suppose that the table is not for age catalog index , therefore MySQL This line of data cannot be located directly .MySQL Will give all the rows of this table

Lock , But after adding row locks ,MySQL The filter will be performed again , The lock is released when a row that does not meet the condition is found , In the end, only those who meet

The conditions are right . However, the process of locking and unlocking at one time has a great impact on the performance , If it's a big watch , It is suggested to design the index reasonably

The problem of unreal reading

The solution to the concurrency problem is row locking , Solving unreal reading also depends on the implementation of locking mechanism , Use clearance lock .MySQL Combine row lock and clearance lock together , You can solve the problem of concurrent write and slow read , This lock is called next-key lock

for example ： select * from tb_student Can get age=10 and age=30 The data of , An index database will be created and maintained B+ Trees , Trees can be used to quickly locate row records

For specific row data , for example age=10 and age=30 The data of , Add a row lock , according to age=10 and age=30 The whole interval can be divided into 3 part ,（ Negative infinity ,10]、(10,30) and [30, Positive infinity ) Three parts , Here 3 A gap lock can be added to each section

Business A	Business B
begin	begin
select * from tb_users
update tb_users set name=‘zhangsan’ where age=10
	insert into tb_users values(null,‘lisi’,10);
select * from tb_users where age=10 Solve the phantom reading problem
commit	commit

In the transaction A Before submitting , Business B The insert operation of can only wait , This is actually the clearance lock effect .

• If there is an index in the table , In fact, row locks can be used directly , If it's not an index column , Then the database will apply a gap lock to the entire table .

• MySQL Of innodb The engine can support transactions , Which can be read repeatedly is MySQL Default isolation level

• Read uncommitted and serialized basically do not need to consider the isolation level , Because the read is uncommitted, there is no lock restriction ; Serialization is equivalent to single threaded execution , It's too inefficient

• Read has been submitted to solve the dirty read problem , Row locking solves the problem of concurrent updates , also MySQL Introducing row locks in repeatable reads + The combination of gap lock can realize repeatable reading

Pessimistic lock and optimistic lock

Pessimistic locking ： When data is obtained, it will be locked directly , Shared resources are used by only one thread at a time , Other threads block waiting . Row locks are provided in the database 、 Table lock, etc. , Lock the data before use . For example, ticketing system select * from ticket where id=100 for update

Optimism lock ： The database system does not come with it , Need to develop and implement . Optimistic locking is the operation of data without any special processing , That is, no lock , Conflict judgment is performed only when updating

Add an extra column to the data table ：version Data version number or time stamp timestamp, Each time the data is modified, the version number will be added 1

• update tb_users set age=80,version=2 where id=5 and version=1 The version number of the first read data is 1, The conditions for modification are version=1 The data of . If there is a transaction in the middle that has modified the data , The version number is definitely not 1, Therefore, the return result of this statement is 0

• If the result of execution is 0, Then read the data again , Perform the modification operation again

JDBC Transaction implementation

Use JDBC Connect mysql By default, every connection is automatically committed . If needed JDBC Execute multiple statements , And ask to form a transaction to execute together

1、 Turn off automatic submission before execution , Set up manual commit transactions Connection The object of .setAutoCommit(false)

2、 If it works , Commit transactions manually Connection The object of .commit();

3、 If an exception occurs during execution , Then manually roll back the undo operation Connection The object of .rollback();

4、 Additional explanation ： Hope to form a habit , Closing Connection Before the object of , Set the connection object back to auto commit ,Connection The object of .setAutoCommit(true)

Because in actual development , Every time I get a connection , It doesn't have to be a new connection , It's the old connection from the connection pool , And closing is not really closing , But also the connection pool , For others to use . In case someone gets it , Thought it was submitted automatically , But not commit, The final data didn't work .

Generally speaking, when it comes to transaction processing , Then the business logic will be more complex . For example, when shopping cart settlement ,1） Add a record to the order table .2） Add more than one order detail record in the order detail , Indicates what the order bought .3） Modify the sales volume and inventory quantity of the commodity table .

Two modification statements are used to simulate a simple transaction .

• update t_department set description = ‘xx’ where did = 2;

• update t_department set description = ‘yy’ where did = 3;

I hope the two statements will either succeed together , Or roll back together . To create failure , You can deliberately write the second sentence wrong update t_department set description = ‘yy’ （ Less where） did = 3;

Connection conn = null; 
try {
    
    Class.forName("com.mysql.cj.jdbc.Driver");
    conn = DriverManager.getConnection("jdbc:mysql:///test? serverTimezone=UTC", "root", "123456"); 
    //  By default, single sentence single transaction , Define the transaction scope manually if necessary , The relationship needs to be submitted automatically  
    conn.setAutoCommit(false);
    PreparedStatement ps = conn.prepareStatement("insert into t1 values(?,?)");
    ps.setInt(1, 125);
    ps.setString(2, "xxx5"); 
    int len = ps.executeUpdate();
    ps.setObject(1, "tttt");// Wrong data type  
    ps.setString(2, "66666"); 
    ps.executeUpdate();
    conn.commit();//  Commit transaction 
} catch (Exception e) {
     
    if (conn != null) 
        conn.rollback();// Rollback undo transaction 
    System.out.println(e.getMessage()); 
} finally {
    
    // If direct connection is used ,conn It doesn't matter whether to restore the original submission method ; If you use a connection pool, you must restore the original self connection   Dynamic submission method  
    if (conn != null)
        conn.close(); 
}

Partition processing

Generally, the created table corresponds to a set of storage files , When the amount of data is large MySQL The performance of began to decline

Solution ： If the data in the data table has a specific business meaning, the characteristics of the data , The data in the table can be dispersed into multiple storage files , To ensure the execution efficiency of a single file .

The most common method of dividing documents is to follow id Value for partitioning , Different partitions correspond to different storage problems . use id Of hash Value for partitioning , It's actually right 10 Take the mold , The data can be evenly distributed to 10 In a file

create table tb_article( 
    id int primary key,
    title varchar(32), 
    content mediumtext 
) partition by hash(id) partitions 10;--  according to id Of hash Value for partitioning , The total is divided into 10 Districts 

[ Failed to transfer the external chain picture , The origin station may have anti-theft chain mechanism , It is suggested to save the pictures and upload them directly (img-GUXLI2R7-1654399757232)(C:\Users\Administrator\AppData\Roaming\Typora\typora-user-images\image-20220605112310220.png)]

The table partition on the server side is transparent to the client , Client or insert data as usual , However, the server side will store data according to the set partition algorithm

PreparedStatement ps = conn.prepareStatement("insert into tb_article values(?,?,?)"); 
for (int i = 0; i < 10000; i++) {
     
    ps.setInt(1, i+1); 
    ps.setString(2, i + "_name");
    ps.setString(3, i+"_content");
    ps.executeUpdate();
}

Purpose of table partition

• Logical data segmentation

• Improve the application speed of single write or read

• Improve the speed of partition range query

• Split data can have multiple physical file paths

• Efficient preservation of historical data

Partition algorithm

MySQL The common partition types supported are Range、List、Hash、key Partition , among range Most commonly

• Range Range ： Allow data to be divided into different ranges , For example, a table can be divided into several partitions by year

• List Predefined list ： Allows the system to segment data by predefined list values

• hash Hash ： Allows you to change one or more columns in a table hash key Calculate , Finally through this hash The code corresponds the data to different partitions

• key Key value ： yes hash An extension of a partition , there hash key By mysql System generated

• Compound mode ： Combination of multiple modes , For example, it has been range On the partitioned table , Once again, the partition in it hash Partition

When specifying the column name in the partition, you need to use the primary key column or part of the primary key , Otherwise, the setting fails

hash Hash partition

It is generally used to split data files evenly without following business rules , The output result has nothing to do with whether the input is regular , Only applicable to integer fields

create table tb_emp( 
    id int primary key auto_increment, 
    name varchar(32) not null, 
    hiredate date default '1989-2-3'
)partition by hash(id) partitions 4;

General requirements hash It is better to have a certain linear relationship between the values in , Otherwise, the partition data will not be evenly distributed .

Keyword partition key

key For strings , Than hash() One more step to calculate an integer from the string , Then take the mold and calculate

create table tb_article(
    id int auto_increment, 
    title varchar(64) comment ' Article title ', 
    content text, 
    primary key(id,title) 
)partition by key(title) partitions 10;

Range partitioning ****range

range Is to partition according to a specified data size range , For example, partition the data according to the publishing time of the article

Get the timestamp select unix_timestamp('2022-4-30 23:59:59 ') 1651334399

select unix_timestamp('2022-3-31 23:59:59 ') 1648742399

create table tb_article(
    id int auto_increment,
    title varchar(32) not null,
    pub_date int, primary key(id,pub_date) 
) partition by range(pub_date)(
    -- 2022 year 3 Monthly and previous data  
    partition p202203 values less than (1648742399),
    -- 2022 year 4 Month of data  
    partition p202204 values less than (1651334399),
    -- 2022 year 4 Data after months 
    partition p202205 values less than maxvalue 
);

[ Failed to transfer the external chain picture , The origin station may have anti-theft chain mechanism , It is suggested to save the pictures and upload them directly (img-YpOEYCD0-1654399757232)(C:\Users\Administrator\AppData\Roaming\Typora\typora-user-images\image-20220605112745213.png)]

among maxvalue Represents the maximum value .MySQL Allow the use of... In partition keys null, The partition key is allowed to be a field , It can also be an expression . commonly MySQL Your partition will null As 0 Or the minimum value . We need to pay attention to ：range in null As a minimum ;list in null Must appear in the enumeration list , Otherwise, it will not be dealt with ;hash perhaps key partition null Be used as a 0 Value processing

Conditional operators can only be used less than, Therefore, the small value is required to be in the front

List partition

List partition is also a conditional partition , Partition using list values . The list value should be a discrete value , The range partition is a continuous value

create table tb_artitle(
    id int auto_increment, 
    title varchar(32) not null 
    status tinyint(1), --  Used to indicate the status of the article , for example 0 draft 、1 Complete unpublished 、2 The published 、3 Off the shelf  
    primary key(id,status)
)partition by list(status)( 
    partition writing values in (1,0), --  Indicates the article being written 
    partition publishing values in(2,3) --  Indicates a completed article  
);