Tencent cloud database mysql:sql flow restriction

background

In database related problems , We often encounter high concurrency , There are many reasons for this phenomenon , For example, slow query accumulation , Sudden increase of business flow, etc . Usually when facing this kind of problem ,DBA Will be more passive , The conventional means are only regular batch kill Inefficient queries , Until the development students temporarily adjust the business query or front-end business flow restriction .

Regular batch kill There are many problems , for example kill The operation of does not necessarily kill inefficient queries in time ; It is impossible for regular batch queries to keep up with the speed of queries initiated by the program , In essence, the database is still in hang A dead state , The business continues to suffer .

For this kind of scenario , Tencent cloud database MySQL A new function is implemented at the kernel level ：SQL Current limiting , And it's packaged in DBbrain in , Flexible use of this function can quickly and effectively deal with such problems .

Function entry

Enter the database intelligent manager DBbrain Main console for , The page is as follows ：

DBBrain

After entering the real-time session page , You can see that there is SQL This function of current limiting tab：

Real time session

SQL The detailed configuration interface of the current limiting task is shown in the following figure ：

SQL Current limiting

The main options work as follows ：

• SQL type ： Current limiting can be carried out SQL type , Include select,delete,update,insert,replace.
• Maximum number of concurrent ： Limited... Can be executed normally SQL Of The number of connections .
• Way of execution ： Automatically close after a certain time ; Or it can last forever , Until it's closed manually .
• Current limiting time ： Duration before automatic shutdown , The maximum is 1440 minute .
• SQL keyword ： Current limited SQL Statement keywords , You can use the table name , Or is it SQL Part of the text .

It should be noted that , The maximum concurrent number refers to the number of connections , and No QPS, Therefore, this restriction can not effectively control the overall QPS, If SQL It's very efficient , So even if the concurrency is limited to 1, Holistic QPS It may not be very low .

The actual effect

To verify the actual effect , This test uses Tencent cloud database MySQL 5.7 The latest edition , And use the ECS of Tencent cloud to deploy sysbench client , Use read_only Test model to simulate business access pressure . After a period of time ,sysbench The output data of is as follows ：

[ 100s ] thds: 24 tps: 616.50 qps: 8633.22 (r/w/o: 8633.22/0.00/0.00) lat (ms,95%): 40.37 err/s: 0.00 reconn/s: 0.00
[ 110s ] thds: 24 tps: 620.00 qps: 8678.82 (r/w/o: 8678.82/0.00/0.00) lat (ms,95%): 39.65 err/s: 0.00 reconn/s: 0.00
[ 120s ] thds: 24 tps: 620.38 qps: 8689.77 (r/w/o: 8689.77/0.00/0.00) lat (ms,95%): 39.65 err/s: 0.00 reconn/s: 0.00
[ 130s ] thds: 24 tps: 620.81 qps: 8686.73 (r/w/o: 8686.73/0.00/0.00) lat (ms,95%): 39.65 err/s: 0.00 reconn/s: 0.00
[ 140s ] thds: 24 tps: 617.70 qps: 8652.90 (r/w/o: 8652.90/0.00/0.00) lat (ms,95%): 40.37 err/s: 0.00 reconn/s: 0.00
[ 150s ] thds: 24 tps: 611.00 qps: 8549.66 (r/w/o: 8549.66/0.00/0.00) lat (ms,95%): 41.85 err/s: 0.00 reconn/s: 0.00
[ 160s ] thds: 24 tps: 614.50 qps: 8604.52 (r/w/o: 8604.52/0.00/0.00) lat (ms,95%): 41.10 err/s: 0.00 reconn/s: 0.00
[ 170s ] thds: 24 tps: 619.29 qps: 8670.92 (r/w/o: 8670.92/0.00/0.00) lat (ms,95%): 39.65 err/s: 0.00 reconn/s: 0.00
[ 180s ] thds: 24 tps: 619.71 qps: 8672.68 (r/w/o: 8672.68/0.00/0.00) lat (ms,95%): 40.37 err/s: 0.00 reconn/s: 0.00

This is for all sbtest Table in the query , Set up 1 Minute current limit , And the concurrency number is 1：

Set current limiting

See the actual effect after setting ：

[ 240s ] thds: 24 tps: 620.71 qps: 8690.32 (r/w/o: 8690.32/0.00/0.00) lat (ms,95%): 39.65 err/s: 0.00 reconn/s: 0.00
[ 250s ] thds: 24 tps: 619.50 qps: 8669.86 (r/w/o: 8669.86/0.00/0.00) lat (ms,95%): 39.65 err/s: 0.00 reconn/s: 0.00
[ 260s ] thds: 24 tps: 620.20 qps: 8683.36 (r/w/o: 8683.36/0.00/0.00) lat (ms,95%): 39.65 err/s: 0.00 reconn/s: 0.00
[ 270s ] thds: 24 tps: 612.60 qps: 8582.43 (r/w/o: 8582.43/0.00/0.00) lat (ms,95%): 41.85 err/s: 0.00 reconn/s: 0.00
[ 280s ] thds: 24 tps: 618.31 qps: 8650.48 (r/w/o: 8650.48/0.00/0.00) lat (ms,95%): 40.37 err/s: 0.00 reconn/s: 0.00
[ 290s ] thds: 24 tps: 619.40 qps: 8671.66 (r/w/o: 8671.66/0.00/0.00) lat (ms,95%): 40.37 err/s: 0.00 reconn/s: 0.00
[ 300s ] thds: 24 tps: 618.59 qps: 8664.02 (r/w/o: 8664.02/0.00/0.00) lat (ms,95%): 40.37 err/s: 0.00 reconn/s: 0.00
[ 310s ] thds: 24 tps: 619.61 qps: 8671.11 (r/w/o: 8671.11/0.00/0.00) lat (ms,95%): 39.65 err/s: 0.00 reconn/s: 0.00
[ 320s ] thds: 24 tps: 264.90 qps: 6586.44 (r/w/o: 6586.44/0.00/0.00) lat (ms,95%): 376.49 err/s: 2246.51 reconn/s: 0.00
[ 330s ] thds: 24 tps: 52.30 qps: 5346.70 (r/w/o: 5346.70/0.00/0.00) lat (ms,95%): 1648.20 err/s: 3503.80 reconn/s: 0.00
[ 340s ] thds: 24 tps: 54.10 qps: 5452.58 (r/w/o: 5452.58/0.00/0.00) lat (ms,95%): 1352.03 err/s: 3456.59 reconn/s: 0.00
[ 350s ] thds: 24 tps: 54.60 qps: 5416.30 (r/w/o: 5416.30/0.00/0.00) lat (ms,95%): 1427.08 err/s: 3521.60 reconn/s: 0.00
[ 360s ] thds: 24 tps: 53.20 qps: 5383.31 (r/w/o: 5383.31/0.00/0.00) lat (ms,95%): 1648.20 err/s: 3565.41 reconn/s: 0.00
[ 370s ] thds: 24 tps: 56.60 qps: 5580.50 (r/w/o: 5580.50/0.00/0.00) lat (ms,95%): 1327.91 err/s: 3376.50 reconn/s: 0.00
[ 380s ] thds: 24 tps: 434.60 qps: 7581.38 (r/w/o: 7581.38/0.00/0.00) lat (ms,95%): 42.61 err/s: 1178.20 reconn/s: 0.00
[ 390s ] thds: 24 tps: 612.80 qps: 8581.10 (r/w/o: 8581.10/0.00/0.00) lat (ms,95%): 41.10 err/s: 0.00 reconn/s: 0.00
[ 400s ] thds: 24 tps: 616.19 qps: 8631.43 (r/w/o: 8631.43/0.00/0.00) lat (ms,95%): 40.37 err/s: 0.00 reconn/s: 0.00
[ 410s ] thds: 24 tps: 620.09 qps: 8680.52 (r/w/o: 8680.52/0.00/0.00) lat (ms,95%): 39.65 err/s: 0.00 reconn/s: 0.00
[ 420s ] thds: 24 tps: 620.40 qps: 8685.16 (r/w/o: 8685.16/0.00/0.00) lat (ms,95%): 39.65 err/s: 0.00 reconn/s: 0.00
[ 430s ] thds: 24 tps: 619.01 qps: 8662.79 (r/w/o: 8662.79/0.00/0.00) lat (ms,95%): 40.37 err/s: 0.00 reconn/s: 0.00
[ 440s ] thds: 24 tps: 619.10 qps: 8667.34 (r/w/o: 8667.34/0.00/0.00) lat (ms,95%): 39.65 err/s: 0.00 reconn/s: 0.00

You can see in the sysbench Side , There's a lot of err,QPS Down about 35%, And average delay 95 There has been a huge increase in the quantile , continued 1 It will return to normal in minutes . After the current limit is completed ,DBbrain You can see the details of the actual limits ：

Limiting effect

Expand the

actually , A little study will reveal , This SQL Setting page of current limit , You can set the concurrency number to 0, for example ：

Concurrent as 0

Then before and after the actual effect , Compare the differences ：

mysql> delete from sbtest.sbtest1 where id = 1;
Query OK, 1 row affected (0.01 sec)

mysql> delete from sbtest.sbtest1 where id = 1;
ERROR 5001 (HY000): SQL rejected by CDB_SQL_FILTER
mysql>

You can see that after it takes effect ,delete Their behavior was directly rejected , Take a look at this time DBbrain Details of the restrictions ：

Limiting effect

So this function , When you need to restrict certain operations on certain tables , It can be used as SQL Firewall to use , For example, as in this example , Limit sbtest All on the table delete operation .

To sum up

DBbrain As a default free service , It can greatly improve users' screening of Tencent cloud database MySQL The efficiency of the problem , And timely discover and solve a large number of hidden dangers , You can learn more about the functions .

DBbrain What is it? ？

Intelligent database manager （TencentDB for DBbrain,DBbrain） It is a product launched by Tencent cloud to provide users with database performance optimization 、 Security 、 Database autonomous cloud service with management and other functions .

DBbrain Use machine learning 、 Big data means 、 Expert experience engine quickly replicates the mature experience of senior database administrators , A large number of traditional manual database operation and maintenance work will be intelligentized , Serve enterprises on and under the cloud , Effectively ensure the security of database services 、 Stable and efficient operation .