A slow SQL drags the whole system down

1. Own case

A few days ago, the monitoring probe suddenly gave an alarm , Manually calling the production interface is found to be particularly slow , At ordinary times A The interface that can respond in hundreds of milliseconds in the service becomes 3-5 second , When it is slow, it takes more than ten seconds , Sometimes it just times out .

First, check the related services , Find out A service cpu And memory are normal , The service log is also output normally , Service problems are eliminated , I once thought it was the Internet that fluctuated , No reason found , Restart the service directly , The problem has not been solved .

Until later, someone said whether there was a problem with the database , Then view the database , Sure enough, I found a pile update The statement is stuck waiting for execution , Then take a look at update Of where Conditions , This field has no index , The reason is found , Because the update condition has no index , Lead to update Statements are particularly slow , The business here is batch update , In an instant, a large number of update statements will enter sql, This leads to a large consumption of database resources , Thus affecting the whole service （ This update The sentence is B service , namely B The service is slow sql It takes up a lot of database resources , This causes all other services to respond slower ）.

After checking the reason, an index is added to the corresponding field , Then it gets faster , The service gradually recovers stability .（ Note that the index is directly added here , At the same time, if this statement continues to enter the database, I think it may lead to locking the table , Because this business is special , Only a few people use , I call them directly to stop using , Then add the index ）

Be careful ： Slow here sql yes update sentence , I wanted to DBA direct kill Of , But because of update sentence DBA Dare not kill, Say you can only rollback , If it is select sentence , directly kill.

Here I have another idea is , Deactivate the interface first , Prevent a lot of update Statement continues to enter the database , Then add the index , So you can add a switch to the system , Configure whether this interface is enabled , You can start and stop an interface at any time , When something goes wrong with an interface , Just stop , Then seize the time to solve , Then enable the interface .

2. Several similar cases found on the Internet , Make a note of

1. Case study ： One slow SQL Drag down the whole system

One day, I suddenly found a crazy alarm on the service detection interface 、 At the same time, the database CPU Consumption also alarm , Finally, the system cannot be accessed ;

At first, I thought there was a problem with the service , After the service is restarted, the phenomenon remains ;

After checking the database, we found , A lot of slow SQL Blocking waiting for execution ：

See which tables are locked ：show OPEN TABLES where In_use > 0;

Query in progress SQL, Find a lot SQL Execution blocked for hundreds of seconds

select * from information_schema.processlist where db=‘ db_xxx ‘ and info is not null;

The process of directly fetching the index ID, To assemble into kill sentence , Take it out and execute it , Kill the blocking indexing process .

select concat(‘kill ‘, id,‘;‘) from information_schema.processlist where db=‘db_xxx ‘ and info is not null;

Found that after killing once , Soon, a large number of execution blocking SQL, and 90% The above is one SQL.

kill N After that, I still can't , There will still be ......

Implement this article SQL Pick it out , For query criteria , Composite index created ,SQL The blockage disappears ,

The system has also returned to normal .....

Reference resources ：MySQL Optimize 5 And CPU Consumption is too high （ One slow SQL Drag down the whole system ）