Positioning of soaring problems caused by online MySQL CPU

problem

One day, I suddenly received an alert email , The server CPU Threshold exceeded , And has been consistently high

The analysis reason

1. Use htop View resource consumption , according to CPU Usage is in descending order , All the discoveries are mysqld Process occupation CPU Very high
2. Get into mysql Command line show processlist; View the currently executing commands , After many executions show processlist Found several fixed sql It's been implemented , And the parameters passed each time are different

mysql> show processlist;
+--------+------+-------------------+--------------------------+---------+------+--------------+-----------------------------------------------------------------------------------------------+
| Id     | User | Host              | db                       | Command | Time | State        | Info                                                                                          |
+--------+------+-------------------+--------------------------+---------+------+--------------+-----------------------------------------------------------------------------------------------+
| 130878 | root | localhost:43314   | bomconfig                | Sleep   |   77 |              | NULL                                                                                          |
| 132762 | root | localhost:34342   | ccsx_weibao              | Query   |    0 | Sending data | SELECT vir.*, 1 as business_type
                FROM ccsx_data.v_install_record vir
                WHERE vir.id = 1948  |
| 132841 | root | localhost:42218   | ccsx                     | Sleep   |  574 |              | NULL                                                                                          |
| 132955 | root | localhost:54332   | ccsx                     | Sleep   |  693 |              | NULL                                                                                          |
| 133058 | root | localhost:38132   | bomconfig                | Sleep   |  434 |              | NULL                                                                                          |
| 133243 | root | localhost:58272   | ccsx_data                | Sleep   |  539 |              | NULL                                                                                          |
| 133347 | root | localhost:41100   | ccsx_weibao              | Query   |    0 | Sending data | select * from ccsx_weibao.install_record_accessory where host_id=2106                         |
| 133416 | root | localhost:47832   | international_ccsx_data  | Sleep   |  519 |              | NULL                                                                                          |
| 133454 | root | localhost:51226   | internatinal_ccsx_weibao | Sleep   | 1194 |              | NULL                                                                                          |
| 133459 | root | localhost:51642   | ccsx                     | Sleep   |    1 |              | NULL                                                                                          |
| 133466 | root | localhost:51956   | bomconfig                | Sleep   |  488 |              | NULL                                                                                          |
| 133467 | root | localhost:51958   | bomconfig                | Sleep   |   77 |              | NULL                                                                                          |
| 133469 | root | localhost:52616   | ccsx_data                | Sleep   |  540 |              | NULL                                                                                          |
| 133519 | root | localhost:57874   | ccsx_weibao              | Query   |    0 | Sending data | SELECT vir.*, 1 as business_type
                FROM ccsx_data.v_install_record vir
                WHERE vir.id = 19042 |

After the first 2 Step execution , Guess someone is for The query is called in the loop , Different parameters are passed each time

3. In order to confirm the correctness of my analysis , Copy show processlist The results of the sql, Go to the project code query , Finally, the following code is traced , Indeed, queries are used in the loop ,fuck！

   	@Override
   	public List<InstallRecordVO> getInstalls(List<Integer> ids) {
         
   		List<InstallRecordVO> installRecords = new ArrayList<>();
   		if (CollectionUtil.isNotEmpty(ids)){
         
   			ids.forEach(t->{
         
   				InstallRecordVO installRecordVO = installRecordMapper.queryById(t);
   				if(installRecordVO!=null){
         
   					List<InstallRecordAccessory> installRecordAccessories = installRecordAccessoryMapper.queryByHostId(installRecordVO.getId());
   		//…… Omitted code 
   					List<SysFile> sysFileList = sysFileMapper.queryByBusinessInfo(paramMap);
   		//…… Omitted code 
   				}
   			});
   		}
   		return installRecords;
   	}
   

4. Just having the above code is not completely sure , This loop has been running online , So I decided to use arthas To further determine , So I put arths attach Use on projects in the production environment trace Track the three queries in the loop respectively , See if it is being implemented , As a result, all three are being implemented , It confirms that the cycle has been executing , therefore cpu Here's the problem .

trace com.sonoscape.ccs.data.mapper.InstallRecordMapper queryById  -n 5 --skipJDKMethod false 

trace com.sonoscape.ccs.data.mapper.InstallRecordAccessoryMapper queryByHostId  -n 5 --skipJDKMethod false 

trace com.sonoscape.ccs.data.mapper.SysFileMapper queryByBusinessInfo  -n 5 --skipJDKMethod false 

5. Finally, I found the person in charge of this part of the code , His ids The parameter should have been filtered , Not much in theory , As a result, the query results in ids It's actually full table , spit

Conclusion

Don't easily execute in a loop sql