Record a MYCAT connection and solve the problems of communications link failure

Project scenario ：

Multithreading processing data push ： Yes A and B Two tables ,A Table is MySQL Connect ,B Table is Mycat Connect , Each has 200 Split sheet ,A and B Joint query splicing data is pushed to other systems for use . The current scheme is A surface 1-200 Read data page by page in separate tables , Get id Gather to B Logical table query of table , Splicing data .

---

Problem description

Adjust the page size to 1000, The program reports an error after running for a period of time , The specific errors are as follows , Later, the paging size was reduced , No report error , And then adjust to 5000, The running time of the program is greatly reduced and errors are reported .

[2022-07-20 16:36:30.723] [ Public area data query thread 781294] [WARN ] com.zaxxer.hikari.pool.ProxyConnection - HikariPool-1 - Connection [email protected] marked as broken because of SQLSTATE(08S01), ErrorCode(0)
com.mysql.cj.jdbc.exceptions.CommunicationsException: Communications link failure

The last packet successfully received from the server was 1,882 milliseconds ago. The last packet sent successfully to the server was 1,883 milliseconds ago.
        at com.mysql.cj.jdbc.exceptions.SQLError.createCommunicationsException(SQLError.java:174)
        at com.mysql.cj.jdbc.exceptions.SQLExceptionsMapping.translateException(SQLExceptionsMapping.java:64)
        at com.mysql.cj.jdbc.ClientPreparedStatement.executeInternal(ClientPreparedStatement.java:953)
        at com.mysql.cj.jdbc.ClientPreparedStatement.execute(ClientPreparedStatement.java:370)
        at com.zaxxer.hikari.pool.ProxyPreparedStatement.execute(ProxyPreparedStatement.java:44)
        at com.zaxxer.hikari.pool.HikariProxyPreparedStatement.execute(HikariProxyPreparedStatement.java)
        at org.apache.ibatis.executor.statement.PreparedStatementHandler.query(PreparedStatementHandler.java:63)
        at org.apache.ibatis.executor.statement.RoutingStatementHandler.query(RoutingStatementHandler.java:79)
        at org.apache.ibatis.executor.SimpleExecutor.doQuery(SimpleExecutor.java:63)
        at org.apache.ibatis.executor.BaseExecutor.queryFromDatabase(BaseExecutor.java:325)
        at org.apache.ibatis.executor.BaseExecutor.query(BaseExecutor.java:156)
        at org.apache.ibatis.executor.CachingExecutor.query(CachingExecutor.java:109)
        at org.apache.ibatis.executor.CachingExecutor.query(CachingExecutor.java:83)
        at org.apache.ibatis.session.defaults.DefaultSqlSession.selectList(DefaultSqlSession.java:148)
        at org.apache.ibatis.session.defaults.DefaultSqlSession.selectList(DefaultSqlSession.java:141)
        at sun.reflect.GeneratedMethodAccessor34.invoke(Unknown Source)
        at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
        at java.lang.reflect.Method.invoke(Method.java:498)
        at org.mybatis.spring.SqlSessionTemplate$SqlSessionInterceptor.invoke(SqlSessionTemplate.java:434)
        at com.sun.proxy.$Proxy54.selectList(Unknown Source)
        at org.mybatis.spring.SqlSessionTemplate.selectList(SqlSessionTemplate.java:231)
        at org.apache.ibatis.binding.MapperMethod.executeForMany(MapperMethod.java:128)
        at org.apache.ibatis.binding.MapperMethod.execute(MapperMethod.java:68)
        at org.apache.ibatis.binding.MapperProxy.invoke(MapperProxy.java:53)
        at com.sun.proxy.$Proxy58.querySource(Unknown Source)
        at gov.zwfw.iam.datacutover.service.impl.NaturalServiceImpl.moveData(NaturalServiceImpl.java:63)
        at gov.zwfw.iam.datacutover.thread.NaturalDataThread.run(NaturalDataThread.java:43)
        at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
        at java.util.concurrent.FutureTask.run(FutureTask.java:266)
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
        at java.lang.Thread.run(Thread.java:748)
Caused by: com.mysql.cj.exceptions.CJCommunicationsException: Communications link failure

Cause analysis ：

This problem begins to appear , I started from navicat I inquired about sql The state of ,SHOW FULL PROCESSLIST; Found most of sql Of state by Waiting for query cache lock,

So search this question , It is found that there is a problem with query caching , And then according to show status like '%Qcache%'; Query cache related information , as follows
according to show variables like '%query_cache%'; Query cache settings
After querying relevant documents , Query cache is turned off , Get into mysql Of my.cnf file , Set the following two properties
All in one operation , Restart the program ,sql The state is no problem , however , After a period of time, it still reports an error , It's just a little longer , therefore , Check this question again , Some documents say that the timeout of idle connection has exceeded mysql Connection time , therefore , Inquire about mysql Connection timeout for ,show global variables like "wait_timeout";
The beginning is 3000, Then I set the timeout as the default 8 Hours , namely 28800,set global wait_timeout=28800; Note that this operation must be in MySQL Connect , Or just modify it my.cnf file
meanwhile , The following configuration is set on the program configuration

validationQuery: SELECT 1
testWhileIdle: true
timeBetweenEvictionRunsMillis: 28000

And then restart the program , After a while , Report errors .............
This operation down , It has wasted my whole day , If you don't do well, you will be scolded , Still, , It occurred to me that , It's always been mycat Connection error , Then I read the log , Find out console.log The log will be reported after the program reports an error oom, Is it right? mycat There's something wrong with the settings for , therefore , Open the mycat Configuration of , see wrapper.conf file , It is found that a direct memory configuration is 2g, Then I had a whim to try this bigger , Changed to 8g, This time, , The program runs longer , But the error is still reported in the end
Then I took a look mycat Log , The operation log has been brushing sql, At this time , With my eyes, I found the problem

Execute a in the logic table sql, It needs to be executed once in each partition table , It's so immortal that I use it here in Go to query , At this time, I probably understood , Due to the increasing amount of data , Queries slow down , No connection caused mycat Feign death , This is why it's OK to reduce the page size , And restart after reporting an error mycat It's still working .

Solution ：

take mycat At the time of inquiry in Range query is changed to equivalent query , such , Although every time I have to go to the circulation Library , But at least the program ran , The performance of circular query is also better than the previous in Range query is much faster , meanwhile , The paging size can also be set to a larger value , Problem solving .