JDBC streaming query and cursor query

The source code is based on mysql-connector-5.1

1. obtain statement

PreparedStatement statement = conn.prepareStatement(sql,ResultSet.TYPE_FORWARD_ONLY,ResultSet.CONCUR_READ_ONLY);
ResultSet resultSet = statement.executeQuery();

prepareStatement() In the method resultSetType The default value is ResultSet.TYPE_FORWARD_ONLY,resultSetConcurrency The default value is ResultSet.CONCUR_READ_ONLY

2. Get the result set

com.mysql.jdbc.MysqlIO

protected ResultSetImpl getResultSet(StatementImpl callingStatement, long columnCount, int maxRows, int resultSetType, int resultSetConcurrency,
        boolean streamResults, String catalog, boolean isBinaryEncoded, Field[] metadataFromCache) throws SQLException {
    
    if (this.connection.versionMeetsMinimum(5, 0, 2) && this.connection.getUseCursorFetch() && isBinaryEncoded && callingStatement != null
        && callingStatement.getFetchSize() != 0 && callingStatement.getResultSetType() == ResultSet.TYPE_FORWARD_ONLY) {
    
        if (usingCursor) {
    
            RowData rows = new RowDataCursor(this, prepStmt, fields);
            ResultSetImpl rs = buildResultSetWithRows(callingStatement, catalog, fields, rows, resultSetType, resultSetConcurrency, isBinaryEncoded);
            if (usingCursor) {
    
                rs.setFetchSize(callingStatement.getFetchSize());
            }
            return rs;
        }
    }
    if (!streamResults) {
    
        rowData = readSingleRowSet(columnCount, maxRows, resultSetConcurrency, isBinaryEncoded, (metadataFromCache == null) ? fields : metadataFromCache);
    } else {
    
        rowData = new RowDataDynamic(this, (int) columnCount, (metadataFromCache == null) ? fields : metadataFromCache, isBinaryEncoded);
        this.streamingData = rowData;
    }

    ResultSetImpl rs = buildResultSetWithRows(callingStatement, catalog, (metadataFromCache == null) ? fields : metadataFromCache, rowData, resultSetType,
            resultSetConcurrency, isBinaryEncoded);

    return rs;
}

From the above code , Common query , The result set returned by the churn query is different from that of the cursor query .

3. Common query

The default is normal query
stay readSingleRowSet() In the method , Add all data to rows(ArrayList) in , Return static row data (RowDataStatic)

4. Streaming query

Do not load all data into memory at once , Calling next() When the method is used ,MySQL The driver only gets from the network data stream 1 Data , Then return to the application

Opening mode

1. Set up statement.setFetchSize(Integer.MIN_VALUE);
2. call createStreamingResultSet()

void enableStreamingResults() throws SQLException;
mysql-connector-8
((com.mysql.cj.jdbc.JdbcStatement)statement).enableStreamingResults()
mysql-connector-5.1
((com.mysql.jdbc.Statement)statement).enableStreamingResults()

StatementImpl.java
/** * We only stream result sets when they are forward-only, read-only, and the * fetch size has been set to Integer.MIN_VALUE * * @return true if this result set should be streamed row at-a-time, rather * than read all at once. */
protected boolean createStreamingResultSet() {
    
    return ((this.query.getResultType() == Type.FORWARD_ONLY) && (this.resultSetConcurrency == java.sql.ResultSet.CONCUR_READ_ONLY)
            && (this.query.getResultFetchSize() == Integer.MIN_VALUE));
}

The first two parameters are the default values , So just set it up fetchSize by Integer.MIN_VALUE that will do , This is mysql-connector Special values customized in , stay JDBC There is no specification in the interface .
enableStreamingResults() You can also enable streaming queries , But it's not jdbc Interface .

The process of obtaining data

1. Return dynamic row data
   
2. Every time you call resultSet.next() when , Will get a piece of data
   

5. Cursor query

Every time you retrieve from the server fetchSize Row data , call next() After traversing , It will fetch data from the server again .

Opening mode

url=jdbc:mysql://192.168.9.19:3307/test?useCursorFetch=true
statement.setFetchSize(20);

The process of obtaining data

1. Return cursor row data
   
2. Every time you call resultSet.next() when , Judge the present fetch The location of , If not less than (size-1) Words , It will fetch data again
   
3. stay fetchMoreRows() I will give you mysql Send a message with fetchSize Of fetch command , call nextRow Take the data
   

6. summary

1. These three methods will call MysqlIO#ResultSetRow nextRow() Method to get data .
2. Cursor query is also a kind of streaming query , Only the amount of data read from the server each time can be set .
3. Streaming query ( Include cursor ) Will use blocking methods from mysql Take the data , So you have to read all the rows in the result set ( Or shut down ) Then you can initiate other queries on this connection , Otherwise, an exception will be thrown .
   There are some caveats with this approach. You must read all of the rows in the result set (or close it) before you can issue any other queries on the connection, or an exception will be thrown.