Data batch writing

Write data in bulk

1.Statement write in

@Test
public void testInsert2() {
    

    Connection conn = null;
    PreparedStatement ps = null;
    try {
    
        //  Turn off auto submit 
        conn.setAutoCommit(false);

        long start = System.currentTimeMillis();
		// 1. Get the connection 
        conn = JDBCUtils.getConnection();
        
        // 2. establish statement object 
        ps = conn.createStatement();

        for(int i = 1 ; i <= 20000 ; i++) {
    
            String sql = "insert into goods(name)values("+ "name_" + i +")";

            //1. " Save " sql
            ps.addBatch(sql);

            if(i % 500 == 0) {
    

                //2.  Save enough 500, Do it once batch
                ps.executeBatch();
                
                //3.  Submit 
                conn.commit();
            }
        }

        long end = System.currentTimeMillis();

        System.out.println(" The time spent is :" + (end - start));

    } catch (Exception e) {
    
        e.printStackTrace();
    } finally {
    
        try {
    
            if (ps != null) {
    
                ps.close();
            }
            if (conn != null) {
    
                conn.close();
            }
        } catch (SQLException e) {
    
            e.printStackTrace();
        }
    }
}

2.PreparedStatement Batch write

@Test
public void testInsert2() {
    

    Connection conn = null;
    PreparedStatement ps = null;
    try {
    
        //  Turn off auto submit 
        conn.setAutoCommit(false);

        long start = System.currentTimeMillis();
		// 1. Get the connection 
        conn = JDBCUtils.getConnection();
        
        // 2. establish prepareStatement object 
        String sql = "insert into goods(name)values(?)";
        ps = conn.prepareStatement(sql);

        for(int i = 1 ; i <= 20000 ; i++) {
    
            ps.setObject(1, "name_" + i);	// The representative is set to the first ? The value of position number is Object type 

            //1. " Save " sql
            ps.addBatch();

            if(i % 500 == 0) {
    

                //2.  Save enough 500, Do it once batch
                ps.executeBatch();
                
                //3.  Submit 
                conn.commit();

                //4.  Empty batch
                ps.clearBatch();
            }
        }

        long end = System.currentTimeMillis();

        System.out.println(" The time spent is :" + (end - start));

    } catch (Exception e) {
    
        e.printStackTrace();
    } finally {
    
        try {
    
            if (ps != null) {
    
                ps.close();
            }
            if (conn != null) {
    
                conn.close();
            }
        } catch (SQLException e) {
    
            e.printStackTrace();
        }
    }
}

3.Statement and PreparedStatement Relationships and differences

Statement and PreparedStatement The relationship and difference between .
Relationship ：PreparedStatement Inherited from Statement, It's all interfaces
difference ：PreparedStatement You can use placeholders , It's precompiled , Batch processing is better than Statement Efficient

How to choose to use Statement and PreparedStatement object

JDBC Driver optimization is based on what features are used . choice PreparedStatement still Statement It depends on how you use them . about Carry out only once SQL Statement selection Statement It's the best . contrary , If SQL Statement is selected for multiple executions PreparedStatement It's the best .

PreparedStatement The first execution cost of is very high . Its performance is reflected in the following repeated execution

Statement object ：
Simple to perform without parameters SQL sentence ;
characteristic ：
a. Execute only single sql sentence ;
b. Can only execute... Without parameters sql sentence ;
c. From the perspective of operational principles , Database received sql After the statement, you need to sql Statement is compiled before execution ;
d. Compare with other interfaces , It is suitable for executing a single without parameters sql sentence , In this case, the implementation efficiency is relatively high .

PreparedStatement object
Execute with or without IN Precompiled parameters SQL sentence ;
characteristic ：
a. Inherited from Statement Interface （ It means that the functions are relatively more comprehensive ）;
b. With precompiled features ;
c. Batch processing sql sentence ;
d. Handle with unknown parameters sql sentence ;
e. It's safe , That is, malicious sql Statement injection attack ;
f. In dealing with a single statement , Execution efficiency is not Statement fast ;
g. Improve the readability and maintainability of the program .

---

Execution efficiency （ If they are all inserted in the database table 10000 In case of records ）：

1. Use Statement object when 31 second
2. precompile PreparedStatement when 14 second
3. Use PreparedStatement + The batch when 485 millisecond

1. Use Statement object
Using range ： When performing similar SQL( Structure is the same , The specific values are different ) The number of statements is relatively small
advantage ： Grammar is simple
shortcoming ： Hard coding is inefficient , Poor safety .
principle ： Hard encoding , Each execution is similar SQL Will compile

public void exec(Connection conn){
    
    try {
    
        Long beginTime = System.currentTimeMillis();
        conn.setAutoCommit(false);// Set up manual submission 
        Statement st = conn.createStatement();
        for(int i=0;i<10000;i++){
    
            String sql="insert into t1(id) values ("+i+")";
            st.executeUpdate(sql);
        }
        Long endTime = System.currentTimeMillis();
        System.out.println("Statement when ："+(endTime-beginTime)/1000+" second ");// computing time 
        st.close();
        conn.close();
    } catch (SQLException e) {
                 
        e.printStackTrace();
    }
}

execution time ：Statement when ：31 second

2. precompile PreparedStatement
Using range ： When performing similar sql The number of statements is more （ For example, user login , Frequent operations on tables …） The sentence is the same , But the specific values are different , It's called dynamic SQL
advantage ： Statement is compiled only once , Reduce compilation times . Improved security （ Stopped SQL Inject ）
shortcoming : Perform nonsimilar SQL When the sentence is , Slower .
principle ： be similar SQL Compile only once , Reduce compilation times
Case execution process :

public void exec2(Connection conn){
    
    try {
    
        Long beginTime = System.currentTimeMillis();
        conn.setAutoCommit(false);// Manual submission 
        PreparedStatement pst = conn.prepareStatement("insert into t1(id) values (?)");
        for(int i=0;i<10000;i++){
    
            pst.setInt(1, i);
            pst.execute();   
        }
        conn.commit();
        Long endTime = System.currentTimeMillis();
        System.out.println("Pst when :"+(endTime-beginTime)+" second ");// computing time 
        pst.close();
        conn.close();
    } catch (SQLException e) {
                   
        e.printStackTrace();
    }
}

execution time ：Pst when :14 second

3. Use PreparedStatement + The batch
Using range ： You need to update multiple records in the database table at one time
advantage ： Reduce and SQL Number of engine interactions , Improve efficiency again , Similar statements are compiled only once , Reduce compilation times . Improved security （ Stopped SQL Inject ）
shortcoming :
principle ： The batch : Reduce and SQL Number of engine interactions , Pass to at one time SQL Engine multiple SQL.
A term is used to explain ：
PL/SQL engine ： stay oracle In the implementation of pl/sql The engine of code , Standard found in implementation sql I'll give it to you sql The engine processes .
SQL engine ： Executive Standards sql The engine of .
Case execution process :

public void exec3(Connection conn){
    
     try {
    
           conn.setAutoCommit(false);
           Long beginTime = System.currentTimeMillis();
           PreparedStatement pst = conn.prepareStatement("insert into t1(id) values (?)");
 
          for(int i=1;i<=10000;i++){
       
                pst.setInt(1, i);
                pst.addBatch();// Add batch , package 
                if(i%1000==0){
    // You can set different sizes ; Such as 50,100,500,1000 wait 
                      pst.executeBatch();
                      conn.commit();
                      pst.clearBatch();
                }//end of if
           }//end of for
           pst.executeBatch();
           Long endTime = System.currentTimeMillis();
           System.out.println("pst+batch when ："+(endTime-beginTime)+" millisecond ");
           pst.close();
           conn.close();
      } catch (SQLException e) {
    
            e.printStackTrace();
      }
}

execution time ：pst+batch when ：485 millisecond