MYSQL-Batch insert data

Bulk insert data

Stored procedures also have return values, stored procedures have one or more return values, and functions have one and only one return value

Insert 1000w data into the table

1. Build a table

create database bigData;use bigData;create table dept(id int unsigned primary key auto_increment,deptno mediumint unsigned not null default 0,dname varchar(20) not null default "",loc varchar(13) not null default "")engine=innodb default charset=GBK;CREATE TABLE emp(id int unsigned primary key auto_increment,empno mediumint unsigned not null default 0,ename varchar(20) not null default "",job varchar(9) not null default "",mgr mediumint unsigned not null default 0,hiredate date not null,sal decimal(7,2) not null,comm decimal(7,2) not null,deptno mediumint unsigned not null default 0)ENGINE=INNODB DEFAULT CHARSET=GBK;

2. Set the parameter log_bin_trust_function_creators

Since mysql itself will generate an error when inserting large data, a parameter, an enabled function module such as binary log, must be set.

Create a function, if an error is reported: This function has none of-DETERMINISTIC....#Because the slow query log is turned on, because we turned on bin-log, we must specify a function for our functionshow variables like 'log_bin_trust_function_creators';set global log_bin_trust_function_creators=1;#After adding parameters in this way, if mysqld restarts, the above parameters will disappear again, permanent method:My.ini[mysqld] under windows plus log_bin_trust_function_creators=1My.cnf[mysqld] under letc/my.cnf under linux plus log_bin_trust_function_creators=1

3. Create a random function to ensure that each data is different

• Randomly generated string
• Randomly generated department number

//Function: Randomly generate stringsdelimiter $$create function ran_string(n int) returns varchar(255)begindeclare chars_str varchar(100) default 'abcdefghijklmnopqrstuvwxyzABCDEFGHIJKLMNOPQRSTUVWXYZ';declare return_str varchar(255) default '';declare i int default 0;while i < n doset return_str = concat(return_str,substring(chars_str,floor(1+rand()*52),1));set i=i+1;end while;return return_str;end $$//Function: Randomly generate department numberdelimiter $$create function rand_num() returns int(5)begindeclare i int default 0;set i=floor(100+rand()*10);return i;end $$# If you want to delete the function, execute: drop function rand_str

4. Create a stored procedure

• Create a stored procedure to insert data into the emp table
• Create a stored procedure to insert data into the dept table

#Stored procedure: create a stored procedure that inserts data into the emp tabledelimiter $$ :create procedure insert_emp(in start int(10),in max_num int(10))begindeclare i int default 0;set autocommit = 0; #Set autocommit to 0, so there is no need to commit every time a piece of data is inserted, and repeated commits affect execution efficiencyrepeatset i = i+1;insert into emp(empno,ename,job,mgr,hiredate,sal,comm,deptno) values((start+i),ran_string(6),'salesman',0001,curdate(),2000,400,rand_num());until i=max_numend repeat;commit; #Automatic one-time commitend $$#delete# DELIMITER;# drop PROCEDURE insert_emp;#Stored procedure: Create a stored procedure that inserts data into the dept tabledelimiter $$create procedure insert_dept(in start int(10),in max_num int(10))begindeclare i int default 0;set autocommit = 0;repeatset i = i+1;insert into dept(deptno,dname,loc) values((start+i),ran_string(10),ran_string(8));until i=max_numend repeat;commit;end $$#delete#DELIMITER;#drop PROCEDURE insert_dept;

5. Call the stored procedure

delimiter ; #Restore to; endCALL insert_dept(100,10);CALL insert_emp(100,5);

When there is a large amount of data, it takes a lot of time.The select may take a long time. At this time, if the slow query log is enabled, the time-out sql statement will be recorded in the log.

Delete indexes on a table in batches

DELIMITER $$CREATE PROCEDURE `proc_drop_index`(dbname VARCHAR(200), tablename VARCHAR(200))BEGINDECLARE done INT DEFAULT 0;DECLARE ct INT DEFAULT 0;DECLARE _index VARCHAR(200) DEFAULT '';DECLARE _cur CURSOR FOR SELECT index_name FROM information_schema.STATISTICS WHEREtable_schema=dbname AND table_name=tablename AND seq_in_index=1 AND index_name <>'PRIMARY' ;DECLARE CONTINUE HANDLER FOR NOT FOUND set done=2 ;OPEN_cur;FETCH _cur INTO _index;WHILE _index<>'' DOSET @str = CONCAT("drop index ",_index," on ",tablename );PREPARE sql_str FROM @str ;EXECUTE sql_str;DEALLOCATE PREPARE sql_str;SET _index='';FETCH _cur INTO _index;END WHILE;CLOSE_cur;END $$#implementCALL proc_drop_index("dbname","tablename");