Batch execute SQL file

One . function

Match the commands that handle the database , It can be used for batch data migration and import .

Two . Operation steps

step1. Generate what needs to be done SQL file

Generate what needs to be done sql file （ Each row of a sql command ）. Examples are as follows ：

delete from lbs_poi where poiid=1000052855 limit 1;
delete from lbs_poi where poiid=1000052856 limit 1;
delete from lbs_poi where poiid=1000052857 limit 1;
delete from lbs_poi where poiid=1000052858 limit 1;
delete from lbs_poi where poiid=1000052859 limit 1;

step2. The original sql File split by fixed number of lines

jar Bag storage location ：10.74.28.129 /data1/bingqing/shAndJar/splitFile.jar splitFile.jar

The above directory is the machine operated by the author , In the company's intranet , If you are not connected to the intranet , Please enter the intranet first .

perform jar Package view jar Call description ：

/data1/bingqing/shAndJar # java -jar splitFile.jar
split file with fixed lineNum
paras: inputFilePath encode fileLineNum
sample: java -jar splitFile.jar poiToDeleteSql.sql utf8 1000

According to the instructions , The target sql Split the file into multiple sub files .

step3. Manually monitor the execution of stalled scripts

shell Script storage location ：10.74.28.129 /data1/bingqing/shAndJar/sleepSql.sh

The script content ：

/data1/bingqing/shAndJar # cat sleepSql.sh

#!/bin/sh
echo '------------------------' >> deletelog.txt

current=`date "+%Y-%m-%d %H:%M:%S"`
#echo $current
echo $current >> deletelog.txt

#mysql -uLBQ -pPSBPkJaHX6aTeWEOds -hm4000i.randa.grid.sina.com.cn --default-character-set=utf8 -P4000 LBQ < poiToDeleteSqlAll2.sql


for i in {3001..10000};
do
#echo $i
echo $i >> deletelog.txt
mysql -uLBQ -pPSBPkJaHX6aTeWEOds -hm5000i.randa.grid.sina.com.cn --default-character-set=utf8 -P4000 LBQ < poiToDeleteSqlAll${i}.sql

current=`date "+%Y-%m-%d %H:%M:%S"`
#echo $current
echo $current >> deletelog.txt

sleep 3s
done


echo 'over!' >> deletelog.txt

notes ：

1.mysql -uLBQ -pPSBPkJaHX6aTeWEOds -hm4000i.randa.grid.sina.com.cn --default-character-set=utf8 -P4000 LBQ < poiToDeleteSqlAll${i}.sql

This is the database connection string , The parameter analysis is as follows ：

-uLBQ  Specify user name （ Sometimes we are used to using the names of databases or departments ）

-pPSBPkJaHX6aTeWEOds  password 

-hm4000i.randa.grid.sina.com.cn  Connect objects or connect targets , This parameter contains the host name , port , The mode of the target library is master or slave , And jdbc Medium MySQL Connection configuration is the same . among hm The table name connects to the main database , If it is hs From the library ;4000 Is the port number ; The suffix is host name （ The intranet is configured , Use the domain name directly ）

--default-character-set=utf8  Specify the encoding format , Prevent garbled code during operation 


-P4000  Designated port 

LBQ  Specify database