Lightning data import

Use Dumpling The backup data

Reference resources Dumpling Introduction and use _ A big river -CSDN Blog

Edit profile

[lightning]
# journal 

level = "info"
file = "tidb-lightning.log"

[tikv-importer]
#  Choose to use local Back end 
backend = "tidb"

#  Set the temporary storage address of sorted key value pairs ,  The target path directory needs to be empty 
sorted-kv-dir = "/export/servers/temp/tiuplightning/kv_data"


[mydumper]
#  Source data directory 
data-source-dir = "/export/servers/temp/tiuplightning/jcdp"

[tidb]
#  Target cluster information 
host = "XX.XX.XX.XX"
port = 4000
user = "root"
password = "password"
#  Table structure information from TiDB Status port acquisition of , status-port = 10080
#  colony pd Address 
pd-addr = "XX.XX.XX.XX:2379"

Start import

Switch to toolkit Of bin Directory to start importing , 

nohup ./tidb-lightning -config  tidb-lightning.toml > nohup.out

Check the status

•   During the import process , Can pass tidb Of show processlist Command to see multi-threaded concurrent import ;
• If the import is successful , nohup.out  and tidb-lightning.log  Display at the end of the line "tidb lightning exit"

+---+----------------------------------------------+-------------+--------+
| # | CHECK ITEM                                   | TYPE        | PASSED |
+---+----------------------------------------------+-------------+--------+
| 1 | Source csv files size is proper              | performance | true   |
+---+----------------------------------------------+-------------+--------+
| 2 | checkpoints are valid                        | critical    | true   |
+---+----------------------------------------------+-------------+--------+
| 3 | table schemas are valid                      | critical    | true   |
+---+----------------------------------------------+-------------+--------+
| 4 | Cluster is available                         | critical    | true   |
+---+----------------------------------------------+-------------+--------+
| 5 | Lightning has the correct storage permission | critical    | true   |
+---+----------------------------------------------+-------------+--------+
tidb lightning exit