DataX installation

One 、 download tar package

github Address ：https://github.com/alibaba/DataX
download tar Package address ：http://datax-opensource.oss-cn-hangzhou.aliyuncs.com/datax.tar.gz

Two 、 Upload tar Unzip the package and install

Upload to server /opt/ And decompress

 tar -xzvf datax.tar.gz

  
   

  
   

    
1
   

3、 ... and 、 Upload and unzip datax-web

link : https://pan.baidu.com/s/1B8bb_KeyIYvpbI5s3IwCjA Extraction code : 3pah
Upload the compressed package to the service directory , Here I upload to /usr/local/ Next , You can upload it to the directory you want

unzip datax-web-2.1.2.zip

  
   

  
   

    
1
   

Four 、 establish datax-web database

sql The files are in the following directory

Four 、 Modify the configuration file

1. Modify database connection
modules Under the datax-admin Under the conf Under the bootstrap.properties

2. Be careful datax-executor Under the bin Under the env.properties Of PYTOHON_PATH Configuration address of , It needs to be installed with us datax The catalog is consistent

5、 ... and 、 start-up datax-web And visit

perform datax-web-2.1.2 Under the bin Under the start-all.sh Start all modules

visit ：http:// The server IP:9527/index.html#/dashboard

6、 ... and 、 Record the problems that may be encountered in the actual development process

1. solve Oracle No, dba Permission query of users and tables failed

getSQLQueryTables Method sql Medium dba_tables It is amended as follows all_tables.
getSQLQueryTableSchema Method sql Medium dba_users It is amended as follows all_users.

  
   

  
   

    
1
    
2
   

Modify the specified method after downloading the source code , pack jar Post replacement datax-admin Under the lib Under the datax-web-2.1.2.jar

2. solve datax Write data to db2 database

datax There is no independent plug-in support for now db2, Need to use general RDBMS; Here is write db2, Use rdbmswriter, The corresponding directory is /datax/plugin/writer/rdbmswriter/
Modify... In this directory plugin.json, Register database driver

{
     
    "name": "rdbmswriter",
    "class": "com.alibaba.datax.plugin.reader.rdbmswriter.RdbmsWriter",
    "description": "useScene: prod. mechanism: Jdbc connection using the database, execute select sql, retrieve data from the ResultSet. warn: The more you know about the database, the less problems you encounter.",
    "developer": "alibaba",
    "drivers":["com.ibm.db2.jcc.DB2Driver"]
}

  
   

  
   

    
1
    
2
    
3
    
4
    
5
    
6
    
7
   

Upload db2 Drive to libs, As shown in the figure

job The files are all in /datax/job Catalog
Create the job Of json file

Execute task commands

python /opt/datax/bin/datax.py /opt/datax/job/xx.json

  
   

  
   

    
1