Tdengine can read and write through dataX

Small  T  Reading guide ：

DataX  It is an offline data synchronization tool open source by Alibaba cloud / platform , At present, it has supported the efficient data synchronization function including a variety of mainstream heterogeneous data sources . Our R & D team has completed  

TDengine

  The adaptation of , Related code has been  DataX  Officially accepted and incorporated into the trunk .

DataX

  It is an offline data synchronization tool open source by Alibaba cloud / platform , At present, it has realized the following functions:  MySQL、Oracle、OceanBase、SQL Server、PostgreSQL、HDFS、Hive、HBase  And other heterogeneous data sources .

DataX  The design idea is ：

DataX  As a data synchronization framework , The synchronization of different data sources is abstracted as reading data from data sources  

Reader

  plug-in unit , And write data to the target  

Writer

  plug-in unit , Theoretically  DataX  The framework can support data synchronization of any data source type . meanwhile  DataX  Plug in system as a set of ecosystem , Each access to a new set of data sources , The interworking between the data source and the existing data source can be realized .

Based on this design idea , Our R & D team has completed  TDengine  The adaptation of , Realized  

TDengineReader

  and  

TDengineWriter

  Two plug-ins , And be  DataX  Official acceptance , Merged into its trunk .

TDengineReader

  The features provided include ：

• Supported by  SQL  Data filtering ;

• Perform task segmentation according to time interval ;

• Support  TDengine  All data types of ;

• Support batch reading , Can pass  
  batchSize
    Parameter controls the size of the batch pull result set , To improve read performance .

TDengineWriter

  Supported features include ：

• Support  
  OpenTSDB
    Of  json  Line protocol in format , have access to  TDengine  Of  Schemaless  Write in  TDengine.

• Support batch write , Can pass  
  batchSize
    Parameter controls the number of batch writes , This improves write performance .

at present  TDengine  Has joined  DataX  List of officially supported databases .

Now? , If the user wants to change the history  Database（ such as  MySQL、OpenTSDB  etc. ） Data from to  TDengine, Or will  TDengine  Data export in , You can use it  DataX  To implement the .

More usage details , You can refer to “ be based on  DataX  Of  TDengine  Data migration tools ” One article .

be based on  DataX  Of  TDengineWriter  and  TDengineReader  And the latest developments , Please pay attention to ：

https://github.com/taosdata/DataX

Want to know more

TDengine Database

Specific details of , Welcome to

GitHub

View the relevant source code on .