The data synchronization tool dataX has officially supported reading and writing tdengine

Small  T  Reading guide ：DataX It is an offline data synchronization tool open source by Alibaba cloud / platform , At present, it has supported the efficient data synchronization function including a variety of mainstream heterogeneous data sources . Our R & D team has completed TDengine The adaptation of , Related code has been DataX Officially accepted and incorporated into the trunk .

DataX（https://github.com/alibaba/DataX  You need to copy the link to the browser to view ） It is an offline data synchronization tool open source by Alibaba cloud / platform , At present, it has realized the following functions: MySQL、Oracle、OceanBase、SQL Server、PostgreSQL、HDFS、Hive、HBase And other heterogeneous data sources .

DataX The design idea is ：

DataX As a data synchronization framework , The synchronization of different data sources is abstracted as reading data from data sources Reader plug-in unit , And write data to the target Writer plug-in unit , Theoretically DataX The framework can support data synchronization of any data source type . meanwhile DataX Plug in system as a set of ecosystem , Each access to a new set of data sources , The interworking between the data source and the existing data source can be realized .

Based on this design idea , Our R & D team has completed TDengine The adaptation of , Realized TDengineReader and TDengineWriter Two plug-ins , And be DataX Official acceptance , Merged into its trunk .

TDengineReader The features provided include ：

1. Supported by SQL Data filtering ;

2. Perform task segmentation according to time interval ;

3. Support TDengine All data types of ;

4.  Support batch reading , Can pass  batchSize  Parameter controls the size of the batch pull result set , To improve read performance .

TDengineWriter Supported features include ：

1. Support OpenTSDB Of json Line protocol in format , have access to TDengine Of Schemaless Write in TDengine.

2.  Support batch write , Can pass  batchSize  Parameter controls the number of batch writes , This improves write performance .

at present TDengine Has joined DataX List of officially supported databases .

Now? , If the user wants to change the history Database（ such as MySQL、OpenTSDB etc. ） Data from to TDengine, Or will TDengine Data export in , You can use it DataX To implement the .

More usage details , You can click at the end of the article Read the original Reference resources “ be based on DataX Of TDengine Data migration tools ” One article .

be based on DataX Of TDengineWriter and TDengineReader And the latest developments , Please pay attention to ：https://github.com/taosdata/DataX（ You need to copy the link to the browser to view ）

---

Click on Read the original , To view more  DataX Use details ！

This article is from WeChat official account. - TDengine（taosdata_news）.
If there is any infringement , Please contact the [email protected] Delete .
Participation of this paper “OSC Source creation plan ”, You are welcome to join us , share .