当前位置:网站首页>Tdengine can read and write through dataX, a data synchronization tool
Tdengine can read and write through dataX, a data synchronization tool
2022-07-05 09:44:00 【Tdengine】
Small T Reading guide :DataX It is an offline data synchronization tool open source by Alibaba cloud / platform , At present, it has supported the efficient data synchronization function including a variety of mainstream heterogeneous data sources . Our R & D team has completed TDengine The adaptation of , Related code has been DataX Officially accepted and incorporated into the trunk .
DataX It is an offline data synchronization tool open source by Alibaba cloud / platform , At present, it has realized the following functions: MySQL、Oracle、OceanBase、SQL Server、PostgreSQL、HDFS、Hive、HBase And other heterogeneous data sources .
DataX The design idea is :
DataX As a data synchronization framework , The synchronization of different data sources is abstracted as reading data from data sources
Reader
plug-in unit , And write data to the targetWriter
plug-in unit , Theoretically DataX The framework can support data synchronization of any data source type . meanwhile DataX Plug in system as a set of ecosystem , Each access to a new set of data sources , The interworking between the data source and the existing data source can be realized .
Based on this design idea , Our R & D team has completed TDengine The adaptation of , Realized TDengineReader
and TDengineWriter
Two plug-ins , And be DataX Official acceptance , Merged into its trunk .
TDengineReader
The features provided include :
- Supported by SQL Data filtering ;
- Perform task segmentation according to time interval ;
- Support TDengine All data types of ;
- Support batch reading , Can pass
batchSize
Parameter controls the size of the batch pull result set , To improve read performance .
TDengineWriter
Supported features include :
- Support OpenTSDB Of json Line protocol in format , have access to TDengine Of Schemaless Write in TDengine.
- Support batch write , Can pass
batchSize
Parameter controls the number of batch writes , This improves write performance .
at present TDengine Has joined DataX List of officially supported databases .
Now? , If the user wants to change the history Database( such as MySQL、OpenTSDB etc. ) Data from to TDengine, Or will TDengine Data export in , You can use it DataX To implement the .
More usage details , You can refer to “ be based on DataX Of TDengine Data migration tools ” One article .
be based on DataX Of TDengineWriter and TDengineReader And the latest developments , Please pay attention to :https://github.com/taosdata/DataX
Want to know more TDengine Database Specific details of , Welcome to GitHub View the relevant source code on .
边栏推荐
- Greendao reported an error in qigsaw, could not init daoconfig
- 搞数据库是不是越老越吃香?
- 【愚公系列】2022年7月 Go教学课程 003-IDE的安装和基本使用
- 百度智能小程序巡檢調度方案演進之路
- 【数组的中的某个属性的监听】
- STM32 simple multi-level menu (array table lookup method)
- Progressive JPEG pictures and related
- LeetCode 496. 下一个更大元素 I
- SQL learning alter add new field
- idea用debug调试出现com.intellij.rt.debugger.agent.CaptureAgent,导致无法进行调试
猜你喜欢
Community group buying has triggered heated discussion. How does this model work?
Can't find the activitymainbinding class? The pit I stepped on when I just learned databinding
E-commerce apps are becoming more and more popular. What are the advantages of being an app?
c语言指针深入理解
OpenGL - Coordinate Systems
Unity SKFramework框架(二十二)、Runtime Console 运行时调试工具
Unity SKFramework框架(二十四)、Avatar Controller 第三人称控制
【阅读笔记】图对比学习 GNN+CL
How to improve the operation efficiency of intra city distribution
Applet data attribute method
随机推荐
LeetCode 31. Next spread
Kotlin introductory notes (V) classes and objects, inheritance, constructors
Kotlin introductory notes (VII) data class and singleton class
如何正确的评测视频画质
C语言-从键盘输入数组二维数组a,将a中3×5矩阵中第3列的元素左移到第0列,第3列以后的每列元素行依次左移,原来左边的各列依次绕到右边
Unity skframework framework (XXIII), minimap small map tool
idea用debug调试出现com.intellij.rt.debugger.agent.CaptureAgent,导致无法进行调试
Cloud computing technology hotspot
A keepalived high availability accident made me learn it again
从“化学家”到开发者,从甲骨文到 TDengine,我人生的两次重要抉择
【对象数组a与对象数组b取出id不同元素赋值给新的数组】
How to empty uploaded attachments with components encapsulated by El upload
[ManageEngine] how to make good use of the report function of OpManager
【el-table如何禁用】
SQL learning alter add new field
Resolve the horizontal (vertical) sliding conflict between viewpager and WebView
[JS sort according to the attributes in the object array]
【sourceTree配置SSH及使用】
【愚公系列】2022年7月 Go教学课程 003-IDE的安装和基本使用
La voie de l'évolution du système intelligent d'inspection et d'ordonnancement des petites procédures de Baidu