The open source project Taier version 1.1 was officially released, and the list of new functions is fast

2022 year 5 month 8 Japan ,Taier 1.1  Official release ！

This version update is for  Flink Upgrade your support to  Flink1.12, Support for multiple Stream type Mission , The new version of the usage document has been pushed in the community , You can download it at any time .

github Address ：

https://github.com/DTStack/Taier

gitee Address ：

https://gitee.com/dtstack_dev_0/taier

Taier 1.1 Version introduction

Taier It's a distributed Visualization DAG Task scheduling system , It is an important hub of the overall architecture of the data platform in several stacks , Be responsible for scheduling a large amount of daily tasks .

It aims to reduce ETL Development costs , Improve the stability of big data platform , So that big data developers can Taier Develop business logic directly , Instead of caring about the complex dependencies of tasks and the architecture implementation of the underlying big data platform , Focus more on the business .

Taier1.0 Version on 2022 year 2 Published in , stay 1.0 The day after the release ,1.1 The iteration of version has been put on the agenda , It was officially released yesterday .

This version is updated , Focus on solving Taier The problem of the fitness of ： Yes Flink Of Support Upgrade to 1.12;Taier Medium Spark SQL and Flink SQL The two components also support users to customize functions in tasks , Obviously let Taier Of Malleability With better play ; newly added There are many examples Hive SQL Types of tasks 、 Powerful functions such as real-time task operation and maintenance .

This version update is for Taier The inherent advantages of , It also strengthens and improves the user experience , Further refinement to improve product performance .

Taier 1.1  Function details

01  Yes Flink Version supports upgrading to 1.12

Taier As a distributed Visualization DAG Task scheduling system , use ChunJun As a distributed data synchronization tool .1.1 Version will be Flink Version upgrade to 1.12 , Support ChunJun 1.12 New in version transformer Operators and all Flink Original grammar and Function

02  Data synchronization supports script mode 、 The incremental synchronization

Data synchronization task except Wizard mode ,1.1 New data synchronization script mode in version . Script mode through json How to configure , Don't need to rely on datasourcex Supported data sources , Directly through json Submit tasks in the way of configuration , Script mode json Seamless format compatibility  ChunJun  Data format , Users can debug the data synchronization of various data sources through script mode .

03   newly added Hive SQL

Apache Hive It's a system built on Hadoop Top level data warehouse , A structured data file can be mapped to a database table , And provide simple SQL Query function , Can be SQL The statement is converted to MapReduce Task to run .Taier1.1 Version added Hive SQL , Support docking Hive Different versions of .

04   Add a variety of real-time tasks

Add real-time acquisition task , Support will MySQL、Oracle Synchronize your data to Kafka.

newly added Flink SQL Mission , Pass standard SQL Semantic development helps to quickly complete the configuration of data tasks .

05   Add real-time task operation and maintenance

You can view the relevant index information of real-time tasks and the detailed log information of tasks through the real-time operation and maintenance center .

06   Support user-defined functions

User defined functions （User Defined Function, abbreviation UDF）, Is that users use system functions in addition to , Self created function , Used to meet personalized computing needs . User defined functions are similar to ordinary system functions in use .

at present Taier1.1 edition Spark SQL and Flink SQL All tasks support custom functions .

07   New Diablo theme online

Taier Development interface Diablo theme online , Provide a variety of topic switching , Users can choose .Taier 1.1 Users can choose freely Dark Default Subject or Light Default Theme and interface style , The user experience has improved significantly .

The future planning

Taier As a new open source project , Our iterations and updates are ongoing , follow-up Taier Will be in Extensibility 、 User autonomy Continue to explore and expand , For example, we are trying to make users based on Taier To customize the types of tasks you need, and so on .

Taier Every progress of is inseparable from the help and advice of community developers , I hope you will keep an eye on it , and Taier Move on together , Keep climbing new peaks ！