Flink weapon: introduction to the open source platform streamx

StreamX brief introduction

StreamX yes Flink & Spark Rapid development of scaffolding , Open source stream batch one-stop big data platform .

StreamX Support Flink Many versions , Support Flink SQL WebIDE, Support Flink SQL check . since 2021 year 3 Open source since January , Provides a range of out of the box Conncetor, Standardized configuration 、 Development 、 test 、 Deploy 、 monitor 、 The whole process of operation and maintenance , Provide scala and java Two sets api, Its ultimate goal is to build a one-stop big data platform , Stream batch integration , Lake warehouse integrated solution .

It can be understood as an open source version 、 Simplified alicloud real-time computing Flink Version or Tencent cloud Flow calculation Oceanus platform .

The architecture is as follows ：

StreamX framework

StreamX features ：

• Development scaffold
• Many versions Flink Support (1.11,x, 1.12.x, 1.13 )
• A series of out of the box connectors
• Support project compilation function (maven compile )
• Online parameter configuration
• Support Applicaion Pattern , Yarn-Per-Job mode
• Quick daily operation ( Mission start-up 、 stop it 、savepoint, from savepoint recovery )
• Support flame chart
• Support notebook( Online task development )
• Project configuration and dependency versioning management
• Support task backup 、 Roll back ( Configure rollback )
• Online management depends on (maven pom) And customization jar
• Customize udf、 Connectors, etc
• Flink SQL WebIDE
• Support catalog、hive
• If the task fails to run, send an alarm message
• Support failed restart retry
• From task Development Stage to Deployment management Full link support
• ...

Reference reading ：

[1] Streamx Github:  https://github.com/streamxhub/streamx

[2] StreamX Official website ：http://www.streamxhub.com/

[3] StreamX documentation : http://www.streamxhub.com/zh/doc/guide/intro/

[4] Baili International StreamX Practice on ：https://mp.weixin.qq.com/s/0WvUiMgzRXRWiaRkiIz7qA