当前位置：网站首页>Flink's sink: a preliminary study

Flink's sink: a preliminary study

2020-11-08 16:12:00 【Programmer Xinchen】

Want to make APP Same thing as WeChat , Can run small programs smoothly ？ | Experience will send you to Xinjiang 、 Huawei 、 Cherry keyboard ！>>>

Welcome to visit mine GitHub

https://github.com/zq2599/blog_demos

Content ： All original articles classified summary and supporting source code , involve Java、Docker、Kubernetes、DevOPS etc. ;

About sink

The following figure comes from Flink official , In the red box is sink, Visible real-time data from Source Start at , stay Transformation After the business logic is completed in sink end , therefore sink It can be used to process the calculation results , For example, console output or save database ： Insert picture description here

About 《Flink Of sink actual combat 》 Series articles

This article is about 《Flink Of sink actual combat 》 The first chapter of , In order to have a preliminary understanding of sink, Through the basic API and addSink Method analysis and research , Lay a good foundation for the subsequent coding practice ;

Full series Links

Start with an instance code

Here is a simple flink Application code , The red box print The way is sink operation ：
Here's the official sink The way , All are DataStream Class API, It can be directly called sink, Just in the code print It's one of them ：
Let's look at the picture above API Source code , First look at print Method , stay DataStream.java in , as follows , It's actually called addSink Method , Participation is PrintSinkFunction：
The other is often used API yes writeAsText, Source code is as follows , Called writeUsingOutputFormat Method ：
track writeUsingOutputFormat Discovery is also called addSink, Participation is OutputFormatSinkFunction：
print and writeAsText Behind the call addSink, So another common writeAsCsv Methods? ？ It's a call addSink？ Open it and see , and writeAsText Same call writeUsingOutputFormat, In this method, you are calling addSink：
in summary ,data sink The key is addSink Input , namely SinkFunction Interface implementation , Use class diagrams to visualize common sink How is ability realized ：

Insert picture description here 8. From the figure above, we can see the abstract class RichSinkFunction With all kinds of sink Ability is closely related , We should focus on it , Show method signature on class diagram , Here's the picture ： 9. As shown in the figure above ,RichSinkFunction There is no content in itself , But it does SinkFunction, Inherit AbstractRichFunction, yes RichFunction and SinkFunction The combination of these two characteristics ; 10. RichFunction The characteristics of are in front of 《Flink Of DataSource Trilogy 》 We already know , It's about resources open and close; 11. SinkFunction The characteristics of ？ It's obviously used to process the calculation results , The class diagram shows two invoke Method , Take a look at the official PrintSinkFunction.java： Insert picture description here 12. writer.write(record) The source code is PrintSinkOutputWriter.java, As shown below ：

Summary

thus , We're right Flink Of sink With a basic understanding ：

Be responsible for the processing of real-time calculation results ( Like output or persistence );
The main implementation is to call DataStream.addSink Method ;
Various sink The realization of ability , The main way is to achieve addSink The interface defined by the input parameter of the method ;

Later chapters , Together with sink Let's practice the coding of the aspect , The direction of actual combat ： Experience the official sink Ability , Customize sink Ability to achieve ;