4 senior experts share the insider architecture design and implementation principles of Flink technology with years of experience in large factories

Have you heard of flink?

Apache Flink is an open source stream processing framework developed by the Apache Software Foundation. Its core is a distributed stream data stream engine written in Java and Scala.Flink executes arbitrary streaming data programs in a data-parallel and pipelined manner, and Flink's pipelined runtime system can execute batch and stream processing programs.In addition, Flink's runtime itself supports the execution of iterative algorithms.

Since everyone has a simple understanding of what flink is, today I will share with you a technical document of Flink technology insider architecture design and implementation principle shared by four senior experts who have summarized and shared their experience in large factories for many years.,Hope everyone likes it!!

What is the design and implementation principle of Flink technology insider architecture?

This is a work that analyzes the architecture design and implementation principles of Flink by analyzing the implementation of each functional module of Flink from the perspective of source code.

It will guide readers to better perform performance tuning, availability assurance, performance optimization and secondary development of Flink.The four authors are very senior Flink experts, and some of the authors are maintainers and reformers of Flink source code. This article summarizes their large-scale Flink practical experience in Alibaba, Ant Group, Didi and other enterprises.

Introduction

The full text consists of 11 chapters, the main contents are as follows.

Chapter 1Preparation before reading Flink source code,

Mainly introduces the acquisition, compilation, debugging of Flink source code, as well as its directory structure, design concept and basic architecture.In addition, the design differences between Flink and Hadoop MapReduce and Spark are also analyzed.

Second~Chapter 8 Flink function module source code and architecture analysis,

From the perspective of source code, analyzes Flink's programming model and API, runtime components and communication, state management and fault tolerance, task submission and execution, Flink network stack, Flink Connector, deployment mode and other functional modules.Strive to allow readers to deeply understand the principles of Flink, rather than just follow the book to read the Flink source code.The principles and ideas are not limited to a certain framework and technology. After understanding the principles and ideas of Flink, it can be extended to other frameworks and technologies.

9th～Chapter 10 Flink Table, SQL and Flink CEP,

The programming method gradually evolves from the original Map-Reduce-based task mode to SQL. SQL means standard and efficiency.These two chapters focus on the implementation process of StreamTableEnvironment, the parsing process of SQL, the internal implementation principle of Table Connector and Flink CEP.

Chapter 11Flink monitoring,

 Mainly explains Flink's monitoring indicators, common system indicators and monitoring system construction, as well as common positioning methods.

Friends who want to learn this technical document "Flink Technology Insider Architecture Design and Implementation Principles" should pay attention!

Add

maxiaoye5397

to get