Installation and deployment of Flink

Want to make APP Same thing as WeChat , Can run small programs smoothly ？ | Experience will send you to Xinjiang 、 Huawei 、 Cherry keyboard ！>>>

One . Flink The download

Installation package download address , Select the corresponding Hadoop Of Flink Version download

 

 

 

 Flink  There are three deployment modes , Namely  Local、Standalone Cluster  and  Yarn Cluster.

Two . Local Pattern

about  Local  Mode ,JobManager  and  TaskManager  Will share one  JVM  To complete  Workload. If you want to validate a simple application ,Local  Patterns are the most convenient . Most of the practical applications use  Standalone  perhaps  Yarn Cluster, and local Mode just unzip the installation package and start it up （./bin/start-local.sh） that will do , There's no demonstration here .

3、 ... and . Standalone HA Pattern

Standalone Pattern as the name suggests , It is scheduled to execute on the local cluster , Independent of external scheduling mechanisms such as YARN,  Generally, it needs to be configured as HA, prevent Jobmanager All of a sudden , The whole cluster or task execution fails . So let's talk about that Standalone HA Mode building and installation

When Flink Program runtime , If jobmanager collapse , Then the whole program will fail . In order to prevent jobmanager A single point of failure , With the help of zookeeper The coordination mechanism of , Can achieve jobmanager Of HA To configure —-1 Lord （leader） More from （standby）. there HA Configuration only involves standalone Pattern ,yarn The mode will not be considered for the moment . 

In this case, plan Jobmanager:hadoop01,hadoop02( One active, One standby);Taskmanager：hadoop02,hadoop03;zookeeper colony

1.   Cluster deployment planning

The name of the node	master	worker	zookeeper
hadoop01	master	worker	zookeeper
hadoop02	master	worker	zookeeper
hadoop03		woker	zookeeper

 

 

 

 

 

 

 

2.  decompression

[hadoop@hadoop01 apps]$ tar -zxvf flink-1.7.2-bin-scala_2.11.tgz -C ./

  [hadoop@hadoop01 apps]$ ls
  azkaban flink-1.7.2 flink-1.7.2-bin-scala_2.11.tgz flume-1.8.0 hadoop-2.7.4 jq kafka_2.11-0.11 zkdata zookeeper-3.4.10 zookeeper.out

3.  Modify the configuration file

To configure masters file

　　 This file is used to specify the master node and its web Access port , For clusters Jobmanager,vi masters, add to master:8081

[hadoop@hadoop01 conf]$ vim masters hadoop01:8081
hadoop02:8081

To configure slaves file , This file is used to specify the slave node , For clusters taskManager. Add the following

[hadoop@hadoop01 conf]$ vim slaveshadoop01hadoop02hadoop03

The configuration file flink-conf.yaml