Flink submitter

flink There are two ways to submit procedures ：
1、Standalone HA
2、Flink on yarn
First, introduce the first method with examples ：
#Standalone HA

package cn.wc

import org.apache.flink.api.java.utils.ParameterTool
import org.apache.flink.streaming.api.scala._

object StreamWordCount01 {
    
  def main(args: Array[String]): Unit = {
    
    // Create an environment 
    val env = StreamExecutionEnvironment.getExecutionEnvironment

    // Reading data Source
    val tool = ParameterTool.fromArgs(args)
    val host = tool.get("host")
    val port = tool.getInt("port")
    //val inputStream = env.socketTextStream("master", 1314)
    val inputStream = env.socketTextStream(host, port)

    // Conversion operation 
    val result = inputStream.flatMap(_.split(" "))
      .map((_, 1))
      .keyBy(_._1)
      .sum(1)

    // Output Sink
    result.print()
    // perform 
    env.execute()
  }
}

1. Start the cluster in the virtual machine first start-cluster.sh
2. stay master Node boot port ：nc -lk 1314

[root@master ~]# nc -lk 1314
aa bb vv
aakk nn ee

3. Type the program as jar package . stay master:8081 Of Web Interface selection Submit new Job -> AddNew, Select the typed jar package .


4. Fill in the full class name 、 Parallelism 、 Parameters . single click Show Plan, You can see this below job The execution calculation of . Priority of parallelism ： Parallelism in code > The parallelism specified when submitting > Parallelism in the configuration file .
Click on Submit Submit the program . Our program began to run .

5. stay Task Manager Click on Task. see Stdout. The result printed by our program will be displayed .


By default, our streaming computing program will not stop automatically . Can be found in Web The interface stops manually
stop . In the top right corner of the Cancel. Or close the port directly .

#Flink on yarn
Use... In actual development Flink on yarn There are many modes
1. start-up yarn, Switch to slave1 node ：
start-yarn.sh
2.yarn-session.sh( Open up resources ) + flink run( Submit tasks )
The first jar Upload package to virtual machine

[root@master ~]# ll
 Total usage  208
drwxr-xr-x. 2 root root     55 8 month   22 20:25 bin
-rw-r--r--. 1 root root 211918 8 month   28 19:17 flink0823-1.0.jar
drwxr-xr-x. 2 root root     70 8 month   23 09:06 script

3. Open up resources ,master node
stay yarn Previous start one Flink conversation , Execute the following command ：
yarn-session.sh -n 2 -tm 800 -s 1 -d
explain :
#-n To apply for 2 A container , This is how many taskmanager
-tm Represent each TaskManager The memory size of
-s Represent each TaskManager Of slots Number
-d Indicates that the later program mode is running

[root@master ~]# yarn-session.sh -n 2 -tm 800 -s 1 -d

4. Open port

[root@master ~]# nc -lk 1314

5.flink run( Submit tasks ),master node

[root@master ~]# flink run -c cn.wc.StreamWordCount01 ./flink0823-1.0.jar --host master --port 1314

6. stay yarn Of 8088 The interface can see Flink session cluster Running .

7、 Go first yarn Of web Interface , adopt yarn Look at the interface again flink Program