Flink deployment mode (II) - three deployment modes of yarn

be based on flink1.15.0

Session Mode on yarn

Turn on flink yarn-session colony

./bin/yarn-session.sh -d \
-Dtaskmanager.numberOfTaskSlots=5 \
-Dflink.cluster.name=flink-yarn-session \
-Dyarn.application.name="flink-yarn-session" \
-Dresourcemanager.taskmanager-timeout=30000 \
-Drest.flamegraph.enabled=true 

Submit tasks

• Mode one , Don't specify yarn app-id Submit

./bin/flink run ./examples/streaming/TopSpeedWindowing.jar
perhaps

• Mode two , Specify manually flink colony ,application_1652928425471_0282 yes flink colony yarn app-id

./bin/flink run -t yarn-session -d
-Dyarn.application.id=application_1652928425471_0285
./examples/streaming/TopSpeedWindowing.jar

• Mode three , stay flink web Interface upload jar Package submission

Enter the client interface

./bin/yarn-session.sh -id application_1652928425471_0252
Stop the task gracefully

echo “stop” | ./bin/yarn-session.sh -id application_1652928425471_0245

Force to kill the mission

yarn application -kill application_1652928425471_0266

NOTE： The only way to start a cluster is jobmanager, Submit job Just to go yarn Apply for resource launch taskmanager, Task completion automatically releases taskmanager

Per-Job Mode on yarn

start-up perjob colony

./bin/flink run -t yarn-per-job --detached \
-Dyarn.application.name="flink-yarn-perjob" \
-Dflink.cluster.name=flink-yarn-perjob \
-Dtaskmanager.numberOfTaskSlots=5 \
./examples/streaming/TopSpeedWindowing.jar

View the job
./bin/flink list -t yarn-per-job -Dyarn.application.id=application_1652928425471_0252
Cancel a certain job

./bin/flink cancel -t yarn-per-job -Dyarn.application.id=application_1652928425471_0252
./bin/flink cancel -t yarn-per-job -Dyarn.application.id=application_1652928425471_0252 ac02048e90c0ad7ec370542b412e498c

**NOTE：** stay perjob Pattern , Cancel a certain job It is equivalent to stopping the cluster

Application Mode on yarn

flink1.11 Later introduced
start-up application colony

./bin/flink run-application \
-t yarn-application \
-Dyarn.application.name="flink-yarn-application" \
-Dflink.cluster.name=flink-yarn-perjob \
-Dtaskmanager.numberOfTaskSlots=5 \
-Djobmanager.memory.process.size=1024m \
-Dtaskmanager.memory.process.size=1024m \
-Drest.flamegraph.enabled=true \
./examples/streaming/TopSpeedWindowing.jar

View the job

./bin/flink list -t yarn-application -Dyarn.application.id=application_1652928425471_0268

Cancel a certain job

./bin/flink cancel -t yarn-application -Dyarn.application.id=application_1652928425471_0268
./bin/flink cancel -t yarn-application -Dyarn.application.id=application_1652928425471_0254 17e0ba867df35d98c2d70be8103fff2e

**NOTE：** stay perjob Pattern , Cancel a certain job It is equivalent to stopping the cluster

In addition, it can be combined with yarn Rely on pre upload settings , To optimize job Submission effect of