5 environment construction spark on yarn

1 SparkOnYarn The essence

2 To configure spark on yarn Environmental Science

This part refers directly to <<spark Deployment documents .doc>> that will do

3 Deployment mode DeployMode

4 Spark On Yarn Summary of the two models

5 Extended reading ： Detailed flow of the two modes

5 summary

1 SparkOnYarn The essence ？
Master from ResourceManager Instead of
Worker from NodeManager Instead of
Driver Can run in a container (Cluster Pattern ) Or in the client process (Client Pattern )
Executor All running in YARN In the container provided
2 Why Spark On YARN?
Improve resource utilization , Already exist YARN Let's Spark received YARN Scheduling can better control
Improve resource utilization and facilitate management