当前位置:网站首页>Why is the cluster mode of spark on Yan better than the client mode

Why is the cluster mode of spark on Yan better than the client mode

2022-07-06 21:43:00 Big data Xiaochen

  • 1、 Because in this way , bring Driver towards Executor Our communications are all in the same cluster network segment , High communication efficiency . And vice versa yarn-client Mode Driver In the submitted network segment ,Executor We are in yarn The network segment of the cluster ,2 The network segments may be different , Communication efficiency is low .

  • 2、 Generally, the machines submitted in the enterprise are so 1-2 platform , If everyone uses the same machine client Modes run their respective Spark application , Then it will cause all Driver Processes are started on this machine , The burden on this machine is heavy , All tasks are affected . And random and idle WorkerNode Scattered all Driver, Load balancing .

 

原网站

版权声明
本文为[Big data Xiaochen]所创,转载请带上原文链接,感谢
https://yzsam.com/2022/02/202202131122070927.html