当前位置:网站首页>Spark independent cluster dynamic online and offline worker node
Spark independent cluster dynamic online and offline worker node
2022-07-06 16:37:00 【Ruo Miaoshen】
List of articles
( One ) Offline Worker node
My operation : Just turn it off ……
Anyway, the work will naturally be handed over to other nodes .
Of course, it should be like this , Log in to the node host that needs to be offline :
Execute script to shut down the machine Worker:
$> $SPARK_HOME/sbin/stop-worker.sh
Notice there's another one stop-workers.sh
, It is used to close all Worker Of , Don't get me wrong .
See clearly one more s!!!
If you are in the Master I accidentally executed this multiple S Script for (Master To all Worker It must be configured ssh Key login ),
Then it's all closed ……
Before shutdown :
After shutting down :
After a while DEAD Of Worker Will disappear ( The cluster must take some time to see if it can reconnect ).
.
( Two ) go online Worker node
Log in to the node host that needs to go online :
Execute script to start the machine Worker, And connect to Spark Master:
$> $SPARK_HOME/sbin/start-worker.sh spark://vm00:7077
Notice there's another one start-workers.sh
Script . The difference is the same as above .
Last , If it's a new one Worker, Don't forget it. $SPARK_CONF_DIR
Editor inside workers
( or slaves
).
Add new Worker The host name , It is convenient for all clusters to start together next time .
by the way , A new start Worker There are new ones ID.
Uh …… End .
边栏推荐
- 拉取分支失败,fatal: ‘origin/xxx‘ is not a commit and a branch ‘xxx‘ cannot be created from it
- (lightoj - 1354) IP checking (Analog)
- 软通乐学-js求字符串中字符串当中那个字符出现的次数多 -冯浩的博客
- 图图的学习笔记-进程
- China double brightening film (dbef) market trend report, technical dynamic innovation and market forecast
- How to insert mathematical formulas in CSDN blog
- Chapter 7__ consumer_ offsets topic
- useEffect,函数组件挂载和卸载时触发
- Pull branch failed, fatal: 'origin/xxx' is not a commit and a branch 'xxx' cannot be created from it
- QNetworkAccessManager实现ftp功能总结
猜你喜欢
sublime text 代码格式化操作
软通乐学-js求字符串中字符串当中那个字符出现的次数多 -冯浩的博客
原生js实现全选和反选的功能 --冯浩的博客
新手必会的静态站点生成器——Gridsome
Spark独立集群动态上线下线Worker节点
QT simulates mouse events and realizes clicking, double clicking, moving and dragging
分享一个在树莓派运行dash应用的实例。
Codeforces round 797 (Div. 3) no f
Local visualization tools are connected to redis of Alibaba cloud CentOS server
OneForAll安装使用
随机推荐
QWidget代码设置样式表探讨
QNetworkAccessManager实现ftp功能总结
Useeffect, triggered when function components are mounted and unloaded
Raspberry pie 4B installation opencv3.4.0
Market trend report, technological innovation and market forecast of double door and multi door refrigerators in China
Share an example of running dash application in raspberry pie.
AcWing——第55场周赛
读取和保存zarr文件
使用jq实现全选 反选 和全不选-冯浩的博客
第五章 Yarn资源调度器
QT implementation fillet window
Advancedinstaller安装包自定义操作打开文件
Double specific tyrosine phosphorylation regulated kinase 1A Industry Research Report - market status analysis and development prospect prediction
Raspberry pie 4b64 bit system installation miniconda (it took a few days to finally solve it)
计算时间差
MariaDB的安装与配置
Codeforces Round #799 (Div. 4)A~H
Research Report on market supply and demand and strategy of Chinese table lamp industry
QT实现窗口置顶、置顶状态切换、多窗口置顶优先关系
Base dice (dynamic programming + matrix fast power)