当前位置:网站首页>Spark独立集群动态上线下线Worker节点
Spark独立集群动态上线下线Worker节点
2022-07-06 09:29:00 【若苗瞬】
(一)下线Worker节点
我的操作:关机就是了……
反正工作自然会交给其它节点做的。
当然其实应该这样,登录到需要下线的节点主机:
执行脚本关闭本机Worker:
$> $SPARK_HOME/sbin/stop-worker.sh
注意还有一个stop-workers.sh
,是用来关闭全部Worker的,别搞错了。
看清楚多一个s!!!
如果你在Master上不小心执行了这个多个S的脚本(Master到所有Worker一定都配置了ssh密钥登录),
那么就全关完了……
关机前:
关机后:
过一段时间DEAD的Worker就会消失(集群总要留点时间看它能不能重新连接嘛)。
.
(二)上线Worker节点
登录到需要上线的节点主机:
执行脚本启动本机Worker,并连接到Spark Master:
$> $SPARK_HOME/sbin/start-worker.sh spark://vm00:7077
注意还有一个start-workers.sh
脚本。区别同上。
最后,如果是一个新的Worker,别忘了在$SPARK_CONF_DIR
里面编辑workers
(或slaves
)。
加入新Worker的主机名,方便下次集群全部一起启动。
对了,新启动的Worker有新的ID。
呃……完。
边栏推荐
- 简单尝试DeepFaceLab(DeepFake)的新AMP模型
- Click QT button to switch qlineedit focus (including code)
- Suffix expression (greed + thinking)
- Pytorch extract skeleton (differentiable)
- Codeforces Round #802(Div. 2)A~D
- Research Report on market supply and demand and strategy of double drum magnetic separator industry in China
- Openwrt source code generation image
- Bisphenol based CE Resin Industry Research Report - market status analysis and development prospect forecast
- Pull branch failed, fatal: 'origin/xxx' is not a commit and a branch 'xxx' cannot be created from it
- Codeforces Round #801 (Div. 2)A~C
猜你喜欢
Pytorch extract skeleton (differentiable)
2078. Two houses with different colors and the farthest distance
顺丰科技智慧物流校园技术挑战赛(无t4)
OneForAll安装使用
1013. Divide the array into three parts equal to and
Codeforces Round #802(Div. 2)A~D
Raspberry pie 4b64 bit system installation miniconda (it took a few days to finally solve it)
1689. Ten - the minimum number of binary numbers
Codeforces round 797 (Div. 3) no f
Problem - 922D、Robot Vacuum Cleaner - Codeforces
随机推荐
Generate random password / verification code
业务系统兼容数据库Oracle/PostgreSQL(openGauss)/MySQL的琐事
(POJ - 3258) River hopper (two points)
1323. Maximum number of 6 and 9
Opencv learning log 28 -- detect the red cup cover
1005. Maximized array sum after K negations
SF smart logistics Campus Technology Challenge (no T4)
计算时间差
Study notes of Tutu - process
Research Report on market supply and demand and strategy of double drum magnetic separator industry in China
Li Kou - 298th weekly match
Socket communication
Pull branch failed, fatal: 'origin/xxx' is not a commit and a branch 'xxx' cannot be created from it
Maximum product (greedy)
Kubernetes集群部署
Bidirectional linked list - all operations
Acwing: Game 58 of the week
(lightoj - 1370) Bi shoe and phi shoe (Euler function tabulation)
QT按钮点击切换QLineEdit焦点(含代码)
Market trend report, technical innovation and market forecast of double-sided foam tape in China