当前位置:网站首页>【解决】Final app status- UNDEFINED, exitCode- 16
【解决】Final app status- UNDEFINED, exitCode- 16
2022-07-07 01:59:00 【骑着蜗牛向前跑】
异常信息
ERROR yarn.ApplicationMaster: RECEIVED SIGNAL TERM
INFO yarn.ApplicationMaster: Final app status: UNDEFINED, exitCode: 16, (reason: Shutdown hook called before final status was reported.)
INFO util.ShutdownHookManager: Shutdown hook called
我的场景是:java 代码中使用 SparkLauncher
提交 spark 任务的 jar 包到 yarn 集群执行,出现了上述异常。遂 google 正确的打开方式。
解决办法
- 在
${HADOOP_HOME}/etc/hadoop/yarn-site.xml
中添加如下配置:
<property>
<name>yarn.nodemanager.aux-services</name>
<value>spark_shuffle,mapreduce_shuffle</value>
</property>
<property>
<name>yarn.nodemanager.aux-services.spark_shuffle.class</name>
<value>org.apache.spark.network.yarn.YarnShuffleService</value>
</property>
- 查看
${HADOOP_HOME}/share/yarn/lib
目录下是否有spark-*-yarn-shuffle.jar
这个jar包,*
是版本号。如果没有,把 spark中对应的 jar包拷贝到该目录下:
cd ${HADOOP_HOME}/share/yarn/lib
cp ${SPARK_HOME}/yarn/spark-2.4.8-yarn-shuffle.jar ./
- 重启 yarn。
cd ${HADOOP_HOME}
./sbin/stop-yarn.sh
./sbin/start-yarn.sh
边栏推荐
- C language interview to write a function to find the first occurrence of substring m in string n.
- Audio distortion analysis of DSP and DAC based on adau1452
- Experience sharing of contribution of "management world"
- JMeter's own functions are not enough? Why don't you develop one yourself
- Haqi projection Black Horse posture, avec seulement six mois de forte pénétration du marché des projecteurs de 1000 yuans!
- From "running distractor" to data platform, Master Lu started the road of evolution
- Talking about reading excel with POI
- Navicat导入15G数据报错 【2013 - Lost connection to MySQL server during query】 【1153:Got a packet bigger】
- K8s running Oracle
- You don't know the complete collection of recruitment slang of Internet companies
猜你喜欢
3428. 放苹果
HKUST & MsrA new research: on image to image conversion, fine tuning is all you need
FlexRay通信协议概述
雷特智能家居龙海祁:从专业调光到全宅智能,20年专注成就专业
693. 行程排序
Common problems of caching in high concurrency scenarios
软件测试的几个关键步骤,你需要知道
Rk3399 platform development series explanation (WiFi) 5.52. Introduction to WiFi framework composition
What are the classic database questions in the interview?
Overview of FlexRay communication protocol
随机推荐
牛客小白月赛52 E.分组求对数和(二分&容斥)
当我们谈论不可变基础设施时,我们在谈论什么
Force deduction 62 different paths (the number of all paths from the upper left to the lower right of the matrix) (dynamic planning)
How to set up in touch designer 2022 to solve the problem that leap motion is not recognized?
哈趣投影黑馬之姿,僅用半年强勢突圍千元投影儀市場!
JVM monitoring and diagnostic tools - command line
JMeter's own functions are not enough? Why don't you develop one yourself
Redis (I) -- getting to know redis for the first time
A very good JVM interview question article (74 questions and answers)
3428. 放苹果
你不知道的互联网公司招聘黑话大全
C language interview to write a function to find the first public string in two strings
字符串常量与字符串对象分配内存时的区别
C language sorting (to be updated)
JWT 认证
Change the original style of UI components
那些自损八百的甲方要求
Ha Qu projection dark horse posture, only half a year to break through the 1000 yuan projector market!
postgresql 数据库 timescaledb 函数time_bucket_gapfill()报错解决及更换 license
Bypass open_ basedir