【解决】Final app status- UNDEFINED, exitCode- 16

异常信息

ERROR yarn.ApplicationMaster: RECEIVED SIGNAL TERM
INFO yarn.ApplicationMaster: Final app status: UNDEFINED, exitCode: 16, (reason: Shutdown hook called before final status was reported.)
INFO util.ShutdownHookManager: Shutdown hook called

我的场景是：java 代码中使用 SparkLauncher 提交 spark 任务的 jar 包到 yarn 集群执行，出现了上述异常。遂 google 正确的打开方式。

解决办法

1. 在 ${HADOOP_HOME}/etc/hadoop/yarn-site.xml 中添加如下配置：

<property>
    <name>yarn.nodemanager.aux-services</name>
    <value>spark_shuffle,mapreduce_shuffle</value>
</property>
<property>
    <name>yarn.nodemanager.aux-services.spark_shuffle.class</name>
    <value>org.apache.spark.network.yarn.YarnShuffleService</value>
</property>

2. 查看 ${HADOOP_HOME}/share/yarn/lib 目录下是否有 spark-*-yarn-shuffle.jar 这个jar包， * 是版本号。如果没有，把 spark中对应的 jar包拷贝到该目录下：

cd ${HADOOP_HOME}/share/yarn/lib
cp ${SPARK_HOME}/yarn/spark-2.4.8-yarn-shuffle.jar ./

3. 重启 yarn。

cd ${HADOOP_HOME}
./sbin/stop-yarn.sh
./sbin/start-yarn.sh

既然是Google的，那就附上人家链接。