[solution] final app status- undefined, exitcode- 16

Abnormal information

ERROR yarn.ApplicationMaster: RECEIVED SIGNAL TERM
INFO yarn.ApplicationMaster: Final app status: UNDEFINED, exitCode: 16, (reason: Shutdown hook called before final status was reported.)
INFO util.ShutdownHookManager: Shutdown hook called

My scene is ：java The code uses SparkLauncher Submit spark Mission jar Package to yarn The cluster to perform , The above exception occurred . Sui google The right way to open .

terms of settlement

1. stay ${HADOOP_HOME}/etc/hadoop/yarn-site.xml Add the following configuration ：

<property>
    <name>yarn.nodemanager.aux-services</name>
    <value>spark_shuffle,mapreduce_shuffle</value>
</property>
<property>
    <name>yarn.nodemanager.aux-services.spark_shuffle.class</name>
    <value>org.apache.spark.network.yarn.YarnShuffleService</value>
</property>

2. see ${HADOOP_HOME}/share/yarn/lib Is there any spark-*-yarn-shuffle.jar This jar package , * It's the version number . without , hold spark Corresponding jar Copy the package to this directory ：

cd ${HADOOP_HOME}/share/yarn/lib
cp ${SPARK_HOME}/yarn/spark-2.4.8-yarn-shuffle.jar ./

3. restart yarn.

cd ${HADOOP_HOME}
./sbin/stop-yarn.sh
./sbin/start-yarn.sh

Since it is Google Of , Then attach someone else's link .