[email protected] word count jar / hadoop jar test word count 1.0 SNAPSHOT.jar wyh.test.mapreduce.WordCountMain ...">

当前位置:网站首页>运行mapreduce任务缺失setJarByClass()报错找不到类

运行mapreduce任务缺失setJarByClass()报错找不到类

2022-06-10 15:29:00 QYHuiiQ

报错信息中提示说是类找不到,但是直观来看,我的自定义Map类是在的,但是还是报错:

[[email protected] word_count_jar]# hadoop jar test_word_count-1.0-SNAPSHOT.jar wyh.test.mapreduce.WordCountMain
22/05/27 07:34:08 INFO client.RMProxy: Connecting to ResourceManager at hadoop01/192.168.126.132:8032
22/05/27 07:34:09 WARN mapreduce.JobResourceUploader: No job jar file set.  User classes may not be found. See Job or Job#setJar(String).
22/05/27 07:34:09 INFO input.FileInputFormat: Total input paths to process : 1
22/05/27 07:34:10 INFO mapreduce.JobSubmitter: number of splits:1
22/05/27 07:34:10 INFO mapreduce.JobSubmitter: Submitting tokens for job: job_1653650542989_0001
22/05/27 07:34:10 INFO mapred.YARNRunner: Job jar is not present. Not adding any jar to the list of resources.
22/05/27 07:34:11 INFO impl.YarnClientImpl: Submitted application application_1653650542989_0001
22/05/27 07:34:11 INFO mapreduce.Job: The url to track the job: http://hadoop01:8088/proxy/application_1653650542989_0001/
22/05/27 07:34:11 INFO mapreduce.Job: Running job: job_1653650542989_0001
22/05/27 07:34:24 INFO mapreduce.Job: Job job_1653650542989_0001 running in uber mode : false
22/05/27 07:34:24 INFO mapreduce.Job:  map 0% reduce 0%
22/05/27 07:34:30 INFO mapreduce.Job: Task Id : attempt_1653650542989_0001_m_000000_0, Status : FAILED
Error: java.lang.RuntimeException: java.lang.ClassNotFoundException: Class wyh.test.mapreduce.WordCountMapper not found
        at org.apache.hadoop.conf.Configuration.getClass(Configuration.java:2267)
        at org.apache.hadoop.mapreduce.task.JobContextImpl.getMapperClass(JobContextImpl.java:187)
        at org.apache.hadoop.mapred.MapTask.runNewMapper(MapTask.java:747)
        at org.apache.hadoop.mapred.MapTask.run(MapTask.java:341)
        at org.apache.hadoop.mapred.YarnChild$2.run(YarnChild.java:164)
        at java.security.AccessController.doPrivileged(Native Method)
        at javax.security.auth.Subject.doAs(Subject.java:422)
        at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1754)
        at org.apache.hadoop.mapred.YarnChild.main(YarnChild.java:158)
Caused by: java.lang.ClassNotFoundException: Class wyh.test.mapreduce.WordCountMapper not found
        at org.apache.hadoop.conf.Configuration.getClassByName(Configuration.java:2171)
        at org.apache.hadoop.conf.Configuration.getClass(Configuration.java:2265)
        ... 8 more

原因在于我的hadoop是集群环境,而在job主类中我没有设置要运行的类,添加如下配置即可:

job.setJarByClass(WordCountMain.class);

 再次运行,报错消失。

 

原网站

版权声明
本文为[QYHuiiQ]所创,转载请带上原文链接,感谢
https://blog.csdn.net/QYHuiiQ/article/details/125009902