运行hadoop 报错 No job jar file set. User classes may not be found. See Job or Job#setJar(String)

时间:2025-02-18 08:01:12

如下创建主类程序

public class JobMain extends Configured implements Tool {
@Override
public int run(String[] args) throws Exception {
//1:创建job对象
Job job = Job.getInstance(super.getConf(), "mapreduce_sort");
...
}

结果单机上跑没有问题,但是放在集群上就会出问题
报如下错误

20/12/19 19:03:26 INFO : Connecting to ResourceManager at node01/192.168.177.101:8032
20/12/19 19:03:26 WARN : No job jar file set.  User classes may not be found. See Job or Job#setJar(String).
20/12/19 19:03:26 INFO : Total input paths to process : 1
20/12/19 19:03:30 INFO : number of splits:1
20/12/19 19:03:30 INFO : Submitting tokens for job: job_1608426890333_0007
20/12/19 19:03:30 INFO : Job jar is not present. Not adding any jar to the list of resources.
20/12/19 19:03:30 INFO : Submitted application application_1608426890333_0007
20/12/19 19:03:30 INFO : The url to track the job: http://node01:8088/proxy/application_1608426890333_0007/
20/12/19 19:03:30 INFO : Running job: job_1608426890333_0007
20/12/19 19:04:10 INFO : Job job_1608426890333_0007 running in uber mode : true
20/12/19 19:04:10 INFO :  map 100% reduce 0%
20/12/19 19:04:11 INFO :  map 100% reduce 100%
20/12/19 19:04:12 INFO : Job job_1608426890333_0007 failed with state FAILED due to: Task failed task_1608426890333_0007_m_000000
Job failed as tasks failed. failedMaps:1 failedReduces:0

20/12/19 19:04:12 INFO : Counters: 17
        Job Counters 
                Failed map tasks=1
                Failed reduce tasks=1
                Launched map tasks=1
                Launched reduce tasks=1
                Other local map tasks=1
                Total time spent by all maps in occupied slots (ms)=1065
                Total time spent by all reduces in occupied slots (ms)=155
                TOTAL_LAUNCHED_UBERTASKS=2
                NUM_UBER_SUBMAPS=1
                NUM_UBER_SUBREDUCES=1
                NUM_FAILED_UBERTASKS=2
                Total time spent by all map tasks (ms)=1065
                Total time spent by all reduce tasks (ms)=155
                Total vcore-milliseconds taken by all map tasks=1065
                Total vcore-milliseconds taken by all reduce tasks=155
                Total megabyte-milliseconds taken by all map tasks=1090560
                Total megabyte-milliseconds taken by all reduce tasks=158720

查阅了一下资料,原因是找不到job任务运行的资源,如map类、reduce类等。

public class JobMain extends Configured implements Tool {

    @Override
    public int run(String[] args) throws Exception {
        //1:创建job对象
        Job job = Job.getInstance(super.getConf(), "mapreduce_sort");
        //只要是Initmain类所在包下的任一个类名都可以,默认为当前 job所在类名
        job.setJarByClass(JobMain.class);
        ...

再回到集群上运行就好了。