oozie中的蜂巢动作失败（在cloudera CDH 4.1.1上）

When I run a hive script with from oozie od CDH 4.1.1

当我用oozie od CDH 4.1.1运行一个配置单元脚本时

The run fails with:

运行失败：

Error Code  JA018
Error Message   org/apache/hadoop/hive/cli/CliDriver

Details
Property    Value
External Id job_201211281608_0112
External Status FAILED/KILLED
Data    None
Start time   Sat, 01 Dec 2012 03:02:37
End time     Sat, 01 Dec 2012 03:03:07
Id  0000007-121128160850795-oozie-oozi-W@ExchangeRateTest
Retries 0
TrackerUri  overlord-datanode1:8021
Transition  kill

Googling JA018 reveals only one cryptic hint: JA018 is output directory exists error in workflow map-reduce action.

谷歌搜索JA018只显示一个神秘的提示：JA018输出目录存在工作流映射中的错误 - 减少动作。

I copied my hiv-site.xml to HDFS and set in the workflow.xml: oozie.hive.defaults /user/hue/oozie/workspaces/overlord-oozie-1/hive-site.xml

我将我的hiv-site.xml复制到HDFS并在workflow.xml中设置：oozie.hive.defaults /user/hue/oozie/workspaces/overlord-oozie-1/hive-site.xml

Here is the complete workflow.xml:

这是完整的workflow.xml：

<workflow-app name="HiveTest" xmlns="uri:oozie:workflow:0.4">
    <start to="ExchangeRateTest"/>
    <action name="ExchangeRateTest">
        <hive xmlns="uri:oozie:hive-action:0.2">
            <job-tracker>${jobTracker}</job-tracker>
            <name-node>${nameNode}</name-node>
            <prepare>
                <delete path="${nameNode}${jobOutput}"/>
            </prepare>
            <configuration>
                <property>
                    <name>oozie.use.system.libpath</name>
                    <value>true</value>
                </property>
                <property>
                    <name>oozie.hive.defaults</name>
                    <value>/user/hue/oozie/workspaces/_overlord_-oozie-1/hive-site.xml</value>
                </property>
            </configuration>
            <script>/user/hue/oozie/workspaces/_overlord_-oozie-1/03_update_exchange_rates_final.hive</script>
              <param>OUTPUT=${jobOutput}</param>
        </hive>
        <ok to="end"/>
        <error to="kill"/>
    </action>
    <kill name="kill">
        <message>Action failed, error message[${wf:errorMessage(wf:lastErrorNode())}]</message>
    </kill>
    <end name="end"/>
</workflow-app>

How can get this to work?

怎么能让这个工作？

1 个解决方案

#1

The class org/apache/hadoop/hive/cli/CliDriver is required for execution of a Hive Action. This much is obvious from the error message. This class is within this jar file: hive-cli-0.7.1-cdh3u5.jar. (In my case cdh3u5 in my cloudera version).

执行Hive Action需要类org / apache / hadoop / hive / cli / CliDriver。从错误消息中可以明显看出这一点。这个类在这个jar文件中：hive-cli-0.7.1-cdh3u5.jar。 （就我的情况而言，我的cloudera版本中的cdh3u5）。

Oozie checks for this jar in the ShareLib directory. The location of this directory is usually configured in hive-site.xml, with the property name as oozie.service.WorkflowAppService.system.libpath, so Oozie should find the jar easily.

Oozie在ShareLib目录中检查此jar。此目录的位置通常在hive-site.xml中配置，其属性名称为oozie.service.WorkflowAppService.system.libpath，因此Oozie应该可以轻松找到jar。

But in my case, hive-site.xml did not include this property, so Oozie didn't know where to look for this jar, hence the java.lang.NoClassDefFoundError.

但在我的情况下，hive-site.xml没有包含这个属性，所以Oozie不知道在哪里寻找这个jar，因此java.lang.NoClassDefFoundError。

To resolve this, I had to include a parameter in my job.properties file to point oozie to the location of the ShareLib directory, as follows: oozie.libpath=${nameNode}/user/oozie/share/lib. (depends on where SharedLib directory is configured on your cluster).

要解决这个问题，我必须在job.properties文件中包含一个参数，以便将oozie指向ShareLib目录的位置，如下所示：oozie.libpath = $ {nameNode} / user / oozie / share / lib。 （取决于在群集上配置SharedLib目录的位置）。