遇到一个很头疼的问题，使用jdbc方式连接hive，在进行
INSERT OVERWRITE DIRECTORY '*******路径'
select MediaId(url),ChannelId(url),PostionId(url),OrderId(url),BusinessId(url),CityId(url),UniqueID(mobile,ip,ua),OsName(ua),PhoneName(ua),AreaCode(mobile,ip),ShowNum(url),ClickNum(url)
from logs
where year='####' and month='##' and day='##' and hour='##' and MediaId(url) is not null and status in(###,###)
执行的时候报错控制台输出错误信息是
java.sql.SQLException: Query returned non-zero code: 9, cause: FAILED: Execution Error, return code 2 from org.apache.hadoop.hive.ql.exec.MapRedTask
at org.apache.hadoop.hive.jdbc.HivePreparedStatement.executeImmediate(HivePreparedStatement.java:177)
at org.apache.hadoop.hive.jdbc.HivePreparedStatement.execute(HivePreparedStatement.java:128)
at com.****.db.DBCon.execute(DBCon.java:131)
at com.****.etl.Etl.doETL(Etl.java:54)
at com.****.aplus.UploadLogs.main(UploadLogs.java:86)
Caused by: HiveServerException(message:Query returned non-zero code: 9, cause: FAILED: Execution Error, return code 2 from org.apache.hadoop.hive.ql.exec.MapRedTask, errorCode:9, SQLState:08S01)
at org.apache.hadoop.hive.service.ThriftHive$execute_result.read(ThriftHive.java:1318)
at org.apache.thrift.TServiceClient.receiveBase(TServiceClient.java:78)
at org.apache.hadoop.hive.service.ThriftHive$Client.recv_execute(ThriftHive.java:105)
at org.apache.hadoop.hive.service.ThriftHive$Client.execute(ThriftHive.java:92)
at org.apache.hadoop.hive.jdbc.HivePreparedStatement.executeImmediate(HivePreparedStatement.java:175)
... 4 more
查看50030页面
，其中将hive语句转换为MapReduce时出现异常，查看jobtracker日志发现如下错误
java.lang.RuntimeException: Error in configuring object
at org.apache.hadoop.util.ReflectionUtils.setJobConf(ReflectionUtils.java:93)
at org.apache.hadoop.util.ReflectionUtils.setConf(ReflectionUtils.java:64)
at org.apache.hadoop.util.ReflectionUtils.newInstance(ReflectionUtils.java:117)
at org.apache.hadoop.mapred.MapTask.runOldMapper(MapTask.java:432)
at org.apache.hadoop.mapred.MapTask.run(MapTask.java:372)
at org.apache.hadoop.mapred.Child$4.run(Child.java:255)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:396)
at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1059)
at org.apache.hadoop.mapred.Child.main(Child.java:249)
Caused by: java.lang.reflect.InvocationTargetException
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
at java.lang.reflect.Method.invoke(Method.java:597)
at org.apache.hadoop.util.ReflectionUtils.setJobConf(ReflectionUtils.java:88)
... 9 more
Caused by: java.lang.RuntimeException: Error in configuring object
at org.apache.hadoop.util.ReflectionUtils.setJobConf(ReflectionUtils.java:93)
at org.apache.hadoop.util.ReflectionUtils.setConf(ReflectionUtils.java:64)
at org.apache.hadoop.util.ReflectionUtils.newInstance(ReflectionUtils.java:117)
at org.apache.hadoop.mapred.MapRunner.configure(MapRunner.java:34)
... 14 more
Caused by: java.lang.reflect.InvocationTargetException
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
at java.lang.reflect.Method.invoke(Method.java:597)
at org.apache.hadoop.util.ReflectionUtils.setJobConf(ReflectionUtils.java:88)
... 17 more
Caused by: java.lang.RuntimeException: Map operator initialization failed
at org.apache.hadoop.hive.ql.exec.ExecMapper.configure(ExecMapper.java:121)
... 22 more
Caused by: org.apache.hadoop.hive.ql.metadata.HiveException: java.lang.NullPointerException
at org.apache.hadoop.hive.ql.exec.FilterOperator.initializeOp(FilterOperator.java:76)
at org.apache.hadoop.hive.ql.exec.Operator.initialize(Operator.java:357)
at org.apache.hadoop.hive.ql.exec.Operator.initialize(Operator.java:433)
at org.apache.hadoop.hive.ql.exec.Operator.initializeChildren(Operator.java:389)
at org.apache.hadoop.hive.ql.exec.TableScanOperator.initializeOp(TableScanOperator.java:166)
at org.apache.hadoop.hive.ql.exec.Operator.initialize(Operator.java:357)
at org.apache.hadoop.hive.ql.exec.MapOperator.initializeOp(MapOperator.java:427)
at org.apache.hadoop.hive.ql.exec.Operator.initialize(Operator.java:357)
at org.apache.hadoop.hive.ql.exec.ExecMapper.configure(ExecMapper.java:98)
... 22 more
Caused by: java.lang.NullPointerException
at org.apache.hadoop.hive.ql.exec.FunctionRegistry.isStateful(FunctionRegistry.java:1127)
at org.apache.hadoop.hive.ql.exec.ExprNodeGenericFuncEvaluator.<init>(ExprNodeGenericFuncEvaluator.java:107)
at org.apache.hadoop.hive.ql.exec.ExprNodeEvaluatorFactory.get(ExprNodeEvaluatorFactory.java:48)
at org.apache.hadoop.hive.ql.exec.ExprNodeGenericFuncEvaluator.<init>(ExprNodeGenericFuncEvaluator.java:97)
at org.apache.hadoop.hive.ql.exec.ExprNodeEvaluatorFactory.get(ExprNodeEvaluatorFactory.java:48)
at org.apache.hadoop.hive.ql.exec.FilterOperator.initializeOp(FilterOperator.java:70)
... 30 more
大家有遇到过这种错误吗？帮忙解决下 谢谢
6 个解决方案
#1
忘了补充下 add jar ****和create temporary function ****都做了，添加的类名是没问题的
#2
不能就这么沉下去啊 ，大神们来看看啊
#3
我顶 我顶 我顶
#4
我这段时间实习时也出现这个错误，你用的也是hadoop 1.0吧。 这个关闭重新启动后就又能正常工作了，使用一段时间后又会出现这个情况。网上说是1.0的个BUG，缓存进程释放不了，只有重启后才会释放。 查看job日志时确实报的 内存溢出，重启后执行，job跑过去了。
#5
检查了下代码最终发现代码中使用到for循环sql连接、add jar 和create funtion写在for外面，而在for循环里面有重新建立sql连接去执行sql，导致找不到udf，谢谢楼上热情解答
#6
能把你eclipse 里添加jar和创建function 还有执行sql的语句给我吗？？？
#1
忘了补充下 add jar ****和create temporary function ****都做了，添加的类名是没问题的
#2
不能就这么沉下去啊 ，大神们来看看啊
#3
我顶 我顶 我顶
#4
我这段时间实习时也出现这个错误，你用的也是hadoop 1.0吧。 这个关闭重新启动后就又能正常工作了，使用一段时间后又会出现这个情况。网上说是1.0的个BUG，缓存进程释放不了，只有重启后才会释放。 查看job日志时确实报的 内存溢出，重启后执行，job跑过去了。
#5
检查了下代码最终发现代码中使用到for循环sql连接、add jar 和create funtion写在for外面，而在for循环里面有重新建立sql连接去执行sql，导致找不到udf，谢谢楼上热情解答
#6
能把你eclipse 里添加jar和创建function 还有执行sql的语句给我吗？？？