I'm having a trouble for two days already, and can't find any solutions.
我已经有两天麻烦了,找不到任何解决方案。
I'm getting
ClassNotFoundException: org.apache.zeppelin.spark.ZeppelinContext
when using input value inside spark DataFrame's filter method.
在spark DataFrame的filter方法中使用输入值时。
val city = z.select("City",cities).toString
oDF.select("city").filter(r => city.equals(r.getAs[String]("city"))).count()
I even tried copying the input value to another val with
我甚至尝试将输入值复制到另一个val
new String(bytes[])
but still get the same error.
但仍然得到相同的错误。
The same code work seamlessly if instead of getting the value from z.select I declare as a String literal
如果不是从z.select获取值,我将无缝地工作,我声明为String文字
city: String = "NY" org.apache.spark.SparkException: Job aborted due to stage failure: Task 0 in stage 49.0 failed 4 times, most recent failure: Lost task 0.3 in stage 49.0 (TID 277, 10.6.60.217): java.lang.NoClassDefFoundError: Lorg/apache/zeppelin/spark/ZeppelinContext;
city:String =“NY”org.apache.spark.SparkException:作业因阶段失败而中止:阶段49.0中的任务0失败4次,最近失败:阶段49.0中失去任务0.3(TID 277,10.6.60.217): java.lang.NoClassDefFoundError:Lorg / apache / zeppelin / spark / ZeppelinContext;
1 个解决方案
#1
0
You are taking this in the wrong direction:
你正朝着错误的方向走:
val city="NY"
gives you a scala String with NY as the string, but when you say
给你一个以NY作为字符串的scala字符串,但是当你说
z.select("City",cities)
then this returns you dataFrame and then you are converting this object to String using method toString and then trying to compare.!
然后这会返回dataFrame,然后使用方法toString将此对象转换为String,然后尝试比较。
This wont work !
这不行!
What you can do is either collect one dF and then pass the scala String accordingly into the other Df or you can do a join if you want to do it for multiple values.
您可以做的是收集一个dF然后将scala字符串相应地传递到另一个Df中,或者如果要为多个值执行连接,则可以执行连接。
But this approach will not work for sure !
但这种方法无法确定!
#1
0
You are taking this in the wrong direction:
你正朝着错误的方向走:
val city="NY"
gives you a scala String with NY as the string, but when you say
给你一个以NY作为字符串的scala字符串,但是当你说
z.select("City",cities)
then this returns you dataFrame and then you are converting this object to String using method toString and then trying to compare.!
然后这会返回dataFrame,然后使用方法toString将此对象转换为String,然后尝试比较。
This wont work !
这不行!
What you can do is either collect one dF and then pass the scala String accordingly into the other Df or you can do a join if you want to do it for multiple values.
您可以做的是收集一个dF然后将scala字符串相应地传递到另一个Df中,或者如果要为多个值执行连接,则可以执行连接。
But this approach will not work for sure !
但这种方法无法确定!