Please pardon my basic question. I need to use a few classes off the latest code of Apache Spark for Spark Streaming so I can use them in my test classes e.g. MicroBatchExecution.
请原谅我的基本问题。我需要使用Apache Spark for Spark Streaming的最新代码中的几个类,所以我可以在我的测试类中使用它们,例如MicroBatchExecution。
However, the Maven dependencies available don't have a few classes such as the one in the link.
但是,可用的Maven依赖项没有一些类,例如链接中的类。
How do I access these?
我如何访问这些?
1 个解决方案
#1
2
The current stable release of Spark - 2.2.1 - was released on Dec 01, 2017.
目前稳定版本的Spark - 2.2.1 - 已于2017年12月1日发布。
The class you referenced (MicroBatchExecution) was added in this commit on Dec 14th so it is not available in the current release.
您引用的类(MicroBatchExecution)已于12月14日在此提交中添加,因此在当前版本中不可用。
Since that class has been merged to master you can expect that it will be made available in the next stable release (2.3.0).
由于该类已合并为master,因此可以预期它将在下一个稳定版本(2.3.0)中提供。
I don't think Spark publish snapshots to a publically available repo so your options are:
我不认为Spark将快照发布到公开可用的仓库,因此您的选择是:
- Build the JAR yourself
自己构建JAR
Or
- Wait for the current SNAPSHOT version (
2.3.0-SNAPSHOT) to be published as an official version (2.3.0)
等待当前的SNAPSHOT版本(2.3.0-SNAPSHOT)作为正式版本发布(2.3.0)
#1
2
The current stable release of Spark - 2.2.1 - was released on Dec 01, 2017.
目前稳定版本的Spark - 2.2.1 - 已于2017年12月1日发布。
The class you referenced (MicroBatchExecution) was added in this commit on Dec 14th so it is not available in the current release.
您引用的类(MicroBatchExecution)已于12月14日在此提交中添加,因此在当前版本中不可用。
Since that class has been merged to master you can expect that it will be made available in the next stable release (2.3.0).
由于该类已合并为master,因此可以预期它将在下一个稳定版本(2.3.0)中提供。
I don't think Spark publish snapshots to a publically available repo so your options are:
我不认为Spark将快照发布到公开可用的仓库,因此您的选择是:
- Build the JAR yourself
自己构建JAR
Or
- Wait for the current SNAPSHOT version (
2.3.0-SNAPSHOT) to be published as an official version (2.3.0)
等待当前的SNAPSHOT版本(2.3.0-SNAPSHOT)作为正式版本发布(2.3.0)