Eclipse + Idea + Maven + Scala + Spark +sbt

时间:2023-03-08 22:02:56

http://jingpin.jikexueyuan.com/article/47043.html

新的scala 编译器idea使用

https://www.jetbrains.com/idea/help/creating-and-running-your-scala-application.html

遇到问题,解决:spark 1.5.2只与scala 2.10.*系列兼容

建立SBT project,scala code一定写在src/main/scala下面,如果没写main,就extend app就可以

https://www.youtube.com/watch?v=TBuX5HtDoT8

Idea 产生 jar文件:

http://blog.****.net/honjane/article/details/40739337

比较规范的IDEA的scala用法参见:

https://www.youtube.com/watch?v=QY8l6puu1x0

手动安装sbt

http://www.scala-sbt.org/0.13/tutorial/zh-cn/Manual-Installation.html

spark cassandra 安装:

https://www.codementor.io/data-science/tutorial/installing-cassandra-spark-linux-debian-ubuntu-14

需要 ./sbt/sbt assembly 之后产生jar文件

为了实现sbt sbt-idea,需要加build.sbt文件

cat ~/.sbt/0.13/plugins/build.sbt
addSbtPlugin("com.github.mpeltonen" % "sbt-idea" % "1.5.1")