Eclipse和Scala安装见:https://blog.csdn.net/qq_25948717/article/details/80404158‘
https://blog.csdn.net/qq_25948717/article/details/80758713
去官网下载插件:http://scala-ide.org/download/prev-stable.html
解压后,把其features和plugins下的所有jar包都copy到eclipse下对应的features和plugins目录下
进入到eclipse安装目录:../eclipse/eclipse启动eclipse
Eclipse打开之后,从file-->new-->others-->Scala Wizards-->Scala Project新建一个project,我命名为SparkWordCount
右击SparkWordCount,-->properties-->Java Build Path-->Libraries-->Add External Jars,选择spark安装目录下lib/的assembly jar包
创建class:
package spark
import org.apache.spark.SparkContext
import org.apache.spark.SparkContext._
import org.apache.spark.SparkConf
class wordcount {
def main(args: Array[String]) {
val infile = "/public/users/yexin/SparkWorks/data/test.txt" // Should be some file on your system
val conf = new SparkConf().setAppName("word count")
val sc = new SparkContext(conf)
val indata = sc.textFile(infile, 2).cache()
val words = indata.flatMap(line => line.split(" ")).map(word => (word,1)).reduceByKey((a,b) => (a+b))
words.saveAsTextFile("/public/users/yexin/SparkWorks/data/output")
println("All words are counted!")
}
}