scala集合转化为DS/DF
case class TestPerson(name: String, age: Long, salary: Double)
val tom = TestPerson("Tom Hanks",,35.5)
val sam = TestPerson("Sam Smith",,40.5)
val PersonList = mutable.MutableList[TestPerson]()
PersonList += tom
PersonList += sam val personDS = PersonList.toDS()
println(personDS.getClass)
personDS.show() val personDF = PersonList.toDF()
println(personDF.getClass)
personDF.show()
personDF.select("name", "age").show()
输出结果：
class org.apache.spark.sql.Dataset +---------+---+------+
| name|age|salary|
+---------+---+------+
|Tom Hanks| | 35.5|
|Sam Smith| | 40.5|
+---------+---+------+ class org.apache.spark.sql.DataFrame +---------+---+------+
| name|age|salary|
+---------+---+------+
|Tom Hanks| | 35.5|
|Sam Smith| | 40.5|
+---------+---+------+ +---------+---+
| name|age|
+---------+---+
|Tom Hanks| |
|Sam Smith| |
+---------+---+
参考资料：
https://*.com/questions/39397652/convert-scala-list-to-dataframe-or-dataset