以豆瓣电影:霸王别姬为例
其主演存在多个,同时每个演员都有自己的id及name属性,而类型仅仅有name属性不包含id属性,
因此主演对应为Array[struct]类型,可表示为
[{"id":1003494,"name":"张国荣"},
{"id":1050265,"name":"张丰毅"},
{"id":1035641,"name":"巩俐"},
{"id":1000905,"name":"葛优"},
{"id":1011479,"name":"英达"}]
类型对应为Array类型,可表示为
["剧情","爱情","同性"]
生成数据文件,其内容为
1291546|霸王别姬|1003494$张国荣^1050265$张丰毅^1035641$巩俐^1000905$葛优^1011479$英达|剧情^爱情^同性
定义Hive表结构为
CREATE TABLE o_douban(
doubanid bigint,
title string,
starring array<struct<id:bigint,name:string>>,
genre array<string>)
ROW FORMAT DELIMITED
FIELDS TERMINATED BY '|'
COLLECTION ITEMS TERMINATED BY '^'
MAP KEYS TERMINATED BY '$'
将数据文件load进hive表
LOAD DATA LOCAL INPATH '/home/admin/test/douban.txt' OVERWRITE INTO TABLE douban
查询结果为
hive> select doubanid, title, editor, genre from o_douban
> where doubanid = 1291546;
1291546|
霸王别姬|
[{"id":1003494,"name":"张国荣"},
{"id":1050265,"name":"张丰毅"},
{"id":1035641,"name":"巩俐"},
{"id":1000905,"name":"葛优"},
{"id":1011479,"name":"英达"}]|
剧情^爱情^同性