Hive:posexplode v.s. explode 实现列转行-lateral view posexplode(array)示例

时间:2024-04-19 17:46:09
select id,tim,single_id,single_tim from test.a 
lateral view posexplode(split(id,',')) t as single_id_index, single_id
lateral view posexplode(split(tim,',')) t as single_yim_index, single_tim
where single_id_index = single_yim_index;
-- 输出:
id	tim	single_id	single_tim
a,b,c,d	2:00,3:00,4:00,5:00	a	2:00
a,b,c,d	2:00,3:00,4:00,5:00	b	3:00
a,b,c,d	2:00,3:00,4:00,5:00	c	4:00
a,b,c,d	2:00,3:00,4:00,5:00	d	5:00
f,b,c,d	1:10,2:20,3:30,4:40	f	1:10
f,b,c,d	1:10,2:20,3:30,4:40	b	2:20
f,b,c,d	1:10,2:20,3:30,4:40	c	3:30
f,b,c,d	1:10,2:20,3:30,4:40	d	4:40

注意事项

  1. lateral view [outer] explode(),如果要拆分的字段有null值,需要使用lateral view outer 替代,避免数据缺失,参考文档见:[https://blog.****.net/weixin_45857425/article/details/117933039];(https://blog.****.net/weixin_45857425/article/details/117933039)
  2. lateral view的位置是from后where条件前;
  3. 生成的虚拟表的表名不可省略;
  4. from后可带多个lateral view;

参考文档:
重点阅读:Hive–sql中的explode()函数和posexplode()函数
explode和lateral view explode使用记录