在Python中的数据流中写入动态目标到云存储

I was trying to read from a big file in cloud storage and shard them according to a given field.

我试图从云存储中的大文件中读取并根据给定字段对它们进行分片。

I'm planning to Read | Map(lambda x: (x[key field], x)) | GroupByKey | Write to file with the name of the key field.

我打算阅读| Map（lambda x：（x [key field]，x））| GroupByKey |使用键字段的名称写入文件。

However I couldn't find a way to write dynamically to cloud storage. Is this functionality supported?

但是我找不到动态写入云存储的方法。是否支持此功能？

Thank you, Yiqing

谢谢你，易清

1 个解决方案

#1

Yes, you can use the FileSystems API to create the files.

是的，您可以使用FileSystems API来创建文件。