I am trying to upload data into bigquery partitioned table using dataflow .I have successfully uploaded data on date basis and fetched this data on monthly basis using bigquery but my moto is to upload data on monthly basis/yearly basis. Is there any way to do that using dataflow.
我正在尝试使用数据流将数据上传到bigquery分区表。我已成功上传日期数据并使用bigquery每月获取此数据，但我的moto是每月/每年上传数据。有没有办法使用数据流来做到这一点。
1 个解决方案
#1
1
You can have "monthly" partitions by using the date for the start of each month. For August, for example, you would store everything in the yourtable$20170801 partition. You would need to have some application-side logic to determine the appropriate $YYYYmmdd suffix for the table into which you are writing using Dataflow.
您可以使用每月开头的日期来获得“每月”分区。例如，对于八月，您可以将所有内容存储在yourtable $ 20170801分区中。您需要使用一些应用程序端逻辑来确定使用Dataflow编写的表的相应$ YYYYmmdd后缀。
#1
1
You can have "monthly" partitions by using the date for the start of each month. For August, for example, you would store everything in the yourtable$20170801 partition. You would need to have some application-side logic to determine the appropriate $YYYYmmdd suffix for the table into which you are writing using Dataflow.
您可以使用每月开头的日期来获得“每月”分区。例如，对于八月，您可以将所有内容存储在yourtable $ 20170801分区中。您需要使用一些应用程序端逻辑来确定使用Dataflow编写的表的相应$ YYYYmmdd后缀。