如何在Dataflow中使用BigQuery Standard SQL？

I would like to run a simple query using BigQuery Standard SQL within dataflow but I can't find where to enable this option. How can I do that?

我想在数据流中使用BigQuery Standard SQL运行一个简单的查询，但我找不到启用此选项的位置。我怎样才能做到这一点？

pipeline.apply(Read.named(metricName + " Read").fromQuery("select * from table1 UNION DISTINCT select * from table2"));

When I try to run it I receive the error:

当我尝试运行它时，我收到错误：

2016-07-20T13:35:22.543Z: Error:   (6e0ad847af078af9): Workflow failed. Causes: (fe6c7bcb1a35a057): S01:warehouse_handled_returns Read/DataflowPipelineRunner.BatchBigQueryIONativeRead+ParMultiDo(FormatData)+warehouse_handled_returns Write/DataflowPipelineRunner.BatchBigQueryIOWrite/DataflowPipelineRunner.BatchBigQueryIONativeWrite failed., (7f29f1d9435d27bc): BigQuery execution failed., (7f29f1d9435d2823): Error:
Message: Encountered "" at line 23, column 27.

HTTP Code: 400

3 个解决方案

#1

You can now use standard SQL with Dataflow.

您现在可以将标准SQL与Dataflow一起使用。

https://cloud.google.com/dataflow/model/bigquery-io

PCollection<TableRow> weatherData = p.apply(
BigQueryIO.Read
.named("ReadYearAndTemp")
.fromQuery("SELECT year, mean_temp FROM `samples.weather_stations`")
.usingStandardSql();

#2

Until DataFlow formally supports BigQuery Standard SQL, one workaround is to start query with the following comment:

在DataFlow正式支持BigQuery Standard SQL之前，一种解决方法是使用以下注释开始查询：

#StandardSQL

This will instruct BigQuery to use Standard SQL instead of Legacy SQL

这将指示BigQuery使用标准SQL而不是旧版SQL

#3

Dataflow SDK for Java supports BigQuery's Standard SQL dialect beginning in version 1.8.0.

Dataflow SDK for Java支持从1.8.0版开始的BigQuery标准SQL方言。

#1