Per the question re: making a custom Sink to update Schemas dynamically in Dataflow, I was wondering if the Patch
operation is exposed in BigQueryIO API somewhere?
根据问题re:使用自定义接收器在Dataflow中动态更新Schema,我想知道Patch操作是否在某个地方的BigQueryIO API中公开?
This is a crucial piece of updating schemas on the fly. We are merging schemas as they come in in backwards compatible ways.
这是动态更新模式的关键部分。我们正在合并模式,因为它们以向后兼容的方式进入。
1 个解决方案
#1
1
This is not in the SDK itself, but can use a standard pattern of side-effecting from a DoFn. Specifically, you'll want to make sure that the BigQuery client you create is marked transient, as DoFns must be serializable.
这不在SDK本身,但可以使用DoFn的标准副作用模式。具体来说,您需要确保您创建的BigQuery客户端标记为瞬态,因为DoFns必须是可序列化的。
class PatchFn extends DoFn<> {
private transient BigQuery bq;
...
}
#1
1
This is not in the SDK itself, but can use a standard pattern of side-effecting from a DoFn. Specifically, you'll want to make sure that the BigQuery client you create is marked transient, as DoFns must be serializable.
这不在SDK本身,但可以使用DoFn的标准副作用模式。具体来说,您需要确保您创建的BigQuery客户端标记为瞬态,因为DoFns必须是可序列化的。
class PatchFn extends DoFn<> {
private transient BigQuery bq;
...
}