Cloud Dataflow BQ输出挂起作业，用于TLS握手错误

My Cloud Dataflow job hangs.

我的Cloud Dataflow作业挂起。

Pipeline:

管道：

Pipeline p = Pipeline.create(options);
p.apply(TextIO.Read.named("ReadFiles").from(options.getInputFile()).withCompressionType(CompressionType.GZIP))
.apply(new ProcessRecords(eventType))
.apply(BigQueryIO.Write.to(options.getOutput()).withSchema(options.getBigQuerySchema())
        .withCreateDisposition(BigQueryIO.Write.CreateDisposition.CREATE_NEVER)
        .withWriteDisposition(BigQueryIO.Write.WriteDisposition.WRITE_TRUNCATE));
p.run();

From worker logs all I can see is

从工人日志我可以看到的是

Kubelet is healthy?: true
TLS handshake error from 172.17.0.2:41759: EOF
shutdown[7231]: shutting down for system halt

The same job when I output to a GCS file it works fine.

输出到GCS文件时的工作相同，工作正常。

JobID: 2016-12-28_15_23_54-4009520083656203603

JobID：2016-12-28_15_23_54-4009520083656203603

I have tried this multiple times but every time its the same issue.

我已多次尝试过，但每次都是同样的问题。

1 个解决方案

#1

There was nothing wrong with the pipeline construct code or data flow. TextIO write operations are faster than BigQueryIO write.

管道构造代码或数据流没有任何问题。 TextIO写操作比BigQueryIO写快。

In case you every notice all your steps to simply move to "Running" state and not show the rate of processing or anything else, there is nothing wrong with it.

如果您每次都注意到所有步骤都只是转到“运行”状态并且没有显示处理速率或其他任何内容，那么它没有任何问题。

Digging into the worker logs might show TLS handshake error or shutdown or system halt. Thats absolutely normal. Just let the job run further and it should be fine.

挖掘工作日志可能会显示TLS握手错误或关闭或系统暂停。这绝对正常。只是让工作进一步运行，它应该没问题。

#1