从有界数据流管道到pub / sub的确定性发布顺序

时间:2021-11-23 14:41:44

Is there a way to publish messages from a bounded dataflow pipeline (batch mode) into PubSubIO.Write in a deterministic order, so that they are delivered in that order, at least approximately. E.g. using timestamp or natural/sorted ordering.

有没有办法以确定的顺序将有限数据流管道(批处理模式)中的消息发布到PubSubIO.Write中,以便按顺序传递它们,至少大约是这样。例如。使用时间戳或自然/排序顺序。

1 个解决方案

#1


3  

There is no way to deterministically order data written into PubSubIO.Write. Even if messages were published into Pub/Sub in order, your subscribers would not receive them in order as Cloud Pub/Sub has no ordering guarantees. In general, it is good to avoid a dependency on order as much as you can with Pub/Sub, though there are some techniques to deal with it. I recommend reading the newly-published Message Ordering doc.

没有办法确定性地将数据写入PubSubIO.Write中。即使消息按顺序发布到Pub / Sub,您的订阅者也不会按顺序接收它们,因为Cloud Pub / Sub没有订购保证。一般来说,尽管有一些技术可以处理它,但最好尽可能避免依赖于Pub / Sub对订单的依赖。我建议您阅读新发布的消息订购文档。

#1


3  

There is no way to deterministically order data written into PubSubIO.Write. Even if messages were published into Pub/Sub in order, your subscribers would not receive them in order as Cloud Pub/Sub has no ordering guarantees. In general, it is good to avoid a dependency on order as much as you can with Pub/Sub, though there are some techniques to deal with it. I recommend reading the newly-published Message Ordering doc.

没有办法确定性地将数据写入PubSubIO.Write中。即使消息按顺序发布到Pub / Sub,您的订阅者也不会按顺序接收它们,因为Cloud Pub / Sub没有订购保证。一般来说,尽管有一些技术可以处理它,但最好尽可能避免依赖于Pub / Sub对订单的依赖。我建议您阅读新发布的消息订购文档。