Wondering if there is some kind of "hook" to place a piece of code that would be executed when apache beam pipeline is being shutdown (for whatever reason - crash, cancel)
想知道是否存在某种“挂钩”来放置一段代码，该代码将在apache beam pipeline关闭时执行（无论出于何种原因 - 崩溃，取消）
I need to delete a subscription on pubsub topic each time Dataflow is stopped.
每次Dataflow停止时，我都需要删除pubsub主题上的订阅。
1 个解决方案
#1
0
Apache Beam is not naturally suited for this sort of flow. For this you may want to look at an orchestration engine, such as Apache Airflow.
Apache Beam自然不适合这种流程。为此，您可能需要查看编排引擎，例如Apache Airflow。
With Airflow you should be able to schedule any sort of script to run after a Beam pipeline finishes/fails/is cancelled, etc. Take a look at it!
使用Airflow，您应该能够安排任何类型的脚本在Beam管道完成/失败/取消后运行等等。看看它！
#1
0
Apache Beam is not naturally suited for this sort of flow. For this you may want to look at an orchestration engine, such as Apache Airflow.
Apache Beam自然不适合这种流程。为此，您可能需要查看编排引擎，例如Apache Airflow。
With Airflow you should be able to schedule any sort of script to run after a Beam pipeline finishes/fails/is cancelled, etc. Take a look at it!
使用Airflow，您应该能够安排任何类型的脚本在Beam管道完成/失败/取消后运行等等。看看它！