I've got a dataflow job that pulls messages off of several Google Pub/Sub topics, does some parallel processing on the individual elements contained within those messages, then passes the collection on for further consumption by various resources. I'd like to put together a Stackdriver dashboard showing how many individual elements have been processed for each topic. Each ParDo step outputs a PCollection.
我有一个数据流工作，从几个Google Pub / Sub主题中提取消息，对这些消息中包含的各个元素进行一些并行处理，然后传递集合以供各种资源进一步使用。我想整理一个Stackdriver仪表板，显示每个主题处理了多少个别元素。每个ParDo步骤输出PCollection。
I've set up a dashboard using ElementCount, but I'm only able to filter by job, not by step. If I mouseover the line in the chart produced using ElementCount, I can see counts for every single step. Indeed, it does appear that the metrics for these are being reported, as I can use the gcloud commandline utility in the following manner:
我已经使用ElementCount设置了一个仪表板，但我只能按工作过滤，而不是按步骤过滤。如果我将鼠标悬停在使用ElementCount生成的图表中的行，我可以看到每一步的计数。实际上，似乎正在报告这些指标，因为我可以通过以下方式使用gcloud命令行实用程序：
gcloud beta dataflow metrics list [jobid] --filter ElementCount
...
name:
context:
original_name: extract_value_topic_1/Map-out0-ElementCount
output_user_name: extract_value_topic_1/Map-out0
name: ElementCount
origin: dataflow/v1b3
scalar: 7000
updateTime: '2017-05-03T18:13:22.804Z'
---
name:
context:
original_name: extract_value_topic_2/Map-out0-ElementCount
output_user_name: extract_value_topic_2/Map-out0
name: ElementCount
origin: dataflow/v1b3
scalar: 12000
updateTime: '2017-05-03T18:13:22.804Z'
I have several of these, but I don't see a straightforward way of building Stackdriver charts based on them (aside from logging to the console for every element processed then using that to generate a log-based metric, but that seems like it'd be extremely inefficient on a number of levels.) Am I missing something? How would one create a chart based on these ElementCounts?
我有其中的几个，但我没有看到基于它们构建Stackdriver图表的简单方法（除了为处理的每个元素记录到控制台，然后使用它来生成基于日志的度量，但这看起来像' d在许多层面上效率极低。）我错过了什么吗？如何根据这些ElementCounts创建图表？
Edit: Additionally, if I open up the Metrics Explorer I can enter dataflow/job/element_count into the search box then pcollection into the filter box, but I'm unable to build a dashboard with this chart in it as the filter selection in the dashboard chart builder does not allow for filtering by pcollection.
编辑：此外，如果我打开Metrics Explorer，我可以在搜索框中输入dataflow / job / element_count，然后将pcollection输入到过滤器框中，但是我无法在其中构建一个带有此图表的仪表板作为过滤器选择仪表板图表构建器不允许通过pcollection进行过滤。
1 个解决方案
#1
0
Unfortunately, you currently cannot build a dashboard with a filter on a metric label. As you noticed, the new (Beta) Metric Explorer provides the filtering functionality and the Stackdriver team is actively working on providing that functionality to the dashboard charts as well. I will follow up if I receive any further updates or details from the Stackdriver team. --Andrea
遗憾的是，您目前无法在度量标签上构建带过滤器的仪表板。正如您所注意到的，新的（Beta）Metric Explorer提供了过滤功能，Stackdriver团队也在积极地为仪表板图表提供该功能。如果我收到Stackdriver团队的任何进一步更新或详细信息，我会跟进。 --Andrea
#1
0
Unfortunately, you currently cannot build a dashboard with a filter on a metric label. As you noticed, the new (Beta) Metric Explorer provides the filtering functionality and the Stackdriver team is actively working on providing that functionality to the dashboard charts as well. I will follow up if I receive any further updates or details from the Stackdriver team. --Andrea
遗憾的是，您目前无法在度量标签上构建带过滤器的仪表板。正如您所注意到的，新的（Beta）Metric Explorer提供了过滤功能，Stackdriver团队也在积极地为仪表板图表提供该功能。如果我收到Stackdriver团队的任何进一步更新或详细信息，我会跟进。 --Andrea