Apache Beam Java SDK和PubSub Source的示例

I'm struggling to get Google Cloud Dataflow Java SDK to work with a PubSub Source. The minimalistic code below always comes back with a 401 (Unauthorized) error. I've tried to figure out how to provide the credentials but couldn't find a way to get it to work. I've used the gcloud auth login beforehand to make sure I'm logged in and the subscription exists.

我很难让Google Cloud Dataflow Java SDK与PubSub Source一起使用。下面的简约代码总是返回401（未授权）错误。我试图找出如何提供凭据，但找不到让它工作的方法。我事先使用了gcloud auth登录来确保我已登录并且订阅存在。

The following error is thrown:

抛出以下错误：

[ERROR] Failed to execute goal org.codehaus.mojo:exec-maven-plugin:1.4.0:java (default-cli) on project my-dataflow: An exception occured while executing the Java class. null: InvocationTargetException: com.google.api.client.googleapis.json.GoogleJsonResponseException: 401 Unauthorized
[ERROR] {
[ERROR] "code" : 401,
[ERROR] "errors" : [ {
[ERROR] "domain" : "global",
[ERROR] "message" : "Request is missing required authentication credential. Expected OAuth 2 access token, login cookie or other valid authentication credential. See https://developers.google.com/identity/sign-in/web/devconsole-project.",
[ERROR] "reason" : "unauthorized"
[ERROR] } ],
[ERROR] "message" : "Request is missing required authentication credential. Expected OAuth 2 access token, login cookie or other valid authentication credential. See https://developers.google.com/identity/sign-in/web/devconsole-project.",
[ERROR] "status" : "UNAUTHENTICATED"
[ERROR] }

My code is as follows:

我的代码如下：

public class MyDataflow{

  public static void main(String[] args) {

    PipelineOptions options = PipelineOptionsFactory.create();

    PubsubOptions dataflowOptions = options.as(PubsubOptions.class);
    dataflowOptions.setStreaming(true);

    Pipeline p = Pipeline.create(options);

    p.apply(PubsubIO.<String>read().subscription("projects/my-project/subscription/my-subscription")
            .withCoder(StringUtf8Coder.of()));

    // Run the pipeline.
    p.run().waitUntilFinish();
  }
}

1 个解决方案

#1

I had to set the GOOGLE_APPLICATION_CREDENTIALS to point to my json credential file.

我必须将GOOGLE_APPLICATION_CREDENTIALS设置为指向我的json凭据文件。

https://developers.google.com/identity/protocols/application-default-credentials

https://developers.google.com/identity/protocols/application-default-credentials