如何在Google Cloud数据流管道项目​​pom中使用beam-sdks-java-core-2.2.0.jar?

时间:2021-09-13 15:37:08

I recently set up a Google Cloud dataflow pipeline project using Google Java eclipse plugin. And the default SKD version is 2.1.0 in pom.

我最近使用Google Java eclipse插件设置了Google Cloud数据流管道项目​​。默认的SKD版本是pom中的2.1.0。

<dependencies>
<dependency>
  <groupId>com.google.cloud.dataflow</groupId>
  <artifactId>google-cloud-dataflow-java-sdk-all</artifactId>
  <version>2.1.0</version>
</dependency>
... ...

And the code can be deployed, but it has NullPointerException exception when inserting data into big query. And then I found the following article, it said the issue is fixed in version 2.2.0, https://issues.apache.org/jira/browse/BEAM-2834.

代码可以部署,但在将数据插入大查询时会出现NullPointerException异常。然后我发现了以下文章,它说该问题已在2.2.0版本中修复,https://issues.apache.org/jira/browse/BEAM-2834。

But I have no idea how to do it. I changed the pom version from 2.1.0 to 2.2.0 and all the dependencies are gone and the error is Missing artifact com.google.cloud.dataflow:google-cloud-dataflow-java-sdk-all:jar:2.2.0 and the versions of google-cloud-dataflow-java-sdk-all-2.1.0.jar and beam-sdks-java-core-2.1.0.jar are the same. I cannot only change beam-sdks-java-core-2.1.0.jar version to 2.2.0

但我不知道该怎么做。我将pom版本从2.1.0更改为2.2.0并且所有依赖项都消失了,错误是Missing artifact com.google.cloud.dataflow:google-cloud-dataflow-java-sdk-all:jar:2.2.0 google-cloud-dataflow-java-sdk-all-2.1.0.jar和beam-sdks-java-core-2.1.0.jar的版本是相同的。我不仅可以将beam-sdks-java-core-2.1.0.jar版本更改为2.2.0

If anybody can show me an example about how to update the jar version or how to fix the NullPointerException exception, that will be a great help for me.

如果有人可以向我展示如何更新jar版本或如何修复NullPointerException异常的例子,那对我来说将是一个很大的帮助。

Thanks a lot in advance.

非常感谢提前。

=============================================

=============================================

This is my pom

这是我的pom

<?xml version="1.0" encoding="UTF-8"?>
<!--~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
  ~ Copyright (C) 2017 Google Inc.
  ~
  ~ Licensed under the Apache License, Version 2.0 (the "License"); you may not
  ~ use this file except in compliance with the License. You may obtain a copy of
  ~ the License at
  ~
  ~ http://www.apache.org/licenses/LICENSE-2.0
  ~
  ~ Unless required by applicable law or agreed to in writing, software
  ~ distributed under the License is distributed on an "AS IS" BASIS, WITHOUT
  ~ WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the
  ~ License for the specific language governing permissions and limitations under
  ~ the License.
  ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~-->
<project xmlns="http://maven.apache.org/POM/4.0.0"
    xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
    xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd">
  <modelVersion>4.0.0</modelVersion>

  <groupId>com.company.product</groupId>
  <artifactId>my-product</artifactId>
  <version>0.0.1-SNAPSHOT</version>

  <properties>
    <project.build.sourceEncoding>UTF-8</project.build.sourceEncoding>
  </properties>

  <repositories>
    <repository>
      <id>ossrh.snapshots</id>
      <name>Sonatype OSS Repository Hosting</name>
      <url>https://oss.sonatype.org/content/repositories/snapshots/</url>
      <releases>
        <enabled>false</enabled>
      </releases>
      <snapshots>
        <enabled>true</enabled>
      </snapshots>
    </repository>
  </repositories>

  <build>
   <plugins>
      <plugin>
        <groupId>org.apache.maven.plugins</groupId>
        <artifactId>maven-compiler-plugin</artifactId>
        <version>3.6.1</version>
        <configuration>
          <source>1.8</source>
          <target>1.8</target>
        </configuration>
      </plugin>
    </plugins>

    <pluginManagement>
      <plugins>
        <plugin>
          <groupId>org.codehaus.mojo</groupId>
          <artifactId>exec-maven-plugin</artifactId>
          <version>1.5.0</version>
          <configuration>
            <cleanupDaemonThreads>false</cleanupDaemonThreads>
          </configuration>
        </plugin>
      </plugins>
    </pluginManagement>
  </build>

  <dependencies>
    <dependency>
      <groupId>com.google.cloud.dataflow</groupId>
      <artifactId>google-cloud-dataflow-java-sdk-all</artifactId>
      <version>2.1.0</version>
    </dependency>

    <!-- slf4j API frontend binding with JUL backend -->
    <dependency>
      <groupId>org.slf4j</groupId>
      <artifactId>slf4j-api</artifactId>
      <version>1.7.14</version>
    </dependency>
    <dependency>
      <groupId>org.slf4j</groupId>
      <artifactId>slf4j-jdk14</artifactId>
      <version>1.7.14</version>
    </dependency>
  </dependencies>
</project>

1 个解决方案

#1


0  

2.2.0 is not yet available from Maven i.e. Google haven't cut from head/master and made a the new 2.2.0 build. AFAIK, it's currently in progress.

目前尚无法从Maven获得2.2.0,即谷歌没有从主/主人那里切入,并制作了新的2.2.0版本。 AFAIK,它目前正在进行中。

https://github.com/GoogleCloudPlatform/DataflowJavaSDK/releases https://mvnrepository.com/artifact/com.google.cloud.dataflow/google-cloud-dataflow-java-sdk-all

https://github.com/GoogleCloudPlatform/DataflowJavaSDK/releases https://mvnrepository.com/artifact/com.google.cloud.dataflow/google-cloud-dataflow-java-sdk-all

If you want, you could download the latest version and build/use it yourself - if you can't wait for the 2.2.0 version.

如果您愿意,可以下载最新版本并自行构建/使用 - 如果您不能等待2.2.0版本。

#1


0  

2.2.0 is not yet available from Maven i.e. Google haven't cut from head/master and made a the new 2.2.0 build. AFAIK, it's currently in progress.

目前尚无法从Maven获得2.2.0,即谷歌没有从主/主人那里切入,并制作了新的2.2.0版本。 AFAIK,它目前正在进行中。

https://github.com/GoogleCloudPlatform/DataflowJavaSDK/releases https://mvnrepository.com/artifact/com.google.cloud.dataflow/google-cloud-dataflow-java-sdk-all

https://github.com/GoogleCloudPlatform/DataflowJavaSDK/releases https://mvnrepository.com/artifact/com.google.cloud.dataflow/google-cloud-dataflow-java-sdk-all

If you want, you could download the latest version and build/use it yourself - if you can't wait for the 2.2.0 version.

如果您愿意,可以下载最新版本并自行构建/使用 - 如果您不能等待2.2.0版本。