在构建机器上清除Maven本地存储库。

时间:2022-02-26 00:00:36

On a CI build server, the local Maven repository fills up the file system repetitively (after a few days). What strategy are others doing to trim the local repository in such a case? -Max

在CI构建服务器上,本地Maven存储库会重复地填充文件系统(几天之后)。在这种情况下,其他人会采取什么策略来削减本地存储库?-麦克斯

6 个解决方案

#1


27  

The Maven dependency plugin has a purge-local-repository goal that allows you to delete the dependencies for a given project from the local repository, if this is run say once a day on each project the snapshots will not accumulate.

Maven依赖插件有一个purg -local-repository目标,它允许您从本地存储库中删除给定项目的依赖项,如果这是在每个项目上运行一天一次,那么快照将不会累积。


Alternatively there's a more scorched-earth approach you could take. As the problem is typically the timestamped snapshot artifacts, you could use the maven-antrun-plugin to delete all files that match the resource collection pattern.

或者,你可以采取一种更加焦土的方法。由于问题是典型的时间戳快照工件,您可以使用maven-antrun-plugin来删除与资源收集模式匹配的所有文件。

For example (note this might need some tweaking as I've done it from memory):

例如(注意这可能需要一些调整,就像我从内存中做的那样):

<plugin>
  <artifactId>maven-antrun-plugin</artifactId>
  <executions>
    <execution>
      <phase>package</phase>
      <configuration>
        <tasks>
          <delete>
            <fileset dir="${settings.localRepository}">
              <include name="**/*.jar"/>
              <exclude name="**/*.pom"/>
              <exclude name="**/*.war"/>
              <exclude name="**/*.ear"/>
              <exclude name="**/*.md5"/>
              <exclude name="**/*.sha"/>
              <!--any other extensions?...-->
              <!--match the timestamp pattern-->
              <containsregexp expression="[0-9]{8}.[0-9]{6}-[0-9]+"/>
            </fileset>
          </delete>
        </tasks>
      </configuration>
      <goals>
        <goal>run</goal>
      </goals>
    </execution>
  </executions>
</plugin>

#2


18  

If you're using hudson, you can set up a scheduled job to just delete the entire repository once a day or something like that. I've got a job called hudson-maven-repo-clean which has this configuration:

如果您使用的是hudson,您可以设置一个调度的作业,以便每天一次或类似地删除整个存储库。我有一个叫hudson- mavenrepoclean的工作它有这样的配置:

  • Build / Execute shell: rm -rf ~hudson/.m2/repository
  • 构建/执行shell: rm -rf ~hudson/.m2/存储库。
  • Build Triggers / Build periodically: 0 0 * * *
  • 构建触发器/定期构建:0 * * *。

#3


4  

In addition to purge-local-repository (which reads to me like a nuclear option, as it only offers an excludes configuration as opposed to an explicit includes), take a look at the Remove Project Artifact mojo. I'm looking to implement it now, as my exact use case is to clear out large WAR and EAR snapshots that are being built on my CI (and sometimes workstation) machines.

除了权限本地存储库(这对我来说就像一个核选项,因为它只提供一个排除配置,而不是显式的包含),请查看删除项目工件mojo。我现在想要实现它,因为我的确切用例是清除在我的CI(有时是工作站)机器上构建的大型WAR和EAR快照。

#4


3  

We use especially for this purpose the build-helper plugin. In our company parent pom is the remove-project-artifact goal embedded in the profile for our hudson builds. This way all old versions of this artifact are removed prior to installing the currently build version.

我们特别为此目的使用了build-helper插件。在我们的公司,父pom是在我们的hudson构建的概要文件中嵌入的remove- project工件目标。这样,在安装当前构建版本之前,将删除该工件的所有旧版本。

...
<profile>
  <id>hudson</id>
  <activation>
    <property>
      <name>BUILD_TAG</name>
    </property>
  </activation>
  <build>
    <plugins>
      <plugin>
        <groupId>org.codehaus.mojo</groupId>
        <artifactId>build-helper-maven-plugin</artifactId>
        <version>1.7</version>
        <executions>
          <execution>
            <id>remove-old-artifacts</id>
            <phase>package</phase>
            <goals>
              <goal>remove-project-artifact</goal>
            </goals>
            <configuration>
              <removeAll>true</removeAll>
            </configuration>
          </execution>
        </executions>
      </plugin>
  ...

Using removeAll set to true will wipe out all other snapshots except the one your working on. This can be dangerous as it may mean snapshots for a branch will be wiped out as well.

使用removeAll设置为true将删除所有其他快照,除了您正在处理的快照。这可能是危险的,因为它可能意味着分支的快照也将被清除。

For instance if you have a snapshot 1.0.0.18-SNAPSHOT representing HEAD and snapshot 1.0.1.17-SNAPSHOT representing a branch, running this plugin with 1.0.0.18-SNAPSHOT build will wipe the 1.0.1.17-SNAPSHOt folder.

例如,如果您有一个快照1.0.0.18快照,表示HEAD和snapshot 1.0.1.17-SNAPSHOT表示一个分支,那么运行这个带有1.0.0.18快照构建的插件将会擦去1.0.1.17快照文件夹。

To get around this scenario the removeAll should be set to false.

为了绕过这个场景,removeAll应该设置为false。

#5


1  

We have employed a slightly different (and devious) technique. All artifacts that build "large things" (EARs, WARs, TARs) have their deploy location overriden like so:

我们采用了一种稍微不同的(和迂回的)技术。所有制造“大东西”(耳朵,战争,塔)的工件都有他们的部署位置:

<properties>
   <discard-me-in-bit-bucket>file://${basedir}/target/_DELETEME</discard-me-in-bit-bucket> 
</properties>

<distributionManagement>
  <repository>
    <id>upload-InternalSite</id>
    <name>SoftwareLibrary External</name>
    <url>${discard-me-in-bit-bucket}</url>
    <layout>legacy</layout>
    <uniqueVersion>false</uniqueVersion>
  </repository>
  <snapshotRepository>
    <id>upload-InternalSite</id>
    <name>Repository Name</name>
    <url>${discard-me-in-bit-bucket}</url>
    <layout>legacy</layout>
    <uniqueVersion>false</uniqueVersion>
  </snapshotRepository>
</distributionManagement>

This strategy causes the deploy goal to put things in the target directory, which of course is destroyed by the next CLEAN operation. To get even more aggressive, we have a postbuild step that does this:

该策略导致部署目标将东西放到目标目录中,当然,这将被下一个干净操作销毁。为了变得更有侵略性,我们有一个这样的postbuild步骤:

find -type d -name '*_DELETEME' -exec rm -rf '{}' ';' -prune || echo $?

We employ yet one more strategy, too. In Hudson/Jenkins we provide a settings file to place the .m2 repository in the workspace for the job. This allows us to delete the entire repository before or after the job. It also makes artifacts visible in the workspace which aids in debugging some problems.

我们还采用了另一种策略。在Hudson/Jenkins中,我们提供了一个设置文件,将.m2存储库放置在工作空间中。这允许我们在作业之前或之后删除整个存储库。它还使工作空间中可见的工件可以帮助调试一些问题。

#6


0  

How big is the file system? We have 10gb allocated to builds and zap snapshots older than 30 days every night. That seems to work

文件系统有多大?我们有10gb的配置用于构建和zap快照,每天晚上超过30天。这似乎工作

Are you doing builds every X hours or when code changes? Switching to code changes will reduce the number of artifacts without reducing coverage.

你是在做每一个X小时的构建还是在代码变更的时候?切换到代码更改将减少工件数量而不减少覆盖率。

Are you installing all snapshots locally? You don't need to do this in all cases. In most cases, just those snapshots that are actively developed dependancies need to be installed locally.

您是否在本地安装所有快照?你不需要在所有情况下都这么做。在大多数情况下,仅仅需要在本地安装那些积极开发依赖的快照。

Are you installing EAR/WAR files locally? You probably don't need them either.

您在本地安装EAR/WAR文件吗?你可能也不需要它们。

How many workspaces are you keeping? We use hudson and keep only the last 5 builds.

你有多少工作空间?我们使用hudson,只保留最后5个版本。

#1


27  

The Maven dependency plugin has a purge-local-repository goal that allows you to delete the dependencies for a given project from the local repository, if this is run say once a day on each project the snapshots will not accumulate.

Maven依赖插件有一个purg -local-repository目标,它允许您从本地存储库中删除给定项目的依赖项,如果这是在每个项目上运行一天一次,那么快照将不会累积。


Alternatively there's a more scorched-earth approach you could take. As the problem is typically the timestamped snapshot artifacts, you could use the maven-antrun-plugin to delete all files that match the resource collection pattern.

或者,你可以采取一种更加焦土的方法。由于问题是典型的时间戳快照工件,您可以使用maven-antrun-plugin来删除与资源收集模式匹配的所有文件。

For example (note this might need some tweaking as I've done it from memory):

例如(注意这可能需要一些调整,就像我从内存中做的那样):

<plugin>
  <artifactId>maven-antrun-plugin</artifactId>
  <executions>
    <execution>
      <phase>package</phase>
      <configuration>
        <tasks>
          <delete>
            <fileset dir="${settings.localRepository}">
              <include name="**/*.jar"/>
              <exclude name="**/*.pom"/>
              <exclude name="**/*.war"/>
              <exclude name="**/*.ear"/>
              <exclude name="**/*.md5"/>
              <exclude name="**/*.sha"/>
              <!--any other extensions?...-->
              <!--match the timestamp pattern-->
              <containsregexp expression="[0-9]{8}.[0-9]{6}-[0-9]+"/>
            </fileset>
          </delete>
        </tasks>
      </configuration>
      <goals>
        <goal>run</goal>
      </goals>
    </execution>
  </executions>
</plugin>

#2


18  

If you're using hudson, you can set up a scheduled job to just delete the entire repository once a day or something like that. I've got a job called hudson-maven-repo-clean which has this configuration:

如果您使用的是hudson,您可以设置一个调度的作业,以便每天一次或类似地删除整个存储库。我有一个叫hudson- mavenrepoclean的工作它有这样的配置:

  • Build / Execute shell: rm -rf ~hudson/.m2/repository
  • 构建/执行shell: rm -rf ~hudson/.m2/存储库。
  • Build Triggers / Build periodically: 0 0 * * *
  • 构建触发器/定期构建:0 * * *。

#3


4  

In addition to purge-local-repository (which reads to me like a nuclear option, as it only offers an excludes configuration as opposed to an explicit includes), take a look at the Remove Project Artifact mojo. I'm looking to implement it now, as my exact use case is to clear out large WAR and EAR snapshots that are being built on my CI (and sometimes workstation) machines.

除了权限本地存储库(这对我来说就像一个核选项,因为它只提供一个排除配置,而不是显式的包含),请查看删除项目工件mojo。我现在想要实现它,因为我的确切用例是清除在我的CI(有时是工作站)机器上构建的大型WAR和EAR快照。

#4


3  

We use especially for this purpose the build-helper plugin. In our company parent pom is the remove-project-artifact goal embedded in the profile for our hudson builds. This way all old versions of this artifact are removed prior to installing the currently build version.

我们特别为此目的使用了build-helper插件。在我们的公司,父pom是在我们的hudson构建的概要文件中嵌入的remove- project工件目标。这样,在安装当前构建版本之前,将删除该工件的所有旧版本。

...
<profile>
  <id>hudson</id>
  <activation>
    <property>
      <name>BUILD_TAG</name>
    </property>
  </activation>
  <build>
    <plugins>
      <plugin>
        <groupId>org.codehaus.mojo</groupId>
        <artifactId>build-helper-maven-plugin</artifactId>
        <version>1.7</version>
        <executions>
          <execution>
            <id>remove-old-artifacts</id>
            <phase>package</phase>
            <goals>
              <goal>remove-project-artifact</goal>
            </goals>
            <configuration>
              <removeAll>true</removeAll>
            </configuration>
          </execution>
        </executions>
      </plugin>
  ...

Using removeAll set to true will wipe out all other snapshots except the one your working on. This can be dangerous as it may mean snapshots for a branch will be wiped out as well.

使用removeAll设置为true将删除所有其他快照,除了您正在处理的快照。这可能是危险的,因为它可能意味着分支的快照也将被清除。

For instance if you have a snapshot 1.0.0.18-SNAPSHOT representing HEAD and snapshot 1.0.1.17-SNAPSHOT representing a branch, running this plugin with 1.0.0.18-SNAPSHOT build will wipe the 1.0.1.17-SNAPSHOt folder.

例如,如果您有一个快照1.0.0.18快照,表示HEAD和snapshot 1.0.1.17-SNAPSHOT表示一个分支,那么运行这个带有1.0.0.18快照构建的插件将会擦去1.0.1.17快照文件夹。

To get around this scenario the removeAll should be set to false.

为了绕过这个场景,removeAll应该设置为false。

#5


1  

We have employed a slightly different (and devious) technique. All artifacts that build "large things" (EARs, WARs, TARs) have their deploy location overriden like so:

我们采用了一种稍微不同的(和迂回的)技术。所有制造“大东西”(耳朵,战争,塔)的工件都有他们的部署位置:

<properties>
   <discard-me-in-bit-bucket>file://${basedir}/target/_DELETEME</discard-me-in-bit-bucket> 
</properties>

<distributionManagement>
  <repository>
    <id>upload-InternalSite</id>
    <name>SoftwareLibrary External</name>
    <url>${discard-me-in-bit-bucket}</url>
    <layout>legacy</layout>
    <uniqueVersion>false</uniqueVersion>
  </repository>
  <snapshotRepository>
    <id>upload-InternalSite</id>
    <name>Repository Name</name>
    <url>${discard-me-in-bit-bucket}</url>
    <layout>legacy</layout>
    <uniqueVersion>false</uniqueVersion>
  </snapshotRepository>
</distributionManagement>

This strategy causes the deploy goal to put things in the target directory, which of course is destroyed by the next CLEAN operation. To get even more aggressive, we have a postbuild step that does this:

该策略导致部署目标将东西放到目标目录中,当然,这将被下一个干净操作销毁。为了变得更有侵略性,我们有一个这样的postbuild步骤:

find -type d -name '*_DELETEME' -exec rm -rf '{}' ';' -prune || echo $?

We employ yet one more strategy, too. In Hudson/Jenkins we provide a settings file to place the .m2 repository in the workspace for the job. This allows us to delete the entire repository before or after the job. It also makes artifacts visible in the workspace which aids in debugging some problems.

我们还采用了另一种策略。在Hudson/Jenkins中,我们提供了一个设置文件,将.m2存储库放置在工作空间中。这允许我们在作业之前或之后删除整个存储库。它还使工作空间中可见的工件可以帮助调试一些问题。

#6


0  

How big is the file system? We have 10gb allocated to builds and zap snapshots older than 30 days every night. That seems to work

文件系统有多大?我们有10gb的配置用于构建和zap快照,每天晚上超过30天。这似乎工作

Are you doing builds every X hours or when code changes? Switching to code changes will reduce the number of artifacts without reducing coverage.

你是在做每一个X小时的构建还是在代码变更的时候?切换到代码更改将减少工件数量而不减少覆盖率。

Are you installing all snapshots locally? You don't need to do this in all cases. In most cases, just those snapshots that are actively developed dependancies need to be installed locally.

您是否在本地安装所有快照?你不需要在所有情况下都这么做。在大多数情况下,仅仅需要在本地安装那些积极开发依赖的快照。

Are you installing EAR/WAR files locally? You probably don't need them either.

您在本地安装EAR/WAR文件吗?你可能也不需要它们。

How many workspaces are you keeping? We use hudson and keep only the last 5 builds.

你有多少工作空间?我们使用hudson,只保留最后5个版本。