避免使用git和make重新编译

时间:2021-07-07 16:23:50

I have two development branches in git and I frequently need to change between the two. However, the really frustrating thing is that every time I change branches in git, the entire project gets rebuilt because the file-system timestamps for some files will change.

我在git中有两个开发分支,我经常需要在两者之间进行更改。然而,真正令人沮丧的是每次我在git中更改分支时,整个项目都会重建,因为某些文件的文件系统时间戳会发生变化。

Ofc, the makefiles are configured to build the project into two different build directories .

Ofc,makefile被配置为将项目构建到两个不同的构建目录中。

Is there any way around this? Compilation is a very long and time-consuming process...

有没有办法解决?编译是一个非常漫长而耗时的过程......

Edit:- This is a slightly more detailed explanation of the question... Say I have a header files Basic.h which is included in a number of other files. Basic.h is different between branch 1 and branch 2.

编辑: - 这是一个稍微更详细的问题解释...说我有一个头文件Basic.h,它包含在许多其他文件中。 Basic.h在分支1和分支2之间是不同的。

Now let's say I have compiled branch 1 into build_branch1 and branch 2 into build_branch2. Say I have branch 2 currently checked out. Now I checkout branch 1 and change File1.cpp and recompile. Ideally, since only File1.cpp has changed since I compiled it the last time, this is the only file that should be recompiled.

现在假设我已将分支1编译为build_branch1,将分支2编译为build_branch2。说我目前检查了分支2。现在我签出分支1并更改File1.cpp并重新编译。理想情况下,因为自上次编译以来只有File1.cpp发生了变化,所以这是唯一应该重新编译的文件。

However, since Basic.h has it's timestamp changed due to the checkout, all files that are including Basic.h will get recompiled. I want to avoid this.

但是,由于Basic.h由于结帐而更改了时间戳,因此将重新编译包含Basic.h的所有文件。我想避免这个。

5 个解决方案

#1


13  

Git changes only the files that are updated between branches. But if your compiler does a full rebuild even if any single file was changed you can always clone and checkout your different branches into different directories. That's like:

Git只更改分支之间更新的文件。但是,如果您的编译器执行完全重建,即使任何单个文件已更改,您也可以始终克隆并将不同的分支签出到不同的目录中。这就像:

/your-repo-name.branch1
/your-repo-name.branch2

This takes extra disk space but is much more convinient than switching divergent branches in a huge repo.

这需要额外的磁盘空间,但比在巨大的回购中切换不同的分支更方便。

#2


8  

Another partial answer: compiler cache.

另一个部分答案:编译器缓存。

When you switch back to the original branch and rebuild, although the dependencies say that numerous files dependent on Basic.h have to be rebuilt, the object files can be pulled from a compiler cache.

当您切换回原始分支并重建时,尽管依赖项表示必须重建依赖于Basic.h的大量文件,但可以从编译器缓存中提取目标文件。

ccache ( http://ccache.samba.org/ ) still has to do some fairly expensive work (processing the pre-processed translation unit, since an entire translation unit is used a hash key) but it's a lot cheaper than compiling.

ccache(http://ccache.samba.org/)仍然需要做一些相当昂贵的工作(处理预处理的翻译单元,因为整个翻译单元使用了哈希键),但它比编译便宜很多。

In some cases ccache can eliminate a compile in the face of a change that does not affect it, like some whitespace changes. For instance if you change a comment in a dependent file (header or source), that does not invalidate the cached object file.

在某些情况下,ccache可以在面对不影响它的更改时消除编译,就像一些空格更改一样。例如,如果更改依赖文件(标头或源)中的注释,则不会使缓存的对象文件失效。

So it can help even if you do a git pull and pick up a new change to Basic.h you haven't seen before.

因此,即使您执行git pull并获取您之前从未见过的Basic.h的新更改,它也可以提供帮助。

#3


3  

If you know which files actually need compilation, and do those manually, GNU Make (at least, I don't know about other implementations) has a flag for you: -t, which basically runs over your Makefile and changes timestamps instead of running commands.

如果您知道哪些文件实际需要编译,并且手动执行,GNU Make(至少,我不知道其他实现)有一个标志:-t,它基本上运行在Makefile上并更改时间戳而不是运行命令。

You'll still need to update the files that need updating before using this flag, or you'll end up with object files that are legitimately out-of-date but look updated. See the linked doc for details.

在使用此标志之前,您仍需要更新需要更新的文件,否则您最终会得到合法过期但看起来更新的目标文件。有关详细信息,请参阅链接文档

The -o option might also interest you, depending on how much changes when you switch branches.

-o选项可能也会让您感兴趣,具体取决于切换分支时的更改量。

#4


2  

There is no good way to preserve timestamps and not get into trouble in your build environment. Use git clone / git checkout branchB to create a second working directory for building and working with branch B. Don't have your Makefile build both, have each one build into its own build directory.

没有好的方法可以保留时间戳,也不会在构建环境中遇到麻烦。使用git clone / git checkout branchB创建第二个工作目录,用于构建和使用分支B.不要同时构建Makefile,将每个构建到自己的构建目录中。

#5


2  

If Basic.h actually differs between the branches, then the only fix is to break it up into smaller files with more fine-grained dependencies, so that less stuff is rebuilt when it changes.

如果Basic.h实际上在分支之间存在差异,那么唯一的解决方法是将其分解为具有更细粒度依赖关系的较小文件,以便在更改时重建更少的东西。

But suppose that Basic.h is actually exactly the same between branches, but git is changing its timestamp. This is a false trigger, like doing touch Basic.h, which reveals a limitation of timestamp-based build systems. Ideally we want a build system to rebuild when contents change, not when timestamps change. Timestamps are used because they are a cheap substitute for detecting a content change. The superior method is for the build system to keep hashes of all files and detect actual modifications, without regard for the time stamp. This also fixes problems like "clock skew detected; your build may be incomplete".

但是假设Basic.h实际上在分支之间完全相同,但是git正在改变它的时间戳。这是一个错误的触发器,就像触摸Basic.h一样,它揭示了基于时间戳的构建系统的局限性。理想情况下,我们希望构建系统在内容更改时重建,而不是在时间戳更改时重建。使用时间戳是因为它们是检测内容更改的廉价替代品。优越的方法是使构建系统保持所有文件的哈希并检测实际修改,而不考虑时间戳。这也解决了“检测到时钟偏差;您的构建可能不完整”等问题。

You're not going to get this kind of build system out of Make; but you may be able to work some aspects of it with regard to certain files. For instance, you can write your Make rules such that your object files do not depend directly on Basic.h but on Basic.h.sha, which is a stamp file. Now, what is in Basic.h.sha? This file contains a SHA256 hash of Basic.h.

你不会从Make中获得这种构建系统;但是你可以针对某些文件来处理它的某些方面。例如,您可以编写Make规则,使您的目标文件不直接依赖于Basic.h,而是依赖于Basic.h.sha,这是一个标记文件。现在,Basic.h.sha中有什么?该文件包含Basic.h的SHA256哈希。

Every time you run Make, logic in the Makefile computes the hash over Basic.h and compares it to the hash stored in Basic.h.sha. If they differ, then Basic.h.sha is overwritten with the new hash. Thereby, its timestamp is bumped, triggering a rebuild.

每次运行Make时,Makefile中的逻辑都会计算Basic.h上的哈希值,并将其与Basic.h.sha中存储的哈希值进行比较。如果它们不同,则使用新哈希覆盖Basic.h.sha。从而,它的时间戳被撞击,触发重建。

BASIC_H_SHA := $(shell sha256sum Basic.h)
BASIC_H_SHA_OLD := $(shell cat Basic.h.sha)

ifneq ($(BASIC_H_SHA),$(BASIC_H_SHA_OLD))
$(info Basic.h has changed!)
DUMMY := $(shell echo "$(BASIC_H_SHA)" > Basic.h.sha)
endif

I have implemented logic along these lines to make modules dependent on changes to their respective CFLAGS. Basically, we can turn a change in anything into a touch of a timestamp file, which controls what is built.

我已经沿着这些线实现了逻辑,使模块依赖于各自CFLAGS的变化。基本上,我们可以将任何内容中的更改转换为时间戳文件的触摸,该文件控制所构建的内容。

#1


13  

Git changes only the files that are updated between branches. But if your compiler does a full rebuild even if any single file was changed you can always clone and checkout your different branches into different directories. That's like:

Git只更改分支之间更新的文件。但是,如果您的编译器执行完全重建,即使任何单个文件已更改,您也可以始终克隆并将不同的分支签出到不同的目录中。这就像:

/your-repo-name.branch1
/your-repo-name.branch2

This takes extra disk space but is much more convinient than switching divergent branches in a huge repo.

这需要额外的磁盘空间,但比在巨大的回购中切换不同的分支更方便。

#2


8  

Another partial answer: compiler cache.

另一个部分答案:编译器缓存。

When you switch back to the original branch and rebuild, although the dependencies say that numerous files dependent on Basic.h have to be rebuilt, the object files can be pulled from a compiler cache.

当您切换回原始分支并重建时,尽管依赖项表示必须重建依赖于Basic.h的大量文件,但可以从编译器缓存中提取目标文件。

ccache ( http://ccache.samba.org/ ) still has to do some fairly expensive work (processing the pre-processed translation unit, since an entire translation unit is used a hash key) but it's a lot cheaper than compiling.

ccache(http://ccache.samba.org/)仍然需要做一些相当昂贵的工作(处理预处理的翻译单元,因为整个翻译单元使用了哈希键),但它比编译便宜很多。

In some cases ccache can eliminate a compile in the face of a change that does not affect it, like some whitespace changes. For instance if you change a comment in a dependent file (header or source), that does not invalidate the cached object file.

在某些情况下,ccache可以在面对不影响它的更改时消除编译,就像一些空格更改一样。例如,如果更改依赖文件(标头或源)中的注释,则不会使缓存的对象文件失效。

So it can help even if you do a git pull and pick up a new change to Basic.h you haven't seen before.

因此,即使您执行git pull并获取您之前从未见过的Basic.h的新更改,它也可以提供帮助。

#3


3  

If you know which files actually need compilation, and do those manually, GNU Make (at least, I don't know about other implementations) has a flag for you: -t, which basically runs over your Makefile and changes timestamps instead of running commands.

如果您知道哪些文件实际需要编译,并且手动执行,GNU Make(至少,我不知道其他实现)有一个标志:-t,它基本上运行在Makefile上并更改时间戳而不是运行命令。

You'll still need to update the files that need updating before using this flag, or you'll end up with object files that are legitimately out-of-date but look updated. See the linked doc for details.

在使用此标志之前,您仍需要更新需要更新的文件,否则您最终会得到合法过期但看起来更新的目标文件。有关详细信息,请参阅链接文档

The -o option might also interest you, depending on how much changes when you switch branches.

-o选项可能也会让您感兴趣,具体取决于切换分支时的更改量。

#4


2  

There is no good way to preserve timestamps and not get into trouble in your build environment. Use git clone / git checkout branchB to create a second working directory for building and working with branch B. Don't have your Makefile build both, have each one build into its own build directory.

没有好的方法可以保留时间戳,也不会在构建环境中遇到麻烦。使用git clone / git checkout branchB创建第二个工作目录,用于构建和使用分支B.不要同时构建Makefile,将每个构建到自己的构建目录中。

#5


2  

If Basic.h actually differs between the branches, then the only fix is to break it up into smaller files with more fine-grained dependencies, so that less stuff is rebuilt when it changes.

如果Basic.h实际上在分支之间存在差异,那么唯一的解决方法是将其分解为具有更细粒度依赖关系的较小文件,以便在更改时重建更少的东西。

But suppose that Basic.h is actually exactly the same between branches, but git is changing its timestamp. This is a false trigger, like doing touch Basic.h, which reveals a limitation of timestamp-based build systems. Ideally we want a build system to rebuild when contents change, not when timestamps change. Timestamps are used because they are a cheap substitute for detecting a content change. The superior method is for the build system to keep hashes of all files and detect actual modifications, without regard for the time stamp. This also fixes problems like "clock skew detected; your build may be incomplete".

但是假设Basic.h实际上在分支之间完全相同,但是git正在改变它的时间戳。这是一个错误的触发器,就像触摸Basic.h一样,它揭示了基于时间戳的构建系统的局限性。理想情况下,我们希望构建系统在内容更改时重建,而不是在时间戳更改时重建。使用时间戳是因为它们是检测内容更改的廉价替代品。优越的方法是使构建系统保持所有文件的哈希并检测实际修改,而不考虑时间戳。这也解决了“检测到时钟偏差;您的构建可能不完整”等问题。

You're not going to get this kind of build system out of Make; but you may be able to work some aspects of it with regard to certain files. For instance, you can write your Make rules such that your object files do not depend directly on Basic.h but on Basic.h.sha, which is a stamp file. Now, what is in Basic.h.sha? This file contains a SHA256 hash of Basic.h.

你不会从Make中获得这种构建系统;但是你可以针对某些文件来处理它的某些方面。例如,您可以编写Make规则,使您的目标文件不直接依赖于Basic.h,而是依赖于Basic.h.sha,这是一个标记文件。现在,Basic.h.sha中有什么?该文件包含Basic.h的SHA256哈希。

Every time you run Make, logic in the Makefile computes the hash over Basic.h and compares it to the hash stored in Basic.h.sha. If they differ, then Basic.h.sha is overwritten with the new hash. Thereby, its timestamp is bumped, triggering a rebuild.

每次运行Make时,Makefile中的逻辑都会计算Basic.h上的哈希值,并将其与Basic.h.sha中存储的哈希值进行比较。如果它们不同,则使用新哈希覆盖Basic.h.sha。从而,它的时间戳被撞击,触发重建。

BASIC_H_SHA := $(shell sha256sum Basic.h)
BASIC_H_SHA_OLD := $(shell cat Basic.h.sha)

ifneq ($(BASIC_H_SHA),$(BASIC_H_SHA_OLD))
$(info Basic.h has changed!)
DUMMY := $(shell echo "$(BASIC_H_SHA)" > Basic.h.sha)
endif

I have implemented logic along these lines to make modules dependent on changes to their respective CFLAGS. Basically, we can turn a change in anything into a touch of a timestamp file, which controls what is built.

我已经沿着这些线实现了逻辑,使模块依赖于各自CFLAGS的变化。基本上,我们可以将任何内容中的更改转换为时间戳文件的触摸,该文件控制所构建的内容。