Is there any software available in linux which compiles a source code containing large number of files parallely on either multicore or distributed systems. Libraries like gcc or xserver takes very time for compilation on unicore/dual machine and most of the times it is frustrating when you need lot of recompilation. Is there any technique for compiling such source code parallely ?
Linux中是否有任何软件可以在多核或分布式系统上并行编译包含大量文件的源代码。像gcc或xserver这样的库需要花时间在unicore / dual机器上进行编译,而且大多数情况下,当你需要大量的重新编译时它会令人沮丧。是否存在并行编译此类源代码的技术?
3 个解决方案
#1
28
On distributed-memory systems, you can use distcc to farm out compile jobs to other machines. This takes a little bit of setup, but it can really speed up your build if you happen to have some extra machines around.
在分布式内存系统上,您可以使用distcc将编译作业分配到其他计算机。这需要一些设置,但如果您碰巧有一些额外的机器,它可以真正加快您的构建。
On shared-memory multicore systems, you can just use make -j
, which will try to spawn build jobs based on the dependencies in your makefiles. You can run like this:
在共享内存多核系统上,您可以使用make -j,它将尝试根据makefile中的依赖项生成构建作业。你可以这样运行:
$ make -j
which will impose no limit on the number of jobs spawned, or you can run with an integer parameter:
这将对生成的作业数量没有限制,或者您可以使用整数参数运行:
$ make -j8
which will limit the number of concurrent build jobs. Here, the limit is 8 concurrent jobs. Usually you want this to be something close to the number of cores on your system.
这将限制并发构建作业的数量。这里,限制是8个并发作业。通常,您希望它与您系统上的核心数接近。
#3
3
GNU make (the standard make installed on Linux systems) supports parallel command execution - take a look at http://www.gnu.org/software/make/manual/make.html#Parallel
GNU make(Linux系统上安装的标准make)支持并行命令执行 - 请查看http://www.gnu.org/software/make/manual/make.html#Parallel
#1
28
On distributed-memory systems, you can use distcc to farm out compile jobs to other machines. This takes a little bit of setup, but it can really speed up your build if you happen to have some extra machines around.
在分布式内存系统上,您可以使用distcc将编译作业分配到其他计算机。这需要一些设置,但如果您碰巧有一些额外的机器,它可以真正加快您的构建。
On shared-memory multicore systems, you can just use make -j
, which will try to spawn build jobs based on the dependencies in your makefiles. You can run like this:
在共享内存多核系统上,您可以使用make -j,它将尝试根据makefile中的依赖项生成构建作业。你可以这样运行:
$ make -j
which will impose no limit on the number of jobs spawned, or you can run with an integer parameter:
这将对生成的作业数量没有限制,或者您可以使用整数参数运行:
$ make -j8
which will limit the number of concurrent build jobs. Here, the limit is 8 concurrent jobs. Usually you want this to be something close to the number of cores on your system.
这将限制并发构建作业的数量。这里,限制是8个并发作业。通常,您希望它与您系统上的核心数接近。
#2
#3
3
GNU make (the standard make installed on Linux systems) supports parallel command execution - take a look at http://www.gnu.org/software/make/manual/make.html#Parallel
GNU make(Linux系统上安装的标准make)支持并行命令执行 - 请查看http://www.gnu.org/software/make/manual/make.html#Parallel