Using GNU make, is it possible to create a set of targets that will never be scheduled at the same time when using the "--jobs" option?
使用GNU make,是否可以创建一组在使用“--jobs”选项时永远不会同时安排的目标?
Background:
To make this a little more concrete, consider a makefile of the form
为了使这更具体,请考虑表单的makefile
p1: ...deps... # no parallelization conflicts (can run at the same time as p*, e*)
...rules...
p2: ...deps... # no parallelization conflicts (can run at the same time as p*, e*)
...rules...
p3: ...deps... # no parallelization conflicts (can run at the same time as p*, e*)
...rules...
e1: ...deps... # cannot run at same time as any other e*
...rules...
e2: ...deps... # cannot run at same time as any other e*
...rules...
e3: ...deps... # cannot run at same time as any other e*
...rules...
The main thing I need to do is make sure that e1, e2, and e3 never are being processed at the same time because they do some work on an embedded device with limited resources. They crash if multiple ones of them are executing at the same time. p1, p2, and p3 can be executed in parallel with anything, including any e* job.
我需要做的主要是确保e1,e2和e3永远不会同时处理,因为它们在资源有限的嵌入式设备上做了一些工作。如果它们中的多个同时执行,它们会崩溃。 p1,p2和p3可以与任何东西并行执行,包括任何e *作业。
Note that the actual makefile has a few thousand targets with a dependency tree that's about 10 levels deep, so I'm hoping there's a way to do this that (a) doesn't require running make serially and (b) preserves the benefits of encoding the dependency tree in a makefile.
请注意,实际的makefile有几千个目标,其依赖关系树大约有10个级别,所以我希望有一种方法可以做到这一点:(a)不需要连续运行make和(b)保留的好处在makefile中编码依赖关系树。
2 个解决方案
#1
6
One option for you is to use "flock" to run the "e" rules under an exclusive lock. See man flock(1) for details. For example, instead of
一个选项是使用“flock”在独占锁下运行“e”规则。有关详细信息,请参阅man flock(1)。例如,而不是
e2: deps
my_cmd foo bar
You can have
你可以有
e2: deps
flock .embedded-device-lock -c my_cmd foo bar
What happens then is that all the "e" targets get started by make in parallel (possibly), but the actual commands will be executed serially.
然后会发生的是,所有“e”目标都由make并行(可能)启动,但实际命令将按顺序执行。
#2
2
It's not a perfect solution, but you could use an order-only prerequisite to impose a specific ordering on the e* targets:
这不是一个完美的解决方案,但您可以使用仅限订单的先决条件在e *目标上强制执行特定排序:
e1: ...deps...
...commands...
e2: ...deps... | e1
...commands...
e3: ...deps... | e2 e1
...commands...
The prerequisites after the pipe symbol '|' are order-only: they don't force, say, e3 to be updated if e1 or e2 has changed, but they do require that all commands for e1 and e2 finish running before the commands for e3 are started.
管道符号“|”后的先决条件如果e1或e2已经改变,它们不会强迫e3更新,但它们确实要求e1和e2的所有命令在e3和e2的命令开始之前完成运行。
The disadvantage of this is that it imposes a specific ordering for these mutually exclusive prerequisites, rather than letting make
pick the order, but in practice you can probably figure out a reasonable order manually.
这样做的缺点是它对这些互斥的先决条件强加了特定的顺序,而不是让make选择订单,但实际上你可以手动找出合理的订单。
#1
6
One option for you is to use "flock" to run the "e" rules under an exclusive lock. See man flock(1) for details. For example, instead of
一个选项是使用“flock”在独占锁下运行“e”规则。有关详细信息,请参阅man flock(1)。例如,而不是
e2: deps
my_cmd foo bar
You can have
你可以有
e2: deps
flock .embedded-device-lock -c my_cmd foo bar
What happens then is that all the "e" targets get started by make in parallel (possibly), but the actual commands will be executed serially.
然后会发生的是,所有“e”目标都由make并行(可能)启动,但实际命令将按顺序执行。
#2
2
It's not a perfect solution, but you could use an order-only prerequisite to impose a specific ordering on the e* targets:
这不是一个完美的解决方案,但您可以使用仅限订单的先决条件在e *目标上强制执行特定排序:
e1: ...deps...
...commands...
e2: ...deps... | e1
...commands...
e3: ...deps... | e2 e1
...commands...
The prerequisites after the pipe symbol '|' are order-only: they don't force, say, e3 to be updated if e1 or e2 has changed, but they do require that all commands for e1 and e2 finish running before the commands for e3 are started.
管道符号“|”后的先决条件如果e1或e2已经改变,它们不会强迫e3更新,但它们确实要求e1和e2的所有命令在e3和e2的命令开始之前完成运行。
The disadvantage of this is that it imposes a specific ordering for these mutually exclusive prerequisites, rather than letting make
pick the order, but in practice you can probably figure out a reasonable order manually.
这样做的缺点是它对这些互斥的先决条件强加了特定的顺序,而不是让make选择订单,但实际上你可以手动找出合理的订单。