如何使用异步。并行限制以最大限度地增加(并行)运行进程的数量?

时间:2022-05-04 13:51:03

Is it possible to set a Limit to parallel running processes with async.parallelLimit ? I used following very simple code to test how it works.

是否可以对使用异步的并行运行进程设置限制?parallelLimit吗?我使用以下非常简单的代码来测试它是如何工作的。

var async = require("async");
var i = 0;

function write() {
    i++;
    console.log("Done", i);
}

async.parallelLimit([
    function(callback) {
        write();
        callback();
    }
], 10, function() {
    console.log("finish");
});

Of course everything what I got back was this:

当然,我得到的是:

Done 1
finish

In my case I have a function wich is called very often, but I only want it to run only 5 times simultaneously. The other calls should be queued. (Yes I know about async.queue but, this is not what I want in this question).

在我的例子中,我有一个函数叫wich,但是我只希望它同时运行5次。其他调用应该排队。(是的,我知道异步。但这不是我想要的)。

So now the question is, is this possible with async.parallelLimit?

那么现在的问题是,异步是否可以实现。

//EDIT:

/ /编辑:

I actually have something similar to this:

实际上我有一些类似的东西

for(var i = 0; i < somthing.length; i++) { //i > 100  for example
    write();
}

And 'cause of the non synchronicity of node.js this could run 100 times at the same time. But how shell I limit the parallel running processes in this case?

因为节点的非同步。这个可以同时运行100次。但是在这种情况下,我如何限制并行运行过程呢?

2 个解决方案

#1


3  

Very short answer; Yes. That's exactly what asyncParallelLimit does.

很短的答案;是的。这就是asyncParallelLimit的作用。

In your case, you are passing only one function to parallelLimit. That's why it only get's called once. If you were to pass an array with this same function many times, it will get executed as many times as you put it in the array.

在你的例子中,你只传递一个函数到并行极限。这就是为什么它只被调用一次。如果你多次传递一个具有相同函数的数组,它将被执行多少次,就像你把它放入数组一样。

Please note that your example function doesn't actually do any work asynchronously. As such, this example function will always get executed in series. If you have a function that does async work, for example a network request or file i/o, it will get executed in parallel.

请注意,示例函数实际上没有异步执行任何工作。因此,这个示例函数将始终以串行方式执行。如果您有一个执行异步工作的函数,例如一个网络请求或文件i/o,它将被并行执行。

A better example-function for a async workload would be:

异步工作负载的一个更好的例子是:

function(callback) {
    setTimeout(function(){
        callback();
    }, 200);
}

#2


1  

For completion, to add to the existing answer, if you want to run the same function multiple times in parallel with a limit, here's how you do it:

对于完成,要添加到现有的答案中,如果您想要并行地运行同一函数多次,并同时运行一个极限,您可以这样做:

  // run 'my_task' 100 times, with parallel limit of 10

  var my_task = function(callback) { ... };
  var when_done = function(err, results) { ... };

  // create an array of tasks
  var async_queue = Array(100).fill(my_task);

  async.parallelLimit(async_queue, 10, when_done);

#1


3  

Very short answer; Yes. That's exactly what asyncParallelLimit does.

很短的答案;是的。这就是asyncParallelLimit的作用。

In your case, you are passing only one function to parallelLimit. That's why it only get's called once. If you were to pass an array with this same function many times, it will get executed as many times as you put it in the array.

在你的例子中,你只传递一个函数到并行极限。这就是为什么它只被调用一次。如果你多次传递一个具有相同函数的数组,它将被执行多少次,就像你把它放入数组一样。

Please note that your example function doesn't actually do any work asynchronously. As such, this example function will always get executed in series. If you have a function that does async work, for example a network request or file i/o, it will get executed in parallel.

请注意,示例函数实际上没有异步执行任何工作。因此,这个示例函数将始终以串行方式执行。如果您有一个执行异步工作的函数,例如一个网络请求或文件i/o,它将被并行执行。

A better example-function for a async workload would be:

异步工作负载的一个更好的例子是:

function(callback) {
    setTimeout(function(){
        callback();
    }, 200);
}

#2


1  

For completion, to add to the existing answer, if you want to run the same function multiple times in parallel with a limit, here's how you do it:

对于完成,要添加到现有的答案中,如果您想要并行地运行同一函数多次,并同时运行一个极限,您可以这样做:

  // run 'my_task' 100 times, with parallel limit of 10

  var my_task = function(callback) { ... };
  var when_done = function(err, results) { ... };

  // create an array of tasks
  var async_queue = Array(100).fill(my_task);

  async.parallelLimit(async_queue, 10, when_done);