Node.js：使用异步库并行运行两个任务，完成前两个任务时执行最后一个任务

Scenario

Need to run two tasks in parallel, and run the third task after firstTask && secondTask are done. Have been using async library and the code works, but want to know if there is something I could improve about my code.

需要并行运行两个任务,并在firstTask && secondTask完成后运行第三个任务。一直在使用异步库和代码工作,但想知道是否有一些我可以改进我的代码。

Details

Task 1: readFileNames: reads a folder and returns an array of file names.

任务1:readFileNames:读取文件夹并返回文件名数组。

Task 2: copyFile: copy a config file from src folder to a destination folder.

任务2:copyFile:将配置文件从src文件夹复制到目标文件夹。

Task 3: writeConfig: write the result of readFileNames into the config file located at the destination folder.

任务3:writeConfig:将readFileNames的结果写入位于目标文件夹的配置文件中。

Questions

Should I combine the parallel control flow with eachSync? Also, was wondering if promises would help me achieve what I am trying to do? And which approach is better in terms of performance? Async vs Q or should I use a more abstracted library like orchestrator?

我应该将并行控制流与eachSync结合使用吗?还有,想知道承诺是否会帮助我实现我想做的事情?哪种方法在性能方面更好? Async vs Q还是应该使用像orchestrator这样更抽象的库?

Below is what I have so far, it works but was wondering if there is a better way of doing it:

下面是我到目前为止,它的工作原理,但想知道是否有更好的方法:

Code

var async = require("async");
var fs = require("fs-extra");
var dir = require("node-dir");
var path = require("path");
var _ = require("underscore");

var readFileNames = function (src, cb) {
  dir.files(src, function (err, files) {
    if (err) { return cb(err); }
    return cb(files);
  });
};

var copyFile = function (src, dest, cb) {
  fs.copy(src, dest, function (err) {
    if (err) { return cb(err); }
    return cb();
  });
};

var writeConfig = function (destFile, content, cb) {
  fs.appendFile(destFile, content, function (err) {
    if (err) { return cb(err); }
    return cb();
  });
};

var modulesFolder = path.join(__dirname, "modules");
var srcFile = path.join(__dirname, "src", "config.json");
var destFile = path.join(__dirname, "dest", "config.json");

async.parallel(
  [
    function (callback) {
      readFileNames(modulesFolder, function (files) {
        callback(null, files);
      });
    },
    function (callback) {
      copyFile(srcFile, destFile, function () {
        callback(null, "");
      });
    }
  ],
  // last callback
  function (err, results) {
    var toWrite = _.flatten(results);
    toWrite.forEach(function (content) {
      if(content) {
        writeConfig(destFile, content + "\n", function () {
        });
      }
    });
    console.log("done");
  }
);

files

├── dest
├── main.js
├── modules
│   ├── module1.txt
│   └── module2.txt
├── node_modules
│   ├── async
│   ├── fs-extra
│   └── node-dir
├── package.json
└── src
    └── config.json

2 个解决方案

#1

I have been using async library and the code works, but want to know if there is something I could improve about my code.

我一直在使用异步库和代码工作,但想知道我的代码是否有可以改进的东西。

You are using way too many anonymous function expressions. Just pass the callback you are receiving around! All those helper functions (including the named ones) are pretty superfluous.

您正在使用太多匿名函数表达式。只需通过您收到的回调!所有这些辅助函数(包括命名函数)都是多余的。

Also, your readFileNames function does not follow the node callback conventions. I assume you don't intend to write the error into your result file?

此外,您的readFileNames函数不遵循节点回调约定。我假设您不打算将错误写入结果文件中?

Also your final callback does ignore errors.

您的最终回调也会忽略错误。

Should I combine the parallel control flow with eachSync?

我应该将并行控制流与eachSync结合使用吗?

I guess you mean eachSeries, not eachSync? Yes, that would be reasonable, if you expect the appendFile calls to chain anyway. But you could just as well use parallel again, which would more closely relate to the .forEach you're currently doing. But in case, you should use an async iteration method, as currently you are logging "done" too early.

我猜你的意思是每个系列,而不是每个同步?是的,如果您期望appendFile调用仍然链接,那将是合理的。但你也可以再次使用并行,这与你目前正在做的.forEach更紧密相关。但万一,您应该使用异步迭代方法,因为目前您过早地记录“完成”。

So rather do

所以宁愿这样做

var async = require("async");
var fs = require("fs-extra");
var dir = require("node-dir");
var path = require("path");

var modulesFolder = path.join(__dirname, "modules");
var srcFile = path.join(__dirname, "src", "config.json");
var destFile = path.join(__dirname, "dest", "config.json");

async.parallel([
  function (callback) {
    dir.files(modulesFolder, callback);
  },
  function (callback) {
    fs.copy(srcFile, destFile, callback);
  }
], function(err, results) {
  if (err)
    return console.error(err);
  var toWrite = results[0] // ignore the result of the copy action
  async.eachSeries(toWrite, function(content, callback) {
    // if(content) not sure if needed at all
      fs.appendFile(destFile, content + "\n", callback);
  }, function(err) {
    if (err)
      return console.error(err);
    console.log("done");
  });
});

Also, was wondering if promises would help me achieve what I am trying to do?

还有,想知道承诺是否会帮助我实现我想做的事情?

Yes, they could do this as well, and possibly even easier and more straightforward (if you're accustomed to them). They'd also simplify error handling a lot.

是的,他们也可以做到这一点,甚至可能更容易和更直接(如果你习惯了它们)。它们还可以简化错误处理。

And which approach is better in terms of performance?

哪种方法在性能方面更好?

Neither. The performance of this little script is bound by the IO capabilities of your system, and the degree of parallelisation your algorithm uses. Which can be achieved by either library.

都不是。这个小脚本的性能受系统的IO功能以及算法使用的并行化程度的约束。这可以通过任何一个库来实现。

#2

Your use of async.parallel() looks fine to me, and if it gets the job done then you're done. Regarding performance, have you parallelized all tasks that can be done in parallel? How fast is your disk IO? These questions matter significantly more than your choice of which async/promise library to use.

您对async.parallel()的使用对我来说很好,如果它完成了工作,那么您就完成了。关于性能,您是否并行化了可以并行完成的所有任务?你的磁盘IO有多快?这些问题比您选择使用哪个异步/承诺库更重要。

With that said, promise libraries like Q would typically slow things down a bit when compared to async because they'll tend to do a process.nextTick at times when it's not strictly necessary, but this performance degradation is quite small. In the vast majority of cases, performance concerns shouldn't dictate your choice of async/promise library.

有了这样的说法,与异步相比,像Q这样的承诺库通常会减慢速度,因为它们往往会做一个process.nextTick有时候并不是绝对必要的,但这种性能下降非常小。在绝大多数情况下,性能问题不应该决定您选择的异步/承诺库。