避免唯一错误E11000与承诺

I have been using this mongoose plugin to perform findOrCreate which is used very often in the codebase.

我一直在使用这个mongoose插件来执行findOrCreate，在代码库中经常用到findOrCreate。

I recently realized that performing multiple asynchronous findOrCreate operations when the unique index is created easily leads to an E11000 duplicate key error.

我最近意识到，在创建惟一索引时执行多个异步findOrCreate操作容易导致E11000重复键错误。

An example can be described by the following using Promise.all. Suppose name is unique then:

下面使用promisee .all对一个示例进行描述。假设名称是唯一的:

const promises = await Promise.all([
  Pokemon.findOrCreate({ name: 'Pikachu' }),
  Pokemon.findOrCreate({ name: 'Pikachu' }),
  Pokemon.findOrCreate({ name: 'Pikachu' })
]);

The above will certainly fail since findOrCreate is not atomic. It makes sense after thinking about it why it fails but, what I would like is a streamlined way of approaching this problem.

由于findOrCreate不是原子的，所以上面的操作肯定会失败。在思考它为什么会失败之后，它是有意义的，但是，我想要的是一种解决这个问题的流线型方法。

Many of my models use findOrCreate and they are all subject to this problem. One solution that comes to mind would be to create a plugin that would catch the error and then return the result of find however, there may be a better approach here - possibly a native mongoose one that I am not aware of.

我的许多模型都使用findOrCreate，它们都受到这个问题的影响。我想到的一个解决方案是创建一个插件来捕获错误，然后返回find的结果。然而，这里可能有更好的方法——可能是一个我不知道的本地mongoose。

1 个解决方案

#1

It certainly depends on your intended usage of this, but I would say overall that "plugins" are just not required. The basic functionality you are looking for is already "built in" to MongoDB with "upserts".

这当然取决于您对它的预期使用，但是我要说的是，总的来说，“插件”并不是必需的。您正在寻找的基本功能已经“内置”到MongoDB并带有“upserts”。

By definition, an "upsert" cannot produce a "duplicate key error" as long as the query condition to "select" the document is issued using the "unique key" for the collection. In this case "name".

根据定义，“upsert”不能产生“重复的关键错误”，只要查询条件“选择”文档，就会使用“唯一键”来进行集合。在这种情况下,“名字”。

In a nutshell you can mimic the same behavior as above by simply doing:

简单地说，你可以通过简单地做:

let results = await Promise.all([
  Pokemon.findOneAndUpdate({ "name": "Pikachu" },{},{ "upsert": true, "new": true }),
  Pokemon.findOneAndUpdate({ "name": "Pikachu" },{},{ "upsert": true, "new": true }),
  Pokemon.findOneAndUpdate({ "name": "Pikachu" },{},{ "upsert": true, "new": true })
]);

Which would simply "create" the item on the first call where it did not already exist, or "return" the existing item. This is how "upserts" work.

它只会在第一次调用中“创建”不存在的项，或者“返回”现有项。这就是“支持者”的工作方式。

[
  {
    "_id": "5a022f48edca148094f30e8c",
    "name": "Pikachu",
    "__v": 0
  },
  {
    "_id": "5a022f48edca148094f30e8c",
    "name": "Pikachu",
    "__v": 0
  },
  {
    "_id": "5a022f48edca148094f30e8c",
    "name": "Pikachu",
    "__v": 0
  }
]

If you really did not care about "returning" each call and simply wanted to "update or create", then it's actually far more efficient to simply send one request with bulkWrite():

如果您真的不关心“返回”每个调用，而只想要“更新或创建”，那么使用bulkWrite()发送一个请求实际上要高效得多:

// Issue a "batch" in Bulk
let result = await Pokemon.bulkWrite(
  Array(3).fill(1).map( (e,i) => ({
    "updateOne": {
      "filter": { "name": "Pikachu" },
      "update": {
        "$set": { "skill": i }
      },
      "upsert": true
    }
  }))
);

So instead of awaiting the server to resolve three async calls, you only make one which either "creates" the item or "updates" with anything you use in the $set modifier when found. These are applied on every match including the first, and if you want "only on create" there is $setOnInsert to do that.

因此，与等待服务器解析三个异步调用不同，您只执行一个调用，该调用将使用在$set修饰符中找到的任何内容“创建”项或“更新”项。这些应用于每个匹配，包括第一个匹配，如果您想要“仅在create上”，可以使用$setOnInsert来实现这一点。

Of course this is just a "write", so it really depends on whether it is important to you to return the modified document or not. So "bulk" operations simply "write" and they do not return, but instead return information on the "batch" indicating what was "upserted" and what was "modified" as in:

当然，这只是一个“写”，因此它实际上取决于返回修改后的文档对您是否重要。因此，“批量”操作简单地“写入”，而它们不返回，而是返回“批”的信息，指示“向上”和“修改”的内容:

{
  "ok": 1,
  "writeErrors": [],
  "writeConcernErrors": [],
  "insertedIds": [],
  "nInserted": 0,
  "nUpserted": 1,           // <-- created 1 time
  "nMatched": 2,            // <-- matched and modified the two other times
  "nModified": 2,
  "nRemoved": 0,
  "upserted": [
    {
      "index": 0,
      "_id": "5a02328eedca148094f30f33"  // <-- this is the _id created in upsert
    }
  ],
  "lastOp": {
    "ts": "6485801998833680390",
    "t": 23
  }
}

So if you do want a "return", then a more typical case is to separate which data you want on "create" and which is needed on "update". Noting that the $setOnInsert is essentially "implied" for whatever values are in the "query" condition to select the document:

因此，如果您确实想要一个“返回”，那么更典型的情况是将“创建”上需要的数据和“更新”上需要的数据分开。注意到$setOnInsert本质上是“暗示”用于“查询”条件中选择文档的任何值:

// Issue 3 pokemon as separate calls
let sequence = await Promise.all(
  Array(3).fill(1).map( (e,i) =>
    Pokemon.findOneAndUpdate(
      { name: "Pikachu" },
      { "$set": { "skill": i } },
      { "upsert": true, "new": true }
    )
  )
);

Which would show the modifications applied in "sequence" of each atomic transaction:

将显示在每个原子事务的“序列”中应用的修改:

[
  {
    "_id": "5a02328fedca148094f30f38",
    "name": "Pikachu",
    "__v": 0,
    "skill": 0
  },
  {
    "_id": "5a02328fedca148094f30f39",
    "name": "Pikachu",
    "__v": 0,
    "skill": 1
  },
  {
    "_id": "5a02328fedca148094f30f38",
    "name": "Pikachu",
    "__v": 0,
    "skill": 2
  }
]

So generally it's "upserts" that you want here, and depending on your intent you either use separate calls to return each modification/creation or you issue your "writes" in a batch.

一般来说，它是你想要的“upserts”，根据你的意图，你可以使用单独的调用来返回每个修改/创建，或者在批处理中发出“写入”。

As a complete listing to demonstrate all the above:

作为一份完整的清单来演示以上所有内容:

const mongoose = require('mongoose'),
      Schema = mongoose.Schema;

mongoose.Promise = global.Promise;
mongoose.set('debug', true);

const uri = 'mongodb://localhost/test',
      options = { useMongoClient: true };

const pokemonSchema = new Schema({
  name: String,
  skill: Number
},{ autoIndex: false });

pokemonSchema.index({ name: 1 },{ unique: true, background: false });

const Pokemon = mongoose.model('Pokemon', pokemonSchema);

function log(data) {
  console.log(JSON.stringify(data, undefined, 2))
}

(async function() {

  try {

    const conn = await mongoose.connect(uri,options);

    // Await index creation, otherwise we error
    await Pokemon.ensureIndexes();

    // Clean data for test
    await Pokemon.remove();

    // Issue 3 pokemon as separate calls
    let pokemon = await Promise.all(
      Array(3).fill(1).map( e =>
        Pokemon.findOneAndUpdate({ name: "Pikachu" },{},{ "upsert": true, "new": true })
      )
    );

    log(pokemon);

    // Clean data again
    await Pokemon.remove();


    // Issue a "batch" in Bulk
    let result = await Pokemon.bulkWrite(
      Array(3).fill(1).map( (e,i) => ({
        "updateOne": {
          "filter": { "name": "Pikachu" },
          "update": {
            "$set": { "skill": i }
          },
          "upsert": true
        }
      }))
    );

    log(result);

    let allPokemon = await Pokemon.find();
    log(allPokemon);

    // Clean data again
    await Pokemon.remove();

    // Issue 3 pokemon as separate calls
    let sequence = await Promise.all(
      Array(3).fill(1).map( (e,i) =>
        Pokemon.findOneAndUpdate(
          { name: "Pikachu" },
          { "$set": { "skill": i } },
          { "upsert": true, "new": true }
        )
      )
    );

    log(sequence);


  } catch(e) {
    console.error(e);
  } finally {
    mongoose.disconnect();
  }


})()

Which would produce the output ( for those too lazy to run themselves ):

这将产生输出(对于那些懒得自己运行的人):

Mongoose: pokemons.ensureIndex({ name: 1 }, { unique: true, background: false })
Mongoose: pokemons.remove({}, {})
Mongoose: pokemons.findAndModify({ name: 'Pikachu' }, [], { '$setOnInsert': { __v: 0 } }, { upsert: true, new: true, remove: false, fields: {} })
Mongoose: pokemons.findAndModify({ name: 'Pikachu' }, [], { '$setOnInsert': { __v: 0 } }, { upsert: true, new: true, remove: false, fields: {} })
Mongoose: pokemons.findAndModify({ name: 'Pikachu' }, [], { '$setOnInsert': { __v: 0 } }, { upsert: true, new: true, remove: false, fields: {} })
[
  {
    "_id": "5a023461edca148094f30f82",
    "name": "Pikachu",
    "__v": 0
  },
  {
    "_id": "5a023461edca148094f30f82",
    "name": "Pikachu",
    "__v": 0
  },
  {
    "_id": "5a023461edca148094f30f82",
    "name": "Pikachu",
    "__v": 0
  }
]
Mongoose: pokemons.remove({}, {})
Mongoose: pokemons.bulkWrite([ { updateOne: { filter: { name: 'Pikachu' }, update: { '$set': { skill: 0 } }, upsert: true } }, { updateOne: { filter: { name: 'Pikachu' }, update: { '$set': { skill: 1 } }, upsert: true } }, { updateOne: { filter: { name: 'Pikachu' }, update: { '$set': { skill: 2 } }, upsert: true } } ], {})
{
  "ok": 1,
  "writeErrors": [],
  "writeConcernErrors": [],
  "insertedIds": [],
  "nInserted": 0,
  "nUpserted": 1,
  "nMatched": 2,
  "nModified": 2,
  "nRemoved": 0,
  "upserted": [
    {
      "index": 0,
      "_id": "5a023461edca148094f30f87"
    }
  ],
  "lastOp": {
    "ts": "6485804004583407623",
    "t": 23
  }
}
Mongoose: pokemons.find({}, { fields: {} })
[
  {
    "_id": "5a023461edca148094f30f87",
    "name": "Pikachu",
    "skill": 2
  }
]
Mongoose: pokemons.remove({}, {})
Mongoose: pokemons.findAndModify({ name: 'Pikachu' }, [], { '$setOnInsert': { __v: 0 }, '$set': { skill: 0 } }, { upsert: true, new: true, remove: false, fields: {} })
Mongoose: pokemons.findAndModify({ name: 'Pikachu' }, [], { '$setOnInsert': { __v: 0 }, '$set': { skill: 1 } }, { upsert: true, new: true, remove: false, fields: {} })
Mongoose: pokemons.findAndModify({ name: 'Pikachu' }, [], { '$setOnInsert': { __v: 0 }, '$set': { skill: 2 } }, { upsert: true, new: true, remove: false, fields: {} })
[
  {
    "_id": "5a023461edca148094f30f8b",
    "name": "Pikachu",
    "__v": 0,
    "skill": 0
  },
  {
    "_id": "5a023461edca148094f30f8b",
    "name": "Pikachu",
    "__v": 0,
    "skill": 1
  },
  {
    "_id": "5a023461edca148094f30f8b",
    "name": "Pikachu",
    "__v": 0,
    "skill": 2
  }
]

N.B The $setOnInsert is also "implied" in all "mongoose" operations for the purpose of applying the __v key. So unless you turn this off, that statement is always "merged" with whatever is issued and thus allows the {} in the first example "update" block which would be an error in the core driver due to no update modifier being applied, yet mongoose adds this one for you.

N。为了应用__v键，在所有的“mongoose”操作中，$setOnInsert也是“隐含的”。因此，除非您关闭该语句，否则该语句总是与所发出的内容“合并”在第一个示例“update”块中，这将是由于没有应用更新修饰符而导致的核心驱动程序中的错误，然而mongoose为您添加了这个。

Also note that bulkWrite() does not actually reference the "schema" for the model and bypasses it. This is why there is no __v in those issued updates, and it does indeed bypass all validation as well. This is usually not an issue, but it is something you should be aware of.

还要注意的是，bulkWrite()并不实际引用模型的“模式”，并绕过它。这就是为什么在那些发布的更新中没有__v，而且它确实也绕过了所有的验证。这通常不是问题，但你应该意识到这一点。