使用javascript从数组中删除重复的对象

I am trying to figure out an efficient way to remove objects that are duplicates from an array and looking for the most efficient answer. I looked around the internet everything seems to be using primitive data... or not scalable for large arrays. This is my current implementation which is can be improved and want to try to avoid labels.

我试图找出一种有效的方法来从数组中删除重复的对象，并寻找最有效的答案。我环顾互联网，一切似乎都在使用原始数据...或者不能扩展到大型数组。这是我目前的实现，可以改进，并希望尝试避免标签。

 Test.prototype.unique = function (arr, artist, title, cb) {
        console.log(arr.length);
        var n, y, x, i, r;
        r = [];      
        o: for (i = 0, n = arr.length; i < n; i++) {

          for (x = 0, y = r.length; x < y; x++) {

                if (r[x].artist == arr[i].artist && r[x].title == arr[i].title) {
                    continue o;
                }
            }
            r.push(arr[i]);
        }

        cb(r);
    };

and the array looks something like this:

并且数组看起来像这样：

[{title: sky, artist: jon}, {title: rain, artist: Paul}, ....]

Order does not matter, but if sorting makes it more efficient then I am up for the challenge...

订单无关紧要，但如果排序使其更有效率，那么我就迎接挑战......

and for people who do not know o is a label and it is just saying jump back to the loop instead of pushing to the new array.

并且对于那些不知道o的人来说是一个标签，它只是说跳回循环而不是推送到新阵列。

Pure javascript please no libs.

纯javascript请没有库。

ANSWERS SO FAR:

回答如此：

The Performance Test for the answers below: http://jsperf.com/remove-duplicates-for-loops

以下答案的性能测试：http：//jsperf.com/remove-duplicates-for-loops

8 个解决方案

#1

I see, the problem there is that the complexity is squared. There is one trick to do it, it's simply by using "Associative arrays".

我知道，问题在于复杂性是平方的。有一个技巧可以做到，它只是使用“关联数组”。

You can get the array, loop over it, and add the value of the array as a key to the associative array. Since it doesn't allow duplicated keys, you will automatically get rid of the duplicates.

您可以获取数组，在其上循环，并将数组的值添加为关联数组的键。由于它不允许重复键，因此您将自动删除重复项。

Since you are looking for title and artist when comparing, you can actually try to use something like:

由于您在比较时正在寻找标题和艺术家，您实际上可以尝试使用以下内容：

var arrResult = {};
for (i = 0, n = arr.length; i < n; i++) {
    var item = arr[i];
    arrResult[ item.title + " - " + item.artist ] = item;
}

Then you just loop the arrResult again, and recreate the array.

然后你再次循环arrResult，并重新创建数组。

var i = 0;
var nonDuplicatedArray = [];    
for(var item in arrResult) {
    nonDuplicatedArray[i++] = arrResult[item];
}

Updated to include Paul's comment. Thanks!

更新以包括保罗的评论。谢谢！

#2

Here is a solution that works for me.

这是一个适合我的解决方案。

Helper functions:

助手功能：

// sorts an array of objects according to one field
// call like this: sortObjArray(myArray, "name" );
// it will modify the input array
sortObjArray = function(arr, field) {
    arr.sort(
        function compare(a,b) {
            if (a[field] < b[field])
                return -1;
            if (a[field] > b[field])
                return 1;
            return 0;
        }
    );
}

// call like this: uniqueDishes = removeDuplicatesFromObjArray(dishes, "dishName");
// it will NOT modify the input array
// input array MUST be sorted by the same field (asc or desc doesn't matter)
removeDuplicatesFromObjArray = function(arr, field) {
    var u = [];
    arr.reduce(function (a, b) {
        if (a[field] !== b[field]) u.push(b);
        return b;
    }, []);
    return u;
}

and then simply call:

然后简单地打电话：

        sortObjArray(dishes, "name");
        dishes = removeDuplicatesFromObjArray(dishes, "name");

#3

Basic sort-then-unique implementation, fiddle HERE:

基本排序然后独特的实现，在这里提示：

function unique(arr) {
    var comparer = function compareObject(a, b) {
        if (a.title == b.title) {
            if (a.artist < b.artist) {
                return -1;
            } else if (a.artist > b.artist) {
                return 1;
            } else {
                return 0;
            }
        } else {
            if (a.title < b.title) {
                return -1;
            } else {
                return 1;
            }
        }
    }

    arr.sort(comparer);
    console.log("Sorted: " + JSON.stringify(arr));
    for (var i = 0; i < arr.length - 1; ++i) {
        if (comparer(arr[i], arr[i+1]) === 0) {
            arr.splice(i, 1);
            console.log("Splicing: " + JSON.stringify(arr));
        }
    }
    return arr;
}

It may or may not be the most efficient, and should be entirely scalable. I've added some console.logs so you can see it as it works.

它可能是也可能不是最有效的，并且应该是完全可扩展的。我添加了一些console.logs，所以你可以看到它的工作原理。

EDIT

编辑

In the interest of saving on the space the function used, I did that for loop at the end, but it seems likely that didn't properly find only unique results (depsite it passing my simple jsfiddle test). Please try replacing my for loop with the following:

为了节省使用函数的空间，我最后做了循环，但似乎没有正确找到唯一的结果（depsite它通过我简单的jsfiddle测试）。请尝试使用以下内容替换我的for循环：

var checker;
var uniqueResults = [];
for (var i = 0; i < arr.length; ++i) {
    if (!checker || comparer(checker, arr[i]) != 0) {
        checker = arr[i];
        uniqueResults.push(checker);
    }
}
return uniqueResults;

#4

I use this function. its not doing any sorting, but produces result. Cant say about performance as never measure it.

我用这个功能。它没有做任何排序，但产生结果。无法衡量绩效，因为从来没有衡量过。

var unique = function(a){
    var seen = [], result = [];
    for(var len = a.length, i = len-1; i >= 0; i--){
        if(!seen[a[i]]){
            seen[a[i]] = true;
            result.push(a[i]);
        }
    }
    return result;
}

var ar = [1,2,3,1,1,1,1,1,"", "","","", "a", "b"]; console.log(unique(ar));// this will produce [1,2,3,"", "a", "b"] all unique elements.

var ar = [1,2,3,1,1,1,1,1，“”，“”，“”，“”，“a”，“b”]; console.log（unique（ar））; //这将生成[1,2,3，“”，“a”，“b”]所有唯一元素。

#5

Below is Henrique Feijo's answer with ample explanation and an example that you can cut and paste:

下面是Henrique Feijo的答案，有充分的解释和一个可以剪切和粘贴的例子：

Goal: Convert an array of objects that contains duplicate objects (like this one)...

目标：转换包含重复对象的对象数组（如此对象）...

[
    {
        "id": 10620,
        "name": "Things to Print"
    },
    {
        "id": 10620,
        "name": "Things to Print"
    },
    {
        "id": 4334,
        "name": "Interesting"
    }
]

... Into an array of objects without duplicate objects (like this one):

...进入没有重复对象的对象数组（如下所示）：

[
    {
        "id": 10620,
        "name": "Things to Print"
    },
    {
        "id": 4334,
        "name": "Interesting"
    }
]

Explanation provided in the comments:

评论中提供的说明：

    var allContent = [{
      "id": 10620,
      "name": "Things to Print"
    }, {
      "id": 10620,
      "name": "Things to Print"
    }, {
      "id": 4334,
      "name": "Interesting"
    }]

     //Put Objects Into As Associative Array. Each key consists of a composite value generated by each set of values from the objects in allContent.
    var noDupeObj = {} //Create an associative array. It will not accept duplicate keys.
    for (i = 0, n = allContent.length; i < n; i++) {
      var item = allContent[i]; //Store each object as a variable. This helps with clarity in the next line.
      noDupeObj[item.id + "|" + item.name] = item; //This is the critical step.
      //Here, you create an object within the associative array that has a key composed of the two values from the original object. 
      // Use a delimiter to not have foo+bar handled like fo+obar
      //Since the associative array will not allow duplicate keys, and the keys are determined by the content, then all duplicate content are removed. 
      //The value assigned to each key is the original object which is along for the ride and used to reconstruct the list in the next step.
    }

     //Recontructs the list with only the unique objects left in the doDupeObj associative array
    var i = 0;
    var nonDuplicatedArray = [];
    for (var item in noDupeObj) {
      nonDuplicatedArray[i++] = noDupeObj[item]; //Populate the array with the values from the noDupeObj.
    }

    console.log(nonDuplicatedArray)

#6

Below code compares object with JSON as String format and removes duplicates and works fine with simple arrays.

下面的代码将对象与JSON比较为String格式，并删除重复项，并与简单数组一起使用。

    Array.prototype.unique=function(a){
     return function(){
        return this.filter(a)
     }
   }(
   function(a,b,c){
     var tmp=[]; 
     c.forEach(function(el){
        tmp.push(JSON.stringify(el))
    }); 
    return tmp.indexOf(JSON.stringify(a),b+1)<0
  })

#7

If you are using underscore js, it is easy to remove duplicate object. http://underscorejs.org/#uniq

如果您使用下划线js，则很容易删除重复的对象。 http://underscorejs.org/#uniq

#8

function remove_duplicates(objectsArray) {
    var arr = [], collection = []; 
    $.each(objectsArray, function (index, value) {
        if ($.inArray(value.id, arr) == -1) { 
            arr.push(value.id);
            collection.push(value);
        }
    });
    return collection;
}