I'm trying to implement an table-view for large collections of semi-complex objects on Vue 2. Basically the idea is to collect anywhere between 50 000 to 100 000 rows from DB into JS cache, which is then analyzed dynamically to build table-view with real-time-filters (text-search). Each row within table is toggleable, meaning that clicking the row changes the row to edit-mode, which enables Excel-like editing for that specific field/cell.
我正在尝试为Vue 2上的大型半复杂对象集合实现表视图。基本的想法是从DB中收集5到10万行到JS缓存的任何地方，然后动态地分析这些行，用实时过滤器(文本搜索)构建表视图。表中的每一行都是可切换的，这意味着单击该行会将该行更改为编辑模式，从而允许对特定字段/单元格进行类似于excel的编辑。
Each object has about ~100-150 fields/properties, but only certain amount of 'em are shown at any given moment within table (table columns can be toggled in real-time). For large datasets it seems that DB is pushing about ~10-100mb of JSON-data, which in this use-case is acceptable. Renderwise the performance isn't an issue -- filters work fast enough and only limited amount of results are rendered to DOM.
每个对象有大约100-150个字段/属性，但是在表内的任何给定时刻只显示一定数量的em(可以实时切换表列)。对于大型数据集，DB似乎正在推动大约10-100mb的json数据，在这个用例中是可以接受的。从渲染的角度来看，性能并不是问题——过滤器的工作速度足够快，并且只向DOM呈现有限的结果。
Everything already works, filters, listing ~100 rows against filters (+ "show 100 more"-mechanism etc), but I hit memory limit when I loaded around 8000 objects into array. This seems to reserve 2 gigabytes of RAM, which after Chrome stops running JS-code all together (even though strangely I'm not getting any kind of warning/error).
所有的东西都已经工作了，过滤器，列表~100行针对过滤器(+“显示100多”-机制等等)，但是当我将大约8000个对象加载到数组中时，我达到了内存限制。这似乎是预留了2g内存，在Chrome完全停止运行js代码之后(尽管奇怪的是我没有得到任何警告/错误)。
I benchmarked memory usage for rows and it seems that ~1000 rows reserves around 300mb of memory. This is most likely reserved by Vue reactivity observers.
我对行的内存使用进行了基准测试，结果发现~1000行存储了大约300mb的内存。这很可能是由Vue反应观察人士保留的。
Three questions:
三个问题:
- Is there a way to toggle reactivity for specific array-list objects (by index or such), so that objects within array itself are unobserved/non-mutable unless specifically called to become mutable (ie. when user clicks row, which enables edit-mode)?
- 是否有一种方法可以为特定的arraylist对象(通过索引或诸如此类)进行切换，从而使数组内的对象本身是未被观察到的/非可变的，除非被特别调用成为可变的(ie)。当用户单击row时，它支持编辑模式)?
- How would you implement handling of large datasets for Vue, as reactivity seems to bottleneck the memory usage? Please do not suggest "limit the results within backend", because it's not the solution which I'm seeking here (even though I may need to create two-part filtering, one for fetching smaller initial dataset and one for realtime filtering). Basically I'm trying to push boundaries of "end of memory" from 8 000 -> 80 000 by re-thinking the data-architecture with Vue. Is the only problem having dataset storaged within Vue's data-variables as reactive?
- 如何为Vue实现对大型数据集的处理，因为反应性似乎限制了内存的使用?请不要建议“限制后端结果”，因为它不是我在这里寻找的解决方案(尽管我可能需要创建两部分过滤，一个用于获取更小的初始数据集，一个用于实时过滤)。基本上，我试图通过使用Vue重新思考数据架构，将“内存结束”的边界从8000 - >80000扩展到8000。在Vue的数据变量中存储数据的唯一问题是反应性吗?
- One idea I have is to turn that "items" -dataset to non-observable/non-reactive with Object.freeze or some similar approach and have table to render two datasets: one for non-reactive and one for those which are currently within edit-mode (which would be pushed to "editableItems" dataset when row is clicked)... any suggestions here (anything simpler, so that I'm able to handle everything within one array?)
- 我的一个想法是将“项”-dataset转换为non-observable/non-reactive with Object.freeze或类似的方法，并使用table来呈现两个数据集:一个是non-reactive数据集，另一个是那些当前处于edit-mode的数据集(当单击row时，该数据集将被推到“editableItems”数据集)……这里有什么建议吗(有什么更简单的吗，这样我就可以在一个数组中处理所有东西了?)
I have done similar application on Angular 1, and it handled 50 000 rows quite well, so I'm sure it should be doable within Vue 2 as well... should be just a matter of finding a way on handling reactivity.
我在角1上也做过类似的应用，它能很好地处理5万行，所以我确信它在Vue 2中也可以。应该只是找到一种处理反应性的方法。
2 个解决方案
#1
2
From everything I've read, I see that you just don't need reactivity for that data, because:
从我读到的所有资料中，我发现你不需要对数据进行反应，因为:
Each row within table is toggleable, meaning that clicking the row changes the row to edit-mode, which enables Excel-like editing for that specific field/cell
表中的每一行都是可切换的，这意味着单击该行会将该行更改为编辑模式，从而允许对特定字段/单元格进行类似于excel的编辑
This means rows are not editable and data cannot be mutated without user interaction.
这意味着行是不可编辑的，数据在没有用户交互的情况下是不能改变的。
Each object has about ~100-150 fields/properties, but only certain amount of 'em are shown at any given moment within table (table columns can be toggled in real-time).
每个对象有大约100-150个字段/属性，但是在表内的任何给定时刻只显示一定数量的em(可以实时切换表列)。
You keep fields reactive but not display them.
您保持字段是被动的，但不显示它们。
And now your questions
现在你的问题
Is there a way to toggle reactivity for specific array-list objects (by index or such), so that objects within array itself are unobserved/non-mutable unless specifically called to become mutable (ie. when user clicks row, which enables edit-mode)?
是否有一种方法可以为特定的arraylist对象(通过索引或诸如此类)进行切换，从而使数组内的对象本身是未被观察到的/非可变的，除非被特别调用成为可变的(ie)。当用户单击row时，它支持编辑模式)?
If there's a single item that can be edited at a time, then why keep everything reactive? You can easily use a single variable to listen for that changes.
如果有一个单独的条目可以一次编辑，那么为什么要保持所有内容的反应性呢?您可以很容易地使用单个变量侦听该更改。
How would you implement handling of large datasets for Vue, as reactivity seems to bottleneck the memory usage?
如何为Vue实现对大型数据集的处理，因为反应性似乎限制了内存的使用?
It's all about implementation - you rarely end up in a situation when you need a huge list of items to be reactive. The more items you have, the more events needs to happen in order to use the reactivity. If you have 50k items and there are just a few events to mutate (like user modifying data manually), then you can easily listen for those events and make the reactivity manually rather than leave Vue handle all the data. You can check Vuex that can make your life a bit easier for you :)
这一切都与实现有关——当您需要大量的项目列表时，您很少会最终陷入被动的境地。您拥有的项越多，需要发生的事件就越多，才能使用反应性。如果您有50k个项目，并且只有几个事件需要修改(比如用户手工修改数据)，那么您可以轻松地侦听这些事件，并手动执行反应，而不是让Vue处理所有数据。你可以检查一下Vuex，它可以让你的生活更轻松一点:
One idea I have is to turn that "items" -dataset to non-observable/non-reactive with Object.freeze or some similar approach and have table to render two datasets: one for non-reactive and one for those which are currently within edit-mode (which would be pushed to "editableItems" dataset when row is clicked)
我的一个想法是用Object.freeze或类似的方法将“项”-dataset转换为non-observable/non-reactive，并使用表来呈现两个数据集:一个是non-reactive数据集，另一个是那些当前处于edit-mode的数据集(当单击row时，该数据集将被推到“editableItems”数据集)
This is kind of going in the right direction but there is no need to support two arrays. Imagine using something like this:
这是一个正确的方向，但不需要支持两个数组。想象一下使用这样的东西:
data: function() {
return {
editingItem: {}
// when editing is enabled bind the input fields to this item
}
},
created: function() {
this.items = [] // your items, can be used in markdown in the loop, but won't be reactive!
},
watch: {
editingItem: function(data) {
// this method will be called whenever user edits the input fields
// here you can do whatever you want
// like get item's id, find it in the array and update it's properties
// something like manual reactivity ;)
}
}
#2
3
It's been a while since I asked this question and I finally got to optimize this part of my project. I'd like to give few pointers for anyone having these performance and/or memory-issues.
我问这个问题已经有一段时间了，我最终优化了我项目的这一部分。我想给那些有这些性能和/或内存问题的人一些建议。
Vue documentation never really explained it, but as Andrey pointed out you CAN use the component-object as an data-storage for your custom objects & object-lists. After all, it's just an normal javascript-object.
Vue文档从来没有真正解释过它，但是正如Andrey指出的，您可以使用组件对象作为自定义对象和对象列表的数据存储。毕竟，它只是一个普通的javascript对象。
After optimization my list component setup looks somewhat like this:
优化后，我的列表组件设置如下:
module.exports = {
items: [],
mixins: [sharedUtils],
data: function() {
return {
columns: {
all: []
etc... Lot's of data & methods
The items-array is filled with thousands of complex objects (about 80mb of data, 6mb compressed) which I'm handling as non-reactive. This proved to be less of an issue than I would have thought -- Instead of using v-for directly against items I was already using structure in which I triggered filtering of this array whenever user clicked some filter-button and/or inputted string-filtering (such as name). Basically this "processFilters"-method goes through non-responsive items-array and returns filteredItems, which is stored in data-context. Thus it automatically becomes reactive as it's mutated.
items-array中填充了数千个复杂对象(大约80mb的数据，6mb的压缩)，我将以无反应方式处理这些对象。事实证明，这并不像我想象的那么严重——我使用的不是v，而是已经使用的结构，每当用户单击某个过滤按钮和/或输入字符串过滤(如名称)时，我就触发对该数组的过滤。基本上，这个“processfilter”方法通过无响应的条目数组并返回filtereditem，后者存储在数据上下文中。因此，当它发生突变时，它会自动反应。
<tr v-for="item in filteredItems"
This way all the items within filteredItems stay reactive, but also lose reactivity when they are filtered out, thus saving bunch-load of memory. Whopping 1200mb shrunk to 400mb, which was exactly what I was looking for. Clever!
这样，filtereditem中的所有项都保持反应性，但在过滤时也会失去反应性，从而节省了大量内存。巨大的1200mb缩小到400mb，这正是我要找的。聪明!
There are few issues which need to be addressed. Since items doesn't exist in data-context you cannot use it directly within template. This means that instead of writing...
需要解决的问题很少。由于项目不存在于数据上下文，所以不能在模板中直接使用它。这意味着，与其写作……
<div v-if="items.length > 0 && everythingElseIsReady">
... I had to store length of items-array to separate data prop. This could have been fixed with computed value as well, but I like to keep those properties existing.
…我必须存储项目数组的长度来分离数据支柱。这也可以用计算值来固定，但是我喜欢保留这些属性。
Giving up the reactivity of your main data-array isn't such a bad thing after all - The most important part is to understand that modifications which are made directly against items within that base-array are never triggering any changes to UI and/or sub-components (douh). This shouldn't be such an issue as long as you separate your code in such a way that you have "hidden data container" which holds all the results from backend, and you have smaller (filtered) presentation array of that large container. By using good REST-architecture you should already be good to go with non-reactive data-storage, as long as you remember to check that after saving the item within non-reactive data storage has also been updated to latest revision.
放弃主数据数组的反应性毕竟不是一件坏事——最重要的是要理解，直接针对该基数组中的项进行的修改不会触发对UI和/或子组件(douh)的任何更改。只要以“隐藏数据容器”的方式分离代码，就不应该出现这种问题，“隐藏数据容器”保存来自后端的所有结果，并且拥有该大型容器的更小(经过过滤的)表示数组。通过使用良好的REST-architecture，您应该已经能够很好地使用非反应性数据存储，只要您记得在将项保存到非反应性数据存储中之后检查它是否已经更新到最新的修订。
Additionally I was baffled by how little it matters performance-wise how many micro-components there are against hundreds of rows. The render takes a hit obviously, but even if I were to pass large props thousands of times (as I have thousands of instances of input-cells) it didn't seem to hit the memory. One of this kind of objects is my global translations-key/value-pair object, having over 20 000 lines of translated strings... but it still didn't matter. This makes sense, as Javascript uses object-references and Vue Core seems to be properly coded, so as long as you use such configuration objects as props you are simply referring from thousands of objects to the same data-set.
此外，我还感到困惑的是，就性能而言，有多少微型组件与数百行相对。渲染显然会受到影响，但即使我要传递数千次大型道具(因为我有数千个输入单元)，它似乎也不会影响到内存。这类对象之一是我的全局平移-键/值对对象，有超过20000行翻译字符串…但这并不重要。这是有道理的，因为Javascript使用对象引用，而Vue Core似乎是正确编码的，所以只要您使用这些配置对象作为道具，您就可以从数千个对象引用到相同的数据集。
Finally, I'd say start going crazy with complex CRUD objects without fear of hitting memory limit!
最后，我要说的是，开始疯狂地使用复杂的CRUD对象，不要担心会影响内存限制!
Huge thanks for Andrey Popov for giving nudge towards right direction!
非常感谢安德烈·波波夫的指点!
#1
2
From everything I've read, I see that you just don't need reactivity for that data, because:
从我读到的所有资料中，我发现你不需要对数据进行反应，因为:
Each row within table is toggleable, meaning that clicking the row changes the row to edit-mode, which enables Excel-like editing for that specific field/cell
表中的每一行都是可切换的，这意味着单击该行会将该行更改为编辑模式，从而允许对特定字段/单元格进行类似于excel的编辑
This means rows are not editable and data cannot be mutated without user interaction.
这意味着行是不可编辑的，数据在没有用户交互的情况下是不能改变的。
Each object has about ~100-150 fields/properties, but only certain amount of 'em are shown at any given moment within table (table columns can be toggled in real-time).
每个对象有大约100-150个字段/属性，但是在表内的任何给定时刻只显示一定数量的em(可以实时切换表列)。
You keep fields reactive but not display them.
您保持字段是被动的，但不显示它们。
And now your questions
现在你的问题
Is there a way to toggle reactivity for specific array-list objects (by index or such), so that objects within array itself are unobserved/non-mutable unless specifically called to become mutable (ie. when user clicks row, which enables edit-mode)?
是否有一种方法可以为特定的arraylist对象(通过索引或诸如此类)进行切换，从而使数组内的对象本身是未被观察到的/非可变的，除非被特别调用成为可变的(ie)。当用户单击row时，它支持编辑模式)?
If there's a single item that can be edited at a time, then why keep everything reactive? You can easily use a single variable to listen for that changes.
如果有一个单独的条目可以一次编辑，那么为什么要保持所有内容的反应性呢?您可以很容易地使用单个变量侦听该更改。
How would you implement handling of large datasets for Vue, as reactivity seems to bottleneck the memory usage?
如何为Vue实现对大型数据集的处理，因为反应性似乎限制了内存的使用?
It's all about implementation - you rarely end up in a situation when you need a huge list of items to be reactive. The more items you have, the more events needs to happen in order to use the reactivity. If you have 50k items and there are just a few events to mutate (like user modifying data manually), then you can easily listen for those events and make the reactivity manually rather than leave Vue handle all the data. You can check Vuex that can make your life a bit easier for you :)
这一切都与实现有关——当您需要大量的项目列表时，您很少会最终陷入被动的境地。您拥有的项越多，需要发生的事件就越多，才能使用反应性。如果您有50k个项目，并且只有几个事件需要修改(比如用户手工修改数据)，那么您可以轻松地侦听这些事件，并手动执行反应，而不是让Vue处理所有数据。你可以检查一下Vuex，它可以让你的生活更轻松一点:
One idea I have is to turn that "items" -dataset to non-observable/non-reactive with Object.freeze or some similar approach and have table to render two datasets: one for non-reactive and one for those which are currently within edit-mode (which would be pushed to "editableItems" dataset when row is clicked)
我的一个想法是用Object.freeze或类似的方法将“项”-dataset转换为non-observable/non-reactive，并使用表来呈现两个数据集:一个是non-reactive数据集，另一个是那些当前处于edit-mode的数据集(当单击row时，该数据集将被推到“editableItems”数据集)
This is kind of going in the right direction but there is no need to support two arrays. Imagine using something like this:
这是一个正确的方向，但不需要支持两个数组。想象一下使用这样的东西:
data: function() {
return {
editingItem: {}
// when editing is enabled bind the input fields to this item
}
},
created: function() {
this.items = [] // your items, can be used in markdown in the loop, but won't be reactive!
},
watch: {
editingItem: function(data) {
// this method will be called whenever user edits the input fields
// here you can do whatever you want
// like get item's id, find it in the array and update it's properties
// something like manual reactivity ;)
}
}
#2
3
It's been a while since I asked this question and I finally got to optimize this part of my project. I'd like to give few pointers for anyone having these performance and/or memory-issues.
我问这个问题已经有一段时间了，我最终优化了我项目的这一部分。我想给那些有这些性能和/或内存问题的人一些建议。
Vue documentation never really explained it, but as Andrey pointed out you CAN use the component-object as an data-storage for your custom objects & object-lists. After all, it's just an normal javascript-object.
Vue文档从来没有真正解释过它，但是正如Andrey指出的，您可以使用组件对象作为自定义对象和对象列表的数据存储。毕竟，它只是一个普通的javascript对象。
After optimization my list component setup looks somewhat like this:
优化后，我的列表组件设置如下:
module.exports = {
items: [],
mixins: [sharedUtils],
data: function() {
return {
columns: {
all: []
etc... Lot's of data & methods
The items-array is filled with thousands of complex objects (about 80mb of data, 6mb compressed) which I'm handling as non-reactive. This proved to be less of an issue than I would have thought -- Instead of using v-for directly against items I was already using structure in which I triggered filtering of this array whenever user clicked some filter-button and/or inputted string-filtering (such as name). Basically this "processFilters"-method goes through non-responsive items-array and returns filteredItems, which is stored in data-context. Thus it automatically becomes reactive as it's mutated.
items-array中填充了数千个复杂对象(大约80mb的数据，6mb的压缩)，我将以无反应方式处理这些对象。事实证明，这并不像我想象的那么严重——我使用的不是v，而是已经使用的结构，每当用户单击某个过滤按钮和/或输入字符串过滤(如名称)时，我就触发对该数组的过滤。基本上，这个“processfilter”方法通过无响应的条目数组并返回filtereditem，后者存储在数据上下文中。因此，当它发生突变时，它会自动反应。
<tr v-for="item in filteredItems"
This way all the items within filteredItems stay reactive, but also lose reactivity when they are filtered out, thus saving bunch-load of memory. Whopping 1200mb shrunk to 400mb, which was exactly what I was looking for. Clever!
这样，filtereditem中的所有项都保持反应性，但在过滤时也会失去反应性，从而节省了大量内存。巨大的1200mb缩小到400mb，这正是我要找的。聪明!
There are few issues which need to be addressed. Since items doesn't exist in data-context you cannot use it directly within template. This means that instead of writing...
需要解决的问题很少。由于项目不存在于数据上下文，所以不能在模板中直接使用它。这意味着，与其写作……
<div v-if="items.length > 0 && everythingElseIsReady">
... I had to store length of items-array to separate data prop. This could have been fixed with computed value as well, but I like to keep those properties existing.
…我必须存储项目数组的长度来分离数据支柱。这也可以用计算值来固定，但是我喜欢保留这些属性。
Giving up the reactivity of your main data-array isn't such a bad thing after all - The most important part is to understand that modifications which are made directly against items within that base-array are never triggering any changes to UI and/or sub-components (douh). This shouldn't be such an issue as long as you separate your code in such a way that you have "hidden data container" which holds all the results from backend, and you have smaller (filtered) presentation array of that large container. By using good REST-architecture you should already be good to go with non-reactive data-storage, as long as you remember to check that after saving the item within non-reactive data storage has also been updated to latest revision.
放弃主数据数组的反应性毕竟不是一件坏事——最重要的是要理解，直接针对该基数组中的项进行的修改不会触发对UI和/或子组件(douh)的任何更改。只要以“隐藏数据容器”的方式分离代码，就不应该出现这种问题，“隐藏数据容器”保存来自后端的所有结果，并且拥有该大型容器的更小(经过过滤的)表示数组。通过使用良好的REST-architecture，您应该已经能够很好地使用非反应性数据存储，只要您记得在将项保存到非反应性数据存储中之后检查它是否已经更新到最新的修订。
Additionally I was baffled by how little it matters performance-wise how many micro-components there are against hundreds of rows. The render takes a hit obviously, but even if I were to pass large props thousands of times (as I have thousands of instances of input-cells) it didn't seem to hit the memory. One of this kind of objects is my global translations-key/value-pair object, having over 20 000 lines of translated strings... but it still didn't matter. This makes sense, as Javascript uses object-references and Vue Core seems to be properly coded, so as long as you use such configuration objects as props you are simply referring from thousands of objects to the same data-set.
此外，我还感到困惑的是，就性能而言，有多少微型组件与数百行相对。渲染显然会受到影响，但即使我要传递数千次大型道具(因为我有数千个输入单元)，它似乎也不会影响到内存。这类对象之一是我的全局平移-键/值对对象，有超过20000行翻译字符串…但这并不重要。这是有道理的，因为Javascript使用对象引用，而Vue Core似乎是正确编码的，所以只要您使用这些配置对象作为道具，您就可以从数千个对象引用到相同的数据集。
Finally, I'd say start going crazy with complex CRUD objects without fear of hitting memory limit!
最后，我要说的是，开始疯狂地使用复杂的CRUD对象，不要担心会影响内存限制!
Huge thanks for Andrey Popov for giving nudge towards right direction!
非常感谢安德烈·波波夫的指点!