如何Javascript /节点。js引擎在内存中存储JSON ?

I have a 70 MB JSON file, which I read from a Node.js script and assign to a variable.

我有一个70mb的JSON文件，它是从一个节点读取的。脚本，并分配给一个变量。

console.log(process.memoryUsage());
let data=require('../newJSON.json');
console.log(process.memoryUsage());

Output:

输出:

{ rss: 28184576,
  heapTotal: 6283264,
  heapUsed: 4199672,
  external: 8252 }
{ rss: 724721664,
  heapTotal: 695595008,
  heapUsed: 663708016,
  external: 8252 }

It seems that 70 MB JSON takes 632 MB of memory. I am interested in understanding how does JSON is stored into memory by Node Js/ Javascript?

看起来70mb的JSON占用了632mb的内存。我感兴趣的是如何通过Node Js/ Javascript将JSON存储到内存中?

1 个解决方案

#1

First off, JSON is just a string representation of objects. There is nothing special about "JSON objects" -- the JSON parser parses the JSON string and creates regular JavaScript objects from it. This:

首先，JSON只是对象的字符串表示形式。“JSON对象”没有什么特别之处——JSON解析器解析JSON字符串并从中创建常规的JavaScript对象。这样的:

var a = JSON.parse('{"foo": "bar"}');

and this:

这:

var a = new Object(); a.foo = "bar";

are completely equivalent.

是完全相同的。

Object storage in memory is complicated, because modern JavaScript engines have pretty nifty optimizations for various different circumstances depending on what your code is doing.

内存中的对象存储是复杂的，因为现代JavaScript引擎对各种不同情况进行了漂亮的优化，这取决于您的代码在做什么。

JSON string length and size of the corresponding object in memory are not strictly correlated; in most cases the JSON representation is expected to be smaller, sometimes by a lot. E.g. for the innermost nesting of your example: "a":0, takes 6 bytes, whereas for one more property in the created object, you need:

在内存中对应对象的JSON字符串长度和大小没有严格关联;在大多数情况下，JSON表示会更小，有时会更小。

one pointer for the property's name, "a"
属性名称的一个指针，“a”
one pointer for the property's attributes (writable, enumerable, configurable)
属性属性的一个指针(可写、可枚举、可配置)
one pointer for the property's value, 0
属性值为0的指针。
assuming the object is in dictionary mode: on average, approximately two pointers of slack
假设对象处于字典模式:平均来说，大约有两个松弛指针

On a 64-bit platform, that adds up to ~40 bytes.

在一个64位平台上，加起来大约是40字节。

If you look at an entire object of similar shape: {"a":0,"b":1} is 13 characters, whereas the memory requirement is:

如果您查看一个具有类似形状的完整对象:{“a”:0，“b”:1}为13个字符，而内存要求为:

"map" pointer
“地图”指针
"elements" pointer (unused)
“元素”指针(未使用)
out-of-object "properties" pointer (unused)
out-of-object“属性”指针(未使用)
value of first property (0)
第一属性值(0)
value of second property (1)
第二属性值(1)
the object's "map": 11 pointers (could be shared with other objects of the same shape, but if you have only one such object, there's nothing to share it with)
对象的“map”:11个指针(可以与相同形状的其他对象共享，但是如果只有一个这样的对象，就没有什么可与之共享的了)
the object's map's property descriptors: 10 pointers
对象映射的属性描述符:10个指针

In total, 26 pointers or 208 bytes.

总共26个指针或208字节。

Lastly, there's a chance that some of the memory usage you see is from temporary objects that the GC will clean up over time.

最后，您看到的一些内存使用可能来自临时对象，GC将随时间清理这些对象。

#1