节点。在内存或数据库中存储对象?

时间:2021-09-21 22:01:52

Am developing a node js application, which reads a json list from a centralised db

正在开发一个节点js应用程序,它从集中的db中读取json列表

List Object is around 1.2mb(if kept in txt file)

List对象大约1.2mb(如果保存在txt文件中)

Requirement is like, data is to be refreshed every 24 hours, so i kept a cron job for it

需求是这样的,数据每24小时更新一次,所以我为它保留了一个cron作业。

Now after fetching data i keep it into a db(couchbase) which is locally running on my server

现在,在获取数据之后,我将它保存到本地运行的db(couchbase)中

Data access is very frequent, i get around 1 or 2 req per sec and nearly all req need that Object

数据访问非常频繁,我大约每秒钟得到1或2个req,几乎所有req都需要那个对象

Is it good to keep that Object as a in memory object in node js or to keep it in local db ?

将该对象保存为节点js中的内存对象,还是将其保存在本地db中?

What are the advantages and disadvantages of both ?

两者的优缺点是什么?

Object only read for all requests , only written once by cron job

对象只读取所有请求,仅由cron job编写一次

it's a high end system, i7 quad core, 16gb ram

这是一个高端系统,i7四核,16gb内存

2 个解决方案

#1


2  

  1. It depends from your hardware
  2. 这取决于你的硬件。
  3. If this object is immutable, per requests, it's better to keep it in memory. If no - depends.
  4. 如果这个对象是不可变的,那么最好将它保存在内存中。如果没有,视情况而定。
  5. In any case workflow open connection to db - fetch data - return result - free data will consume more resources than caching in memory.
  6. 在任何情况下,工作流打开连接到db - fetch数据-返回结果-免费数据将消耗更多的资源而不是缓存在内存中。

For example, in our project we processing high definition images, and keep all objects in memory - 3-7mb in raw format. Tests shows that this is much efficient than usage of any caching systems, such as redis or couch base.

例如,在我们的项目中,我们处理高清晰度图像,并将所有对象保存在内存中——3-7mb为原始格式。测试表明,这比使用任何缓存系统(如redis或couch base)都要有效得多。

#2


1  

I would keep the most recent version as memory object, and store it as well. That way you have a backup if anything crashes. If you edit the file however, I would only keep it as database object.

我将把最近的版本保存为内存对象,并将其存储。这样你就有了备份。但是,如果您编辑这个文件,我将只将它作为数据库对象保存。

Accessing the DB for that object every 2 seconds would probably work fine, but 1.2MB of memory is not that much and if you can keep that contained, your server won't likely run into problems.

每两秒钟访问该对象的DB可能会运行得很好,但是1.2MB的内存并不是那么多,如果您能够保持它的受控性,您的服务器就不会出现问题。

The DB is a little slow compared to memory, but has the advantage to (most likely) be thread-safe. If you would edit the document, you could run into thread problems with a memory object.

与内存相比,DB有点慢,但它的优势(很可能)是线程安全的。如果您要编辑文档,您可能会遇到内存对象的线程问题。

You know the application and the requirements, you should be able to tell if you would need a thread-safe database, or if you need to safe your memory on the server. If you don't know, we need to see the actual code and use-cases to tell you what you could do best.

您了解应用程序和需求,您应该能够判断是否需要线程安全的数据库,或者是否需要在服务器上保护内存。如果您不知道,我们需要看到实际的代码和用例来告诉您什么是最好的。

#1


2  

  1. It depends from your hardware
  2. 这取决于你的硬件。
  3. If this object is immutable, per requests, it's better to keep it in memory. If no - depends.
  4. 如果这个对象是不可变的,那么最好将它保存在内存中。如果没有,视情况而定。
  5. In any case workflow open connection to db - fetch data - return result - free data will consume more resources than caching in memory.
  6. 在任何情况下,工作流打开连接到db - fetch数据-返回结果-免费数据将消耗更多的资源而不是缓存在内存中。

For example, in our project we processing high definition images, and keep all objects in memory - 3-7mb in raw format. Tests shows that this is much efficient than usage of any caching systems, such as redis or couch base.

例如,在我们的项目中,我们处理高清晰度图像,并将所有对象保存在内存中——3-7mb为原始格式。测试表明,这比使用任何缓存系统(如redis或couch base)都要有效得多。

#2


1  

I would keep the most recent version as memory object, and store it as well. That way you have a backup if anything crashes. If you edit the file however, I would only keep it as database object.

我将把最近的版本保存为内存对象,并将其存储。这样你就有了备份。但是,如果您编辑这个文件,我将只将它作为数据库对象保存。

Accessing the DB for that object every 2 seconds would probably work fine, but 1.2MB of memory is not that much and if you can keep that contained, your server won't likely run into problems.

每两秒钟访问该对象的DB可能会运行得很好,但是1.2MB的内存并不是那么多,如果您能够保持它的受控性,您的服务器就不会出现问题。

The DB is a little slow compared to memory, but has the advantage to (most likely) be thread-safe. If you would edit the document, you could run into thread problems with a memory object.

与内存相比,DB有点慢,但它的优势(很可能)是线程安全的。如果您要编辑文档,您可能会遇到内存对象的线程问题。

You know the application and the requirements, you should be able to tell if you would need a thread-safe database, or if you need to safe your memory on the server. If you don't know, we need to see the actual code and use-cases to tell you what you could do best.

您了解应用程序和需求,您应该能够判断是否需要线程安全的数据库,或者是否需要在服务器上保护内存。如果您不知道,我们需要看到实际的代码和用例来告诉您什么是最好的。