I'm new to web development, so please excuse me if this question is trivial.
我是网络开发的新手,所以如果这个问题很简单,请原谅。
I have a background in machine learning, and am currently working on an application that involves various uses of machine learning techniques. When one is writing such an application, where are feature weights normally stored? The idea is that as users add data, the model will update feature weights to better learn the data.
我有机器学习的背景,目前正在开发涉及机器学习技术的各种用途的应用程序。在编写这样的应用程序时,通常存储的特征权重在哪里?这个想法是,当用户添加数据时,模型将更新要素权重以更好地学习数据。
Does one normally store feature weights in a database? How does one normally deal with this scenario? For example, suppose that I have a trained model and want to evaluate for a certain input. If feature weights are stored in a database, that requires looking up possibly several different values, which seems very computationally expensive.
通常会在数据库中存储要素权重吗?通常如何处理这种情况?例如,假设我有一个训练有素的模型,并想要评估某个输入。如果要素权重存储在数据库中,则需要查找可能有几个不同的值,这在计算上看起来非常昂贵。
What are the best practices for managing this?
管理此问题的最佳做法是什么?
1 个解决方案
#1
0
It would really depend on a specific case and environment. If performance matters, you would probably want keep them in memory and store in the database or file only for the means of keeping them permanently.
这实际上取决于具体的案例和环境。如果性能很重要,您可能希望将它们保存在内存中并存储在数据库或文件中,仅用于永久保存它们。
When loading the application, it would be best to load the full model into memory. This might or might not be possible, based on the available resources and the amount of data. If your model is too large, you could pre-load the needed parts as required.
加载应用程序时,最好将完整模型加载到内存中。根据可用资源和数据量,这可能是也可能是不可能的。如果您的模型太大,您可以根据需要预加载所需的零件。
You would need to make expensive I/O operations only on application load and when model changes (and even for the changes you can defer the write ops).
您只需要在应用程序加载时以及模型更改时进行昂贵的I / O操作(甚至可以在更改时推迟写入操作)。
You can find similar discussion here: Neural Network / Machine Learning memory storage
你可以在这里找到类似的讨论:神经网络/机器学习内存存储
#1
0
It would really depend on a specific case and environment. If performance matters, you would probably want keep them in memory and store in the database or file only for the means of keeping them permanently.
这实际上取决于具体的案例和环境。如果性能很重要,您可能希望将它们保存在内存中并存储在数据库或文件中,仅用于永久保存它们。
When loading the application, it would be best to load the full model into memory. This might or might not be possible, based on the available resources and the amount of data. If your model is too large, you could pre-load the needed parts as required.
加载应用程序时,最好将完整模型加载到内存中。根据可用资源和数据量,这可能是也可能是不可能的。如果您的模型太大,您可以根据需要预加载所需的零件。
You would need to make expensive I/O operations only on application load and when model changes (and even for the changes you can defer the write ops).
您只需要在应用程序加载时以及模型更改时进行昂贵的I / O操作(甚至可以在更改时推迟写入操作)。
You can find similar discussion here: Neural Network / Machine Learning memory storage
你可以在这里找到类似的讨论:神经网络/机器学习内存存储