I'm searching for best solution to store sensitive data in database. I know that this is common problem and i have done my homework (at least this is what i think), but i wanted to ask here before i will make a decision.
我正在寻找将敏感数据存储在数据库中的最佳解决方案。我知道这是常见的问题而且我完成了我的作业(至少这是我的想法),但我想在此之前先问我做出决定。
Assumptions:
- Encrypted data needs to be decrypted. We are talking about SMTP credentials like username, password, host, port itp.
加密数据需要解密。我们正在谈论SMTP凭据,如用户名,密码,主机,端口itp。
I was thinking about 2 concepts:
我在考虑两个概念:
-
Encrypt data with help of passlib.totp library. To make those data a bit safer i will keep key in separate file. Then from what i can see i can use this library to decrypt data to plain text using my key.
借助passlib.totp库加密数据。为了使这些数据更安全,我会将密钥保存在单独的文件中。然后从我可以看到我可以使用此库使用我的密钥将数据解密为纯文本。
-
The other concept was to encrypt and decrypt data during query request with help of postgres:
另一个概念是在postgres的帮助下在查询请求期间加密和解密数据:
insert into demo(pw) values ( encrypt( 'data', 'key', 'aes') );And:
decrypt(pw, 'key', 'aes'), 'utf-8')Here the key will be stored also in separate file.
这里的密钥也将存储在单独的文件中。
So my questions are:
所以我的问题是:
- What is better approach to encrypt / decrypt data, in code or in database?
- Are there any better (stronger) libraries to use than passlib.totp -> i have no experience with that library (i'm aware that encryption / decryption is not the moste secure way of storing password -> password supposed to be hased but i need it in plain text to use users smtp gate).
在代码或数据库中加密/解密数据的更好方法是什么?
有没有比passlib.totp更好(更强)的库使用 - >我没有使用该库的经验(我知道加密/解密不是存储密码的最安全的方式 - >密码应该被hased但是我需要在纯文本中使用用户smtp gate)。
3 个解决方案
#1
1
2) The other concept was to encrypt and decrypt data during query request with help of postgres: insert into demo(pw) values ( encrypt( 'data', 'key', 'aes') ); and decrypt(pw, 'key', 'aes'), 'utf-8') Here the key will be stored also in separate file.
2)另一个概念是在postgres的帮助下在查询请求期间加密和解密数据:插入demo(pw)值(encrypt('data','key','aes'));和解密(pw,'key','aes'),'utf-8')这里的密钥也将存储在单独的文件中。
I wouldn't recommend that, because it's way too easy for the keys to get exposed in pg_stat_activity, the logs, etc. PostgreSQL doesn't have log masking features that would protect against that.
我不建议这样做,因为密钥很容易暴露在pg_stat_activity,日志等中.PostgreSQL没有日志屏蔽功能可以防止这种情况发生。
I strongly advise you to use app-side crypto. Use a crypto offload device if security is crucial, so key extraction isn't possible for most attackers. Or require the key to be unlocked by the admin entering a passphrase at app start, so the key is never stored unencrypted on disk - then the attacker has to steal it from memory. But even an unencrypted key file somewhere non-obvious is better than in-db crypto, IMO, since it at least separates the key from the data.
我强烈建议你使用app-side crypto。如果安全性至关重要,请使用加密卸载设备,因此大多数攻击者无法进行密钥提取。或者要求管理员在应用启动时输入密码来解锁密钥,因此密钥永远不会以未加密的方式存储在磁盘上 - 然后攻击者必须从内存中窃取密钥。但即使是非显而易见的未加密密钥文件也比in-db加密,IMO更好,因为它至少将密钥与数据分开。
#2
0
Ultimately, your application needs to be able to recover the plaintext passwords using some sort of key. If your system is compromised, you have to assume that the malicious user will simply find your key (whether in the database or on disk) and perform the exact same decryption that your application performs.
最终,您的应用程序需要能够使用某种键来恢复明文密码。如果您的系统遭到入侵,您必须假设恶意用户只需找到您的密钥(无论是在数据库中还是在磁盘上),并执行与您的应用程序执行完全相同的解密。
When your system needs to store passwords in a recoverable form, i.e. to authenticate with external systems, at best you can only obfuscate that information. This is what people mean when they refer to "security through obscurity", and it's a bad idea.
当您的系统需要以可恢复的形式存储密码时,即通过外部系统进行身份验证时,您最多只能对该信息进行模糊处理。这就是人们提到“通过默默无闻的安全”时的意思,这是一个坏主意。
When you give the appearance of security, without actually securing something, then that can make things even more dangerous. You, or other people administering the system, may overlook other important security measures because they believe that there is already a layer of protection for the sensitive information. It can create a social situation where sensitive information may be more likely to leak, than it would if it was assumed that the only way to safeguard the info is to safeguard the systems that hold it. It can also cause people to believe that if data is stolen from the database, "maybe it's okay because they can't decrypt it". This is how you end up responsible for leaking credentials to the wide world, because you must assume that if your application can get the plaintext data, so can an attacker that has compromised your application.
当你给出安全的外观而没有实际保护某些东西时,那会使事情变得更加危险。您或其他管理系统的人可能会忽略其他重要的安全措施,因为他们认为敏感信息已经存在一层保护。它可以创造一种敏感信息可能更容易泄漏的社交场合,而不是假设保护信息的唯一方法是保护持有它的系统。它还可以使人们相信,如果数据从数据库中被盗,“也许它没关系,因为它们无法解密”。这就是你最终要向全世界泄露凭据的责任,因为你必须假设如果你的应用程序可以获得纯文本数据,那么攻击者也可以破坏你的应用程序。
There may be a very very small advantage to encrypting a multipart key on multiple systems (i.e. part on the file system, part in the database), so that somebody who gains access to one system doesn't necessarily gain access to the other. It's reasonable to say that this might, in fact, delay an attack, or deter a lazy attacker. Generally speaking however, the place where your application lives has access to both these things anyway so if your application is compromised then you must assume that the data is compromised. (Edit: You mentioned in another comment you have users who a) shouldn't know the passwords stored in the DB, but b) will have access to the DB directly. Believe me, there is a non-zero chance of one of those users getting all of those passwords if you do this. By going down this path, you're putting your faith in a faulty layer of protection.)
加密多个系统上的多部分密钥(即文件系统的一部分,数据库中的部分)可能有非常小的优点,因此获得对一个系统的访问权的人不一定能够访问另一个系统。事实上,这可能会延迟攻击,或阻止懒惰的攻击者,这是合理的。但是,一般来说,您的应用程序所在的位置无论如何都可以访问这些内容,因此如果您的应用程序被泄露,那么您必须假设数据已被泄露。 (编辑:你在另一个评论中提到你有用户a)不应该知道存储在数据库中的密码,但是b)可以直接访问数据库。相信我,如果你这样做,其中一个用户获得所有这些密码的概率非零。沿着这条路走下去,你就会把自己的信仰放在一个错误的保护层上。)
tl;dr reversible encryption when storing sensitive data rarely has practical, real security value. If you're trying to tick a compliance check box, and you have no power to overrule someone up the chain who needs you to tick that box, then by all means implement something that "encrypts" the data. But if you're actually trying to secure your system, look elsewhere: here be dragons.
tl; dr存储敏感数据时的可逆加密很少具有实用,真实的安全价值。如果您正在尝试勾选合规性复选框,并且您没有权力推翻需要您勾选该框的链上的某个人,那么无论如何都要实施“加密”数据的内容。但是,如果你真的想要保护你的系统,那就看看其他地方:这里是龙。
#3
0
You may check Vault project, a tool for managing secrets:
您可以查看Vault项目,这是一个管理机密的工具:
General Secret Storage
一般秘密存储
At a bare minimum, Vault can be used for the storage of any secrets. For example, Vault would be a fantastic way to store sensitive environment variables, database credentials, API keys, etc.
最低限度,Vault可用于存储任何机密。例如,Vault是存储敏感环境变量,数据库凭据,API密钥等的绝佳方式。
Compare this with the current way to store these which might be plaintext in files, configuration management, a database, etc. It would be much safer to query these using vault read or the API. This protects the plaintext version of these secrets as well as records access in the Vault audit log.
将其与当前存储这些文件,配置管理,数据库等明文的方式进行比较。使用文件库读取或API查询这些文件会更安全。这可以保护这些机密的纯文本版本以及Vault审核日志中的记录访问权限。
Employee Credential Storage
员工凭证存储
While this overlaps with "General Secret Storage", Vault is a good mechanism for storing credentials that employees share to access web services. The audit log mechanism lets you know what secrets an employee accessed and when an employee leaves, it is easier to roll keys and understand which keys have and haven't been rolled.
虽然这与“一般秘密存储”重叠,但Vault是一种很好的机制,用于存储员工共享以访问Web服务的凭据。审计日志机制可以让您了解员工访问的秘密以及员工离职时,更容易滚动密钥并了解哪些密钥已经滚动和未滚动。
Vault server stores data in encrypted form. Data can be retrieved via command line or REST API. Server must be in unsealed state to return decrypted data - unsealing needs specific number of shards of the master key. Once server is restarted, you need to unseal it again.
Vault服务器以加密形式存储数据。可以通过命令行或REST API检索数据。服务器必须处于未密封状态才能返回解密数据 - 启封需要特定数量的主密钥分片。重新启动服务器后,您需要再次启动它。
#1
1
2) The other concept was to encrypt and decrypt data during query request with help of postgres: insert into demo(pw) values ( encrypt( 'data', 'key', 'aes') ); and decrypt(pw, 'key', 'aes'), 'utf-8') Here the key will be stored also in separate file.
2)另一个概念是在postgres的帮助下在查询请求期间加密和解密数据:插入demo(pw)值(encrypt('data','key','aes'));和解密(pw,'key','aes'),'utf-8')这里的密钥也将存储在单独的文件中。
I wouldn't recommend that, because it's way too easy for the keys to get exposed in pg_stat_activity, the logs, etc. PostgreSQL doesn't have log masking features that would protect against that.
我不建议这样做,因为密钥很容易暴露在pg_stat_activity,日志等中.PostgreSQL没有日志屏蔽功能可以防止这种情况发生。
I strongly advise you to use app-side crypto. Use a crypto offload device if security is crucial, so key extraction isn't possible for most attackers. Or require the key to be unlocked by the admin entering a passphrase at app start, so the key is never stored unencrypted on disk - then the attacker has to steal it from memory. But even an unencrypted key file somewhere non-obvious is better than in-db crypto, IMO, since it at least separates the key from the data.
我强烈建议你使用app-side crypto。如果安全性至关重要,请使用加密卸载设备,因此大多数攻击者无法进行密钥提取。或者要求管理员在应用启动时输入密码来解锁密钥,因此密钥永远不会以未加密的方式存储在磁盘上 - 然后攻击者必须从内存中窃取密钥。但即使是非显而易见的未加密密钥文件也比in-db加密,IMO更好,因为它至少将密钥与数据分开。
#2
0
Ultimately, your application needs to be able to recover the plaintext passwords using some sort of key. If your system is compromised, you have to assume that the malicious user will simply find your key (whether in the database or on disk) and perform the exact same decryption that your application performs.
最终,您的应用程序需要能够使用某种键来恢复明文密码。如果您的系统遭到入侵,您必须假设恶意用户只需找到您的密钥(无论是在数据库中还是在磁盘上),并执行与您的应用程序执行完全相同的解密。
When your system needs to store passwords in a recoverable form, i.e. to authenticate with external systems, at best you can only obfuscate that information. This is what people mean when they refer to "security through obscurity", and it's a bad idea.
当您的系统需要以可恢复的形式存储密码时,即通过外部系统进行身份验证时,您最多只能对该信息进行模糊处理。这就是人们提到“通过默默无闻的安全”时的意思,这是一个坏主意。
When you give the appearance of security, without actually securing something, then that can make things even more dangerous. You, or other people administering the system, may overlook other important security measures because they believe that there is already a layer of protection for the sensitive information. It can create a social situation where sensitive information may be more likely to leak, than it would if it was assumed that the only way to safeguard the info is to safeguard the systems that hold it. It can also cause people to believe that if data is stolen from the database, "maybe it's okay because they can't decrypt it". This is how you end up responsible for leaking credentials to the wide world, because you must assume that if your application can get the plaintext data, so can an attacker that has compromised your application.
当你给出安全的外观而没有实际保护某些东西时,那会使事情变得更加危险。您或其他管理系统的人可能会忽略其他重要的安全措施,因为他们认为敏感信息已经存在一层保护。它可以创造一种敏感信息可能更容易泄漏的社交场合,而不是假设保护信息的唯一方法是保护持有它的系统。它还可以使人们相信,如果数据从数据库中被盗,“也许它没关系,因为它们无法解密”。这就是你最终要向全世界泄露凭据的责任,因为你必须假设如果你的应用程序可以获得纯文本数据,那么攻击者也可以破坏你的应用程序。
There may be a very very small advantage to encrypting a multipart key on multiple systems (i.e. part on the file system, part in the database), so that somebody who gains access to one system doesn't necessarily gain access to the other. It's reasonable to say that this might, in fact, delay an attack, or deter a lazy attacker. Generally speaking however, the place where your application lives has access to both these things anyway so if your application is compromised then you must assume that the data is compromised. (Edit: You mentioned in another comment you have users who a) shouldn't know the passwords stored in the DB, but b) will have access to the DB directly. Believe me, there is a non-zero chance of one of those users getting all of those passwords if you do this. By going down this path, you're putting your faith in a faulty layer of protection.)
加密多个系统上的多部分密钥(即文件系统的一部分,数据库中的部分)可能有非常小的优点,因此获得对一个系统的访问权的人不一定能够访问另一个系统。事实上,这可能会延迟攻击,或阻止懒惰的攻击者,这是合理的。但是,一般来说,您的应用程序所在的位置无论如何都可以访问这些内容,因此如果您的应用程序被泄露,那么您必须假设数据已被泄露。 (编辑:你在另一个评论中提到你有用户a)不应该知道存储在数据库中的密码,但是b)可以直接访问数据库。相信我,如果你这样做,其中一个用户获得所有这些密码的概率非零。沿着这条路走下去,你就会把自己的信仰放在一个错误的保护层上。)
tl;dr reversible encryption when storing sensitive data rarely has practical, real security value. If you're trying to tick a compliance check box, and you have no power to overrule someone up the chain who needs you to tick that box, then by all means implement something that "encrypts" the data. But if you're actually trying to secure your system, look elsewhere: here be dragons.
tl; dr存储敏感数据时的可逆加密很少具有实用,真实的安全价值。如果您正在尝试勾选合规性复选框,并且您没有权力推翻需要您勾选该框的链上的某个人,那么无论如何都要实施“加密”数据的内容。但是,如果你真的想要保护你的系统,那就看看其他地方:这里是龙。
#3
0
You may check Vault project, a tool for managing secrets:
您可以查看Vault项目,这是一个管理机密的工具:
General Secret Storage
一般秘密存储
At a bare minimum, Vault can be used for the storage of any secrets. For example, Vault would be a fantastic way to store sensitive environment variables, database credentials, API keys, etc.
最低限度,Vault可用于存储任何机密。例如,Vault是存储敏感环境变量,数据库凭据,API密钥等的绝佳方式。
Compare this with the current way to store these which might be plaintext in files, configuration management, a database, etc. It would be much safer to query these using vault read or the API. This protects the plaintext version of these secrets as well as records access in the Vault audit log.
将其与当前存储这些文件,配置管理,数据库等明文的方式进行比较。使用文件库读取或API查询这些文件会更安全。这可以保护这些机密的纯文本版本以及Vault审核日志中的记录访问权限。
Employee Credential Storage
员工凭证存储
While this overlaps with "General Secret Storage", Vault is a good mechanism for storing credentials that employees share to access web services. The audit log mechanism lets you know what secrets an employee accessed and when an employee leaves, it is easier to roll keys and understand which keys have and haven't been rolled.
虽然这与“一般秘密存储”重叠,但Vault是一种很好的机制,用于存储员工共享以访问Web服务的凭据。审计日志机制可以让您了解员工访问的秘密以及员工离职时,更容易滚动密钥并了解哪些密钥已经滚动和未滚动。
Vault server stores data in encrypted form. Data can be retrieved via command line or REST API. Server must be in unsealed state to return decrypted data - unsealing needs specific number of shards of the master key. Once server is restarted, you need to unseal it again.
Vault服务器以加密形式存储数据。可以通过命令行或REST API检索数据。服务器必须处于未密封状态才能返回解密数据 - 启封需要特定数量的主密钥分片。重新启动服务器后,您需要再次启动它。