如何优化快速插入表?

时间:2021-10-13 16:54:48

I have a log table that will receive inserts from several web apps. I wont be doing any searching/sorting/querying of this data. I will be pulling the data out to another database to run reports. The initial table is strictly for RECEIVING the log messages.

我有一个日志表,将从几个Web应用程序接收插入。我不会对这些数据进行任何搜索/排序/查询。我将把数据拉到另一个数据库来运行报告。初始表严格用于RECEIVING日志消息。

Is there a way to ensure that the web applications don't have to wait on these inserts? For example I know that adding a lot of indexes would slow inserts, so I won't. What else is there? Should I not add a primary key? (Each night the table will be pumped to a reports DB which will have a lot of keys/indexes)

有没有办法确保Web应用程序不必等待这些插入?例如,我知道添加大量索引会减慢插入速度,所以我不会。那里还有什么?我不应该添加主键吗? (每晚会将表格抽取到报告数据库中,该报告数据库将包含大量键/索引)

3 个解决方案

#1


9  

If performance is key, you may not want to write this data to a database. I think most everything will process a database write as a round-trip, but it sounds like you don't want to wait for the returned confirmation message. Check if, as S. Lott suggests, it might not be faster to just append a row to a simple text file somewhere.

如果性能是关键,您可能不希望将此数据写入数据库。我认为大多数事情都会将数据库写入处理为往返,但听起来你不想等待返回的确认消息。检查是否像S. Lott所说的那样,将行添加到某个简单的文本文件可能不会更快。

If the database write is faster (or necessary, for security or other business/operational reasons), I would put no indexes on the table--and that includes a primary key. If it won't be used for reads or updates, and if you don't need relational integrity, then you just don't need a PK on this table.

如果数据库写入更快(或者出于安全性或其他业务/操作原因需要),我不会在表上放置任何索引 - 包括主键。如果它不会用于读取或更新,并且如果您不需要关系完整性,那么您只需要在此表上使用PK。

To recommend the obvious: as part of the nightly reports run, clear out the contents of the table. Also, never reset the database file sizes (ye olde shrink database command); after a week or so of regular use, the database files should be as big as they'll ever need to be and you won't have to worry about the file growth performance hit.

推荐显而易见的:作为夜间报告运行的一部分,清除表格的内容。另外,永远不要重置数据库文件大小(ye olde shrink database命令);经过一周左右的常规使用后,数据库文件应该是他们需要的大小,您不必担心文件增长性能受到影响。

#2


5  

Here are a few ideas, note for the last ones to be important you would have extremly high volumns:

这里有一些想法,请注意最后的重要一点,你会有极高的音量:

  • do not have a primary key, it is enforced via an index
  • 没有主键,它通过索引强制执行
  • do not have any other index
  • 没有任何其他索引
  • Create the database large enough that you do not have any database growth
  • 创建足够大的数据库,使您没有任何数据库增长
  • Place the database on it's own disk to avoid contention
  • 将数据库放在自己的磁盘上以避免争用
  • Avoid software RAID
  • 避免使用软件RAID
  • place the database on a mirrored disk, saves the calculating done on RAID 5
  • 将数据库放在镜像磁盘上,保存在RAID 5上完成的计算

#3


4  

No keys, no constraints, no validation, no triggers, No calculated columns

没有键,没有约束,没有验证,没有触发器,没有计算列

If you can, have the services insert async, so as to not wait for the results (if that is acceptable).

如果可以,请让服务插入异步,以便不等待结果(如果可以接受)。

You can even try to insert into a "daily" table, which should then be less records, and then move this across before the batch runs at night.

您甚至可以尝试插入“每日”表格,这应该是较少的记录,然后在批处理晚上运行之前移动它。

But mostly on the table NO KEYS/Validation (PK and Unique indexes will kill you)

但主要是在表上NO KEYS / Validation(PK和Unique索引会杀了你)

#1


9  

If performance is key, you may not want to write this data to a database. I think most everything will process a database write as a round-trip, but it sounds like you don't want to wait for the returned confirmation message. Check if, as S. Lott suggests, it might not be faster to just append a row to a simple text file somewhere.

如果性能是关键,您可能不希望将此数据写入数据库。我认为大多数事情都会将数据库写入处理为往返,但听起来你不想等待返回的确认消息。检查是否像S. Lott所说的那样,将行添加到某个简单的文本文件可能不会更快。

If the database write is faster (or necessary, for security or other business/operational reasons), I would put no indexes on the table--and that includes a primary key. If it won't be used for reads or updates, and if you don't need relational integrity, then you just don't need a PK on this table.

如果数据库写入更快(或者出于安全性或其他业务/操作原因需要),我不会在表上放置任何索引 - 包括主键。如果它不会用于读取或更新,并且如果您不需要关系完整性,那么您只需要在此表上使用PK。

To recommend the obvious: as part of the nightly reports run, clear out the contents of the table. Also, never reset the database file sizes (ye olde shrink database command); after a week or so of regular use, the database files should be as big as they'll ever need to be and you won't have to worry about the file growth performance hit.

推荐显而易见的:作为夜间报告运行的一部分,清除表格的内容。另外,永远不要重置数据库文件大小(ye olde shrink database命令);经过一周左右的常规使用后,数据库文件应该是他们需要的大小,您不必担心文件增长性能受到影响。

#2


5  

Here are a few ideas, note for the last ones to be important you would have extremly high volumns:

这里有一些想法,请注意最后的重要一点,你会有极高的音量:

  • do not have a primary key, it is enforced via an index
  • 没有主键,它通过索引强制执行
  • do not have any other index
  • 没有任何其他索引
  • Create the database large enough that you do not have any database growth
  • 创建足够大的数据库,使您没有任何数据库增长
  • Place the database on it's own disk to avoid contention
  • 将数据库放在自己的磁盘上以避免争用
  • Avoid software RAID
  • 避免使用软件RAID
  • place the database on a mirrored disk, saves the calculating done on RAID 5
  • 将数据库放在镜像磁盘上,保存在RAID 5上完成的计算

#3


4  

No keys, no constraints, no validation, no triggers, No calculated columns

没有键,没有约束,没有验证,没有触发器,没有计算列

If you can, have the services insert async, so as to not wait for the results (if that is acceptable).

如果可以,请让服务插入异步,以便不等待结果(如果可以接受)。

You can even try to insert into a "daily" table, which should then be less records, and then move this across before the batch runs at night.

您甚至可以尝试插入“每日”表格,这应该是较少的记录,然后在批处理晚上运行之前移动它。

But mostly on the table NO KEYS/Validation (PK and Unique indexes will kill you)

但主要是在表上NO KEYS / Validation(PK和Unique索引会杀了你)