so I have an application that is processing a large text document which is composed of many logical documents. Say for eg it has 200,000 logical documents. Right now I read one logical document and do the transformation and write logical document out. This works well because nothing is stored in memory so the workflow ends up finishing fast but the number of IOPS end up being high! Is there any recommendation on how I can reduce the number of IOPS ?
所以我有一个处理大型文本文档的应用程序,该文档由许多逻辑文档组成。比如说,它有200,000个逻辑文件。现在我读了一个逻辑文档并进行转换并写出逻辑文档。这很好用,因为没有任何内容存储在内存中,因此工作流程最终快速完成,但IOPS的数量最终会很高!有关如何减少IOPS数量的建议吗?
1 个解决方案
#1
0
If this is a single pass through the document, increase your read buffer size (i.e. 2 reads with a buffer size of 1024 becomes 1 read with a buffer size of 2048, which means less overhead).
如果这是单次传递文档,请增加读取缓冲区大小(即,缓冲区大小为1024的2次读取将变为1次读取,缓冲区大小为2048,这意味着开销更少)。
If you want more detailed suggestions, you'll need to provide more details in your question.
如果您需要更详细的建议,则需要在问题中提供更多详细信息。
#1
0
If this is a single pass through the document, increase your read buffer size (i.e. 2 reads with a buffer size of 1024 becomes 1 read with a buffer size of 2048, which means less overhead).
如果这是单次传递文档,请增加读取缓冲区大小(即,缓冲区大小为1024的2次读取将变为1次读取,缓冲区大小为2048,这意味着开销更少)。
If you want more detailed suggestions, you'll need to provide more details in your question.
如果您需要更详细的建议,则需要在问题中提供更多详细信息。