I have hypothetical scenario where a file handle opened in asynchronous mode, and some threads which are appending to that file handle. They append by setting the Offset and OffsetHigh parts of the OVERLAPPED structure to 0xFFFFFFFF, as documented in the MSDN article for WriteFile.
我有一个假设的场景,其中一个文件句柄在异步模式下打开,一些线程附加到该文件句柄。它们通过将OVERLAPPED结构的Offset和OffsetHigh部分设置为0xFFFFFFFF来附加,如WriteFile的MSDN文章中所述。
Can I issue a second write in append mode like this before the first append completes, and expect the file to contain the entire contents of the first append followed by the entire contents of the second append? Or must I wait to issue the following asynchronous write until the previous write completes?
我可以在第一个附加完成之前以这样的附加模式发出第二个写入,并期望该文件包含第一个附加的全部内容,后跟第二个附加的全部内容吗?或者我必须等待发出以下异步写入,直到上一次写入完成?
2 个解决方案
#1
0
Yes. It works. I worked at a company that used a similar scheme, although to get their seek calls to work each time, the predefined the file at a known size (about 2Gb...) then truncated the file at the end.
是。有用。我在一家使用类似方案的公司工作,虽然每次都要求他们的搜索工作,预定义的文件大小已知(大约2Gb ......)然后在最后截断文件。
However, you can just "append" by going to the right location before each write. You'll have to handle the position yourself though.
但是,您可以通过在每次写入之前到达正确的位置来“追加”。你必须自己处理这个职位。
And also each thread must access the file atomically, "of course."
并且每个线程也必须以原子方式访问文件,“当然。”
A simple example:
一个简单的例子:
lock mutex
seek to position
write data
position += data size
unlock mutex
Of course here I assume that the file is properly opened before you call this function from any thread.
当然在这里我假设在从任何线程调用此函数之前,文件已正确打开。
The one thing that you cannot do, unless you create a large file first (which is very fast since files with all zeroes are created virtually), is seek at a position depending on something such as a frame number. So if thread 3 wants to write at "size * 3" and that happens before thread 2 writes at "size * 2" then the seek() will fail...
你不能做的一件事,除非你先创建一个大文件(这是非常快的,因为虚拟地创建全零的文件),取决于诸如帧号之类的东西寻找位置。因此,如果线程3想要写入“size * 3”并且在线程2写入“size * 2”之前发生,则seek()将失败...
#2
0
You should never issue multiple outstanding WriteFile operations with offset set to 0xFFFFFFFF even from a single thread. This will cause issue where multiple calls try to access the end of data at the same time and will lead to data corruption. This is due to the fact that if WriteFile operates in asynch mode and there are other outstanding WriteFile in process using the end of file, some operations might write data to the end of file and other outstanding operations will get wrong end of file pointer. In short you should use 0xFFFFFFFF only 1 time and wait for the operation to finish to issue another one using that offset. Otherwise you need to calculate the offsets yourself so that each outstanding write operation uses a unique offset. This bug took me 3 days to find about due to poor MSDN documentation about that offset.
即使从单个线程,也不应发出偏移设置为0xFFFFFFFF的多个未完成的WriteFile操作。这将导致多个调用同时尝试访问数据末尾并导致数据损坏的问题。这是因为如果WriteFile在异步模式下运行并且在使用文件末尾的进程中有其他未完成的WriteFile,则某些操作可能会将数据写入文件末尾,而其他未完成的操作将得到错误的文件结束指针。简而言之,您应该只使用0xFFFFFFFF一次并等待操作完成以使用该偏移发出另一个操作。否则,您需要自己计算偏移量,以便每个未完成的写操作使用唯一的偏移量。由于关于该偏移的MSDN文档很差,这个错误花了我3天时间才发现。
#1
0
Yes. It works. I worked at a company that used a similar scheme, although to get their seek calls to work each time, the predefined the file at a known size (about 2Gb...) then truncated the file at the end.
是。有用。我在一家使用类似方案的公司工作,虽然每次都要求他们的搜索工作,预定义的文件大小已知(大约2Gb ......)然后在最后截断文件。
However, you can just "append" by going to the right location before each write. You'll have to handle the position yourself though.
但是,您可以通过在每次写入之前到达正确的位置来“追加”。你必须自己处理这个职位。
And also each thread must access the file atomically, "of course."
并且每个线程也必须以原子方式访问文件,“当然。”
A simple example:
一个简单的例子:
lock mutex
seek to position
write data
position += data size
unlock mutex
Of course here I assume that the file is properly opened before you call this function from any thread.
当然在这里我假设在从任何线程调用此函数之前,文件已正确打开。
The one thing that you cannot do, unless you create a large file first (which is very fast since files with all zeroes are created virtually), is seek at a position depending on something such as a frame number. So if thread 3 wants to write at "size * 3" and that happens before thread 2 writes at "size * 2" then the seek() will fail...
你不能做的一件事,除非你先创建一个大文件(这是非常快的,因为虚拟地创建全零的文件),取决于诸如帧号之类的东西寻找位置。因此,如果线程3想要写入“size * 3”并且在线程2写入“size * 2”之前发生,则seek()将失败...
#2
0
You should never issue multiple outstanding WriteFile operations with offset set to 0xFFFFFFFF even from a single thread. This will cause issue where multiple calls try to access the end of data at the same time and will lead to data corruption. This is due to the fact that if WriteFile operates in asynch mode and there are other outstanding WriteFile in process using the end of file, some operations might write data to the end of file and other outstanding operations will get wrong end of file pointer. In short you should use 0xFFFFFFFF only 1 time and wait for the operation to finish to issue another one using that offset. Otherwise you need to calculate the offsets yourself so that each outstanding write operation uses a unique offset. This bug took me 3 days to find about due to poor MSDN documentation about that offset.
即使从单个线程,也不应发出偏移设置为0xFFFFFFFF的多个未完成的WriteFile操作。这将导致多个调用同时尝试访问数据末尾并导致数据损坏的问题。这是因为如果WriteFile在异步模式下运行并且在使用文件末尾的进程中有其他未完成的WriteFile,则某些操作可能会将数据写入文件末尾,而其他未完成的操作将得到错误的文件结束指针。简而言之,您应该只使用0xFFFFFFFF一次并等待操作完成以使用该偏移发出另一个操作。否则,您需要自己计算偏移量,以便每个未完成的写操作使用唯一的偏移量。由于关于该偏移的MSDN文档很差,这个错误花了我3天时间才发现。