The CreateIoCompletionPort
function allows the creation of a new I/O completion port and the registration of file handles to an existing I/O completion port.
CreateIoCompletionPort函数允许创建一个新的I/O完成端口,并将文件句柄注册到一个现有的I/O完成端口。
Then, I can use any function, like a recv
on a socket or a ReadFile
on a file with a OVERLAPPED
structure to start an asynchronous operation.
然后,我可以使用任何函数,比如在一个套接字上的recv或一个文件上的ReadFile,它有一个重叠的结构来启动一个异步操作。
I have to check whether the function call returned synchronously although it was called with an OVERLAPPED
structure and in this case handle it directly. In the other case, when ERROR_IO_PENDING
is returned, I can use the GetQueuedCompletionStatus
function to be notified when the operation completes.
我必须检查函数调用是否同步返回,尽管它是由一个重叠的结构调用的,在这个例子中直接处理它。在另一种情况下,当返回ERROR_IO_PENDING时,我可以使用GetQueuedCompletionStatus函数在操作完成时得到通知。
The question which arise are:
出现的问题是:
-
How can I remove a handle from the I/O completion port? For example, when I add sockets to the IOCP, how can I remove closed ones? Should I just re-register another socket with the same completion key?
如何从I/O完成端口移除句柄?例如,当我为IOCP添加套接字时,如何删除闭包?我是否应该重新注册一个具有相同完成键的套接字?
-
Also, is there a way to make the calls ALWAYS go over the I/O completion port and don't return synchronously?
另外,是否有一种方法可以让调用总是遍历I/O完成端口,而不会同时返回?
-
And finally, is it possible for example to
recv
asynchronously but tosend
synchronously? For example when a simple echo service is implemented: Can I wait with an asynchronousrecv
for new data butsend
the response in a synchronous way so that code complexity is reduced? In my case, I wouldn'trecv
a second time anyways before the first request was processed.最后,是否有可能以异步方式进行,但同时发送?例如,当实现简单的echo服务时:我是否可以用异步recv等待新数据,但以同步方式发送响应,从而减少代码的复杂性?在我的例子中,在第一个请求被处理之前,我不会再重复第二次。
-
What happens if an asynchronous
ReadFile
has been requested, but before it completes, aWriteFile
to the same file should be processed. Will theReadFile
be cancelled with an error message and I have to restart the read process as soon as the write is complete? Or do I have to cancel theReadFile
manually before writing? This question arises in combination with a communication device; so, the write and read should not do problems if happening concurrently.如果请求了异步读取文件,但在它完成之前,应该处理相同文件的WriteFile。是否会用错误消息取消ReadFile,并且在写入完成后,我必须重新启动读进程?或者我必须在写之前手动取消ReadFile吗?这个问题与通信设备一起出现;所以,写和读不能同时发生问题。
4 个解决方案
#1
10
How can I remove a handle from the I/O completion port?
如何从I/O完成端口移除句柄?
In my experience you can't disassociate a handle from a completion port. However, you may disable completion port notification by setting the low-order bit of your OVERLAPPED
structure's hEvent
field: See the documentation for GetQueuedCompletionStatus.
在我的经验中,您不能将句柄与完成端口分离。但是,您可以通过设置重叠结构的hEvent字段的低阶位来禁用完成端口通知:查看GetQueuedCompletionStatus的文档。
For example, when I add sockets to the IOCP, how can I remove closed ones? Should I just re-register another socket with the same completion key?
例如,当我为IOCP添加套接字时,如何删除闭包?我是否应该重新注册一个具有相同完成键的套接字?
It is not necessary to explicitly disassociate a handle from an I/O completion port; closing the handle is sufficient. You may associate multiple handles with the same completion key; the best way to figure out which request is associated with the I/O completion is by using the OVERLAPPED
structure. In fact, you may even extend OVERLAPPED
to store additional data.
不必显式地将句柄与I/O完成端口相关联;关闭手柄就足够了。您可以将多个句柄与相同的完成键关联;找出与I/O完成相关的请求的最佳方法是使用重叠结构。实际上,您甚至可以扩展重叠来存储额外的数据。
Also, is there a way to make the calls ALWAYS go over the I/O completion port and don't return synchronously?
另外,是否有一种方法可以让调用总是遍历I/O完成端口,而不会同时返回?
That is the default behavior, even when ReadFile
/WriteFile
returns TRUE
. You must explicitly call SetFileCompletionNotificationModes to tell Windows to not enqueue a completion packet when TRUE
and ERROR_SUCCESS
are returned.
这是默认的行为,即使当ReadFile/WriteFile返回TRUE时也是如此。您必须显式地调用setfilecompletionnotificationmode,以告知窗口在返回TRUE和ERROR_SUCCESS时不需要对完成包进行排队。
is it possible for example to
recv
asynchronously but tosend
synchronously?例如,是否可以以异步方式进行,但同步发送?
Not by using recv
and send
; you need to use functions that accept OVERLAPPED
structures, such as WSARecv
, WSASend
, or alternatively ReadFile
and WriteFile
. It might be more handy to use the latter if your code is meant to work multiple types of I/O handles, such as both sockets and named pipes. Those functions provide a synchronous mode, so if you use those them you can mix asynchronous and synchronous calls.
不使用recv和send;您需要使用接受重叠结构的函数,比如WSARecv、WSASend,或者是ReadFile和WriteFile。如果您的代码想要处理多种类型的I/O句柄,比如套接字和命名管道,那么使用后者可能会更方便。这些函数提供了同步模式,因此如果您使用这些函数,您可以混合异步调用和同步调用。
What happens if an asynchronous ReadFile has been requested, but before it completes, a WriteFile to the same file should be processed?
如果请求了一个异步读取文件,但在它完成之前,应该处理同一个文件的WriteFile,会发生什么情况呢?
There is no implicit cancellation. As long as you're using separate OVERLAPPED
structures for each read/write to a full-duplex device, I see no reason why you can't do concurrent I/O operations.
没有隐式的取消。只要你在一个全双工设备上使用单独的重叠结构,我看不出为什么你不能做并行I/O操作。
#2
6
As I’ve already pointed out there, the commonly held belief that it is impossible to remove handles from completion ports is wrong, probably caused by the abscence of any hint whatsoever on how to do this from nearly all documentation I could find. Actually, it’s pretty easy:
正如我已经指出的,通常认为不可能从完成端口上删除句柄的想法是错误的,这可能是由于任何关于如何从几乎所有我能找到的文档中实现这一点的提示所引起的。实际上,它很简单:
Call NtSetInformationFile
with the FileReplaceCompletionInformation
enumerator value for FileInformationClass
and a pointer to a FILE_COMPLETION_INFORMATION
structure for the FileInformation
parameter. In this structure, set the Port
member to NULL
(or nullptr
, in C++) to disassociate the file from the port it’s currently attached to (I guess if it isn’t attached to any port, nothing would happen), or set Port
to a valid HANDLE
to another completion port to associate the file with that one instead.
调用NtSetInformationFile,其中包含filereplacecompletioninformationlist值,用于FileInformationClass和FILE_COMPLETION_INFORMATION结构的一个指针,用于FileInformation参数。在此结构中,将港口成员设置为NULL(或nullptr c++)分离的文件从港口目前相连(我猜如果不是任何港口,不会发生),或一组端口有效处理另一个完成端口关联的文件。
#3
3
First some important corrections.
一些重要的修正。
In case the overlapped I/O operation completes immediately (ReadFile
or similar I/O function returns success) - the I/O completion is already scheduled to the IOCP.
如果重叠的I/O操作立即完成(ReadFile或类似的I/O函数返回成功)- I/O完成已经计划到IOCP。
Also, according to your questions I think you confuse between the file/socket handles, and the specific I/O operations issued on them.
另外,根据您的问题,我认为您混淆了文件/套接字句柄和针对它们的特定I/O操作。
Now, regarding your questions:
现在,关于你的问题:
- AFAIK there is no conventional way to remove a file/socket handle from the IOCP (usually you just don't have to do this). You talk about removing closed handles from the IOCP, which is absolutely incorrect. You can't remove a closed handle, because it does not reference a valid kernel object anymore!
- AFAIK没有传统的方法从IOCP中删除一个文件/套接字句柄(通常您不需要这样做)。您谈到了从IOCP删除关闭的句柄,这是绝对错误的。您不能删除一个闭句柄,因为它不再引用一个有效的内核对象!
A more correct question should be how the file/socket should be properly closed. The answer is: just close your handle. All the outstanding I/O operations (issued on this handle) will return soon with an error code (abortion). Then, in your completion routine (the one that calls GetQueuedCompletionStatus
in a loop) should perform the per-I/O needed cleanup.
一个更正确的问题应该是如何正确关闭文件/套接字。答案是:关上你的把手。所有未完成的I/O操作(在此句柄上发布)将很快返回错误代码(人工流产)。然后,在您的完成例程中(在循环中调用GetQueuedCompletionStatus的那个)应该执行每个i /O需要的清理。
-
As I've already said, all the I/O completion arrives at IOCP in both synchronous and asynchronous cases. The only situation where it does not arrive at IOCP is when an I/O completes synchronously with an error. Anyway, if you want a unified processing - in such a case you may post an artificial completion data to IOCP (use
PostQueuedCompletionStatus
).正如我已经说过的,所有的I/O完成都在同步和异步的情况下到达IOCP。只有当I/O与错误同步完成时,才会到达IOCP。无论如何,如果您想要一个统一的处理——在这种情况下,您可以将人工完成数据发布到IOCP(使用PostQueuedCompletionStatus)。
-
You should use
WSASend
andWSARecv
(notrecv
andsend
) for overlapped I/O. Nevertheless, even of the socket was opened with flagWSA_FLAG_OVERLAPPED
- you are allowed to call the I/O functions without specifying theOVERLAPPED
structure. In such a case those functions work synchronously. So that you may decide on synchronous/asynchronous modes for every function call.您应该使用WSASend和WSARecv(不是recv和send)来进行重叠I/O。尽管如此,即使是使用标志wsa_flag_重叠来打开套接字,也可以调用I/O函数,而无需指定重叠结构。在这种情况下,这些函数是同步工作的。因此,您可以决定每个函数调用的同步/异步模式。
-
There is no problem to mix overlapped read/write requests. The only delicate point here is what happens if you try to read the data from the file position where you're currently writing to. The result may depend on subtle things, such as order of completion of I/Os by the hardware, some PC timing parameters and etc. Such a situation should be avoided.
混合读/写请求是没有问题的。这里唯一的微妙之处是,如果您试图从当前正在写入的文件位置读取数据,会发生什么情况。结果可能取决于一些细微的事情,比如硬件的完成顺序,一些PC的定时参数等等。这样的情况应该避免。
#4
0
How can I remove a handle from the I/O completion port? For example, when I add sockets to the IOCP, how can I remove closed ones? Should I just re-register another socket with the same completion key?
如何从I/O完成端口移除句柄?例如,当我为IOCP添加套接字时,如何删除闭包?我是否应该重新注册一个具有相同完成键的套接字?
You've got it the wrong way around. You set the I/O completion port to be used by a file object - when the file object is deleted, you have nothing to worry about. The reason you're getting confused is because of the way Win32 exposes the underlying native API functionality (CreateIoCompletionPort
does two very different things in one function).
你弄错了。您设置了一个文件对象使用的I/O完成端口——当文件对象被删除时,您无需担心。您感到困惑的原因是,Win32公开了底层的本地API功能(CreateIoCompletionPort在一个函数中做了两件非常不同的事情)。
Also, is there a way to make the calls ALWAYS go over the I/O completion port and don't return synchronously?
另外,是否有一种方法可以让调用总是遍历I/O完成端口,而不会同时返回?
This is how it's always been. Only starting with Windows Vista can you customize how the completion notifications are handled.
事情就是这样的。只有从Windows Vista开始,您才能定制如何处理完成通知。
What happens if an asynchronous ReadFile has been requested, but before it completes, a WriteFile to the same file should be processed. Will the ReadFile be cancelled with an error message and I have to restart the read process as soon as the write is complete?
如果请求了异步读取文件,但在它完成之前,应该处理相同文件的WriteFile。是否会用错误消息取消ReadFile,并且在写入完成后,我必须重新启动读进程?
I/O operations in Windows are asynchronous inherently, and requests are always queued. You may not think this is so because you have to specify FILE_FLAG_OVERLAPPED
in CreateFile
to turn on asynchronous I/O. However, at the native layer, synchronous I/O is really an add-on, convenience thing where the kernel keeps track of the file position for you and waits for the I/O to complete before returning.
Windows中的I/O操作本身是异步的,请求总是排队。您可能不认为这是这样的,因为您必须在CreateFile中指定file_flag_重叠来打开异步I/O。但是,在本机层,同步I/O实际上是一个附加的、方便的东西,内核可以为您跟踪文件位置,并等待I/O在返回之前完成。
#1
10
How can I remove a handle from the I/O completion port?
如何从I/O完成端口移除句柄?
In my experience you can't disassociate a handle from a completion port. However, you may disable completion port notification by setting the low-order bit of your OVERLAPPED
structure's hEvent
field: See the documentation for GetQueuedCompletionStatus.
在我的经验中,您不能将句柄与完成端口分离。但是,您可以通过设置重叠结构的hEvent字段的低阶位来禁用完成端口通知:查看GetQueuedCompletionStatus的文档。
For example, when I add sockets to the IOCP, how can I remove closed ones? Should I just re-register another socket with the same completion key?
例如,当我为IOCP添加套接字时,如何删除闭包?我是否应该重新注册一个具有相同完成键的套接字?
It is not necessary to explicitly disassociate a handle from an I/O completion port; closing the handle is sufficient. You may associate multiple handles with the same completion key; the best way to figure out which request is associated with the I/O completion is by using the OVERLAPPED
structure. In fact, you may even extend OVERLAPPED
to store additional data.
不必显式地将句柄与I/O完成端口相关联;关闭手柄就足够了。您可以将多个句柄与相同的完成键关联;找出与I/O完成相关的请求的最佳方法是使用重叠结构。实际上,您甚至可以扩展重叠来存储额外的数据。
Also, is there a way to make the calls ALWAYS go over the I/O completion port and don't return synchronously?
另外,是否有一种方法可以让调用总是遍历I/O完成端口,而不会同时返回?
That is the default behavior, even when ReadFile
/WriteFile
returns TRUE
. You must explicitly call SetFileCompletionNotificationModes to tell Windows to not enqueue a completion packet when TRUE
and ERROR_SUCCESS
are returned.
这是默认的行为,即使当ReadFile/WriteFile返回TRUE时也是如此。您必须显式地调用setfilecompletionnotificationmode,以告知窗口在返回TRUE和ERROR_SUCCESS时不需要对完成包进行排队。
is it possible for example to
recv
asynchronously but tosend
synchronously?例如,是否可以以异步方式进行,但同步发送?
Not by using recv
and send
; you need to use functions that accept OVERLAPPED
structures, such as WSARecv
, WSASend
, or alternatively ReadFile
and WriteFile
. It might be more handy to use the latter if your code is meant to work multiple types of I/O handles, such as both sockets and named pipes. Those functions provide a synchronous mode, so if you use those them you can mix asynchronous and synchronous calls.
不使用recv和send;您需要使用接受重叠结构的函数,比如WSARecv、WSASend,或者是ReadFile和WriteFile。如果您的代码想要处理多种类型的I/O句柄,比如套接字和命名管道,那么使用后者可能会更方便。这些函数提供了同步模式,因此如果您使用这些函数,您可以混合异步调用和同步调用。
What happens if an asynchronous ReadFile has been requested, but before it completes, a WriteFile to the same file should be processed?
如果请求了一个异步读取文件,但在它完成之前,应该处理同一个文件的WriteFile,会发生什么情况呢?
There is no implicit cancellation. As long as you're using separate OVERLAPPED
structures for each read/write to a full-duplex device, I see no reason why you can't do concurrent I/O operations.
没有隐式的取消。只要你在一个全双工设备上使用单独的重叠结构,我看不出为什么你不能做并行I/O操作。
#2
6
As I’ve already pointed out there, the commonly held belief that it is impossible to remove handles from completion ports is wrong, probably caused by the abscence of any hint whatsoever on how to do this from nearly all documentation I could find. Actually, it’s pretty easy:
正如我已经指出的,通常认为不可能从完成端口上删除句柄的想法是错误的,这可能是由于任何关于如何从几乎所有我能找到的文档中实现这一点的提示所引起的。实际上,它很简单:
Call NtSetInformationFile
with the FileReplaceCompletionInformation
enumerator value for FileInformationClass
and a pointer to a FILE_COMPLETION_INFORMATION
structure for the FileInformation
parameter. In this structure, set the Port
member to NULL
(or nullptr
, in C++) to disassociate the file from the port it’s currently attached to (I guess if it isn’t attached to any port, nothing would happen), or set Port
to a valid HANDLE
to another completion port to associate the file with that one instead.
调用NtSetInformationFile,其中包含filereplacecompletioninformationlist值,用于FileInformationClass和FILE_COMPLETION_INFORMATION结构的一个指针,用于FileInformation参数。在此结构中,将港口成员设置为NULL(或nullptr c++)分离的文件从港口目前相连(我猜如果不是任何港口,不会发生),或一组端口有效处理另一个完成端口关联的文件。
#3
3
First some important corrections.
一些重要的修正。
In case the overlapped I/O operation completes immediately (ReadFile
or similar I/O function returns success) - the I/O completion is already scheduled to the IOCP.
如果重叠的I/O操作立即完成(ReadFile或类似的I/O函数返回成功)- I/O完成已经计划到IOCP。
Also, according to your questions I think you confuse between the file/socket handles, and the specific I/O operations issued on them.
另外,根据您的问题,我认为您混淆了文件/套接字句柄和针对它们的特定I/O操作。
Now, regarding your questions:
现在,关于你的问题:
- AFAIK there is no conventional way to remove a file/socket handle from the IOCP (usually you just don't have to do this). You talk about removing closed handles from the IOCP, which is absolutely incorrect. You can't remove a closed handle, because it does not reference a valid kernel object anymore!
- AFAIK没有传统的方法从IOCP中删除一个文件/套接字句柄(通常您不需要这样做)。您谈到了从IOCP删除关闭的句柄,这是绝对错误的。您不能删除一个闭句柄,因为它不再引用一个有效的内核对象!
A more correct question should be how the file/socket should be properly closed. The answer is: just close your handle. All the outstanding I/O operations (issued on this handle) will return soon with an error code (abortion). Then, in your completion routine (the one that calls GetQueuedCompletionStatus
in a loop) should perform the per-I/O needed cleanup.
一个更正确的问题应该是如何正确关闭文件/套接字。答案是:关上你的把手。所有未完成的I/O操作(在此句柄上发布)将很快返回错误代码(人工流产)。然后,在您的完成例程中(在循环中调用GetQueuedCompletionStatus的那个)应该执行每个i /O需要的清理。
-
As I've already said, all the I/O completion arrives at IOCP in both synchronous and asynchronous cases. The only situation where it does not arrive at IOCP is when an I/O completes synchronously with an error. Anyway, if you want a unified processing - in such a case you may post an artificial completion data to IOCP (use
PostQueuedCompletionStatus
).正如我已经说过的,所有的I/O完成都在同步和异步的情况下到达IOCP。只有当I/O与错误同步完成时,才会到达IOCP。无论如何,如果您想要一个统一的处理——在这种情况下,您可以将人工完成数据发布到IOCP(使用PostQueuedCompletionStatus)。
-
You should use
WSASend
andWSARecv
(notrecv
andsend
) for overlapped I/O. Nevertheless, even of the socket was opened with flagWSA_FLAG_OVERLAPPED
- you are allowed to call the I/O functions without specifying theOVERLAPPED
structure. In such a case those functions work synchronously. So that you may decide on synchronous/asynchronous modes for every function call.您应该使用WSASend和WSARecv(不是recv和send)来进行重叠I/O。尽管如此,即使是使用标志wsa_flag_重叠来打开套接字,也可以调用I/O函数,而无需指定重叠结构。在这种情况下,这些函数是同步工作的。因此,您可以决定每个函数调用的同步/异步模式。
-
There is no problem to mix overlapped read/write requests. The only delicate point here is what happens if you try to read the data from the file position where you're currently writing to. The result may depend on subtle things, such as order of completion of I/Os by the hardware, some PC timing parameters and etc. Such a situation should be avoided.
混合读/写请求是没有问题的。这里唯一的微妙之处是,如果您试图从当前正在写入的文件位置读取数据,会发生什么情况。结果可能取决于一些细微的事情,比如硬件的完成顺序,一些PC的定时参数等等。这样的情况应该避免。
#4
0
How can I remove a handle from the I/O completion port? For example, when I add sockets to the IOCP, how can I remove closed ones? Should I just re-register another socket with the same completion key?
如何从I/O完成端口移除句柄?例如,当我为IOCP添加套接字时,如何删除闭包?我是否应该重新注册一个具有相同完成键的套接字?
You've got it the wrong way around. You set the I/O completion port to be used by a file object - when the file object is deleted, you have nothing to worry about. The reason you're getting confused is because of the way Win32 exposes the underlying native API functionality (CreateIoCompletionPort
does two very different things in one function).
你弄错了。您设置了一个文件对象使用的I/O完成端口——当文件对象被删除时,您无需担心。您感到困惑的原因是,Win32公开了底层的本地API功能(CreateIoCompletionPort在一个函数中做了两件非常不同的事情)。
Also, is there a way to make the calls ALWAYS go over the I/O completion port and don't return synchronously?
另外,是否有一种方法可以让调用总是遍历I/O完成端口,而不会同时返回?
This is how it's always been. Only starting with Windows Vista can you customize how the completion notifications are handled.
事情就是这样的。只有从Windows Vista开始,您才能定制如何处理完成通知。
What happens if an asynchronous ReadFile has been requested, but before it completes, a WriteFile to the same file should be processed. Will the ReadFile be cancelled with an error message and I have to restart the read process as soon as the write is complete?
如果请求了异步读取文件,但在它完成之前,应该处理相同文件的WriteFile。是否会用错误消息取消ReadFile,并且在写入完成后,我必须重新启动读进程?
I/O operations in Windows are asynchronous inherently, and requests are always queued. You may not think this is so because you have to specify FILE_FLAG_OVERLAPPED
in CreateFile
to turn on asynchronous I/O. However, at the native layer, synchronous I/O is really an add-on, convenience thing where the kernel keeps track of the file position for you and waits for the I/O to complete before returning.
Windows中的I/O操作本身是异步的,请求总是排队。您可能不认为这是这样的,因为您必须在CreateFile中指定file_flag_重叠来打开异步I/O。但是,在本机层,同步I/O实际上是一个附加的、方便的东西,内核可以为您跟踪文件位置,并等待I/O在返回之前完成。