使用scatter-gather处理短读/写的技术？

Scatter-gather - readv()/writev()/preadv()/pwritev() - reads/writes a variable number of iovec structs in a single system call. Basically it reads/write each buffer sequentially from the 0th iovec to the Nth. However according to the documentation it can also return less on the readv/writev calls than was requested. I was wondering if there is a standard/best practice/elegant way to handle that situation.

Scatter-gather - readv()/ writev()/ preadv()/ pwritev() - 在单个系统调用中读/写可变数量的iovec结构。基本上它从第0个iovec到第N个顺序读/写每个缓冲区。但是根据文档,它在readv / writev调用上的返回次数也少于请求的次数。我想知道是否有一种标准/最佳实践/优雅方式来处理这种情况。

If we are just handling a bunch of character buffers or similar this isn't a big deal. But one of the niceties is using scatter-gather for structs and/or discrete variables as the individual iovec items. How do you handle the situation where the readv/writev only reads/writes a portion of a struct or half of a long or something like that.

如果我们只是处理一堆字符缓冲区或类似的东西,这不是什么大问题。但其中一个细节是使用分散 - 聚集结构和/或离散变量作为单独的iovec项目。你如何处理readv / writev只读取/写入一个结构的一部分或一半长或类似的情况。

Below is some contrived code of what I am getting at:

下面是我得到的一些人为的代码:

int fd;

struct iovec iov[3];

long aLong = 74775767;
int  aInt  = 949;
char aBuff[100];  //filled from where ever

ssize_t bytesWritten = 0;
ssize_t bytesToWrite = 0;

iov[0].iov_base = &aLong;
iov[0].iov_len = sizeof(aLong);
bytesToWrite += iov[0].iov_len;

iov[1].iov_base = &aInt;
iov[1].iov_len = sizeof(aInt);
bytesToWrite += iov[1].iov_len;

iov[2].iov_base = &aBuff;
iov[2].iov_len = sizeof(aBuff);
bytesToWrite += iov[2].iov_len;

bytesWritten = writev(fd, iov, 3);

if (bytesWritten == -1)
{
    //handle error
}

if (bytesWritten < bytesToWrite)
    //how to gracefully continue?.........

3 个解决方案

#1

Use a loop like the following to advance the partially-processed iov:

使用如下所示的循环来推进部分处理的iov:

for (;;) {
    written = writev(fd, iov+cur, count-cur);
    if (written < 0) goto error;
    while (cur < count && written >= iov[cur].iov_len)
        written -= iov[cur++].iov_len;
    if (cur == count) break;
    iov[cur].iov_base = (char *)iov[cur].iov_base + written;
    iov[cur].iov_len -= written;
}

Note that if you don't check for cur < count you will read past the end of iov which might contain zero.

请注意,如果您不检查cur ,您将读取超过iov结尾的值,该值可能包含零。

#2

AFAICS the vectored read/write functions work the same wrt short reads/writes as the normal ones. That is, you get back the number of bytes read/written, but this might well point into the middle of a struct, just like with read()/write(). There is no guarantee that the possible "interruption points" (for lack of a better term) coincide with the vector boundaries. So unfortunately the vectored IO functions offer no more help for dealing with short reads/writes than the normal IO functions. In fact, it's more complicated since you need to map the byte count into an IO vector element and offset within the element.

AFAICS向量读/写功能与普通读/写功能的读写/写操作相同。也就是说,你得到了读/写的字节数,但这可能指向一个结构的中间,就像read()/ write()一样。无法保证可能的“中断点”(缺少更好的术语)与矢量边界一致。因此,不幸的是,向量IO函数在处理短读/写操作方面没有比普通IO函数更多的帮助。事实上,它更复杂,因为您需要将字节计数映射到IO向量元素并在元素内进行偏移。

Also note that the idea of using vectored IO for individual structs or data items might not work that well; the max allowed value for the iovcnt argument (IOV_MAX) is usually quite small, something like 1024 or so. So if you data is contiguous in memory, just pass it as a single element rather than artificially splitting it up.

另请注意,对单个结构或数据项使用向量IO的想法可能效果不佳; iovcnt参数(IOV_MAX)的最大允许值通常非常小,大约1024左右。因此,如果数据在内存中是连续的,只需将其作为单个元素传递,而不是人为地将其拆分。

#3

-1

Vectored write will write all the data you have provided with one call to "writev" function. So byteswritten will be always be equal to total number of bytes provided as input. this is what my understanding is.

向量写入将通过一次调用“writev”函数写入您提供的所有数据。因此,byteswritten将始终等于作为输入提供的总字节数。这就是我的理解。

Please correct me if I am wrong

如果我错了,请纠正我

#1