多个线程和多个TCP连接

I have an application with multiple threads, one thread is creating 4 tcp connections, then this thread creates another thread that will handle the receive function for these 4 connections using poll or anything else, and the original thread (the first one) start sending messages to these 4 connections(round-robin). it's like below pseudo code,

我有一个具有多个线程的应用程序,一个线程正在创建4个tcp连接,然后该线程创建另一个线程,它将使用poll或其他任何东西处理这4个连接的接收函数,并且原始线程(第一个)开始发送消息到这4个连接(循环)。这就像下面的伪代码,

main()
{

    pthread_create( &thread1, NULL, sending_thread, (void*) args);

}

void *sending_thread( void *ptr )
{
    int i=0;
    int *connections;
    connections = create_connections();

    pthread_create( &thread1, NULL, receiving_thread, (void*)&connections);
        while(1) {

        send(connections[i], "test"); 
        i++;
        if (i > 3)
        i=0;             

        }
}

void *receiving_thread( void *ptr )
{
    char *msg; 
        msg = receive(connections[i]);
        save_received_msg_to_disk(msg);

}

My question is How can I check my connections and bring up the disconnected one? for example, let's say connection1 went down, do I need to create another connection with the same fd, which is connection[1] in this case? or is there other ways to handle this case?

我的问题是如何检查我的连接并启动断开的连接?例如,假设connection1关闭,我是否需要使用相同的fd创建另一个连接,在这种情况下是连接[1]?还是有其他方法来处理这种情况?

Environment is C/pthread in Linux

环境是Linux中的C / pthread

1 个解决方案

#1

Here are some points based on your code and comments.

以下是基于您的代码和注释的一些要点。

• While threads do allow some things happen in parallel, they usually have a non-zero cost in terms of more involved design that always boils down to synchronization. Like in your case there's no easy way of re-establishing one of the connections and sharing it between two threads again.

虽然线程确实允许某些事情并行发生,但它们通常在涉及更多设计方面具有非零成本,总是归结为同步。就像你的情况一样,没有简单的方法可以重新建立其中一个连接并再次在两个线程之间共享它。

• I haven't ever seen an app where input and output streams are completely independent. I can imagine something like a tunneling proxy or a TCP-based VPN where that might make sense, but in general case some higher level protocol still imposes some request-response semantics that again would impose/require arbitration among sending and receiving threads.

我还没有见过输入和输出流完全独立的应用程序。我可以想象类似于隧道代理或基于TCP的VPN,这可能是有意义的,但在一般情况下,一些更高级别的协议仍然强加一些请求 - 响应语义,这些语义再次强加/要求在发送和接收线程之间进行仲裁。

• Things are different when you need throughput and when you try to minimize latency. Sleep-waiting on a mutex is usually OK for the former, but is rarely a good idea for the latter. Round-robin writing to several blocking sockets from a single thread hurts both.

当您需要吞吐量并尝试最小化延迟时,情况会有所不同。对于前者来说,在互斥上等待睡眠通常是可以的,但对后者来说很少是个好主意。从单个线程循环写入几个阻塞套接字会伤害两者。

• If you have heavy output stream(s) and sparse input, it only makes sense to use something like epoll(7) to detect saturated connections and be notified when they are available again then starve others.

如果您有大量的输出流和稀疏输入,那么只使用像epoll(7)这样的东西来检测饱和连接,并在它们再次可用时通知然后让其他人饿死是有意义的。

I know this doesn't answer your question directly, but my rant list did not fit into the comments. Hope this helps a little.

我知道这不能直接回答你的问题,但我的咆哮清单不适合评论。希望这有所帮助。

Edit 0:

Here's the usual setup with epoll(7):

以下是epoll(7)的常用设置:

• Make your sockets non-blocking (fcntl(2) with O_NONBLOCK).

使您的套接字无阻塞(带O_NONBLOCK的fcntl(2))。

• Set epoll_data.fd to your socket descriptor for each potential channel (four sockets in your example). Other options are possible with union epoll_data if you want to keep more complex structures then just socket descriptors.

将epoll_data.fd设置为每个潜在通道的套接字描述符(示例中为四个套接字)。如果你想保留更复杂的结构,那么只需要套接字描述符,就可以使用union epoll_data。

• Use EPOLLIN and EPOLLET to get edge-triggered behavior, i.e. be woken up when input buffer becomes not empty.

使用EPOLLIN和EPOLLET获取边沿触发行为,即当输入缓冲区变空时唤醒。

• Only set EPOLLOUT if you get EWOULDBLOCK from a write(2), otherwise do output as usual. Same logic here with EPOLLET to detect output buffer space becoming available.

如果从write(2)获得EWOULDBLOCK,则仅设置EPOLLOUT,否则像往常一样输出。与EPOLLET相同的逻辑用于检测输出缓冲区空间是否可用。

• Use EPOLLRDHUP to detect other side disconnecting cleanly (for abrupt disconnects you need to handle EPIPE error form write(2)).

使用EPOLLRDHUP检测另一侧是否干净地断开连接(对于突然断开连接,您需要处理EPIPE错误形式write(2))。

• epoll_wait(2) gives you back number of events to iterate through. Do separate checks for input (events & EPOLLIN) and output (events & EPOLLOUT).

epoll_wait(2)为您提供迭代的事件数。对输入(事件和EPOLLIN)和输出(事件和EPOLLOUT)进行单独检查。

• On input read from data.fd (or otherwise associated socket) until you get EWOULDBLOCK.

在从data.fd(或其他关联的套接字)读取的输入上,直到获得EWOULDBLOCK。

• On output write until you get EWOULDBLOCK or you don't have more output data pending (remove EPOLLOUT in that case).

在输出写入之前,直到您获得EWOULDBLOCK或者您没有更多的输出数据待处理(在这种情况下删除EPOLLOUT)。

It looks like a lot, but is pretty simple once you get a hang of it.

它看起来很多,但是一旦掌握它就很简单。

You can also do non-blocking connect(2), which is probably a good idea if you ever want to re-establish broken streams without hurting others that are still chugging along (connect(2) returns -1 with errno(3) set to EINPROGRESS and you wait for the socket to become writable as above).

你也可以做非阻塞连接(2),这可能是一个好主意,如果你想重新建立破坏的流而不会伤害仍在继续的其他人(connect(2)返回-1与errno(3)设置到EINPROGRESS并等待套接字变为可写如上)。