具有异步功能的跨平台、C/ c++ HTTP库

I'm looking for a C/C++ library that will work on Windows and Linux which will allow me to asychronously query multiple webservers (1000's per minute) for page headers and download web pages in much the same way WinHttp library does in a windows environment.

我正在寻找一个C/ c++库，它可以在Windows和Linux上运行，允许我对多个web服务器(每分钟1000次)进行同步查询页面标题，并以Windows环境下WinHttp库的方式下载web页面。

So far I've come across libCurl which seems to do what I want but the asychronous aspect looks suspect.

到目前为止，我遇到了libCurl，它似乎可以做我想做的事情，但是异步方面看起来很可疑。

How easy do you think it would be to bypass the idea of using a library and write something simple from scratch based on sockets that could achieve this?

您认为绕过使用库的想法，根据可以实现这一点的套接字从头编写一些简单的东西有多容易?

Any comments, advice or suggestions would be very welcomed.

欢迎任何意见、建议或建议。

Addendum:- Any body have comments about doing this with libCurl, I said the asychronous aspect may look suspect but does anyone have any experience of of it?

附录:-任何一个人对用libCurl做这个有意见，我说过异步方面可能看起来可疑，但是有人有过这样的经历吗?

2 个解决方案

#1

Try libevent HTTP routines. You create an HTTP connection and provide a callback which is invoked when a response arrives (or timeout event fires).

尝试libevent HTTP的例程。创建HTTP连接并提供回调，当响应到达(或超时事件触发)时调用回调。

Updated: I built a distributed HTTP connection-throttling proxy and used both th e client and server portions within the same daemon, all on a single thread. It worked great.

更新:我构建了一个分布式HTTP连接-节流代理，并在同一个守护进程中使用e客户机和服务器部分，所有这些都在一个线程上。这工作很好。

If you're writing an HTTP client, libevent should be a good fit. The only limitation I ran into with the server side was lack of configuration options -- the API is a bit sparse if you want to start adding more advanced features; which I expected since it was never intended to replace general-purpose web servers like Apache, Nginx. For example I patched it to add a custom subroutine to limit the overall size of an inbound HTTP request (e.g. close connection after 10MB read). The code is very well-written and the patch was easy to implement.

如果您正在编写一个HTTP客户端，libevent应该是一个不错的选择。我在服务器端遇到的唯一限制是缺少配置选项——如果您想开始添加更高级的特性，API有点稀疏;这是我期望的，因为它从未打算取代像Apache、Nginx这样的通用web服务器。例如，我修补了它以添加自定义子例程来限制入站HTTP请求的总体大小(例如，在读取10MB后关闭连接)。代码写得很好，补丁很容易实现。

I was using the 1.3.x branch; the 2.x branch has some serious performance improvements over the older releases.

我用的是1。3。x分行;的2。与旧版本相比，x branch有一些严重的性能改进。

Code example: Found a few minutes and wrote a quick example. This should get you acquainted with the libevent programming style:

代码示例:找到几分钟并编写一个快速示例。这将使您熟悉libevent编程风格:

#include <stdio.h>
#include <event.h>
#include <evhttp.h>

void
_reqhandler(struct evhttp_request *req, void *state)
{
    printf("in _reqhandler. state == %s\n", (char *) state);
    if (req == NULL) {
        printf("timed out!\n");
    } else if (req->response_code == 0) {
        printf("connection refused!\n");
    } else if (req->response_code != 200) {
        printf("error: %u %s\n", req->response_code, req->response_code_line);
    } else {
        printf("success: %u %s\n", req->response_code, req->response_code_line);
    }
    event_loopexit(NULL);
}

int
main(int argc, char *argv[])
{
    const char *state = "misc. state you can pass as argument to your handler";
    const char *addr = "127.0.0.1";
    unsigned int port = 80;
    struct evhttp_connection *conn;
    struct evhttp_request *req;

    printf("initializing libevent subsystem..\n");
    event_init();

    conn = evhttp_connection_new(addr, port);
    evhttp_connection_set_timeout(conn, 5);
    req = evhttp_request_new(_reqhandler, (void *)state);
    evhttp_add_header(req->output_headers, "Host", addr);
    evhttp_add_header(req->output_headers, "Content-Length", "0");
    evhttp_make_request(conn, req, EVHTTP_REQ_GET, "/");

    printf("starting event loop..\n");
    event_dispatch();

    return 0;
}

Compile and run:

编译并运行:

% gcc -o foo foo.c -levent
% ./foo    
initializing libevent subsystem..
starting event loop..
in _reqhandler. state == misc. state you can pass as argument to your handler
success: 200 OK

#2

Microsoft's cpprestsdk is an cross platform http library that enables communications with http servers. Here is some sample code on msdn. This uses boost asio on linux and WinHttp on windows

微软的cpprestsdk是一个跨平台的http库，可以与http服务器进行通信。下面是msdn上的一些示例代码。这在linux上使用boost asio, windows上使用WinHttp

#1