如何同时执行多个Guzzle请求?

时间:2021-12-04 20:41:18

I can perform single requests using Guzzle and I'm very pleased with Guzzle's performance so far however, I read in the Guzzle API something about MultiCurl and Batching.

我可以使用Guzzle执行单个请求,到目前为止我对Guzzle的性能非常满意,但是我在Guzzle API中读到了一些关于MultiCurl和Batching的内容。

Could someone explain to me how to make multiple requests at the same time? Async if possible. I don't know if that is what they mean with MultiCurl. Sync would also be not a problem. I just want to do multiple requests at the same time or very close (short space of time).

有人可以向我解释如何同时发出多个请求吗?尽可能异步。我不知道这是不是他们对MultiCurl的意思。同步也不是问题。我只是想同时或非常接近(短时间)做多个请求。

2 个解决方案

#1


20  

From the docs: http://guzzle3.readthedocs.org/http-client/client.html#sending-requests-in-parallel

来自文档:http://guzzle3.readthedocs.org/http-client/client.html#sending-requests-in-parallel

For an easy to use solution that returns a hash of request objects mapping to a response or error, see http://guzzle3.readthedocs.org/batching/batching.html#batching

有关易于使用的解决方案,它返回映射到响应或错误的请求对象的哈希,请参阅http://guzzle3.readthedocs.org/batching/batching.html#batching

Short example:

简短的例子:

<?php

$client->send(array(
    $client->get('http://www.example.com/foo'),
    $client->get('http://www.example.com/baz'),
    $client->get('http://www.example.com/bar')
));

#2


18  

An update related to the new GuzzleHttp guzzlehttp/guzzle

与新GuzzleHttp guzzlehttp / guzzle相关的更新

Concurrent/parallel calls are now run through a few different methods including Promises.. Concurrent Requests

并发/并行调用现在通过几种不同的方法运行,包括Promises .. Concurrent Requests

The old way of passing a array of RequestInterfaces will not work anymore.

传递RequestInterfaces数组的旧方法将不再起作用。

See example here

见这里的例子

    $newClient = new  \GuzzleHttp\Client(['base_uri' => $base]);
    foreach($documents->documents as $doc){

        $params = [
            'language' =>'eng',
            'text' => $doc->summary,
            'apikey' => $key
        ];

        $requestArr[$doc->reference] = $newClient->getAsync( '/1/api/sync/analyze/v1?' . http_build_query( $params) );
    }

    $time_start = microtime(true);
    $responses = \GuzzleHttp\Promise\unwrap($requestArr); //$newClient->send( $requestArr );
    $time_end = microtime(true);
    $this->get('logger')->error(' NewsPerf Dev: took ' . ($time_end - $time_start) );

Update: As suggested in comments and asked by @sankalp-tambe, you can also use a different approach to avoid that a set of concurrent request with a failure will not return all the responses.

更新:正如@ sankalp-tambe在评论中所建议的那样,您也可以使用不同的方法来避免一组带有失败的并发请求不会返回所有响应。

While the options suggested with Pool is feasible i still prefer promises.

虽然建议使用Pool的选项是可行的,但我仍然更喜欢承诺。

An example with promises is to use settle and and wait methods instead of unwrap.

承诺的一个例子是使用结算和等待方法而不是解包。

The difference from the example above would be

与上面的例子不同的是

$responses = \GuzzleHttp\Promise\settle($requestArr)->wait(); 

I have created a full example below for reference on how to handle the $responses too.

我在下面创建了一个完整的示例,以获取有关如何处理$ response的参考。

require __DIR__ . '/vendor/autoload.php';
use GuzzleHttp\Client as GuzzleClient;
use GuzzleHttp\Promise as GuzzlePromise;

$client = new GuzzleClient(['timeout' => 12.0]); // see how i set a timeout
$requestPromises = [];
$sitesArray = SiteEntity->getAll(); // returns an array with objects that contain a domain

foreach ($sitesArray as $site) {
    $requestPromises[$site->getDomain()] = $client->getAsync('http://' . $site->getDomain());
}

$results = GuzzlePromise\settle($requestPromises)->wait();

foreach ($results as $domain => $result) {
    $site = $sitesArray[$domain];
    $this->logger->info('Crawler FetchHomePages: domain check ' . $domain);

    if ($result['state'] === 'fulfilled') {
        $response = $result['value'];
        if ($response->getStatusCode() == 200) {
            $site->setHtml($response->getBody());
        } else {
            $site->setHtml($response->getStatusCode());
        }
    } else if ($result['state'] === 'rejected') { 
        // notice that if call fails guzzle returns is as state rejected with a reason.

        $site->setHtml('ERR: ' . $result['reason']);
    } else {
        $site->setHtml('ERR: unknown exception ');
        $this->logger->err('Crawler FetchHomePages: unknown fetch fail domain: ' . $domain);
    }

    $this->entityManager->persist($site); // this is a call to Doctrines entity manager
}

This example code was originally posted here.

此示例代码最初发布在此处。

#1


20  

From the docs: http://guzzle3.readthedocs.org/http-client/client.html#sending-requests-in-parallel

来自文档:http://guzzle3.readthedocs.org/http-client/client.html#sending-requests-in-parallel

For an easy to use solution that returns a hash of request objects mapping to a response or error, see http://guzzle3.readthedocs.org/batching/batching.html#batching

有关易于使用的解决方案,它返回映射到响应或错误的请求对象的哈希,请参阅http://guzzle3.readthedocs.org/batching/batching.html#batching

Short example:

简短的例子:

<?php

$client->send(array(
    $client->get('http://www.example.com/foo'),
    $client->get('http://www.example.com/baz'),
    $client->get('http://www.example.com/bar')
));

#2


18  

An update related to the new GuzzleHttp guzzlehttp/guzzle

与新GuzzleHttp guzzlehttp / guzzle相关的更新

Concurrent/parallel calls are now run through a few different methods including Promises.. Concurrent Requests

并发/并行调用现在通过几种不同的方法运行,包括Promises .. Concurrent Requests

The old way of passing a array of RequestInterfaces will not work anymore.

传递RequestInterfaces数组的旧方法将不再起作用。

See example here

见这里的例子

    $newClient = new  \GuzzleHttp\Client(['base_uri' => $base]);
    foreach($documents->documents as $doc){

        $params = [
            'language' =>'eng',
            'text' => $doc->summary,
            'apikey' => $key
        ];

        $requestArr[$doc->reference] = $newClient->getAsync( '/1/api/sync/analyze/v1?' . http_build_query( $params) );
    }

    $time_start = microtime(true);
    $responses = \GuzzleHttp\Promise\unwrap($requestArr); //$newClient->send( $requestArr );
    $time_end = microtime(true);
    $this->get('logger')->error(' NewsPerf Dev: took ' . ($time_end - $time_start) );

Update: As suggested in comments and asked by @sankalp-tambe, you can also use a different approach to avoid that a set of concurrent request with a failure will not return all the responses.

更新:正如@ sankalp-tambe在评论中所建议的那样,您也可以使用不同的方法来避免一组带有失败的并发请求不会返回所有响应。

While the options suggested with Pool is feasible i still prefer promises.

虽然建议使用Pool的选项是可行的,但我仍然更喜欢承诺。

An example with promises is to use settle and and wait methods instead of unwrap.

承诺的一个例子是使用结算和等待方法而不是解包。

The difference from the example above would be

与上面的例子不同的是

$responses = \GuzzleHttp\Promise\settle($requestArr)->wait(); 

I have created a full example below for reference on how to handle the $responses too.

我在下面创建了一个完整的示例,以获取有关如何处理$ response的参考。

require __DIR__ . '/vendor/autoload.php';
use GuzzleHttp\Client as GuzzleClient;
use GuzzleHttp\Promise as GuzzlePromise;

$client = new GuzzleClient(['timeout' => 12.0]); // see how i set a timeout
$requestPromises = [];
$sitesArray = SiteEntity->getAll(); // returns an array with objects that contain a domain

foreach ($sitesArray as $site) {
    $requestPromises[$site->getDomain()] = $client->getAsync('http://' . $site->getDomain());
}

$results = GuzzlePromise\settle($requestPromises)->wait();

foreach ($results as $domain => $result) {
    $site = $sitesArray[$domain];
    $this->logger->info('Crawler FetchHomePages: domain check ' . $domain);

    if ($result['state'] === 'fulfilled') {
        $response = $result['value'];
        if ($response->getStatusCode() == 200) {
            $site->setHtml($response->getBody());
        } else {
            $site->setHtml($response->getStatusCode());
        }
    } else if ($result['state'] === 'rejected') { 
        // notice that if call fails guzzle returns is as state rejected with a reason.

        $site->setHtml('ERR: ' . $result['reason']);
    } else {
        $site->setHtml('ERR: unknown exception ');
        $this->logger->err('Crawler FetchHomePages: unknown fetch fail domain: ' . $domain);
    }

    $this->entityManager->persist($site); // this is a call to Doctrines entity manager
}

This example code was originally posted here.

此示例代码最初发布在此处。