GuzzleHttp Promises problem: requests are running synchronously

32 Views Asked by At

I have a problem. In the code below I use GuzzleHttp Promises to handle asynchronous requests. What happens here is that the requests start asynchronous but when I enter in the loop in the then() call the foreach loop is called synchronously as you can see from the output. Code:

$badgeboxApi = new BadgeboxApi(null, $api_key, $api_secret);

        $countArray = $badgeboxApi->getCompanyCount();

        if (empty($countArray["value"])) {
            Log::info("Could not retrieve company employees count");
            return 1;
        }

        $page = 0;
        $limit = 100;

        $max = $countArray["value"];
        $totalPages = ceil($max / $limit);

        $handler = new CurlHandler([
            'handle_factory' => new CurlFactory(1)
        ]);

        $client = new Client([
            'base_uri' => env('BADGEBOX_BASE_URL'),
            'timeout' => PHP_INT_MAX,
            'connect_timeout' => PHP_INT_MAX,
            'handler' => HandlerStack::create($handler)
        ]);


        $promises = (function () use ($client, $limit, $totalPages, $api_key, $api_secret, $company, $badgeboxApi) {
            for ($page = 0; $page < $totalPages; $page++) {
                $paramsStr = "?api_key=$api_key&api_secret=$api_secret&page=$page&limit=$limit";
                yield $client->postAsync(
                    env('BADGEBOX_BASE_URL') .
                    "/server/api/" .
                    env('BADGEBOX_API_VERSION') .
                    "/company/employees" .
                    $paramsStr
                )->then(function ($response) use ($company, $badgeboxApi, $paramsStr, $page) {
                    $_data = json_decode($response->getBody(), true);
                    if (!empty($_data["employees"])) {
                        Log::info("Count employees");
                        Log::info(count($_data["employees"]));
                        foreach ($_data["employees"] as $k => $employee) {
                            Log::info(
                                "Iteration " . ($k) .
                                "\$page=$page"
                            );
                            try {
                                GetOrSetManager::getOrSetUserGetEmployee($company, $badgeboxApi, $employee["id"]);
                            } catch (\Exception $e) {
                                Log::info("GetOrSetManager error");
                                Log::info($e->getMessage());
                            }
                        }
                    }
                });
            }
        })();
        $eachPromise = new \GuzzleHttp\Promise\EachPromise($promises, [
            'concurrency' => 150,
            'fulfilled' => function ($response) {

            },
            'rejected' => function ($reason) {
            }
        ]);
        $eachPromise->promise()->wait();

OUTPUT:

[2024-02-12 08:35:27] production.INFO: Count employees  
[2024-02-12 08:35:27] production.INFO: 100  
[2024-02-12 08:35:27] production.INFO: Iteration 0$page=0
[2024-02-12 08:35:28] production.INFO: Count employees  
[2024-02-12 08:35:28] production.INFO: 100  
[2024-02-12 08:35:28] production.INFO: Iteration 0$page==1
[2024-02-12 08:35:29] production.INFO: Count employees  
[2024-02-12 08:35:29] production.INFO: 100  
[2024-02-12 08:35:29] production.INFO: Iteration 0$page=2
[2024-02-12 08:35:29] production.INFO: Count employees  
[2024-02-12 08:35:29] production.INFO: 100  
[2024-02-12 08:35:29] production.INFO: Iteration 0$page=3
[2024-02-12 08:35:30] production.INFO: Count employees  
[2024-02-12 08:35:30] production.INFO: 100  
[2024-02-12 08:35:30] production.INFO: Iteration 0$page=4
[2024-02-12 08:35:31] production.INFO: Count employees  
[2024-02-12 08:35:31] production.INFO: 100  
[2024-02-12 08:35:31] production.INFO: Iteration 0$page=5
[2024-02-12 08:35:32] production.INFO: Count employees  
[2024-02-12 08:35:32] production.INFO: 100  
[2024-02-12 08:35:32] production.INFO: Iteration 0$page=6
[2024-02-12 08:35:33] production.INFO: Count employees  
[2024-02-12 08:35:33] production.INFO: 100  
[2024-02-12 08:35:33] production.INFO: Iteration 0$page=7
[2024-02-12 08:35:34] production.INFO: Count employees  
[2024-02-12 08:35:34] production.INFO: 100  
[2024-02-12 08:35:34] production.INFO: Iteration 0$page=8
[2024-02-12 08:35:35] production.INFO: Count employees  
[2024-02-12 08:35:35] production.INFO: 100  
[2024-02-12 08:35:35] production.INFO: Iteration 0$page=9
[2024-02-12 08:35:36] production.INFO: Count employees  
[2024-02-12 08:35:36] production.INFO: 100  
[2024-02-12 08:35:36] production.INFO: Iteration 0$page=10
[2024-02-12 08:35:37] production.INFO: Count employees  
[2024-02-12 08:35:37] production.INFO: 100  
[2024-02-12 08:35:37] production.INFO: Iteration 0$page=11
[2024-02-12 08:35:38] production.INFO: Iteration 1$page=11
[2024-02-12 08:35:39] production.INFO: Iteration 2$page=11
[2024-02-12 08:35:40] production.INFO: Iteration 3$page=11
[2024-02-12 08:35:41] production.INFO: Iteration 4$page=11
[2024-02-12 08:35:43] production.INFO: Iteration 5$page=11

As you can see the first 10 requests are running asynchronous, but then the 11th "then" function is run synchronous (The number of iterations after page=10 is consecutive). I didn't write all the logs, but what happens is that after page=11 it runs all iterations on page=10, page=9, page=8 one after another. What can I do to improve performance? Anybody knows why my script has this behaviour?

0

There are 0 best solutions below