Google API PHP library files.get in parallel

148 Views Asked by At

I want to run files.list to get the files list on a users Drive, and then run files.get in parallel to download the files. The problem is that when the parallel operation is preceded by a single request (files.list or files.get) to the API, the Google client breaks with "Unsupported operand types" in /vendor/guzzlehttp/guzzle/src/Handler/CurlFactory.php line 140.

I'm using the official Google PHP client together with Amphp's parallelMap().

My code looks like this:

function __construct($token, $refresh_token)
{
    $google_client = new \Google_Client();
    // ...........
    $this->api = new \Google_Service_Drive($google_client);
}


public function downloadFile($remote_file_id)
{
    $file = $this->api->files->get($remote_file_id, ['alt' => 'media']);
    return $file->getBody()->getContents();
}


public function downloadFiles($files_ids)
{
    $download_jobs = parallelMap($files_ids, function ($file_id) {
        return $this->downloadFile($file_id);
    });

    return wait($download_jobs);
}

// $storage = new Storage(..., ...);

$files1 = [
    0 => "1s9aaHl8UsTswPMsXAt6X_ooDY0idv27dfSR4ysWBI5J2KSuvqw",
    1 => "1ZKQmozeA9TfUDp5dTs7CDuRcuesBhkK_iUe4XG3L1nE9ySRz6Q",
    2 => "12kGcMS_l58cGJWvdOyQRLqwbbWhTHeA_OBuVITbeMcN0V0PnPA",
    3 => "1ETVMmfno_DUn7KloyUpzukOFlt0QcinqZ0C2pRgff-osIHyf-Q",
    4 => "1P6CIe7A8qsgLLHf3AZn0Gj4YsTHPsGIZRok0_Pke60QeAXGoMg",
    5 => "1d8K7WvDqT40b1MGIDWMxK010invlmaX4e6qdc2pui7TV3Z4liQ",
 ];

 $downloaded = $storage->downloadFile($files1[0]); // if uncommented, this causes downloadFiles to break
 $downloaded = $storage->downloadFiles($files1);

It almost seems like the first request does something internally to the Google client and then parallel requests stop working. If I only run downloadFiles(), it works smoothly. Even this works:

    $files = [];
    $files[] = $storage->downloadFiles([$files1[0]]);
    $files[] = $storage->downloadFiles([$files1[1]]);
    $files[] = $storage->downloadFiles([$files1[2]]);
    $files[] = $storage->downloadFiles([$files1[3]]);
  1. Why does Google client library break when run in parallel?
  2. What would be the best practice to run Drive downloads in parallel on PHP+Laravel? I am following the best practices from Google, handle their exceptions and each error code individually.

I am running PHP 7.2.11

0

There are 0 best solutions below