Is it possible to have multiple different amphp workers on the same core doing file_get_contents concurrently?

52 Views Asked by At

I am implementing AMPHP for multithreading, replacing an existing "multicurl callback" solution where new php processes were created by doing an external call to the server from within the code.

Given I have an 8 core CPU, when I submit 50 "Task"s from Amp\Parallel\Worker\Task which each call file_get_contents on a url, what happens?

Is it that 50 threads/processes will be created, each of which will hog a core until their file_get_contents is done? Or is it that they will suspend and resume the processes as needed?

From this thread: Running multiple processes in parallel in php It sounds like they will hog cores...

If I switch file_get_contents to the AMPHP http-client, will this behaviour change (because it's "non-blocking"), even if I don't use Promises?

I don't write StackOverflow questions very much, so please let me know if I need to edit my question for clarification.

1

There are 1 best solutions below

0
Peter Job On

This is only a partial answer to this question, but I have done some tests which show that with a simple substitution of file_get_contents for the AMPHP http-client HttpClientBuilder, I have got significant improvements:

Worker time with file_get_contents for 4 workers: 
0.4008469581604
Worker time with HTTP client for 4 workers:
0.33068490028381
Time with just HTTP client for 4 requests:
0.63406896591187
Worker time with file_get_contents for 16 workers: 
1.4112739562988
Worker time with HTTP client for 16 workers:
0.33482003211975
Time with just HTTP client for 16 requests:
2.6101880073547
Worker time with file_get_contents for 32 workers: 
1.4165980815887
Worker time with HTTP client for 32 workers:
0.33442378044128
Time with just HTTP client for 32 requests:
5.2062652111053
Worker time with file_get_contents for 64 workers: 
2.4925398826599
Worker time with HTTP client for 64 workers:
1.4523921012878
Time with just HTTP client for 64 requests:
10.369027853012

My test function is:

public function test100Workers(){
        $nWorkersArray = [4, 16, 32, 64];

        foreach ($nWorkersArray as $nWorkers){
            $futures = [];

            //Workers
            $t1 = microtime(true);
            for ($i = 0; $i < $nWorkers; $i++) {
                $futures[] = submit(new FileReadTask(false))->getFuture();
            }
            $this->assertTrue(await($futures)[0]);

            //Manual
            $t2 = microtime(true);

            //AMPHP Http client
            for ($i = 0; $i < $nWorkers; $i++) {
                $futures[] = submit(new FileReadTask(true))->getFuture();
            }
            $this->assertTrue(await($futures)[0]);
            $t3 = microtime(true);

            //AMPHP Http client no workers
            for ($i = 0; $i < $nWorkers; $i++) {
                $x = new FileReadTask(true);
                $x->getWebsite();
            }
            $t4 = microtime(true);

            echo "\nWorker time with file_get_contents for $nWorkers workers: \n";
            print_r($t2 - $t1);

            echo "\nWorker time with HTTP client for $nWorkers workers:\n";
            print_r($t3 - $t2);

            echo "\nTime with just HTTP client for $nWorkers requests:\n";
            print_r($t4 - $t3);
        }

    }

Where FileReadTask is:


class FileReadTask implements Task {
    public function __construct(
        private bool $ampHttp = false
    ) {
    }

    public function run(Channel $channel, Cancellation $cancellation): mixed {
        $x = $this->getWebsite();
        return true;
    }

    public function getWebsite(){
        if ($this->ampHttp){
            $request = new Request("https://httpbin.org");
            $x = HttpClientBuilder::buildDefault()->request($request);
            $y = $x->getHeaders();
        }else{
            $x = file_get_contents("https://httpbin.org");
        }
        return $x;
    }
}

This suggests that Amp http-client is "less blocking" than file_get_contents, but only when you use it in the environment of a worker.