Managing multiple PHP worker child processes using Symfony2 Process

2.7k Views Asked by At

I want to use Symfony2 Process to create a process with multiple worker child process.

Basically I have a skeleton executable PHP script like below

#!/usr/bin/env php
<?php

include "vendor/autoload.php";
$maxChild = 5;
$sleep = 1;
$child = 0;
while(true){
    if ($child < $maxChild){
        $random = rand(3, 9);
        $date = date('YmdHis');
        $process = new \Symfony\Component\Process\Process(<<<CMD
                sleep {$random} && \
                touch /tmp/foo-{$child}-{$random}-{$date}-`date +%Y%m%d%H%M%S` && \
                echo 'foo'
CMD
        );

    }
    sleep($sleep);
}

The problem is how to spawn a new child worker as soon as a child worker finishes.

As far as I know, Process::start() is the way to run the process asynchronously.

So if I run this

 #!/usr/bin/env php
<?php

include "vendor/autoload.php";
$maxChild = 5;
$sleep = 1;
$child = 0;
while(true){
    if ($child < $maxChild){
        $random = rand(3, 9);
        $date = date('YmdHis');
        $process = new \Symfony\Component\Process\Process(<<<CMD
                sleep {$random} && \
                touch /tmp/foo-{$child}-{$random}-{$date}-`date +%Y%m%d%H%M%S` && \
                echo 'foo'
CMD
        );

        $process->start(function($type, $output) use (&$child){
            echo "{$output}\n";
            $child--;
        });
        $pid = $process->getPid();
        echo "Spawning #{$child}. pid: {$pid}. random: {$random}\n";
        $child++;

    }
    sleep($sleep);
}

the child process will run asynchronously because of I see this

$ ls --full-time /tmp/ | grep foo
-rw-rw-r-- 1 petra   petra        0 2014-01-29 15:47:52.208715691 +0700 foo-0-5-20140129154747-20140129154752
-rw-rw-r-- 1 petra   petra        0 2014-01-29 15:47:53.208715735 +0700 foo-1-5-20140129154748-20140129154753
-rw-rw-r-- 1 petra   petra        0 2014-01-29 15:47:58.212715939 +0700 foo-2-9-20140129154749-20140129154758
-rw-rw-r-- 1 petra   petra        0 2014-01-29 15:47:53.212715734 +0700 foo-3-3-20140129154750-20140129154753
-rw-rw-r-- 1 petra   petra        0 2014-01-29 15:47:59.216715981 +0700 foo-4-8-20140129154751-20140129154759

for example file foo-4-8-20140129154751-20140129154759 is created around the random time generated, which is 8 seconds.

But then the only way I could know which process is ending is by using Process::wait() but that would block the entire process. So, when the main loop is blocked waiting for a child process to finish, another child may be ended early.

I initially want to execute another PHP file from inside loop using php file.php command since most of the business logic is in the same codebase. So trying this first using PHP is my first intuition.

So my questions are

  1. Is there a way to spawn a new child process soon after a child process ended within number of max child but without blocking using Symfony2 Process?
  2. Or is there a mature PHP library to do this (not framework, though since I already use a framework, and preferably PHP 5.3+ library)?
  3. If I can not do this using PHP, is there another way to do this maybe perhaps using Python or PERL? I kinda need a simple one-file script solution. Or maybe I could use some software (preferably installable in Linux) that requires small configuration and less coding?

UPDATE

I just learned about supervisor. I hope this is suitable for my needs. So I'm thinking instead of creating a PHP script to spawn child processes, maybe I can make one script using infinite loop and spawn a blocking child process and then set the supervisor to run several of the script. But for the sake of curiosity, it would be nice if someone can answer the questions above.

0

There are 0 best solutions below