exec() should be asynchronous, but it isn't

Question

I'm creating a web application using the Yii 2 Framework, running on an Apache webserver with PHP (the server is Ubuntu).

A big part of the app involves users uploading video, and that video being run through FFMpeg to be resaved twice, once as an MP4 and again as a WEBM. FFMpeg also extracts a frame, which is then run through Imagick to be sized correctly.

All of that takes a significant amount of time, so rather than presenting the user with a loading screen for possibly 5-10 minutes, I've opted to put all that processing in a console command, which then runs asynchronously in the background, and emails them when their video is done processing.

Here's the relevant part of the upload form model:

// if the new database entry successfully saved
if($video->save()){

    // define the target filename and full target filepath
    $target_name = uniqid();
    $target_path = Yii::getAlias('@webroot'). '/videos/' . $target_name;

    // get the current working directory (should be /models)
    $cwd = getcwd();

    // move up one directory to the app base 
    chdir('../');

    // prepare the shell command to process the video
    $command = escapeshellcmd("php yii video/process " . $fileFullPath . " " . $target_name . " " . $target_path . " " . $video->id . " " . $video->name . " " . Yii::$app->language . " " . Yii::$app->homeUrl . " " . $this->email . " >/dev/null 2>&1 &");

    // execute the shell command
    exec($command);

    // change the working directory back to the original
    chdir($cwd);

    // return the ID of the uploaded video
    return $video->id;
}

At the end of the shell command, you can see the /dev/null redirect that should cause the command to execute asynchronously, allowing the PHP script to continue on and return the uploaded video ID to the Controller.

Here's a somewhat shortened version of the VideoController method actionProcess:

public function actionProcess($source_path, $target_name, $target_path, $id, $first_name, $language, $homeUrl, $email)
{
    $ffmpeg = FFMpeg\FFMpeg::create(['timeout' => 7200, 'ffmpeg.threads' => 4]);
    $video = $ffmpeg->open($source_path);
    $dimension = new FFMpeg\Coordinate\Dimension(1280, 1280);

    $video
        ->filters()
        ->resize($dimension, 'inset')
        ->synchronize();

    $video
        ->frame(FFMpeg\Coordinate\TimeCode::fromSeconds(2))
        ->save($target_path . '.png');

    $video
        ->save(new FFmpeg\Format\Video\X264(), $target_path . '.mp4')
        ->save(new FFMpeg\Format\Video\WebM(), $target_path . '.webm');

    @unlink($source_path);

    $image = Imagick::open($target_path . '.png');

    //////////
    // There's a big if statement here controlling whether to crop the image vertically or horizontally to get the desired size.
    // Didn't seem necessary to include.
    //////////

    $image->saveTo($target_path . '.png');

    $video = Video::find()->where(['id' => $id])->one();
    $video->path = $target_name;
    $video->published = Video::IS_PUBLISHED;
    $video->save();

    //////////
    // There's another large code block here to send an email to the user.
    // Also didn't seem necessary to include.
    //////////

    return 0;
}

As you might be able to tell, I'm using the PHP-FFmpeg library (https://github.com/PHP-FFMpeg/PHP-FFMpeg) to call FFMpeg, and I'm using tpmanc's Yii2 Imagick library (https://github.com/tpmanc/yii2-imagick) to implement Imagick.

So, with all of that said: the exec() command SHOULD be getting implemented asynchronously, however it isn't. Uploading a video results in the video being uploaded, and then waiting around for another 5-10 minutes while the video processing works and finishes before finally loading the 'successfully uploaded' page.

Here's the thing: it WAS working. I tested it very early on in the development cycle and it was fine. Then I commented out the /dev/null redirect on the shell command so I could debug while developing, and now that I've added it back, it doesn't seem to be working anymore. What could be causing the above command to NOT execute asynchronously?

EDIT: I should also add that the only changes made, were made to the console-executed PHP script. No changes were made to the script executing the actual command itself (the upload form model) in between when it worked, and when it didn't. So either there's a glaringly obvious typo I'm missing, or something in the console command is overriding the /dev/null redirect and forcing the form model to wait for the script to finish, which doesn't seem like it should actually be possible, though of course I could be wrong on that count.

UPDATE: I wound up using Cron to accomplish this rather than manually calling a script every time a video is uploaded. That said, I think this question should remain open as the actual question is as-yet unresolved: why would the above exec() not execute asynchronously?

FINAL EDIT: Welp, here come the downvote fairies. Consider the question closed.

Answer 1

I have the same problem, years ago with videos. PHP is not async. I suggest to you use React PHP (http://reactphp.org/) or create JobQueue and/or Cron task lists to process this after. You can create the video and upload, on show to user the video, show message, "Rendering" and create a sample thumbnail to videos when not created the thumbnail to him.

I founded this example using Yii to JobQueue, but search better after. https://github.com/yiisoft/yii2-queue