My webserver has been experiencing a problem of php-fpm active processes slowly increasing till the pm.max_children setting is reached, at which point it's stuck and I need to restart php-fpm. (os: ubuntu 20.0.4, webserver: Caddy, php-fpm version: 7.1, pm = dynamic, running Laravel 5.5 framework)
I've enable the php-fpm status page and found that many processes are stuck in the "Getting request informations" state. Example row from output: of /status?html&full (this has been stuck here for over an hour)
| pid | state | start time | start since | requests | request duration | request method | request uri | content length | user | script | last request cpu | last request memory |
|---|---|---|---|---|---|---|---|---|---|---|---|---|
| 1772235 | Getting request informations | 24/Jun/2021:15:03:07 +0000 | 5111 | 131 | 4625314443 | POST | /api.php?t=removed&e=/role/checkOut/3461 | 5542139 | - | /var/www/nameremoved/app/fe/production/api.php | 0.00 | 0 |
Can anyone shed some light on what the "Getting request informations" state is? I can't seem to find anywhere it's documented.
In php.ini I have: max_execution_time = 180 Yet this seems to be ignored.. The scripts being run are from Laravel 5.5 and definitely shouldn't take more than a few seconds to execute - they are just basic database operations, maybe with file uploads that could be up to 500MB
I guess my next step could be to set the php-fpm setting: request_terminate_timeout and see if that terminates the processes.
The strange thing is I have an identical server set up in a different location (requests are routed to either server based on location) which does not have this problem.
Any advice appreciated :)
UPDATE 25/6/2021 Still happening, it seems to be only for POST requests with file uploads
UPDATE 29/6/2021 I've set request_terminate_timeout=2h this successfully kills the requests stuck in the "Getting request informations" state.. so this kinda solves the problem but I still have no idea what was causing it
UPDATE 16/6/2022 Now using Php 8.1, Laravel 8, Caddy v 2.4.6 same problem still occurring. I've added global before and after middleware in Laravel to log each http request with php-fpm process id to try to find the culprit, but it seems the problem is occuring before the before middleware is even being hit..
I have the same behavior with Ubuntu 20.04.3 LTS, Laravel 8, php-fpm 7.4 and caddy 2.4.5.
Restarting either the caddy or php-fpm service immediately frees up the processes. So I first quickly "fixed" it by restarting caddy every 15 minutes via crontab.
Since this doesn't happen with nginx, I'm now running caddy -> nginx -> php-fpm, it works so far.