We need your help/opinion on some issues we're having with slowness in our production app. Some context: we have two AWS EC2 servers (t3a.2xlarge) running a symfony 2.8 app (php 5.6) through php-fpm/nginx, which connects mostly to mariadb, but also sometimes to mongodb.
When we have less than ~700-800 users, the performance is ok. With more than that, we start to have some serious performance problems, and the CPU of the machines reaches 80-100%.
In one of those moments, for a simple API call, our datadog shows the following trace:
The AppKernel is taking 7 seconds until the first mysql query is done. That controller action code is quite simple, the problem is not in that code. In a normal situation, this call takes 200ms total. We have this same problem in other controllers actions. We already fine tuned a lot of the php-fpm and opcache parameters to:
pm.max_children: 216
pm.start_servers: 54
pm.max_spare_servers: 162
pm.min_spare_servers: 54
pm.process_idle_timeout: 10s
pm.max_requests: 500
realpath_cache_size: 4096k
realpath_cache_ttl: 300
We're basically struggling a bit to find the real cause of this slowness. Is it just a matter of cpu, because the php5.6/symfony 2.8 is a slow and old stack? Or is there some fine-tuning we can make so that the AppKernel becomes faster?
Although we are migrating all this stack to a more recent one, we still have plenty of symfony 2.8 code running with php 5.6, so this is still pretty important to us.
I would really appreciate your help on this. Thanks!