[ Nextcloud ] Last update php-fpm high number of threads

[zerpex]

Hi,

Since today, my server faced a really high load average :

root@aragog:# top -b -n 1 | head
top - 18:36:02 up  2:13,  1 user,  load average: **17,00, 15,81, 11,89**
Tasks: 222 total,   1 running, 221 sleeping,   0 stopped,   0 zombie
%Cpu(s):  1,5 us,  0,4 sy,  0,0 ni, **96,4 id**,  1,7 wa,  0,0 hi,  0,1 si,  0,0 st
KiB Mem:   8179716 total,  5321888 used,  2857828 free,   115024 buffers
KiB Swap:  5103612 total,        0 used,  5103612 free.  3841176 cached Mem

  PID USER      PR  NI    VIRT    RES    SHR S  %CPU %MEM     TIME+ COMMAND
 3040 root      20   0  699676 111952   7176 S  12,6  1,4   0:09.74 guacd
 3077 root      20   0   25760   2852   2404 R   6,3  0,0   0:00.01 top
    1 root      20   0   29416   5200   3100 S   0,0  0,1   0:00.69 systemd

While throubleshooting, I noticed that :
1/ My CPUs were idle
2/ There was a large number of php-fpm processes in nextcloud's container :

/ # ps auxf
PID   USER     TIME   COMMAND
    1 1000       0:00 /bin/s6-svscan /etc/s6.d
   52 1000       0:00 s6-supervise php
   53 1000       0:00 s6-supervise nginx
   54 1000       0:00 s6-supervise cron
   55 1000       0:00 php-fpm: master process (/php/etc/php-fpm.conf)
   56 1000       0:00 nginx: master process nginx
   57 1000       0:00 {run} /bin/sh ./run
   58 1000       0:00 php -d memory_limit=1g -f /nextcloud/cron.php
   59 1000       0:00 nginx: worker process
   60 1000       0:00 nginx: worker process
  125 1000       0:00 php-fpm: pool www
  126 1000       0:00 php-fpm: pool www
  127 1000       0:00 php-fpm: pool www
  128 1000       0:00 php-fpm: pool www
  129 1000       0:00 php-fpm: pool www
  130 1000       0:00 php-fpm: pool www
  134 1000       0:00 php-fpm: pool www
  137 1000       0:00 php-fpm: pool www
  138 1000       0:00 php-fpm: pool www
  139 1000       0:00 php-fpm: pool www
  142 1000       0:00 php-fpm: pool www
  143 1000       0:00 php-fpm: pool www
  147 root       0:00 sh
  152 1000       0:00 php-fpm: pool www
  153 1000       0:00 php-fpm: pool www
  154 1000       0:00 php-fpm: pool www

(I only have 2 cores on this server)

Looking at php-fpm's conf, I noticed that :

/ # cat /php/etc/php-fpm.conf
[global]
daemonize = no
error_log = /php/logs/error.log

[www]
listen = /php/run/php-fpm.sock
pm = dynamic
pm.max_children = **15**
pm.start_servers = 2
pm.min_spare_servers = 1
pm.max_spare_servers = 6
chdir = /
request_terminate_timeout = 0
env[PATH] = /usr/local/bin:/usr/bin:/bin
php_admin_value[post_max_size] = 30G
php_admin_value[upload_max_filesize] = 30G
php_admin_value[max_execution_time] = 10800
php_admin_value[max_input_time] = 3600
php_admin_value[expose_php] = Off
php_admin_value[memory_limit] = 512M
php_admin_value[session.save_path] = "/php/session"

I changed "15" by "2", restarted the container and everything went fine.

Will you please update your image or do I have to share php-fpm.conf with my host in order to keep the good setting ?

Regards,
z.

[benyanke]

Would dropping the number of workers slow things down? It's my understanding that in dynamic mode, the max children are only reached if the other children are already overloaded.

Given that, I'm thinking you're correct that the best way to modify this would be to share in the config file from your host as a mount.

[JOduMonT]

@zerpex I have the same issue on a brand new nextcloud 14 installed an debian 9 (no docker)
https://help.nextcloud.com/t/how-many-max-children-do-you-have/38340