Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

cpu high load with php8 #99

Open
shuyinglee opened this issue Aug 17, 2022 · 2 comments
Open

cpu high load with php8 #99

shuyinglee opened this issue Aug 17, 2022 · 2 comments

Comments

@shuyinglee
Copy link

Hi, I upgrated php8 with pecl memcache 8.0, compiled perfectly, BUT while I pushed the WE B server to LIVE, cpu load 100% in 5miniutes. i had to take it off from LB server. I tried it more times and the result was the same.
then i checked logs files:

php-fpm.log
[17-Aug-2022 10:27:19] WARNING: [pool www] child 162297, script '/htdocs/www/index/htdocs/index.php' (request: "GET /index.php") executing too slow (4.499987 sec), logging
[17-Aug-2022 10:27:19] WARNING: [pool www] child 162295, script '/htdocs/www/index/htdocs/index.php' (request: "POST /index.php") executing too slow (5.330986 sec), logging
[17-Aug-2022 10:27:19] WARNING: [pool www] child 162293, script '/htdocs/www/index/htdocs/index.php' (request: "HEAD /index.php") executing too slow (5.096421 sec), logging
[17-Aug-2022 10:27:19] WARNING: [pool www] child 162286, script '/htdocs/www/index/htdocs/index.php' (request: "GET /index.php") executing too slow (4.393981 sec), logging
[17-Aug-2022 10:27:19] WARNING: [pool www] child 162269, script '/htdocs/www/index/htdocs/index.php' (request: "GET /index.php") executing too slow (4.516977 sec), logging
[17-Aug-2022 10:27:20] WARNING: [pool www] child 162309, script '/htdocs/www/index/htdocs/index.php' (request: "POST /index.php") executing too slow (4.227775 sec), logging
[17-Aug-2022 10:27:20] WARNING: [pool www] child 162308, script '/htdocs/www/index/htdocs/index.php' (request: "POST /index.php") executing too slow (4.999566 sec), logging

php-fpm.log.slow
[17-Aug-2022 11:30:06] [pool www] pid 1019
script_filename = /htdocs/www/index/htdocs/index.php
[0x00007fcf55c149d0] addserver() /htdocs/www/library/Lib/cache_mem.php:15
[0x00007fcf55c14820] __construct() /htdocs/www/index/tmp/~run.php(50) : eval()'d code:1
[0x00007fcf55c14770] INCLUDE_OR_EVAL /htdocs/www/index/tmp/~run.php:50
[0x00007fcf55c145a0] loadClass() /htdocs/www/index/tmp/~run.php:50
[0x00007fcf55c14490] lib() /htdocs/www/library/Lib/cache.php:18
[0x00007fcf55c14330] __construct() /htdocs/www/index/tmp/~run.php(50) : eval()'d code:1
[0x00007fcf55c14280] INCLUDE_OR_EVAL /htdocs/www/index/tmp/~run.php:50
[0x00007fcf55c140b0] loadClass() /htdocs/www/index/tmp/~run.php:50
[0x00007fcf55c13fa0] lib() /htdocs/www/index/model/cache.php:6
[0x00007fcf55c13f10] init() /htdocs/www/index/tmp/~run.php:49
[0x00007fcf55c13e90] __construct() /htdocs/www/index/tmp/~run.php(50) : eval()'d code:1
[0x00007fcf55c13e00] INCLUDE_OR_EVAL /htdocs/www/index/tmp/~run.php:50
[0x00007fcf55c13c30] loadClass() /htdocs/www/index/tmp/~run.php:50
[0x00007fcf55c13950] model() /htdocs/www/index/tmp/~run.php:46
[0x00007fcf55c13830] __construct() /htdocs/www/index/tmp/~run.php:44
[0x00007fcf55c134b0] _dispatch() /htdocs/www/index/tmp/~run.php:44
[0x00007fcf55c133f0] dispatch() /htdocs/www/index/tmp/~run.php:44
[0x00007fcf55c13350] run() /htdocs/www/index/htdocs/index.php:18

[17-Aug-2022 11:30:06] [pool www] pid 1045
script_filename = /htdocs/www/index/htdocs/index.php
[0x00007fcf55c149d0] addserver() /htdocs/www/library/Lib/cache_mem.php:15
[0x00007fcf55c14820] __construct() /htdocs/www/index/tmp/~run.php(50) : eval()'d code:1
[0x00007fcf55c14770] INCLUDE_OR_EVAL /htdocs/www/index/tmp/~run.php:50
[0x00007fcf55c145a0] loadClass() /htdocs/www/index/tmp/~run.php:50
[0x00007fcf55c14490] lib() /htdocs/www/library/Lib/cache.php:18
[0x00007fcf55c14330] __construct() /htdocs/www/index/tmp/~run.php(50) : eval()'d code:1
[0x00007fcf55c14280] INCLUDE_OR_EVAL /htdocs/www/index/tmp/~run.php:50
[0x00007fcf55c140b0] loadClass() /htdocs/www/index/tmp/~run.php:50
[0x00007fcf55c13fa0] lib() /htdocs/www/index/model/cache.php:6
[0x00007fcf55c13f10] init() /htdocs/www/index/tmp/~run.php:49
[0x00007fcf55c13e90] __construct() /htdocs/www/index/tmp/~run.php(50) : eval()'d code:1
[0x00007fcf55c13e00] INCLUDE_OR_EVAL /htdocs/www/index/tmp/~run.php:50
[0x00007fcf55c13c30] loadClass() /htdocs/www/index/tmp/~run.php:50
[0x00007fcf55c13950] model() /htdocs/www/index/tmp/~run.php:46
[0x00007fcf55c13830] __construct() /htdocs/www/index/tmp/~run.php:44
[0x00007fcf55c134b0] _dispatch() /htdocs/www/index/tmp/~run.php:44
[0x00007fcf55c133f0] dispatch() /htdocs/www/index/tmp/~run.php:44
[0x00007fcf55c13350] run() /htdocs/www/index/htdocs/index.php:18

CODE: /htdocs/www/library/Lib/cache_mem.php [line 15]
$this->_mem = new Memcache;
foreach($config['servers'] as $server){
$this->_mem->addServer($server['host'], $server['port'],true,$server['weight']); //it is line 15, also tried third args to FALSE, it's the same result
}

I did some additional changes in code
if I removed parts of memcache addserver, get, set etc. in my code, the CPU load was going to VERY VERY LOW, excuted faster and faster.

MY SERVER EVN:
memcache Server:
/usr/local/servers/memcached-1.5.9/bin/memcached

CentOS Linux release 8.2.2004 (Core)

PHP 8.1.8 (cli) (built: Aug 16 2022 10:12:36) (NTS)
Copyright (c) The PHP Group
Zend Engine v4.1.8, Copyright (c) Zend Technologies

memcache
memcache support => enabled
Version => 4.0.5.2 (it should be 8.0 in pecl.php.net)
Revision => $Revision$
Directive => Local Value => Master Value
memcache.allow_failover => 1 => 1
memcache.chunk_size => 32768 => 32768
memcache.compress_threshold => 20000 => 20000
memcache.default_port => 11211 => 11211
memcache.hash_function => crc32 => crc32
memcache.hash_strategy => consistent => consistent
memcache.lock_timeout => 15 => 15
memcache.max_failover_attempts => 20 => 20
memcache.prefix_host_key => Off => Off
memcache.prefix_host_key_remove_subdomain => Off => Off
memcache.prefix_host_key_remove_www => On => On
memcache.prefix_static_key => no value => no value
memcache.protocol => ascii => ascii
memcache.redundancy => 1 => 1
memcache.session_prefix_host_key => Off => Off
memcache.session_prefix_host_key_remove_subdomain => Off => Off
memcache.session_prefix_host_key_remove_www => On => On
memcache.session_prefix_static_key => no value => no value
memcache.session_redundancy => 2 => 2
memcache.session_save_path => no value => no value

@shuyinglee
Copy link
Author

i think i have found out the reason.
I pushed about 100 servers with different port into memcache pool, and the 4th args "weight" was the same value which is 100.
that caused very slow. after i remove the 4th arg "weight" , it works very very goog.

the bug still exists, i just resolved my issue.
...
$this->_mem = new Memcache;
foreach($config['servers'] as $server){
$this->_mem->addServer($server['host'], $server['port'],true,$server['weight']); //it is line 15, also tried third args to FALSE, it's the same result
}

@TysonAndre
Copy link
Contributor

TysonAndre commented Oct 15, 2022

https://www.php.net/manual/en/memcache.addserver.php

weight
Number of buckets to create for this server which in turn control its probability of it being selected. The probability is relative to the total weight of all servers.

you're adding 100*100 buckets - I'm not familiar with what libmemcached is doing or which version you're using, but it might also be sorting the entire list of buckets every time addServer is called (100 times, for lists 50100 buckets long) (and doing more work to find your key in 100100 buckets than for just 100 buckets if it uses binary search)

smaller weights are faster but would be less evenly balanced, though 100 is excessively large

https://en.wikipedia.org/wiki/Consistent_hashing

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants