Replies: 3 comments 1 reply
-
Beta Was this translation helpful? Give feedback.
0 replies
-
What are the workloads of the volumes? Are they IO intensive? |
Beta Was this translation helpful? Give feedback.
1 reply
-
With 1.6 and v2 engine there is information about high CPU usage: As far as I see I don't have v2 volumes yet, |
Beta Was this translation helpful? Give feedback.
0 replies
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
-
There are several bugs in github about longhorn high cpu usage, but I'm not sure if I want to talk about bug or is it feature of this system. Also I think that this may be better place for discussion that bug reports.
According to docs:
My context is: small home cluster with 5 nodes (1 master), most are power efficient 4 cores nucs, with 8-64G of RAM, all SSD, few nvmes, 1G ethernet, all debian bookworm x86. There is nothing deployed with super high usage, most things is just idle, used to test interesting services with not much workload. Right now there are about 55 volumes. Longhorn 1.6.1, k3s 1.28.6
Recently I noticed that most of CPU usage comes from longhorn.
Nothing bad in logs, except 1 or two nodes with errors about wrong iscsi targets (this is already reported, happens all the time on some nodes).
I was sure that most of work on cluster is due to prometheus stack, for sure there is plenty of work to done there, it also uses biggest volume 120G., It's still way less than longhorn takes. I can see same cpu usage in htop on nodes (about 30% of each).
This leads to question - is mostly idle cluster requires so much CPU just for longhorn? I'm sure that there is nothing in background tasks (like backup, snapshots or recovery). I already reduced replicas for prometheus (to 2) and maybe I'll setup this bit differently (there is no real data there, I can always prune there and collect new).
For now I'm looking for debug - why it's taking so much cpu, how to reduce that and how to choose in future wisely.
As I said - cluster is mostly idle, not that much IO except prometheus.
Beta Was this translation helpful? Give feedback.
All reactions