Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Optimize pillar/zedbox memory usage #3860

Open
eriknordmark opened this issue Apr 17, 2024 · 0 comments
Open

Optimize pillar/zedbox memory usage #3860

eriknordmark opened this issue Apr 17, 2024 · 0 comments
Labels
bug Something isn't working enhancement New feature or request

Comments

@eriknordmark
Copy link
Contributor

The history is that at first we had separate golang binaries (for each of pkg/pillar/cmd/) but the disk image size of each one of them was >10Mbytes and it seemed wasteful to have 10-12 of them. So we introduced a single zedbox binary with symlinks (from /opt/zededa/bin/ to zedbox) thus the total disk space usage was about 30Mbytes.

A few years later we realize that the total memory usage (RSS) for these processes could grow quite large. To address this we made zedbox run as a service (where each /opt/zededa/bin/* would request zedbox to start a service - unless run as inline). The service is implemented by the single zedbox process calling the Run() function for the service, which kicks off its go routines.
This made total RSS consumption smaller. But is is still quite large.

Note that none of the above changes had any impact on the structure of each pillar microservice - apart from the main() function being renamed to Run(). Thus it should be easy to revert this to having separate processes.

It turns out that the disk size of the binaries is not a concern, so if we can get a handle on the RSS consumption it would make sense to have smaller go binaries. That would also have some security benefits since it would later allow us to use standard linux mechanisms to restrict which pillar microservices can access the network or various directories etc.

So the question is can be make either the current zedbox process have smaller RSS, or break apart into separate processes for each /opt/zededa/bin/* and have those have smaller RSS. It is not clear whether this RSS comes from init code in the various golang packages, but having smaller processes with less package imports would presumably make this easier to investigate.

@eriknordmark eriknordmark added bug Something isn't working enhancement New feature or request labels Apr 17, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something isn't working enhancement New feature or request
Projects
None yet
Development

No branches or pull requests

1 participant