Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Memory leak in kubos_system due to logging dependency #9

Open
MoralCode opened this issue Mar 29, 2022 · 5 comments
Open

Memory leak in kubos_system due to logging dependency #9

MoralCode opened this issue Mar 29, 2022 · 5 comments
Labels
bug Something isn't working

Comments

@MoralCode
Copy link
Member

Description
When creating a custom service that configures KubOS logging functionality, the service increasingly uses more system memory as it runs. This memory consumption appears to be proportional to the number of log statements made by the service. This appears to be due to a possible memory leak in the log4rs/log4rs-syslog pair of libraries (which appears to mostly be on the syslog side). I have reported this issue upstream in im-0/log4rs-syslog#33

System Details

  • beaglebone black
  • Kubos 1.21.0

Severity

  • Medium - Irritating bug, but there's a workaround

To Reproduce
see instructions in upstream issue

Bug Output
this bug is present if a service uses more and more memory as it runs as shown by the 6th column of the ps aux | grep <service-name> command. Valgrind also shows memory leaks (see upstream issue for some output)

Workaround
The workaround is to remove the line Logger::init("service-name"); from the service. This will likely break logging. Another option depending on how fast the memory leaks is to put logging behind a conditional compilation guard to exclude it from specific builds when logging is not needed for debugging or memory use is important.

@MoralCode MoralCode added the bug Something isn't working label Mar 29, 2022
@patrickoppel
Copy link

I can confirm this issue also appeared on the ISIS iOBC

@patrickoppel
Copy link

Just tested your minimal code from im-0/log4rs-syslog#33.

I get a very similar error when I set let stdout-log = true and let syslog = false
mem_leak_stdout.txt

@MoralCode
Copy link
Member Author

I saw that too when i was testing this. From what i can tell the stdout appender appears to be fine. That said, i was seeing some weird results when playing with that other variable, so im not really sure whats going on there

@patrickoppel
Copy link

Any updates on this?

@MoralCode
Copy link
Member Author

no updates as far as I'm aware. I haven't had much chance to look into this further and probably won't for the forseeable future.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something isn't working
Projects
None yet
Development

No branches or pull requests

2 participants