Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Memory leak (possible goroutine leak) in throttledQuery #229

Open
carbocation opened this issue Nov 30, 2017 · 1 comment
Open

Memory leak (possible goroutine leak) in throttledQuery #229

carbocation opened this issue Nov 30, 2017 · 1 comment

Comments

@carbocation
Copy link
Contributor

I run a long-lived binary that queries multiple social media providers. I use anaconda to query Twitter. It makes hundreds of queries per minute, and over the course of days/weeks, it ends up using all available RAM on the system.

Profiling, it seems that throttledQuery itself is the source of the leaks. Here is the inuse_space graph from the profile (note: it's an SVG and can be clicked on / zoomed in):
profiled image

As you can see, I terminated the program when 300MB had leaked (the rest of the program uses a few dozen MB total). I'm guessing that this is a goroutine leak, but I'm still working on profiling that.

@carbocation
Copy link
Contributor Author

Memory leaks are anticipated in the docs ( https://github.com/ChimeraCoder/anaconda#query-queue-persistence ) but it seems that rather than coding around them, it might be possible to address the leaks by restructuring the function itself.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant