Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Bug: memory leaking in MSGGATEWAY #2176

Open
Sylariam opened this issue Apr 10, 2024 · 2 comments
Open

Bug: memory leaking in MSGGATEWAY #2176

Sylariam opened this issue Apr 10, 2024 · 2 comments
Labels
kind/bug Categorizes issue or PR as related to a bug.

Comments

@Sylariam
Copy link

Sylariam commented Apr 10, 2024

What happened?

This monday I upgraded server side to v3.6.0 using Helm Chart. For some reason MSGGATEWAY memory usage suddenly spiked, with no signs of stopping. This directly led to our IM service becoming unavailable, which coincided with our peak business hours.
The quantity of error logs also increased during that period, including entries such as "read tcp :80->:5644: i/o timeout" and "websocket: close 1006 (abnormal closure): unexpected EOF".

MSGGATEWAY memory increased all the way to 11GiB. No stopping before our manual intervention.
img_v3_029q_7c008829-eb5a-43d8-8146-90a9fc4b128g

OpenIM's official Grafana dashboard stats:
img_v3_029q_4972ff6b-d852-47dc-b3c6-edb8a0d6c76g

Online users:
image

What did you expect to happen?

no memory leaking plz

How can we reproduce it (as minimally and precisely as possible)?

Helm charts deploy
v3.6.0

Anything else we need to know?

No response

version

```console $ {name} version # paste output here ```

Cloud provider

OS version

```console # On Linux: $ cat /etc/os-release # paste output here $ uname -a # paste output here # On Windows: C:\> wmic os get Caption, Version, BuildNumber, OSArchitecture # paste output here ```

Install tools

@Sylariam Sylariam added the kind/bug Categorizes issue or PR as related to a bug. label Apr 10, 2024
@Sylariam
Copy link
Author

Setting aside the issue at hand, the current Kubernetes deployment lacks resource limitations, which would lead to unlimited consumption of cluster resources if any similar problems arise. This needs to be addressed.

@pokid
Copy link

pokid commented Apr 15, 2024

i got this problem too.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
kind/bug Categorizes issue or PR as related to a bug.
Projects
None yet
Development

No branches or pull requests

2 participants