Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

go.opentelemetry.io/otel may cause fatal error out of memory allocating heap arena metadata #1067

Open
LINJINTIANDE opened this issue Sep 22, 2022 · 2 comments
Labels
kind/bug Kind: Bug

Comments

@LINJINTIANDE
Copy link

LINJINTIANDE commented Sep 22, 2022

Describe the bug:

When running 1960322 highly actor tasks, the cpu is full, resulting in downtime.
Comment out the following lines of code, the task can run normally.

_, span := otel.Tracer("").Start(ctx, "RawActorExtractor.Extract")

defer span.End()

if span.IsRecording() {

span.SetAttributes(a.Attributes()...)

span.SetAttributes(a.Attributes()...)

image

Steps to Reproduce:

Running a walk job like this:
lily job run --storage=lily --tasks=actor --storage=lily walk --from=1960322 --to=1960322

Lily Version: v0.12.0

@LINJINTIANDE LINJINTIANDE added the kind/bug Kind: Bug label Sep 22, 2022
@frrist
Copy link
Member

frrist commented Sep 26, 2022

Interesting. Can you share the command used to start the daemon? Was tracing enabled?

@LINJINTIANDE
Copy link
Author

run lily's script
image

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
kind/bug Kind: Bug
Projects
None yet
Development

No branches or pull requests

2 participants