Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Akumuli crashing on query #362

Open
sptrakesh opened this issue Jun 5, 2020 · 2 comments
Open

Akumuli crashing on query #362

sptrakesh opened this issue Jun 5, 2020 · 2 comments

Comments

@sptrakesh
Copy link

I am running via the latest Docker image (akumuli:0.8.80-skylake). The following query via the Grafana plugin causes the container to crash.

spt_akumuli.1.sullcoizlnqg@docker-desktop    | 2020-06-05 23:02:06,816 http [INFO] Cursor 140025295594008 created
spt_akumuli.1.sullcoizlnqg@docker-desktop    | 2020-06-05 23:02:06,816 http [INFO] Cursor 140025295594008 started
spt_akumuli.1.sullcoizlnqg@docker-desktop    | 2020-06-05 23:02:06,817 Main [INFO] Parsing query:
spt_akumuli.1.sullcoizlnqg@docker-desktop    | 2020-06-05 23:02:06,818 Main [INFO] {
spt_akumuli.1.sullcoizlnqg@docker-desktop    |     "select": "request.counter",
spt_akumuli.1.sullcoizlnqg@docker-desktop    |     "range": {
spt_akumuli.1.sullcoizlnqg@docker-desktop    |         "from": "20200605T230206.779",
spt_akumuli.1.sullcoizlnqg@docker-desktop    |         "to": "20200605T200206.779"
spt_akumuli.1.sullcoizlnqg@docker-desktop    |     },
spt_akumuli.1.sullcoizlnqg@docker-desktop    |     "where": "",
spt_akumuli.1.sullcoizlnqg@docker-desktop    |     "order-by": "series",
spt_akumuli.1.sullcoizlnqg@docker-desktop    |     "apply": "",
spt_akumuli.1.sullcoizlnqg@docker-desktop    |     "limit": "1000000",
spt_akumuli.1.sullcoizlnqg@docker-desktop    |     "pivot-by-tag": [
spt_akumuli.1.sullcoizlnqg@docker-desktop    |         "resource"
spt_akumuli.1.sullcoizlnqg@docker-desktop    |     ]
spt_akumuli.1.sullcoizlnqg@docker-desktop    | }
spt_akumuli.1.sullcoizlnqg@docker-desktop    | 
spt_akumuli.1.sullcoizlnqg@docker-desktop    | 2020-06-05 23:02:06,819 Main [ERROR] MergeIterator - broken invariant
spt_akumuli.1.sullcoizlnqg@docker-desktop    | 2020-06-05 23:02:06,820 Main [ERROR] Terminating (core dumped)

Akumuli logs also show this error:

2020-06-05 23:09:30,277 Main [INFO] 18446744073709550400 Trying to open tree, repair status - OK, addr: 20537
2020-06-05 23:09:30,278 Main [ERROR] Empty rescue points list found, leaf-node data was lost
2020-06-05 23:09:30,278 Main [ERROR] Repair needed, id=1145
@Lazin
Copy link
Member

Lazin commented Jun 8, 2020

How your tags look like? Is 'resource' the only tag in the series name?

@sptrakesh
Copy link
Author

No, there are additional tags such as method, resource, ipaddress, status and mimeType.

Sorry, an OS update wiped out the old logs. I was able to trace the issue back further to what looked like a database corruption (probably caused when writing the series). The query causing the crash seemed to be an after effect of that corruption.

Will create another issue if I run into it again. Until then probably only option is close this issue.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants