-
Notifications
You must be signed in to change notification settings - Fork 255
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Cache errors with queries with large domain lists in net_dns_record
#4116
Comments
This issue is stale because it has been open 60 days with no activity. Remove stale label or comment or this will be closed in 30 days. |
Hey @tinder-tder, my apologies for the late response. Could you please try the above using the latest version(0.21.3) of Steampipe? |
@bigdatasourav it looks like those warnings no longer showup with the latest steampipe version and latest plugin version (0.12) but now the query doesnt complete.
|
@tinder-tder, When you said that the |
@bigdatasourav no errors it just hangs, i have tried to change the query around and it looks like it just takes longer now. Connecting in service mode via redash it used to finish but now its timing out (client connection dies and the transaction gets rolled back). I can connect locally and run the same query via psql and it will eventually finish. |
@kaidaguerre, as discussed on the above Slack thread, could you please look into it when you get a chance? |
Hi @tinder-tder are you still seeing this issue? |
@kaidaguerre Yes, we are basically splitting up queries now into smaller chunks to get around it for now. The cache erros are not showing up in the logs. we can close this if there isnt anything we can do |
Describe the bug
Caching errors/corruption(?) causing plugin to exit or not complete queries. It seems to happen only when doing a large amount of queries for a domain list > 2k. I have attempted to throttle the plugin with the limiter block config but it doesn't not fix it.
Error running query: rpc error: code = Unknown desc = IterateSet called for callId net-1699322677353 but there is no in-progress ‘set’ operation
which is also logged in the plugin log)select from steampipe_internal.meta_cache(‘off’);
before running the query prevents errors/warnings in the plugin and seems to run faster when it does complete (works in redash)sample of logs when caching is ON:
and
Steampipe version (
steampipe -v
)v0.21.1
Plugin version (
steampipe plugin list
)0.11.1
To reproduce
create a large list of domains to scan and run a query against the
net_dns_record
table with all domains in the listExpected behavior
query to complete
Additional context
slack thread
The text was updated successfully, but these errors were encountered: