-
Notifications
You must be signed in to change notification settings - Fork 17
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
khepri_machine: Run the query anonymous function from the calling process #226
base: main
Are you sure you want to change the base?
Conversation
Codecov ReportAttention:
Additional details and impacted files@@ Coverage Diff @@
## main #226 +/- ##
==========================================
+ Coverage 88.89% 88.90% +0.01%
==========================================
Files 20 20
Lines 2890 2902 +12
==========================================
+ Hits 2569 2580 +11
- Misses 321 322 +1
Flags with carried forward coverage won't be shown. Click here to find out more.
☔ View full report in Codecov by Sentry. |
…cess [Why] Before this change, the anonymous function passed to `khepri_machine:fold/5` was executed by the Ra server process. This had two downsides: * it blocked the Ra server to serve writes * the anonymous function couldn't query the Ra server itself because it would deadlock [How] The alternative implemented in this patch is to get the whole Khepri tree from the Ra server, then run the anonymous function, this time from the calling process. The downside is that the whole Khepri tree is copied to that calling process.
3d67a60
to
cb52d57
Compare
I'm still not sure this alternative approach is what we want. Copying the entire Khepri tree to the calling process could be very expensive and the memory footprint could sky-rocket. Another solution would be to try to run the anonymous function from the Ra process like before. But if it exits with a Or simply, we accept a What do you think, @dcorbacho and @the-mikedavis? To give some context to this issue, when RabbitMQ queries transient queues from the database, it queries the Khepri members from a Khepri query anonymous function: khepri:fold(
StoreId,
PathPattern,
fun(QueuePath, NodeProps) ->
...
%% This queries the Ra server and can't be executed by the Ra server
%% itself because it would deadlock. gen_statem throws a `calling_self`
%% exception.
Members = khepri:members(StoreId),
...
end,
[]). |
I pushed an additional commit to implement that option that lets the caller decide if it wants the tree to be copied to be able to do nested queries. I would love some feedback on the approach and the option name :-) |
a862938
to
ac306e2
Compare
We ended up going a different direction on this in the server similar to the-mikedavis/rabbitmq-server@aed8ddd. We use @dumbbell should we close out this PR for now? We could re-open it if we find another use-case and want to make it possible to do while still calling |
Why
Before this change, the anonymous function passed to
khepri_machine:fold/5
was executed by the Ra server process. This had two downsides:How
The alternative implemented in this patch is to get the whole Khepri tree from the Ra server, then run the anonymous function, this time from the calling process.
The downside is that the whole Khepri tree is copied to that calling process.