Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Investigate ways to make this plugin compatible with Khepri-enabled clusters #272

Open
michaelklishin opened this issue Mar 13, 2024 · 0 comments
Assignees

Comments

@michaelklishin
Copy link
Member

michaelklishin commented Mar 13, 2024

Problem Definition

This plugin currently does not run on Khepri-enabled nodes (in Khepri-enabled clusters, that is, since Khepri cannot be enabled selectively).

And it won't be just a matter of adapting the code paths to use either of the data stores, like it is the case in most code paths in RabbitMQ and its plugins:

  1. Khepri has a documented limitation where it stores the entire data set both on disk and in memory
  2. This plugin stores the entire message content in the table, therefore, it explicitly goes against the recommendation for the current version of Khepri

Possible Solutions

Given that #229 will take a long time to finish regardless of the details, perhaps this plugin should be reworked to not store the payload in Mnesia.

DETS is one readily available option with the known 4 GiB limit per file.

The plugin can also be adapted to start and use a single local copy of Mnesia in ways that do not interfere with Khepri.

Introducing a dependency on a pure Erlang K/V store is also possible but there aren't many actively maintained ones that do not pull in RocksDB, LevelDB, and other Windows-incompatible or Windows-is-an-afterthought options.

@lukebakken lukebakken self-assigned this Mar 13, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants