[Proposal] Switch memcached client in vmq_diversity to a modern implementation

[codeadict]

Describe your issue or feature request

vmq_diversity uses mcd which is a library not receiving updates anymore and has dependencies on a very old lager library and other Basho libraries that are unnecessary for verneMQ. A more modern client named Mero (https://github.com/AdRoll/mero) has built-in pooling, supports clustered Memcached, including the AWS ElasticCache and more importantly has zero dependencies.

While working on the OTP logger branch I found this still leaves lager around so wanted to file this ticket to address this low-hanging fruit before the logging.

Describe the solution you'd like

Use mero for the Memcached client in vmq_diversity. This will mean more settings will be added to the schema but it shouldn't be had to keep backward compatible.

Describe alternatives you've considered

No response

Environment

• VerneMQ Version:
• OS:
• Erlang/OTP version (if building from source):
• Cluster size/standalone:

Add any other context, code samples or screenshots about the issue report here.

No response

Code of Conduct

• I agree to follow the VerneMQ's Code of Conduct

[ioolkos]

@codeadict thanks for the suggestion, looks like a "no-brainer" to me. Even when adding more settings etc.
+1 on this.

---

👉 Thank you for supporting VerneMQ: https://github.com/sponsors/vernemq
👉 Using the binary VerneMQ packages commercially (.deb/.rpm/Docker) requires a paid subscription.

[ioolkos]

By the way, let me know, if you feel we have better options for the other DB drivers in vmq_diversity too.

---

👉 Thank you for supporting VerneMQ: https://github.com/sponsors/vernemq
👉 Using the binary VerneMQ packages commercially (.deb/.rpm/Docker) requires a paid subscription.

[codeadict]

Sure thing, the MongoDB one we probably need to find another driver since the current one doesn't support newer MongoDB 6.x+

[mths1]

@ioolkos , @codeadict : Any news on that one? I'd also like to remove lager and this is one of the dependencies. As an intermediate solution we could also fork mcd (not a big deal, as it doesn't receive any updates anyway), remove lager and the cluster code (which Verne does not seem to use) and keep our own for till we move away from mcd.

[ioolkos]

Ah, I wanted to check out Mero back then. In any case, we have a maintained lib with it, so it should be the way to go. OTOH... in terms of effort, if there's a huge difference, I'm fine cloning MCD too.

---

👉 Thank you for supporting VerneMQ: https://github.com/sponsors/vernemq
👉 Using the binary VerneMQ packages commercially (.deb/.rpm/Docker) requires a paid subscription.

[ioolkos]

@mths1 there's even a fake_lager project https://github.com/g-andrade/fake_lager to ease transition to error logger :)

---

👉 Thank you for supporting VerneMQ: https://github.com/sponsors/vernemq
👉 Using the binary VerneMQ packages commercially (.deb/.rpm/Docker) requires a paid subscription.

[codeadict]

I have a branch somewhere for the logger transition

…

On Tue, Dec 5, 2023, 4:23 AM ioolkos ***@***.***> wrote: @mths1 <https://github.com/mths1> there's even a fake_lager project https://github.com/g-andrade/fake_lager to ease transition to error logger :) ------------------------------ 👉 Thank you for supporting VerneMQ: https://github.com/sponsors/vernemq 👉 Using the binary VerneMQ packages commercially (.deb/.rpm/Docker) requires a paid subscription. — Reply to this email directly, view it on GitHub <#2106 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AABJD6G2ET64HYPEN4K5PK3YH3RXNAVCNFSM6AAAAAAVPAK6MCVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMYTQNBQGM3DAOJYGU> . You are receiving this because you were mentioned.Message ID: ***@***.***>

[mths1]

Hi @codeadict : Meanwhile I got verne to compile with logger and removed all dependencies to lager. The only thing that is missing right now is configuration. Did you work on a nice way to keep the current config options log.xxx?

Additional question, do we want the log to look exactly like lagger output, or is is it fine to go with what logger produces (didn't compare too much yet).

[ioolkos]

From talking to use cases, there are certainly folks scanning logs for events and also frequency of events.
I don't think we need to keep the lager output format literally, though.
Here's some info how Rabbit switched to logger: https://github.com/rabbitmq/rabbitmq-server/blob/e06a8b49103483139eb72e209a37cd00c80e2fff/release-notes/3.9.0.md?plain=1#L35

JSON format might be interesting, and switching log levels dynamically with vmq-admin.