Scaling large MUCs

[kawaii44]

Hi. I have a large persistent MUC with about 30K online users joined. With so many users, it's really slow to join the MUC, get message history, send messages, etc. There is still plenty of CPU and memory available on the cluster of ejabberd machines. Strangely, the MUC seems to get faster over time. It's like there is cache that has to first warm up. Also, I've confirmed there are no long-running SQL statements. I'm running Postgres as the backend database for ejabberd and Reddis for the cache. Is there anything I can do to tune ejabberd for large MUCs? Any ideas?

BTW: the slowness seems to occur only as the number of people joining grows. Once the total participants stabilizes, then it gets way faster.

[badlop]

I have a large persistent MUC with about 30K online users joined.

I guess you refer to a single MUC room, not to the whole MUC service, as described in https://xmpp.org/extensions/xep-0045.html#terms-rooms

There is still plenty of CPU and memory available on the cluster of ejabberd machines.

There may be resources on other ejabberd nodes, but all your 30k clients are connecting to the same MUC room, to the same ejabberd node, right? Maybe you can split the usage of that single room into separated rooms (by spoken language, country, topic, whatever) and let each room be located in different nodes.

On the other hand, I imagine in production you don't plan to put 30.000 human beings chatting in the same room, right? That would be total madness. And their individual presence (join/leave/i'm away/...) is not important information for the other 19.999 human beings in that room.

So, if you are interested in messages, not in presence, check MUC/Sub: clients subscribe to the room instead of join the room. They send and receive messages, without the presence traffic.

Check also XEP-0060: Publish-Subscribe which starts saying:

This specification defines an XMPP protocol extension for generic publish-subscribe functionality. The protocol enables XMPP entities to create nodes (topics) at a pubsub service and publish information at those nodes; an event notification (with or without payload) is then broadcasted to all entities that have subscribed to the node.

On the other hand, you can take a look at https://docs.ejabberd.im/admin/configuration/basic/#shapers

[kawaii44]

Thanks @badlop. You understood my issue correctly and your advice is perfect. You are right that I definitely don't want each user to receive presence notifications for the other 29.999 people in the MUC. I hadn't thought about that! :). My XMPP client is just ignoring those presence notifications, so I didn't notice the storm of messages. No doubt that is why it's slow only when new people are joining the MUC. I'll try out MUC/Sub instead as you suggest.