I've got some debug logs I can provide on request.

@hashworks if it still an issue, debug logs could be handy. It's probably the ZNC bouncer that maybe replays messages?

I've sent you debug logs from 2021-08-21 by mail. I've also re-activated the debug logs on my end, but I don't want to run it like that forever – the bug occurs once every few weeks and -debug generates a lot of output in that time.

Well, according to the debug log it's receiving those duplicate messages from the irc server, so I think its your ZNC that's causing it. Matterbridge itself doesn't cache those messages so it can't generate duplicate messages itself.

Is there an option to do this without ZNC, just let matterbridge connect directly to the IRC server?

I have been seeing the same issue with a matterbridge bot that I run for Wikimedia projects. Today I spent some quality time with the logs from that bot and I think I have figured out why the duplication happens.

The basic issue is that when ZNC gets disconnected from an upstream IRC network matterbridge does not get disconnected from the ZNC instance. Later when ZNC is again connected to the upstream IRC network the bot sees a set of messages from the IRC server (proxied by ZNC) including the events which tell matterbridge that a new IRC session has started. When handleNewConnection is called to deal with the connection event matterbridge ends up installing a duplicate set of event handlers on the same connection to the intermediate ZNC instance which then leads to the bot seeing all IRC traffic as duplicated. There are a few more details and logs to back this up at https://phabricator.wikimedia.org/T305487#7833726

Is there an option to do this without ZNC, just let matterbridge connect directly to the IRC server?

In the case of my bot, not really? I added the ZNC to help with numerous issues of the bot's IRC client failing to reconnect cleanly and entering crash loops which triggered bans from libera.chat. See https://phabricator.wikimedia.org/T264212 and https://phabricator.wikimedia.org/T292640 for additional context if desired. All of the connection storm issues stopped immediately with the introduction of ZNC as an intermediate to the libera.chat servers.

I wonder if a fix is a simple as adding a call to i.Handlers.ClearAll() in handleNewConnection just before the existing calls to i.Handlers.AddBg and Add?

Thanks!

I wonder if a fix is a simple as adding a call to i.Handlers.ClearAll() in handleNewConnection just before the existing calls to i.Handlers.AddBg and Add?

My #1795 patch does not seem to actually fix this bug. I just deployed the v1.25.1 containing the proposed fix and was able to recreate the double IRC client event notification by issuing a /znc Jump to make my ZNC disconnect and reconnect to the upstream irc network. Following the reconnect matterbridge's debug logs are again showing two copies of each irc event.

@bd808 your patch has been reverted because of #1815

This bug makes the bridge virtually unusable for me at the moment. Room users (IRC - Matrix in my case) aren't happy to have duplicate messages until I wake up and fix it by restarting, so it's a serious issue for me.

Is there any news since the last patch getting reverted? It's been a while.

In addition, should this bug really be marked as "closed", given the patch that originally closed it has been reverted?

I use Matterbridge with ZNC. I haven't seen duplicate messages yet. Calling i.Handlers.ClearAll() was wrong. I've kept the following code in bridge/irc/handlers.go, but I don't know whether that change removes duplicate messages or whether my setup is simply different.

diff --git a/bridge/irc/handlers.go b/bridge/irc/handlers.go
index 74db7685..9080a6f7 100644
--- a/bridge/irc/handlers.go
+++ b/bridge/irc/handlers.go
@@ -122,6 +122,15 @@ func (b *Birc) handleNewConnection(client *girc.Client, event girc.Event) {
        i := b.i
        b.Nick = event.Params[0]

+       i.Handlers.Clear("PRIVMSG")
+       i.Handlers.Clear("CTCP_ACTION")
+       i.Handlers.Clear(girc.RPL_TOPICWHOTIME)
+       i.Handlers.Clear(girc.NOTICE)
+       i.Handlers.Clear("JOIN")
+       i.Handlers.Clear("PART")
+       i.Handlers.Clear("QUIT")
+       i.Handlers.Clear("KICK")
+       i.Handlers.Clear("INVITE")
        i.Handlers.AddBg("PRIVMSG", b.handlePrivMsg)
        i.Handlers.AddBg("CTCP_ACTION", b.handlePrivMsg)
        i.Handlers.Add(girc.RPL_TOPICWHOTIME, b.handleTopicWhoTime)

My ZNC configuration mostly uses the defaults. I haven't changed any buffer sizes. My networks load nothing but an authentication module and stickychan.

<Network libera>
    Encoding = ^UTF-8
    FloodBurst = 9
    FloodRate = 2.00
    IRCConnectEnabled = true
    JoinDelay = 0
    LoadModule = sasl
    LoadModule = stickychan
    Server = irc.libera.chat +6697
    TrustAllCerts = false
    TrustPKI = true
</Network>

The corresponding Matterbridge configuration:

[irc.libera]
Nick="example"
Password="example/libera:secret"
Server="127.0.0.1:6667"
Charset="utf-8"
NoSendJoinPart=true
StripMarkdown=true

Just FYI, I'm still getting duplicate messages. But no time how to debug this ATM.

I use Matterbridge with ZNC. I haven't seen duplicate messages yet. Calling i.Handlers.ClearAll() was wrong. I've kept the following code in bridge/irc/handlers.go, but I don't know whether that change removes duplicate messages or whether my setup is simply different.

diff --git a/bridge/irc/handlers.go b/bridge/irc/handlers.go
index 74db7685..9080a6f7 100644
--- a/bridge/irc/handlers.go
+++ b/bridge/irc/handlers.go
@@ -122,6 +122,15 @@ func (b *Birc) handleNewConnection(client *girc.Client, event girc.Event) {
        i := b.i
        b.Nick = event.Params[0]

+       i.Handlers.Clear("PRIVMSG")
+       i.Handlers.Clear("CTCP_ACTION")
+       i.Handlers.Clear(girc.RPL_TOPICWHOTIME)
+       i.Handlers.Clear(girc.NOTICE)
+       i.Handlers.Clear("JOIN")
+       i.Handlers.Clear("PART")
+       i.Handlers.Clear("QUIT")
+       i.Handlers.Clear("KICK")
+       i.Handlers.Clear("INVITE")
        i.Handlers.AddBg("PRIVMSG", b.handlePrivMsg)
        i.Handlers.AddBg("CTCP_ACTION", b.handlePrivMsg)
        i.Handlers.Add(girc.RPL_TOPICWHOTIME, b.handleTopicWhoTime)

Your fix seems to work for me as well @voegelas. You should submit it as a PR. :)