Text is appended to overrides/rspamd/milter_headers.conf when mounted in antispam container

[unikitty37]

Environment & Version

Environment

• docker compose
• kubernetes
• docker swarm

Version

• Version: 2.0

Description

I have an override file for rspamd's milter_headers.conf in /mailu/overrides/rspamd/milter_headers.conf.

On upgrading to version 2.0, rspamd no longer processes this. Examining etc/rspamd/local.d/milter_headers.conf in the antispam container shows this:

authenticated_headers = ["authentication-results"];
skip_local = false;
skip_authenticated = false;
use = ["x-spamd-bar", "x-spam-level", "x-virus", "authentication-results"];
routines {
  authentication-results {
    add_smtp_user = false;
  }
  x-virus {
    symbols = ["CLAM_VIRUS", "FPROT_VIRUS", "JUST_EICAR"];
  }
}
.include(try=true,priority=1,duplicate=merge) "/overrides/milter_headers.conf"

I'm not sure where the bit at the end comes from, but rspamd isn't expecting it do be there; it isn't added in version 1.9.

The upgrade documentation mentions that the override is mounted in a new location, but that doesn't seem to affect this.

This causes the x-spam-level header to not be added to incoming messages; as I'm using a Sieve rule rather than the built-in spam filtering to move spam (to work around #1552) this causes all spam to be unceremoniously dumped in my inbox. I've had to revert to 1.9 in order to keep things usable.

Replication Steps

1. Add the following to /mailu/overrides/rspamd/milter_headers.conf:

authenticated_headers = ["authentication-results", "x-spam-status", "x-spamd-bar", "x-spam-level"];
local_headers = ["authentication-results", "x-spam-status", "x-spamd-bar", "x-spam-level"];
skip_local = false;
skip_authenticated = false;

use = ["x-spamd-bar", "x-spam-level", "x-virus", "authentication-results", "x-spam-status"];

routines {
  x-virus {
    symbols = ["CLAM_VIRUS", "FPROT_VIRUS", "JUST_EICAR"];
  }
}

2. Start mailu.
3. Send a non-spammy email to yourself.

Observed behaviour

The x-spam-status header is not added to the received message.

In the antispam container, the line .include(try=true,priority=1,duplicate=merge) "/overrides/milter_headers.conf" has been added to the end of /etc/rspamd/local.d/milter_headers.conf.

Expected behaviour

The x-spam-status header is added to the received message.

In the antispam container, /etc/rspamd/local.d/milter_headers.conf contains exactly what was added in step 1.

Logs

antispam_1  | 2023-05-31 18:02:07 #1(main) <d55yww>; lua; milter_headers.lua:620: invalid config for milter_headers: field "skip_authenticated": expected type "boolean", got "table"

[Diman0]

Since Mailu 2.0, the config in the overrides folder merges with existing config.
The files in /mailu/overrides/rspamd are placed in /overrides in the antispam container. This config file is included via the include statement .include(try=true,priority=1,duplicate=merge) "/overrides/milter_headers.conf"

This is not going to change. Overrides should not completely override the mailu config. This can result in Mailu config in rspamd to stop working.

If you want the old behaviour, you can add the following host mount in docker-compose.yml:

  antispam:
...
    volumes:
      - "/mailu/filter:/var/lib/rspamd"
      - "/mailu/overrides/rspamd:/overrides:ro"
      - "/mailu/overrides/rspamd_override.d:/etc/rspamd/override.d

and then place your override file in:
/mailu/overrides/rspamd_override.d/

[Diman0]

Hmm. Testing with your config in the rspamd override.d folder, I only have these x-spam* headers:

X-Spamd-Bar: -
X-Spam-Status: No, score=-1.00 

This might be a bug in rspamd. You could consider creating an issue in the upstream project: https://github.com/rspamd/rspamd/issues

[unikitty37]

Thanks — mailu's default config appears to be doing exactly the same as my override file anyway, so it should be displaying the X-Spam-Status header without my override, but it doesn't.

If I remove my override file, and just use mailu's config, the X-Spam-Status header still doesn't display, and I see this error in the logs:

antispam_1  | 2023-05-31 18:02:07 #1(main) <d55yww>; lua; milter_headers.lua:620: invalid config for milter_headers: field "skip_authenticated": expected type "boolean", got "table"

Quite what's happening to make it think false is a table rather than a boolean is anybody's guess!

(As for your second comment, I'm only getting the X-Spamd-Bar and X-Spam-Status with 1.9 as well, so it's probably an error in my config rather than an rspamd bug, but the X-Spam-Status is the header my rules work from, so I've not bothered to look into that part of it any further 😁 )

[stale]

Issues not for bugs, enhancement requests or discussion go stale after 21 days of inactivity. This issue will be automatically closed after 14 days.
For all metrics refer to the stale.yml file.
Github issues are not meant for user support. For user-support questions, reach out on the matrix support channel.

Mark the issue as fresh by simply adding a comment to the issue.
If this issue is safe to close, please do so now.

[unikitty37]

Does anybody know why I'm getting that type error when using the default? The lack of spam headers is preventing me from upgrading past 1.9…

[stale]

Issues not for bugs, enhancement requests or discussion go stale after 21 days of inactivity. This issue will be automatically closed after 14 days.
For all metrics refer to the stale.yml file.
Github issues are not meant for user support. For user-support questions, reach out on the matrix support channel.

Mark the issue as fresh by simply adding a comment to the issue.
If this issue is safe to close, please do so now.

[ouafnico]

I got exactly the same problem.
Was looking why rspamd is not putting x-spam headers.

Got the error 'invalid config for milter_headers: field "skip_authenticated": expected type "boolean", got "table"' too.
I'm trying to comment the variable in milter_headers.conf to see.

[stale]

Issues not for bugs, enhancement requests or discussion go stale after 21 days of inactivity. This issue will be automatically closed after 14 days.
For all metrics refer to the stale.yml file.
Github issues are not meant for user support. For user-support questions, reach out on the matrix support channel.

Mark the issue as fresh by simply adding a comment to the issue.
If this issue is safe to close, please do so now.

[unikitty37]

This is still an issue for me. I'm still stuck on 1.9 as a result, and it's stopping me from recommending Mailu as a solution for other people.

[stale]

Issues not for bugs, enhancement requests or discussion go stale after 21 days of inactivity. This issue will be automatically closed after 14 days.
For all metrics refer to the stale.yml file.
Github issues are not meant for user support. For user-support questions, reach out on the matrix support channel.

Mark the issue as fresh by simply adding a comment to the issue.
If this issue is safe to close, please do so now.