Scan file that were previously on the whitelist

[Clevero]

It would be cool if files that were (not anymore) on the whitelist and re-incoming as mail are actually scanned

Expected Behavior

1. Attachment of an incoming mail is not scanned since it's mime type is on the whitelist
2. Mimy type is removed from the whitelist
3. Exact same attachment from step one is incoming and is scanned since it's no longer on the whitelist

Current Behavior

1. Attachment of an incoming mail is not scanned since it's on the whitelist
2. Mimy type is removed from the whitelist
3. Exact same attachment from step one is incoming and is not scanned since peekaboo already has an DB entry that says "File type is not on the list of types to analyse (set(['application/hta', 'text/html']))"

Steps to Reproduce

1. Send an attachment ABC with a mime type that is on the whitelist
2. Remove that mime type from the whitelist
3. Re-Send the attachment ABC

Context (Environment)

I tried to enable scanning of .hta files and tested it with the file that was previously ignored
#89

After editing, my current ruleset.conf looks like this:

[file_type_on_whitelist]
#whitelist.1 : text/plain
whitelist.1 : message/rfc822
whitelist.2 : inode/x-empty
whitelist.3 : application/pkcs7-signature
whitelist.4 : application/x-pkcs7-signature
whitelist.5 : application/pkcs7-mime
whitelist.6 : application/x-pkcs7-mime
#whitelist.8 : text/html

[file_type_on_greylist]
greylist.1  : application/octet-stream
...
greylist.35 : application/vnd.ms-excel
greylist.36 : application/msword
greylist.37 : application/hta
greylist.38 : text/html

[Jack28]

Any change in the configuration or setup that can change the result of a scan should always result in a reset or cleanup of the database (given the "known" rule is enabled).

This is for config changes and also for example if you update the software of pdf reader there are vulnerabilites fixed and new vulnerabilities introduced. All previous results can be wrong.

It can be discussed to keep malicious results.

HTA files are in my experience never legitimate and are blocked by amavis.

The database cleanup functionality could be useful and could be included in the (future) peekaboo-util tool.

[Clevero]

I understand and agree that changing the configuration is also changing the significance of "unknown" results.

In my opinion, "bad" should be kept. I do not want to pass a malicious file when it was rated as malicious in the past even if the file cannot damage my current environment since its targeting an old CVE that was patched on our systems.

I think for "ignored" results, it would solve my use case if there would be a tool where I can easily purge those results. But in the meantime I delete those via an SQl query.

Thanks for your answer!

[Jack28]

Our draft of peekaboo-util:

peekaboo-util
--scan-file <file>
--prune-reports <seconds> - clear reports older than <seconds>
--reload - config reload
--restart - graceful restart
--shutdown - graceful shutdown
--status - report running status and service health
--stats - report some stats such as samples scanned and outcomes achieved