Geolocation Allowed Cyber Protection

[philipianpearce]

What is needed to make this work is a method to test the destination IP after DNS look-up. As the DNS look-up is normally done after the request checks, I may need to add an extra storyboard 'entry hook' to the e2g code as well as adding a ipdestinationin state to the storyboarding. (This may also be useful for deployments where a restricted DNS is used as it should then be possible to replace any RDNS block page with an e2g one).

I will look at this once v5.5.3 is settled in the first v5.6.

Follow on from issue #733

[philipianpearce]

Feature added to v5.6.dev

Please test and comment

[Dalacor]

Thank you for doing this. I have only just become aware of this update as I have not updated E2guardian for a while. I will test this out on my system when I have some time. I have found that using Geolocation in iptables firewall itself, has been quite effective as 99% of the websites we use are all in the same 10 odd countries. The only issue with cloudflare websites. So this update should fix that and it is far better to restrict the countries in E2guardian and exempt specific websites. I will try and test this out in March.

[Dalacor]

I have installed E2guardian 5.6 pre release version. First Bugs:

e2guardian.conf file

#preauthstoryboard = '/usr/local/e2guardian/etc/e2guardian/preauth.story' - need to remove rem as causes error when E2guardian starts because of listdir now being set to enabled.

sitelist = 'name=allowedtld,messageno=752,path=LISTDIR/allowedtldlist' is missing the messageno=752

common.story file

if(responsecodein,nolog) return setnolog - need to rem out as causes error when E2guardian starts. Not sure if I need this for Geolocation to work properly?

Feedback on blanket block

I have not been able to test Geolocation because I discovered something that is actually a design flaw going back to the days of Dansguardian. Dansguardian was inherently designed around a concept of blocking things explicitly, whereas the concept of Allow Geolocation, Allow TLD, allowed extensions and allowed mimetypes work on the concept of everything is blocked by default and only a small subset are allowed.

So the E2guardian logs only show TLD, mimetypes, extensions and presumably geolocations information when they are actually blocked. Which means it is impossible to run E2guardian for say 6 months and a have complete record of all tld, mimetypes, extensions and geolocations that are actually used in web traffic. I had a lot of problems using allowed extensions and allowed mimetypes for this precise reason I now realise. Because I had no idea what extensions and mimetypes are actually being used. I ended up having to turn it off in the end because I was constantly adding more and more mimetypes and extensions to the allowed list.

Would it be possible to update the log format so it is possible to see what countries, extensions and mimetypes are actually being used for web connections. For consistency, it would probably be advisable to include a TLD field as well, although it's actually very easy to create an allowed list of tld as 99% of websites use .com, org, .net, country specific tld and maybe .io. Beyond that, virtually no legitimate websites uses anything else. So a tld field is not strictly speaking necessary. I see it is possible to configure the log file to show the fields that you actually want to see, so this may be a better option in that it can be added for users who want to see this information or even when testing to see what mimetypes etc are actually being used. Then once blanket mode is enabed, one could in theory then remove those fields. I saw a config file somewhere yesterday when I was looking at the issue.

While I managed to narrow the number of countries being used (via iptables), I don't actually know how many countries are actually needed in terms of 99% of websites. The cloudflare factor is the reason why I have about a dozen countries required instead of possibly 5.

So I would consider it essential to record in the logs what countries web connections are being made to. As far as mimetypes and extensions are concerned, I am wondering whether I should be controlling this at the firewall or just relying on something like Applocker on Windows to prevent users and malware from running not approved software. There are so many mimetypes being used by websites nowadays just to visit the website, that I am not sure if this is the best approach. In addition, a lot of websites don't even register what mimetype they are using when it is blocked.

TLD and Geolocation is definitely very effective for very little work at protecting computers from Cyber Breaches. So they are definitely worth creating fields in the log files. Extensions and Mimetypes - I am not so sure about at this point. Applocker may be a simpler and more effective approach.

[philipianpearce]

@Dalacor Thanks for the feedback.

1. Bugs - Can you post the errors you are getting for these?

sitelist = 'name=allowedtld,messageno=752,path=LISTDIR/allowedtldlist' is missing the messageno=752 - As the block is induced when the tld is not in the list a list based messageno is not required - the messageno is defined in the storyboard.
ifnot(sitein,allowedtld,752) return setblock.

2. Logging - mime-type can be displayed in logs. However, where a site or url is explicitly or blanket blocked or the https site is an exception e2g does not have access to this information as mimetype is in the server response header. I'll look at adding extra fields to logs where possible (easy to add tld as it is derived from url) when I do more work on v5.6.
3. Will comment on your general points later.

[Dalacor]

Am I correct in saying that you are moving to a design where it is no longer necessary to have the messageno in the e2guardian file. The reason why I added it, was because on a previous version of E2guardian I was getting a message 888 translation key missing or something like that.

I am not sure about extensions and mimetypes at the firewall. I think Applocker might be a more effective solution here and is a lot simpler to setup. A big problem with the current extension blocklist is that they are about 20 years old and many of the extensions are obsolete and missing modern extensions. I would think about firewall file restrictions versus something like Applocker before spending a lot of time on extensions.

The point I was making about the logs, was not what the logs show when the website is blocked, bur rather for all connections - successful and not successful. It would be very useful to run the system for 6 months and then be able to very quickly see what top levels domains, extensions and countries are actually being used.

Bug Error Messages:

Error opening storyboard file (does it exist?): $prefix/etc/e2guardian/preauth.story - this is being caused by listdir being enabled now. The only way to fix this is to remove # from #preauthstoryboard = '/usr/local/e2guardian/etc/e2guardian/preauth.story'

When I fix the above, the next and last error is:

SB warning: Undefined list nolog used at line 326 of common.story - I have to rem this line out and then E2guardian loads.

[philipianpearce]

No the design has not changed regarding messageno. The storyboard messageno, if present. will override the list definitions. See https://github.com/e2guardian/e2guardian/blob/v5.6.dev/notes/V5_Storyboard.pdf.

Having messageno in list definitions allow different types of lists (site, url, regexp, ext, etc) to have different messages. However, where ifnot() is used or non-list conditions, if you wish to generate a message then messageno has to be defined in the storyboard command.

Will respond on other points later.