Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Improve IP Geolocation #1922

Open
BelleNottelling opened this issue Dec 5, 2023 · 1 comment · May be fixed by #1929
Open

Improve IP Geolocation #1922

BelleNottelling opened this issue Dec 5, 2023 · 1 comment · May be fixed by #1929
Assignees
Labels
enhancement New feature or request

Comments

@BelleNottelling
Copy link
Member

Presently the implementation for IP geolocation in FOSSBilling is a tad troublesome for a few reasons.

Source

FOSSBilling's IP geolocation database is from MaxMind which presents us with a few (smaller) issues:

  1. We can't easily automatically update it. Our version is actually nearly a full year out of date at this point. MaxMind does give the functionality to update it programmatically, but we need to provide credentials to do so which makes it only properly viable through the release process. Better than nothing, but still imperfect, especially if we go a long time between releases.
  2. The licensing for it requires attribution which we currently aren't doing right. There is only a note inside of our readme, however there should instead be attribution given on each page the data is used. Officially, this is only inside a few pages in the admin panel, but as there's a twig filter for it, that could easily be added anywhere else. (Relates to Legal review prior to release of v1.0 #1678)

Functionality

  • On top of the aforementioned issues, we also face the issue of not providing any way to provide an alternative source, leaving people to have to replace the file that's shipped with FOSSBilling. Their version will then get replaces with each FOSSBilling update.
  • We only use it for visual purposes in the admin panel, but this data could be leveraged a bit better such as with the fingerprint system to detect if a user's country based on IP address suddenly changes, which would be a strong indication of their session being hijacked.
    • We actually do use the cloudflare provided country, but that's only useful for people who are using cloudflare and have that header enabled.
  • Can't update without needing to update the entire app.

Fix

I will work to address the source issue and the more important missing functionality for 0.6.1.

Finding a better source

For a new source that'll be a better fit to ship out-of-the-box, this GitHub repository automatically syncs with various IP geolocation DBs, 3 of which are under the CC0 1.0 license meaning we wouldn't need to even consider if attribution is given correctly and can simply use the data within FOSSBilling.

This repository is a GO project which can be used to build a .mmdb database off of those databases which is fantastic as that means we already include the needed PHP dependency to read this DB and it also means we'd be sticking to one of the most common formats that commercial databases come in.

Updating

Combining those two and some fairly simple automation would give us a stress free database that's automatically updated every 24 hours and that could even be updated directly within the app. That means no longer having a DB for it in the source code and we wouldn't even need to bundle it within a release. FOSSBilling itself should be able to automatically fetch and update the DB using something like a scheduled task.

Custom Databases

Since we'd be sticking with the tried-and-true .mmdb format, all we need to do to give people the ability to use their own database is just to give a path configuration where they can point FOSSBilling to a database of their choosing on the disk. All IP geolocation DB providers I've seen give this as a format option so that generally means for most people they don't need to worry about format compatibility and as such, neither do we.

Although this option does mean that FOSSBilling itself doesn't provide functionality to automatically fetch this data from anything except our out-of-the-box source, it should be pretty easy for someone to make a custom module that does handle the process of interacting with whatever commercial source, downloading the DB, and then storing it in the correct location.

Functionality

I'll work to implement a new class / expand the system module which should make easier for us to expand the functionality of this later down the line in a way that's easy for ourselves and developers to use & will likely add some smaller items while I'm in there.

@BelleNottelling BelleNottelling added the enhancement New feature or request label Dec 5, 2023
@BelleNottelling BelleNottelling self-assigned this Dec 5, 2023
@BelleNottelling
Copy link
Member Author

I've spent some time on this today specifically focusing on the automation part and have created the following repository:
https://github.com/HostByBelle/IP-Geolocation-DB

This will once per day pull in the CC0 and PDDL licensed databases (both are public domain) from ip-location-db, compile various .mmdb databases, and then publish those to a new release. There are 5 total pairs of DBs, each one with both IPv4 and IPv6 available.

So we have a few different options depending on the source we trust the most and what info we want to have out of the box, but regardless of which we choose from that they all will be public domain and very easy for us to programmatically update.

@BelleNottelling BelleNottelling linked a pull request Dec 6, 2023 that will close this issue
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement New feature or request
Projects
Status: No status
Development

Successfully merging a pull request may close this issue.

1 participant