Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Source ASN DB missing data #12

Open
Xorlent opened this issue Jul 6, 2022 · 2 comments
Open

Source ASN DB missing data #12

Xorlent opened this issue Jul 6, 2022 · 2 comments

Comments

@Xorlent
Copy link

Xorlent commented Jul 6, 2022

Doing QC on the source TSV file, I found it's missing quite a few IP ranges. I recommend utilizing a different source. The ip2location lite ASN DB appears to be complete, updated monthly:
https://lite.ip2location.com/database-asn

Here is a modified version of the shell script if you decide to adopt it. One benefit is there would no longer be a need for ipcalc. The download would be similar (sign-up and token required), using unzip instead of gzip. There are licensing restrictions/attribution to consider if deciding to use ip2location.

Process ASN.txt and output asn-preprocess.txt (IPv4 only)

cat ~/ASN.txt | grep -v '^#' | awk '{print $1}' | grep '^AS' | while read asn; do
echo "Processing $asn"
awk -F "","" '{if($4 == '${asn:2}') print $3}' ~/IP2LOCATION-LITE-ASN.CSV >> ~/asn-preprocess.txt
done

Make sure all addresses are valid and properly formatted, outputting ipv4.txt

perl cleanup.pl ~/asn-preprocess.txt | grep -E '/(2[0-4]|1[0-9]|[0-9])$' > ~/ipv4.txt

@splitice
Copy link
Member

splitice commented Jul 8, 2022

Please submit a PR with your proposed automation changes

@splitice
Copy link
Member

I don't see any licence issues as long as we don't plan on redistributing the licence. There would however need to be a PR supporting this and a comparison.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants