Opened 7 years ago

Closed 7 years ago

#6318 closed task (fixed)

Compare relay IPs between maxmind db and blockfinder db

Reported by: arma Owned by: karsten
Priority: Medium Milestone:
Component: Metrics/Analysis Version:
Severity: Keywords:
Cc: nickm, ioerror, karsten Actual Points:
Parent ID: Points:
Reviewer: Sponsor:

Description

Maxmind's geoip db is rotting (#6266). I hear Jake's blockfinder produces similar results. We should check how similar they are in practice. The first question is: do the two databases agree about the IPs in the consensus?

Child Tickets

Change History (9)

comment:1 Changed 7 years ago by arma

Karsten, is this easy for you to do using the scripts you used last time?

comment:2 Changed 7 years ago by karsten

Owner: set to karsten
Status: newassigned

Not easy, but doable. I don't have these scripts with me, but I can look into this problem next week.

Jake, where would I find blockfinder's database, or how would I generate one?

comment:3 Changed 7 years ago by ioerror

git clone the blockfinder code from github - then run ./blockfinder -i to set up the basic local databases and then run ./blockfinder -r $IP or something like that...

comment:4 Changed 7 years ago by ioerror

Here is the git repo and project url:

https://www.github.com/ioerror/blockfinder

comment:5 Changed 7 years ago by karsten

Hmm, blockfinder -i doesn't work for me:

Updating delegation cache...
Fetching ftp://ftp.arin.net/pub/stats/arin/delegated-arin-latest.md5
Fetching  0.0  kilobytes
[====================================================================>] 0.00 K/s
IOError(2, 'No such file or directory')
Fetching ftp://ftp.arin.net/pub/stats/arin/delegated-arin-latest
Fetching  3077.0  kilobytes
[==================================================================>] 112.47 K/s
Fetching ftp://ftp.ripe.net/ripe/stats/delegated-ripencc-latest.md5
Fetching  -1.0  kilobytes

IOError(2, 'No such file or directory')
Fetching ftp://ftp.ripe.net/ripe/stats/delegated-ripencc-latest
Fetching  -1.0  kilobytes

Fetching ftp://ftp.afrinic.net/pub/stats/afrinic/delegated-afrinic-latest.md5

After that line, it just sits there doing nothing. I aborted the script after a few minutes.

Can you generate a .csv file for me that contains IP address ranges and country codes?

comment:6 Changed 7 years ago by ioerror

Those files are certainly there - I'd grab them manually, put them on your own space, patch up blockfinder and try again.

comment:7 in reply to:  6 Changed 7 years ago by karsten

Looks like afrinic screwed up their -latest link. I replaced -latest with -20120709 and got ./blockfinder -i working. Maybe they'll fix it tomorrow.

Now, how do I export blockfinder's address ranges and country codes to a .csv file?

comment:8 Changed 7 years ago by ioerror

Perhaps send them an email?

You'll have to implement such an export - it's all packed in the CSV. However, you can also just look up every IP with maxmind, output it to a file, and do the same with blockfinder. blockfinder supports maxmind's geoip, so you can use the same tool, if you want.

comment:9 Changed 7 years ago by karsten

Resolution: fixed
Status: assignedclosed

This is now done. blockfinder can now be used to compare multiple data sources and show different types of conflicts. Closing.

Note: See TracTickets for help on using tickets.