Opened 4 years ago

Closed 4 years ago

#14679 closed enhancement (wontfix)

geoip seems to have problem with unmapped regions

Reported by: starlight Owned by:
Priority: Medium Milestone: Tor: 0.2.7.x-final
Component: Core Tor/Tor Version:
Severity: Keywords: geoip
Cc: karsten Actual Points:
Parent ID: Points:
Reviewer: Sponsor:

Description

New RIR country mappings for China and
other countries have appeared between
two JP mappings. Have observed that
The August geoip file (one alpha TBB
version back) has this region incorrectly
tagged as JP instead of unmapped:

721420288,737479679,JP

or 43.0.0.0 thru 43.245.11.255

Below are the mappings obtained directly
from the RIR .txt files, first on 7/9/14
and next on 2/1/15 that show the
progression. The RIR entries were
gathered with FTP on the dates indicated,
concatenated and sorted by IP address
(is used for the input to a firewall
rule generation script here).

This can lead to various errors in the
Tor reporting systems. I wonder if the
error was in the MaxMind DB or is a
problem with the script. Any reason
why the RIR files should not be used
since MaxMind is for high-precision
geolocation and Tor does not appear
to need anything beyond country-level
granularity?

apnic|JP|ipv4|43.232.0.0|262144|20090723|allocated|A92D3CAE
apnic|JP|ipv4|43.244.0.0|65536|20090723|allocated|A9290F05
apnic|JP|ipv4|43.232.0.0|262144|20090723|allocated|A92D3CAE
apnic|CN|ipv4|43.236.0.0|1024|20140812|allocated|A9280685
apnic|CN|ipv4|43.236.4.0|1024|20140812|allocated|A923FF55
apnic|CN|ipv4|43.236.8.0|1024|20140812|allocated|A924A73C
apnic|CN|ipv4|43.236.12.0|1024|20140812|allocated|A924E1C6
apnic|CN|ipv4|43.236.16.0|1024|20140812|allocated|A9297CAD
apnic|CN|ipv4|43.236.20.0|1024|20140812|allocated|A92D8B9C
apnic|CN|ipv4|43.236.24.0|1024|20140812|allocated|A9285E72
apnic|CN|ipv4|43.236.28.0|1024|20140812|allocated|A92CAE4C
apnic|CN|ipv4|43.236.32.0|1024|20140812|allocated|A9235801
apnic|CN|ipv4|43.236.36.0|1024|20140812|allocated|A9231336
apnic|CN|ipv4|43.236.40.0|1024|20140812|allocated|A921CFB0
apnic|CN|ipv4|43.236.44.0|1024|20140812|allocated|A921E120
apnic|CN|ipv4|43.236.48.0|1024|20140812|allocated|A92B6C38
apnic|CN|ipv4|43.236.52.0|1024|20140812|allocated|A9214493
apnic|CN|ipv4|43.236.56.0|1024|20140812|allocated|A92A078D
apnic|CN|ipv4|43.236.60.0|1024|20140812|allocated|A9222C44
apnic|CN|ipv4|43.236.64.0|1024|20140812|allocated|A92AC4D4
apnic|CN|ipv4|43.236.68.0|1024|20140812|allocated|A9248801
apnic|CN|ipv4|43.236.72.0|1024|20140812|allocated|A92C6276
apnic|CN|ipv4|43.236.76.0|1024|20140812|allocated|A9255117
apnic|CN|ipv4|43.236.80.0|1024|20140814|allocated|A925A3C8
apnic|CN|ipv4|43.236.84.0|1024|20140814|allocated|A92929E9
apnic|CN|ipv4|43.236.88.0|1024|20140814|allocated|A92B22D6
apnic|CN|ipv4|43.236.92.0|1024|20140814|allocated|A92B8B8F
apnic|CN|ipv4|43.236.96.0|1024|20140814|allocated|A9286880
apnic|CN|ipv4|43.236.100.0|1024|20140814|allocated|A921BAD4
apnic|CN|ipv4|43.236.104.0|1024|20140814|allocated|A9234954
apnic|CN|ipv4|43.236.108.0|1024|20140814|allocated|A92492E2
apnic|CN|ipv4|43.236.112.0|1024|20140814|allocated|A9247007
apnic|CN|ipv4|43.236.116.0|1024|20140814|allocated|A92E2330
apnic|CN|ipv4|43.236.120.0|1024|20140814|allocated|A92B5A6F
apnic|CN|ipv4|43.236.124.0|1024|20140814|allocated|A929E193
apnic|CN|ipv4|43.236.128.0|1024|20140814|allocated|A9241939
apnic|CN|ipv4|43.236.132.0|1024|20140814|allocated|A92537DA
apnic|CN|ipv4|43.236.136.0|1024|20140814|allocated|A92505C1
apnic|CN|ipv4|43.236.140.0|1024|20140814|allocated|A9225E10
apnic|CN|ipv4|43.236.144.0|1024|20140815|allocated|A9250C53
apnic|CN|ipv4|43.236.148.0|1024|20140815|allocated|A926728C
apnic|CN|ipv4|43.236.152.0|1024|20140815|allocated|A92CAFF0
apnic|CN|ipv4|43.236.156.0|1024|20140815|allocated|A9226987
apnic|CN|ipv4|43.236.160.0|1024|20140815|allocated|A92420AC
apnic|CN|ipv4|43.236.164.0|1024|20140815|allocated|A9276330
apnic|CN|ipv4|43.236.168.0|1024|20140815|allocated|A92AF498
apnic|CN|ipv4|43.236.172.0|1024|20140815|allocated|A92CF084
apnic|CN|ipv4|43.236.176.0|1024|20140815|allocated|A924E88E
apnic|CN|ipv4|43.236.180.0|1024|20140815|allocated|A92FC4EB
apnic|CN|ipv4|43.236.184.0|1024|20140815|allocated|A92E533B
apnic|CN|ipv4|43.236.188.0|1024|20140815|allocated|A92B6713
apnic|CN|ipv4|43.236.192.0|1024|20140815|allocated|A9287919
apnic|CN|ipv4|43.236.196.0|1024|20140815|allocated|A92F5052
apnic|CN|ipv4|43.236.200.0|1024|20140815|allocated|A9281A7E
apnic|CN|ipv4|43.236.204.0|1024|20140815|allocated|A927E4E8
apnic|CN|ipv4|43.236.208.0|1024|20140815|allocated|A92F2F64
apnic|CN|ipv4|43.236.212.0|1024|20140815|allocated|A9261B2C
apnic|CN|ipv4|43.236.216.0|1024|20140815|allocated|A9273302
apnic|CN|ipv4|43.236.220.0|1024|20140815|allocated|A929F8EF
apnic|CN|ipv4|43.236.224.0|1024|20140815|allocated|A927837F
apnic|CN|ipv4|43.236.228.0|1024|20140815|allocated|A92C3B04
apnic|CN|ipv4|43.236.232.0|1024|20140815|allocated|A9240D41
apnic|CN|ipv4|43.236.236.0|1024|20140815|allocated|A92786CD
apnic|CN|ipv4|43.236.240.0|1024|20140815|allocated|A924E553
apnic|CN|ipv4|43.236.244.0|1024|20140815|allocated|A9272E63
apnic|CN|ipv4|43.236.248.0|1024|20140815|allocated|A921E93F
apnic|CN|ipv4|43.236.252.0|1024|20140815|allocated|A928C453
apnic|CN|ipv4|43.237.0.0|1024|20140815|allocated|A9258D34
apnic|CN|ipv4|43.237.4.0|1024|20140815|allocated|A92C1208
apnic|CN|ipv4|43.237.8.0|1024|20140815|allocated|A92C5B33
apnic|CN|ipv4|43.237.12.0|1024|20140815|allocated|A92C3D5B
apnic|CN|ipv4|43.237.16.0|1024|20140815|allocated|A921E557
apnic|CN|ipv4|43.237.20.0|1024|20140815|allocated|A9256378
apnic|CN|ipv4|43.237.24.0|1024|20140815|allocated|A92527E4
apnic|CN|ipv4|43.237.28.0|1024|20140815|allocated|A9221767
apnic|CN|ipv4|43.237.32.0|1024|20140815|allocated|A92DF90C
apnic|CN|ipv4|43.237.36.0|1024|20140815|allocated|A929451F
apnic|CN|ipv4|43.237.40.0|1024|20140815|allocated|A922A115
apnic|CN|ipv4|43.237.44.0|1024|20140815|allocated|A92F02DA
apnic|CN|ipv4|43.237.48.0|1024|20140815|allocated|A92942B0
apnic|CN|ipv4|43.237.52.0|1024|20140818|allocated|A921642B
apnic|CN|ipv4|43.237.56.0|1024|20140818|allocated|A9224EBD
apnic|CN|ipv4|43.237.60.0|1024|20140818|allocated|A928102E
apnic|CN|ipv4|43.237.64.0|1024|20140818|allocated|A9273AD2
apnic|CN|ipv4|43.237.68.0|1024|20140818|allocated|A92AB56A
apnic|CN|ipv4|43.237.72.0|1024|20140818|allocated|A92CF927
apnic|CN|ipv4|43.237.76.0|1024|20140801|allocated|A92E4AD3
apnic|CN|ipv4|43.237.80.0|1024|20140818|allocated|A9255482
apnic|CN|ipv4|43.237.84.0|1024|20140818|allocated|A92B91BD
apnic|CN|ipv4|43.237.88.0|1024|20140819|allocated|A92F7EA2
apnic|CN|ipv4|43.237.92.0|1024|20140819|allocated|A9243B23
apnic|CN|ipv4|43.237.96.0|1024|20140819|allocated|A92520AE
apnic|CN|ipv4|43.237.100.0|1024|20140819|allocated|A92A8F00
apnic|CN|ipv4|43.237.104.0|1024|20140819|allocated|A924BBCC
apnic|CN|ipv4|43.237.108.0|1024|20140819|allocated|A926B018
apnic|CN|ipv4|43.237.112.0|1024|20140819|allocated|A92AC194
apnic|CN|ipv4|43.237.116.0|1024|20140819|allocated|A92B9D9A
apnic|CN|ipv4|43.237.120.0|1024|20140819|allocated|A927DC49
apnic|CN|ipv4|43.237.124.0|1024|20140819|allocated|A92A3FB3
apnic|CN|ipv4|43.237.128.0|1024|20140819|allocated|A924F22E
apnic|CN|ipv4|43.237.132.0|1024|20140821|allocated|A923F49E
apnic|CN|ipv4|43.237.136.0|1024|20140821|allocated|A92541A4
apnic|CN|ipv4|43.237.140.0|1024|20140821|allocated|A92AEF65
apnic|CN|ipv4|43.237.144.0|1024|20140821|allocated|A928DCEC
apnic|CN|ipv4|43.237.148.0|1024|20140821|allocated|A9291AF2
apnic|CN|ipv4|43.237.152.0|1024|20140821|allocated|A927C0AE
apnic|CN|ipv4|43.237.156.0|1024|20140821|allocated|A92A595A
apnic|CN|ipv4|43.237.160.0|1024|20140821|allocated|A929958A
apnic|CN|ipv4|43.237.164.0|1024|20140821|allocated|A926C39C
apnic|CN|ipv4|43.237.168.0|1024|20140821|allocated|A92BC443
apnic|CN|ipv4|43.237.172.0|1024|20140821|allocated|A92632A4
apnic|CN|ipv4|43.237.176.0|1024|20140821|allocated|A924FA45
apnic|CN|ipv4|43.237.180.0|1024|20140821|allocated|A923F8A8
apnic|CN|ipv4|43.237.184.0|1024|20140821|allocated|A92BD07F
apnic|CN|ipv4|43.237.188.0|1024|20140821|allocated|A925B6E0
apnic|CN|ipv4|43.237.192.0|1024|20140821|allocated|A92BF8F6
apnic|CN|ipv4|43.237.196.0|1024|20140821|allocated|A921F2D8
apnic|CN|ipv4|43.237.200.0|1024|20140821|allocated|A922D354
apnic|CN|ipv4|43.237.204.0|1024|20140821|allocated|A92BFA01
apnic|CN|ipv4|43.237.208.0|1024|20140821|allocated|A92AF3CE
apnic|CN|ipv4|43.237.212.0|1024|20140821|allocated|A927A209
apnic|CN|ipv4|43.237.216.0|1024|20140821|allocated|A92EDF82
apnic|CN|ipv4|43.237.220.0|1024|20140821|allocated|A9297B1B
apnic|CN|ipv4|43.237.224.0|1024|20140821|allocated|A92495ED
apnic|CN|ipv4|43.237.228.0|1024|20140822|allocated|A92963CB
apnic|CN|ipv4|43.237.232.0|1024|20140822|allocated|A92A6E4F
apnic|CN|ipv4|43.237.236.0|1024|20140822|allocated|A92B2654
apnic|CN|ipv4|43.237.240.0|1024|20140822|allocated|A9279D91
apnic|CN|ipv4|43.237.244.0|1024|20140822|allocated|A9287217
apnic|CN|ipv4|43.237.248.0|1024|20140822|allocated|A92FF75C
apnic|CN|ipv4|43.237.252.0|1024|20140822|allocated|A92545E7
apnic|CN|ipv4|43.238.0.0|1024|20140822|allocated|A92267AD
apnic|CN|ipv4|43.238.4.0|1024|20140822|allocated|A92E1505
apnic|CN|ipv4|43.238.8.0|1024|20140822|allocated|A92D6B79
apnic|CN|ipv4|43.238.12.0|1024|20140822|allocated|A92F2002
apnic|CN|ipv4|43.238.16.0|1024|20140822|allocated|A9264944
apnic|CN|ipv4|43.238.20.0|1024|20140822|allocated|A928DBB3
apnic|CN|ipv4|43.238.24.0|1024|20140822|allocated|A925E9D1
apnic|CN|ipv4|43.238.28.0|1024|20140822|allocated|A92BC2DB
apnic|CN|ipv4|43.238.32.0|1024|20140822|allocated|A9271F34
apnic|CN|ipv4|43.238.36.0|1024|20140822|allocated|A92696F1
apnic|CN|ipv4|43.238.40.0|1024|20140822|allocated|A9293561
apnic|CN|ipv4|43.238.44.0|1024|20140822|allocated|A925F505
apnic|CN|ipv4|43.238.48.0|1024|20140822|allocated|A9274ECD
apnic|CN|ipv4|43.238.52.0|1024|20140822|allocated|A9221D69
apnic|CN|ipv4|43.238.56.0|1024|20140822|allocated|A9273EF7
apnic|CN|ipv4|43.238.60.0|1024|20140822|allocated|A9290A17
apnic|CN|ipv4|43.238.64.0|1024|20140822|allocated|A92D17B9
apnic|CN|ipv4|43.238.68.0|1024|20140822|allocated|A926EE30
apnic|CN|ipv4|43.238.72.0|1024|20140822|allocated|A927CD1B
apnic|CN|ipv4|43.238.76.0|1024|20140822|allocated|A9213B22
apnic|CN|ipv4|43.238.80.0|1024|20140822|allocated|A92FF552
apnic|CN|ipv4|43.238.84.0|1024|20140822|allocated|A9226207
apnic|CN|ipv4|43.238.88.0|1024|20140822|allocated|A92F517A
apnic|CN|ipv4|43.238.92.0|1024|20140822|allocated|A924E7E3
apnic|CN|ipv4|43.238.96.0|1024|20140822|allocated|A92288F4
apnic|CN|ipv4|43.238.100.0|1024|20140822|allocated|A92C2324
apnic|CN|ipv4|43.238.104.0|1024|20140822|allocated|A92ED314
apnic|CN|ipv4|43.238.108.0|1024|20140822|allocated|A9280226
apnic|CN|ipv4|43.238.112.0|1024|20140822|allocated|A92F4881
apnic|CN|ipv4|43.238.116.0|1024|20140822|allocated|A9245A32
apnic|CN|ipv4|43.238.120.0|1024|20140822|allocated|A92A9D36
apnic|CN|ipv4|43.238.124.0|1024|20140822|allocated|A9277D6A
apnic|CN|ipv4|43.238.128.0|1024|20140822|allocated|A9297A5F
apnic|CN|ipv4|43.238.132.0|1024|20140822|allocated|A9284431
apnic|CN|ipv4|43.238.136.0|1024|20140822|allocated|A92EFE66
apnic|CN|ipv4|43.238.140.0|1024|20140822|allocated|A92EBA20
apnic|CN|ipv4|43.238.144.0|1024|20140822|allocated|A9233E64
apnic|CN|ipv4|43.238.148.0|1024|20140822|allocated|A92DCC1C
apnic|CN|ipv4|43.238.152.0|1024|20140822|allocated|A9246956
apnic|CN|ipv4|43.238.156.0|1024|20140822|allocated|A92135D9
apnic|CN|ipv4|43.238.160.0|1024|20140822|allocated|A9211012
apnic|CN|ipv4|43.238.164.0|1024|20140801|allocated|A92D46EF
apnic|CN|ipv4|43.238.168.0|1024|20140822|allocated|A926DF5D
apnic|CN|ipv4|43.238.172.0|1024|20140822|allocated|A926412B
apnic|CN|ipv4|43.238.176.0|1024|20140823|allocated|A9288EE5
apnic|CN|ipv4|43.238.180.0|1024|20140823|allocated|A92680ED
apnic|CN|ipv4|43.238.184.0|1024|20140823|allocated|A92D8989
apnic|CN|ipv4|43.238.188.0|1024|20140823|allocated|A9261EAE
apnic|CN|ipv4|43.238.192.0|1024|20140823|allocated|A92DE4B7
apnic|CN|ipv4|43.238.196.0|1024|20140823|allocated|A92979C1
apnic|CN|ipv4|43.238.200.0|1024|20140823|allocated|A9281D6A
apnic|CN|ipv4|43.238.204.0|1024|20140823|allocated|A921B5EC
apnic|CN|ipv4|43.238.208.0|1024|20140823|allocated|A92D7548
apnic|CN|ipv4|43.238.212.0|1024|20140823|allocated|A92F63E6
apnic|CN|ipv4|43.238.216.0|1024|20140823|allocated|A92A472F
apnic|CN|ipv4|43.238.220.0|1024|20140823|allocated|A92FCB9E
apnic|CN|ipv4|43.238.224.0|1024|20140823|allocated|A922248C
apnic|CN|ipv4|43.238.228.0|1024|20140823|allocated|A92753E0
apnic|CN|ipv4|43.238.232.0|1024|20140823|allocated|A92E35BD
apnic|CN|ipv4|43.238.236.0|1024|20140823|allocated|A9231113
apnic|CN|ipv4|43.238.240.0|1024|20140823|allocated|A921B428
apnic|CN|ipv4|43.238.244.0|1024|20140823|allocated|A92D518E
apnic|CN|ipv4|43.238.248.0|1024|20140823|allocated|A92F7B52
apnic|CN|ipv4|43.238.252.0|1024|20140823|allocated|A92673EF
apnic|CN|ipv4|43.239.0.0|1024|20140823|allocated|A92ED1B6
apnic|CN|ipv4|43.239.4.0|1024|20140823|allocated|A928BCC2
apnic|CN|ipv4|43.239.8.0|2048|20140801|allocated|A928EEFA
apnic|CN|ipv4|43.239.16.0|1024|20140823|allocated|A92A4917
apnic|CN|ipv4|43.239.20.0|1024|20140823|allocated|A92B1513
apnic|CN|ipv4|43.239.24.0|1024|20140823|allocated|A92B5680
apnic|CN|ipv4|43.239.28.0|1024|20140823|allocated|A926F8AD
apnic|CN|ipv4|43.239.32.0|1024|20140823|allocated|A92AE769
apnic|CN|ipv4|43.239.36.0|1024|20140823|allocated|A92584A2
apnic|CN|ipv4|43.239.40.0|1024|20140823|allocated|A92B7C8D
apnic|CN|ipv4|43.239.44.0|1024|20140826|allocated|A92EA8D8
apnic|CN|ipv4|43.239.48.0|1024|20140826|allocated|A9255ECE
apnic|CN|ipv4|43.240.0.0|1024|20140808|allocated|A9245F5B
apnic|IN|ipv4|43.240.4.0|1024|20140808|allocated|A92A99C8
apnic|IN|ipv4|43.240.8.0|1024|20140808|allocated|A929FD70
apnic|HK|ipv4|43.240.12.0|1024|20140811|assigned|A912F9EE
apnic|HK|ipv4|43.240.16.0|1024|20140811|allocated|A9161E39
apnic|MY|ipv4|43.240.20.0|1024|20140811|allocated|A911C812
apnic|TW|ipv4|43.240.24.0|1024|20140811|allocated|A92A8502
apnic|HK|ipv4|43.240.28.0|1024|20140811|allocated|A91148DA
apnic|NZ|ipv4|43.240.32.0|1024|20140811|allocated|A91178F4
apnic|HK|ipv4|43.240.36.0|1024|20140811|allocated|A91DBAA3
apnic|AU|ipv4|43.240.40.0|1024|20140811|allocated|A91F0C6D
apnic|TW|ipv4|43.240.44.0|1024|20140811|allocated|A92654C8
apnic|CN|ipv4|43.240.48.0|1024|20140812|allocated|A91FAEBB
apnic|BD|ipv4|43.240.52.0|1024|20140812|allocated|A919F7C3
apnic|CN|ipv4|43.240.56.0|1024|20140812|allocated|A929B383
apnic|CN|ipv4|43.240.60.0|1024|20140812|allocated|A926B432
apnic|IN|ipv4|43.240.64.0|1024|20140812|assigned|A92591B0
apnic|CN|ipv4|43.240.68.0|1024|20140813|allocated|A92AEC47
apnic|CN|ipv4|43.240.72.0|1024|20140813|allocated|A922EA36
apnic|CN|ipv4|43.240.76.0|1024|20140813|allocated|A9226672
apnic|ID|ipv4|43.240.80.0|1024|20140813|allocated|A929649C
apnic|CN|ipv4|43.240.84.0|1024|20140813|allocated|A9213A8B
apnic|GU|ipv4|43.240.88.0|1024|20140814|allocated|A91AEE28
apnic|HK|ipv4|43.240.92.0|1024|20140814|allocated|A918A4F9
apnic|AU|ipv4|43.240.96.0|1024|20140814|allocated|A91EF379
apnic|BD|ipv4|43.240.100.0|1024|20140814|allocated|A912F1D5
apnic|TW|ipv4|43.240.104.0|1024|20140814|allocated|A9244E76
apnic|TW|ipv4|43.240.108.0|1024|20140814|allocated|A92E5629
apnic|IN|ipv4|43.240.112.0|1024|20140814|assigned|A92C2018
apnic|HK|ipv4|43.240.116.0|1024|20140814|assigned|A91FEBD5
apnic|AU|ipv4|43.240.120.0|1024|20140815|allocated|A91604BF
apnic|CN|ipv4|43.240.124.0|1024|20140815|allocated|A92D2629
apnic|CN|ipv4|43.240.128.0|1024|20140815|allocated|A92164FC
apnic|CN|ipv4|43.240.132.0|1024|20140815|allocated|A92F9AAC
apnic|CN|ipv4|43.240.136.0|1024|20140815|allocated|A92D96CC
apnic|VU|ipv4|43.240.140.0|1024|20140815|allocated|A9198D02
apnic|AU|ipv4|43.240.144.0|1024|20140815|assigned|A91D9C74
apnic|ID|ipv4|43.240.148.0|1024|20140815|allocated|A928CAEB
apnic|TW|ipv4|43.240.152.0|1024|20140815|allocated|A9128CFD
apnic|CN|ipv4|43.240.156.0|1024|20140817|allocated|A92165C7
apnic|CN|ipv4|43.240.160.0|1024|20140817|allocated|A92343E4
apnic|CN|ipv4|43.240.164.0|1024|20140817|allocated|A92A3454
apnic|CN|ipv4|43.240.168.0|1024|20140817|allocated|A9233F2A
apnic|CN|ipv4|43.240.172.0|1024|20140817|allocated|A92424F0
apnic|CN|ipv4|43.240.176.0|1024|20140817|allocated|A9215166
apnic|CN|ipv4|43.240.180.0|1024|20140817|allocated|A92BBA0E
apnic|CN|ipv4|43.240.184.0|1024|20140817|allocated|A92316BF
apnic|CN|ipv4|43.240.188.0|1024|20140817|allocated|A92B437A
apnic|CN|ipv4|43.240.192.0|1024|20140817|allocated|A92EF2D4
apnic|CN|ipv4|43.240.196.0|1024|20140817|allocated|A92304D0
apnic|CN|ipv4|43.240.200.0|1024|20140817|allocated|A928D723
apnic|CN|ipv4|43.240.204.0|1024|20140818|allocated|A928605F
apnic|CN|ipv4|43.240.208.0|1024|20140818|allocated|A922F1D1
apnic|CN|ipv4|43.240.212.0|1024|20140818|allocated|A92BE58B
apnic|CN|ipv4|43.240.216.0|1024|20140818|allocated|A92990CC
apnic|CN|ipv4|43.240.220.0|1024|20140818|allocated|A927275C
apnic|ID|ipv4|43.240.224.0|1024|20140818|assigned|A9256239
apnic|ID|ipv4|43.240.228.0|1024|20140818|allocated|A92C0686
apnic|AU|ipv4|43.240.232.0|1024|20140819|allocated|A91BA5D9
apnic|CN|ipv4|43.240.236.0|1024|20140819|allocated|A91993B2
apnic|CN|ipv4|43.240.240.0|1024|20140819|allocated|A92893D1
apnic|CN|ipv4|43.240.244.0|1024|20140819|allocated|A922778B
apnic|CN|ipv4|43.240.248.0|1024|20140819|allocated|A924A5EC
apnic|CN|ipv4|43.240.252.0|1024|20140819|allocated|A9280802
apnic|CN|ipv4|43.241.0.0|1024|20140819|allocated|A92919C9
apnic|CN|ipv4|43.241.4.0|1024|20140819|allocated|A9292EE4
apnic|CN|ipv4|43.241.8.0|1024|20140819|allocated|A9263E1C
apnic|CN|ipv4|43.241.12.0|1024|20140819|allocated|A92CDE40
apnic|CN|ipv4|43.241.16.0|1024|20140819|allocated|A92E30BE
apnic|CN|ipv4|43.241.20.0|1024|20140819|allocated|A929B816
apnic|IN|ipv4|43.241.24.0|1024|20140819|allocated|A929A778
apnic|IN|ipv4|43.241.28.0|1024|20140819|allocated|A9210077
apnic|TW|ipv4|43.241.32.0|1024|20140819|allocated|A9280A11
apnic|IN|ipv4|43.241.36.0|1024|20140819|allocated|A921731B
apnic|IN|ipv4|43.241.40.0|1024|20140820|allocated|A91955C8
apnic|HK|ipv4|43.241.44.0|1024|20140820|allocated|A9112801
apnic|CN|ipv4|43.241.48.0|1024|20140820|allocated|A92A93D9
apnic|AU|ipv4|43.241.52.0|1024|20140820|allocated|A91CE3A3
apnic|TH|ipv4|43.241.56.0|1024|20140820|allocated|A9197650
apnic|IN|ipv4|43.241.60.0|1024|20140820|assigned|A927A4D3
apnic|IN|ipv4|43.241.64.0|1024|20140820|allocated|A9257E0E
apnic|IN|ipv4|43.241.68.0|1024|20140820|assigned|A92AD9A6
apnic|HK|ipv4|43.241.72.0|1024|20140821|allocated|A91E859E
apnic|CN|ipv4|43.241.76.0|1024|20140821|allocated|A92A8710
apnic|CN|ipv4|43.241.80.0|1024|20140821|allocated|A923107F
apnic|CN|ipv4|43.241.84.0|1024|20140821|allocated|A92F4260
apnic|CN|ipv4|43.241.88.0|1024|20140821|allocated|A92699FC
apnic|CN|ipv4|43.241.92.0|1024|20140821|allocated|A92B56DD
apnic|MY|ipv4|43.241.96.0|1024|20140821|allocated|A917AAF3
apnic|HK|ipv4|43.241.100.0|1024|20140821|allocated|A91BFB3A
apnic|KR|ipv4|43.241.104.0|1024|20140821|allocated|A92C9868
apnic|KR|ipv4|43.241.108.0|1024|20140821|allocated|A9250106
apnic|CN|ipv4|43.241.112.0|1024|20140821|allocated|A9234CC5
apnic|IN|ipv4|43.241.116.0|1024|20140821|allocated|A92EB885
apnic|IN|ipv4|43.241.120.0|1024|20140821|allocated|A9273B82
apnic|IN|ipv4|43.241.124.0|1024|20140821|assigned|A92CC880
apnic|IN|ipv4|43.241.128.0|1024|20140821|assigned|A929F8DA
apnic|IN|ipv4|43.241.132.0|1024|20140821|assigned|A92A9622
apnic|BT|ipv4|43.241.136.0|1024|20140822|allocated|A91C4F06
apnic|IN|ipv4|43.241.140.0|1024|20140822|assigned|A927462E
apnic|IN|ipv4|43.241.144.0|1024|20140822|assigned|A9246A0E
apnic|IN|ipv4|43.241.148.0|1024|20140822|assigned|A9249549
apnic|HK|ipv4|43.241.152.0|1024|20140822|allocated|A914FC2C
apnic|HK|ipv4|43.241.156.0|1024|20140822|allocated|A913ADDE
apnic|TW|ipv4|43.241.160.0|1024|20140825|allocated|A91415FB
apnic|WS|ipv4|43.241.164.0|1024|20140825|allocated|A9157216
apnic|CN|ipv4|43.241.168.0|1024|20140825|allocated|A92A952E
apnic|CN|ipv4|43.241.172.0|1024|20140825|allocated|A92AD17D
apnic|CN|ipv4|43.241.176.0|1024|20140825|allocated|A92DE080
apnic|CN|ipv4|43.241.180.0|1024|20140825|allocated|A92CE9FD
apnic|CN|ipv4|43.241.184.0|1024|20140825|allocated|A92DC495
apnic|AU|ipv4|43.241.188.0|1024|20140825|allocated|A91EAE3D
apnic|IN|ipv4|43.241.192.0|1024|20140825|assigned|A921DBEA
apnic|CN|ipv4|43.241.196.0|1024|20140826|allocated|A9175E72
apnic|AU|ipv4|43.241.200.0|1024|20140826|allocated|A91CE5A6
apnic|HK|ipv4|43.241.204.0|1024|20140826|allocated|A91C9BBC
apnic|CN|ipv4|43.241.208.0|1024|20140826|allocated|A9295EB9
apnic|CN|ipv4|43.241.212.0|1024|20140826|allocated|A92C1AFB
apnic|CN|ipv4|43.241.216.0|1024|20140826|allocated|A924C325
apnic|CN|ipv4|43.241.220.0|1024|20140826|allocated|A92FF0F9
apnic|CN|ipv4|43.241.224.0|1024|20140826|allocated|A92D54BC
apnic|CN|ipv4|43.241.228.0|1024|20140826|allocated|A92EA51E
apnic|CN|ipv4|43.241.232.0|1024|20140826|allocated|A92339F4
apnic|CN|ipv4|43.241.236.0|1024|20140826|allocated|A92ED32F
apnic|CN|ipv4|43.241.240.0|1024|20140826|allocated|A92703C3
apnic|IN|ipv4|43.241.244.0|1024|20140826|allocated|A92D5159
apnic|CN|ipv4|43.241.248.0|1024|20140826|allocated|A9278C20
apnic|CN|ipv4|43.241.252.0|1024|20140827|allocated|A91F1446
apnic|HK|ipv4|43.242.0.0|1024|20140827|assigned|A911BF0E
apnic|SG|ipv4|43.242.4.0|1024|20140827|assigned|A918E908
apnic|CN|ipv4|43.242.8.0|1024|20140827|allocated|A928DA24
apnic|CN|ipv4|43.242.12.0|1024|20140827|allocated|A92ECA54
apnic|CN|ipv4|43.242.16.0|1024|20140827|allocated|A9292A56
apnic|CN|ipv4|43.242.20.0|1024|20140827|allocated|A926606F
apnic|CN|ipv4|43.242.24.0|1024|20140827|allocated|A92FF2EF
apnic|CN|ipv4|43.242.28.0|1024|20140828|allocated|A9252AE2
apnic|HK|ipv4|43.242.32.0|1024|20140828|allocated|A91148DA
apnic|IN|ipv4|43.242.36.0|1024|20140828|allocated|A926BE36
apnic|AU|ipv4|43.242.40.0|1024|20140829|allocated|A912D0F9
apnic|CN|ipv4|43.242.44.0|1024|20140829|allocated|A91F9090
apnic|CN|ipv4|43.242.48.0|1024|20140829|allocated|A9258485
apnic|CN|ipv4|43.242.52.0|1024|20140829|allocated|A9273C1A
apnic|CN|ipv4|43.242.56.0|1024|20140829|allocated|A92F899E
apnic|CN|ipv4|43.242.60.0|1024|20140829|allocated|A92570C4
apnic|CN|ipv4|43.242.64.0|1024|20140829|allocated|A9212E90
apnic|AU|ipv4|43.242.68.0|1024|20140829|allocated|A913B369
apnic|CN|ipv4|43.242.72.0|1024|20140829|allocated|A9260215
apnic|CN|ipv4|43.242.76.0|1024|20140829|allocated|A92D7082
apnic|CN|ipv4|43.242.80.0|1024|20140829|allocated|A92D1926
apnic|CN|ipv4|43.242.84.0|1024|20140829|allocated|A92D3AEE
apnic|CN|ipv4|43.242.88.0|1024|20140829|allocated|A92A0662
apnic|CN|ipv4|43.242.92.0|1024|20140829|allocated|A92CE529
apnic|CN|ipv4|43.242.96.0|1024|20140829|allocated|A928E1D4
apnic|PK|ipv4|43.242.100.0|1024|20140901|allocated|A9150F66
apnic|IN|ipv4|43.242.104.0|1024|20140901|allocated|A9197CED
apnic|HK|ipv4|43.242.108.0|1024|20140901|allocated|A9151E1B
apnic|KR|ipv4|43.242.112.0|1024|20140901|allocated|A92306ED
apnic|IN|ipv4|43.242.116.0|1024|20140901|assigned|A92DA014
apnic|IN|ipv4|43.242.120.0|1024|20140901|assigned|A923D25F
apnic|IN|ipv4|43.242.124.0|1024|20140901|allocated|A92CCE82
apnic|HK|ipv4|43.242.128.0|1024|20140902|allocated|A918F20B
apnic|JP|ipv4|43.242.132.0|1024|20140902|allocated|A9132F31
apnic|JP|ipv4|43.242.136.0|1024|20140902|allocated|A91BCE7A
apnic|HK|ipv4|43.242.140.0|1024|20140902|assigned|A91B4E2B
apnic|CN|ipv4|43.242.144.0|1024|20140902|allocated|A9288DF5
apnic|CN|ipv4|43.242.148.0|1024|20140902|allocated|A92AB359
apnic|CN|ipv4|43.242.152.0|1024|20140902|allocated|A9143CB3
apnic|CN|ipv4|43.242.156.0|1024|20140902|allocated|A92F06FE
apnic|CN|ipv4|43.242.160.0|1024|20140902|allocated|A9274FD4
apnic|CN|ipv4|43.242.164.0|1024|20140903|allocated|A92FBBB4
apnic|CN|ipv4|43.242.168.0|1024|20140903|assigned|A9187465
apnic|HK|ipv4|43.242.172.0|1024|20140903|allocated|A91A5F7D
apnic|PK|ipv4|43.242.176.0|1024|20140904|allocated|A9112E46
apnic|CN|ipv4|43.242.180.0|1024|20140904|allocated|A92C8E05
apnic|SG|ipv4|43.242.184.0|1024|20140904|assigned|A9196D9E
apnic|CN|ipv4|43.242.188.0|1024|20140904|allocated|A926E739
apnic|CN|ipv4|43.242.192.0|1024|20140904|allocated|A92588F6
apnic|CN|ipv4|43.242.196.0|1024|20140904|allocated|A92C0EE3
apnic|HK|ipv4|43.242.200.0|1024|20140904|allocated|A91AD0B2
apnic|CN|ipv4|43.242.204.0|1024|20140904|allocated|A9282EEC
apnic|IN|ipv4|43.242.208.0|1024|20140904|allocated|A92B3CB0
apnic|IN|ipv4|43.242.212.0|1024|20140904|assigned|A924C5E4
apnic|CN|ipv4|43.242.216.0|1024|20140904|allocated|A92C7487
apnic|CN|ipv4|43.242.220.0|1024|20140904|allocated|A9273278
apnic|IN|ipv4|43.242.224.0|1024|20140904|assigned|A92D23D5
apnic|IN|ipv4|43.242.228.0|1024|20140904|assigned|A92E3EAD
apnic|HK|ipv4|43.242.232.0|1024|20140904|assigned|A91CCAC4
apnic|HK|ipv4|43.242.236.0|1024|20140905|allocated|A918B0B9
apnic|MN|ipv4|43.242.240.0|1024|20140905|allocated|A9119510
apnic|IN|ipv4|43.242.244.0|1024|20140905|allocated|A9185413
apnic|IN|ipv4|43.242.248.0|1024|20140905|allocated|A91EF246
apnic|CN|ipv4|43.242.252.0|1024|20140905|allocated|A927BDFF
apnic|AU|ipv4|43.243.0.0|1024|20140905|allocated|A9144463
apnic|CN|ipv4|43.243.4.0|1024|20140905|allocated|A92E2C74
apnic|CN|ipv4|43.243.8.0|1024|20140905|allocated|A9260DBB
apnic|CN|ipv4|43.243.12.0|1024|20140905|allocated|A923F0AA
apnic|CN|ipv4|43.243.16.0|1024|20140908|allocated|A92C8DEB
apnic|AU|ipv4|43.243.20.0|256|20140908|allocated|A913B673
apnic|NZ|ipv4|43.243.21.0|256|20141009|assigned|A919EF51
apnic|NZ|ipv4|43.243.22.0|512|20141015|assigned|A91D51C1
apnic|CN|ipv4|43.243.24.0|1024|20140908|allocated|A91D14BE
apnic|HK|ipv4|43.243.28.0|1024|20140908|allocated|A91F3256
apnic|HK|ipv4|43.243.32.0|1024|20140908|allocated|A913BE6F
apnic|IN|ipv4|43.243.36.0|1024|20140908|assigned|A928B5ED
apnic|HK|ipv4|43.243.40.0|1024|20140909|allocated|A91E3E27
apnic|HK|ipv4|43.243.44.0|1024|20140909|allocated|A91716DE
apnic|HK|ipv4|43.243.48.0|1024|20140909|allocated|A91B07E7
apnic|HK|ipv4|43.243.52.0|1024|20140909|assigned|A915A421
apnic|NZ|ipv4|43.243.56.0|1024|20140909|allocated|A91AF3C6
apnic|NZ|ipv4|43.243.60.0|1024|20140909|allocated|A915CADD
apnic|HK|ipv4|43.243.64.0|1024|20140909|allocated|A9150E55
apnic|HK|ipv4|43.243.68.0|1024|20140909|allocated|A91958FF
apnic|HK|ipv4|43.243.72.0|1024|20140909|allocated|A91FEDE7
apnic|IN|ipv4|43.243.76.0|1024|20140909|allocated|A929A2CB
apnic|IN|ipv4|43.243.80.0|1024|20140909|allocated|A92BE6A6
apnic|IN|ipv4|43.243.84.0|1024|20140909|assigned|A927BCF2
apnic|CN|ipv4|43.243.88.0|1024|20140910|allocated|A9270073
apnic|HK|ipv4|43.243.92.0|1024|20140910|allocated|A912F535
apnic|NP|ipv4|43.243.96.0|1024|20140911|allocated|A91BC966
apnic|HK|ipv4|43.243.100.0|1024|20140911|allocated|A911BEA2
apnic|HK|ipv4|43.243.104.0|1024|20140911|allocated|A91DA04E
apnic|HK|ipv4|43.243.108.0|1024|20140911|allocated|A9158E7B
apnic|HK|ipv4|43.243.112.0|1024|20140911|allocated|A9169A03
apnic|AU|ipv4|43.243.116.0|1024|20140911|allocated|A91BF27F
apnic|HK|ipv4|43.243.120.0|1024|20140911|assigned|A91CA496
apnic|PH|ipv4|43.243.124.0|1024|20140912|allocated|A9189099
apnic|CN|ipv4|43.243.128.0|1024|20140912|allocated|A928CEBE
apnic|CN|ipv4|43.243.136.0|1024|20140912|allocated|A92A8C20
apnic|ID|ipv4|43.243.140.0|1024|20140912|allocated|A9249C46
apnic|CN|ipv4|43.243.144.0|1024|20140912|allocated|A92E862B
apnic|CN|ipv4|43.243.148.0|1024|20140912|allocated|A924A525
apnic|ID|ipv4|43.243.152.0|1024|20140912|allocated|A926A79D
apnic|CN|ipv4|43.243.156.0|1024|20140912|allocated|A92BA715
apnic|MN|ipv4|43.243.160.0|1024|20140912|allocated|A915EAD0
apnic|HK|ipv4|43.243.164.0|1024|20140912|allocated|A91E9D58
apnic|CN|ipv4|43.243.168.0|1024|20140913|allocated|A9156B81
apnic|IN|ipv4|43.243.172.0|1024|20140914|assigned|A9294627
apnic|HK|ipv4|43.243.176.0|1024|20140915|allocated|A91CA496
apnic|CN|ipv4|43.243.180.0|1024|20140915|allocated|A927D1BA
apnic|ID|ipv4|43.243.184.0|1024|20140915|allocated|A9194F88
apnic|CN|ipv4|43.243.188.0|1024|20140915|allocated|A91C3B4F
apnic|HK|ipv4|43.243.192.0|1024|20140916|allocated|A91DD0D3
apnic|NZ|ipv4|43.243.196.0|1024|20140916|allocated|A916ED18
apnic|AU|ipv4|43.243.200.0|1024|20140916|assigned|A9110308
apnic|BD|ipv4|43.243.204.0|1024|20140916|allocated|A91A1EE0
apnic|HK|ipv4|43.243.208.0|1024|20140916|allocated|A911A35E
apnic|IN|ipv4|43.243.212.0|1024|20140916|allocated|A92786FF
apnic|KR|ipv4|43.243.216.0|1024|20140916|allocated|A92BB735
apnic|HK|ipv4|43.243.220.0|1024|20140916|allocated|A9132407
apnic|HK|ipv4|43.243.224.0|1024|20140917|allocated|A914B459
apnic|CN|ipv4|43.243.228.0|1024|20140917|allocated|A924128D
apnic|CN|ipv4|43.243.232.0|1024|20140918|allocated|A926E9BA
apnic|AU|ipv4|43.243.236.0|1024|20140918|allocated|A91B0EF4
apnic|HK|ipv4|43.243.240.0|1024|20140918|allocated|A9110650
apnic|CN|ipv4|43.243.244.0|1024|20140918|allocated|A92C895C
apnic|HK|ipv4|43.243.248.0|1024|20140918|allocated|A91DE034
apnic|TW|ipv4|43.243.252.0|1024|20140918|allocated|A91D92FB
apnic|JP|ipv4|43.244.0.0|65536|20090723|allocated|A9290F05

Child Tickets

Change History (20)

comment:1 Changed 4 years ago by starlight

I have a perl script that I could easily modify to generate the IPv4 geoip file from the RIR files and am willing to contribute. It redistributes RIR blocks into correct CIDR ranges if geoip requires that. If not it's a fairly trivial matter of coalescing adjacent country ranges (without the gaps, of course).

Don't have an IPv6 flavor, but it's probably not hard to adapt from the IPv4 logic.

comment:2 Changed 4 years ago by nickm

Cc: karsten added
Component: - Select a componentTor
Milestone: Tor: 0.2.6.x-final

Interesting! Our existing scripts are in src/geoip; more wouldn't hurt, if they improve the quality and accuracy of our results.

comment:3 Changed 4 years ago by starlight

I'll work on it over the next couple of week.

comment:4 Changed 4 years ago by karsten

starlight, before you go write code, please note that changing tor's IP-to-country database is not something we should do without doing some research first. Maybe take a look at this report where I compared databases a few years ago. It may be that the situation has changed and MaxMind doesn't have the best database for our purposes anymore. I'd want to see some kind of proof for such a statement before switching, though.

comment:5 Changed 4 years ago by starlight

I was thinking of delaying for a different reason. If MaxMind is a sponsor to any degree it would likely be best to retain them as the source. Possibly they clean the RIR (Regional Internet Database) information, though in general RIR data is the primary source for IP block country code assignments.

My guess is that the existing script has a bug where it coalesces non-contiguous blocks. I was thinking of looking at it but it seemed to me like the MaxMind database cannot be downloaded freely, so there's no way to test.

comment:6 Changed 4 years ago by starlight

Just checked the November geopip found in TBB 4.5a3.

It has some of the above blocks, but is still badly
out of date. This region is in rapid flux and my guess
is with IPv4 space exhaustion well along, many such
highly fragmented and rapidly evolving regions exist.

Could someone script a daily geoip build that's
posted publicly? That way operators who want
semi-accurate bridge-ips in their relay statistics
can keep it reasonably fresh. This will mitigate
the unallocated region bug as well.

comment:7 Changed 4 years ago by starlight

After a bit if digging I found the "current"
version of geoip, which was built on Jan 7, 2015.
It's found at

https://gitweb.torproject.org/tor.git/tree/src/config

and will presumably always contain the most
recent version of the geoip files.

On causal examination it appears to cover the
new allocations in the above region, so this
resolves the immediate operational annoyance
of incorrect bridge country statistics.

However I believe it is a serious bug that the
conversion script interpolates unallocated
regions bounded by same-country blocks as
belonging to that country.

comment:8 in reply to:  6 Changed 4 years ago by arma

Replying to starlight:

Just checked the November geopip found in TBB 4.5a3.

It has some of the above blocks, but is still badly
out of date.

In the distant past we had a plan for Tor to auto update its geoip file, rather than using the file that's shipped in whatever package you fetch (and which could remain static for a year if there are no updates to that package):
https://gitweb.torproject.org/torspec.git/tree/proposals/126-geoip-reporting.txt#n186

comment:9 in reply to:  7 ; Changed 4 years ago by karsten

Replying to starlight:

However I believe it is a serious bug that the
conversion script interpolates unallocated
regions bounded by same-country blocks as
belonging to that country.

Hmm? Can you give an example?

comment:10 in reply to:  9 Changed 4 years ago by starlight

Replying to karsten:

Hmm? Can you give an example?

Is provided in detail in the opening bug post. The

721420288,737479679,JP

or, converted

43.0.0.0,43.245.11.255,JP

entry in the August file includes the
large empty gap between the two actual
allocations.

apnic|JP|ipv4|43.232.0.0|262144|20090723|allocated|A92D3CAE
apnic|JP|ipv4|43.244.0.0|65536|20090723|allocated|A9290F05

or converted

43.232.0.0/14 to 43.235.255.255
43.244.0.0/16 to 43.244.255.255

clear enough?

comment:11 Changed 4 years ago by karsten

MaxMind's August 7 database contains these entries:

(720896000, 721420287, 'CN')
(721420288, 729808895, 'JP')
(729808896, 734003199, 'JP')
(734003200, 736100351, 'JP')
(736100352, 737148927, 'JP')
(737148928, 737411071, 'JP')
(737411072, 737476607, 'JP')
(737476608, 737478655, 'JP')
(737478656, 737479679, 'JP')
(737479680, 737480703, 'IN')

We don't consider APNIC files or other registry files, and we don't modify MaxMind's entries. The only thing we do is merge entries, if possible, which is the case here. I don't see a bug.

comment:12 Changed 4 years ago by starlight

The August 7 geoip file found at

https://gitweb.torproject.org/tor.git/tree/src/config?id=b98e3f993617fb6cb2e5f41bfa49c16a21965a4c
https://gitweb.torproject.org/tor.git/plain/src/config/geoip?id=b98e3f993617fb6cb2e5f41bfa49c16a21965a4c

and in prior alpha 2 TBB actually contains

720437248,720502783,AU
720502784,721420287,CN
721420288,737479679,JP ###
737479680,737480703,IN
737480704,737490943,JP

So at the time it was generated, either the MaxMind DB
had an incorrect entry or the conversion script
improperly conflated the gap. At the time, the
RIR DB was correct (per the original bug report
above). I'm not saying that the RIR info should
be used, it's just an obvious cross-reference point
and how I discovered the problem.

I assume the sequence next above was freshly
generated from a MaxMind's historical point-in-time
snapshot, and so cannot be considered definitive.

comment:13 Changed 4 years ago by starlight

For anyone who might doubt the veracity of
the original post, here a link to the
original RIR file referenced:

ftp://ftp.apnic.net/pub/stats/apnic/2014/delegated-apnic-extended-20140719.gz

The actual entries are

apnic|JP|ipv4|43.232.0.0|262144|20090723|allocated|A92D3CAE
apnic||ipv4|43.236.0.0|262144||available|
apnic||ipv4|43.240.0.0|262144||available|
apnic|JP|ipv4|43.244.0.0|65536|20090723|allocated|A9290F05

Where the "available" slots were not mentioned in the
bug report because the script here strips them out
for the particular use case.

comment:14 Changed 4 years ago by starlight

And to nail it down ever more firmly, the 8/7 RIR file

ftp://ftp.apnic.net/pub/stats/apnic/2014/delegated-apnic-extended-20140807.gz

contains

apnic|JP|ipv4|43.232.0.0|262144|20090723|allocated|A92D3CAE
apnic||ipv4|43.236.0.0|262144||reserved|
apnic||ipv4|43.240.0.0|262144||available|
apnic|JP|ipv4|43.244.0.0|65536|20090723|allocated|A9290F05

comment:15 Changed 4 years ago by karsten

I just uploaded MaxMind's last four GeoIP databases that we used as input to our script:

Please give an example of two entries in any of these files that are merged in the resulting geoip or geoip6 files which shouldn't be merged. Without such an example, I claim that there is no bug.

And again, we're not considering registry files, and we're not verifying how MaxMind produces their databases.

comment:16 Changed 4 years ago by starlight

I misunderstood the earlier comment as a challenge
to the accuracy of the RIR data or it's interpretation.
Was not clear from the wording itself that
MaxMind was wrong, and all previous discussion
tended to assert perfection of that source.

comment:17 Changed 4 years ago by karsten

Status: newneeds_information
Type: defectenhancement

Okay, now that we agree that there's no bug, I'm changing the ticket type to enhancement. My comment above still stands. Changing status to needs_information until there's research that implies that we should switch sources for tor's geoip and geoip6 files. To be clear, I don't believe that more sources would make these files better. I'm rather concerned that they'll make our database files harder to verify for others.

comment:18 Changed 4 years ago by starlight

Possibly the ticket should be closed. I opened it in a state of concern about why a bridge was wildly misreporting countries. Had noticed a couple of active connections that didn't make sense and dug into it. The offer to provide a RIR conversion script was off-the-cuff.

Without knowing the actual details, I would suppose that the big upside to MaxMind is the service properly attributes blocks that are registered in one country but actually used in a different country, such as the Microsoft Azure block written about in the press.

The problem experienced could have been a one-off glitch, and would have been corrected by keeping the geoip file up-to-date. Arma's point above about moving geoip into a Tor online configuration document might be the best use of time and resources.

One relatively simple improvement would be to run and include a new geoip whenever a TBB is released. Perhaps automate the process and run it daily to make that easier and make a fresh file available to relay operators and users. The file included in TBB 4.5a3 was two months old at the time it was released.

I just reran the RIR-to-firewall script here and observe that in the last six months or so, something like 10000 new IPv4 blocks have been assigned, many of which are V4 end-game 1024 address fragments from wildly mixed assortments of countries. Keeping geoip current has become more important over time.

comment:19 Changed 4 years ago by nickm

Milestone: Tor: 0.2.6.x-finalTor: 0.2.7.x-final

Enhancements from after 1 Feb aren't on track for 0.2.6. I'll branch 0.2.7 really soon though.

comment:20 Changed 4 years ago by karsten

Resolution: wontfix
Status: needs_informationclosed

A few comments before I close this ticket:

MaxMind only update their database at most once per month, usually in the first week or two. We usually take one or two weeks to notice, update files, file a ticket, and merge files. For example, if they update on the 14th and we don't check before the 24th, we'll still ship the file from the previous month. Should be okay.

Switching from shipping two files to providing contents via an online service comes with all kinds of problems from designing a good protocol to making infrastructure available. This is not going to happen unless the benefit outweighs the costs, which is not the case yet.

Updating geoip and geoip6 files is actually pretty simple:

# Last updated based on March 3 2015 Maxmind GeoLite2 Country
# wget https://geolite.maxmind.com/download/geoip/database/GeoLite2-Country.mmdb.gz
# gunzip GeoLite2-Country.mmdb.gz
# python mmdb-convert.py GeoLite2-Country.mmdb

The goal was to make it really easy for people to confirm that we're providing correct files.

All in all, I'm closing this ticket.

Note: See TracTickets for help on using tickets.