Opened 3 years ago

Closed 3 years ago

#19154 closed defect (fixed)

Onionoo stopped to resolve ASN and country code for new relays

Reported by: twim Owned by: karsten
Priority: Medium Milestone:
Component: Metrics/Onionoo Version:
Severity: Major Keywords:
Cc: karsten, iwakeh Actual Points:
Parent ID: Points:
Reviewer: Sponsor:


Since sometime between 17th and 18th of May Onionoo includes no "country" (as well as other "country"-related fields) and "as_number" fields for new relays. Most probably it's all because installed MitM-as-a-Service (the OneMoreStep one) and Onionoo backend cannot update appropriate BGP databases from there [1][2].

Less probably, it's affected by this commit [3] and related to #18989.


Child Tickets

Change History (11)

comment:1 Changed 3 years ago by karsten

Owner: set to karsten
Status: newaccepted

Huh, there's indeed something wrong. I found this in the log:

2016-05-24 14:17:18,656 ERROR o.t.o.u.LookupService:169 Number format exception while parsing line ',1668284,1668284,,0,0,,23.5000,121.0000,51' in /srv/

I'll need to dig deeper, I just wanted to confirm this is a bug.

comment:2 Changed 3 years ago by karsten

Cc: iwakeh added
Severity: NormalMajor
Status: acceptedneeds_review

iwakeh, if you can, please review my branch task-19154. Thanks!

comment:3 Changed 3 years ago by iwakeh

Status: needs_reviewmerge_ready

Looks fine!

One could add a more exaggerated test with many more dummy fields, but that's just nitpicking.

comment:4 Changed 3 years ago by karsten

Added another test and merged to master. Thanks for looking!

Deploying now. It might take an hour or two until the change is effective. Leaving this ticket open until that is the case.

comment:5 Changed 3 years ago by twim

Could you please update the data affected by this bug? It seems to be lots of relays that joined during this period still have no GeoIP data.

comment:6 Changed 3 years ago by karsten

It took a bit longer than anticipated to deploy the patched version, but I think that most relays that joined in the past week or so have GeoIP data by now. Can you check?

comment:7 Changed 3 years ago by twim

Great, every relay since the earliest affected has GeoIP data now.
Thanks for fixing!

comment:8 Changed 3 years ago by karsten

Resolution: fixed
Status: merge_readyclosed

Great, thanks for checking and of course for reporting in the first place! Closing.

comment:9 Changed 3 years ago by cypherpunks

Resolution: fixed
Status: closedreopened

All relays that were first seen yesterday (2016-06-02) have no AS or country data. (tpo instance)

Are we seeing this bug again?

Last edited 3 years ago by cypherpunks (previous) (diff)

comment:10 Changed 3 years ago by twim

Yes, somthening definetely went wrong on 2016-06-01:

$ curl | jq '.relays[]' | jq 'select(.country == null)' | jq '.first_seen' | awk '{print $1}' | sort | uniq -c

1 "2016-05-03
1 "2016-05-05
1 "2016-05-24
1 "2016-05-27
34 "2016-06-01
71 "2016-06-02
16 "2016-06-03

comment:11 Changed 3 years ago by karsten

Resolution: fixed
Status: reopenedclosed

Oh, I know what happened. I forgot to update the crontab line, so that after the most recent server reboot the previous version was run again. Updated now and created #19265 to avoid situations like this in the future. Will kill the current process and switch to the recent version after the current run. Sorry! Closing as fixed even though it may take a few hours for results to include GeoIP results again.

Note: See TracTickets for help on using tickets.