Opened 7 years ago

Closed 6 years ago

#7474 closed defect (wontfix)

Directory authorities don't store 15% or more of uploaded extra-info descriptors

Reported by: karsten Owned by:
Priority: Low Milestone: Tor: unspecified
Component: Core Tor/Tor Version:
Severity: Keywords: tor-auth
Cc: arma, Sebastian, ln5, weasel Actual Points:
Parent ID: Points:
Reviewer: Sponsor:

Description

Could it be that the directory authorities reject a number of extra-info descriptors for some yet unknown reason? Is there anything in the logs?

metrics-db started warning about missing descriptors yesterday:

[java] 2012-11-13 21:05:44 We are missing at least 0.5% of
  server or extra-info descriptors referenced from a consensus
  or vote.

Here are the most recent fractions of descriptors that metrics-db got:

Consensus/Vote, valid-after, votes, server descriptors, extra-infos:
V, 2012-11-14 14:00:00, NA, 5976/5976 (100.0%), 5081/5976 (85.0%)
V, 2012-11-14 14:00:00, NA, 5972/5972 (100.0%), 5077/5972 (85.0%)
V, 2012-11-14 14:00:00, NA, 5976/5976 (100.0%), 5081/5976 (85.0%)
V, 2012-11-14 14:00:00, NA, 5976/5976 (100.0%), 5081/5976 (85.0%)
V, 2012-11-14 14:00:00, NA, 5976/5976 (100.0%), 5081/5976 (85.0%)
V, 2012-11-14 14:00:00, NA, 5976/5976 (100.0%), 5081/5976 (85.0%)
V, 2012-11-14 14:00:00, NA, 5976/5976 (100.0%), 5081/5976 (85.0%)
V, 2012-11-14 14:00:00, NA, 5976/5976 (100.0%), 5081/5976 (85.0%)
V, 2012-11-14 14:00:00, NA, 5956/5956 (100.0%), 5061/5956 (85.0%)
C, 2012-11-14 14:00:00, 9/9 (100.0%), 3173/3173 (100.0%), 3099/3173 (97.7%)

So, while metrics-db manages to find 100.0% of server descriptors, it's down to 85.0% of referenced extra-info descriptors.

Here's a quick look at gabelmoo's cached descriptor files that metrics-db rsyncs and uses as one relay descriptor source:

$ grep -h "^opt extra-info-digest " cached-descriptors* | \
  cut -d" " -f3- | sort | uniq | wc -l
19043
$ grep "^extra-info " cached-extrainfo* | cut -d" " -f3 | \
  sort | uniq | wc -l
6713

I don't know what typical values are, but having only 6713 of 19043 referenced extra-info descriptors seems low.

This could be a problem in metrics-db. But it could also be a problem with the directory authorities.

Child Tickets

Change History (5)

comment:1 Changed 7 years ago by karsten

Correction: when looking at gabelmoo's cached descriptor files, we should compare the same thing in server and extra-info descriptors...

$ grep -h "^opt fingerprint " cached-descriptors* | \
  cut -d" " -f3- | sed 's/ //g' | sort | uniq | wc -l
7381
$ grep "^extra-info " cached-extrainfo* | cut -d" " -f3 | \
  sort | uniq | wc -l
6713

So, 6713 / 7381 = 90.9%, which is still lower than expected. It could be that 9.1% of relays don't publish extra-info descriptors, but that seems too high.

comment:2 Changed 7 years ago by ln5

This is what I see at maatuska:

grep -h "^opt extra-info-digest " cached-descriptors* | cut -d" " -f3- | sort | uniq | wc -l
   16381
grep "^extra-info " cached-extrainfo* | cut -d" " -f3 | sort | uniq | wc -l
    5664

comment:3 Changed 7 years ago by karsten

Priority: criticalminor

Looks like this problem was caused by somebody setting up 2K relays over a short period of time which failed at reliably uploading descriptors to the directory authorities. This has stopped a few days ago. We might want to look at archived votes/consensuses to understand what was happening, but it doesn't look as much as a bug in Tor anymore. Reducing priority to minor.

comment:4 Changed 7 years ago by nickm

Keywords: tor-auth added
Milestone: Tor: unspecified

comment:5 Changed 6 years ago by karsten

Resolution: wontfix
Status: newclosed

I'm not planning to look into this anymore, and I assume nobody else wants to do it. Closing. If somebody else wants to take a look, please re-open and re-assign.

Note: See TracTickets for help on using tickets.