Opened 13 months ago

Last modified 13 months ago

#27047 new enhancement

Authorities should keep recent consensuses, votes, and bandwidth files

Reported by: teor Owned by:
Priority: Medium Milestone: Tor: unspecified
Component: Core Tor/Tor Version:
Severity: Normal Keywords: tor-bwauth, tor-dirauth, needs-design
Cc: irl, metrics-team Actual Points:
Parent ID: #25925 Points:
Reviewer: Sponsor:

Description

Moving https://trac.torproject.org/projects/tor/ticket/21378#comment:12 into a separate ticket:

Quoting teor:

Replying to teor:

Replying to irl:

Using the fixed URL ​http://<hostname>/tor/status-vote/next/bandwidth.z sounds like it would be very easy to add this to CollecTor.

Thanks for the feedback!

We have discussed in the Metrics team extending dir-spec.txt to allow to fetch "recent" files as well as just next/current. In the case that there is a wide CollecTor outage, and we miss a file, it would be good to have those files cached (on a best-effort basis, not necessarily persisted to disk) and available via some URL.

How is this any different to losing descriptors or consensuses?
(Please answer this question on a separate ticket.)

I don't know if karsten already had some ideas about what these URLs would look like, but we should perhaps consider this before implementing changes to dir-spec.txt.

Please open a separate ticket for this feature. It's potentially a large feature. And it's not essential for the initial release of this feature.

If you open a separate ticket for historical directory documents, please make #26698 a child of that ticket. We'll need bandwidth file hashes to work out the exact file used in each vote.

Child Tickets

TicketTypeStatusOwnerSummary
#26698enhancementclosedAuthorities should put a hash of the bandwidth file in their votes
#26797defectnewDirAuths should only read the V3BandwidthsFile once per vote

Change History (2)

comment:1 Changed 13 months ago by teor

Cc: irl added
Keywords: needs-design added

irl, let's design the "recent" URLs in dir-spec.txt when you have time.

Here's some background:

The documents that are used form a tree (strictly, a DAG):

  • every consensus contains a vote-digest for the votes used to create it
  • after #26698, every vote will contain a bandwidth-file-digest for the bandwidth file used to create it

So we need a design to:

  • get recent consensuses by consensus period
  • get (unused) votes by consensus period
  • (unused bandwidth files aren't relevant, because each bandwidth file is tied to a vote)

Normally, there is 1 consensus per hour, but during a consensus failure, authorities attempt to produce 2 consensuses per hour:
https://gitweb.torproject.org/torspec.git/tree/dir-spec.txt#n1575

Directory caches (and authorities) store 72 hours of recent consensus diffs by default:
https://gitweb.torproject.org/torspec.git/tree/dir-spec.txt#n1961

So we should probably try to keep 72 hours of votes and bandwidth files as well.

comment:2 Changed 13 months ago by karsten

Cc: metrics-team added
Note: See TracTickets for help on using tickets.