Opened 5 years ago

Last modified 21 months ago

#10871 new defect

Download more microdescriptors with a shorter request

Reported by: nickm Owned by:
Priority: Medium Milestone: Tor: unspecified
Component: Core Tor/Tor Version:
Severity: Normal Keywords: tor-client, directory, microdesc, prop281, low-bandwidth, sponsor4, 032-unreached
Cc: Actual Points:
Parent ID: Points:
Reviewer: Sponsor: Sponsor4


In a comment on #9969, karsten said:

A few thoughts:

  • Would it help if we implemented /tor/micro/all which is mentioned in dir-spec section 5.2 "Downloading router descriptors or microdescriptors" but which is not implemented yet? Of course, then clients would download the bulk of microdescriptors from a single directory.
  • Do we have to include full digests in requests, or would it be sufficient to ask for the first few digest bytes? Assuming that clients would only accept descriptors matching locally stored full digests. For example, requests could contain only the first 4 (or 8) base64 chars representing the first 3 (or 6) digest bytes. Directories could accept any multiple of 4 base64 chars.
  • Mixing the two ideas, how about we add a way to ask for 1/2, 1/4, etc. of all microdescriptors in a single request? The request could be /tor/micro/all/<base64-prefix>/<bits>, so that /tor/micro/all/A/1 means all digests starting with 0 binary, /tor/micro/all/w/2 means all digests starting with 11 binary, etc. Clients could decide how many requests to send from the number of descriptors they need, which may change over time.

Each of these ideas requires us to upgrade authorities and caches before clients will be able to use them.

I'm giving this a separate ticket since it's going to need analysis which #9969 won't.

Child Tickets

Change History (12)

comment:1 Changed 5 years ago by nickm

When somebody designs this, remember that getting all the microdescriptors that changed since the last consensus you downloaded is really different from getting all the microdescriptors in the latest consensus if you're starting from scratch.

Remember also that the total list of microdescs that a cache will hold is larger --potentially much larger -- than the list that a client needs in order to handle a single consensus. (That's because a cache remembers all the microdescs for the last several consensuses.)

Note finally that this becomes much simpler if the cache has the same consensus (and same last consensus?) that the client does, and harder if the cache doesn't.

comment:2 Changed 5 years ago by nickm

Milestone: Tor: 0.2.5.x-finalTor: 0.2.???

comment:3 Changed 3 years ago by arma

Keywords: low-bandwidth added
Severity: Normal

comment:4 Changed 3 years ago by teor

Milestone: Tor: 0.2.???Tor: 0.3.???

Milestone renamed

comment:5 Changed 2 years ago by nickm

Keywords: tor-03-unspecified-201612 added
Milestone: Tor: 0.3.???Tor: unspecified

Finally admitting that 0.3.??? was a euphemism for Tor: unspecified all along.

comment:6 Changed 2 years ago by nickm

Keywords: sponsor4 added

Bulk-adding "sponsor4" keyword to items that would appear to reduce low-bw clients' directory bandwidth usage. But we shouldn't build these without measurement/proposals: see #21205 and #21209.

comment:7 Changed 2 years ago by nickm

Sponsor: Sponsor4

Setting "sponsor4" sponsor on all tickets that have the sponsor4 keyword, but have no sponsor set.

comment:8 Changed 2 years ago by nickm

Remove Sponsor4 keyword, now that Sponsor4 is the value of the Sponsor field.

comment:9 Changed 2 years ago by nickm

Keywords: tor-03-unspecified-201612 removed

Remove an old triaging keyword.

comment:10 Changed 2 years ago by nickm

Milestone: Tor: unspecifiedTor: 0.3.2.x-final

If we decide to do this, 0.3.2 is the place.

comment:11 Changed 22 months ago by nickm

Keywords: prop281 added; needs-proposal removed

Prop281 describes an approach here.

comment:12 Changed 21 months ago by nickm

Keywords: 032-unreached added
Milestone: Tor: 0.3.2.x-finalTor: unspecified

Move some 0.3.2 items (fewer than I had expected for now) into Unspecified.

Note: See TracTickets for help on using tickets.