Opened 3 months ago

Last modified 2 weeks ago

#24851 new enhancement

create a script that generates the authority format from the authorities in the current consensus

Reported by: teor Owned by:
Priority: Medium Milestone: Tor: unspecified
Component: Core Tor/Fallback Scripts Version:
Severity: Normal Keywords: tor-dirauth, 034-triage-20180328, 034-removed-20180328
Cc: teor@… Actual Points:
Parent ID: #24818 Points: 0.5
Reviewer: Sponsor:

Description

We need to make sure we also:

  • apply address overrides
  • make sure the details match the current list
  • check that all supported Tor versions can parse the list (existing unit tests)

Child Tickets

Change History (8)

comment:1 in reply to:  description Changed 3 months ago by teor

Here is the specification for the authority list:
https://github.com/teor2345/torspec/blob/dir-list/dir-list-spec.txt#L254

The data should be the same as the data in this list:
https://gitweb.torproject.org/tor.git/tree/src/or/config.c#n1079

But it will end up looking a bit like this list:
https://gitweb.torproject.org/tor.git/tree/src/or/fallback_dirs.inc#n27

And we need to get the data from the current Tor consensus using stem, so it's easy to update when it changes:
https://stem.torproject.org/api/descriptor/remote.html#stem.descriptor.remote.get_consensus

It's ok to just get the fields right in a first draft, and then work out the exact order and formatting later.

Replying to teor:

We need to make sure we also:

  • apply address overrides
  • make sure the details match the current list

The address overrides will become obvious when we check against the current list.

  • check that all supported Tor versions can parse the list (existing unit tests)

We can do this by replacing src/or/auth_dirs.inc with the generated list, then running Tor's "make check".
This will only work once #24854 is completed.

comment:2 Changed 3 months ago by teor

Milestone: Tor: 0.3.3.x-finalTor: 0.3.4.x-final

We have time to do these, let's do them well in 0.3.4

comment:4 Changed 3 months ago by atagar

Hi Tim. Is this to generate tor's hardcoded list of authorities? If so then seems a little odd to check for the 'Authority' flag since this means that your script will omit authorities when they're down.

For what it's worth Stem has a get_authorities() which references its own copy of the authority list...

https://stem.torproject.org/api/descriptor/remote.html#stem.descriptor.remote.get_authorities
https://gitweb.torproject.org/stem.git/tree/stem/descriptor/remote.py#n823

However, probably unhelpful due to the clear chicken-and-egg since part of the goal is to sync this listing with the document you generate.

Maybe the best bet would be for your script to include a hardcoded listing of the authority fingerprints, then...

  • Fetch their descriptors to generate the rest of the doc (address, or/dirport, nickname, etc).
  • If any of the hardcoded relays isn't present in the consensus then error.

This way adding/removing authorities is as simple as changing the list of fingerprints and rerunning your script.

Cheers! -Damian

comment:5 in reply to:  4 Changed 3 months ago by teor

Replying to atagar:

Hi Tim. Is this to generate tor's hardcoded list of authorities? If so then seems a little odd to check for the 'Authority' flag since this means that your script will omit authorities when they're down.

For what it's worth Stem has a get_authorities() which references its own copy of the authority list...

https://stem.torproject.org/api/descriptor/remote.html#stem.descriptor.remote.get_authorities
https://gitweb.torproject.org/stem.git/tree/stem/descriptor/remote.py#n823

However, probably unhelpful due to the clear chicken-and-egg since part of the goal is to sync this listing with the document you generate.

Maybe the best bet would be for your script to include a hardcoded listing of the authority fingerprints, then...

  • Fetch their descriptors to generate the rest of the doc (address, or/dirport, nickname, etc).
  • If any of the hardcoded relays isn't present in the consensus then error.

This way adding/removing authorities is as simple as changing the list of fingerprints and rerunning your script.

I want to use the consensus to make sure that we only include valid IPv6 addresses for authorities.
When we start hard-coding ed25519 key fingerprints, we can use it to warn about incorrectly pinned ed25519 keys.
(Key pinning is much less of a concern, but IPv6 addresses can be quite dynamic.)

I don't think it matters whether the other fields are retrieved from the consensus or the descriptor.
So your strategy sounds good.

comment:6 Changed 4 weeks ago by nickm

Keywords: 034-triage-20180328 added

comment:7 Changed 4 weeks ago by nickm

Keywords: 034-removed-20180328 added

Per our triage process, these tickets are pending removal from 0.3.4.

comment:8 Changed 2 weeks ago by nickm

Milestone: Tor: 0.3.4.x-finalTor: unspecified

These tickets, tagged with 034-removed-*, are no longer in-scope for 0.3.4. We can reconsider any of them, if time permits.

Note: See TracTickets for help on using tickets.