create a script that generates the authority format from the authorities in the current consensus

We need to make sure we also:

• apply address overrides
• make sure the details match the current list
• check that all supported Tor versions can parse the list (existing unit tests)

added 034-removed-20180328 034-triage-20180328 component::core tor/fallback scripts parent::24818 points::0.5 priority::medium severity::normal status::new tor-dirauth type::enhancement labels

Here is the specification for the authority list: https://github.com/teor2345/torspec/blob/dir-list/dir-list-spec.txt#L254

The data should be the same as the data in this list: https://gitweb.torproject.org/tor.git/tree/src/or/config.c#n1079

But it will end up looking a bit like this list: https://gitweb.torproject.org/tor.git/tree/src/or/fallback_dirs.inc#n27

And we need to get the data from the current Tor consensus using stem, so it's easy to update when it changes: https://stem.torproject.org/api/descriptor/remote.html#stem.descriptor.remote.get_consensus

It's ok to just get the fields right in a first draft, and then work out the exact order and formatting later.

Replying to teor:

We need to make sure we also:

• apply address overrides
• make sure the details match the current list

The address overrides will become obvious when we check against the current list.

• check that all supported Tor versions can parse the list (existing unit tests)

We can do this by replacing src/or/auth_dirs.inc with the generated list, then running Tor's "make check". This will only work once #24854 (moved) is completed.

We have time to do these, let's do them well in 0.3.4

Trac:
Milestone: Tor: 0.3.3.x-final to Tor: 0.3.4.x-final

You can start with the Stem example here: https://stem.torproject.org/tutorials/mirror_mirror_on_the_wall.html#where-can-i-get-the-current-descriptors

Then modify it to only print the authorities, by matching on the "Authority" flag: https://stem.torproject.org/api/descriptor/router_status_entry.html#stem.descriptor.router_status_entry.RouterStatusEntry

And add the other fields that are in the existing document: https://stem.torproject.org/api/descriptor/router_status_entry.html#stem.descriptor.router_status_entry.RouterStatusEntryV3

You might find that this spec helps, or there might be way too much detail: https://github.com/teor2345/torspec/blob/dir-list/dir-list-spec.txt#L252

Hi Tim. Is this to generate tor's hardcoded list of authorities? If so then seems a little odd to check for the 'Authority' flag since this means that your script will omit authorities when they're down.

For what it's worth Stem has a get_authorities() which references its own copy of the authority list...

https://stem.torproject.org/api/descriptor/remote.html#stem.descriptor.remote.get_authorities https://gitweb.torproject.org/stem.git/tree/stem/descriptor/remote.py#n823

However, probably unhelpful due to the clear chicken-and-egg since part of the goal is to sync this listing with the document you generate.

Maybe the best bet would be for your script to include a hardcoded listing of the authority fingerprints, then...

• Fetch their descriptors to generate the rest of the doc (address, or/dirport, nickname, etc).
• If any of the hardcoded relays isn't present in the consensus then error.

This way adding/removing authorities is as simple as changing the list of fingerprints and rerunning your script.

Cheers! -Damian

Replying to atagar:

Hi Tim. Is this to generate tor's hardcoded list of authorities? If so then seems a little odd to check for the 'Authority' flag since this means that your script will omit authorities when they're down.

For what it's worth Stem has a get_authorities() which references its own copy of the authority list...

https://stem.torproject.org/api/descriptor/remote.html#stem.descriptor.remote.get_authorities https://gitweb.torproject.org/stem.git/tree/stem/descriptor/remote.py#n823

However, probably unhelpful due to the clear chicken-and-egg since part of the goal is to sync this listing with the document you generate.

Maybe the best bet would be for your script to include a hardcoded listing of the authority fingerprints, then...

• Fetch their descriptors to generate the rest of the doc (address, or/dirport, nickname, etc).
• If any of the hardcoded relays isn't present in the consensus then error.

This way adding/removing authorities is as simple as changing the list of fingerprints and rerunning your script.

I want to use the consensus to make sure that we only include valid IPv6 addresses for authorities. When we start hard-coding ed25519 key fingerprints, we can use it to warn about incorrectly pinned ed25519 keys. (Key pinning is much less of a concern, but IPv6 addresses can be quite dynamic.)

I don't think it matters whether the other fields are retrieved from the consensus or the descriptor. So your strategy sounds good.

Trac:
Keywords: N/A deleted, 034-triage-20180328 added

Per our triage process, these tickets are pending removal from 0.3.4.

Trac:
Keywords: N/A deleted, 034-removed-20180328 added

These tickets, tagged with 034-removed-*, are no longer in-scope for 0.3.4. We can reconsider any of them, if time permits.

Trac:
Milestone: Tor: 0.3.4.x-final to Tor: unspecified

Remove useless CC

Trac:
Cc: teor@torproject.org to N/A

After #27914 (moved), fallback-scripts is its own repository. Fallback changes are no longer tied to Tor releases.

Trac:
Milestone: Tor: unspecified to N/A

changed time estimate to 4h

mentioned in issue #24853 (moved)

mentioned in issue #24854 (moved)

mentioned in issue #25538 (moved)

mentioned in issue tpo/core/tor#24853 (closed)

moved to tpo/core/fallback-scripts#24851