Fuzzy match the fallback whitelist

We can rely on Onionoo to tell us when relay fingerprints or addresses change, so we don't need to do an exact match.

But listing fingerprints, addresses and ports in the whitelist is useful so that we know what a relay operator originally asked for.

Fuzzy matching simplifies maintaining the fallback whitelist, because we don't have to ask operators to update their relay details. (Or ask if those details are permanent.)

We deleted the blacklist in #26502 (moved).

changed milestone to %Tor: 0.4.0.x-final

added 034-removed-20180328 034-triage-20180328 035-roadmap-subtask 035-triaged-in-20180711 actualpoints::0.5 component::core tor/tor easy fallback fast-fix intro milestone::Tor: 0.4.0.x-final owner::teor parent::24786 points::1 priority::medium resolution::implemented reviewer::nickm s8-bootstrap severity::normal sponsor::8-can status::closed type::enhancement labels

Trac:
Summary: Just have relay fingerprints in the whitelist to Just have relay fingerprints in the fallback whitelist

We'll still need to check and warn if fallbacks don't have a DirPort. At least until stem gets ORPort support (#18856 (closed)), and we modify Tor and the script to accept relays without DirPorts (#19129 (moved)).

Trac:
Keywords: N/A deleted, 034-triage-20180328 added

Per our triage process, these tickets are pending removal from 0.3.4.

Trac:
Keywords: N/A deleted, 034-removed-20180328 added

These tickets, tagged with 034-removed-*, are no longer in-scope for 0.3.4. We can reconsider any of them, if time permits.

Trac:
Milestone: Tor: 0.3.4.x-final to Tor: unspecified

We should do this ticket in 0.3.4 or 0.3.5, because it's fast, and it lets us skip this slow step in the fallback rebuild process:

• running the script, finding change warnings, contacting operators, and updating their details

Trac:
Keywords: N/A deleted, fast-fix added
Milestone: Tor: unspecified to Tor: 0.3.4.x-final
Status: new to assigned
Owner: N/A to teor

Anyone can do this

Trac:
Owner: teor to N/A

8/150 = 5% of fallbacks are currently down. https://consensus-health.torproject.org/graphs.html At this rate, it will be 2019 before we reach our 25% failure threshold. So we don't need to do a fallback rebuild in 0.3.4.

Trac:
Milestone: Tor: 0.3.4.x-final to Tor: 0.3.5.x-final

We don't need to do this ticket as part of the fallback rebuild, but we can if we want to. (It might be easier to ignore changed addresses, than update all the changed addresses in the whitelist.)

Trac:
Milestone: Tor: 0.3.5.x-final to Tor: unspecified

Trac:
Summary: Just have relay fingerprints in the fallback whitelist to Ignore addresses in the fallback whitelist
Description: We can rely on Onionoo to tell us when relay addresses change, so we don't need to record or check them.

This simplifies maintaining the fallback whitelist, and also simplifies the script.

We should probably still put the full details in the blacklist, but that's rare. We can use generateFallbackDirLine.py for that.

to

We can rely on Onionoo to tell us when relay addresses change, so we don't need to check them. But listing addresses is useful when fingerprints change.

This simplifies maintaining the fallback whitelist, and also simplifies the script.

We should probably still put the full details in the blacklist, but that's rare. We can use generateFallbackDirLine.py for that.

Trac:
Description: We can rely on Onionoo to tell us when relay addresses change, so we don't need to check them. But listing addresses is useful when fingerprints change.

This simplifies maintaining the fallback whitelist, and also simplifies the script.

We should probably still put the full details in the blacklist, but that's rare. We can use generateFallbackDirLine.py for that.

to

We can rely on Onionoo to tell us when relay addresses change, so we don't need to check them. But listing addresses is useful when fingerprints change.

This simplifies maintaining the fallback whitelist, and also simplifies the script.

We will delete the blacklist in #26502 (moved).

Trac:
Keywords: fast-fix deleted, fast-fix 035-roadmap-subtask added

Remove useless CC

Trac:
Cc: teor@torproject.org to N/A

Hi imnotbad, if you're looking for something to work on while #26502 (moved) is being reviewed, you could try this ticket.

We just want to read the id= part of every line from the fallback list, and ignore the addresses and ports. Then when we match against the whitelist, we just want to check the id.

Trac:
Keywords: fallback deleted, easy intro fallback added
Milestone: Tor: unspecified to Tor: 0.3.5.x-final
Cc: N/A to imnotbad

Please base your branch for this ticket on the existing #26502 (moved) ticket, using a command like:

git checkout -b bug24838 bug26502

Edit: typo

Trac:
Keywords: fast-fix 035-roadmap-subtask, easy intro fallback deleted, fallback, 035-triaged-in-20180711, 035-roadmap-subtask, intro, easy, fast-fix added

Make everything that is assigned to no-one new again.

Trac:
Status: assigned to new

Trac:
Milestone: Tor: 0.3.5.x-final to Tor: 0.3.6.x-final

25% of fallbacks are failing, so these tickets need to be done in 0.3.5.

Trac:
Milestone: Tor: 0.3.6.x-final to Tor: 0.3.5.x-final

Trac:
Cc: imnotbad to imnotbad, phoul

I will do this ticket this week.

Trac:
Status: new to assigned
Owner: N/A to teor

See my pull request, which contains the code for this ticket and #28768 (moved): https://github.com/torproject/tor/pull/570

Trac:
Description: We can rely on Onionoo to tell us when relay addresses change, so we don't need to check them. But listing addresses is useful when fingerprints change.

This simplifies maintaining the fallback whitelist, and also simplifies the script.

We will delete the blacklist in #26502 (moved).

to

We can rely on Onionoo to tell us when relay fingerprints or addresses change, so we don't need to do an exact match.

But listing fingerprints, addresses and ports in the whitelist is useful so that we know what a relay operator originally asked for.

Fuzzy matching simplifies maintaining the fallback whitelist, because we don't have to ask operators to update their relay details. (Or ask if those details are permanent.)

We deleted the blacklist in #26502 (moved).
Keywords: N/A deleted, s8-bootstrap added
Sponsor: N/A to Sponsor8-can
Status: assigned to needs_review
Actualpoints: N/A to 0.5
Summary: Ignore addresses in the fallback whitelist to Fuzzy match the fallback whitelist
Milestone: Tor: 0.3.5.x-final to Tor: 0.4.0.x-final

This ticket makes changes to the tor repository, so it goes in the tor component for review.

Trac:
Component: Core Tor/Fallback Scripts to Core Tor/Tor

Trac:
Reviewer: N/A to nickm

I believe I already reviewed and merged this code as part of #28768 (moved). Closing.

Trac:
Status: needs_review to closed
Resolution: N/A to implemented

closed

changed time estimate to 8h

added 4h of time spent

mentioned in issue #24839 (moved)

mentioned in issue #28768 (moved)

mentioned in issue tpo/core/tor#24805 (closed)

moved to tpo/core/tor#24838 (closed)