Opened 4 years ago

Closed 20 months ago

#20910 closed enhancement (duplicate)

Make the fallback list automatically adapt to minor changes

Reported by: teor Owned by:
Priority: Medium Milestone: Tor: unspecified
Component: Core Tor/Fallback Scripts Version:
Severity: Normal Keywords: fallback
Cc: Actual Points:
Parent ID: Points: 1
Reviewer: Sponsor:

Description

I spend a lot of time contacting relay operators about new and changed IP addresses, fingerprints, etc.

It would be great to identify minor changes (such as adding an IPv6 address) and automatically adapt to them, rather than eliminating the relay from the list. In the case of an IPv6 address, automatically adding it should be OK.

Child Tickets

Change History (7)

comment:1 Changed 4 years ago by nickm

Keywords: tor-03-unspecified-201612 added
Milestone: Tor: 0.3.???Tor: unspecified

Finally admitting that 0.3.??? was a euphemism for Tor: unspecified all along.

comment:2 Changed 3 years ago by nickm

Keywords: tor-03-unspecified-201612 removed

Remove an old triaging keyword.

comment:3 Changed 3 years ago by nickm

Component: Core Tor/TorCore Tor/Fallback Scripts
Owner: set to teor

comment:4 Changed 3 years ago by teor

Owner: teor deleted
Status: newassigned

Disowning tickets I don't intend to work on in the next 6 months.

comment:5 Changed 3 years ago by teor

Status: assignednew

Mark all tickets that are assigned to nobody as "new".

comment:6 Changed 3 years ago by teor

This is the original wiki text:

1. Checking Existing Fallbacks

Fallbacks can go down, become slow, or change their identity key (fingerprint), addresses, or ports.
We:

  1. check if any existing fallbacks have changed,
  2. inform relay operators,
  3. modify the hard-coded list, and
  4. modify the whitelist and blacklist.

a. Finding Broken Fallbacks

To check existing fallbacks, we use the existing fallback list as the whitelist.

  1. Run the fallback script in check_existing mode, saving the list:
    scripts/maint/updateFallbackDirs.py check_existing > good_fallbacks 2> good_fallbacks.log
    

This can take a long time, as it downloads ~150MB of OnionOO data, parses it, then tries each fallback's DirPort.

b. Handling Broken Fallbacks

To produce a log that only has the whitelisted fallbacks in it, do:

for fp in `cat scripts/maint/fallback.whitelist | grep id= | cut -d" " -f3 | cut -d= -f2`; do
  grep $fp good_fallbacks.log >> good_fallbacks.whitelist.log;
done

The script will log warnings like:

WARNING::0C2C599AFCB26F5CFC2C7592435924C1D63D9484 excluded: has it gained an IPv6 address [2001:41d0:a:fb7a::1]:9001?
WARNING::FCB6695F8F2DC240E974510A4B3A0F2B12AB5B64 excluded: has it changed IPv4 from 195.154.8.111 to 178.62.12.24?
WARNING::Consensus download: 20.5s too slow from IPredator (197.231.221.211:9030), max download time 15.0s.

It will also produce info-level messages about other excluded fallbacks (see "Handling Missing Fallbacks"):

INFO::01A9258A46E97FF8B2CAC7910577862C14F2C524 not a candidate: running avg too low (0.116029)
INFO::1ECD73B936CB6E6B3CD647CC204F108D9DF2C9F7 not a candidate: guard avg too low (0.000000)
INFO::6A640018EABF3DA9BAD9321AA37C2C87BBE1F907 not a candidate: version not recommended

If relays are down, stem will produce log messages like this:

INFO::Initiating consensus download from armbrust (176.31.180.157:143).
INFO::Unable to retrieve a consensus from armbrust: <urlopen error [Errno 61] Connection refused>

Other relays may have been missing for so long, they aren't even in the OnionOO data.

For each warning and each excluded relay:

  1. email the operator (see "How to Contact Relay Operators"), and ask them if the change is permanent,
    • if there's nothing an operator can do to fix the issue (for example guard average), there's no reason to email them,
  2. remove the fallback from the hard-coded list, by commenting it out,
    • you must modify the start of each line, so that stem's automated parser detects the fallback change,
    • backport the modified hard-coded list to every Tor version >= 0.2.8 that might ever be re-released,
  3. move the relay from the fallback.whitelist to the fallback.blacklist,
    • if the script will pick up improvements automatically (uptime, guard average, version), there's no reason to blacklist the relay.

If the operator has fixed the issue, leave the fallback in the hard-coded list and the whitelist. But if they have a history of unexpected changes, it is safer to remove the relay from the hard-coded list, and blacklist it.

comment:7 Changed 20 months ago by teor

Resolution: duplicate
Status: newclosed

This was done in #24838.

Note: See TracTickets for help on using tickets.