wiki:doc/UpdatingFallbackDirectoryMirrors

Version 47 (modified by teor, 11 months ago) (diff)

Rewrite backporting explanation

Updating Tor's Hard-Coded Fallback Directory Mirror List

This page has instructions for updating Tor's list of fallback directory mirrors.

Read Fallback Directory Mirrors for information about how fallbacks work.

A Typical Release

So you'd like to update Tor's hard-coded list of fallback directory mirrors for the next release?

  1. Ask relay operators to opt-in their stable relays to the fallback list.
  1. Update the fallback whitelist with new and changed relay details
    • File: tor/scripts/maint/fallback.whitelist
  1. Run updateFallbackDirs.py with the new whitelist. You'll need a decent network connection for this. If you have IPv6, please set PERFORM_IPV6_DIRPORT_CHECKS. Please attach the script logs to the trac ticket.
    • Command: tor/scripts/maint/updateFallbackDirs.py > src/or/fallback_dirs.inc 2> fallback_dirs.log
  1. Look at the warnings from the script to see which addresses have changed.
  1. Write a changes file
    • See below for an example, and commands that count fallback changes
  1. Announce the new list
  1. Tell the metrics relay search (atlas) maintainer that the list has changed
    • Email: metrics-team

The list should be rebuilt once ~25% of fallbacks go down. Tor clients are designed to work even if the entire list is down, with only minor delays.

How to Contact Relay Operators

When you email relay operators, use the email address (or other contact) in their relay's ContactInfo.

Individual Emails

Put the operator email in the To field.

Please also CC:

  • the address they replied with,
  • any addresses they CC'd,
  • teor (who did the first few fallback lists), and
  • tor-team.

Use the same subject for similar emails, so tor-team members can filter them out if desired.

Group Emails

Send the email to tor-relays. This allows operators without ContactInfo to opt-in (and it's good for transparency).

Please also CC:

  • teor (who did the first few fallback lists), and
  • the metrics team, who maintain relay search (atlas).

1. Asking Potential Fallbacks to Opt-In

The fallback.whitelist contains relays whose operators opted-in to become a fallback directory mirror at the last release. But new, stable relays could have been started since then, so we need to ask their operators if they want to opt-in. We:

  1. ask relay operators to opt-in,
  2. add opt-ins to the whitelist (and opt-outs to the blacklist).

a. Asking Relay Operators

Contact operators on tor-relays using the process in 'Group Emails' section above.

b. Whitelisting Fallbacks

As responses come in, add them to the whitelist.

If any operators confirm their relays will never be suitable, add them to the blacklist. (This makes it easier to switch to opt-out in future.)

2. Creating a New List of Fallbacks

a. How Many Fallbacks? (optional)

Every extra fallback:

  • spreads the load more evenly,
  • provides diverse IP addresses and ports for clients to contact,
  • provides another relay that can take up the load if some fallbacks fail.

But it also:

  • adds to the size of Tor,
  • adds a relay that might go down or change details,
  • adds to a list that might be blocked on some networks.

The original proposal was to select 20% of the guards (~200 relays) as fallback directory mirrors. Having 100 by the end of the 2.5 years is less than we might have wanted, but since the extra load is so small, it's unlikely to be an issue.

b. Generating the List

We want to find the full list of fallback directory mirrors.

  1. Make sure the whitelist and blacklist and script are the current ones from git master (not the modified ones from other instructions)
  2. If you have a working IPv6 connection (or can set up an IPv6 tunnel), perform IPv6 checks:
    - PERFORM_IPV6_DIRPORT_CHECKS = False if OUTPUT_CANDIDATES else False
    + PERFORM_IPV6_DIRPORT_CHECKS = True
    
  3. Run the fallback selection script, saving the list:
    scripts/maint/updateFallbackDirs.py > src/or/fallback_dirs.inc 2> fallback_dirs.log
    

This can take a long time, as it downloads ~150MB of OnionOO data, parses it, then tries each fallback's DirPort.

Handling Consensus Download Failures

Sometimes fallbacks are temporarily slow to serve consensuses, or temporarily down. Check again, at a different time, and from a different network connection, if you can.

If relays are down a lot, the uptime checks will eliminate them in the next release. But it might be worth removing them from the current release.

c. Confirming Fallback Changes with Operators

Handling Missing Fallbacks

Some of the excluded fallbacks are only logged at info level, because they are excluded by the script.

To find these fallbacks, do:

diff -u src/or/fallback_dirs.inc good_fallbacks

(The diff should be reasonably small, because the fallbacks are sorted in fingerprint order.)

Then, once you have a list of these fallbacks, search the log for their fingerprints. If they aren't in the log, try searching Atlas for their fingerprint. Also try their IPv4 address.

Handling New IPv6 Addresses

Adding an IPv6 address is a special case: clients never knew about the new IPv6 address, so they will continue to bootstrap fine using the IPv4 address. There's no need to remove the relay from the existing hard-coded list.

But the relay's whitelist entry must include the new IPv6 address, so that it can be selected for the next release. (If the IPv6 address isn't stable, then that address can't be added to the whitelist. But since the descriptor and whitelist don't match, the relay can never be selected. We'll fix this in #20175.)

d. Updating the List in Tor's Development Branch

Do:

git add src/or/fallback_dirs.inc

Formatting Changes Files

When we replace the entire list of fallbacks, this is what a changes file should look like:

  o Minor feature (fallback directory mirrors):
    - Replace the 80 remaining fallbacks of the 100 originally introduced
      in Tor 0.2.8.1-alpha in January 2016, with a list of 100 fallbacks
      (40 new, 60 existing, 20 removed) generated in March 2016.
      Closes ticket 17158; patch by teor.

You can use the following commands to count fallbacks:

Remaining/Replacement:

grep -c id= src/or/fallback_dirs.inc

New:

git diff src/or/fallback_dirs.inc | grep id= | grep + | wc -l

Removed (by deletion):

git diff src/or/fallback_dirs.inc | grep id= | grep - | grep -v \* | wc -l

Removed (by comments):

git diff src/or/fallback_dirs.inc | grep id= | grep + | grep \* | wc -l

The existing count is the replacement count, minus the new count. (It can't be calculated directly from the git diff, because diff skips unchanged lines.)

Remember to run make check to check for parse errors and changes file errors.

e. Backporting the List to Other Tor Versions

We backport the fallback list to any tor versions >= 0.2.8 that are still supported. This makes sure that those versions still bootstrap without delay, and avoids them overloading the authorities.