Opened 9 months ago

Last modified 5 weeks ago

#24786 new task

Rebuild the fallback list in 2018

Reported by: teor Owned by:
Priority: Medium Milestone: Tor: 0.3.5.x-final
Component: Core Tor/Fallback Scripts Version:
Severity: Normal Keywords: fallback, 035-roadmap-master, 029-backport, 032-backport-maybe, 033-backport, 034-backport, 034-roadmap-master, 034-triage-20180328, 034-included-20180328, 031-unreached-backport-maybe, 035-triaged-in-20180711
Cc: Actual Points:
Parent ID: Points: 3
Reviewer: Sponsor:

Description

We need to rebuild the list of fallbacks in late 2018 or early 2019.
We usually do this when 25% or more go down.
(This is tracked in #tor-bots on IRC.)

Here are the instructions for running a rebuild:
https://trac.torproject.org/projects/tor/wiki/doc/UpdatingFallbackDirectoryMirrors

Child Tickets

TicketStatusOwnerSummaryComponent
#24803newGenerate a new fallback list in 2018 and backport it to all supported versionsCore Tor/Fallback Scripts
#24804assignedphoulRun an opt-in process for relay operators to become fallbacksCore Tor/Fallback Scripts
#24805needs_revisionphoulUpdate fallback whitelist in late 2018Core Tor/Fallback Scripts
#24838newIgnore addresses in the fallback whitelistCore Tor/Fallback Scripts
#26502closedStop using the fallback blacklist, and delete itCore Tor/Fallback Scripts

Change History (15)

comment:1 Changed 6 months ago by nickm

Keywords: 034-roadmap-master added

The *minimal* version of this ticket -- whatever that entails! -- is on the 034 roadmap.

comment:2 Changed 6 months ago by nickm

Keywords: 034-triage-20180328 added

comment:3 Changed 6 months ago by nickm

Keywords: 034-included-20180328 added

comment:4 Changed 5 months ago by teor

I am very happy with the current set of fallbacks, which we generated in January 2018 using a minimum 90 day stability period.
(The previous stability period was 30 days, which was too low - it led to a rebuild after ~6 months,)

We have only lost 8 falllbacks after ~3 months:
https://consensus-health.torproject.org/graphs.html

At this rate, it will take us ~14 months to lose 25% of our 150 original fallbacks (or ~38 fallbacks).
But we also want to rebuild the list every 12 months, so let's rebuild in January 2019 at the latest, or 0.3.6:
https://trac.torproject.org/projects/tor/wiki/org/teams/NetworkTeam/CoreTorReleases#Current

If we're still doing well with fallbacks in 3-6 months' time, let's send out an email to thank all the fallback operators.

comment:5 Changed 5 months ago by teor

Milestone: Tor: 0.3.4.x-finalTor: 0.3.5.x-final

8/150 = 5% of fallbacks are currently down.
https://consensus-health.torproject.org/graphs.html
At this rate, it will be 2019 before we reach our 25% failure threshold.
So we don't need to do a fallback rebuild in 0.3.4.

comment:6 Changed 5 months ago by teor

Summary: Rebuild the fallback list for 0.3.4 or 0.3.5Rebuild the fallback list in 2018

comment:7 Changed 3 months ago by teor

Keywords: 031-unreached-backport added

0.3.1 is end of life, there are no more backports.
Tagging with 031-unreached-backport instead.

comment:8 Changed 3 months ago by teor

Keywords: 031-unreached-backport-maybe added; 031-backport-maybe removed

0.3.1 is end of life, there are no more 0.3.1 maybe backports.
Tagging with 031-unreached-backport-maybe.

comment:9 Changed 3 months ago by teor

Keywords: 031-unreached-backport removed

Keywords does substrings in search, but not replace.

comment:10 Changed 3 months ago by nickm

Keywords: 035-roadmap-master added; 029-backport 032-backport-maybe 033-backport 034-backport 034-roadmap-master 034-triage-20180328 034-included-20180328 031-unreached-backport-maybe removed

comment:11 Changed 3 months ago by teor

Keywords: 029-backport 032-backport-maybe 033-backport 034-backport 034-roadmap-master 034-triage-20180328 034-included-20180328 031-unreached-backport-maybe added

Restore keywords

comment:12 Changed 3 months ago by teor

Cc: teor@… removed

Remove useless CC

comment:13 Changed 2 months ago by nickm

Keywords: 035-triaged-in-20180711 added

comment:14 Changed 5 weeks ago by nickm

Milestone: Tor: 0.3.5.x-finalTor: 0.3.6.x-final

comment:15 Changed 5 weeks ago by teor

Milestone: Tor: 0.3.6.x-finalTor: 0.3.5.x-final

Hi Colin,

Our monitoring system checked the fallbacks this morning, and found that 25% of them are down. So we should generate a new list of fallbacks in the next few weeks.

Here's what needs to be done:

  • you can do a final call for opt-ins, if you'd like (#24803)
  • Do a final update to the whitelist (#24805)
  • I'll make the fallback script ignore addresses in the whitelist (#24838)
  • I can merge the changes from #24805, #24838, and #26502 into a branch (or master)
  • We can both run the fallback script from that branch to generate a fallback_dirs.inc file (#24803)
    • Last time, pastly and I ran the script, and then we merged our results together
  • I can write a changes file (#24803)
  • I can backport the list to src/or/fallback_dirs.inc in maint-0.2.9 (#24803)

The changes file looks like:

  o Minor features (fallback directory list):
    - Replace the 177 fallbacks originally introduced in Tor 0.2.9.8 in
      December 2016 (of which ~126 were still functional), with a list of
      151 fallbacks (32 new, 119 existing, 58 removed) generated in
      May 2017.
      Resolves ticket 21564.

But I think I might need to tweak the wording to pass make check-changes.

Note: See TracTickets for help on using tickets.