Backport latest fallback list to 0.2.8 and 0.2.9

I'm generating a new list of fallback directory mirrors for release 0.2.9 in #18828 (moved).

We have a few options for dealing with the 0.2.8 list:

• comment-out any broken fallbacks,
• backport the 0.2.9 list,
• do nothing.

I would prefer to have the same src/or/fallback_dirs.inc in every (relevant) Tor release, assuming we do another 0.2.8 series release. This is consistent with how we handle directory authorities and geoip. Otherwise it becomes hard to check multiple fallback lists at once.

But I could be convinced to go with either of the other options.

changed milestone to %Tor: 0.2.8.x-final

added 028-backport 029-backport actualpoints::0.2 component::core tor/fallback scripts fallback milestone::Tor: 0.2.8.x-final owner::teor parent::18828 points::0.1 priority::medium resolution::implemented severity::normal status::closed type::defect labels

We should do the following things before backporting:

• inform the relay operators their relays have been selected for 0.2.9, and
• check the list before the 0.2.9 stable release to ensure the relays are still working.

Trac:
Parent: N/A to #20172 (moved)

I made a branch broken-028-fallbacks on my github. It comments out the broken 0.2.8 fallbacks, leaving 81/100. Another 10 are somewhat suspect (old versions, lost flags).

If this trend keeps going, we could be down to 50/100 by the end of the 2-year 0.2.8 release lifetime. We wanted to refresh the list when it reached 75/100.

So let's refresh both the 0.2.9 list and the 0.2.8 list. I don't think we need to delay any backport, because the new list will be larger and more reliable than the old one.

Trac:
Status: new to assigned
Owner: N/A to teor

In fact, let's do both:

• merge broken-028-fallbacks before the next 0.2.8 release, and
• update the list based on the 0.2.9 list when it's ready.

Replying to teor:

In fact, let's do both:

• merge broken-028-fallbacks before the next 0.2.8 release, and

Split off into #20190 (moved).

• update the list based on the 0.2.9 list when it's ready.

This ticket.

This is the task for backporting the fallback list to 0.2.8 and 0.2.9. The changes to the script, whitelist, and blacklist will be handled in #18828 (moved), and merged to master only.

There will soon be a draft list attached to #18828 (moved), I will revise it around the 17th of December, once relay operators have had a week and a half to send in updates and upgrade old versions.

Trac:
Version: Tor: 0.2.8.7 to Tor: 0.2.8.10
Status: assigned to needs_revision
Summary: Backport 0.2.9 fallback list to 0.2.8? to Backport latest fallback list to 0.2.8 and 0.2.9
Parent: #20172 (moved) to #18828 (moved)
Keywords: N/A deleted, 029-backport added

I've built a draft fallback list using the branch in #18828 (moved). I'll attach the logs to this ticket.

The list was generated using the instructions in: https://trac.torproject.org/projects/tor/wiki/doc/UpdatingFallbackDirectoryMirrors#b.GeneratingtheList

Around 17 December, I will update the final list, and we can backport it to 0.2.8 and 0.2.9.

If someone wants to run a version of tor with the new fallback list, I'd love a review.

The current draft branch with the list is new-fallbacks-028 on my github: https://github.com/teor2345/tor/blob/new-fallbacks-028/src/or/fallback_dirs.inc

Trac:
Status: needs_revision to needs_review
Version: Tor: 0.2.8.10 to N/A

Trac:
draft_fallback_dirs_20161207.log

Fallback selection log for draft 029 fallbacks on 20161207

We could also switch to 3 fallbacks per operator to get ~173 fallbacks, see #20914 (moved)

Latest branch is new-fallbacks-028-v2, I'll update it next weekend as well.

Trac:
Status: needs_review to needs_revision

The latest fallback list is in the branch new-fallbacks-028-20161219 on my github. It is based on maint-0.2.8 (I think around 0.2.8.9, but it should still merge cleanly).

Trac:
Actualpoints: N/A to 0.2
Status: needs_revision to merge_ready

(The log for this list is https://trac.torproject.org/projects/tor/attachment/ticket/18828/fallback_dirs_final_201612190411_fcf19f8b545a30a6cc9e6adde66e8ca2cb6b3bca.log )

Merged to 0.2.8 and forward!

Trac:
Resolution: N/A to implemented
Status: merge_ready to closed

Put all fallback script issues in the fallback scripts component

Trac:
Component: Core Tor/Tor to Core Tor/Fallback Scripts

closed

changed time estimate to 48m

added 1h 36m of time spent

mentioned in issue #20190 (moved)

mentioned in issue #20242 (moved)

mentioned in issue #20243 (moved)

mentioned in issue tpo/core/tor#20243 (closed)

moved to tpo/core/fallback-scripts#20170 (closed)