Update fallback whitelist in late 2018

We need to update the list using:

• operator emails from 2018
• the opt-in process in #24804 (moved)
• running the script, finding change warnings, contacting operators, and updating their details we'll do #24838 (moved) instead

changed milestone to %Tor: 0.4.0.x-final

added 034-removed-20180328 034-triage-20180328 035-removed-20180711 035-roadmap component::core tor/tor fallback milestone::Tor: 0.4.0.x-final owner::phoul parent::24786 points::0.5 priority::medium resolution::fixed s8-bootstrap severity::normal sponsor::8-can status::closed type::enhancement labels

Trac:
Points: N/A to 2
Type: defect to enhancement

An operator sent me an email asking me to add this relay to the whitelist:

https://atlas.torproject.org/#details/BC6B2E2F62ACC5EDECBABE64DA1E48F84DD98B78

It's just changed DirPort, so it wouldn't be in the January 2018 list anyway. (Fallbacks need to have the same address and port for 90 days.)

An operator moved from 217.79.190.25 to 37.157.255.35 and sent this email: https://lists.torproject.org/pipermail/tor-relays/2018-January/014055.html

Three operators sent emails to tor-relays:

https://lists.torproject.org/pipermail/tor-relays/2018-January/014063.html

92412EA1B9AA887D462B51D816777002F4D58907 360CBA08D1E24F513162047BDB54A1015E531534

https://lists.torproject.org/pipermail/tor-relays/2018-January/014064.html

A2A6616723B511D8E068BB71705191763191F6B2

https://lists.torproject.org/pipermail/tor-relays/2018-January/014069.html

E51620B90DCB310138ED89EDEDD0A5C361AAE24E

And one sent an email to me directly, saying their relay's details had changed to:

BC7ACFAC04854C77167C7D66B7E471314ED8C410 144.76.75.184:9001 [2a01:4f8:191:93a2::4]:9001

An operator sent an email to me directly, opting-in this relay:

F2DFE5FA1E4CF54F8E761A6D304B9B4EC69BDAE8

An operator wants this relay taken off the whitelist and put on the blacklist:

855BC2DABE24C861CD887DB9B2E950424B49FC34

I moved it from the blacklist to the whitelist last release by mistake.

An operator wants this relay taken off the whitelist and put on the blacklist:

3B33F6FCA645AD4E91428A3AF7DC736AD9FB727B

It is being shut down.

An operator wants to opt-in: https://lists.torproject.org/pipermail/tor-relays/2018-January/014267.html

D122094E396DF8BA560843E7B983B0EA649B7DF9

An operator wants to opt-in: https://lists.torproject.org/pipermail/tor-relays/2018-January/014276.html

32828476F4F84E15C42B4C360A5CD8DE4C3C2BE7

An operator wants this relay taken off the whitelist and put on the blacklist: 0BEA4A88D069753218EAAAD6D22EA87B9A1319D6 It is being shut down. See #25169 (moved).

An operator wants this relay taken off the whitelist and put on the blacklist: 9FBEB75E8BC142565F12CBBE078D63310236A334 It is being shut down. See #25414 (moved).

Removal for 328E54981C6DDD7D89B89E418724A4A7881E3192 , see #25436 (moved)

Trac:
Keywords: N/A deleted, 034-triage-20180328 added

Per our triage process, these tickets are pending removal from 0.3.4.

Trac:
Keywords: N/A deleted, 034-removed-20180328 added

This ticket is part of the minimal fallback directory refresh for 0.3.4 or 0.3.5.

I can do it, or pastly can do it, or some volunteer may do it.

Edit: autocorrect

Trac:
Owner: N/A to teor
Status: new to assigned

An operator wants the IP address of 6A7551EEE18F78A9813096E82BF84F740D32B911 changed to 94.130.186.5.

They posted to tor-relays, but somehow the message didn't make it into the archives, possibly because the subject was very long.

A relay operator is shutting down these two fallbacks: 175921396C7C426309AB03775A9930B6F611F794 1DBAED235E3957DE1ABD25B4206BE71406FB61F8

https://lists.torproject.org/pipermail/tor-relays/2018-April/014944.html

These tickets, tagged with 034-removed-*, are no longer in-scope for 0.3.4. We can reconsider any of them, if time permits.

Trac:
Milestone: Tor: 0.3.4.x-final to Tor: unspecified

This relay can be added to the whitelist of FallbackDir. https://metrics.torproject.org/rs.html#details/924B24AFA7F075D059E8EEB284CC400B33D3D036

Trac:
Username: crazysane

An operator wants treadstone (185.129.249.124) removed from the whitelist, see #25798 (moved).

This ticket is in scope for 0.3.4 or 0.3.5 as part of the minimal fallback rebuild task.

We will do a rebuild when 25% of fallbacks are down, which is 37/150: https://consensus-health.torproject.org/graphs.html#fallbackdirstatus

Trac:
Points: 2 to 0.5
Milestone: Tor: unspecified to Tor: 0.3.4.x-final

Instead of updating fallback details in the whitelist, we should do #24838 (moved) and ignore them. We believe onionoo, and onionoo checks stability anyway.

Trac:
Description: We need to update the list using:

• operator emails from 2018
• the opt-in process in #24804 (moved)
• running the script, finding change warnings, contacting operators, and updating their details

to

We need to update the list using:

• operator emails from 2018
• the opt-in process in #24804 (moved)
• running the script, finding change warnings, contacting operators, and updating their details we'll do #24838 (moved) instead

Anyone can do this

Trac:
Owner: teor to N/A

8/150 = 5% of fallbacks are currently down. https://consensus-health.torproject.org/graphs.html At this rate, it will be 2019 before we reach our 25% failure threshold. So we don't need to do a fallback rebuild in 0.3.4.

Trac:
Milestone: Tor: 0.3.4.x-final to Tor: 0.3.5.x-final

An operator wants 774555642FDC1E1D4FDF2E0C31B7CA9501C5C9C7 removed from the whitelist: https://lists.torproject.org/pipermail/tor-relays/2018-May/015146.html

An operator wants the following relay removed:

Nickname: DigiGesTor2e1
Fingerprint: 88487BDD980BF6E72092EE690E8C51C0AA4A538C
IP: 176.10.104.243

...
It will be shutdown end of June 2018.

They want to add this relay instead:

Nickname: DigiGesTor3e1
Fingerprint:9AA3FF35E7A549D2337E962333D366E102FE4D50
IP:94.230.208.147
IPv6:2a02:418:6017::147

Trac:
Summary: Update fallback whitelist and blacklist to Update fallback whitelist and blacklist in late 2018

Trac:
Cc: teor@torproject.org to teor@torproject.org, phoul

An operator wants their relay removed from the whitelist: https://lists.torproject.org/pipermail/tor-dev/2018-June/013207.html

An operator wants E947C029087FA1C3499BEF5D4372947C51223D8F added to the fallback list.

Phoul has a draft commit that includes all of the relays above (except for those that don't have DirPorts): https://github.com/Phoul/tor/commit/306ab8ae17b8f39a6b83bcb24fe02afd8389dc13

Another draft has been created with the current operators who have responded to the mailing list post at https://lists.torproject.org/pipermail/tor-relays/2018-June/015521.html and Twitter postings:

https://github.com/Phoul/tor/commit/23d9ffcdf4877f574278b0f0b39ce511072898af

That branch has 20 new and 2 changed IPv4 addresses.

Our target is >30% of the final count having new or changed IPv4 addresses, we currently have a best case scenario of: (2 + 20)/(150 + 20) = 13%.

If we have 170 fallbacks, we can also increase the limit per operator to 170 x 5% = 8 relays. That should gain us 5 more relays from existing large operators, which gives us 15% new or changed IPv4 addresses.

Remove useless CC

Trac:
Cc: teor@torproject.org, phoul to phoul

I have just pushed another commit with 9 more relays added to the fallback list. I've also followed up with large relay operators today to see if they have any more to offer us.

https://github.com/Phoul/tor/commit/d7e7c0d20d15d90882bc0b963f897a99b5d7585a

(there is a spacing issue in this commit on the final line, the following commit fixes it)

Replying to teor:

That branch has 20 new and 2 changed IPv4 addresses.

Our target is >30% of the final count having new or changed IPv4 addresses, we currently have a best case scenario of: (2 + 20)/(150 + 20) = 13%.

If we have 170 fallbacks, we can also increase the limit per operator to 170 x 5% = 8 relays. That should gain us 5 more relays from existing large operators, which gives us 15% new or changed IPv4 addresses.

These tickets are being triaged out of 0.3.5. The ones marked "035-roadmap-proposed" may return.

Trac:
Keywords: N/A deleted, 035-removed-20180711 added
Milestone: Tor: 0.3.5.x-final to Tor: unspecified

https://github.com/Phoul/tor/commit/855feab70d35dd5093aae1f27d6cf19ee6cb3338 3 more fallback directory mirrors added, a number of operators have been contacted to see if they would like any others from their relay families added.

3 more relays added from EmeraldOnion https://github.com/Phoul/tor/commit/54cf36400669c555a3af84b167fd79f19b9ebbe3

Replying to nickm:

These tickets are being triaged out of 0.3.5. The ones marked "035-roadmap-proposed" may return.

Oops, this is on the 0.3.5 roadmap, if:

• we reach the 30% changed/new relays threshold, so the list can help with censorship
• we reach the 25% down relays threshold in consensus health

Here's how we're progressing:

The branch now has ~35 new and ~2 changed IPv4 addresses.

Our target is >30% of the final count having new or changed IPv4 addresses, we currently have a best case scenario of: (2 + 35)/(150 + 35) = 20%.

If we have 185 fallbacks, we can also increase the limit per operator to 185 x 5% = 9 relays. That should gain us 10 more relays from existing large operators, which gives us 24% new or changed IPv4 addresses.

20/150 = 13% of fallbacks are down.

Trac:
Keywords: N/A deleted, 035-roadmap added

Make everything that is assigned to no-one new again.

Trac:
Status: assigned to new

On Monday I sent out another round of emails to large operators asking for fallback mirrors, today I received 1 more and heard from other operators they are considering which to add.

I have added the new one to https://github.com/Phoul/tor/commit/0260b720053717cdc348a51ede1d7e51e7ea2741.

Trac:
Owner: N/A to phoul
Status: new to assigned

2 new relays have been added to https://github.com/Phoul/tor/commit/9aceb928392ef8f3e2f737c14702a9cc330bd109

One fallback has been terminated by the network provider: https://lists.torproject.org/pipermail/tor-relays/2018-August/015886.html

Replying to teor:

One fallback has been terminated by the network provider: https://lists.torproject.org/pipermail/tor-relays/2018-August/015886.html

Thanks for adding this to the ticket, I have updated the repo at https://github.com/Phoul/tor/commit/51f8db1d92d64331d000d2847edb8b66e9fb4c0c

Since #26502 (moved), there is no blacklist any more.

Trac:
Milestone: Tor: unspecified to Tor: 0.3.5.x-final
Summary: Update fallback whitelist and blacklist in late 2018 to Update fallback whitelist in late 2018
Status: assigned to needs_revision

25% of fallbacks are failing, so these tickets need to be done in 0.3.5.

Phoul, see also #27297 (moved).

Replying to teor:

Phoul, see also #27297 (moved).

This relay has been added at https://github.com/Phoul/tor/commit/f0792537e9cf25e76b5cab16635de7b78bfbf8b7