When outputting potential new fallbacks, blacklist the whitelist

When we look for potential new fallback directory mirrors, we want to ignore existing whitelisted fallbacks, as well as the blacklist.

changed milestone to %Tor: unspecified

added component::core tor/fallback scripts fallback milestone::Tor: unspecified owner::haxxpop points::0.5 priority::medium resolution::wontfix severity::normal status::closed type::enhancement labels

Milestone renamed

Trac:
Milestone: Tor: 0.2.??? to Tor: 0.3.???

Finally admitting that 0.3.??? was a euphemism for Tor: unspecified all along.

Trac:
Milestone: Tor: 0.3.??? to Tor: unspecified
Keywords: N/A deleted, tor-03-unspecified-201612 added

Trac:
Owner: N/A to haxxpop
Status: new to assigned

Remove an old triaging keyword.

Trac:
Keywords: tor-03-unspecified-201612 deleted, N/A added

Trac:
Component: Core Tor/Tor to Core Tor/Fallback Scripts

This is the old wiki text:

We want to only find new potential fallbacks, and ask their operators to opt-in. So we add both lists together, and tell the script not to select anything in those lists. (All those operators have already been contacted, and responded.)

i. Make sure the whitelist is the current one from git master (not the modified one from step 1) ii. Make sure the blacklist is saved in a git branch or a backup iii. Append the whitelist to the blacklist

cat scripts/maint/fallback.whitelist >> scripts/maint/fallback.blacklist

iv. Set the log level to INFO for both the script and stem:

- logging.basicConfig(level=logging.WARNING)
+ logging.basicConfig(level=logging.INFO)

- logging.getLogger('stem').setLevel(logging.WARNING)
+ logging.getLogger('stem').setLevel(logging.INFO)

v. Output candidate fallbacks (ignore the whitelist, log operator contact details):

- OUTPUT_CANDIDATES = False
+ OUTPUT_CANDIDATES = True

vi. Run the fallback selection script, saving the list:

scripts/maint/updateFallbackDirs.py > potential_extra_fallbacks 2> potential_extra_fallbacks.log

This can take a long time, as it downloads ~150MB of OnionOO data, parses it, then tries each fallback's DirPort.

We don't plan on contacting operators individually any more.

Trac:
Resolution: N/A to wontfix
Status: assigned to closed

closed

changed time estimate to 4h

mentioned in issue #20876 (moved)

moved to tpo/core/fallback-scripts#20192 (closed)

mentioned in issue tpo/core/fallback-scripts#20876 (closed)