Opened 9 months ago

Closed 18 hours ago

#21619 closed enhancement (fixed)

Flag fallback directory mirrors in Atlas

Reported by: teor Owned by: irl
Priority: Medium Milestone:
Component: Metrics/Atlas Version:
Severity: Normal Keywords: metrics-2018
Cc: Actual Points:
Parent ID: Points:
Reviewer: Sponsor:

Description

Tor keeps a list of fallback directory mirrors in its source code.

consensus-health flags these mirrors using the synthetic flag "FallbackDir".

Could Atlas do something like this, too?
(Like the Not Recommended flag?)

The list is updated every 2-6 months:
https://gitweb.torproject.org/tor.git/tree/src/or/fallback_dirs.inc

And then parsed and included in stem in a friendlier format:
https://gitweb.torproject.org/stem.git/tree/stem/descriptor/fallback_directories.cfg

Child Tickets

Change History (12)

comment:1 in reply to:  description ; Changed 9 months ago by cypherpunks

Replying to teor:

Could Atlas do something like this, too?
(Like the Not Recommended flag?)

To add "Not Recommended" Atlas simply made use of onionoo boolean recommended_version, but there is no fallback flag in onionoo, so this would probably require a onionoo change first unless atlas keeps this list out of band.

comment:2 Changed 9 months ago by atagar

For what it's worth if you want the current list of fallback directories in the tor source here's what you can call (it pulls the list from gitweb)...

https://stem.torproject.org/api/descriptor/remote.html#stem.descriptor.remote.FallbackDirectory.from_remote

comment:3 in reply to:  1 Changed 9 months ago by teor

Replying to cypherpunks:

Replying to teor:

Could Atlas do something like this, too?
(Like the Not Recommended flag?)

To add "Not Recommended" Atlas simply made use of onionoo boolean recommended_version, but there is no fallback flag in onionoo, so this would probably require a onionoo change first unless atlas keeps this list out of band.

Thanks, opened #21620 for this.

comment:4 Changed 9 months ago by irl

Also useful for this would probably be to add "last_changed_address_or_port" to the details pages for relays. Probably in the 3rd column.

comment:5 in reply to:  4 Changed 9 months ago by cypherpunks

Replying to irl:

Also useful for this would probably be to add "last_changed_address_or_port" to the details pages for relays. Probably in the 3rd column.

This should be a separate ticket although i can't see the usefulness of adding it to the details page. The field seems more suitable for the script that generates the fallback list.

comment:6 Changed 8 weeks ago by karsten

Keywords: metrics-2018 added

comment:7 Changed 8 weeks ago by karsten

Owner: changed from irl to metrics-team
Status: newassigned

comment:8 Changed 20 hours ago by irl

Owner: changed from metrics-team to irl
Status: assignedaccepted

comment:9 Changed 20 hours ago by irl

Resolution: fixed
Status: acceptedclosed

This has been fixed in 0d0871c.

Future updates to the directory mirrors will mean that a Python script needs to be run:

python3 scripts/fallback_dir.py > js/fallback_dir.js

This script fetches the list from the tor source code and generates a JavaScript file containing a function for determining whether or not there a relay is a fallback directory.

comment:10 Changed 19 hours ago by teor

I have added instructions to inform the relay search maintainers and the steam maintainer when the fallback directory list changes:
https://trac.torproject.org/projects/tor/wiki/doc/UpdatingFallbackDirectoryMirrors

I wonder if we should automate this somehow.

comment:11 Changed 19 hours ago by atagar

Resolution: fixed
Status: closedreopened

Hi Tim. Stem is able to pull the up-to-date list of fallbacks from tor...

https://gitweb.torproject.org/stem.git/tree/stem/descriptor/remote.py#n1008

Obviously a hack using gitweb but hey, it works.

On Stem's side I don't really need to be notified. As for Atlas why not have it simply use the above method to get them. It could cache the data for, say, a day. Then you wouldn't need to worry about notifying folks at all since Atlas would auto-update itself.

Gonna reopen to get irl's thoughts.

comment:12 Changed 18 hours ago by irl

Resolution: fixed
Status: reopenedclosed

In #23518 this would become possible. Currently Relay Search is static HTML and JavaScript and so does require manual updates. (Sorry, I discussed this with teor in IRC but didn't update this ticket, but I did update #23518).

It may be that in the future this information comes directly from Onionoo too, meaning that Relay Search doesn't need to know about it at all.

I'll add this to the agenda for the next Metrics Team meeting to discuss.

If there are updates before #23518 is implemented, it's just a case of running:

https://gitweb.torproject.org/user/irl/atlas.git/tree/scripts/fallback_dir.py

git add, commit, push and then deploy to the static mirrors.

Note: See TracTickets for help on using tickets.