Opened 11 months ago

Closed 11 months ago

Last modified 5 months ago

#24706 closed enhancement (fixed)

Add a script that creates fallback directory whitelist entries

Reported by: teor Owned by: teor
Priority: Medium Milestone: Tor: 0.3.3.x-final
Component: Core Tor/Fallback Scripts Version:
Severity: Normal Keywords: review-group-28
Cc: Actual Points: 0.4
Parent ID: #22271 Points: 0.2
Reviewer: pastly Sponsor:

Description

I've finally got sick of doing these by hand.

Thanks to John Ricketts for inspiring this change, by sending me a list of ~30 potential fallbacks.

Child Tickets

Change History (15)

comment:1 Changed 11 months ago by teor

Actual Points: 0.20.3
Parent ID: #22321#22271
Status: assignedneeds_review

Please see my branch ticket24706, which adds two fallback helper scripts:

  • Add the generateFallbackDirLine.py script for automatically generating fallback directory mirror lines from relay fingerprints. No more typos!
  • Add the lookupFallbackDirContact.py script for automatically looking up operator contact info from relay fingerprints.

These scripts are written in python using stem.

Edit: fix formatting

Last edited 11 months ago by teor (previous) (diff)

comment:2 Changed 11 months ago by atagar

Hi Tim. Apologies, but mind providing a link for your git repo? Not spotting any 'teor' repos on https://gitweb.torproject.org/ so guess you use github.

comment:3 Changed 11 months ago by atagar

Status: needs_reviewneeds_information

comment:4 Changed 11 months ago by teor

Status: needs_informationneeds_review

It's at https://github.com/teor2345/tor.git

I would appreciate your review on these scripts.
But they are intended for Core Tor's scripts/maint directory, beside the existing updateFallbackDirs.py script.

comment:5 Changed 11 months ago by atagar

Thanks Tim. For other's reference here's the link.

Your generateFallbackDirLine.py looks good, but I'm unsure why you're downloading the full consensus rather than just fetching this from the server descriptor. Doing do does the same, is simpler, and runs far faster.

Otherwise looks good to me!

#!/usr/bin/env python

# Generate a fallback directory whitelist/blacklist line for every fingerprint
# passed as an argument.
#
# Usage:
# generateFallbackDirLine.py fingerprint ...

import sys 
import urllib2

import stem.descriptor.remote
import stem.util.tor_tools

if len(sys.argv) <= 1:
  print('Usage: %s fingerprint ...' % sys.argv[0])
  sys.exit(1)

for fingerprint in sys.argv[1:]:
  if not stem.util.tor_tools.is_valid_fingerprint(fingerprint):
    print("'%s' isn't a valid relay fingerprint" % fingerprint)
    sys.exit(1)

  try:
    desc = stem.descriptor.remote.get_server_descriptors(fingerprint).run()[0]
  except urllib2.HTTPError as exc:
    if exc.code == 404:
      print('# %s not found in current consensus' % fingerprint)
      continue
    else:
      raise

  if not desc.dir_port:
    print("# %s needs a DirPort" % fingerprint)
  else:
    ipv6_addresses = [(address, port) for address, port, is_ipv6 in desc.or_addresses if is_ipv6]
    ipv6_field = ' ipv6=[%s]:%s' % ipv6_addresses[0] if ipv6_addresses else ''
    print('%s:%s orport=%s id=%s%s # %s' % (desc.address, desc.dir_port, desc.or_port, fingerprint, ipv6_field, desc.nickname))

PS. Edited since I didn't catch that you wanted to accept multiple fingerprints.

Last edited 11 months ago by atagar (previous) (diff)

comment:6 Changed 11 months ago by atagar

Oh, I should have asked - do you intend for this to be invoked with just a few fingerprints or a bunch? If you're gonna resolve a ton then making a batch get_server_descriptors() request would of course be faster. Initially thought you just wanted one which is why I wrote it this way.

comment:7 Changed 11 months ago by teor

Actual Points: 0.30.4
Status: needs_reviewneeds_revision

A bunch of fingerprints (up to 30 in the case that triggered the ticket).
But the typical case is 1-5.

I'm not sure about the consensus / descriptor tradeoff.
If the relay is down right now, then it won't be in the consensus, but its descriptor might be in some directory mirrors' caches.
(But it does cause a bit of a delay in finding out that it's down.)
I think this is ok, because our final checks make sure the relay is up.

I'll fix the warning message (s/current consensus/recent descriptors/) and update the branch with this patch.

comment:8 Changed 11 months ago by teor

Reviewer: atagar
Status: needs_revisionmerge_ready

Please see the updated commit on my branch ticket24706 at https://github.com/teor2345/tor/commit/fd1906ade3f14bca619c5d3090a6cdf611fdf769

I fixed some trailing whitespace and revised the error message.

The descriptor method seems to identify down relays ok, so I think we are good here.

Thanks for the review!

comment:9 Changed 11 months ago by atagar

Thanks! Looks good to me. I don't have push access for the core tor repo so Nick or someone will need to do the merge.

comment:10 Changed 11 months ago by nickm

Keywords: review-group-28 added

comment:11 Changed 11 months ago by pastly

Tested both scripts. Look good to me!

comment:12 Changed 11 months ago by teor

This code is now in my branch fallback-code-2018-01 at https://github.com/teor2345/tor.git

comment:13 Changed 11 months ago by teor

Reviewer: atagarpastly

pastly reviewed all of these

comment:14 Changed 11 months ago by teor

Resolution: fixed
Status: merge_readyclosed

This branch has been merged, so these tickets are now implemented,

comment:15 Changed 5 months ago by teor

Cc: teor@… removed

Remove useless Cc

Note: See TracTickets for help on using tickets.