Ticket #22755: stem_descriptor_creation.patch

File stem_descriptor_creation.patch, 8.5 KB (added by atagar, 17 months ago)

Patch file for using stem to create test descriptors.

  • README.rst

    From 74fc3c538f7c473af05d351a444635f45aae86c0 Mon Sep 17 00:00:00 2001
    From: Damian Johnson <atagar@torproject.org>
    Date: Wed, 28 Jun 2017 09:58:22 -0700
    Subject: [PATCH] Use stem to create test descriptors
    
    Stem now provides the ability to create and sign descriptors...
    
      https://stem.torproject.org/tutorials/mirror_mirror_on_the_wall.html#can-i-create-descriptors
    
    Unfortunately I didn't manage to get bridgedb's tests running but its use of
    leekspin looks pretty simple. Replacing it with Stem. There was a comment
    warning that descriptor creation takes a while but for what it's worth that
    doesn't seem to be the case. I'm running on an antiquated ten year old desktop
    and I can generate nineteen signed descriptors a second...
    
      atagar@odin:~/Desktop/tor/bridgedb$ time ./scripts/create-descriptors 250
    
      real  0m13.087s
      user  0m12.753s
      sys 0m0.320s
    
      atagar@odin:~/Desktop/tor/bridgedb$ cat test_descriptors/descriptor_15
      router Unnamed2006509655 194.39.149.152 9001 0 0
      published 2005-06-15 02:26:44
      bandwidth 153600 256000 104590
      reject *:*
      onion-key
      -----BEGIN RSA PUBLIC KEY-----
      NGExNDZmMDU2YWRkYWUwODhiM2ZkNjg3NmNhNDM0YmMzYjc5ZmE5MDViNTRkYjA1
      OGMwOGJmNTNmM2NiNzg3NTRjNTdjMDA1MmZlY2QxNzc5YTAwNTQ1OWYyNmRiN2Jm
      YWFkM2RlNDRhOTQ1ZmRiZjExYTdmOTE4NzkyMTIwNTQ2YzY1NTAzNmM1MDM=
      -----END RSA PUBLIC KEY-----
      signing-key
      -----BEGIN RSA PUBLIC KEY-----
      MIGJAoGBALS1NGORgH3xEVOSd5Zs92z3bcrvjEK6WMYNkUiFUVvi7HhEFOT7vinP
      vxUtPRakjDD7Kxg8LbL7PtyEtm29uSL9ggzZTZIHmMd4dIqAYSVfuN2wj/Stk5qp
      2286BrmhuWyU/ZKLZXrz3g3JOkx9feJ9UxaxGeTGLK626WwMKuHjAgMBAAE=
      -----END RSA PUBLIC KEY-----
      router-signature
      -----BEGIN SIGNATURE-----
      pIEyuX8oUjMu2W1bqZ1KagpKDlRxMGBH8rupoWIuIjkvfBXC4goch9MHsa3S/Mf3
      0Tb0PWS/KVFnQyTIPPIi5+iV5J/9ol87HPmrnCP4yP3NlE/nnlpbIOfuB+vdZv6n
      BApDNLkWuuZCirNOqeps6BVBsH58v35nQoUY+HYOK60=
      -----END SIGNATURE-----
    ---
     README.rst                 |    6 +-----
     bridgedb/Main.py           |    9 ++-------
     bridgedb/runner.py         |   38 +++++++-------------------------------
     doc/HACKING.md             |   11 ++++-------
     scripts/create-descriptors |   29 +++++++++++++++++++++++++++++
     setup.py                   |    2 +-
     6 files changed, 44 insertions(+), 51 deletions(-)
     create mode 100755 scripts/create-descriptors
    
    diff --git a/README.rst b/README.rst
    index faeae5c..49ebe43 100644
    a b To create a bunch of fake bridge descriptors to test BridgeDB, do:: 
    314314
    315315      bridgedb mock [-n NUMBER_OF_DESCRIPTORS]
    316316
    317 Note that you will need to install
    318 `leekspin <https://pypi.python.org/pypi/leekspin>`__ in order to run the
    319 ``bridgedb mock`` command. See ``doc/HACKING.md`` for details.
    320 
    321 And finally, to run the test suites, do::
     317To run the test suites, do::
    322318
    323319      make coverage
    324320
  • bridgedb/Main.py

    diff --git a/bridgedb/Main.py b/bridgedb/Main.py
    index 4990904..e095826 100644
    a b def runSubcommand(options, config): 
    524524    # mentioned above with the email.server and https.server.
    525525    from bridgedb import runner
    526526
    527     statuscode = 0
    528 
    529527    if options.subCommand is not None:
    530528        logging.debug("Running BridgeDB command: '%s'" % options.subCommand)
    531529
    532530        if 'descriptors' in options.subOptions:
    533             statuscode = runner.generateDescriptors(
    534                 options.subOptions['descriptors'], config.RUN_IN_DIR)
     531            runner.generateDescriptors(options.subOptions['descriptors'], config.RUN_IN_DIR)
    535532
    536         logging.info("Subcommand '%s' finished with status %s."
    537                      % (options.subCommand, statuscode))
    538         sys.exit(statuscode)
     533        sys.exit(0)
  • bridgedb/runner.py

    diff --git a/bridgedb/runner.py b/bridgedb/runner.py
    index 597b1b2..dbd921c 100644
    a b def find(filename): 
    8181def generateDescriptors(count=None, rundir=None):
    8282    """Run a script which creates fake bridge descriptors for testing purposes.
    8383
    84     This will run Leekspin_ to create bridge server descriptors, bridge
    85     extra-info descriptors, and networkstatus document.
    86 
    87     .. warning: This function can take a very long time to run, especially in
    88         headless environments where entropy sources are minimal, because it
    89         creates the keys for each mocked OR, which are embedded in the server
    90         descriptors, used to calculate the OR fingerprints, and sign the
    91         descriptors, among other things.
    92 
    93     .. _Leekspin: https://gitweb.torproject.org/user/isis/leekspin.git
    94 
    9584    :param integer count: Number of mocked bridges to generate descriptor
    9685        for. (default: 3)
    9786    :type rundir: string or None
    def generateDescriptors(count=None, rundir=None): 
    10089        directory MUST already exist, and the descriptor files will be created
    10190        in it. If None, use the whatever directory we are currently in.
    10291    """
    103     import subprocess
    104     import os.path
    10592
    106     proc = None
    107     statuscode = 0
    108     script = 'leekspin'
    109     rundir = rundir if os.path.isdir(rundir) else None
     93    from stem.descriptor.server_descriptor import RelayDescriptor
     94
    11095    count = count if count else 3
    111     try:
    112         proc = subprocess.Popen([script, '-n', str(count)],
    113                                 close_fds=True, cwd=rundir)
    114     finally:
    115         if proc is not None:
    116             proc.wait()
    117             if proc.returncode:
    118                 print("There was an error generating bridge descriptors.",
    119                       "(Returncode: %d)" % proc.returncode)
    120                 statuscode = proc.returncode
    121             else:
    122                 print("Sucessfully generated %s descriptors." % str(count))
    123         del subprocess
    124         return statuscode
     96    rundir = rundir if rundir else os.getcwd()
     97
     98    for i in range(count):
     99      with open(os.path.join(rundir, 'descriptor_%i' % i), 'w') as descriptor_file:
     100        descriptor_file.write(RelayDescriptor.content(sign = True))
    125101
    126102def doDumpBridges(config):
    127103    """Dump bridges by assignment to a file.
  • doc/HACKING.md

    diff --git a/doc/HACKING.md b/doc/HACKING.md
    index 22f9b2b..f923039 100644
    a b with password ```writecode```. 
    1212## Generating bridge descriptors
    1313
    1414Developers wishing to test BridgeDB will need to generate mock bridge
    15 descriptors. This is accomplished through the [leekspin
    16 script](https://gitweb.torproject.org/user/isis/leekspin.git). To generate 20
    17 bridge descriptors, change to the bridgedb running directory and do:
     15descriptors. This is accomplished through **create-descriptors.py**. To
     16generate 20 bridge descriptors, change to the bridgedb running directory and
     17run:
    1818
    19     $ leekspin -n 20
     19    $ ./scripts/create-descriptors.py 20
    2020
    2121It is recommended that you generate at least 250 descriptors for testing.
    2222Ideally, even more descriptors should be generated, somewhere in the realm of
    23232000, as certain bugs do not emerge until BridgeDB is processing thousands of
    2424descriptors.
    2525
    26 **Leekspin is for testing purposes only and should never be deployed on a
    27 production server.** We do not want to distribute fake bridges.
    28 
    2926## Git Workflow
    3027
    3128See this article on git branching [workflow][workflow]. The only modifications
  • new file scripts/create-descriptors

    diff --git a/scripts/create-descriptors b/scripts/create-descriptors
    new file mode 100755
    index 0000000..e6711c1
    - +  
     1#!/usr/bin/env python
     2
     3import os
     4import sys
     5
     6try:
     7  import stem
     8  from stem.descriptor.server_descriptor import RelayDescriptor
     9except ImportError:
     10  print('Creating descriptors requires stem (https://stem.torproject.org/)')
     11  sys.exit(1)
     12
     13if not hasattr(RelayDescriptor, 'content'):
     14  print('This requires stem version 1.6 or later, you are running version %s' % stem.__version__)
     15  sys.exit(1)
     16
     17if len(sys.argv) < 2 or not sys.argv[1].isdigit():
     18  print('We need a numeric argument indicating how many descriptors to make.')
     19  sys.exit(1)
     20
     21output_dir = os.path.join(os.getcwd(), 'test_descriptors')
     22
     23if not os.path.exists(output_dir):
     24  os.mkdir(output_dir)
     25
     26for i in range(int(sys.argv[1])):
     27  with open(os.path.join(output_dir, 'descriptor_%i' % i), 'w') as descriptor_file:
     28    descriptor_file.write(RelayDescriptor.content(sign = True))
     29
  • setup.py

    diff --git a/setup.py b/setup.py
    index 97dee01..d772f28 100644
    a b setuptools.setup( 
    382382             'scripts/get-tor-exits'],
    383383    extras_require={'test': ["sure==1.2.2",
    384384                             "coverage==4.2",
    385                              "leekspin==1.1.4"]},
     385                             "cryptography==1.9"]},
    386386    zip_safe=False,
    387387    cmdclass=get_cmdclass(),
    388388    include_package_data=True,