wiki:doc/TorChutneyGuide

Tor Test Networks using Chutney: A Guide

A guide to creating Tor test networks with chutney. (This page is an early draft outline of the documentation and functionality we want to have.)

Features which are not yet implemented are marked NYI, with a corresponding Trac ticket number and link (if it exists).

Chutney README

The chutney README gives detailed descriptions and command-line examples for each chutney feature:

https://gitweb.torproject.org/chutney.git/tree/README

Testing an installed Tor version

Run a Single Test

Get the latest version of chutney:

git clone https://git.torproject.org/chutney.git

If you already have tor installed, you can run a test network:

chutney/tools/test-network.sh

Testing while Building Tor from Source

Run a Single Test

git clone https://git.torproject.org/tor.git

git clone https://git.torproject.org/chutney.git

cd tor

sh autogen.sh

./configure

make test-network


The ./configure step will fail if there are missing libraries, and will indicate which those are. Run ./configure again after installing all the necessary libraries.

Setting the CHUTNEY_PATH environmental variable

Recent versions of Tor (0.2.7.3-alpha (?) and later) will autodetect the $CHUTNEY_PATH if the chutney directory is next to the tor directory (#16903).

If your tor and chutney source directories are not side-by-side, you'll neeed to set the CHUTNEY_PATH environmental variable to the path to the chutney directory (not the script itself).

You can do this for one command:

Or while the shell is running:

Or for all future shells:

Test Functionality

Recent versions of Tor (0.2.7.3-alpha (?) and later) run a comprehensive test of most functionality, including authorities, bridges, clients, exits, and hidden services (but no IPv6 for compatibility reasons). See tickets #16945 (tor) & #16946 (chutney). Older versions of Tor run a simpler test of authorities, clients, and exits.

Resolving Test Failures

Chutney occasionally fails to bootstrap or verify. Sometimes, and on some platforms / machines, bootstrap is slower than we expect (25 seconds in 0.2.6 - 0.2.7.2-alpha), causing intermittent failures.

Recent tor versions (#16952, merge pending, 0.2.7.3-alpha?):

  • Retry boostrap (configure and launch) once on failure, configurable via $CHUTNEY_RECONFIGURES or --reconfigures
  • Retry verify every 5 seconds from $BOOTSTRAP_TIME (now 20s) to $MAX_BOOTSTRAP_TIME (60s), configurable via --time and --max-time

These changes make src/test/test-network.sh robust against intermittment failures.

However, sometimes chutney fails to launch due to a persistent error. The most common issue is having a previous chutney network active. Old chutney networks are typically killed by chutney/tools/bootstrap-network.sh using chutney stop, but sometimes this doesn't happen. The tor processes launched by chutney can be found using a command like ps auxwww | grep chutney | grep tor. Ensure each pid doesn't belong to a process you want, like a system tor or an unrelated process, then kill each chutney tor pid using kill. Alternately, each tor instance's pid files can be found in the chutney/net/nodes directory. (Old nodes directories are renamed using an increasing numeric timestamp as a suffix.)

NYI: Better yet, make these instructions part of the chutney failure message when some tor processes fail to launch. (#17002)

Run Comprehensive Tests

git clone https://git.torproject.org/tor.git

git clone https://git.torproject.org/chutney.git

cd tor

sh autogen.sh

./configure

make test-network-all


make test-network-all runs comprehensive tests. It reports results PASS / SKIP / FAIL like the existing make check output. (#16953)

Base Test Suite:

  • basic-min: authorities, exit relays, exit client
  • bridges-min: authorities, bridge authority, bridge, bridge-exit client
  • hs-min: authorities, non-exit relays, hidden service, hs client
  • single-onion: authorities, non-exit relays, single onion service, hs client

IPv6 Test Suite: only run if ping6 ::1 works.

  • bridges+ipv6-min: authorities, bridge authority, ipv4/ipv6 bridge, ipv6 bridge client
  • ipv6-exit-min: authorities, ipv4/ipv6 exit, ipv4 exit client asking for ipv6 address
  • hs-ipv6: authorities, ipv4/ipv6 non-exit relays, ipv6 hidden service, ipv4 hs client, ipv6 hs client
  • single-onion-ipv6: authorities, ipv4/ipv6 non-exit relays, ipv6 single onion service, ipv4 hs client, ipv6 hs client

NYI: #17011 - test actual IPv6 connectivity (not just IPv4 on IPv6 instances); #16971 - remove requirement for IPv6 DNS on IPv6 exits.

Mixed Version Test Suite: only run if a stable version of tor exists. NYI: try harder to find one (#17015)

  • mixed: mixed authorities, mixed exit relays, mixed clients

Log (.log) and result (.trs) files are available in the test_network_log directory in the build directory.

Testing using the Tor Test Network Script

tor/src/test/test-network.sh

List script arguments and corresponding chutney environmental variables in a table

List templates in a table

Describe performance testing results, and what they do and don't mean (localhost, no network hardware involved, more parallelism than you'll get as a relay)

Testing using Chutney Commands

chutney/chutney

Manually configure a network, a minimal set of steps similar to what tor/src/test/test-network.sh does

List chutney commands in a table

Reference environmental variables and templates from previous section

Other Usability Improvements

NYI: There are some minor changes to chutney which would make it much more usable, such as sensible error messages on failure. Let's work out which of these we can knock over quickly. See the full list of proposed chutney changes in #16949.

Modifying Chutney Tests

The torrc templates and test network configurations included with chutney can be modified using any text editor.

Modifying torrc Templates

chutney/torrc_templates

The torrc template files use the tor options syntax, with the addition of a few templating commands, such as file includes. Some template expansions are performed to make configuration easier. The most important template expansion creates a list of test network authorities in every torrc file.

List templating commands

List "magic" expansion operations, including the one done in the launch script

Modifying Test Network Configurations

chutney/networks

How this currently works:

  • each network is a separate Python source file
  • tags identify clients?, exits and hidden services so chutney can run tests for each client and each hidden service (exits are chosen at random) (there are some subtleties I'm missing here about different kinds of tags)
  • other things in current files I've forgotten

How this could work: (NYI: #16954)

Refactor the existing chutney templates so it's easier to mix and match particular authority, middle relay, client (standard, bridge, IPv6 bridge) and endpoint (exit, hidden service) nodes. Also make version mixing easier. Have a replaceable minimal default? 3 authorities, 1 client, 1 bridge client, 1 bridge, 1 middle, 1 exit, 1 hs

  • each network includes 3 authorities, 1 client or bridge client, 1 exit or hs, and however many middles are required to do a fast bootstrap (assuming reachability)
  • work out a way to select between client / bridge client, and exit or hs endpoints
  • create a way to do a full bootstrap (no assuming reachability) by flipping one setting (change torrc options, add middle relays to some minimum of auth + exit + middle: 4? 5?)
  • work out a way to add IPv6 bridges, old tor versions, and other features I'm forgetting
  • work out a way to scale network size by multiplication, either of all nodes, or of particular node roles (start point, relay, end point), or of individual node configurations
  • work out what syntax Python needs to include one configuration in another

Maybe it could look like this:

  • core: authorities, non-exit relays?
  • client: client, bridge, IPv6 bridge (choose at least one)
  • endpoint: exit, hs (choose at least one)
  • bootstrap: fast or full (choose one)
  • version: built, system (choose at least one)

Then combined to give us all the networks that currently exist, without needing to cut and paste between network configurations.

chutney/net

You can also allow chutney to configure a network, and then modify the generated torrc files yourself. (List commands with reference to chutney command section above.)

Last modified 6 months ago Last modified on Apr 10, 2017, 3:52:28 AM