Run torperf on the sat link
It would be interesting to run the 15 torperf runs described in the torperf git repository on the sat link: https://gitweb.torproject.org/torperf.git/blob_plain/HEAD:/measurements-HOWTO
Info level logs, and periodic captures (every few hours) of the state files would also be useful.
Activity
At $7 per MB, that's $35 per data point for the 5MB torperf runs. 50KB 480 times a day, plus 1MB 48 times a day, plus 5MB 24 times a day, equals 192MB or $1344 a day. Not counting the overhead that Tor induces.
15 of them running would be $20k a day. Not to mention the fact that they'd get in each other's way, right?
Wait, 480 runs means starting a new download every 3 minutes? That's probably not enough time to finish a 50 KB download. The current default is starting a new download every 5 minutes. We should try 5 minutes for the sat link, too, or we'll end up with too many timeouts.
When you have the data, I can visualize them and compare them to our current measurements.
-
Replying to karsten:
Wait, 480 runs means starting a new download every 3 minutes? That's probably not enough time to finish a 50 KB download. The current default is starting a new download every 5 minutes. We should try 5 minutes for the sat link, too, or we'll end up with too many timeouts.
Yes, you're right. Somebody should recheck all other statements I made on that day too. :}
We should tweak the 'cbtquantile' consensus parameter a bit and see what effect changing it has on our regular torperf runs and these. Tweaking it in the consensus not only allows us to get two tests for free at the same time, it also allows us to see what the network-wide effects are.
I suggest that we try 60, 70 and the default (80). Each run should last for at least a few hours (probably 6-8 to ensure consensus propagation and stabilization) and we should ensure each torperf has already learned a timeout at the start of the test.
How many days will we have in the test lab?
-
If we have multiple days, each test should be performed on the same time periods of each day. I can assist remotely, but if we just let the torperfs run continuous for three days, and I flip the switch on my dirauth each day at the same time, this should be super easy to do. We can also do it in only two days if we can pick a good guess for the lowest value we're comfortable using, and just try that out.
-
Replying to mikeperry:
We should tweak the 'cbtquantile' consensus parameter a bit and see what effect changing it has on our regular torperf runs and these. Tweaking it in the consensus not only allows us to get two tests for free at the same time, it also allows us to see what the network-wide effects are.
Changing a network-wide parameter to test the performance of a single Tor client seems like the wrong approach. We won't get two tests for free, but we'll influence the results of the sat link measurements in a way we don't understand. We'll always have time to play with consensus parameters and our standard torperfs at a later time.
Is there a way to change the 'cbtquantile' parameter on the Tor client only? If not, should we just patch it for the measurements?
Also, keep in mind that 288 data points per day are not much to compare them to the standard torperfs, so running an unchanged torperf on the sat link for a couple of days might be useful.
-
Replying to karsten:
Changing a network-wide parameter to test the performance of a single Tor client seems like the wrong approach. We won't get two tests for free, but we'll influence the results of the sat link measurements in a way we don't understand. We'll always have time to play with consensus parameters and our standard torperfs at a later time.
Is there a way to change the 'cbtquantile' parameter on the Tor client only? If not, should we just patch it for the measurements?
I totally disagree. The cbtquantile is a parameter that could very well have emergent effects. It's exactly the type of thing that you could set to a value like '20' on just one tor client and arrive at the conclusion that it works great. Testing a value out on the whole network in this situation is therefore actually like 3 tests for free even, not just 2. We get to see what sort of load that this induces on our servers and we get to see if the combination of more circuit creation and the lower value actually improves one or both of the sat link and the broadband link.
Really the only reason not to do it yet is that we don't have enough clients yet running 0.2.2.x. But this isn't a valid reason not to try it either. It just means that we'll see less of an increased load on our nodes..
Also, keep in mind that 288 data points per day are not much to compare them to the standard torperfs, so running an unchanged torperf on the sat link for a couple of days might be useful.
This is a different matter. Maybe we should just run the 50kb slightly more often? If we can't fetch 50kb in 3 min, isn't that pretty much a failure anyways? Circuits should be timing out on the order of 7-30 seconds... Test #2477 (moved) should also give us data on this for the sat link, so we should certainly try this first and see what the minimum retry value is.
-
Replying to mikeperry:
Testing a value out on the whole network in this situation is therefore actually like 3 tests for free even, not just 2.
OK, I don't know enough about 'cbtquantile' to make a solid statement here. My comment was more a general one about changing more than one variable at once (here: ctbquantiles on an unknown number of Tor clients, not just on the one we're measuring).
Also, keep in mind that 288 data points per day are not much to compare them to the standard torperfs, so running an unchanged torperf on the sat link for a couple of days might be useful.
This is a different matter. Maybe we should just run the 50kb slightly more often? If we can't fetch 50kb in 3 min, isn't that pretty much a failure anyways? Circuits should be timing out on the order of 7-30 seconds... Test #2477 (moved) should also give us data on this for the sat link, so we should certainly try this first and see what the minimum retry value is.
Our only data comes from torperfs running on excellent network connections. Hard to say if 3 minutes are a useful timeout for a sat link.
-
Trac:
Parent: N/A to #2492 (moved) I've setup a torperf instance that only runs the 50KB tests from a vanilla Tor client (only section 1 and not section 2). They're running in a cronjob as of about twenty minutes ago.
mikeperry - can you OTR me and I'll show you what I'm capturing so far?
Trac:
Status: new to assigned
Owner: ioerror to mikeperryI just noticed that after running
start-torsI had the following in my terminal buffer:File "../extra_stats.py", line 15 if not self._conn ^ SyntaxError: invalid syntaxIt looks like this is having issues:
cd ~/torperf/torclient50kb && python ../extra_stats.py 10020 ../50kb.extradata &TorCtl is importing properly - I'm not sure why Python (2.6.6) is complaining.
Is there a unit test suite that I can run?
Trac:
Status: assigned to needs_review-
Replying to ioerror:
In the torperf HOWTO - The package name is no longer 'timeout, it's now 'coreutils'
Right, that has changed in Squeeze. Sebastian, please merge branch timeout in my public repository.
Not sure about the Python problem.
Trac:
extra_stats.py.patchI've applied the above patch and now start Tor like so:
$ cat start-tors #!/bin/bash cd ~/torperf/torclient50kb && tor -f ~/torperf/torclient50kb/torrc sleep 5 cd ~/torperf/torclient50kb && python ../extra_stats.py 10020 ../50kb.extradata &Running the above results in the following output:
Feb 08 00:10:43.276 [notice] Tor v0.2.2.22-alpha (git-21b3de6cf37d4e60). This is experimental software. Do not rely on it for strong anonymity. (Running on Linux x86_64) Feb 08 00:10:43.281 [notice] Initialized libevent version 1.4.13-stable using method epoll. Good. Feb 08 00:10:43.281 [notice] Opening Socks listener on 127.0.0.1:9020 Feb 08 00:10:43.282 [notice] Opening Control listener on 127.0.0.1:10020Trac:
measurements-HOWTO.patch