Opened 9 years ago

Closed 6 years ago

#2565 closed enhancement (wontfix)

Redesign Python parts in Torperf

Reported by: karsten Owned by:
Priority: Low Milestone:
Component: Archived/Torperf Version:
Severity: Keywords:
Cc: Sebastian, mikeperry Actual Points:
Parent ID: #7168 Points: 8
Reviewer: Sponsor:

Description

We're stuffing more and more functionality into Torperf. See #2551 for outputting circuit build times and #2554 for adding hidden service time components.

I think it's time to consider a redesign of the Python parts. Having a Python script for writing controller events to disk, a Python script for influencing guard selection, and a cronjob for actually making the requests is already hackish. Adding circuit build times and hidden-service events stretches this even more.

We could have a single Python program that reads from a configuration file what to do, starts Tor, connects to its control port, periodically calls the C program to make requests, and collects all the data we want. This would also make it much easier to set up Torperf.

Sebastian, Mike: Does this make sense?

Child Tickets

TicketTypeStatusOwnerSummary
#3279enhancementclosedtombWrite a Torperf skeleton in Python that parses Torperf's configuration files
#3280enhancementclosedtombMake the new Python Torperf write torrc's
#3281enhancementclosedtombEnable the new Python Torperf to start and stop tor processes
#3282enhancementclosedtombExtend the new Python Torperf to talk to tor's control port
#3283enhancementclosedtombMake the new Torperf call trivsocks-client to make request and timeout runs that take too long
#3284enhancementclosedtombMake the new Torperf generate the new .mergedata output directly
#3285enhancementclosedtombWrite Python version of filter.R to parse Torperf's new .mergedata format

Change History (13)

comment:1 Changed 9 years ago by aaronsw

This is the theory behind torperf2. It doesn't have all the features of the old torperf yet, though.

https://github.com/aaronsw/torperf/

comment:2 Changed 9 years ago by karsten

Parent ID: #2607

comment:3 Changed 9 years ago by karsten

Owner: karsten deleted
Status: newassigned

comment:4 Changed 9 years ago by karsten

Keywords: TorPerfIteration20110305 added
Parent ID: #2607

comment:5 Changed 9 years ago by karsten

We should look at run_test.py when implementing this ticket and steal the logic to call trivsocks-client. Once we're done with this ticket, let's kill run_test.py.

comment:6 Changed 9 years ago by dchasteen

Points: 4

Added points from sprint planning meeting.

comment:7 Changed 9 years ago by karsten

Component: MetricsTorperf
Keywords: TorPerfIteration20110305 removed
Points: 48
Priority: normalminor

We didn't touch this ticket in our March 5 Torperf sprint. Removing the keyword and bumping points to 8.

I'm also setting priority to minor, because we're changing Torperf's output so frequently these days that external users won't enjoy setting up their own Torperf anyway. We should finish working on the data formats first before trying to make Torperf more usable.

comment:8 Changed 9 years ago by karsten

See also #2766.

comment:9 Changed 8 years ago by karsten

Here's my vague idea of how a redesigned Torperf could look like.

We'd have a single config file that contains all settings that we currently write to the crontab and to the start script. A sample config file might look like this:

TorperfGeneral
  DataDirectory=/home/torperf/torperf/
  TorBinary=/home/torperf/tor/src/or/tor
  SourceString=ferrinii
  StartSocksPorts=9020
  StartControlPorts=10020
  HttpServer=torperf.torproject.org

TorperfRun
  Start=*/5
  FileSize=50*1024
  Timeout=295

TorperfRun
  TorBinary=/home/torperf/tor-75/src/or/tor
  Start=2,7,12,17,22,27,32,37,42,47,52,57
  FileSize=50*1024
  Timeout=295
  Guards=slowratio
  Cbt=75

We'd also have shell scripts to start and stop all Torperf runs described in the config file. The only required change to the crontab would be to add a @reboot line containing the start script.

Torperf would:

  • create any missing Tor data directories and torrc's,
  • start tors,
  • connect to all the control ports to obtain path information and possibly influence guard node selection,
  • regularly call trivsocks-client to make requests,
  • timeout runs that take too long, and
  • write everything to a single output file, possibly in the format described in #3036.

comment:10 Changed 8 years ago by tomb

I'm adding here Ian's request here so we don't forget it:

Could the new version support optimistic data?  Here's a patch for the
existing version:


https://trac.torproject.org/projects/tor/attachment/ticket/1849/optimistic-data.diff

comment:11 Changed 8 years ago by tomb

I am adding Arturo Filastò's request here so we won't overlook it:

I have some requirements for torperf. I would like it to be able to give
me some useful data about tor2web improvements.

We have made a list of the things we think we are interesting in taking
note of: https://github.com/globaleaks/tor2web-2.0/issues/2.

comment:12 Changed 7 years ago by arma

Parent ID: #7168

comment:13 Changed 6 years ago by karsten

Resolution: wontfix
Status: assignedclosed

There's a new attempt to rewrite Torperf, starting with a design document. Closing all tickets related to the earlier attempt, because they're not relevant anymore.

Note: See TracTickets for help on using tickets.