Opened 8 years ago

Closed 8 years ago

Last modified 8 years ago

#2551 closed enhancement (implemented)

Torperf should output circuit build times

Reported by: mikeperry Owned by: mikeperry
Priority: Medium Milestone:
Component: Metrics/CollecTor Version:
Severity: Keywords: TorPerfIteration20110305 MikePerryIteration20110305
Cc: Sebastian Actual Points: 3
Parent ID: Points: 3
Reviewer: Sponsor:

Description (last modified by arma)

The extra_data.py listener for TorPerf should be outputting circuit build times for research and analysis. I can help with this. It shouldn't be too difficult to do if we just add CIRC events to its EventListener.

The .extradata file format may change, which may or may not make some people sad. I'm not sure if anyone besides us is looking at those files though.

Child Tickets

Attachments (1)

50kb.extradata (312.7 KB) - added by mikeperry 8 years ago.
New extra_data.py sample output.

Download all attachments as: .zip

Change History (17)

comment:1 Changed 8 years ago by arma

Description: modified (diff)

comment:2 Changed 8 years ago by karsten

Cc: Sebastian added

I don't think anyone depends on the .extradata file format yet. I'm only putting it on the website, but not doing anything with it.

What would be a good format?

Who would implement this? You? Sebastian?

comment:3 Changed 8 years ago by karsten

See also #2565 for some thoughts on redesigning Torperf to output circuit build times.

comment:4 Changed 8 years ago by mikeperry

We also record BUILDTIMEOUT_SET control port events as part of this, or perhaps for a new data set + plot.

comment:5 Changed 8 years ago by mikeperry

Points: 3

Alter extra_data.py to observer circ events: 1
Filter out irrelevant circuits: 2

comment:6 Changed 8 years ago by mikeperry

Keywords: TorPerfIteration20110305 MikePerryIteration0305 added
Owner: changed from karsten to mikeperry
Status: newassigned

comment:7 Changed 8 years ago by karsten

Parent ID: #2607

comment:8 Changed 8 years ago by mikeperry

Keywords: MikePerryIteration20110305 added; MikePerryIteration0305 removed

Changed 8 years ago by mikeperry

Attachment: 50kb.extradata added

New extra_data.py sample output.

comment:9 Changed 8 years ago by mikeperry

Status: assignedneeds_review

This should be done now. See mikeperry/bugs2551+2590 for the implementation.

comment:10 Changed 8 years ago by karsten

Parent ID: #2607

comment:11 Changed 8 years ago by karsten

Python gives me this traceback:

WARN[Wed Feb 23 13:10:14 2011]:No event for: 650 BUILDTIMEOUT_SET COMPUTED TOTAL_TIMES=1000 TIMEOUT_MS=27131 XM=1412 ALPHA=0.544520 CUTOFF_QUANTILE=0.800000 TIMEOUT_RATE=0.158000 CLOSE_MS=284850 CLOSE_RATE=0.064000
Traceback (most recent call last):
  File "/srv/torperf.torproject.org/torperf/TorCtl/TorCtl.py", line 746, in _eventLoop
    self._handleFn(timestamp, reply)
  File "/srv/torperf.torproject.org/torperf/TorCtl/TorCtl.py", line 1347, in _handle1
    l.listen(event)
  File "/srv/torperf.torproject.org/torperf/TorCtl/TorCtl.py", line 1292, in listen
    self._map1.get(event.event_name, self.unknown_event)(event)
  File "../extra_stats.py", line 53, in buildtimeout_set_event
    result = b.event_name + " " +b.body
AttributeError: BuildTimeoutSetEvent instance has no attribute 'body'

Are we using the same TorCtl? I'm running 7329ceb4fe44c3557742ea596aeca7ea5f9e76f7.

I hot-fixed this by replacing b.body with "(unknown body)" just to make it work.

comment:12 Changed 8 years ago by mikeperry

Ugh, I guess I fell asleep before pushing this. Update your TorCtl checkout and it should fix this issue. I want to store that .body as-is.

I also fixed a spurious WARN in extra_data.py, so you might want to update that too.

comment:13 Changed 8 years ago by karsten

There's another problem in that code. Here's my hotfix:

diff --git a/TorCtl.py b/TorCtl.py
index 4319242..65a91f7 100755
--- a/TorCtl.py
+++ b/TorCtl.py
@@ -1431,7 +1431,7 @@ class EventHandler(EventSink):
       m = re.match(r"(\d+)\s+(\d+)\s+(\d+)", body)
       if not m:
         raise ProtocolError("STREAM_BW event misformatted.")
-      event = StreamBwEvent(evtype, *m.groups(), body=body)
+      event = StreamBwEvent(evtype, *m.groups())
     elif evtype == "BW":
       m = re.match(r"(\d+)\s+(\d+)", body)
       if not m:

comment:14 Changed 8 years ago by mikeperry

Try:

diff --git a/TorCtl.py b/TorCtl.py
index 4319242..5403a95 100755
--- a/TorCtl.py
+++ b/TorCtl.py
@@ -266,8 +266,8 @@ class ORConnEvent(Event):
     self.ncircs = ncircs
 
 class StreamBwEvent(Event):
-  def __init__(self, event_name, strm_id, written, read, body):
-    Event.__init__(self, event_name, body)
+  def __init__(self, event_name, saved_body, strm_id, written, read):
+    Event.__init__(self, event_name, saved_body)
     self.strm_id = int(strm_id)
     self.bytes_read = int(read)
     self.bytes_written = int(written)
@@ -1431,7 +1431,7 @@ class EventHandler(EventSink):
       m = re.match(r"(\d+)\s+(\d+)\s+(\d+)", body)
       if not m:
         raise ProtocolError("STREAM_BW event misformatted.")
-      event = StreamBwEvent(evtype, *m.groups(), body=body)
+      event = StreamBwEvent(evtype, body, *m.groups())
     elif evtype == "BW":
       m = re.match(r"(\d+)\s+(\d+)", body)
       if not m:

comment:15 Changed 8 years ago by karsten

Resolution: implemented
Status: needs_reviewclosed

Torperfs on ferrinii look good! Merging and closing.

comment:16 Changed 8 years ago by mikeperry

Actual Points: 3
Note: See TracTickets for help on using tickets.