Opened 8 years ago

Closed 3 years ago

Last modified 3 years ago

#2554 closed task (fixed)

Write script to reconstruct hidden service time components from controller events

Reported by: arma Owned by: dgoulet
Priority: Medium Milestone: Tor: 0.2.9.x-final
Component: Archived/Torperf Version:
Severity: Normal Keywords: tor-hs
Cc: teor, rob.g.jansen@… Actual Points:
Parent ID: Points: medium
Reviewer: Sponsor: SponsorR-must

Description

When we get #1944 going, we're going to start wondering what the breakdown of "time before connected cell" is.

Does torperf get its statistics of time breakdown just from the socks port, or does it learn things over the control port too, or what? We should figure out what time components we want to track, and then figure out how to export them so torperf can track them for posterity.

We may also find that we want to remember the circuits for the other components of the rendezvous, similar to the extension we're considering for #2551.

Child Tickets

Change History (18)

comment:1 Changed 8 years ago by karsten

Torperf, the C program, gets its timing information only from talking to Tor's SOCKS port. Torperf, the Python part, does learn things from the control port, namely path information and maybe circuit build times when we implement #2551. It should be possible to add hidden service time components to one of the Python scripts.

See also #2565 for some thoughts to redesign Torperf to add hidden service time components.

comment:2 Changed 8 years ago by aaronsw

torperf2 does this by parsing the log events that come from the controller port.

https://github.com/aaronsw/torperf/

comment:3 Changed 8 years ago by aaronsw

Status: newneeds_review

comment:4 Changed 8 years ago by karsten

Owner: karsten deleted
Status: needs_reviewassigned

comment:5 Changed 8 years ago by karsten

Keywords: TorPerfIteration20110305 added

comment:6 Changed 8 years ago by dchasteen

Points: 2

Added points from sprint planning meeting.

comment:7 Changed 8 years ago by karsten

Component: MetricsTorperf
Keywords: TorPerfIteration20110305 removed
Points: 24
Priority: normalminor

We didn't start working on this ticket in our March 5 Torperf sprint. Removing the keyword and bumping points to 4. Also setting priority to minor, because we should finish our substantial changes to Torperf first before looking at Torperf for hidden services.

comment:8 Changed 8 years ago by karsten

See #3036 for thoughts on adding the various hidden service substeps to the .mergedata format.

comment:9 Changed 5 years ago by arma

Keywords: SponsorR added
Priority: minornormal

comment:10 Changed 4 years ago by teor

Cc: teor added

comment:11 Changed 4 years ago by arma

Cc: dgoulet added
Summary: extend torperf to record hidden service time componentswrite script to reconstruct hidden service time components from controller events

I spoke to David about this process yesterday. Rob wrote a stem script to dump a bunch of controller events while we load an onion service. David is working on a parsing script to piece together the events for each onion service we're visiting, so we can a) understand which components of the onion service rendezvous worked, and b) track how long each component takes.

As a double bonus, Rob can run the same stem script on his OnionPerf, and then we can use the same parsing script to reconstruct the component timing and success rates on the OnionPerf actions.

And as a triple bonus, onion crawlers like Ahmia or SRI's crawler can dump these events too, and we can use the same script to reconstruct in that case too.

I'm going to revise the title here to indicate that torperf probably is no longer the place to do this reconstruction.

comment:12 Changed 4 years ago by arma

comment:13 Changed 4 years ago by dgoulet

Cc: dgoulet removed
Keywords: tor-hs added; SponsorR removed
Milestone: Tor: 0.2.8.x-final
Owner: set to dgoulet
Parent ID: #2552
Points: 4medium
Sponsor: SponsorR
Status: assignedaccepted
Summary: write script to reconstruct hidden service time components from controller eventsWrite script to reconstruct hidden service time components from controller events
Type: enhancementtask

comment:14 Changed 4 years ago by robgjansen

Cc: rob.g.jansen@… added
Severity: Normal

comment:15 Changed 3 years ago by nickm

Milestone: Tor: 0.2.8.x-finalTor: 0.2.9.x-final

These seem like features, or like other stuff unlikely to be possible this month. Bumping them to 0.2.9

comment:16 Changed 3 years ago by dgoulet

Sponsor: SponsorRSponsorR-must

@rob: Is this something that OnionPerf does now and if yes, we should open specific tickets if we are missing some control events that we would like to track for the HS dance and close this ticket.

comment:17 in reply to:  16 Changed 3 years ago by dgoulet

Resolution: fixed
Status: acceptedclosed

Replying to dgoulet:

@rob: Is this something that OnionPerf does now and if yes, we should open specific tickets if we are missing some control events that we would like to track for the HS dance and close this ticket.

Hrm, actually I found #17627 about missing events for the "HS dance". I'm closing this in favor of that ticket and from the fact that we have OnionPerf doing what we want. Please re-open if you think otherwise.

comment:18 Changed 3 years ago by robgjansen

OnionPerf does not currently do anything with HS_DESC events.

https://github.com/robgjansen/onionperf/blob/master/onionperf/analysis.py#L673

AIUI, #17627 is about adding additional events to Tor so that something like OnionPerf can actually link up everything. You may want to use this ticket to track the task of actually doing something with those new Tor events once they exist (i.e., updating OnionPerf to support HS_DESC events, or writing a new script if you don't want to do it in OnionPerf).

IMHO, OnionPerf is the correct place to add the new logic.

Note: See TracTickets for help on using tickets.