Opened 5 years ago

Last modified 2 years ago

#13792 new enhancement

HS statistics for private tor network to gather info on services, clients and relays

Reported by: dgoulet Owned by:
Priority: Medium Milestone: Tor: unspecified
Component: Core Tor/Tor Version:
Severity: Normal Keywords: tor-hs, performance, metrics, privcount-maybe
Cc: karsten, asn, rob.g.jansen@… Actual Points:
Parent ID: Points:
Reviewer: Sponsor: SponsorQ-can

Description (last modified by arma)

This ticket is for gathering HS statistics for a private tor network.

Reference: #13509

Child Tickets

TicketTypeStatusOwnerSummary
#13802enhancementcloseddgouletAdd general trace-event logging instrumentation to Tor

Attachments (1)

task-13509-analysis.txt (3.8 KB) - added by karsten 5 years ago.
Quick analysis of #1944 experiment to answer some of dgoulet's questions on #13509

Download all attachments as: .zip

Change History (18)

comment:1 Changed 5 years ago by dgoulet

Keywords: tor-hs SponsorR added

comment:2 Changed 5 years ago by robgjansen

Cc: rob.g.jansen@… added

Changed 5 years ago by karsten

Attachment: task-13509-analysis.txt added

Quick analysis of #1944 experiment to answer some of dgoulet's questions on #13509

comment:3 Changed 5 years ago by karsten

I just added a text file that I wrote about a month ago that might answer some of dgoulet's questions raised on #13509. Maybe it's still useful.

comment:4 Changed 5 years ago by dgoulet

Ok so the initial post for the framework to measure performance is here:

https://lists.torproject.org/pipermail/tor-dev/2014-December/007989.html

The post is mostly instructions and where to get the tools for it. I'll detail here the measurement we have up to now thus helping us add or modify or remove some.

Introduction Point

  • Circuit build time
    • Time from circuit is launched to "has opened" event.
  • Established time
    • Time of rend_service_launch_establish_intro())
  • Ready time
    • From the start of rend_service_launch_establish_intro() to the INTRO_ESTABLISHED cell
  • Cannibalized (0|1)

Rendezvous Point

  • Circuit build time
    • Time from circuit is launched to "has opened" event.
  • Established time
    • Time from the creation of the rdv point up to finishing handling the RENDEZVOUS_ESTABLISHED cell.
  • Establish rdv to RENDEZVOUS2 cell
    • Time from the ESTABLISH_RENDEZVOUS cell to the RENDEZVOUS2 end of processing cell.
  • Ready time
    • Time from the creation of the rdv point up to receiving the RENDEZVOUS2 cell which indicates that data is ready for transmission.
  • Cannibalized (0|1)

HSDir Client

  • Fetch time
    • Time of rend_client_refetch_v2_renddesc()
  • Store time
    • When a fetched desc. is successfully stored (rend_cache_store_v2_desc_as_client())

Note that it's not that trivial to match client/service trace events since well this is an anonimity network and it's by design that we can't match both. However in a controlled network of 1 client and 1 HS, there are some assumptions we can make vis-a-vis ordering of events thus possible to match them.

We can do much more but this is the intial part I have which gives us pretty graph already and a clear picture of the timings in the HS subsystem.

comment:5 in reply to:  4 ; Changed 5 years ago by asn

Replying to dgoulet:

Ok so the initial post for the framework to measure performance is here:

https://lists.torproject.org/pipermail/tor-dev/2014-December/007989.html

The post is mostly instructions and where to get the tools for it. I'll detail here the measurement we have up to now thus helping us add or modify or remove some.

Introduction Point

  • Circuit build time
    • Time from circuit is launched to "has opened" event.
  • Established time
    • Time of rend_service_launch_establish_intro())
  • Ready time
    • From the start of rend_service_launch_establish_intro() to the INTRO_ESTABLISHED cell
  • Cannibalized (0|1)

Rendezvous Point

  • Circuit build time
    • Time from circuit is launched to "has opened" event.
  • Established time
    • Time from the creation of the rdv point up to finishing handling the RENDEZVOUS_ESTABLISHED cell.
  • Establish rdv to RENDEZVOUS2 cell
    • Time from the ESTABLISH_RENDEZVOUS cell to the RENDEZVOUS2 end of processing cell.
  • Ready time
    • Time from the creation of the rdv point up to receiving the RENDEZVOUS2 cell which indicates that data is ready for transmission.
  • Cannibalized (0|1)

HSDir Client

  • Fetch time
    • Time of rend_client_refetch_v2_renddesc()
  • Store time
    • When a fetched desc. is successfully stored (rend_cache_store_v2_desc_as_client())

Note that it's not that trivial to match client/service trace events since well this is an anonimity network and it's by design that we can't match both. However in a controlled network of 1 client and 1 HS, there are some assumptions we can make vis-a-vis ordering of events thus possible to match them.

We can do much more but this is the intial part I have which gives us pretty graph already and a clear picture of the timings in the HS subsystem.

That's a great start! BTW, are the graphs for the HSDir measurements somewhere public? I'm interested to see the variance in the time of fetching a descriptor.

On the topic of HSDirs, it should now be easy to do #13208 too, right? However, I'm not sure how useful this would be in a private network, where there is not that big number of relays and also natural churn is not very natural

Also, I noticed that all stats come from clients or hidden services. You haven't instrumented the IP or HSDir code itself, right? Do you think that would be useful? For example, to check from the IP's PoV the "time from receiving ESTABLISH_INTRO to sending INTRO_ESTABLISHED cell back to the HS". Are the two clusters visible there too? If yes, it's definitely an IP issue.

Also, maybe #13239 would be a useful experiment. Basically checking the performance baseline with 2 preemptive circuits vs 3 preemptive circuits.

comment:6 in reply to:  5 Changed 5 years ago by dgoulet

That's a great start! BTW, are the graphs for the HSDir measurements somewhere public? I'm interested to see the variance in the time of fetching a descriptor.

I have one yes but it has few data points for now. I would like one with hundreds at least but it takes forever with chutney (because of the bootstrap time) so I still have to set an overnight experiment.

https://people.torproject.org/~dgoulet/hs/hsdir-025.png

On the topic of HSDirs, it should now be easy to do #13208 too, right? However, I'm not sure how useful this would be in a private network, where there is not that big number of relays and also natural churn is not very natural

Very easy yes! We can match in the trace the fetch + store very easily. However, as you said, I've never seen that on chutney because well not really overloaded :).

Also, I noticed that all stats come from clients or hidden services. You haven't instrumented the IP or HSDir code itself, right? Do you think that would be useful? For example, to check from the IP's PoV the "time from receiving ESTABLISH_INTRO to sending INTRO_ESTABLISHED cell back to the HS". Are the two clusters visible there too? If yes, it's definitely an IP issue.

Well in a way I did. The "tor_trace_hs_relay_cell_start/done()" tracepoint, you see all HS cells. This is how I can match cells going to certain intro points from the HS.

Good idea for a graph! I can definitely do that!

Also, maybe #13239 would be a useful experiment. Basically checking the performance baseline with 2 preemptive circuits vs 3 preemptive circuits.

Indeed, with a patch that does both, we can easily graph the differences.

That's I think one of the beauty of this stats collection, quickly we can see if we make progress or not :)

comment:7 Changed 5 years ago by nickm

Milestone: Tor: 0.2.???

Some of this will land in different tor versions.

comment:8 Changed 4 years ago by arma

Description: modified (diff)

comment:9 Changed 4 years ago by nickm

Keywords: SponsorR removed
Sponsor: SponsorR

Bulk-replace SponsorR keyword with SponsorR sponsor field in Tor component.

comment:10 Changed 3 years ago by dgoulet

Sponsor: SponsorRSponsorR-can

Move those from SponsorR to SponsorR-can.

comment:11 Changed 3 years ago by teor

Milestone: Tor: 0.2.???Tor: 0.3.???

Milestone renamed

comment:12 Changed 3 years ago by nickm

Keywords: tor-03-unspecified-201612 added
Milestone: Tor: 0.3.???Tor: unspecified

Finally admitting that 0.3.??? was a euphemism for Tor: unspecified all along.

comment:13 Changed 2 years ago by nickm

Keywords: tor-03-unspecified-201612 removed

Remove an old triaging keyword.

comment:14 Changed 2 years ago by dgoulet

Keywords: performance added
Severity: Normal
Sponsor: SponsorR-can
Type: taskenhancement

comment:15 Changed 2 years ago by nickm

Keywords: metrics privcount-maybe added

comment:17 Changed 2 years ago by nickm

Sponsor: SponsorQ-can
Note: See TracTickets for help on using tickets.