Opened 7 years ago

Last modified 22 months ago

#7358 new task

Decide on list of stats to collect

Reported by: robgjansen Owned by:
Priority: Medium Milestone: Tor: unspecified
Component: Core Tor/Tor Version:
Severity: Normal Keywords: performance, simulation, statistics, tor-relay, tor-client
Cc: arma, karsten, robgjansen Actual Points:
Parent ID: #7357 Points:
Reviewer: Sponsor:

Description (last modified by robgjansen)

Here are the statistics I speculate will be useful, and may or may not already be available in some form, and may only be available externally (outside of the Tor code). Keep in mind that some of this is not intended to be collected outside of an experimentation environment, else proper aggregation/scrubbing is required.

client

  • circuit build times, build timeouts
  • which relays were chosen for each circuit, and during which time intervals
  • number of streams over time
  • stream throughput over time
  • how long streams have been active/inactive
  • number of and bandwidth expended by client directory operations

client+relay

  • cell statistics: number queued and processed, waiting times
  • total number of circuits and the various connection types (AP, OR, EXIT, DIR) over time
  • throughput of circuits and the various connection types over time
  • when steams, circuits, or connections change active/inactive status
  • how fast/often token buckets were emptied/empty

relay

  • protocol overheads (raw client data vs protocol traffic)
  • number of and bandwidth expended by directory server operations
  • crypto statistics (see #7134)

What am I missing?

Child Tickets

Change History (9)

comment:1 Changed 7 years ago by robgjansen

Type: defecttask

comment:2 in reply to:  description ; Changed 7 years ago by karsten

Replying to robgjansen:

Client

  • download statistics (how long to get to first, last, ? byte)
  • circuit build times, build timeouts
  • which relays were chosen for each circuit, and during which time intervals

This is basically what Torperf does. The first item is what trivsocks-client tells us, and the second and third items are what we learn from control port events. Implementing the first item in Tor using control port events is sure tempting. We lack some information, e.g., the exact timestamp when Torperf started the download and the expected number of bytes we want to download. But maybe we can compensate for that. For example, we could collect times between sending the first byte and receiving 1B, 1kB, 2kB, 5kB, 10kB, 20kB, 50kB, 100kB, 200kB, 500kB, and so on, up to the last byte. Not exactly the same as the deciles we have so far, but should be fine for most purposes. Also, we'll have to do that for all circuits that the client opens on behalf of the user.

Another item for the client section here might be directory client operations. We might want to keep track how many directory requests a client has made and how many bytes it has sent or received for that. Then we can compare different directory designs more easily.

Also, I think we need to move all statistics from client+relay that have to do with streams to the client-only section.

client+relay

  • cell statistics: # queued, processed, waiting times
  • total number of or connections, circuits, and streams over time

What about other connections than OR?

  • various throughputs (stream, circuit, connection) over various intervals (last second, 10 seconds, 60 seconds, 300 seconds, ? seconds)
  • when steams, circuits, or connections change active/inactive status
  • indications of congestion (inferred by how fast/often token buckets were emptied/empty, queuing times from above)

I assume these will all be implemented as asynchronous control port events, right? Which of them will be emitted whenever there's a change, and which will be emitted periodically?

Another item might be statistics on crypto operations as described in #7134, but without the aggregation step that isn't necessary if we collect these statistics in a simulation/testing environment. The two can probably share a lot of code.

relay

  • protocol overheads (raw client data vs protocol traffic)

We also have statistics on bi-directional connection usage already in Tor. But these are probably contained somewhere in the client+relay section.

And we might add statistics on directory server operations with the same reasoning as adding directory client operations.

What am I missing?

Not sure, I guess we'll find out while working off this list. Count me in, this is fun stuff. :)

comment:3 in reply to:  2 Changed 7 years ago by robgjansen

Replying to karsten:

I assume these will all be implemented as asynchronous control port events, right? Which of them will be emitted whenever there's a change, and which will be emitted periodically?

I must confess I'm not very familiar with the ControlPort functionality, and could use some help designing this properly. I'd guess our options are either, e.g., CircuitBuilt type events or CircuitBuildingStatsReady type events. Lets discuss this in #7359.

comment:4 in reply to:  2 ; Changed 7 years ago by robgjansen

Description: modified (diff)

Replying to karsten:

Replying to robgjansen:

Client

  • download statistics (how long to get to first, last, ? byte)
  • circuit build times, build timeouts
  • which relays were chosen for each circuit, and during which time intervals

This is basically what Torperf does. The first item is what trivsocks-client tells us, and the second and third items are what we learn from control port events. Implementing the first item in Tor using control port events is sure tempting. We lack some information, e.g., the exact timestamp when Torperf started the download and the expected number of bytes we want to download. But maybe we can compensate for that. For example, we could collect times between sending the first byte and receiving 1B, 1kB, 2kB, 5kB, 10kB, 20kB, 50kB, 100kB, 200kB, 500kB, and so on, up to the last byte. Not exactly the same as the deciles we have so far, but should be fine for most purposes. Also, we'll have to do that for all circuits that the client opens on behalf of the user.

I'm of the opinion that we let the user application (TorPerf, etc.) continue measuring end-user-specific performance characteristics, precisely because of the lack of information and imprecision that you mentioned. Tor should stick to things its good at measuring, like throughput.

Another item for the client section here might be directory client operations. We might want to keep track how many directory requests a client has made and how many bytes it has sent or received for that. Then we can compare different directory designs more easily.

Also, I think we need to move all statistics from client+relay that have to do with streams to the client-only section.

Great! I've updated the description.

client+relay

  • cell statistics: # queued, processed, waiting times
  • total number of or connections, circuits, and streams over time

What about other connections than OR?

Good point :) Updated.

  • various throughputs (stream, circuit, connection) over various intervals (last second, 10 seconds, 60 seconds, 300 seconds, ? seconds)
  • when steams, circuits, or connections change active/inactive status
  • indications of congestion (inferred by how fast/often token buckets were emptied/empty, queuing times from above)

I assume these will all be implemented as asynchronous control port events, right? Which of them will be emitted whenever there's a change, and which will be emitted periodically?

Another item might be statistics on crypto operations as described in #7134, but without the aggregation step that isn't necessary if we collect these statistics in a simulation/testing environment. The two can probably share a lot of code.

This does seem important. My vision of a "stats" module in #7359 should help avoid duplication of code and help separate statistics from functionality.

relay

  • protocol overheads (raw client data vs protocol traffic)

We also have statistics on bi-directional connection usage already in Tor. But these are probably contained somewhere in the client+relay section.

And we might add statistics on directory server operations with the same reasoning as adding directory client operations.

Updated.

What am I missing?

Not sure, I guess we'll find out while working off this list. Count me in, this is fun stuff. :)

Awesome:)

comment:5 Changed 7 years ago by robgjansen

Description: modified (diff)

Should we add #7134 to #7357?

comment:6 Changed 7 years ago by robgjansen

Milestone: Tor: 0.2.4.x-finalTor: unspecified

comment:7 in reply to:  4 Changed 7 years ago by karsten

Replying to robgjansen:

I'm of the opinion that we let the user application (TorPerf, etc.) continue measuring end-user-specific performance characteristics, precisely because of the lack of information and imprecision that you mentioned. Tor should stick to things its good at measuring, like throughput.

Ah, okay. I read your statement in #7357 ("Not only will this standardize stats across simulations/real experiments/tor metrics, clients like TorPerf would only have to worry about creating requests and not collecting stats."), and I found the idea of having Tor output basic performance data about Torperf's download tempting. But either way works.

comment:8 in reply to:  5 Changed 7 years ago by karsten

Replying to robgjansen:

Should we add #7134 to #7357?

Sure, why not. I just did that.

comment:9 Changed 22 months ago by teor

Severity: Normal

Set all open tickets without a severity to "Normal"

Note: See TracTickets for help on using tickets.