Opened 4 years ago

Last modified 8 months ago

#21205 new project

Instrument clients to measure directory usage

Reported by: nickm Owned by:
Priority: Medium Milestone: Tor: unspecified
Component: Core Tor/Tor Version:
Severity: Normal Keywords: sponsor4, 034-triage-20180328, 034-removed-20180328
Cc: ahf Actual Points:
Parent ID: Points: parent
Reviewer: Sponsor: Sponsor4


We want to reduce the directory overhead needed for low-bandwidth clients. To do this well, we should make it so a client can measure how much stuff it downloads, and how much of what kind of traffic it uses to download it.

In general, this should be local-only measurements for use while testing: there's no reason we need a complex infrastructure for this one, since the results should be pretty uniform given the same client software in the same circumstances.

Here are some strawman scenarios for client usage which we might want to think about:

  • Client is completely unused.
  • Client is completely unused, but restarted or HUPed once every N hours.
  • Client is used to fetch a .onion website once every N hours.
  • Client is used to fetch a http website once every N hours.
  • Client is turned on, and connected to (say) IRC all the time.

The relevant values of "N" are probably something like: .5 hours, 1 hour, 4 hours, 8 hours, ... up to a week?

This is a parent-ticket. Child tickets will focus on particular measurements we want.

Child Tickets

#21206newMeasure client up/down bandwidth for directory requests, split by typeCore Tor/Tor
#21207newTest scenarios for clients that are idle for large periods of timeCore Tor/Tor
#21208newMeasure overall client bandwidth usage and circuit countsCore Tor/Tor

Change History (16)

comment:1 Changed 4 years ago by ahf

Cc: ahf added

Additional possible scenarios for client usage after a discussion in #tor-dev:

  • Client turns on, fetches an HTTP website, idles for N minutes, then shuts down for M minutes, then starts again.
  • Add systematic "noise" to some of the already listed tests: For example: client is completely unused, but have a flaky network connection.
  • For the IRC test (long-lasting connection): add occasional retries and some bursty behaviour.

Roger additionally suggested using tgen, from Shadow, to generate realistic-looking network traffic, but since this is about directory connectivity performance enhancements, we decided to not look at it for now.

comment:2 Changed 4 years ago by isabela

Sponsor: Sponsor4

comment:3 Changed 4 years ago by teor

One relevant value of N is the consensus expiry time (0-3 hours), another is the reasonably live time (24 hours after expiry, so 24-27 hours), another is the relay descriptor mandatory refresh time (I think this is 18 hours, but you should check this), and another is the microdescriptor expiry time (which I think is a week, but you should check this). A week is also the time that every descriptor must change in, as onion keys are rotated every week.

Also, we might want to do a scenario where the client is on but the network is disconnected.

comment:4 Changed 4 years ago by nickm

Keywords: TorCoreTeam201703 added

comment:5 Changed 4 years ago by ahf

Owner: set to ahf
Status: newassigned

comment:6 Changed 3 years ago by dgoulet

Just want to point out that #13208 has been merged so the src/trace/ framework can be used to add tracing event although if this is to be shipped in production, I would personally want to avoid using --enable-event-tracing (even though no events exists yet) :)

comment:7 Changed 3 years ago by nickm

Remove Sponsor4 keyword, now that Sponsor4 is the value of the Sponsor field.

comment:8 Changed 3 years ago by nickm

Keywords: TorCoreTeam201703 removed

comment:9 Changed 3 years ago by dgoulet

Milestone: Tor: 0.3.1.x-finalTor: 0.3.2.x-final

No chance this can get in 031... This is very close to a whole new feature.

comment:10 Changed 3 years ago by nickm

Milestone: Tor: 0.3.2.x-finalTor: 0.3.3.x-final

comment:11 Changed 3 years ago by nickm

Milestone: Tor: 0.3.3.x-finalTor: 0.3.4.x-final

Mark a lot of assigned/needs_revision tickets as 0.3.4. If you think this should happen in 0.3.3 instead, just let me know?

comment:12 Changed 3 years ago by nickm

Keywords: 034-triage-20180328 added

comment:13 Changed 3 years ago by nickm

Keywords: 034-removed-20180328 added

Per our triage process, these tickets are pending removal from 0.3.4.

comment:14 Changed 3 years ago by nickm

Milestone: Tor: 0.3.4.x-finalTor: unspecified

These tickets, tagged with 034-removed-*, are no longer in-scope for 0.3.4. We can reconsider any of them, if time permits.

comment:15 Changed 17 months ago by gaba

Owner: ahf deleted

Liberating some of the tickets that ahf had.

comment:16 Changed 8 months ago by teor

Status: assignednew

Change tickets that are assigned to nobody to "new".

Note: See TracTickets for help on using tickets.