Opened 22 months ago

Last modified 14 months ago

#29005 new enhancement

PrivCount proof of concept: implement consumed bandwidth counters

Reported by: teor Owned by:
Priority: Medium Milestone: Tor: unspecified
Component: Core Tor/Tor Version:
Severity: Normal Keywords: privcount, 040-unreached-20190109, 041-accepted-20190115, network-team-unreached-2019-Q1Q2
Cc: irl Actual Points: 0.4
Parent ID: #27908 Points: 2
Reviewer: Sponsor:


We want to add a copy of the current consumed bandwidth statistic to PrivCount:

  • consumed bandwidth by relay flags

We can defer these statistics, because they are not as sensitive to manipulation by hostile clients or services:

  • Bandwidth spent on answering directory requests
  • Fraction of connections used uni-/bidirectionally

We will need counters for:

  • read-history
  • write-history

Split each counter into 4 (G, E, G and E, not G and not E) using these flags:

  • Guard
  • Exit and not BadExit

We can do the G/E split on the Data Collector (DC).
We can also do a smaller check aggregation for each group on the Tally Reporter (TR).

Check that relays are Running.
We can do the Running check on the DC.
We should also check Running on the TR, and exclude DCs that weren't Running at all during the period.

How often should we update the bandwidth counters?

If we update counters at the end of the day, we can match the current statistics exactly:
"we only include bandwidth histories for a given day if a relay was listed as running in a consensus at least once on that day. We attribute bandwidth to guards and/or exits if a relay was a guard and/or exit at least in one consensus on a day."
But we would be storing some sensitive data in memory for the whole day.

Instead, we could update the counters whenever we queue data, based on the flags in the current consensus we have. (The difference will likely be minimal.)

If updating the counters multiple times per second is too CPU-intensive, we can update them every few seconds. If that's too often, we can update them just before we delete an old consensus.


Child Tickets

#24250defectnewIn a private network some relays advertise zero bandwidth-observed
#29019enhancementneeds_revisionMake relays report bandwidth usage more often in test networks

Change History (18)

comment:1 Changed 22 months ago by teor

Type: defectenhancement

These should all be enhancements

comment:2 Changed 22 months ago by teor

Points: 2

If we hard-code a lot of things, this should not take more than a few days, depending on how many changes we need to make to the existing tor bandwidth reporting code.

comment:3 Changed 22 months ago by teor

Actual Points: 0.4

I have a proof-of-concept for PrivCount in extra-info descriptors in

Let's review it in #29004.

comment:4 Changed 22 months ago by teor

Keywords: 040-unreached-20190109 041-proposed added
Milestone: Tor: 0.4.0.x-finalTor: unspecified

These tasks are on the 0.4.1 roadmap for PrivCount and Sponsor V.

comment:5 Changed 22 months ago by teor

Keywords: 041-proposed-on-roadmap added; 041-proposed removed
Milestone: Tor: unspecifiedTor: 0.4.1.x-final

Let's review these tickets at the next meeting using our 041-proposed process.

They're on the roadmap, so the review should focus on ticket size and team capacity (and sponsor expectations).

comment:6 Changed 22 months ago by teor

Keywords: 041-accepted-20190115 added; 041-proposed-on-roadmap removed

These PrivCount tickets are on the 041 roadmap, we accepted their points estimates in 041 without discussion.

comment:7 Changed 21 months ago by nickm

Sponsor: SponsorV

comment:8 Changed 20 months ago by teor

Keywords: network-team-roadmap-2019-Q1Q2 added

This is part of an existing roadmap task.

comment:9 Changed 20 months ago by teor

Status: assignedneeds_revision

I have part of a patch for this ticket, but it needs to be revised after #29018 and #29019, and with #29005.

Last edited 20 months ago by teor (previous) (diff)

comment:10 Changed 20 months ago by teor

Keywords: backlog added

comment:11 Changed 18 months ago by teor

I have incomplete branches for #29004, #29005, and #29019 here:

I think all the necessary code is present in these branches.

But it needs some cleanup:

  • rebase on to the current master,
  • put the commits on the right branches
  • make sure it does what these tickets say it should do

I'm happy to do that after I come back from leave. I am also happy if Nick wants to clean up this code.

comment:12 Changed 18 months ago by nickm

Milestone: Tor: 0.4.1.x-finalTor: 0.4.2.x-final

Move privcount tickets to 0.4.2

comment:13 Changed 17 months ago by teor

Keywords: network-team-roadmap-unreached-2019-Q1Q2 added; backlog network-team-roadmap-2019-Q1Q2 removed

comment:14 Changed 17 months ago by teor

Keywords: network-team-unreached-2019-Q1Q2 added; network-team-roadmap-unreached-2019-Q1Q2 removed

comment:15 Changed 17 months ago by teor

Owner: teor deleted
Status: needs_revisionassigned

comment:16 Changed 17 months ago by gaba

Sponsor: SponsorV

comment:17 Changed 16 months ago by irl

Cc: irl added

comment:18 Changed 14 months ago by nickm

Milestone: Tor: 0.4.2.x-finalTor: unspecified
Status: assignednew

Move privcount tickets from 0.4.2 to "Unspecified"; mark as new.

Note: See TracTickets for help on using tickets.