Opened 6 weeks ago

Last modified 3 weeks ago

#29005 assigned enhancement

PrivCount proof of concept: implement consumed bandwidth counters

Reported by: teor Owned by: teor
Priority: Medium Milestone: Tor: 0.4.1.x-final
Component: Core Tor/Tor Version:
Severity: Normal Keywords: privcount, 040-unreached-20190109, 041-accepted-20190115
Cc: Actual Points: 0.4
Parent ID: #27908 Points: 2
Reviewer: Sponsor: SponsorV

Description

We want to add a copy of the current consumed bandwidth statistic to PrivCount:

  • consumed bandwidth by relay flags

https://metrics.torproject.org/bandwidth-flags.html

We can defer these statistics, because they are not as sensitive to manipulation by hostile clients or services:

  • Bandwidth spent on answering directory requests
  • Fraction of connections used uni-/bidirectionally

We will need counters for:

  • read-history
  • write-history

Split each counter into 4 (G, E, G and E, not G and not E) using these flags:

  • Guard
  • Exit and not BadExit

We can do the G/E split on the Data Collector (DC).
We can also do a smaller check aggregation for each group on the Tally Reporter (TR).

Check that relays are Running.
We can do the Running check on the DC.
We should also check Running on the TR, and exclude DCs that weren't Running at all during the period.

How often should we update the bandwidth counters?

If we update counters at the end of the day, we can match the current statistics exactly:
"we only include bandwidth histories for a given day if a relay was listed as running in a consensus at least once on that day. We attribute bandwidth to guards and/or exits if a relay was a guard and/or exit at least in one consensus on a day."
But we would be storing some sensitive data in memory for the whole day.

Instead, we could update the counters whenever we queue data, based on the flags in the current consensus we have. (The difference will likely be minimal.)

If updating the counters multiple times per second is too CPU-intensive, we can update them every few seconds. If that's too often, we can update them just before we delete an old consensus.

Sources:
https://metrics.torproject.org/reproducible-metrics.html#traffic
https://trac.torproject.org/projects/tor/wiki/org/meetings/2018MexicoCity/Notes/PrivCount

Child Tickets

TicketTypeStatusOwnerSummary
#24250defectnewIn a private network some relays advertise zero bandwidth-observed
#29019enhancementassignedteorMake relays report bandwidth usage more often in test networks

Change History (7)

comment:1 Changed 6 weeks ago by teor

Type: defectenhancement

These should all be enhancements

comment:2 Changed 6 weeks ago by teor

Points: 2

If we hard-code a lot of things, this should not take more than a few days, depending on how many changes we need to make to the existing tor bandwidth reporting code.

comment:3 Changed 6 weeks ago by teor

Actual Points: 0.4

I have a proof-of-concept for PrivCount in extra-info descriptors in https://github.com/torproject/tor/pull/640

Let's review it in #29004.

comment:4 Changed 6 weeks ago by teor

Keywords: 040-unreached-20190109 041-proposed added
Milestone: Tor: 0.4.0.x-finalTor: unspecified

These tasks are on the 0.4.1 roadmap for PrivCount and Sponsor V.

comment:5 Changed 6 weeks ago by teor

Keywords: 041-proposed-on-roadmap added; 041-proposed removed
Milestone: Tor: unspecifiedTor: 0.4.1.x-final

Let's review these tickets at the next meeting using our 041-proposed process.

They're on the roadmap, so the review should focus on ticket size and team capacity (and sponsor expectations).

comment:6 Changed 5 weeks ago by teor

Keywords: 041-accepted-20190115 added; 041-proposed-on-roadmap removed

These PrivCount tickets are on the 041 roadmap, we accepted their points estimates in 041 without discussion.

comment:7 Changed 3 weeks ago by nickm

Sponsor: SponsorV
Note: See TracTickets for help on using tickets.