Opened 3 years ago

Last modified 17 months ago

#20403 new enhancement

Make it easier for relay operators to find their observed bandwidth

Reported by: teor Owned by: tom
Priority: Medium Milestone:
Component: Metrics/Consensus Health Version:
Severity: Normal Keywords:
Cc: tom, atagar, metrics-team Actual Points:
Parent ID: Points:
Reviewer: Sponsor:

Description

Hi tom, thanks for #20372, makes a big difference to relay operators.

I was answering another relay operator question today, and realised I needed the observed bandwidth. Because they were measured high, but then limited by the observed bandwidth.

Is that the sort of thing we can put on consensus health?
Under the fingerprint? (Because it's set by the relay, not an authority.)
Or another row?
Also, should we show the consensus weight?

Or is that just too confusing, and I should direct them to use Atlas, and hover over the bandwidth heading? (If it works, which it doesn't on high security tor browser.)

Child Tickets

Change History (6)

comment:1 Changed 3 years ago by tom

Cc: atagar added

So depictor (consensus-health) uses stem to get its info, so it's really easy for me to include any information here: https://stem.torproject.org/api/descriptor/router_status_entry.html It's really difficult to get anything else - downloading descriptors for the network would take a _lot_ of time, just downloading the votes and consesnsuses takes 10+ minutes.

I'm a little confused by the terms you're using. When you say "observed bandwidth" do you mean the relay's 'advertised bandwidth' a relay operator puts in the config, the unit-less values the bwauths calculate and vote on, or a third thing I'm not recalling? (And if it is the third thing, where does that get exposed: Consensus, MicroConsensus, Descriptor, Microdescriptor, or ExtraInfo?) And then the same question for "consensus weight".

Looking closer at the stem documentation, I might have done things wrong in #20372 actually. Adding Damian to confirm the below is correct:

  • On a vote .measured is the bwauth's vote for the relay's (unitless) bandwidth value
  • On a consensus .measured is the average* of the bwauths votes for the relay's (unitless) bandwidth value
  • On a vote or consensus .bandwidth is the _relay's_ advertised bandwidth (and is what teor wants available)

*not really average but let's pretend it is for simplicity

(Currently, the value on each Authority is the .measured but the value on the consensus is .bandwidth - which would be wrong as the consensus would be reporting the self-reported advertised bandwidth...)

comment:2 in reply to:  1 ; Changed 3 years ago by teor

Replying to tom:

So depictor (consensus-health) uses stem to get its info, so it's really easy for me to include any information here: https://stem.torproject.org/api/descriptor/router_status_entry.html It's really difficult to get anything else - downloading descriptors for the network would take a _lot_ of time, just downloading the votes and consesnsuses takes 10+ minutes.

I think I might be asking for the "bandwidth (int) -- bandwidth claimed by the relay (in kb/s)", but I am not sure how it is sourced.

I'm a little confused by the terms you're using. When you say "observed bandwidth" do you mean the relay's 'advertised bandwidth' a relay operator puts in the config, the unit-less values the bwauths calculate and vote on, or a third thing I'm not recalling? (And if it is the third thing, where does that get exposed: Consensus, MicroConsensus, Descriptor, Microdescriptor, or ExtraInfo?)

By "observed bandwdith" I mean the "bandwdith-observed" figure from the descriptor "bandwdith" line:
https://gitweb.torproject.org/torspec.git/tree/dir-spec.txt#n418

I'd like it more easily available because it's one of the two figures that an operator doesn't set themselves - the other is the measured bandwidth.

And used in conjunction with the advertised bandwidth to calculate the bandwidth in the consensus "w" line:
https://gitweb.torproject.org/torspec.git/tree/dir-spec.txt#n2268

(That figure is useful, but more confusing, because it's sometimes the advertised bandwidth, and sometimes the observed bandwidth.)

And then the same question for "consensus weight".

Whatever Atlas uses for consensus weight in the relay details page.
I assume it's measured bandwidth from the consensus w line, but I'm not sure.

If it is, it's already there in the consensus column of the relay flags table.

https://gitweb.torproject.org/torspec.git/tree/dir-spec.txt#n2274

Looking closer at the stem documentation, I might have done things wrong in #20372 actually. Adding Damian to confirm the below is correct:

  • On a vote .measured is the bwauth's vote for the relay's (unitless) bandwidth value
  • On a consensus .measured is the average* of the bwauths votes for the relay's (unitless) bandwidth value
  • On a vote or consensus .bandwidth is the _relay's_ advertised bandwidth (and is what teor wants available)

Not quite, I think it's the minimum of advertised and observed, which confuses a lot of operators, because sometimes it's controlled by the torrc, and other times by their relay's self-reported performance:
https://gitweb.torproject.org/torspec.git/tree/dir-spec.txt#n418

*not really average but let's pretend it is for simplicity

(low-median)

(Currently, the value on each Authority is the .measured but the value on the consensus is .bandwidth - which would be wrong as the consensus would be reporting the self-reported advertised bandwidth...)

That's strange, because all the figures in the table appear to be a low-median (consensus .measured) to me.

comment:3 in reply to:  2 Changed 3 years ago by tom

Replying to teor:

Replying to tom:

So depictor (consensus-health) uses stem to get its info, so it's really easy for me to include any information here: https://stem.torproject.org/api/descriptor/router_status_entry.html It's really difficult to get anything else - downloading descriptors for the network would take a _lot_ of time, just downloading the votes and consesnsuses takes 10+ minutes.

I think I might be asking for the "bandwidth (int) -- bandwidth claimed by the relay (in kb/s)", but I am not sure how it is sourced.

So I'm pretty sure stem's documentation is wrong.

On my relay (rittervg)'s consensus.routersC0EDB08D7540D1DD3CA69809ED17D979F51B66E3? https://atlas.torproject.org/#details/C0EDB08D7540D1DD3CA69809ED17D979F51B66E3 RouterStatusEntry:

  • .is_unmeasured is false
  • .measured is empty
  • .bandwidth is the value from the consensus w line (the agreed upon value by the bwauths)

If I manually pull all the server descriptors (which I'm trying to avoid but I guess I could) I get the following new values:

  • .observed_bandwidth - 1434333
  • .average_bandwidth - This is the value Atlas displays under 'Advertised Bandwidth'
  • .burst_bandwidth - 1536000

My relay's torrc says

  • RelayBandwidthRate 1250 KBytes
  • RelayBandwidthBurst 1500 KBytes

I'm a little confused by the terms you're using. When you say "observed bandwidth" do you mean the relay's 'advertised bandwidth' a relay operator puts in the config, the unit-less values the bwauths calculate and vote on, or a third thing I'm not recalling? (And if it is the third thing, where does that get exposed: Consensus, MicroConsensus, Descriptor, Microdescriptor, or ExtraInfo?)

By "observed bandwdith" I mean the "bandwdith-observed" figure from the descriptor "bandwdith" line:
https://gitweb.torproject.org/torspec.git/tree/dir-spec.txt#n418

I'd like it more easily available because it's one of the two figures that an operator doesn't set themselves - the other is the measured bandwidth.

And used in conjunction with the advertised bandwidth to calculate the bandwidth in the consensus "w" line:
https://gitweb.torproject.org/torspec.git/tree/dir-spec.txt#n2268

(That figure is useful, but more confusing, because it's sometimes the advertised bandwidth, and sometimes the observed bandwidth.)

And .observed_bandwidth from the descriptor also

And then the same question for "consensus weight".

Whatever Atlas uses for consensus weight in the relay details page.
I assume it's measured bandwidth from the consensus w line, but I'm not sure.

If it is, it's already there in the consensus column of the relay flags table.

https://gitweb.torproject.org/torspec.git/tree/dir-spec.txt#n2274

Yes, I'm 95% sure this is working correctly currently.

Looking closer at the stem documentation, I might have done things wrong in #20372 actually. Adding Damian to confirm the below is correct:

  • On a vote .measured is the bwauth's vote for the relay's (unitless) bandwidth value
  • On a consensus .measured is the average* of the bwauths votes for the relay's (unitless) bandwidth value
  • On a vote or consensus .bandwidth is the _relay's_ advertised bandwidth (and is what teor wants available)

Not quite, I think it's the minimum of advertised and observed, which confuses a lot of operators, because sometimes it's controlled by the torrc, and other times by their relay's self-reported performance:
https://gitweb.torproject.org/torspec.git/tree/dir-spec.txt#n418

*not really average but let's pretend it is for simplicity

(low-median)

Ah interesting, so it just takes the median value (and if there's an even number, the lower of the two in the middle? Didn't realize that.

(Currently, the value on each Authority is the .measured but the value on the consensus is .bandwidth - which would be wrong as the consensus would be reporting the self-reported advertised bandwidth...)

That's strange, because all the figures in the table appear to be a low-median (consensus .measured) to me.

Yup, turns out the stem documentation seems to be wrong.

comment:4 Changed 3 years ago by tom

Component: MetricsMetrics/Consensus Health
Owner: set to tom

comment:5 Changed 3 years ago by tom

Opened #20493 for the stem documentation.

comment:6 Changed 17 months ago by irl

Cc: metrics-team added

Adding metrics-team to cc

Note: See TracTickets for help on using tickets.