Opened 19 months ago

Last modified 13 months ago

#24834 new enhancement

Map consensus weight vs bandwidth for each bandwidth authority's votes

Reported by: teor Owned by: metrics-team
Priority: Medium Milestone:
Component: Metrics/Relay Search Version:
Severity: Normal Keywords: tor-bwauth-needs
Cc: ln5, tom, juga Actual Points:
Parent ID: #24674 Points:
Reviewer: Sponsor:

Description

The new map on Relay Search is great!
The consensus weight versus bandwidth map has been really helpful for analysing bandwidth authority behaviour.
We now know that the measurements are biased towards the north of the northern hemisphere. (We don't know why yet: maybe the Internet is better, maybe the relays are all there, or maybe the bandwidth authorities are all there.)

I can look at the bias for the Guard and Exit flags by doing an aggregate search, then mapping it.

But I'd also like to see the bias for each bandwidth authority, using the bandwidth figures in their votes.
Is this possible in Relay Search?

Having this map on a country level would help us when we start experimenting with moving the servers around, or pooling them, or using a CDN.

Child Tickets

TicketTypeStatusOwnerSummary
#16843enhancementassignedmetrics-teamAdd all bwauth measurements (from votes)

Change History (15)

comment:1 Changed 19 months ago by teor

Maybe we also want to map consensus weight for each authority while we're doing this.

comment:2 Changed 19 months ago by irl

This would require information that is not in Onionoo. I don't know if the consensus contains this information or if it would only come from votes (I know consensus-health displays it in the consensus column, but is that synthentic?). Onionoo doesn't have votes yet.

comment:3 in reply to:  2 Changed 19 months ago by teor

Replying to irl:

This would require information that is not in Onionoo.

Yes, that's ticket #16843.

I don't know if the consensus contains this information or if it would only come from votes (I know consensus-health displays it in the consensus column, but is that synthentic?). Onionoo doesn't have votes yet.

consensus-health gets bandwidths from the votes and the consensus. The only synthetic flag is the deciding bwauth(s), which is calculated by matching the consensus bandwidth with the vote bandwidths.

(Consensus-health doesn't load descriptors.)

comment:4 Changed 18 months ago by irl

Ok, good to know we have a ticket for adding the data. I couldn't remember if we did or not.

As a quick and rather horrible hack: https://people.torproject.org/~irl/volatile/rsvotes/

This uses screen scraping of consensus health to generate some JSON that can be consumed by Relay Search, and needs to be manually updated. The data was based on the consensus published 2018-01-09 04:00:00.

If consensus health could write out the JSON, then these maps could in theory be integrated into Relay Search but proper caching would be needed to avoid repeatedly fetching large documents.

comment:5 Changed 18 months ago by teor

consensus-health uses stem: a small stem program could write out the per-bandwidth-authority votes. And the output would be much smaller than consensus-health's flags table.

comment:6 Changed 18 months ago by teor

What format did you need?

comment:7 Changed 18 months ago by teor

(Also, the map is missing Faravahar. Was it missing from that particular vote?)

comment:8 Changed 18 months ago by atagar

consensus-health uses stem: a small stem program could write out the per-bandwidth-authority votes

Hi Tim. For what it's worth we already have that in our examples...

https://stem.torproject.org/tutorials/examples/votes_by_bandwidth_authorities.html

comment:9 Changed 18 months ago by irl

If there is a concrete plan and resources available for #16843, I could provide a short-term solution using Stem. I really don't want to do this unless I know it's going to be short-term though.

comment:10 Changed 18 months ago by teor

Cc: tom added

irl, can I ask for an even more horrible hack?

tom has set up a bandwidth authority using Fastly as the server in #24506.
He did an analysis here:
https://tomrittervg.github.io/bwauth-tools/analyses/fastly-01.png

But a map analysis of consensus weight vs bandwidth would be even more helpful to detect the kinds of effects that Mike is concerned about.

comment:11 Changed 18 months ago by tom

All my data is at https://bwauth.ritter.vg/bwauth-patches/

For the vastly comparison start at the file bwscan.20171214-1545 and go until the data runs out.

comment:12 Changed 18 months ago by teor

We'll just want one file for a point-in-time comparison.
So we could use the latest file, or:
https://bwauth.ritter.vg/bwauth-patches/bwscan.20180115-1345

The "bw=" field is the same as the "Bandwidth=" in the votes.

comment:13 Changed 15 months ago by iwakeh

Please be aware of the start for RS into Metrics-Web integration and also refer to the branch in this comment of the parent ticket.

comment:14 Changed 14 months ago by juga

Cc: juga added

comment:15 Changed 13 months ago by teor

Cc: teor@… removed

Remove useless CC

Note: See TracTickets for help on using tickets.