Opened 7 months ago

Last modified 2 hours ago

#25459 merge_ready enhancement

Add graph that compares total consensus weights across bandwidth authorities

Reported by: teor Owned by: karsten
Priority: Medium Milestone:
Component: Metrics/Statistics Version:
Severity: Normal Keywords:
Cc: tom, metrics-team, nikita, juga Actual Points:
Parent ID: Points:
Reviewer: Sponsor:

Description

We were talking today about how to compare bandwidth authorities, and make sure that different implementations are producing similar figures.

One easy way to compare bandwidth authority implementations is to compare their total bandwidth weights.

For example, this is the total bandwidth weight in the consensus:

$ cat Library/Application\ Support/TorBrowser-Data/Tor/cached-microdesc-consensus | grep "^w" | cut -d= -f2 | grep -v Unmeasured | paste -s -d + - | bc
43763163

Do you think we can do a graph of total bandwidth weight for votes?
Should this go somewhere on metrics?
Or should it be a once-off thing that Tom (or someone else) does?

Child Tickets

Attachments (5)

totalcw-2018-03-11.csv (89.3 KB) - added by karsten 7 months ago.
totalcw-2018-03-11.png (264.5 KB) - added by karsten 7 months ago.
totalcw-2018-03-11a.csv (2.1 MB) - added by karsten 7 months ago.
totalcw-2018-03-11a.png (368.5 KB) - added by karsten 7 months ago.
totalcw-2018-10-12.png (136.0 KB) - added by karsten 4 days ago.

Change History (26)

comment:1 Changed 7 months ago by karsten

Owner: changed from metrics-team to karsten
Status: newaccepted

This should start as a once-off thing. I'll make a first graph in the next few hours and post it here.

Changed 7 months ago by karsten

Attachment: totalcw-2018-03-11.csv added

Changed 7 months ago by karsten

Attachment: totalcw-2018-03-11.png added

comment:2 Changed 7 months ago by karsten

Cc: metrics-team added

Here's a graph showing total consensus weights across bandwidth authorities in March 2018:


Let me know if you want something else or another graph for a longer time frame (like all of 2018).

comment:3 Changed 7 months ago by teor

Thanks, this is interesting.

I'd like to see a graph from the "normal state" around September? 2017, through the DDoS in December and failure in February.

comment:4 Changed 7 months ago by karsten

Sure, that'll take a while. I'll post the graph as soon it's ready.

Changed 7 months ago by karsten

Attachment: totalcw-2018-03-11a.csv added

Changed 7 months ago by karsten

Attachment: totalcw-2018-03-11a.png added

comment:5 Changed 7 months ago by karsten

Here's a new graph covering the time between July 2017 and today:


comment:6 Changed 7 months ago by teor

Thanks!
For reference, we'll use this graph in the scaling part of the bandwidth authority session org/meetings/2018Rome/Notes/BandwidthAuthorityRequirements

comment:7 Changed 7 months ago by nikita

Cc: nikita added

comment:8 Changed 3 months ago by karsten

Status: acceptedneeds_information

It seems like the one-off analysis is done and there are no open question or requests for change particular to that.

Does that mean we can close this ticket? Or should we turn it into a ticket for adding a new graph to Tor Metrics at some point in the future?

comment:9 in reply to:  8 Changed 3 months ago by teor

Component: Metrics/AnalysisMetrics/Statistics
Status: needs_informationnew

Replying to karsten:

It seems like the one-off analysis is done and there are no open question or requests for change particular to that.

Yes, thanks, the total consensus weight graphs were really helpful.

Does that mean we can close this ticket? Or should we turn it into a ticket for adding a new graph to Tor Metrics at some point in the future?

It looks like we might need to scale sbws' results to match torflow.
So having these graphs permanently would be useful to determine the scaling parameters.
And they are useful to monitor the health of each bandwidth authority.

So let's make permanent graphs!

comment:10 Changed 3 months ago by karsten

Owner: changed from karsten to metrics-team
Status: newassigned
Summary: Compare total consensus weights across bandwidth authoritiesAdd graph that compares total consensus weights across bandwidth authorities

Sounds good. Blocking on #21378, but once the data is available in CollecTor, let's make a new graph.

comment:11 in reply to:  10 ; Changed 2 months ago by teor

Replying to karsten:

Sounds good. Blocking on #21378, but once the data is available in CollecTor, let's make a new graph.

I don't think archiving bandwidth files (#21378) is blocking this ticket. These graphs show the total bandwidth voted for by each bandwidth authority, and this total bandwidth is in the votes.

Specifically, we want to graph the sum of the "Measured" fields in "w" lines in votes:
https://gitweb.torproject.org/torspec.git/tree/dir-spec.txt#n2337

comment:12 in reply to:  11 Changed 2 months ago by karsten

Replying to teor:

Replying to karsten:

Sounds good. Blocking on #21378, but once the data is available in CollecTor, let's make a new graph.

I don't think archiving bandwidth files (#21378) is blocking this ticket.

Uhm, you're right. We can start working on this ticket as time permits. Sorry for the confusion!

comment:13 Changed 2 months ago by teor

Cc: juga added

No worries!

Please let juga know if there is anything they can do to help with these graphs (for example, reviewing the output).

comment:14 Changed 6 days ago by karsten

Owner: changed from metrics-team to karsten
Status: assignedaccepted

I started working on this ticket and pushed a temporary commit to the task-25459 branch in my repository. I'll now import a few months and then years of data. This may take a while...

Changed 4 days ago by karsten

Attachment: totalcw-2018-10-12.png added

comment:15 Changed 4 days ago by karsten

Importing the entire archive took almost two days, but now it's done, and it only needs to be done once. Here's the output:


What remains is write a description for metrics.json and add any glue that's missing to put the graph on Tor Metrics. I'll work on a patch next week.

comment:16 Changed 30 hours ago by karsten

Status: acceptedneeds_review

My updated task-25459 branch is now ready for review, with the last commit being 962bb6c.

comment:17 Changed 23 hours ago by irl

Status: needs_reviewmerge_ready

Looks good to me.

comment:18 Changed 21 hours ago by karsten

Cool! Created #28052 for the database setup. Will comment here as deployment goes forward.

comment:19 Changed 19 hours ago by teor

The graph looks good to me, and it will help me to answer PrivCount spec questions like "how many bits do we need to store consensus weight?"

comment:20 Changed 19 hours ago by arma

Ha, you can clearly see on that most recent graph where we tried the "enable PID feedback" experiment with torflow. Nice.

comment:21 Changed 2 hours ago by karsten

The graph is now available here: https://metrics.torproject.org/totalcw.html

I still need to deploy the data processor when the current run is done. Leaving the ticket open until then.

Note: See TracTickets for help on using tickets.