Graph results from the torflow to sbws transition

After we get some solid graph methodology that we like from #33076 (moved), we should use those graphs to closely compare the sbws consensus votes to TorFlow consensus votes.

After that, when we try to switch to sbws, we should get before and after graphs of onionperf data, like we decided on for #33076 (moved).

Trac:
Parent Ticket: #33121 (moved)

added actualpoints::2 component::metrics/analysis owner::metrics-team parent::33121 points::4 priority::medium sbws-roadmap severity::normal status::new type::task labels

Trac:
Keywords: N/A deleted, sbws-roadmap added

This one is needed for sbws deployment.

Trac:
Keywords: sbws-roadmap deleted, sbws-roadmap metrics-team-roadmap-2020Q1 added

The goal is to deploy sbws in all bw authorities. We need to fix critical bugs to do this.

Trac:
Parent: N/A to #33121 (moved)

hopefully

Trac:
Summary: Graph results from the sbws to torflow transition to Graph results from the torflow to sbws transition

Mikeperry: Can we have child tickets for the other graphs that will be needed for each transition?

Trac:

From #33076 (moved) (mikeperry's words):

For #33077 (moved), I think it would be helpful to group the votes in some way based on which dirauths are using sbws vs torflow. Not sure if that means overlay all the CDFs or just combine them, or do some additional work to take the median values of measured and min_rate_observed from each set and make one graph for TorFlow and one for sbws votes. On the one hand, we will want to pick an option that is likely to be useful for future experiments. On the other, we will need to really dig into what the network load balancing would look like under each system.

It took me way too long to match directory authority nicknames to the bandwidth authority software they were running around the time of Rob's experiment. It's not particularly hard, but I ran into several metrics-lib defects that I'll still have to fix properly. I finally came up with the newly attached graph that has one CDF per bandwidth authority, colored red for torflow and blue for sbws.

Is this graph useful? Any suggestions how to plot these 15 lines in a way that is even more useful?

Unless I hear otherwise, I'll make two more graphs tomorrow for the time when the first authority switched from torflow to sbws (+/- 5 days) and another one when the second authority switched.

Trac:

I made another graph first that shows when directory authorities switched from torflow to sbws:

It looks like longclaw switched back and forth between sbws and torflow on 2019-02-23 and once more around 2019-03-20, and bastet briefly switched from torflow to sbws on 2019-03-09 before switching back. Oh, and maatuska temporarily switched to sbws between 2019-12-15 and 2019-12-24 before switching permanently on 2020-01-08.

I'll graph the time around the last switches from torflow to sbws. That means the days around 2019-05-28 for the bastet switch and the days around 2020-01-08 for the maatuska switch. I don't have the data to say when exactly longclaw switched, but I can make another graph for that if somebody tells me.

Trac:

Here's the CDF-Relay-Stream-Capacity graph for the time when bastet switched from torflow to sbws:

I had trouble with maatuska's switch, in particular with plotting its torflow values from before the switch. But this graph likely looks the same as the one above, so I'll leave it out for now.

What else do you need here?

Estimating 3 points for making these graphs plus 1 point for cleaning up the code and providing it in metrics-tasks.git. Also estimating that we already spent 2 of these points for making and discussing attached graphs.

Trac:
Points: N/A to 4
Actualpoints: N/A to 2

Move some of the tickets from last metrics roadmap to the roadmap in April.

Trac:
Keywords: sbws-roadmap metrics-team-roadmap-2020Q1 deleted, metrics-team-roadmap-2020April, sbws-roadmap added

Trac:
Keywords: metrics-team-roadmap-2020April deleted, N/A added

changed time estimate to 32h

added 16h of time spent

mentioned in issue #33206 (moved)

mentioned in issue #33121 (moved)