new field: measured flag

added component::metrics/onionoo priority::medium resolution::implemented status::closed type::enhancement labels

Trac:
Username: tyseom
Cc: N/A to tyseom

Instead of making this field a boolean I'd say make it an int that shows the measured bw and if unmeasured has a value of -1.

Trac:
Cc: tyseom to tyseom, aagbsn

The Consensus Weight is the measured bandwidth from bandwidth authorities. If you're going to add a key based on measured status it might also be worth considering a last_measured based on the last time the relay had a weight line in consensus without Unmeasured.

The graphs you are interested in wouldn't be as easy to produce based on a measured key because then you need to check every hour to see if the status for measured changed for every relay. It's also not exactly deterministic when a measurement is taken (based on ticket review). You're asking Onionoo to do what you could just as easily do with CollecTor and basic system tools.

If you're interested in graphing bandwidth data over time you might be better served by the Onionoo weights document which tracks Consensus Weight.

Trac:
Username: leeroy

Replying to leeroy:

You're asking Onionoo to do what you could just as easily do with CollecTor and basic system tools.

Correct, but isn't almost any other field also sourced from CollecTor data as well?

If the number of bw authorities remain as low a currently it would even be handy to include the measurement per bw auth.

I just mean why ask for a time to implement in the next year what you can just as easily do without imposing a load on Onionoo? It's not deterministic when it occurs, so you cannot rely on it until the BWAuth tickets are addressed. When will they be fixed is the better question. If you're looking for measurement per BWAuth, Onionoo is the wrong place to ask for an enhancement. As you say the source is CollecTor which is based on live network data. Is the measurement per BWAuth a part of live network data? Then the component for this ticket is not Onionoo.

Trac:
Username: leeroy

Adding a measured flag to details documents shouldn't be difficult. Though the primary purpose would be for operators to find out easily why their relay is not used, not for developers to graph the fraction of unmeasured relays (using CollecTor's data directly is indeed the better way for that).

A few remarks on the possible specification of such a field:

This field shouldn't be combined with the existing "consensus_weight" field by setting that to -1 if a relay is unmeasured, because that would override the (unmeasured) bandwidth value contained in the consensus. This change to the current field semantics would also constitute a major protocol change, requiring client developers to update their clients.
Turning this field into a "last_measured" field is harder to implement and would require re-parsing descriptor archives. I'd rather want to avoid that.
Including all bwauth measurements would certainly be handy, but that would require parsing votes which we don't do right now. Onionoo is already choking on parsing all the descriptors published every hour, and votes are not exactly tiny. I'd say don't expect this to happen anytime soon. But I agree that it would be really useful to have.

I'm sketching out the implementation off the top of my head, in case somebody else wants to try hacking on this:

Extend NodeDetailsStatusUpdater to include whether a relay was measured in the relay's NodeStatus and to later include that in the relay's DetailsStatus. Look out for NodeStatus.getConsensusWeight() and NodeStatus.setConsensusWeight() to get the idea.
Extend DetailsDocumentWriter to include this information in the relay's DetailsDocument. Similarly, look out for DetailsStatus.getConsensusWeight() for an example.
Include new "measured" field in the ResponseBuilder hack in writeDetailsLines() to support the fields parameter.
Raise protocol version to 2.5 or whatever the next minor version is. Do this in build.xml and ResponseBuilder and web/protocol.html. Describe new field in web/protocol.html.

I probably won't get to it this week, but I'll try next week if nobody else has coded this until then. It shouldn't be hard. Happy to review (partial) patches.

I wrote a simple script that uses stem to produce csv formatted measurement counts from votes (archived or from the Directory Authorities directly) - it could be easily extended to provide per-bwauth values for each relay, per vote.

Note that parsing the votes seems to take ages - I have a really anemic system and the vote parsing script is only parallelized so far as stem DescriptorReader spawns a separate thread.

I had been thinking along the lines of extracting the relavent attributes from the votes, and providing an API endpoint that would emit json, to produce client-side graphs using d3.js - rather than try and render graphs for every relay, every consensus period. Thoughts?

Trac:
unmeasured.py

Script to parse votes and count unmeasured relays

Karsten, implementing last_measured is actually trivial. Onionoo already parses consensus so the time for parsing is already used. Unmeasured is just another piece of data in the weight line so last_measured is based on the last time it was missing. It's trivial because it only requires comparing the timestamp stored (and can be done without string comparison using a hash if needed). It's up to you if it should be implemented though.

A node can have consensus weight without being measured. In which case I don't see how this could possibly help for debugging. Either the BWAuth's are experiencing a known problem (and the problem is easy to identify), or not (and the field does nothing to help). On the other hand an operator can use the weights history document to see changes in bandwidth weight (besides other goodness) in hourly intervals.

It's only if they're new, or reset, and expect a BWAuth scan, to have a problem in obtaining consensus weight. It's also not a future guarantee to be able to get per BWAuth data. Consensus is computed on meeting the threshold number of measurements. From BWAuth tickets dedicated BWAuths scan certain percentiles. So measurements may be the same BWAuth, not unique. In this case you would be parsing votes just to get the different view of bandwidth, from the same BWAuth, at different DirAuth.

But if you're still determined to do this I would propose an optimization. Only parse the current vote, don't parse archives. As a comparison, parsing current consensus takes ~880ms while parsing current votes takes 11s. Which is based on the ideal read benchmark I posted elsewhere.

Trac:
Username: leeroy

Each Bandwidth Authority scans the whole network, but the task is split up across 4 (now 8) processes that run on the same physical machine.

I'm also interested to see (e.g. visualize) how different measurements look as seen by each Bandwidth Authority for a given relay - I think this kind of graph would be helpful to think about how geographic diversity in measurement endpoints affects the bandwidth distribution of the Tor network.

What about the separate tickets for BWAuths. In particular having dedicated BWAuth for certain percentiles. Unless the two changes are coordinated you will eventually end up obtaining the same BWAuth data.

Trac:
Username: leeroy

I'm not sure if I understand what you are suggesting. Currently each bandwidth voting Directory Authority must receive a complete (>60% of the network) set of measurements, which are obtained independently from the other Directory Authority operators. Each Bandwidth Authority operator measures the entire Tor network (in the ideal case).

The tickets about splitting up or balancing Bandwidth Authorities have to do with making the separate scanner processes complete their fraction of the network at the same time as the other scanner processes (in the same Bandwidth Authority installation), so that the whole network is measured on approximately the same time interval. This isn't very pretty -- hope this makes a bit more sense now?

I appreciate the clarification. So the only concern would be if an uncoordinated change to BWAuth behavior occurs. And BWAuths don't have a spec, so it can happen. Wouldn't your desire to visualize measurements be out of scope for Onionoo? It's closer to metrics-web.

Trac:
Username: leeroy

The spec is hidden here: https://gitweb.torproject.org/torflow.git/tree/NetworkScanners/BwAuthority/README.spec.txt

My desire is to provide the appropriate amount of data so that it can be visualized, and yes, I'm not sure where the best home for these metrics might be. Perhaps this is an off-topic aside for this ticket, but perhaps there is a way for onionoo to provide queryable bandwidth history?

Oh, a spec I've not read. Thank you. Why's it hidden?

Visualizing measurements is off-topic. Providing the current measurements data, from votes, just so someone can make graph is out of scope (unless you want to change Onionoo's scope). The BWAuth measurements, and when they occur are a property of BWAuths, not a running relay. Visualizing them is better done from metrics-web. Onionoo already provides the Consensus Weight and Weights history document.

That being said as long as Onionoo doesn't try to build a history object from archives (because that's bad news) it looks doable on an hourly basis. If you really intend to expect an operator to parse votes archives then Onionoo needs to make producing the bandwidth measurements optional to setup.

To reinforce feasibility perhaps it would be better to test it first. An extension of the previous parsing result would be "Parse the current interval vote, obtain an !Iterable mapping fingerprint to set of measurements", and see what kind of memory/time is required. Much better than blindly hacking in new code. But it's up to you's.

Trac:
Username: leeroy

Note that parsing the votes seems to take ages...

Quick thing Aaron: if you're not using Stem 1.4.1 then I'd suggest upgrading since it introduces lazy loading.

new field: measured flag

Child items 0

Activity