Opened 3 years ago

Closed 3 years ago

#23517 closed enhancement (fixed)

Add aggregated results table for relays grouped by country and/or AS

Reported by: karsten Owned by: irl
Priority: Medium Milestone:
Component: Metrics/Relay Search Version:
Severity: Normal Keywords: metrics-2018
Cc: metrics-team Actual Points:
Parent ID: Points:
Reviewer: Sponsor:


This feature request is inspired by #23509, which, IIUC, is about displaying aggregated data on a single page. However, I believe that the feature that I'm describing here is sufficiently different to deserve its own ticket. If that's not the case, feel free to merge tickets!

It would be cool to be able to group relays in the results table by country, by AS, or even by both. (Not by family, because grouping by family can be really tricky.)

The most common use case would be that somebody wants to learn which are the top 10 countries contributing to the network. Or the top 10 ASes. And then they might want to include only relays with the Exit flag. Or filter by a given country and group by AS.

Basically, this feature would make Compass obsolete, AFAICS. The only part in Compass that this would not implement is the "(almost) fast exit" distinction, and that's not a loss.

This feature might be combined with #23509 by including a link from an aggregated results table entry to an aggregated details page that would be implemented in #23509. But that's not a requirement. The aggregated results table in itself is an improvement.

Regarding links, there could be "Top 10 Countries" and "Top 10 ASes" links next to the existing "Top 10 Relays" link.

Regarding the implementation, I think that we could start with an implementation in Atlas and only consider adding another document type for this to Onionoo if the implementation in Atlas does not scale. Atlas would request a details document with only the fields to be aggregated and displayed in the table, but for all relays. That's a potentially quite large document (though it's much smaller when using the fields parameter). But it's probably quite easy to cache that on the server and in the browser (for different aggregations by country or AS). Atlas would then perform all aggregations and subsequent sorting in the browser.

What do you think? Did I miss anything that makes this feature a lot more difficult than it sounds in my description above?

Child Tickets

Change History (4)

comment:1 Changed 3 years ago by irl

I agree that this should be a separate ticket from #23509, I think this should encompass #23511 however with the top relays for an aggregation listed on the aggregated page. I will close that ticket as a duplicate as I think this one captures the vision better.

I'm concerned that this wouldn't be a new document in Onionoo, for two reasons:

1) It's a lot of logic to add to Atlas where we've always tried to minimise the logic and have Atlas serve only as a presentation layer on top of Onionoo
2) It prevents the data being easily reusable by others without their reverse engineering of the Atlas source code

comment:2 Changed 3 years ago by karsten

Hmm, those are two good reasons. Okay, let's not add this logic to Atlas. However, I'm concerned about adding this document to Onionoo, because it's quite different from the existing document types. We might first want to resolve things like #15844 and see if the solution used there also supports aggregated results.

New suggestion to move this ticket forward: Should we look into extending Compass to also offer its results as a JSON document (in addition to the HTML output which we would shut down at some point)? Atlas would then either query Onionoo directly for non-aggregated lists and Compass for aggregated lists. If this works out well, we could talk about moving this code to either Onionoo or Atlas later on. (Note how this wouldn't resolve #23509, though I'm not sure what the plan for that is.)

comment:3 Changed 3 years ago by karsten

Keywords: metrics-2018 added

comment:4 Changed 3 years ago by irl

Resolution: fixed
Status: newclosed

Fixed in 5463eb8 in Relay Search.

I would still like to move the actual aggregation to Onionoo, but for now this works and doesn't seem to have terrible performance. The aggregations load faster than the single relays do so this isn't a massive concern.

Note: See TracTickets for help on using tickets.