Add top-10 relays by bandwidth history to Atlas?

Once upon a time we had a table with the top-10 relays by bandwidth history on the metrics website. We had to take it out for performance reasons. See #4612 (moved) for a discussion about putting it back.

I think the table doesn't belong on the metrics website, because we're in general not looking at single relays there, but only at aggregate statistics. The table would be better suited on Atlas or another Onionoo client.

The question is: would Atlas have room for a table of top-10 relays by bandwidth history? More generally, are there any other top-X relay lists that Atlas could display?

I'd understand if the answer is no, because Atlas is designed around a single search request, but maybe I'm overlooking something.

(If Atlas would want to display this table, I'd be happy to hack on the Onionoo part to provide the data, of course.)

added complete-before-abandoning-globe component::metrics/relay search owner::irl parent::6787 priority::low resolution::fixed severity::normal status::closed type::enhancement labels

I think this is a great idea, and my plans for Atlas is to not have it just centered around one search, but to have also other corollary features.

Where can I get the top 10 relays by bandwidth history? Would Onionoo be providing me with such list?

Replying to hellais:

I think this is a great idea, and my plans for Atlas is to not have it just centered around one search, but to have also other corollary features.

Ah, cool, I wasn't sure about that. Great! :)

Where can I get the top 10 relays by bandwidth history? Would Onionoo be providing me with such list?

Yes, Onionoo would provide you with such lists.

I'm not sure what other lists would be useful to have. That makes it difficult to design a good API. How about a URL like this (not implemented yet):

GET details/relays?orderby=-bwhist&limit=10

The -bwhist would mean "in descending order," but we could also add a desc parameter or something. The result would have its relay elements in sorted order, rather than unsorted as they are right now.

Of course, we could allow the same parameters for all other resources and URLS, too.

This feels like designing a database interface. I wonder if there are databases providing us with such an interface. I could rewrite Onionoo to be just the database importer and let the database do the hard work that databases are designed for. Hmmm.

There, I came up with a new API that uses parameters for filtering and sorting results. The top 10 relays by consensus weights would be (not implemented yet):

GET detail?type=relay&order=-consensus_weight&limit=10

Please review the new API, in particular the Methods section at the end, and let me know what you think. It's already implemented, but not deployed. If you like it, I'll deploy it next week.

I didn't hear any feedback, but I'm deploying the new API anyway. The top 10 relays by consensus weight can now be requested here. The new methods and parameters are described here.

Replying to karsten:

I didn't hear any feedback, but I'm deploying the new API anyway. The top 10 relays by consensus weight can now be requested here. The new methods and parameters are described here.

This looks good. I will be sure to get this into the future versions of Atlas.

Replying to hellais:

This looks good. I will be sure to get this into the future versions of Atlas.

Cool! Let me know if you come up with any other API parameters that you'd like Onionoo to support.

For reference, Globe can show top 10 relays by consensus weight using the GET request shown above.

Trac:
Username: tyseom
Cc: N/A to nusenu@openmailbox.org

I really like this feature in Globe and would love us to add it to Atlas before dumping Globe.

Trac:
Reviewer: N/A to N/A
Sponsor: N/A to N/A
Keywords: N/A deleted, complete-before-abandoning-globe added
Severity: N/A to Normal

Going to take a go at this, as Onionoo already can provide this data it should not be too difficult.

Trac:
Status: new to accepted
Owner: hellais to irl

Trac:
Parent: N/A to #6787 (moved)

A patch implementing this is available at:

https://gitweb.torproject.org/user/irl/atlas.git/patch/?id=4cff4597819a93412d2db14dd6f6cc7625009aa6

Adds a list of the top 10 relays by consensus weight (Fixes: #5430)

 * Adds a navbar to the top of the page for Home, About and Top 10
   Relays.
 * Re-uses the search results view using a hardcoded Onionoo URL (taken
   from Globe) to show a list of the top 10 relays by consensus weight.
 * Disables sorting by default of the datatables, and disables saving of
   state for sorting options. This allows the list of the top 10 to be
   sorted by consensus weight as reported by Onionoo, though manual
   sorting is still possible as before.

Trac:
Status: accepted to needs_review

Thanks irl, these changes look great!

I only have one remark: When testing the patch, I was surprised to see that the results weren't sorted by bandwidth. I think we should either 1) leave it as it is, but add a note on top saying that the results are sorted by consensus weight, or 2) sort the results by bandwidth. I don't care strongly either way. Any opinions?

As this is a "complete-before-abandoning-globe" ticket, I tried to replicate the functionality that Globe had as best I could. This meant sorting by consensus weight.

Adding a title for the search results was the next thing I was planning to do, but I saw this as a separate enhancement. There's no reason I can't make it part of this patch though.

Adding a title for the search results was the next thing I was planning to do, but I saw this as a separate enhancement. There's no reason I can't make it part of this patch though.

I would appreciate that. I think a small info box (maybe an alert div class?) is sufficient, saying something along the lines of "The results are sorted by consensus weight, but you can also sort them by bandwidth by clicking on the column head." I'm afraid that if we are not doing that, we will end up with occasional bug reports that misinterpret the output. Thanks!

https://gitweb.torproject.org/user/irl/atlas.git/patch/?id=223aec37d3b96556da949bf18d72298aa3c9b675

Adds a title to the search results page

 * Displays the search query for searches
 * Displays "Top 10 Relays by Consensus Weight" for the top 10 page to
   avoid confusion as to why the top 10 are not sorted by bandwidth
   (#5430)

This patch comes after the patch above. It appears I'm not allowed to rebase my branches in my personal repo.

Patches look good to me, though I didn't try them out. I agree that sorting by consensus weight is the better short-term solution. Sorting by bandwidth would require an Onionoo extension, which is certainly possible but which should happen after we shut down Globe. Note that sorting by bandwidth using the column header would only apply a different order of the top-10 relays by consensus weight, rather than return the top-10 relays by bandwidth. So, good to go if phw agrees that the second patch looks good. Thanks, irl and phw!

The patch looks great, thanks irl! I had to fix a merge conflict. Do you mind having a quick look at the last commits, to see if I merged your changes correctly? It's all in the master branch at https://gitweb.torproject.org/user/phw/atlas.git/

This all looks correct. (:

If you know of any way of allowing me to force push rebased branches to my user git repo on git.tpo, please let me know.

Replying to irl:

If you know of any way of allowing me to force push rebased branches to my user git repo on git.tpo, please let me know.

Hmm, it's the first time I hear that it's not working. Can you paste the command and result of that command from trying?

A possible workaround would be to commit subsequent patches with --squash or --rebase and ask the maintainer to simply squash those commits when merging.

Apparently it works now, might have been a temporary problem at the time I was trying it. So this is all good, merge away and then this ticket can be closed.

Thanks irl and karsten. Merged and deployed.

Trac:
Resolution: N/A to fixed
Status: needs_review to closed

closed

mentioned in issue #6787 (moved)

mentioned in issue #18893 (moved)