Opened 7 years ago

Closed 2 years ago

#6730 closed enhancement (wontfix)

Display newly added or recently disappeared relays or bridges

Reported by: mo Owned by: metrics-team
Priority: Low Milestone:
Component: Metrics/Compass Version:
Severity: Normal Keywords:
Cc: gsathya Actual Points:
Parent ID: Points:
Reviewer: Sponsor:

Description

It would be nice to have a way to track new nodes appearing in the datasets. At the moment, I compare "by eyes" if there are any new relays popping up.

For example, a color bar for nodes that Compass saw for the first time for a few days or a specified timeframe would be great.

Child Tickets

Change History (10)

comment:1 Changed 7 years ago by mo

Type: defectenhancement

comment:2 Changed 7 years ago by karsten

Agreed, this information would be very useful to have for a community manager person. Unfortunately, it's data that Onionoo doesn't provide yet, and adding that data is blocking on the Python rewrite of Onionoo. Please see #6509 for more details on the data-providing side of things.

Leaving this ticket open to discuss the Compass integration once Onionoo provides the data.

comment:3 Changed 7 years ago by gsathya

Moritz made https://github.com/moba/tor-fast-exits-tracker to track changes. Maybe integrate this into compass?

comment:4 in reply to:  3 Changed 7 years ago by karsten

Replying to gsathya:

Moritz made https://github.com/moba/tor-fast-exits-tracker to track changes. Maybe integrate this into compass?

No. That script seems like a fine hack, but not like a good design. We should not integrate it into Compass. Compass should not create an archive of Onionoo documents. In fact, no Onionoo client should be forced to do that. Such a history based on Onionoo documents starts on the day when you set up the script, and there's no way to import past history. Also, whenever the script fails, there will be missing data that cannot be recovered.

I guess what we should do instead is generalize the desired feature as much as possible and put that into Onionoo itself. Onionoo has access to the last 72 hours of Tor data, so can recover from failures up to 72 hours without missing any data, and it's easy to import descriptors from the past weeks, months, or even years. What's the exact feature here? Maybe we can integrate that data into Onionoo somehow.

Note that this discussion is not quite what this ticket is about, which is to detect newly added or recently disappeared relays or bridges. Please open a new ticket.

comment:5 Changed 7 years ago by karsten

I should note that if it turned out that this feature doesn't fit into Onionoo, the alternative would be to build something that's similar to Onionoo. Such a tool would then obtain relay descriptors from metrics-web's rsync server, parse them via metrics-lib or stem, and keep its own history in a local database. That would be much, much better than archiving Onionoo's data. But before we fork/rewrite Onionoo for this, let's see if we can extend it.

comment:6 Changed 7 years ago by karsten

I just created #7895 for discussing how to integrate Moritz' script into Onionoo. Please take the subsequent discussion there, because this ticket is about detecting newly added or recently disappeared relays or bridges, not about fast exits.

comment:7 in reply to:  2 Changed 7 years ago by karsten

Replying to karsten:

Agreed, this information would be very useful to have for a community manager person. Unfortunately, it's data that Onionoo doesn't provide yet, and adding that data is blocking on the Python rewrite of Onionoo. Please see #6509 for more details on the data-providing side of things.
Leaving this ticket open to discuss the Compass integration once Onionoo provides the data.

#6509 is implemented. Compass or any other Onionoo client can now start using the following two parameters:

  • first_seen_days: Return only relays or bridges which have first been seen during the given range of days ago. A parameter value "x-y" with x >= y returns relays or bridges that have first been seen at least x and at most y days ago. Accepted short forms are "x", "x-", and "-y" which are interpreted as "x-x", "x-infinity", and "0-y".
  • last_seen_days: Return only relays or bridges which have last been seen during the given range of days ago. A parameter value "x-y" with x >= y returns relays or bridges that have last been seen at least x and at most y days ago. Accepted short forms are "x", "x-", and "-y" which are interpreted as "x-x", "x-infinity", and "0-y". Note that relays and bridges that haven't been running in the past week are never included in results, so that setting x to 8 or higher will always lead to an empty result set.

comment:8 Changed 2 years ago by karsten

Severity: Normal
Summary: A way to see changes in datasets over timeDisplay newly added or recently disappeared relays or bridges

Try to make the summary more accurate/useful.

comment:9 Changed 2 years ago by karsten

Owner: set to metrics-team
Status: newassigned

comment:10 Changed 2 years ago by irl

Resolution: wontfix
Status: assignedclosed

This functionality is present in Relay Search.

For newly appeared relays: https://atlas.torproject.org/#search/first_seen_days:-1

For recently disappeared relays: https://atlas.torproject.org/#search/running:false%20last_seen_days:1-2

Note: See TracTickets for help on using tickets.