Add graph with new relays per day (identified by fingerprint)

[cypherpunks]

It is not to easy to spot days that have an unusually high rate of newly added relays by looking at [1]. What about a new graph that shows only the number of new unique relay fingerprints first seen on a given day?

[1] ​https://metrics.torproject.org/networksize.html

[johnbwilliams]

proposed code for requested graph: ​http://rpubs.com/johnbwilliams/networkchanges

[cypherpunks]

Thanks John for working on this.

I think there is a slight misunderstanding what kind of graphs are requested in this ticket, BUT I would still like to see this graphs being added to metrics.

The data available in
​https://metrics.torproject.org/stats/servers.csv
​https://metrics.torproject.org/stats.html#servers

does not contain the required data to draw the graphs requested in this ticket, since servers.csv only shows absolute numbers and does not say anything about when a relay has first been seen on the tor network (a new fingerprint never seen before).
"first_seen" as defined in
​https://onionoo.torproject.org/#details (first_seen field)

• Could you separate relay graphs from bridge graphs (due to the huge spikes for the bridges)?

[cypherpunks]

I filed #22425 for the graphs you added.

[cypherpunks]

The required data could easily be created by fetching all new relays/bridges from onionoo once a day.

[johnbwilliams]

I am interested in your proposed metric of new unique relays - I consider it to be one of many time series in a data warehouse for privacy intelligence which I would like to implement. I would need to collaborate with you and others to identify signals to be included. Interested in collaborating to create this data warehouse?

[karsten]

Tweaked the summary a tiny bit.

A few comments:

• The suggested graph is indeed not correct, because the necessary data is not yet available. I also don't find the suggested graph as useful on its own, which is why I closed #22425. Please refer to the reason for closing that I wrote down there.
• Setting up a data warehouse sounds like a fun project, but it's currently out of scope for metrics team work. That shouldn't stop others from doing it, of course.
• Last but not least, if we want to provide the graph suggested in this ticket, we'll have to write a new data-processing module in metrics-web that provides exactly the data we need.