Opened 5 years ago

Closed 3 years ago

#13709 closed enhancement (wontfix)

Track some Dir Auth network stats

Reported by: grarpamp Owned by:
Priority: Medium Milestone:
Component: Metrics/Website Version:
Severity: Normal Keywords:
Cc: Actual Points:
Parent ID: Points:
Reviewer: Sponsor:

Description

We have client growth charts, need corresponding charts for rest of the net / node types, including that suitable for provisioning such nodes.

bits/s
frames/s
tcp connections/s
tcp connections outstanting

Child Tickets

Change History (7)

comment:1 Changed 5 years ago by karsten

Status: newneeds_information

Can you explain why we'd need statistics on this level of detail?

Also, did you see the available charts on relays and bridges on the Tor Metrics website?

comment:2 Changed 5 years ago by grarpamp

You'd want to know how all components of the net scale as a whole.
You want to be tracking a bunch of stuff so you can see any changes reflected.
Those interested in running parallel tor networks would also want to know this because they will need to bootstrap, support and plan for certain node types, dirauths especially.

I don't believe this chart is from or otherwise reflects the dirauths.
https://metrics.torproject.org/bandwidth.html#dirbytes

comment:3 Changed 5 years ago by karsten

Relays report usage statistics, and directory authorities are relays. It's easy to make graphs for just directory authorities. The reason why there are no such graphs covering only directory authorities on Tor Metrics is that I didn't see an immediate need for them. The average user probably doesn't care, and whoever wants to run their own Tor network should be technically capable of generating these graphs on their own.

But I'm open to change my mind about average users not caring: what specific graphs do you think would the average user want to see on Tor Metrics, and why?

Going back to the start of this ticket, I don't think we need to gather new statistics in extent to what relays, including directory authorities, currently report.

comment:4 Changed 5 years ago by grarpamp

Beyond curiosity and coolness, this is not for users. The dirauths
(set of 10) are a separate bootstrap nondistributed set of auths,
'dirbytes' commingles that subset of 10 and thus doesn't call out
their purpose. The above stats are relevant to anyone wishing to
deploy parallel tor nets such as for data or any other purpose.
They'd want to see them to provision boxes accordingly ahead of
time. ie: If they expect 10k/100k/1M/10M/100M users, what will their
dirauths (that they essentially pay for) roughly look like at the
ISP/box level. Tor at scale is a representative live model that
can't be generated in a tiny simulation sandbox beforehand. Stats
relavent to tor for the same reasons. ie: any future dirauths

You can probably get all this from 'netstat', aggregate and divide
by 10 if you want and remember to update on dirauth change.

comment:5 Changed 5 years ago by karsten

Resolution: fixed
Status: needs_informationclosed

There are ten or so directory authorities, but there are also a few thousand directory mirrors. The "dirbytes" reported by the directory authorities mostly result from new clients bootstrapping, whereas the "dirbytes" reported by the directory mirrors are spent on recurring clients.

The graph on Tor Metrics indeed puts all "dirbytes" together into a single graph. Let me try to explain that a bit better on Tor Metrics: "The following graph shows the portion of consumed bandwidth that directory authorities and mirrors spent on answering directory requests. Not all directories report these statistics, so that the graph shows an estimation of total consumed bandwidth as it would be observed if all directories reported these statistics." If you have suggestions for better text to describe the graph, please let me know.

Apart from that I don't see what's left to do. I don't see why we'd need directory authorities or mirrors to report more data. And I don't see a need to make another graph for this. That's why I'm closing this ticket now. Thanks for your input!

comment:6 Changed 5 years ago by grarpamp

Resolution: fixed
Status: closedreopened

I'll restate another way in one sentence...
Graph the bandwidth [etc] used by the specific subclass of the roughly 10 bootstrap dir auths.

You can't do it today because AFAIK dirbytes does not specifically track the unique bootstrap function of that set of 10 hosts. As you state, dirbytes is composed of both 'auths and mirrors'. AFAIK, mirrors do not carry bootstrap load past the consensus expiry, if at all. Only auths do that. So the real stats for that function are today unavailably buried beneath a single 'dirbytes' graph.

People who want to run parallel tor protocol nets need the auth bandwidth data so they know how much to buy/locate to support their nets at any given scale. They don't care about mirror bandwidth since mirrors come from the user community, not the auth group. [TPO's relation to the current auth group is of no importance here. That there can be more than one tor protocol net in operation is.]

I've no more to add.

comment:7 Changed 3 years ago by karsten

Resolution: wontfix
Severity: Normal
Status: reopenedclosed

I believe that fallback directories introduced a few years ago made the bandwidth-for-bootstrapping reason moot.

And private Tor networks are not something that we care about. It's not something we discourage in any way, but it's also not something that we'd direct development efforts to specifically.

Re-closing for various reasons, some stated in this comment and some in earlier comments.

Note: See TracTickets for help on using tickets.