Exclude exit-port statistics from extra-info descriptors
Very fast relays generate exit-port statistics that require 40-45 KB. These statistics can easily make the relay generate an extra-info descriptor that exceeds the 50 KB upload limit. See #2183 (moved).
Should we exclude exit-port statistics from extra-info descriptors? We could instead append them to a local file, so that they are still available if someone wants them.
I'd like to hear Steven's opinion here, because he was the reason why we implemented these stats a year ago.
Activity
changed milestone to %Tor: 0.2.2.x-final
Replying to Sebastian:
Is there some way to make the stats smaller yet still useful?
There is a way to make the stats smaller by raising the threshold for ports to be included in the stats. I can't say if that makes the stats still useful, because I'm not using these stats right now. I think leaving them unchanged, but excluding them from extra-info descriptors is the better approach here.
-
Replying to Sebastian:
Hrm. If the stats are still useful, we should find a way to report them. If not, we should consider removing them?
The stats can still be useful when they are stored to disk only. We can ask relay operators to turn stats on for a while and send us the file.
-
Replying to Sebastian:
We could do that with all other stats too.
Not really. We rely on some of the other stats to be reported by a lot of relays, so that we can combine them.
Let's wait for Steven to make a statement whether we still need exit-port statistics.
For the purposes I needed this feature, we already have more than enough data, so I think we can safely not include exit-port statistics in the descriptor and just store them locally. We can then ask for selected Tor nodes to give us the file if we need more data in the future.
It might still be desirable to include a compressed version in the descriptor so we could spot any significant changes in network usage. We only need very coarse statistics of the ports which have the highest usage, as all we need to do is spot large changes which merit further investigation.
Steven - what data do you have for these exit ports? I would like to have data for a high speed default exit policy relay, as well as one with the one we recommend on the blog. It would be great to have this over a long period of time. I've been meaning to update the consensus weights based on this data for some time..
-
One possible patch is to raise the threshold of total bytes that a port needs to see in order to be included in exit stats from 0.01% to, say, 0.1% or 1%. Someone should do the math to see what maximum size we could end up with this approach. This patch would affect a single line only, which may be good this close to the -rc.
A different approach is to show the top 10 ports only. I'll think about a possible code change tomorrow. If someone has an idea, please let me know.
-
Replying to karsten:
A different approach is to show the top 10 ports only. I'll think about a possible code change tomorrow. If someone has an idea, please let me know.
There's a possible patch in branch exitstats in my public repository. I'd like to review it myself tomorrow once again before having other people look. Just noticing it here to prevent people from writing a similar patch between now and tomorrow.
moved to tpo/core/tor#2196 (closed)
