Opened 10 years ago

Closed 10 years ago

Last modified 8 years ago

#2196 closed defect (fixed)

Exclude exit-port statistics from extra-info descriptors

Reported by: karsten Owned by:
Priority: Medium Milestone: Tor: 0.2.2.x-final
Component: Core Tor/Tor Version: Tor: 0.2.2.17-alpha
Severity: Keywords: tor-relay
Cc: Actual Points:
Parent ID: Points:
Reviewer: Sponsor:

Description

Very fast relays generate exit-port statistics that require 40-45 KB. These statistics can easily make the relay generate an extra-info descriptor that exceeds the 50 KB upload limit. See #2183.

Should we exclude exit-port statistics from extra-info descriptors? We could instead append them to a local file, so that they are still available if someone wants them.

I'd like to hear Steven's opinion here, because he was the reason why we implemented these stats a year ago.

Child Tickets

Change History (17)

comment:1 Changed 10 years ago by Sebastian

Is there some way to make the stats smaller yet still useful?

comment:2 in reply to:  1 Changed 10 years ago by karsten

Replying to Sebastian:

Is there some way to make the stats smaller yet still useful?

There is a way to make the stats smaller by raising the threshold for ports to be included in the stats. I can't say if that makes the stats still useful, because I'm not using these stats right now. I think leaving them unchanged, but excluding them from extra-info descriptors is the better approach here.

comment:3 Changed 10 years ago by Sebastian

Hrm. If the stats are still useful, we should find a way to report them. If not, we should consider removing them?

comment:4 in reply to:  3 Changed 10 years ago by karsten

Replying to Sebastian:

Hrm. If the stats are still useful, we should find a way to report them. If not, we should consider removing them?

The stats can still be useful when they are stored to disk only. We can ask relay operators to turn stats on for a while and send us the file.

comment:5 Changed 10 years ago by Sebastian

We could do that with all other stats too.

comment:6 in reply to:  5 Changed 10 years ago by karsten

Replying to Sebastian:

We could do that with all other stats too.

Not really. We rely on some of the other stats to be reported by a lot of relays, so that we can combine them.

Let's wait for Steven to make a statement whether we still need exit-port statistics.

comment:7 Changed 10 years ago by sjmurdoch

For the purposes I needed this feature, we already have more than enough data, so I think we can safely not include exit-port statistics in the descriptor and just store them locally. We can then ask for selected Tor nodes to give us the file if we need more data in the future.

It might still be desirable to include a compressed version in the descriptor so we could spot any significant changes in network usage. We only need very coarse statistics of the ports which have the highest usage, as all we need to do is spot large changes which merit further investigation.

comment:8 Changed 10 years ago by nickm

So we could have it give, for example, stats for the top 10 ports?

comment:9 Changed 10 years ago by mikeperry

Steven - what data do you have for these exit ports? I would like to have data for a high speed default exit policy relay, as well as one with the one we recommend on the blog. It would be great to have this over a long period of time. I've been meaning to update the consensus weights based on this data for some time..

comment:10 Changed 10 years ago by karsten

One possible patch is to raise the threshold of total bytes that a port needs to see in order to be included in exit stats from 0.01% to, say, 0.1% or 1%. Someone should do the math to see what maximum size we could end up with this approach. This patch would affect a single line only, which may be good this close to the -rc.

A different approach is to show the top 10 ports only. I'll think about a possible code change tomorrow. If someone has an idea, please let me know.

comment:11 in reply to:  10 Changed 10 years ago by karsten

Replying to karsten:

A different approach is to show the top 10 ports only. I'll think about a possible code change tomorrow. If someone has an idea, please let me know.

There's a possible patch in branch exitstats in my public repository. I'd like to review it myself tomorrow once again before having other people look. Just noticing it here to prevent people from writing a similar patch between now and tomorrow.

comment:12 Changed 10 years ago by karsten

Status: newneeds_review

Please review branch exitstats in my public repository.

comment:13 Changed 10 years ago by nickm

Resolution: fixed
Status: needs_reviewclosed

Looks okay to me. Commenting with some loop invariants, then merging.

comment:14 Changed 10 years ago by mikeperry

Just to verify, all of the bytes for ports beyond the top 10 are properly added to other= still, right?

comment:15 in reply to:  14 Changed 10 years ago by karsten

Replying to nickm:

Looks okay to me. Commenting with some loop invariants, then merging.

Thanks!

Replying to mikeperry:

Just to verify, all of the bytes for ports beyond the top 10 are properly added to other= still, right?

Yes.

comment:16 Changed 8 years ago by nickm

Keywords: tor-relay added

comment:17 Changed 8 years ago by nickm

Component: Tor RelayTor
Note: See TracTickets for help on using tickets.