Opened 7 years ago

Closed 6 years ago

#9658 closed defect (fixed)

Track how many ntor/tap cells we received/processed

Reported by: arma Owned by:
Priority: Medium Milestone: Tor: 0.2.5.x-final
Component: Core Tor/Tor Version:
Severity: Keywords: tor-relay 025-triaged
Cc: Actual Points:
Parent ID: #9657 Points:
Reviewer: Sponsor:

Description (last modified by arma)

As part of #9574 Mike suggested that we make relays report how many TAP and NTor circuit-level handshakes they see.

This data would be most useful if we report "number received" and "number processed" for each one, so we can track both sides of the trends.

I'm not sure if we'll want this in 0.2.4 or 0.2.5 -- it would sure be useful to have it in 0.2.4 along with #9574, but it will depend how klunky the patch is.

Child Tickets

Change History (15)

comment:1 Changed 7 years ago by mikeperry

This has some downsides. It may allow inferences about the guard nodes of popular hidden services to be made using extra-info descriptors. However, the extra-info read+write history lines probably already do this?

What I really want is a way to obtain statistics that can tell us how we might rate-limit heavy clients on a per-orconn basis. But this information is probably just as sensitive to publish without aggregation or alteration first...

comment:2 Changed 7 years ago by arma

Yeah. You're right that this info might be sensitive.

You're also right that maybe we already publish dangerous stuff. But that doesn't necessarily make it ok to publish more such dangerous stuff.

Maybe we should collect these stats and log_notice them for the operator once a day? (or heck, once an hour)

comment:3 Changed 7 years ago by arma

Or we could skip reporting the create_fast cells, which would avoid saying stuff about client connections, right?

comment:4 Changed 7 years ago by arma

Description: modified (diff)
Summary: Add extrainfo lines for how many create cells of each handshake type we received/processedAdd extrainfo lines for how many ntor/tap cells we received/processed

Modified ticket to say we only want ntor and tap cells, since then we won't be counting create_fast's.

That said, I guess if we implement proposal stop-using-create-fast, then relays would start counting these clients.

Maybe we should actively discard (not count) the ones that come from non-relays?

comment:5 Changed 7 years ago by arma

See my ticket9658_local branch for the stats collection. I opted for only collecting stats about tap and ntor onionskins that come from known relays.

ticket9658_local_loud is the same thing but it logs once a minute rather than once an hour, so you can see it working.

comment:6 Changed 7 years ago by arma

For posterity, here's moria5 today:

Sep 04 17:51:34.667 [notice] Circuit handshake stats since last time: 12047/12051 TAP, 20/20 NTor.
Sep 04 17:52:35.705 [notice] Circuit handshake stats since last time: 12052/12053 TAP, 25/25 NTor.
Sep 04 17:53:36.735 [notice] Circuit handshake stats since last time: 11941/11945 TAP, 25/25 NTor.
Sep 04 17:54:37.768 [notice] Circuit handshake stats since last time: 12338/12340 TAP, 26/26 NTor.
Sep 04 17:55:38.803 [notice] Circuit handshake stats since last time: 11979/11981 TAP, 24/24 NTor.
Sep 04 17:56:39.843 [notice] Circuit handshake stats since last time: 12123/12124 TAP, 25/25 NTor.
Sep 04 17:57:40.874 [notice] Circuit handshake stats since last time: 12155/12156 TAP, 26/26 NTor.
Sep 04 17:58:41.914 [notice] Circuit handshake stats since last time: 12369/12372 TAP, 20/20 NTor.
Sep 04 17:59:42.945 [notice] Circuit handshake stats since last time: 11946/11949 TAP, 20/20 NTor.
Sep 04 18:00:43.960 [notice] Circuit handshake stats since last time: 12289/12290 TAP, 24/24 NTor.
Sep 04 18:01:44.983 [notice] Circuit handshake stats since last time: 12250/12250 TAP, 35/35 NTor.
Sep 04 18:02:45.029 [notice] Circuit handshake stats since last time: 11919/11919 TAP, 22/22 NTor.
Sep 04 18:03:46.064 [notice] Circuit handshake stats since last time: 12290/12294 TAP, 22/22 NTor.
Sep 04 18:04:47.111 [notice] Circuit handshake stats since last time: 11999/12005 TAP, 17/17 NTor.

(I believe I'm not dropping any due to overload, so the difference is probably from clients that preemptively gave up, disappeared, etc. before I got around to processing their onionskin.)

comment:7 Changed 7 years ago by arma

Status: newneeds_review

Actually, I've decided I want to merge ticket9658_local to maint-0.2.4 as-is. I added a changes file. It logs every hour, but at this point some more info for the operators is a good thing.

Nick, please review and merge if you like it.

comment:8 Changed 7 years ago by arma

Summary: Add extrainfo lines for how many ntor/tap cells we received/processedTrack how many ntor/tap cells we received/processed

comment:9 Changed 7 years ago by arma

Ok, I looked at it again. I'm going to merge it, to get this release out the door. If there's something wrong with it, we can fix it in the next rc.

(I think reporting stuff in extra info descs could be really dangerous in terms of privacy, but I think reporting this level of detail in local logs should be fine.)

comment:10 Changed 7 years ago by arma

Merged. I'm going to leave the ticket open for a bit in case somebody still wants to take a look.

comment:11 Changed 7 years ago by nickm

rep_hist_note_circuit_handshake_completed should probably be renamed; these handshakes are being assigned to cpuworkers, not "completed."

This should probably be part of the heartbeat code.

comment:12 Changed 7 years ago by arma

What needs review here?

(Nick's suggested refactorings sound fine to me, and maybe even can be completed as 0.2.5 things?)

comment:13 Changed 7 years ago by nickm

Milestone: Tor: 0.2.4.x-finalTor: 0.2.5.x-final
Status: needs_reviewnew

comment:14 Changed 6 years ago by nickm

Keywords: 025-triaged added

I'll do the refactorings nice and quick, hang on.

comment:15 Changed 6 years ago by nickm

Resolution: fixed
Status: newclosed

Looks like this *was* made part of the heartbeat as part of our fix for #10485 . Only remaining thing to do was variable rename, so I just went and did it.

Note: See TracTickets for help on using tickets.