Opened 2 months ago

Last modified 5 weeks ago

#30440 assigned defect

sendme: Service introduction circuit ignore flow control

Reported by: dgoulet Owned by:
Priority: Medium Milestone: Tor: unspecified
Component: Core Tor/Tor Version:
Severity: Normal Keywords: tor-hs, sendme
Cc: Actual Points:
Parent ID: Points: 10
Reviewer: Sponsor:

Description

As preamble, circuit level SENDMEs are end-to-end which means they go from client to exit and vice versa. In other words, they can be described to be from edge connection to edge connection.

Which is exactly where it goes wrong for hidden service cells. First of all, they are not DATA cells so the SENDME logic is entirely ignored for all of them. They are all "circuit establishment" cells (see the list below)...

    case RELAY_COMMAND_ESTABLISH_INTRO:
    case RELAY_COMMAND_ESTABLISH_RENDEZVOUS:
    case RELAY_COMMAND_INTRODUCE1:
    case RELAY_COMMAND_INTRODUCE2:
    case RELAY_COMMAND_INTRODUCE_ACK:
    case RELAY_COMMAND_RENDEZVOUS1:
    case RELAY_COMMAND_RENDEZVOUS2:
    case RELAY_COMMAND_INTRO_ESTABLISHED:
    case RELAY_COMMAND_RENDEZVOUS_ESTABLISHED:

... *except* one single cell which is the INTRODUCE2 cell. A large number of these cells can be put on the same introduction service circuit, basically for each client introducing.

Which means that the intropoint can send as much cell as it wants on the service circuit without being bound by the SENDME flow control logic. Plainly speaking, intro points do not wait for an acknowledgement of the service to send more data, they just shove it all on the circuit.

This most likely render the introduction DoS (#29607) worst because the service actually constantly receives introduction requests as they queue up massively due to the intro point sending them non stop.

If there would be flow control on that circuit, a heavily loaded service (in CPU) would take a while to handle all introduction requests and then the SENDME cell towards the intro point would be only sent when the last request is actually handled and likely have CPU for new ones.

This also prevents us basically from implementing armadev's proposal in #15516 (https://trac.torproject.org/projects/tor/ticket/15516#comment:28).

Child Tickets

Change History (6)

comment:1 Changed 2 months ago by dgoulet

There is a world where we might want to advocate for backport on this. Else, any HS defenses based on flow control (like package window) will take a long time to be usable in the network...

Last edited 2 months ago by dgoulet (previous) (diff)

comment:2 Changed 2 months ago by dgoulet

After much discussions with armadev and asn, the ideal path forward is to use prop289 here.

The service can know (with protover FlowCtrl=1) if the introduction point supports SENDME v1 and if it does, the service should then start emitting them on the intro circuit.

For this to work, we need also the intro point to handle SENDME v1 which means decrease its package_window as INTRO2 cell comes in (not the current behavior). However, because the IP doesn't know if the service uses SENDME, it would only start accepting them once we switch on SENDME v1 in the network (sendme_accept_min_version) which could be some years down the road... But at least, in the meantime, service can start right now emitting SENDMEs v1 for any relay advertising FlowCtrl=1.

  1. There could be an argument for a new consensus param that control the usage of SENDMEs on the intro circuit which would be turned on much later since once we force that on the intro circuit, every service *NOT* sending SENDMEs on the circuit will stop working after 1000 cells because the intro point will reject anything due to flow control.
  1. Another approach is to make intro points right now start decreasing the package window and if it ends up to 0 and it doesn't get the SENDME at that point, the IP can deduce the service is not supporting it and thus we should continue operating like before. And this behavior will at some point fade out itself once every HS out there start using SENDME v1. I currently don't see much of a gain for a service to game this in the future, except maybe voluntarily allow the intro point to route a LOT of INTRO2 cells without flow control...? Again, we could switch off that feature with the consensus or simply by starting releasing a new stable not supporting this.

Overall, this will introduce an information leak: the intro point will know that the service behind an intro circuit is >= "tor_version" ... asn and I aren't too worried about that since any new HS features has that "leak" once deployed on the network (ex: client auth).

comment:3 Changed 2 months ago by dgoulet

Keywords: prop289 added
Milestone: Tor: 0.4.2.x-finalTor: 0.4.1.x-final
Owner: set to dgoulet
Priority: HighVery High
Status: newaccepted

comment:4 Changed 2 months ago by dgoulet

Milestone: Tor: 0.4.1.x-finalTor: unspecified

comment:5 Changed 6 weeks ago by dgoulet

Keywords: prop289 removed
Parent ID: #15516
Points: 210
Priority: Very HighMedium
Sponsor: Sponsor27-can

We aren't going to get there any time soon. Unparenting and no more sponsor.

comment:6 Changed 5 weeks ago by gaba

Owner: dgoulet deleted
Status: acceptedassigned

dgoulet will assign himself to the ones he is working on right now.

Note: See TracTickets for help on using tickets.