Opened 4 months ago

Last modified 5 days ago

#23101 new enhancement

Predict and build specific HS purpose circuits (rather than GENERAL)

Reported by: mikeperry Owned by:
Priority: Medium Milestone: Tor: 0.3.3.x-final
Component: Core Tor/Tor Version:
Severity: Normal Keywords: tor-hs, tor-guard, guard-discovery-prop247-controller
Cc: asn Actual Points:
Parent ID: #13837 Points:
Reviewer: Sponsor:

Description

circuit_predict_and_launch_new() builds CIRCUIT_PURPOSE_C_GENERAL circuits, even for hidden service circuits. When we implement Proposa247/#9001, we will need to give all hidden service circuits their specific purposes right from build time, in order to ensure that they are build using Vanguards. We will also need to disable cannibalization for them, since cannibalized GENERAL circuits will not use our vanguards, either.

This means we need to change the prediction recording and circuit building to record, predict, and build each HS circuit type independently.

Child Tickets

Change History (7)

comment:1 Changed 2 months ago by asn

One approach here is to indeed define new circuit purposes for HS circuits. My main fear here is that these circuit purposes are used for tons of things in the codebase (from circuit timeouts, to cannibalization, to guard stuff, to stats stuff, to basic circuit functionality state machine), and hence adding new purposes means that we need to find all the right places to consider our new purposes which I imagine is gonna be a PITA and perhaps error-prone...

Another approach would be to use the hs_ident and rend_data elements of the circuit_t to distinguish HS circuits from normal circuits. That's what most of the code is doing to detect HS rend circuits.

comment:2 Changed 4 weeks ago by mikeperry

Keywords: guard-discovery-prop247-controller added; guard-discovery removed
Milestone: Tor: unspecifiedTor: 0.3.3.x-final

comment:3 in reply to:  1 Changed 3 weeks ago by teor

Replying to asn:

One approach here is to indeed define new circuit purposes for HS circuits. My main fear here is that these circuit purposes are used for tons of things in the codebase (from circuit timeouts, to cannibalization, to guard stuff, to stats stuff, to basic circuit functionality state machine), and hence adding new purposes means that we need to find all the right places to consider our new purposes which I imagine is gonna be a PITA and perhaps error-prone...

Yes, it will be error-prone. I wonder how we can make this state machine easier to modify in future?

Another approach would be to use the hs_ident and rend_data elements of the circuit_t to distinguish HS circuits from normal circuits. That's what most of the code is doing to detect HS rend circuits.

No, these fields are not populated until the circuit is used for rendezvous.
So they can't be used to identify preemptive circuits that are built with vanguards.

comment:4 Changed 12 days ago by mikeperry

Parent ID: #9001#13837

Re-parenting, because I think this prediction should be fixed for #13837 (and for our experiments using #13837 w/ https://github.com/mikeperry-tor/vanguards).

comment:5 Changed 11 days ago by mikeperry

Asn - you are right to worry about trying to alter how we deal with existing purposes, especially in a way that changes the HS state machine. However, adding a whole new circuit purpose is quite easy. I have done it several times before (including in my draft patch for parent ticket #13837). I know how to do it without issue. Aside from a couple asserts and checks here and there, new purposes get ignored by existing code. That's what circuit purposes are for.

How about this: instead of messing with the existing HS purpose state machine(s), we create a new purpose (say CIRCUIT_PURPOSE_HS_GENERAL), and then in circuit_predict_and_launch_new() we launch new circuits of that type when we predict we need any hidden service circuits. Then we should be able to make just a few changes to circuit_get_open_circ_or_launch(), circuit_launch_by_extend_info(), and circuit_find_to_cannibalize() to use this new circuit type for hidden service circuits (instead of using CIRCUIT_PURPOSE_C_GENERAL for them).

That way circuits with this new purpose will otherwise be ignored unless we are looking to build a new one. And that mechanism for reusing them will be very similar as it is today, just drawing from CIRCUIT_PURPOSE_HS_GENERAL instead of CIRCUIT_PURPOSE_C_GENERAL, and we also don't have to change our current rephist prediction code.

Sound like a plan?

Last edited 11 days ago by mikeperry (previous) (diff)

comment:6 in reply to:  5 ; Changed 9 days ago by asn

Replying to mikeperry:

Asn - you are right to worry about trying to alter how we deal with existing purposes, especially in a way that changes the HS state machine. However, adding a whole new circuit purpose is quite easy. I have done it several times before (including in my draft patch for parent ticket #13837). I know how to do it without issue. Aside from a couple asserts and checks here and there, new purposes get ignored by existing code. That's what circuit purposes are for.

How about this: instead of messing with the existing HS purpose state machine(s), we create a new purpose (say CIRCUIT_PURPOSE_HS_GENERAL), and then in circuit_predict_and_launch_new() we launch new circuits of that type when we predict we need any hidden service circuits. Then we should be able to make just a few changes to circuit_get_open_circ_or_launch(), circuit_launch_by_extend_info(), and circuit_find_to_cannibalize() to use this new circuit type for hidden service circuits (instead of using CIRCUIT_PURPOSE_C_GENERAL for them).

That way circuits with this new purpose will otherwise be ignored unless we are looking to build a new one. And that mechanism for reusing them will be very similar as it is today, just drawing from CIRCUIT_PURPOSE_HS_GENERAL instead of CIRCUIT_PURPOSE_C_GENERAL, and we also don't have to change our current rephist prediction code.

Sound like a plan?

Plausible plan. I'm still a bit concerned but I don't have a better plan tbh.

What would be the interaction between CIRCUIT_PURPOSE_C_GENERAL and CIRCUIT_PURPOSE_HS_GENERAL? e.g. would you be able to cannibalize a C_GENERAL circuit to an HS_GENERAL circuit if we find that we have idle general circuits and we need HS ones (or the opposite)?

comment:7 in reply to:  6 Changed 5 days ago by mikeperry

Replying to asn:

Replying to mikeperry:

Asn - you are right to worry about trying to alter how we deal with existing purposes, especially in a way that changes the HS state machine. However, adding a whole new circuit purpose is quite easy. I have done it several times before (including in my draft patch for parent ticket #13837). I know how to do it without issue. Aside from a couple asserts and checks here and there, new purposes get ignored by existing code. That's what circuit purposes are for.

How about this: instead of messing with the existing HS purpose state machine(s), we create a new purpose (say CIRCUIT_PURPOSE_HS_GENERAL), and then in circuit_predict_and_launch_new() we launch new circuits of that type when we predict we need any hidden service circuits. Then we should be able to make just a few changes to circuit_get_open_circ_or_launch(), circuit_launch_by_extend_info(), and circuit_find_to_cannibalize() to use this new circuit type for hidden service circuits (instead of using CIRCUIT_PURPOSE_C_GENERAL for them).

That way circuits with this new purpose will otherwise be ignored unless we are looking to build a new one. And that mechanism for reusing them will be very similar as it is today, just drawing from CIRCUIT_PURPOSE_HS_GENERAL instead of CIRCUIT_PURPOSE_C_GENERAL, and we also don't have to change our current rephist prediction code.

Sound like a plan?

Plausible plan. I'm still a bit concerned but I don't have a better plan tbh.

What would be the interaction between CIRCUIT_PURPOSE_C_GENERAL and CIRCUIT_PURPOSE_HS_GENERAL? e.g. would you be able to cannibalize a C_GENERAL circuit to an HS_GENERAL circuit if we find that we have idle general circuits and we need HS ones (or the opposite)?

The desired behavior is that this should not be allowed. The idea is that these two types of general circuits will have different path construction mechanisms for the first three hops (vanguards/pinned middles vs normal), and should not be mixed.

In terms of how this will be done - I am going to alter circuit_find_to_cannibalize() so that you must also specify a circuit type to cannibalize from in addition to the one you want to create, and you can't switch between these classes of circuits (in the case where there is nothing to cannibalize in this way, that function can find nothing and return NULL, and the calling code in circuit_launch_by_extend_info() will build a fresh circuit instead). Since circuit_find_to_cannibalize() already behaves as if you specified canibalize_from == CIRCUIT_PURPOSE_C_GENERAL, I don't anticipate that the change to draw from CIRCUIT_PURPOSE_HS_GENERAL will be very complicated.

Note: See TracTickets for help on using tickets.