Opened 4 years ago

Last modified 3 months ago

#17359 new defect

__DisablePredictedCircuits causes bootstrap to hang at "Connecting to Tor Network"

Reported by: teor Owned by:
Priority: Low Milestone: Tor: unspecified
Component: Core Tor/Tor Version:
Severity: Minor Keywords: tor-hs bootstrap sponsor8-maybe
Cc: Actual Points:
Parent ID: Points: small
Reviewer: Sponsor:

Description (last modified by arma)

When __DisablePredictedCircuits is set in the torrc, bootstrap hangs at 80% - "Connecting to Tor Network".

This happens in hidden service configurations, it may happen in other client or server configurations as well.

I think this is because:

  • no predicted circuits are being built, and therefore
  • tor never completes an OR connection, and therefore
  • tor never thinks it has bootstrapped, and therefore
  • tor doesn't make any of the OR connections it would make as part of its configured function

To fix this, we need to either:

  • assume tor is connected to the network if it gets to "Connecting to Tor Network" and __DisablePredictedCircuits is set, or
  • make at least one connection at "Connecting to Tor Network" even if __DisablePredictedCircuits is set

The first risks repeatedly making connections if tor isn't connected to the network, the second risks making connections the user doesn't want.

Child Tickets

Change History (19)

comment:1 Changed 4 years ago by teor

A workaround is to set:

LongLivedPorts
PredictedPortsRelevanceTime 0 seconds
LearnCircuitBuildTimeout 0

But there's no option to reduce the number of hidden service server preemptive circuits (see #17360).

comment:2 Changed 4 years ago by arma

Hm! I think our definition of 100% bootstrapped is that we made a circuit. Is that so terrible a definition?

What situation is causing us to set DisablePredictedCircuits without a controller generating its own circuits? That is, is this just a pathological config, or is it actually a thing that somebody wants to do?

And lastly, be aware that "80% connecting to tor network" is the initial bootstrap message if you start Tor with enough cached dir info. That is, seeing 80% doesn't mean anything at all in this case about whether the network is plugged in.

comment:3 in reply to:  2 Changed 4 years ago by teor

Keywords: tor-hs added

Replying to arma:

Hm! I think our definition of 100% bootstrapped is that we made a circuit. Is that so terrible a definition?

It's an excellent definition. But we require 100% bootstrapped to make circuits in some configs, so it's a circular dependency (on hidden/onion services, and without predicted circuits). Perhaps a solution is to make introduction point connections dependent on 80%, not 100%? (This would also fix the scenario where each type of predicted circuit is disabled individually by a specific option.)

What situation is causing us to set DisablePredictedCircuits without a controller generating its own circuits? That is, is this just a pathological config, or is it actually a thing that somebody wants to do?

It's useful for load-balancing hidden/onion service configurations like (Rendezvous) Single Onion Services (RSOS - #17178, SOS - prop#252), where (almost) every connection is one-hop, and several hundred tor instances might be running. In this case, predicted circuits will never be used (except for descriptor uploads, and not every instance uploads descriptors due to OnionBalance). Predicted circuits represent unnecessary load on the server and network in this case.

And lastly, be aware that "80% connecting to tor network" is the initial bootstrap message if you start Tor with enough cached dir info. That is, seeing 80% doesn't mean anything at all in this case about whether the network is plugged in.

Indeed. I was using chutney on localhost, and it worked without __DisablePredictedCircuits set, so I know the network was plugged in and chutney was working.

comment:4 Changed 4 years ago by nickm

Milestone: Tor: 0.2.8.x-finalTor: 0.2.9.x-final

It is impossible that we will fix all 226 currently open 028 tickets before 028 releases. Time to move some out. This is my second pass through the "new" and tickets, looking for things to move to 0.2.9.

comment:5 Changed 4 years ago by dgoulet

Points: small
Priority: MediumLow

comment:6 Changed 4 years ago by isabela

Milestone: Tor: 0.2.9.x-finalTor: 0.2.???

tickets market to be removed from milestone 029

comment:7 Changed 3 years ago by teor

Milestone: Tor: 0.2.???Tor: 0.3.???

Milestone renamed

comment:8 Changed 3 years ago by nickm

Keywords: tor-03-unspecified-201612 added
Milestone: Tor: 0.3.???Tor: unspecified

Finally admitting that 0.3.??? was a euphemism for Tor: unspecified all along.

comment:9 Changed 3 years ago by teor

This is related to #21073, where PredictedPortsRelevanceTime 0 eventually causes a tor hidden service to stop updating its descriptor. The root cause is that tor relies on predicted circuits to generate network activity, and network activity causes tor to perform certain actions on a regular basis.

comment:10 Changed 3 years ago by nickm

Keywords: tor-03-unspecified-201612 removed

Remove an old triaging keyword.

comment:11 Changed 2 years ago by nickm

Keywords: bootstrap sponsor8-maybe added

comment:12 Changed 14 months ago by teor

#27849 fixes a similar issue with SOCKSPort 0.

comment:13 Changed 13 months ago by teor

This issue might have caused sbws #28639, see #28701.

comment:14 Changed 13 months ago by teor

This issue is related to #28714, but #28714 probably won't fix it.

comment:15 in reply to:  1 ; Changed 12 months ago by teor

Replying to teor:

A workaround is to set:

LongLivedPorts
PredictedPortsRelevanceTime 0 seconds
LearnCircuitBuildTimeout 0

But there's no option to reduce the number of hidden service server preemptive circuits (see #17360).

In #17592 in Tor 0.3.1.1-alpha, we replaced PredictedPortsRelevanceTime and CircuitIdleTimeout with CircuitsAvailableTimeout.

So the current workaround is:

LongLivedPorts
LearnCircuitBuildTimeout 0
# and
PredictedPortsRelevanceTime 0 seconds
# or
CircuitsAvailableTimeout (a small timeout? a large timeout?)

We can't disable predicted ports separately any more, which is annoying.

Last edited 12 months ago by teor (previous) (diff)

comment:16 in reply to:  15 ; Changed 3 months ago by dgoulet

Replying to teor:

In #17592 in Tor 0.3.1.1-alpha, we replaced PredictedPortsRelevanceTime and CircuitIdleTimeout with CircuitsAvailableTimeout.

So the current workaround is:

LongLivedPorts
LearnCircuitBuildTimeout 0
# and
PredictedPortsRelevanceTime 0 seconds
# or
CircuitsAvailableTimeout (a small timeout? a large timeout?)

We can't disable predicted ports separately any more, which is annoying.

I'm having this issue with Exitmap and the above workaround do not work :S ...

No circuit activity stalls bootstrap progression basically :(.

comment:17 Changed 3 months ago by arma

Description: modified (diff)

comment:18 Changed 3 months ago by arma

My first thought is that we should distinguish between "predicted" circuits, meaning circuits that are made for the purpose of having extra clean circuits sitting around if we need them, and other kinds of circuits, such as the ones we use to see if we can connect to the network.

And we should do all of the other things that we need circuits for, even if 'predicted' circuits are disabled.

comment:19 in reply to:  16 Changed 3 months ago by teor

Replying to dgoulet:

Replying to teor:

In #17592 in Tor 0.3.1.1-alpha, we replaced PredictedPortsRelevanceTime and CircuitIdleTimeout with CircuitsAvailableTimeout.

So the current workaround is:

LongLivedPorts
LearnCircuitBuildTimeout 0
# and
PredictedPortsRelevanceTime 0 seconds
# or
CircuitsAvailableTimeout (a small timeout? a large timeout?)

We can't disable predicted ports separately any more, which is annoying.

I'm having this issue with Exitmap and the above workaround do not work :S ...

No circuit activity stalls bootstrap progression basically :(.

Those workarounds make tor stop building as many circuits as possible. They are a workaround for this use case:

"I am running lots of tor instances, and I don't want them to build useless circuits."

I don't know what the workaround for your use case is. But I think your use case is:

"I am running a tor instance, and I don't want it to stall."

Maybe you should try:

LongLivedPorts 80,443
LearnCircuitBuildTimeout 1
PredictedPortsRelevanceTime (Maximum allowed time)
CircuitsAvailableTimeout (I'm not really sure, maybe use the default? Or try a large or small timeout, and see how you go.)

Note: See TracTickets for help on using tickets.