Opened 8 years ago

Closed 7 months ago

#2878 closed enhancement (duplicate)

Don't bootstrap from an old consensus if we're about to replace it

Reported by: Sebastian Owned by:
Priority: High Milestone: Tor: unspecified
Component: Core Tor/Tor Version:
Severity: Major Keywords: performance, bootstrap, tor-client, s8-performance, s8-errors, 040-roadmap-proposed
Cc: catalyst, brade, mcs, intrigeri Actual Points:
Parent ID: #23605 Points:
Reviewer: Sponsor: Sponsor8-can

Description

Starting up maint-0.2.2 with an old data dir, I got this:

[notice] We now have enough directory information to build circuits.
[notice] Bootstrapped 80%: Connecting to the Tor network.
[notice] Bootstrapped 85%: Finishing handshake with first hop.
[notice] Bootstrapped 90%: Establishing a Tor circuit.
[notice] Tor has successfully opened a circuit. Looks like client functionality is working.
[notice] Bootstrapped 100%: Done.
[warn] Please upgrade! This version of Tor (0.2.2.19-alpha) is not recommended, according to the directory authorities. Recommended versions are: 0.2.1.29,0.2.1.30,0.2.2.21-alpha,0.2.2.22-alpha,0.2.2.23-alpha
[notice] Our directory information is no longer up-to-date enough to build circuits: We have only 0/2268 usable descriptors.
[notice] I learned some more directory information, but not enough to build a circuit: We have only 0/2268 usable descriptors.
[notice] Tor has successfully opened a circuit. Looks like client functionality is working.
[notice] I learned some more directory information, but not enough to build a circuit: We have only 96/2268 usable descriptors.
[notice] I learned some more directory information, but not enough to build a circuit: We have only 192/2268 usable descriptors.
[notice] I learned some more directory information, but not enough to build a circuit: We have only 288/2268 usable descriptors.
[notice] I learned some more directory information, but not enough to build a circuit: We have only 384/2268 usable descriptors.
[notice] I learned some more directory information, but not enough to build a circuit: We have only 480/2268 usable descriptors.
[notice] We now have enough directory information to build circuits.

First we make a circuit, then immediately we are say we don't have enough data to make a circuit and get a little spammy about that, too. Maybe there's an easy fix, if not we should postpone this for 0.2.3.x

Child Tickets

Change History (25)

comment:1 Changed 8 years ago by nickm

How old was the datadir?

I think the timeline here is something like:

  • We start with an older consensus and some matching descriptors. The consensus is old enough that we want to replace it, but not so old that we refuse to use it at all.
  • We start fetching a new consensus
  • We build some circuits and announce that we're bootstrapped.
  • The new consensus arrives, and we realize all our old server descriptors are unlisted. Now we know no good servers, so we declare that we're unbootstrapped, and start fetching server descriptors
  • The new server descriptors arrive; we decide that we can build circuits again
  • We build circuits, and declare that we're bootstrapped.

Can we think of a better behavior here? Most improvements that I can think of right now are subtle enough that I'd prefer that they wait for 0.2.3, but if we can come up with something easy and safe, it could go into 0.2.2 IMO.

comment:2 Changed 8 years ago by nickm

Milestone: Tor: 0.2.2.x-finalTor: 0.2.3.x-final
Priority: minormajor

One thing that might work here is to declare that if we start up with a consensus that we want to replace, don't declare ourselves bootstrapped until the download has succeeded or failed.

We could also refrain from building user circuits until we have a recent consensus or have become persuaded that we won't get one.

comment:3 Changed 8 years ago by nickm

We could also refrain from fetching descriptors until we have a recent consensus or have become persuaded that we won't get one.

comment:4 Changed 8 years ago by arma

Summary: Confusing order of log messages if Tor is restarted after a some downtimeDon't bootstrap from an old consensus if we're about to replace it

comment:5 Changed 8 years ago by nickm

So the only hard part of my "hold off on descriptor fetches and circuit building until we have a fresh consensus or are persuaded we won't get one" is the notion of "persuaded that we won't get one." What's the right rule there? We have tried to fetch a new consensus N times and failed?

comment:6 Changed 8 years ago by arma

Keywords: performance bootstrap added

comment:7 Changed 8 years ago by nickm

Milestone: Tor: 0.2.3.x-finalTor: unspecified
Type: defectenhancement

Seems like this doesn't need to go in 0.2.3.x

comment:8 Changed 7 years ago by nickm

Keywords: tor-client added

comment:9 Changed 7 years ago by nickm

Component: Tor ClientTor

comment:10 Changed 2 years ago by nickm

Keywords: sponsor8-maybe added

comment:11 Changed 22 months ago by catalyst

Cc: catalyst added
Severity: Blocker

What's more, the 85% (BOOTSTRAP_STATUS_HANDSHAKE_OR, "Finishing handshake with first hop") is very likely caused by the handshake with a directory mirror to attempt downloading a more current consensus. Look at how control_event_bootstrap() decodes the special status BOOTSTRAP_STATUS_HANDSHAKE. On the other hand, I think it takes a three-hop circuit to reach BOOTSTRAP_STATUS_DONE (100%).

comment:12 Changed 22 months ago by catalyst

Severity: BlockerMajor

comment:13 Changed 22 months ago by mcs

Cc: brade mcs added

comment:14 Changed 20 months ago by catalyst

Keywords: s8-performance s8-errors added; sponsor8-maybe removed
Sponsor: Sponsor8-can

comment:15 Changed 14 months ago by arma

Keywords: 036-proposed added

Adding to 036-proposed on the theory that once we do the "fix 80% bootstrap lie" stuff, it will be time to do this one.

comment:16 Changed 11 months ago by intrigeri

Cc: intrigeri added

comment:17 Changed 9 months ago by teor

Parent ID: #23605

comment:18 Changed 8 months ago by teor

Keywords: 040--roadmap-proposed added

0.3.6 is now 0.4.0: also changing -roadmap to -roadmap-proposed

comment:19 Changed 8 months ago by teor

Keywords: 040-roadmap-proposed added; 036-proposed removed

Oops

comment:20 Changed 8 months ago by teor

Keywords: 040--roadmap-proposed removed

Oops, second stage

comment:21 Changed 7 months ago by nickm

It's possible we don't actually want to do this. This ticket is older than microdescriptors, older than the onion key rotation changes of prop#274 (see #21641) and (I think) older than the live/reasonably live distinction. Most of the microdescriptors from any reasonably live consensus should still be usable these days, especially

The worst case here is when we have a consensus but we shut down before we can fetch any microdescriptors for it, and we start up again in the last minutes when the consensus is still reasonably live. I think even then the damage should be limited.

Taylor nodes that some other issues discussed above are variants on the jump-to-80% problem (see #22266).

I say that we should either wontfix, or mark this as needs_information, said information being "what is the measurable effect of this issue in practice?"

comment:22 Changed 7 months ago by arma

I think you're right that the microdesc part of this ticket is less relevant, in that we won't waste as much bandwidth now as we did in earlier designs.

But here is one concrete issue that I expect still happens (and that is the issue in this ticket as originally described): you start your Tor, it makes circuits, it declares 100% success, and then it declares un-success (because it got the new consensus and that new consensus made us realize we don't have a threshold of the right microdescs yet), and it starts bootstrapping again. If it finishes bootstrapping right after that, all is fine, but what if it fails bootstrapping the second time, like because your directory guards don't have the new microdescs you just learned you want, or something else has gone wrong? Tor Browser will be happily showing you a browser window as if everything is working, when it isn't.

comment:23 in reply to:  22 Changed 7 months ago by catalyst

Status: newneeds_information

Replying to arma:

But here is one concrete issue that I expect still happens (and that is the issue in this ticket as originally described): you start your Tor, it makes circuits, it declares 100% success, and then it declares un-success (because it got the new consensus and that new consensus made us realize we don't have a threshold of the right microdescs yet), and it starts bootstrapping again. If it finishes bootstrapping right after that, all is fine, but what if it fails bootstrapping the second time, like because your directory guards don't have the new microdescs you just learned you want, or something else has gone wrong? Tor Browser will be happily showing you a browser window as if everything is working, when it isn't.

I think the right thing to do in that case is basically #27691. Maybe we should close this as a duplicate, then?

comment:24 Changed 7 months ago by gaba

Yes. Feel free to close it if this is a duplicate.

comment:25 Changed 7 months ago by catalyst

Resolution: duplicate
Status: needs_informationclosed

At this point this mostly seems like a duplicate of #27691.

Note: See TracTickets for help on using tickets.