Opened 9 months ago

Closed 6 months ago

Last modified 5 months ago

#24740 closed defect (fixed)

Tor launches two requests for authority certificates on first bootstrap

Reported by: teor Owned by:
Priority: High Milestone: Tor: 0.3.4.x-final
Component: Core Tor/Tor Version: Tor: 0.2.9.1-alpha
Severity: Normal Keywords: tor-bootstrap, tor-bandwidth, easy, intro, review-group-34, s8-bootstrap
Cc: Actual Points:
Parent ID: Points: 0.5
Reviewer: catalyst Sponsor: Sponsor8-can

Description

When tor first bootstraps, it launches two requests for authority certificates:

  • one is the hinted request that goes to the directory it downloaded the consensus from (ticket?)
  • one is a scheduled request that goes to a random directory

We should delay the first scheduled request slightly (5s?) to allow the first request to complete.
This might be as easy as changing the first number in the authority certificate schedule from 0 to 5.

I thought we avoided fetching certificates if another fetch was in progress: clearly that doesn't happen, and maybe we don't want it to, because we don't want to wait a whole 10 seconds for it to timeout.

Child Tickets

Change History (14)

comment:1 Changed 8 months ago by nickm

Milestone: Tor: 0.3.3.x-finalTor: 0.3.4.x-final
Priority: MediumHigh

comment:2 Changed 7 months ago by fristonio

Which part of the codebase is this hinted request made in, what I got from the code is that when the bootstrapping process starts Tor parses the certificates first from the cached-certs file. Then Tor parses microdesc-consensus from the disk and sets the current consensus based on them during this process it launches certificate fetch if not enough certificate are available to validate the consensus. Then it reloads the router list after this we set up periodic callbacks. I couldn't locate where the hinted request is made. Any help :)

comment:3 in reply to:  2 Changed 7 months ago by teor

Version: Tor: 0.2.9.1-alpha

Replying to fristonio:

Which part of the codebase is this hinted request made in, what I got from the code is that when the bootstrapping process starts Tor parses the certificates first from the cached-certs file.

When Tor first bootstraps, this file does not exist, so there are no cached certificates.
So at this step, Tor does nothing.

Then Tor parses microdesc-consensus from the disk and sets the current consensus based on them

When Tor first bootstraps, this file does not exist, so there is no cached consensus.
So at this step, Tor downloads the consensus:
https://gitweb.torproject.org/tor.git/tree/src/or/networkstatus.c#n961

Then, once it receives the consensus, it sends a request for the certs to the same directory mirror that it just got the consensus from, using the fingerprint of that directory mirror as a hint:
https://gitweb.torproject.org/tor.git/tree/src/or/directory.c#n2614
https://gitweb.torproject.org/tor.git/tree/src/or/networkstatus.c#n1887

during this process it launches certificate fetch if not enough certificate are available to validate the consensus.

This step always happens on first bootstrap, because there are no cached certificates.

Then it reloads the router list after this we set up periodic callbacks. I couldn't locate where the hinted request is made. Any help :)

The periodic callbacks also download certificates:
https://gitweb.torproject.org/tor.git/tree/src/or/networkstatus.c#n1242
This is the source of the duplicate certificate fetch.

We need to move this line:
https://gitweb.torproject.org/tor.git/tree/src/or/networkstatus.c#n1242

To here:
https://gitweb.torproject.org/tor.git/tree/src/or/networkstatus.c#n956

With a comment that says that either:

  • we are waiting for certificates, and we've launched a certificate download
  • we are waiting for a consensus, and we've launched a consensus download, which will launch a certificate download when it completes

This is a bugfix on #18963 in 0.2.9.1-alpha.
But it's only a load issue, so it's not a backport candidate.

comment:4 Changed 7 months ago by fristonio

Ok got that, so the first request that the client make to the directory server is hinted with the digest of the directory server it downloaded the consensus from. During startup scheduled requests are also set up which are made periodically to fetch the consensus from a random directory authority. So essentially in this issue we don't want to launch a certificate fetch always during the scheduled periodic consensus fetch but only in those cases when consensus are waiting for certs?

Thanks teor for your help. :)
Here is the patch https://github.com/fristonio/tor/tree/ticket-24740

Last edited 7 months ago by fristonio (previous) (diff)

comment:5 Changed 7 months ago by teor

Status: newneeds_review

Thanks! Looks good to me.

Does it pass "make check" and "make test-network-all"?
(You'll need chutney installed to run "make test-network-all".)

It's ok if you don't want to do the tests, someone will come and do them eventually.

comment:6 Changed 7 months ago by fristonio

Thanks teor, It passes make check with some tests skipped. I haven't tried make test-network-all yet, I will surely try and set up chutney for this as I think, I should be familiar with chutney for testing.

comment:7 Changed 7 months ago by fristonio

I tried running make test-network-all. It skipped mixed flavors: mixed+hs-v23 rest all tests passed. There was just a warning at the end I could not figure out

Warning: Unable to add signatures to consensus:
Valid-After times do not match when adding detached signatures to consensus Number: 4

`

comment:8 Changed 7 months ago by teor

Yeah, that's a timing bug in chutney, because we run Tor networks really fast. At some point, I'll work out why it's happening, and try to fix it.

If you want the mixed networks to work, you need tor-stable in your path.
Try something like:

ln -s /usr/bin/tor /usr/local/bin/tor-stable

or

alias tor-stable=/usr/bin/tor

But the mixed networks are mainly useful for protocol and consensus changes.

Edit: explain mixed better

Last edited 7 months ago by teor (previous) (diff)

comment:9 Changed 7 months ago by fristonio

Ah Thanks, I created the symlink for tor-stable and all the mixed network tests passed :)

comment:10 Changed 7 months ago by nickm

Keywords: review-group-34 added

comment:11 Changed 7 months ago by dgoulet

Reviewer: catalyst

Reviewer week 03/19th

comment:12 Changed 6 months ago by catalyst

Status: needs_reviewmerge_ready

Code changes look good to me. I haven't exhaustively analyzed it the way teor has, so I'll defer to their judgment about the strategy.

I wish we had a way to automatically test for the intended behavior, but it seems difficult to write such a test given what I see of the existing code structure.

comment:13 Changed 6 months ago by nickm

Resolution: fixed
Status: merge_readyclosed

okay; let's give it a try! Merging to master

comment:14 Changed 5 months ago by catalyst

Keywords: s8-bootstrap added
Note: See TracTickets for help on using tickets.