Opened 3 years ago

Closed 3 years ago

#11612 closed defect (fixed)

tbb bundle with meek takes (literally) hours to connect

Reported by: cypherpunks Owned by: dcf
Priority: Medium Milestone:
Component: Obfuscation/meek Version:
Severity: Keywords:
Cc: Actual Points:
Parent ID: Points:
Reviewer: Sponsor:

Description

I tried out the experimental Tor Browser bundle with meek (version 3.5.4-meek-1-Linux, 32 bit).
When first launching the bundle, it took literally hours to make the first connection to the tor network. The progress bar was hung at ~50%, the console showed error messages like:

[notice] I learned some more directory information, but not enough to build a circuit: We need more microdescriptors: we have 883/4458, and can only build 0% of likely paths. (We have 20% of guards bw, 21% of midpoint bw, and 21% of exit bw.)
[notice] Bootstrapped 72%: Loading relay descriptors.
[warn] Problem bootstrapping. Stuck at 72%: Loading relay descriptors. (DONE; DONE; count 1; recommendation warn)

I quit & restarted it several times, the bootstrapping progress seemed to restart from where it had left off, but it literally took hours before I had a tor browser window.

After that, TBB with meek worked reasonably well, albeit slower than normal Tor Browser

Child Tickets

Change History (7)

comment:1 Changed 3 years ago by dcf

Status: newneeds_information

This sounds reminiscent of a bug I thought was fixed in https://gitweb.torproject.org/pluggable-transports/meek.git/commitdiff/7c84d72e5b92e710ef560f493d836d722a65e4bb: receipt of the big data payloads that come during the downloading of descriptors was exceeding the HTTP helper's built-in timeout.

Could you try increasing the internal timeout, and see if it makes a change? First, extract the bundle fresh. Then:

cd Data/Browser/profile.meek-http-helper/extensions
mv meek-http-helper@bamsoftware.com.xpi meek-http-helper@bamsoftware.com.zip
unzip -d meek-http-helper@bamsoftware.com meek-http-helper@bamsoftware.com.zip

Edit the file meek-http-helper@…/components/main.js and increase the timeout on these two lines to 10.0:

MeekHTTPHelper.LOCAL_READ_TIMEOUT = 2.0;
MeekHTTPHelper.LOCAL_WRITE_TIMEOUT = 2.0;

Then restart the browser.

The only other thing I can think of is #9229, but that seems unlikely, as it only stalls bootstrapping for 60 seconds.

Last edited 3 years ago by dcf (previous) (diff)

comment:2 Changed 3 years ago by cypherpunks

(sorry for the delay in responding)

I re-extracted the tarball, increased the timeout to 10.0 as directed and retried, but it didn't seem to make any difference, the connection process still hangs for hours as before.

The meek-client.log file is full of error messages of the form "error in handling request: EOF"; they appear at roughly 10 second intervals.

comment:3 in reply to:  2 Changed 3 years ago by dcf

Replying to cypherpunks:

(sorry for the delay in responding)

I re-extracted the tarball, increased the timeout to 10.0 as directed and retried, but it didn't seem to make any difference, the connection process still hangs for hours as before.

The meek-client.log file is full of error messages of the form "error in handling request: EOF"; they appear at roughly 10 second intervals.

Thanks for doing the test. What if you increase the timeout to a really large value, like 1000.0?

comment:4 Changed 3 years ago by cypherpunks

I increased the timeout to 1000.0, and it still takes a very long time (several hours) to start.

For example, after re-extracting the tarball, increasing the timeout to 1000.0 as described above, and running start-tor-browser, the bootstrapping(?) process stalled for 1hr and 20mins, after which I got the following error message:

[warn] Problem bootstrapping. Stuck at 67%: Loading relay descriptors. (DONE; DONE; count 1; recommendation warn)
[warn] 1 connections have failed:
[warn]  1 connections died in state handshaking (TLS) with SSL state SSLv3 read finished A in HANDSHAKE

After that, I cancelled the connect to the Tor network dialog and ran start-tor-browser again, this time, after a further ~1hr, I finally got the message

[notice] We now have enough directory information to build circuits.
[notice] Bootstrapped 80%: Connecting to the Tor network.
[notice] Bootstrapped 90%: Establishing a Tor circuit.
[notice] Tor has successfully opened a circuit. Looks like client functionality is working.
[notice] Bootstrapped 100%: Done.

and a Tor Browser window. The meek-client.log is full of the same error messages as before ("error in handling request: EOF", roughly once every 10 seconds)

comment:5 Changed 3 years ago by dcf

I bumped up the timeouts from 10 s to 20 s in the appengine component and in meek-server. Please try it again and see if it makes a difference.

comment:6 Changed 3 years ago by dcf

I think I found the cause of the problem. The Firefox helper requires the request from meek-client to be received all at once in one read; i.e., it doesn't buffer and concatenate multiple reads:

During initial bootstrapping, there are several large requests (up to ~80000 bytes after base64) that the helper is required to read in one go. It's not surpising that the OS might split such a large payload across multiple packets, even on a localhost interface. If I modify meek-client to split its write into two writes, I can reproduce the bootstrapping failure.

comment:7 in reply to:  6 Changed 3 years ago by dcf

Resolution: fixed
Status: needs_informationclosed

Replying to dcf:

I think I found the cause of the problem. The Firefox helper requires the request from meek-client to be received all at once in one read; i.e., it doesn't buffer and concatenate multiple reads:

During initial bootstrapping, there are several large requests (up to ~80000 bytes after base64) that the helper is required to read in one go. It's not surpising that the OS might split such a large payload across multiple packets, even on a localhost interface. If I modify meek-client to split its write into two writes, I can reproduce the bootstrapping failure.

No, I was wrong—the code was already trying to buffer multiple read events. But there was a bug in that code that only arose when it took two or more reads to fill a buffer. I wrote a long explanation in the commit message:

I'm pretty sure this is the source of the problem. If you want to test it, you can patch your main.js file to remove the second argument to the this.buf.subarray call. I'll ask for this fix to go into the next Tor Browser alpha release.

Note: See TracTickets for help on using tickets.