Bundles were made. As discussed with dcf, a few things remain before we deploy meek to real users:
a) Get some people to review its code
b) Get someone to run a stable/fast meek bridge.
c) Figure out if Tor can pay the Appengine bills.
d) Make its TLS layer less easily fingerpintable.
This is my thinking on [[doc/meek#Distinguishability|HTTPS fingerprintability]] and deployment. We should first take care of the trivial fingerprinting issues, namely client ciphersuites and TLS extensions, and then make the first deployment. Those two are the things for which a firewall rule can be written in five minutes. The other issues are things like traffic statistics, which are important but not immediately important in that it takes time to make a classifier and even then it's not 100%.
I'm working on a TBB browser extension to make HTTPS requests on behalf of meek-client. That will get us Firefox's ciphersuites and TLS extensions, and I think is more realistically deployable than, say, packaging another browser in the TBB, and more future-proof than writing a custom OpenSSL or NSS program that closely tries to imitate browser TLS.
I'm working on a TBB browser extension to make HTTPS requests on behalf of meek-client. That will get us Firefox's ciphersuites and TLS extensions, and I think is more realistically deployable than, say, packaging another browser in the TBB, and more future-proof than writing a custom OpenSSL or NSS program that closely tries to imitate browser TLS.
A big consideration here is latency; i.e., proximity to App Engine servers. We're adding an extra roundtrip hop to every communication:
meek-client → App Engine → meek-server
We have no control over the meek-client→App Engine link, but we can try to make the App Engine→meek-server link fast.
Here are sample roundtrip times for a few servers. The test is running live at http://meek-latency-test.appspot.com/. Source code for the test is in the repo here. The horizontal axis is milliseconds.
The measured servers are in order:
my bridge in California (the same one used for flash proxy)
a Google server very close to App Engine (lower bound on roundtrip time)
an HTTPS Google server very close to App Engine (lower bound on HTTPS roundtrip time)
I can add other servers if someone wants to test their own.
It appears that the App Engine requests are coming from 8.35.201.0/24 in the U.S. This link seems to confirm that the servers are in the U.S. Ideally we minimize latency with that netblock.
All that said, I've been using the Greek bridge for a while, and it doesn't feel that slow, but maybe I'm just used to it.
Google is currently offering $500 in Cloud Platform credit. (Cloud Platform includes App Engine.) I applied, describing meek, and received a code for $500.
It says the credit expires three months after it is redeemed. My guess is that we'll use up the three months before we use up the $500. So I think to activate the credit just before public deployment. That should keep us running for three months anyway.
I tested the linux64 bundle yesterday night. It worked great. I wouldn't even notice the second browser, if I hadn't done a pstree, that looks like this:
I didn't test with wireshark to see if the SSL looks firefoxy. I need to do that next.
Also, with the meek bundles, on Tor bootstrap I always get the following error:
Apr 12 14:26:46.000 [warn] Received http status code 404 ("Not found") from server '0.0.2.0:1' while fetching "/tor/keys/fp-sk/27B6B5996C426270A5C95488AA5BCEB6BCC86956-510628E039BC57EB78FD5A324E995690C1BDF1CE".
}}}
If that is really only an amd64 issue could you then disable the hardening just on that architecture? GBUILD_BITS is your friend here.
I tried it again without disabling DEB_BUILD_HARDENING_PIE, and didn't get an error. I'm going to test some bundles that don't touch DEB_BUILD_HARDENING_PIE at all.
In any case, I don't think PIE is going to buy us anything with golang executables.
https://code.google.com/p/go/issues/detail?id=6940#c6: "The Go linker does not currently support building objects that may be linked into a PIE."
I also prepared a parallel rebased merge candidate branch. It has just 8 commits on top of tbb-3.6.1-build1. The second link shows all the changes made in the branch.
Clearly the two programs meek-client-torbrowser and meek-client are the only files that matter. On disk they are 11.2 MB together. When I compress them with xz, they take up 1.7 MB.
1.7M t.tar.xz
The large size of the binaries is caused by static linking by the golang compiler. I don't think we want to try to circumvent Go's preference for static linking (see e.g. http://harmful.cat-v.org/software/dynamic-linking/); compression seems to get rid of most of the redundancy anyway. The large executable sizes are a known problem; see issue 6853: binaries too big and growing and Go Binary Sizes Are Growing out of Control. It's supposed to be better in the upcoming Go 1.3 (we're using Go 1.2). I tried compiling with a 1.3 beta and it does save some bytes, though compression makes it not matter much:
With Go 1.2:
Here are the added files and their sizes for windows (missing doc files are #11834 (closed)). The extra terminateprocess-buffer.exe program is a special program required only on windows. I derived this output from diff -u <(cd 3.6.1/windows && find . -type f -print0 | xargs -0 ls -lh) <(cd 3.6.1-meek-1/windows && find . -type f -print0 | xargs -0 ls -lh).
I also prepared a parallel rebased merge candidate branch. It has just 8 commits on top of tbb-3.6.1-build1. The second link shows all the changes made in the branch.
This looks good to me. Bonus points if you could get rid of these sudo calls (might be a matter of setting $PATH properly). I am currently refactoring the descriptors and getting everything to work without the need for special privileges while building is one (albeit very minor) goal.
I also prepared a parallel rebased merge candidate branch. It has just 8 commits on top of tbb-3.6.1-build1. The second link shows all the changes made in the branch.
This looks good to me. Bonus points if you could get rid of these sudo calls (might be a matter of setting $PATH properly). I am currently refactoring the descriptors and getting everything to work without the need for special privileges while building is one (albeit very minor) goal.
They are built not exactly on top of the 3.6.2 bundles, because I merged with the tag tbb-3.6.2-build5 and the 3.6.2 bundles were apparently built with what is now the head of maint-3.6.
I also have a rebased branch, which I propose for merge:
I know that #11641 (closed) changes the directory structure, which will probably affect the Info.plist thing. Would you like me to rebase somewhere onto master, rather than the maint-3.6 branch?
Did you test with a different Gitian setup to check whether building meek is still reproducible?
I tried building it again, and ran into the trouble in #12382 (closed). However the packages are the same except for the changed langpack files. The en-US packages are identical.
I know that #11641 (closed) changes the directory structure, which will probably affect the Info.plist thing. Would you like me to rebase somewhere onto master, rather than the maint-3.6 branch?
Yes, please. We plan to release a 4.0 alpha which could contain meek to give it a wider testing (and test its reproducibility under release conditions).
I know that #11641 (closed) changes the directory structure, which will probably affect the Info.plist thing. Would you like me to rebase somewhere onto master, rather than the maint-3.6 branch?
Yes, please. We plan to release a 4.0 alpha which could contain meek to give it a wider testing (and test its reproducibility under release conditions).
I started the rebase, but the build hasn't finished, and now I won't be able to finish it until June 19. Please stand by.
Yes, please. We plan to release a 4.0 alpha which could contain meek to give it a wider testing (and test its reproducibility under release conditions).
I tested it and made sure it would bootstrap with meek on all platforms.
I put the second headless app (#11429 (moved)) under TorBrowser/Data/TorBrowser.app.meek-http-helper. It has symlinks to the top-level Contents files, and modified Info.plist.
Here's a summary of added files on every platform.
This looks good to me. The only thing I am not sure is wrt the location for the headless app and the placement of Docs/meek in the OS X case. Let's see what Kathy/Mark are saying. Kathy/Mark: Could you comment on those things? Does there something speak against the current idea? If so, what would be a better place?
This looks good to me. The only thing I am not sure is wrt the location for the headless app and the placement of Docs/meek in the OS X case. Let's see what Kathy/Mark are saying. Kathy/Mark: Could you comment on those things? Does there something speak against the current idea? If so, what would be a better place?
I didn't even notice the wrong placement of the meek docs. That's a bug. I'll put it under TorBrowser/Docs like in the other cases.
This looks good to me. The only thing I am not sure is wrt the location for the headless app and the placement of Docs/meek in the OS X case. Let's see what Kathy/Mark are saying. Kathy/Mark: Could you comment on those things? Does there something speak against the current idea? If so, what would be a better place?
I discussed this with Kathy. It seems a little odd to place an application bundle (even a symlink'd one) under TorBrowser/Data. Maybe move it under TorBrowser/Tor/PluggableTransports/ ?
Also, maybe it is intentional or maybe not that the bundle does not end in .app, but we would have expected to see a name like TorBrowser-meek-http-helper.app. Apparently things work either way though.
The other issue that we will need to research is whether the Firefox updater (#4234 (moved)) can handle symlinks. My guess is that it cannot but maybe we will be pleasantly surprised ;) I think we have to proceed with the symlinks for now in the interest of getting the alpha out.
This looks good to me. The only thing I am not sure is wrt the location for the headless app and the placement of Docs/meek in the OS X case. Let's see what Kathy/Mark are saying. Kathy/Mark: Could you comment on those things? Does there something speak against the current idea? If so, what would be a better place?
I discussed this with Kathy. It seems a little odd to place an application bundle (even a symlink'd one) under TorBrowser/Data. Maybe move it under TorBrowser/Tor/PluggableTransports/ ?
Also, maybe it is intentional or maybe not that the bundle does not end in .app, but we would have expected to see a name like TorBrowser-meek-http-helper.app. Apparently things work either way though.
I avoided ending the file name with ".app" to reduce the likelihood that a curious user would launch the internal bundle by double-clicking. The internal bundle is necessarily configured to bypass the Tor proxy, so it's dangerous if it's confused with the actual Tor Browser. The naming trick worked when it was under TorBrowser/Data (the app appeared as a normal folder), but for some reason it doesn't work under TorBrowser/Tor/PluggableTransports :/ When you double-click the icon, first you get a dialog offering to import your bookmarks, then you get a browser window with the normal Firefox branding. You can't type into it, nor does it appear in the dock, probably because of the LSBackgroundOnly setting.
I avoided ending the file name with ".app" to reduce the likelihood that a curious user would launch the internal bundle by double-clicking. The internal bundle is necessarily configured to bypass the Tor proxy, so it's dangerous if it's confused with the actual Tor Browser. The naming trick worked when it was under TorBrowser/Data (the app appeared as a normal folder), but for some reason it doesn't work under TorBrowser/Tor/PluggableTransports :/ When you double-click the icon, first you get a dialog offering to import your bookmarks, then you get a browser window with the normal Firefox branding. You can't type into it, nor does it appear in the dock, probably because of the LSBackgroundOnly setting.
Almost certainly the app copy does not know where to find its browser profile. I think you have a reasonable point about not wanting users to be tempted to open the app, so not ending in .app makes sense.
It appears that the App Engine requests are coming from 8.35.201.0/24 in the U.S. This link seems to confirm that the servers are in the U.S. Ideally we minimize latency with that netblock.
Here's the official way to get the App Engine IPs: