Tor Browser with Snowflake

added anti-censorship-roadmap-2020 component::circumvention/snowflake priority::very high severity::critical sponsor::28-must status::new type::project ux-team labels

First working bundles with Snowflake, for linux only:

To run,

Open another browser to http://keroserene.net/snowflake/snowflake.html.
Run ./start-tor-browser.desktop
Say yes to "Does your Internet Service Provider (ISP) block or otherwise censor connections to the Tor Network?" and select snowflake from the menu. If all goes well, after a few seconds the other browser should turn green.

In order to build it yourself, you have to do

git clone -b snowflake https://git.torproject.org/user/dcf/tor-browser-bundle.git
cd tor-browser-bundle/gitian
build nightly TORSOCKS=

You'll need to prepared the gitian-builder directory and everything else according to [[doc/TorBrowser/Hacking]] and [[doc/TorBrowser/BuildingWithGitian]]. I did not try targets other than nightly.

Explanations of some noteworthy changes:

I cherry-picked 0001-Upgrade-go-to-1.6.2.patch:ticket:18333|the Go 1.6 patch from #18333 (moved) because arlolra said there is a dependency on Go 1.5 having to do with SRCDIR.
I tried using --no-history when running depot_tools' fetch and gclient, but it doesn't seem to work because the directories are 12 GB and have history.
libwebrtc needs a newer GCC than Debian wheezy has, so I copied code from the firefox descriptor to use our self-built GCC. I also tried building libwebrtc with Clang, which worked, but I couldn't get the resulting library to link with go-webrtc and Cgo.
At https://github.com/keroserene/go-webrtc/issues/23#issue-123390551, serene said that it was necessary to create a custom ninja file, but I didn't find it necessary (nor did I know what to put in the custom file).
The GYP_DEFINES are a result of trial and error and cribbing from the Debian chromium-browser source package.
I had to install Java and set JAVA_HOME to get past what seems to be a trivial step in webrtc/build/gyp_webrtc.
For constructing libwebrtc-magic.a, I used the ar crs command from go-webrtc's build.sh, but I had to exclude an additional file, dump_syms_regtest.o.
I had to disable _GLIBCXX_USE_CXX11_ABI=0 when building go-webrtc, because the prebuilt libwebrtc-magic.a is built an older default _GLIBCXX_USE_CXX11_ABI=0, but the Gitian-built libwebrtc-magic.a is built with the new default which is _GLIBCXX_USE_CXX11_ABI=1.
I used an old version of snowflake, 598e2a3bfb1cd8effab4c1b945035c870be16bff, because that's the parent of the bad commit mentioned in #18653 (moved). It might not be the same problem, but I couldn't get snowflake master to bootstrap past 10%.
I didn't know what to do about webrtc in verify-tags.sh, since the gclient sync step clones a bunch of other repos within itself.

Trac:

Screenshot of the first IRC message sent by a Snowflake Tor Browser, 2016-03-09.

I forgot to mention, I had to increase the size of the base VMs from 15360 to 25600 here and here, or else there wasn't enough space for the webrtc source code and compiled code. That change isn't reflected in the patches in comment:1.

I built bundles with make alpha from 6cd7c411e8. So make alpha works, and you should use that in preference to make nightly. https://people.torproject.org/~dcf/pt-bundle/snowflake/20160510-6.0a5-6cd7c411e891/

I built it twice in a row, and the output files matched.

The whole build (of linux only) took almost 8 hours.

real    471m19.011s
user    3m50.328s
sys     3m25.008s

Trac:
Cc: N/A to mrphs

In order to help out with https://github.com/keroserene/go-webrtc/issues/38, dcf pushed https://gitweb.torproject.org/user/dcf/tor-browser-bundle.git/commit/?h=snowflake&id=4c9dae0f2242361b71f74d0533ee5780fe60dbb8, and built https://people.torproject.org/~dcf/pt-bundle/snowflake/20160608-6.0a5-98487434e0/

The bundles there work fine, but using the magic to build go-webrtc (and thus, help out in the above) runs afoul.

go-webrtc has moved ahead now, so first check out 3a7f6aed75357dc85a7438af10b12c4177cf180f (or, better 5e9969c3bf5e63df086566404241c06279e5d56a).

Then, because the headers might not be what's expected (see https://gitweb.torproject.org/user/dcf/tor-browser-bundle.git/tree/gitian/descriptors/linux/gitian-pluggable-transports.yml?h=snowflake&id=6cd7c411e8910d75f5ef18e86bbe682f943aa229#n341), checkout 8ccf77319e08b86330c0e5b195697fd738c80931 in webrtc.git, and copy them over again.

Now, I'd expect it to work. But, linux 386 yields,

# github.com/keroserene/go-webrtc
/tmp/go-build151695054/github.com/keroserene/go-webrtc/_obj/peerconnection.cc.o: In function `CGO_DeserializeSDP':
./peerconnection.cc:344: undefined reference to `webrtc::JsepSessionDescription::JsepSessionDescription(std::string const&)'
./peerconnection.cc:347: undefined reference to `webrtc::SdpDeserialize(std::string const&, webrtc::JsepSessionDescription*, webrtc::SdpParseError*)'
/tmp/go-build151695054/github.com/keroserene/go-webrtc/_obj/peerconnection.cc.o: In function `CGO_AddIceCandidate':
./peerconnection.cc:386: undefined reference to `webrtc::CreateIceCandidate(std::string const&, int, std::string const&, webrtc::SdpParseError*)'
/tmp/go-build151695054/github.com/keroserene/go-webrtc/_obj/peerconnection.cc.o: In function `Peer::Initialize()':
./peerconnection.cc:54: undefined reference to `rtc::Thread::SetName(std::string const&, void const*)'
./peerconnection.cc:55: undefined reference to `rtc::Thread::SetName(std::string const&, void const*)'
collect2: error: ld returned 1 exit status

and amd64,

# _/home/david/branches/go-webrtc
/usr/bin/ld: skipping incompatible ./lib/libwebrtc-linux-amd64-magic.a when searching for -lwebrtc-linux-amd64-magic
/usr/bin/ld: skipping incompatible ./lib/libwebrtc-linux-amd64-magic.a when searching for -lwebrtc-linux-amd64-magic
/usr/bin/ld: skipping incompatible ./lib/libwebrtc-linux-amd64-magic.a when searching for -lwebrtc-linux-amd64-magic
/usr/bin/ld: cannot find -lwebrtc-linux-amd64-magic
collect2: error: ld returned 1 exit status
FAIL    _/home/david/branches/go-webrtc [build failed]

Maybe this has something to do with what's said above,

libwebrtc needs a newer GCC than Debian wheezy has, so I copied code from the firefox descriptor to use our self-built GCC. I also tried building libwebrtc with Clang, which worked, but I couldn't get the resulting library to link with go-webrtc and Cgo.

Another thing I noticed is that gclient sync -r hash does something funky with master if you aren't already on hash. What I've normally been doing is gclient sync && cd src/ && git checkout hash && cd .. && gclient sync -r hash, but maybe that was just me.

It was also suggested that it might be because go-webrtc sets _GLIBCXX_USE_CXX11_ABI=0 but playing with that didn't produce any useful results.

Anyways, just documenting for posterity.

Trac:
Cc: mrphs to mrphs, mcs

I've been working on getting the mac build to work; however I'm having trouble and it doesn't work yet. I pushed some work-in-progress changes here: https://gitweb.torproject.org/user/dcf/tor-browser-bundle.git/diff/?h=snowflake&id=dcf12df6630b519abecf86e58467aad151816083&id2=39a46f2b37573c3824e7bab8719d2bcd90f7af77

+ echo "print(\"$SDKROOT\")" > build/mac/find_sdk.py

This line prevents an error when the build system runs find_sdk.py, which tries to guess the Mac OS X SDK version of the build system; i.e., it does the wrong thing when cross-compiling:

$ webrtc/build/gyp_webrtc
Updating projects from gyp files...
Traceback (most recent call last):
  File "./build/mac/find_sdk.py", line 91, in <module>
    raise Exception("This script only runs on Mac")
Exception: This script only runs on Mac
gyp: Call to 'python ./build/mac/find_sdk.py 10.10' returned exit status 1 while in all.gyp.

+  sed -i -e 's#\$(SDKROOT)#/usr/lib/apple/SDKs/MacOSX10.6.sdk#g' out/Release/obj/webrtc/tools/*.ninja

For some reason, the $(SDKROOT) variable doesn't get properly substituted in these files, so I tried replacing it with a hardcoded value to get the build to at least progress. With this line, the error is:

$ ninja -C out/Release/
ninja: Entering directory `out/Release/'
ninja: error: obj/webrtc/tools/psnr_ssim_analyzer.ninja:43: bad $-escape (literal $ must be written as $$)
    $(SDKROOT)/System/Library/Frameworks/ApplicationServices.framework
    ^ near here

I first tried building using the usual /home/debian/build/apple-osx/bin/i686-apple-darwin11-g++. However that version of g++ appears to be too old to support the flags that Chromium assumes:

$ ninja -C out/Release/
ninja: Entering directory `out/Release/'
[1/1996] CXX obj/third_party/protobuf/src/google/protobuf/protobuf_lite.wire_format_lite.o
FAILED: obj/third_party/protobuf/src/google/protobuf/protobuf_lite.wire_format_lite.o
/home/debian/build/apple-osx/bin/i686-apple-darwin11-g++ -MMD -MF obj/third_party/protobuf/src/google/protobuf/protobuf_lite.wire_format_lite.o.d -DV8_DEPRECATION_WARNINGS -DCLD_VERSION=2 -D__ASSERT_MACROS_DEFINE_VERSIONS_WITHOUT_UNDERSCORE=0 -DCHROMIUM_BUILD -DUSE_AURA=1 -DUSE_OZONE=1 -DUSE_LIBJPEG_TURBO=1 -DENABLE_ONE_CLICK_SIGNIN -DENABLE_WEBRTC=1 -DENABLE_MEDIA_ROUTER=1 -DENABLE_CONFIGURATION_POLICY -DENABLE_NOTIFICATIONS -DENABLE_HIDPI=1 -DDONT_EMBED_BUILD_METADATA -DFIELDTRIAL_TESTING_ENABLED -DENABLE_TASK_MANAGER=1 -DENABLE_EXTENSIONS=1 -DENABLE_PLUGIN_INSTALLATION=1 -DENABLE_SESSION_SERVICE=1 -DENABLE_THEMES=1 -DENABLE_AUTOFILL_DIALOG=1 -DENABLE_BACKGROUND=1 -DENABLE_SPELLCHECK=1 -DUSE_BROWSER_SPELLCHECKER=1 -DENABLE_CAPTIVE_PORTAL_DETECTION=1 -DENABLE_APP_LIST=1 -DENABLE_SETTINGS_APP=1 -DENABLE_SUPERVISED_USERS=1 -DENABLE_SERVICE_DISCOVERY=1 -DV8_USE_EXTERNAL_STARTUP_DATA -DFULL_SAFE_BROWSING -DSAFE_BROWSING_CSD -DSAFE_BROWSING_DB_LOCAL -DLIBPROTOBUF_EXPORTS -DPROTOBUF_USE_DLLS -DGOOGLE_PROTOBUF_NO_RTTI -DGOOGLE_PROTOBUF_NO_STATIC_INITIALIZER -DUSE_LIBPCI=1 -DUSE_OPENSSL=1 -DNDEBUG -DNVALGRIND -DDYNAMIC_ANNOTATIONS_ENABLED=0 -Igen -I../../third_party/protobuf -I../../third_party/protobuf/src -fstack-protector --param=ssp-buffer-size=4 -Wno-unused-local-typedefs   -c ../../third_party/protobuf/src/google/protobuf/wire_format_lite.cc -o obj/third_party/protobuf/src/google/protobuf/protobuf_lite.wire_format_lite.o
cc1plus: error: unrecognized command line option "-Wno-unused-local-typedefs"

So I tried building it with clang, as in #18331 (moved).

Now the problem I'm having is a lack of a stdlib with C++11 support (features such as nullptr_t) for the x86_64-apple-darwin10 arch:

[3/1996] CXX obj/webrtc/common_audio/resampler/common_audio_sse2.sinc_resampler_sse.o
FAILED: obj/webrtc/common_audio/resampler/common_audio_sse2.sinc_resampler_sse.o
/home/debian/build/clang/bin/clang++ -MMD -MF obj/webrtc/common_audio/resampler/common_audio_sse2.sinc_resampler_sse.o.d -DV8_DEPRECATION_WARNINGS -DCLD_VERSION=2 -D__ASS
LE_ONE_CLICK_SIGNIN -DENABLE_WEBRTC=1 -DENABLE_MEDIA_ROUTER=1 -DENABLE_CONFIGURATION_POLICY -DENABLE_NOTIFICATIONS -DENABLE_HIDPI=1 -DDONT_EMBED_BUILD_METADATA -DFIELDTRI
_AUTOFILL_DIALOG=1 -DENABLE_BACKGROUND=1 -DENABLE_SPELLCHECK=1 -DUSE_BROWSER_SPELLCHECKER=1 -DENABLE_CAPTIVE_PORTAL_DETECTION=1 -DENABLE_APP_LIST=1 -DENABLE_SETTINGS_APP=
_LOCAL -DEXPAT_RELATIVE_PATH -DWEBRTC_POSIX -DWEBRTC_MAC -DUSE_LIBPCI=1 -DUSE_OPENSSL=1 -D__STDC_CONSTANT_MACROS -D__STDC_FORMAT_MACROS -DNDEBUG -DNVALGRIND -DDYNAMIC_ANN
ripts -Wno-unneeded-internal-declaration -Wno-covered-switch-default -Wstring-conversion -Wno-c++11-narrowing -Wno-deprecated-register -Wno-inconsistent-missing-override
msse2 -std=gnu++11 -stdlib=libc++ -mmacosx-version-min=10.6 -target x86_64-apple-darwin10 -isysroot /home/debian/build/MacOSX10.7.sdk -I/home/debian/build/MacOSX10.7.sdk/
/sinc_resampler_sse.cc -o obj/webrtc/common_audio/resampler/common_audio_sse2.sinc_resampler_sse.o
In file included from ../../webrtc/common_audio/resampler/sinc_resampler_sse.cc:14:
In file included from ../../webrtc/common_audio/resampler/sinc_resampler.h:18:
../../webrtc/base/scoped_ptr.h:331:19: error: no type named 'nullptr_t' in namespace 'std'
  scoped_ptr(std::nullptr_t) : impl_(nullptr) {}
             ~~~~~^
../../webrtc/base/scoped_ptr.h:368:30: error: no type named 'nullptr_t' in namespace 'std'
  scoped_ptr& operator=(std::nullptr_t) {
                        ~~~~~^
../../webrtc/base/scoped_ptr.h:380:17: error: no member named 'move' in namespace 'std'
    return std::move(*this);
           ~~~~~^
../../webrtc/base/scoped_ptr.h:491:19: error: no type named 'nullptr_t' in namespace 'std'
  scoped_ptr(std::nullptr_t) : impl_(nullptr) {}
             ~~~~~^
../../webrtc/base/scoped_ptr.h:504:30: error: no type named 'nullptr_t' in namespace 'std'
  scoped_ptr& operator=(std::nullptr_t) {
                        ~~~~~^
../../webrtc/base/scoped_ptr.h:516:17: error: no member named 'move' in namespace 'std'
    return std::move(*this);
           ~~~~~^
6 errors generated.

I wrote a longer howto on building Tor Browser at [[doc/Snowflake#IntegrationwithTorBrowser]].

I pushed new commits that update libwebrtc to branch-heads/52 (as in GH#39) and snowflake to 0ae71952d2. https://gitweb.torproject.org/user/dcf/tor-browser-bundle.git/diff/?h=snowflake&id=a83d3993eebbad215a81b991918b22bb3ca9bcef&id2=484301cf415ef7e5b0e4aaaa05fd6bde1395cb5b Here are bundles, still only working for linux. https://people.torproject.org/~dcf/pt-bundle/snowflake/20160726-6.5a1-81842c68e1e4/

I'm making some progress on the mac build. I set it up to cross-compile libc++ from scratch, giving us a cross-compiled C++11 standard library. That worked to compile a few hundred source files into Mac object files.

There are still some problems. It's becoming clear that the webrtc source isn't really meant to be cross-compiled. On the surface it would appear to be supported, because there are variables like CC_host and CFLAGS_host that usually indicate support for cross-compiling. But it turns out they are not honored everywhere. The first place the build fails is in yasm, which is set up to use host settings for CFLAGS but then uses (incompatible) target settings for LDFLAGS. I hacked around that, but then there are some tools meant to be compiled for the host system that link against libraries compiled for the target system. Now I'm doing some refactoring and trying to find a way around it.

I successfully built TBB (just for linux) with latest snowflake ac9d49b872 Can we bump the version to this latest one? (I can't push to your branch)

To verify:

2213aebf764cfd3bd6c9ba0fa9c13b14f712e9db6c57b6f66404c94261109fdc  tor-browser-linux32-6.5a1_en-US.tar.xz
683ef3a2035633c5aa592b008a523a1ae138e202fa980b6fb92477279f5a2b2a  tor-browser-linux64-6.5a1_en-US.tar.xz

Also, question about including multiplexing in the torrc defaults.

For quicker recovery when browsing and the current remote goes offline, I've manually added the flag -max 3 to the ClientTransportPlugin snowflake line in /Browser/TorBrowser/Data/Tor/torrc. Can you confirm if multiplexing works well on your end / if this might be a sensible default?

If so, would it make sense to update the torrc in the pre-build so that the build itself would generate this default?

Trac:
Cc: mrphs, mcs to mrphs, mcs, serene

Replying to serene:

I successfully built TBB (just for linux) with latest snowflake ac9d49b872 Can we bump the version to this latest one? (I can't push to your branch)

I created #19848 (moved) to give you and arlo access.

Also, question about including multiplexing in the torrc defaults.

For quicker recovery when browsing and the current remote goes offline, I've manually added the flag -max 3 to the ClientTransportPlugin snowflake line in /Browser/TorBrowser/Data/Tor/torrc. Can you confirm if multiplexing works well on your end / if this might be a sensible default?

If so, would it make sense to update the torrc in the pre-build so that the build itself would generate this default?

Edit the files Bundle-Data/PTConfigs/{linux,mac,windows}/torrc-defaults-appendix in order to change command-line defaults.

I'm in the middle of the mac build so I haven't tried ac9d49b872.

I'm making some good progress on the mac build. For the first time, I got a libwebrtc-darwin-amd64-magic.a built. (I had to do some bad things to do it.) The build of go-webrtc failed ultimately because of a few missing Objective-C++ symbols--the build system assumes that only Xcode can compile Objective-C++ and left out a few source files. So now I'm digging into that but my feeling is we're getting pretty close. (Whether the compiled library will actually work is potentially another matter.)

I'm doing a build now to see that I got all my manual steps into the gitian descriptors.

Here are some Linux bundles I built from comment:9,

https://paganini.erinn.org/~arlolra/snowflake/20160806-6.5a1-a83d3993/

They don't match what dcf posted there, but I haven't looked into why yet.

I followed the integration docs but hit a few snags along the way:

Needed to add https://git.torproject.org/builders/tor-browser-bundle.git as a remote and git fetch --tags
During the gclient sync, needed to apt-get install curl pkg-config libgtk2.0-dev libglib2.0-dev
Applied boklm's patch from #19737 (moved) (but that was well documented)
Got prompted for an SSH password, as in #18786 (moved) (debian@localhost's password:), despite being on the tor-browser-builder-4 branch and having the commits referenced there. I did the s/ecdsa/dsa/ dcf suggest, and a make-clean-vm got past that step.

The bundles are around 70 MB each. Next, I'm going to try applying my patch in #19569 (moved) to see if we can get that down.

I at last got libwebrtc built for mac and snowflake linking to it. Does it work? I don't know! I don't have access to a Mac just now. https://gitweb.torproject.org/user/dcf/tor-browser-bundle.git/log/?h=snowflake-mac-1 https://people.torproject.org/~dcf/pt-bundle/snowflake/20160806-6.5a1-965e0daa7d59/

The repository history is a complete disaster, but I decided to just push it as it is to a side branch and we can clean it up later. We'll have to do a huge rebase anyway before merging to master. Here's the diff relative to the last working linux build.

At a high level, the main major changes are 1) using clang and libc++ to build libwebrtc, and 2) factoring out a separate gitian-webrtc.yml descriptor. These changes I'll probably port back to the linux descriptor. About the various other necessary hacks, the best thing I can say is they are finite in number. There was one straight-up bug in libc++'s CMakeLists.txt that I reported upstream. I had to do some finessing of CFLAGS et al. because our clang and 10.7 SDK are a little older than what the build script expect. The build really expects you to be doing Mac builds using Xcode and I had to work around that assumption in a number of places, including a little bit of rewriting of ninja files. The biggest and ugliest part is this patch that gets applied directly to the webrtc sources.

Does it work?

Yup!

I rebased and refactored the snowflake-mac branch. https://gitweb.torproject.org/user/dcf/tor-browser-bundle.git/log/?h=snowflake-mac-2 I'm starting a build with it now. If it works, I'll merge it into the main snowflake branch. Then I want to merge Serene's go-webrtc updates (comment:11), Arlo's datachannel patch (comment:1:ticket:19569), then merge with master to pick up the patch for #19737 (moved).

Tor Browser with Snowflake

Child items 0

Activity