Opened 17 months ago

Last modified 3 weeks ago

#19001 new project

Tor Browser with Snowflake

Reported by: dcf Owned by:
Priority: Medium Milestone:
Component: Obfuscation/Snowflake Version:
Severity: Normal Keywords:
Cc: mrphs, mcs, serene, boklm, arthuredelstein, adrelanos Actual Points:
Parent ID: Points:
Reviewer: Sponsor:

Description

Let's prepare a branch with reproducible builds and a built-in Snowflake client selectable from the menu.

These GitHub issues are relevant:

  • #23 native webrtc dependency build script
  • #29 Reproducible builds

Child Tickets

TicketStatusOwnerSummaryComponent
#19315newInclude libwebrtc license files in bundleObfuscation/Snowflake
#19569newarlolraDataChannel-only libwebrtcObfuscation/Snowflake

Attachments (2)

snowflake-1st.png (103.4 KB) - added by dcf 17 months ago.
Screenshot of the first IRC message sent by a Snowflake Tor Browser, 2016-03-09.
snowflake-client-mac-fw.png (29.6 KB) - added by dcf 13 months ago.

Download all attachments as: .zip

Change History (38)

comment:1 Changed 17 months ago by dcf

First working bundles with Snowflake, for linux only:

Overall diff so far:

To run,

  1. Open another browser to http://keroserene.net/snowflake/snowflake.html.
  2. Run ./start-tor-browser.desktop
  3. Say yes to "Does your Internet Service Provider (ISP) block or otherwise censor connections to the Tor Network?" and select snowflake from the menu.

If all goes well, after a few seconds the other browser should turn green.

In order to build it yourself, you have to do

git clone -b snowflake https://git.torproject.org/user/dcf/tor-browser-bundle.git
cd tor-browser-bundle/gitian
build nightly TORSOCKS=

You'll need to prepared the gitian-builder directory and everything else according to doc/TorBrowser/Hacking and doc/TorBrowser/BuildingWithGitian. I did not try targets other than nightly.

Explanations of some noteworthy changes:

Changed 17 months ago by dcf

Attachment: snowflake-1st.png added

Screenshot of the first IRC message sent by a Snowflake Tor Browser, 2016-03-09.

comment:2 Changed 17 months ago by dcf

I forgot to mention, I had to increase the size of the base VMs from 15360 to 25600 here and here, or else there wasn't enough space for the webrtc source code and compiled code. That change isn't reflected in the patches in comment:1.

comment:3 Changed 17 months ago by dcf

I built bundles with make alpha from 6cd7c411e8. So make alpha works, and you should use that in preference to make nightly.

https://people.torproject.org/~dcf/pt-bundle/snowflake/20160510-6.0a5-6cd7c411e891/

I built it twice in a row, and the output files matched.

The whole build (of linux only) took almost 8 hours.

real    471m19.011s
user    3m50.328s
sys     3m25.008s

comment:4 Changed 15 months ago by mrphs

Cc: mrphs added

comment:5 Changed 15 months ago by arlolra

In order to help out with https://github.com/keroserene/go-webrtc/issues/38,
dcf pushed https://gitweb.torproject.org/user/dcf/tor-browser-bundle.git/commit/?h=snowflake&id=4c9dae0f2242361b71f74d0533ee5780fe60dbb8,
and built https://people.torproject.org/~dcf/pt-bundle/snowflake/20160608-6.0a5-98487434e0/

The bundles there work fine, but using the magic to build go-webrtc (and thus, help out in the above) runs afoul.

go-webrtc has moved ahead now, so first check out 3a7f6aed75357dc85a7438af10b12c4177cf180f (or, better 5e9969c3bf5e63df086566404241c06279e5d56a).

Then, because the headers might not be what's expected (see https://gitweb.torproject.org/user/dcf/tor-browser-bundle.git/tree/gitian/descriptors/linux/gitian-pluggable-transports.yml?h=snowflake&id=6cd7c411e8910d75f5ef18e86bbe682f943aa229#n341), checkout 8ccf77319e08b86330c0e5b195697fd738c80931 in webrtc.git, and copy them over again.

Now, I'd expect it to work. But, linux 386 yields,

# github.com/keroserene/go-webrtc
/tmp/go-build151695054/github.com/keroserene/go-webrtc/_obj/peerconnection.cc.o: In function `CGO_DeserializeSDP':
./peerconnection.cc:344: undefined reference to `webrtc::JsepSessionDescription::JsepSessionDescription(std::string const&)'
./peerconnection.cc:347: undefined reference to `webrtc::SdpDeserialize(std::string const&, webrtc::JsepSessionDescription*, webrtc::SdpParseError*)'
/tmp/go-build151695054/github.com/keroserene/go-webrtc/_obj/peerconnection.cc.o: In function `CGO_AddIceCandidate':
./peerconnection.cc:386: undefined reference to `webrtc::CreateIceCandidate(std::string const&, int, std::string const&, webrtc::SdpParseError*)'
/tmp/go-build151695054/github.com/keroserene/go-webrtc/_obj/peerconnection.cc.o: In function `Peer::Initialize()':
./peerconnection.cc:54: undefined reference to `rtc::Thread::SetName(std::string const&, void const*)'
./peerconnection.cc:55: undefined reference to `rtc::Thread::SetName(std::string const&, void const*)'
collect2: error: ld returned 1 exit status

and amd64,

# _/home/david/branches/go-webrtc
/usr/bin/ld: skipping incompatible ./lib/libwebrtc-linux-amd64-magic.a when searching for -lwebrtc-linux-amd64-magic
/usr/bin/ld: skipping incompatible ./lib/libwebrtc-linux-amd64-magic.a when searching for -lwebrtc-linux-amd64-magic
/usr/bin/ld: skipping incompatible ./lib/libwebrtc-linux-amd64-magic.a when searching for -lwebrtc-linux-amd64-magic
/usr/bin/ld: cannot find -lwebrtc-linux-amd64-magic
collect2: error: ld returned 1 exit status
FAIL    _/home/david/branches/go-webrtc [build failed]

Maybe this has something to do with what's said above,

libwebrtc needs a newer GCC than Debian wheezy has, so ​I copied code from the ​firefox descriptor to use our self-built GCC. I also tried building libwebrtc with Clang, which worked, but I couldn't get the resulting library to link with go-webrtc and Cgo.

Another thing I noticed is that gclient sync -r hash does something funky with master if you aren't already on hash. What I've normally been doing is gclient sync && cd src/ && git checkout hash && cd .. && gclient sync -r hash, but maybe that was just me.

It was also suggested that it might be because go-webrtc sets _GLIBCXX_USE_CXX11_ABI=0 but playing with that didn't produce any useful results.

Anyways, just documenting for posterity.

comment:6 Changed 15 months ago by mcs

Cc: mcs added

comment:7 Changed 14 months ago by dcf

I've been working on getting the mac build to work; however I'm having trouble and it doesn't work yet. I pushed some work-in-progress changes here:

https://gitweb.torproject.org/user/dcf/tor-browser-bundle.git/diff/?h=snowflake&id=dcf12df6630b519abecf86e58467aad151816083&id2=39a46f2b37573c3824e7bab8719d2bcd90f7af77

+ echo "print(\"$SDKROOT\")" > build/mac/find_sdk.py

This line prevents an error when the build system runs find_sdk.py, which tries to guess the Mac OS X SDK version of the build system; i.e., it does the wrong thing when cross-compiling:

$ webrtc/build/gyp_webrtc
Updating projects from gyp files...
Traceback (most recent call last):
  File "./build/mac/find_sdk.py", line 91, in <module>
    raise Exception("This script only runs on Mac")
Exception: This script only runs on Mac
gyp: Call to 'python ./build/mac/find_sdk.py 10.10' returned exit status 1 while in all.gyp.
+  sed -i -e 's#\$(SDKROOT)#/usr/lib/apple/SDKs/MacOSX10.6.sdk#g' out/Release/obj/webrtc/tools/*.ninja

For some reason, the $(SDKROOT) variable doesn't get properly substituted in these files, so I tried replacing it with a hardcoded value to get the build to at least progress. With this line, the error is:

$ ninja -C out/Release/
ninja: Entering directory `out/Release/'
ninja: error: obj/webrtc/tools/psnr_ssim_analyzer.ninja:43: bad $-escape (literal $ must be written as $$)
    $(SDKROOT)/System/Library/Frameworks/ApplicationServices.framework
    ^ near here

I first tried building using the usual /home/debian/build/apple-osx/bin/i686-apple-darwin11-g++. However that version of g++ appears to be too old to support the flags that Chromium assumes:

$ ninja -C out/Release/
ninja: Entering directory `out/Release/'
[1/1996] CXX obj/third_party/protobuf/src/google/protobuf/protobuf_lite.wire_format_lite.o
FAILED: obj/third_party/protobuf/src/google/protobuf/protobuf_lite.wire_format_lite.o
/home/debian/build/apple-osx/bin/i686-apple-darwin11-g++ -MMD -MF obj/third_party/protobuf/src/google/protobuf/protobuf_lite.wire_format_lite.o.d -DV8_DEPRECATION_WARNINGS -DCLD_VERSION=2 -D__ASSERT_MACROS_DEFINE_VERSIONS_WITHOUT_UNDERSCORE=0 -DCHROMIUM_BUILD -DUSE_AURA=1 -DUSE_OZONE=1 -DUSE_LIBJPEG_TURBO=1 -DENABLE_ONE_CLICK_SIGNIN -DENABLE_WEBRTC=1 -DENABLE_MEDIA_ROUTER=1 -DENABLE_CONFIGURATION_POLICY -DENABLE_NOTIFICATIONS -DENABLE_HIDPI=1 -DDONT_EMBED_BUILD_METADATA -DFIELDTRIAL_TESTING_ENABLED -DENABLE_TASK_MANAGER=1 -DENABLE_EXTENSIONS=1 -DENABLE_PLUGIN_INSTALLATION=1 -DENABLE_SESSION_SERVICE=1 -DENABLE_THEMES=1 -DENABLE_AUTOFILL_DIALOG=1 -DENABLE_BACKGROUND=1 -DENABLE_SPELLCHECK=1 -DUSE_BROWSER_SPELLCHECKER=1 -DENABLE_CAPTIVE_PORTAL_DETECTION=1 -DENABLE_APP_LIST=1 -DENABLE_SETTINGS_APP=1 -DENABLE_SUPERVISED_USERS=1 -DENABLE_SERVICE_DISCOVERY=1 -DV8_USE_EXTERNAL_STARTUP_DATA -DFULL_SAFE_BROWSING -DSAFE_BROWSING_CSD -DSAFE_BROWSING_DB_LOCAL -DLIBPROTOBUF_EXPORTS -DPROTOBUF_USE_DLLS -DGOOGLE_PROTOBUF_NO_RTTI -DGOOGLE_PROTOBUF_NO_STATIC_INITIALIZER -DUSE_LIBPCI=1 -DUSE_OPENSSL=1 -DNDEBUG -DNVALGRIND -DDYNAMIC_ANNOTATIONS_ENABLED=0 -Igen -I../../third_party/protobuf -I../../third_party/protobuf/src -fstack-protector --param=ssp-buffer-size=4 -Wno-unused-local-typedefs   -c ../../third_party/protobuf/src/google/protobuf/wire_format_lite.cc -o obj/third_party/protobuf/src/google/protobuf/protobuf_lite.wire_format_lite.o
cc1plus: error: unrecognized command line option "-Wno-unused-local-typedefs"

So I tried building it with clang, as in #18331.

Now the problem I'm having is a lack of a stdlib with C++11 support (features such as nullptr_t) for the x86_64-apple-darwin10 arch:

[3/1996] CXX obj/webrtc/common_audio/resampler/common_audio_sse2.sinc_resampler_sse.o
FAILED: obj/webrtc/common_audio/resampler/common_audio_sse2.sinc_resampler_sse.o
/home/debian/build/clang/bin/clang++ -MMD -MF obj/webrtc/common_audio/resampler/common_audio_sse2.sinc_resampler_sse.o.d -DV8_DEPRECATION_WARNINGS -DCLD_VERSION=2 -D__ASS
LE_ONE_CLICK_SIGNIN -DENABLE_WEBRTC=1 -DENABLE_MEDIA_ROUTER=1 -DENABLE_CONFIGURATION_POLICY -DENABLE_NOTIFICATIONS -DENABLE_HIDPI=1 -DDONT_EMBED_BUILD_METADATA -DFIELDTRI
_AUTOFILL_DIALOG=1 -DENABLE_BACKGROUND=1 -DENABLE_SPELLCHECK=1 -DUSE_BROWSER_SPELLCHECKER=1 -DENABLE_CAPTIVE_PORTAL_DETECTION=1 -DENABLE_APP_LIST=1 -DENABLE_SETTINGS_APP=
_LOCAL -DEXPAT_RELATIVE_PATH -DWEBRTC_POSIX -DWEBRTC_MAC -DUSE_LIBPCI=1 -DUSE_OPENSSL=1 -D__STDC_CONSTANT_MACROS -D__STDC_FORMAT_MACROS -DNDEBUG -DNVALGRIND -DDYNAMIC_ANN
ripts -Wno-unneeded-internal-declaration -Wno-covered-switch-default -Wstring-conversion -Wno-c++11-narrowing -Wno-deprecated-register -Wno-inconsistent-missing-override
msse2 -std=gnu++11 -stdlib=libc++ -mmacosx-version-min=10.6 -target x86_64-apple-darwin10 -isysroot /home/debian/build/MacOSX10.7.sdk -I/home/debian/build/MacOSX10.7.sdk/
/sinc_resampler_sse.cc -o obj/webrtc/common_audio/resampler/common_audio_sse2.sinc_resampler_sse.o
In file included from ../../webrtc/common_audio/resampler/sinc_resampler_sse.cc:14:
In file included from ../../webrtc/common_audio/resampler/sinc_resampler.h:18:
../../webrtc/base/scoped_ptr.h:331:19: error: no type named 'nullptr_t' in namespace 'std'
  scoped_ptr(std::nullptr_t) : impl_(nullptr) {}
             ~~~~~^
../../webrtc/base/scoped_ptr.h:368:30: error: no type named 'nullptr_t' in namespace 'std'
  scoped_ptr& operator=(std::nullptr_t) {
                        ~~~~~^
../../webrtc/base/scoped_ptr.h:380:17: error: no member named 'move' in namespace 'std'
    return std::move(*this);
           ~~~~~^
../../webrtc/base/scoped_ptr.h:491:19: error: no type named 'nullptr_t' in namespace 'std'
  scoped_ptr(std::nullptr_t) : impl_(nullptr) {}
             ~~~~~^
../../webrtc/base/scoped_ptr.h:504:30: error: no type named 'nullptr_t' in namespace 'std'
  scoped_ptr& operator=(std::nullptr_t) {
                        ~~~~~^
../../webrtc/base/scoped_ptr.h:516:17: error: no member named 'move' in namespace 'std'
    return std::move(*this);
           ~~~~~^
6 errors generated.

comment:8 Changed 14 months ago by dcf

I wrote a longer howto on building Tor Browser at doc/Snowflake#IntegrationwithTorBrowser.

comment:10 Changed 14 months ago by dcf

I'm making some progress on the mac build. I set it up to cross-compile libc++ from scratch, giving us a cross-compiled C++11 standard library. That worked to compile a few hundred source files into Mac object files.

There are still some problems. It's becoming clear that the webrtc source isn't really meant to be cross-compiled. On the surface it would appear to be supported, because there are variables like CC_host and CFLAGS_host that usually indicate support for cross-compiling. But it turns out they are not honored everywhere. The first place the build fails is in yasm, which is set up to use host settings for CFLAGS but then uses (incompatible) target settings for LDFLAGS. I hacked around that, but then there are some tools meant to be compiled for the host system that link against libraries compiled for the target system. Now I'm doing some refactoring and trying to find a way around it.

comment:11 Changed 14 months ago by serene

Cc: serene added

I successfully built TBB (just for linux) with latest snowflake ac9d49b872
Can we bump the version to this latest one? (I can't push to your branch)

To verify:

2213aebf764cfd3bd6c9ba0fa9c13b14f712e9db6c57b6f66404c94261109fdc  tor-browser-linux32-6.5a1_en-US.tar.xz
683ef3a2035633c5aa592b008a523a1ae138e202fa980b6fb92477279f5a2b2a  tor-browser-linux64-6.5a1_en-US.tar.xz

Also, question about including multiplexing in the torrc defaults.

For quicker recovery when browsing and the current remote goes offline, I've manually added the flag -max 3 to the ClientTransportPlugin snowflake line in /Browser/TorBrowser/Data/Tor/torrc. Can you confirm if multiplexing works well on your end / if this might be a sensible default?

If so, would it make sense to update the torrc in the pre-build so that the build itself would generate this default?

comment:12 in reply to:  11 Changed 14 months ago by dcf

Replying to serene:

I successfully built TBB (just for linux) with latest snowflake ac9d49b872
Can we bump the version to this latest one? (I can't push to your branch)

I created #19848 to give you and arlo access.

Also, question about including multiplexing in the torrc defaults.

For quicker recovery when browsing and the current remote goes offline, I've manually added the flag -max 3 to the ClientTransportPlugin snowflake line in /Browser/TorBrowser/Data/Tor/torrc. Can you confirm if multiplexing works well on your end / if this might be a sensible default?

If so, would it make sense to update the torrc in the pre-build so that the build itself would generate this default?

Edit the files Bundle-Data/PTConfigs/{linux,mac,windows}/torrc-defaults-appendix in order to change command-line defaults.

I'm in the middle of the mac build so I haven't tried ac9d49b872.

comment:13 Changed 14 months ago by dcf

I'm making some good progress on the mac build. For the first time, I got a libwebrtc-darwin-amd64-magic.a built. (I had to do some bad things to do it.) The build of go-webrtc failed ultimately because of a few missing Objective-C++ symbols--the build system assumes that only Xcode can compile Objective-C++ and left out a few source files. So now I'm digging into that but my feeling is we're getting pretty close. (Whether the compiled library will actually work is potentially another matter.)

I'm doing a build now to see that I got all my manual steps into the gitian descriptors.

comment:14 Changed 14 months ago by arlolra

Here are some Linux bundles I built from comment:9,

https://paganini.erinn.org/~arlolra/snowflake/20160806-6.5a1-a83d3993/

They don't match what dcf posted there, but I haven't looked into why yet.

I followed the integration docs but hit a few snags along the way:

  • Needed to add https://git.torproject.org/builders/tor-browser-bundle.git as a remote and git fetch --tags
  • During the gclient sync, needed to apt-get install curl pkg-config libgtk2.0-dev libglib2.0-dev
  • Applied boklm's patch from #19737 (but that was well documented)
  • Got prompted for an SSH password, as in #18786 (debian@localhost's password:), despite being on the tor-browser-builder-4 branch and having the commits referenced there. I did the s/ecdsa/dsa/ dcf suggest, and a make-clean-vm got past that step.

The bundles are around 70 MB each. Next, I'm going to try applying my patch in #19569 to see if we can get that down.

comment:15 Changed 14 months ago by dcf

I at last got libwebrtc built for mac and snowflake linking to it. Does it work? I don't know! I don't have access to a Mac just now.

https://gitweb.torproject.org/user/dcf/tor-browser-bundle.git/log/?h=snowflake-mac-1
https://people.torproject.org/~dcf/pt-bundle/snowflake/20160806-6.5a1-965e0daa7d59/

The repository history is a complete disaster, but I decided to just push it as it is to a side branch and we can clean it up later. We'll have to do a huge rebase anyway before merging to master. Here's the diff relative to the last working linux build.

At a high level, the main major changes are 1) using clang and libc++ to build libwebrtc, and 2) factoring out a separate gitian-webrtc.yml descriptor. These changes I'll probably port back to the linux descriptor. About the various other necessary hacks, the best thing I can say is they are finite in number. There was one straight-up bug in libc++'s CMakeLists.txt that I reported upstream. I had to do some finessing of CFLAGS et al. because our clang and 10.7 SDK are a little older than what the build script expect. The build really expects you to be doing Mac builds using Xcode and I had to work around that assumption in a number of places, including a little bit of rewriting of ninja files. The biggest and ugliest part is this patch that gets applied directly to the webrtc sources.

Last edited 13 months ago by dcf (previous) (diff)

comment:16 Changed 14 months ago by arlolra

Does it work?

Yup!

comment:17 Changed 13 months ago by dcf

I rebased and refactored the snowflake-mac branch.

https://gitweb.torproject.org/user/dcf/tor-browser-bundle.git/log/?h=snowflake-mac-2

I'm starting a build with it now. If it works, I'll merge it into the main snowflake branch. Then I want to merge Serene's go-webrtc updates (comment:11), Arlo's datachannel patch (comment:1:ticket:19569), then merge with master to pick up the patch for #19737.

comment:18 in reply to:  17 Changed 13 months ago by dcf

Replying to dcf:

I rebased and refactored the snowflake-mac branch.

https://gitweb.torproject.org/user/dcf/tor-browser-bundle.git/log/?h=snowflake-mac-2

I'm starting a build with it now. If it works, I'll merge it into the main snowflake branch. Then I want to merge Serene's go-webrtc updates (comment:11), Arlo's datachannel patch (comment:1:ticket:19569), then merge with master to pick up the patch for #19737.

There were a couple of bugs in snowflake-mac-2 but I fixed them in snowflake-mac-3 and merged all the changes to the main snowflake branch.

https://gitweb.torproject.org/user/dcf/tor-browser-bundle.git/log/?h=snowflake-mac-3
https://gitweb.torproject.org/user/dcf/tor-browser-bundle.git/log/?h=snowflake&id=c2f636b8c367ac562d943489311b579b1a13f980

comment:19 Changed 13 months ago by arlolra

Here's a build of c2f636b8c367ac562d943489311b579b1a13f980 for osx:

https://paganini.erinn.org/~arlolra/snowflake/20160820-6.5a1-c2f636b8/

I noticed that we're putting the snowflake.log beside the snowflake-client.
Maybe that'd be better in ~/Library/Application Support/TorBrowser-Data/

comment:20 in reply to:  19 Changed 13 months ago by yawning

Replying to arlolra:

I noticed that we're putting the snowflake.log beside the snowflake-client.

That breaks GateKeeper signature verification.

Maybe that'd be better in ~/Library/Application Support/TorBrowser-Data/

That's still incorrect, try again.

https://gitweb.torproject.org/torspec.git/tree/pt-spec.txt#n188

comment:22 in reply to:  15 Changed 13 months ago by dcf

Replying to dcf:

There was one straight-up bug in libc++'s CMakeLists.txt that I reported upstream.

The libc++ CMake bug got fixed upstream: https://llvm.org/bugs/show_bug.cgi?id=28831#c3. (Currently their bugzilla is demanding a login to even view the bug, but it just says "Fixed in r280037. Please reopen if you run into any trouble.")

Their fix is on their r280037 and we are currently on ~r247359 (which I chose to be contemporaneous with the already existing clang and llvm revisions).

comment:23 in reply to:  19 Changed 13 months ago by dcf

Replying to arlolra:

Here's a build of c2f636b8c367ac562d943489311b579b1a13f980 for osx:

https://paganini.erinn.org/~arlolra/snowflake/20160820-6.5a1-c2f636b8/

I put my corresponding build here:

https://people.torproject.org/~dcf/pt-bundle/snowflake/20160820-6.5a1-c2f636b8/

Here is the diffoscope:

https://people.torproject.org/~dcf/pt-bundle/snowflake/20160820-6.5a1-c2f636b8/20160820-6.5a1-c2f636b8-dcf-arlolra.diffoscope.txt

Apart from expected differences in bundle.inputs and versions, the only differing file is snowflake-client (and it has a lot of differences). We might want to objdump the two snowflake-clients to see what is going on.

comment:24 in reply to:  17 Changed 13 months ago by dcf

Replying to dcf:

Then I want to merge Serene's go-webrtc updates (comment:11), Arlo's datachannel patch (comment:1:ticket:19569), then merge with master to pick up the patch for #19737.

I pushed:

a2d13e6f42 Bump SNOWFLAKE_TAG to !6cecd31fd896eb26e64ad8bab8a9ea510ec3b21d.
f811240035 Merge tag 'tbb-6.5a2-build2' into snowflake

I didn't do the datachannel patch (#19659) because I wasn't sure it was going to work in the mac descriptor (it probably will, with some work, but I didn't have time for it yet).

Bundle are here:

https://people.torproject.org/~dcf/pt-bundle/snowflake/20160830-6.5a2-f811240035c3/

I tested the linux64 and mac bundles and they work.

This was my first time running the mac bundle. I got a firewall dialog the first time running it:


I let the dialog remain for a few seconds and the connection failed. This is what was at the end of snowflake.log:

2016/08/30 13:32:35 SOCKS accepted:  {0.0.3.0:1  map[]}
2016/08/30 13:32:35 ---- Handler: snowflake assigned ----
2016/08/30 13:32:35 Buffered 235 bytes --> WebRTC
2016/08/30 13:32:40 Traffic Bytes (in|out): 0 | 235 -- (0 OnMessages, 1 Sends)
2016/08/30 13:32:45 WebRTC: At capacity [1/1]  Retrying in 10 seconds...
2016/08/30 13:32:55 WebRTC: At capacity [1/1]  Retrying in 10 seconds...
2016/08/30 13:33:05 WebRTC: At capacity [1/1]  Retrying in 10 seconds...
2016/08/30 13:33:05 WebRTC: No messages received for 30 seconds -- closing stale connection.
2016/08/30 13:33:05 WebRTC: closing PeerConnection
2016/08/30 13:33:05 WebRTC: DataChannel.OnClose [locally]
2016/08/30 13:33:05 WebRTC: Closing
2016/08/30 13:33:05 copy loop ended
2016/08/30 13:33:05 ---- Handler: closed ---
2016/08/30 13:33:05 SOCKS listening...
2016/08/30 13:33:05 WebRTC: melted all 0 snowflakes.
2016/08/30 13:33:05 snowflake is done.

The snowflake proxy itself seemed to stall at this point, saying Status: Serving 1 new client. (Polling in 0 seconds...). I had to refresh the page before I could try to connect again. After I allowed snowflake-client through the firewall, everything worked.

Changed 13 months ago by dcf

Attachment: snowflake-client-mac-fw.png added

comment:25 Changed 13 months ago by dcf

I posted a description of our mac build to the discuss-webrtc mailing list:

https://groups.google.com/forum/?hl=en#!topic/discuss-webrtc/ca_R9ocEHus

comment:26 Changed 12 months ago by dcf

GN migration

https://webrtc.org/native-code/development/ these days says "GYP is no longer supported and likely to break soon. You’re encouraged to migrate to GN as soon as possible."

I pushed a first try at migrating to GN here:

https://gitweb.torproject.org/user/dcf/tor-browser-bundle.git/log/?h=snowflake-gn&id=2bb19e4c15446faf4ae8ac36e9d34847d80b4d8b

The webrtc build works, however it's missing some symbols required by go-webrtc, such as FakeAudioCaptureModule, which aren't accounted for yet in the upstream GN build files. It seems that GN support for webrtc is not complete yet, as this message from August 10, 2016:

You seem to be setting the GN variables. WebRTC still uses GYP by default which may explain why you don't see any change (we're on our way to migrate to GN though).

My plan is to forget about the GN migration until the next time we update the webrtc upstream.

Here's a recent revision (from today, in fact) that seems to be something we need (other webrtc-using projects asked for it):

"Switching to GN from GYP: RTTI flag and static libraries?"
https://groups.google.com/d/msg/discuss-webrtc/wBf7LtuDqrI/fQFet5-QBgAJ
"GN: Change rtc_source_set targets --> rtc_static_library"
https://crrev.com/b62dbbe985c643cf4ee28e4c73c75bb3ef5e4d54

comment:27 Changed 11 months ago by boklm

Cc: boklm added

comment:28 Changed 11 months ago by arthuredelstein

Cc: arthuredelstein added

comment:29 Changed 6 months ago by adrelanos

Cc: adrelanos added

comment:30 in reply to:  26 Changed 6 months ago by dcf

Replying to dcf:

GN migration

https://webrtc.org/native-code/development/ these days says "GYP is no longer supported and likely to break soon. You’re encouraged to migrate to GN as soon as possible."

I pushed a first try at migrating to GN here:

https://gitweb.torproject.org/user/dcf/tor-browser-bundle.git/log/?h=snowflake-gn&id=2bb19e4c15446faf4ae8ac36e9d34847d80b4d8b

The webrtc build works, however it's missing some symbols required by go-webrtc, such as FakeAudioCaptureModule, which aren't accounted for yet in the upstream GN build files. It seems that GN support for webrtc is not complete yet, as this message from August 10, 2016:

You seem to be setting the GN variables. WebRTC still uses GYP by default which may explain why you don't see any change (we're on our way to migrate to GN though).

My plan is to forget about the GN migration until the next time we update the webrtc upstream.

At comment:3:ticket:21748, arlolra posted a branch that among other things uses GN rather than GYP. Does that mean the GN migration is finished, or is there more to do on it?

https://github.com/arlolra/tor-browser-bundle/commits/gn

comment:31 Changed 6 months ago by arlolra

Does that mean the GN migration is finished, or is there more to do on it?

That branch was merged. There's nothing more to do. Well, feel free to double-check the work I did, and suggest improvements. Thanks.

comment:32 Changed 3 months ago by dcf

Status: newneeds_review

mac reproducible build

I've ported the mac build to the GN build system and solved some reproducibility problems. For the first time, I got two consecutive identical working mac builds. I would like someone to please try building

https://gitweb.torproject.org/user/dcf/tor-browser-bundle.git/log/?h=snowflake&id=e084e834184d5ff61aef4c7f172ec883e266bdf7

and comparing the sha256sums to

https://people.torproject.org/~dcf/pt-bundle/snowflake/20170630-7.5a1-e084e834184d/

Here is the cumulative diff:

https://gitweb.torproject.org/user/dcf/tor-browser-bundle.git/diff/?h=snowflake&id=e084e834184d5ff61aef4c7f172ec883e266bdf7&id2=36808fe250f4c1de115fc200e1eb9294cbcdc2c0

This is roughly the procedure to build. For more details, see doc/Snowflake#IntegrationwithTorBrowser.

$ git clone https://git.torproject.org/builders/tor-browser-bundle.git
$ cd tor-browser-bundle
tor-browser-bundle$ git remote add dcf https://git.torproject.org/user/dcf/tor-browser-bundle.git
tor-browser-bundle$ git fetch dcf
tor-browser-bundle$ git checkout -b snowflake --track dcf/snowflake
tor-browser-bundle$ git checkout e084e834184d5ff61aef4c7f172ec883e266bdf7
tor-browser-bundle$ make clean
tor-browser-bundle$ ./mkbundle-mac.sh versions.alpha

I'm also using two locally uncommitted patches, which you may or may not need. The first applies to gitian-builder and works around #22467. The second applies to tor-browser-bundle and works around #20757. These patches are not specific to snowflake at all; I need them whenever I build Tor Browser.

diff --git a/target-bin/upgrade-system.sh b/target-bin/upgrade-system.sh
index 9384229..795c3b9 100644
--- a/target-bin/upgrade-system.sh
+++ b/target-bin/upgrade-system.sh
@@ -6,6 +6,9 @@ set -e

 mkdir -p /var/cache/gitian

+DEBIAN_FRONTEND=noninteractive apt-get -y install grub
+DEBIAN_FRONTEND=noninteractive apt-get -y install linux-image-$(uname -r)
+
 # remove obsolete grub, it causes package dependency issues
 apt-get -q -y purge grub > /dev/null 2>&1 || true

diff --git a/gitian/git-gpg-wrapper b/gitian/git-gpg-wrapper
index f137d6d4..d3dcdf2c 100755
--- a/gitian/git-gpg-wrapper
+++ b/gitian/git-gpg-wrapper
@@ -3,10 +3,10 @@
 # an expired key.
 # https://bugs.torproject.org/19737
 set -e
-if [ $# -eq 4 ] && [ "$1" = '--status-fd=1' ] \
-        && [ "$2" = '--verify' ]
+if [ $# -eq 5 ] && [ "$1" = '--status-fd=1' ] \
+        && [ "$3" = '--verify' ]
 then
-    gpgv "$1" "$3" "$4" | sed 's/^\[GNUPG:\] EXPKEYSIG /\[GNUPG:\] GOODSIG /'
+    gpgv "$1" "$4" "$5" | sed 's/^\[GNUPG:\] EXPKEYSIG /\[GNUPG:\] GOODSIG /'
     exit ${PIPESTATUS[0]}
 else
     exec gpg "$@"

The hacks needed to get the mac version to cross-compile, and build reproducibly, are not terrible--definitely not as bad as they were back in comment:15. Building with Clang and GN helped a lot. Of the 8 small patches applied to the webrtc source code, 4 of them have to do with our use of the 10.7 macOS SDK (instead of a more recent SDK).

The only potentially sketchy patch, which I invite comment on, disables a call to a function that is not present in the 10.7 SDK. The function is not called on non-mac platforms, so it seems safe, but I am not sure.

tl;dr: please try building e084e834184d5ff61aef4c7f172ec883e266bdf7 and check it against these sha256sums.

comment:34 Changed 3 months ago by dcf

Status: needs_reviewnew

Thank you for testing.

I made #22831 requesting a merge of the mac commits.

comment:35 Changed 2 months ago by dcf

I made a post on the discuss-webrtc list explaining our process for cross-compiling libwebrtc for mac.

https://groups.google.com/d/msg/discuss-webrtc/ca_R9ocEHus/LWhtmhhPAwAJ

comment:36 Changed 3 weeks ago by dcf

windows reproducible build

I pushed some preliminary code for getting started with a windows reproducible build.

https://gitweb.torproject.org/user/dcf/tor-browser-bundle.git/log/?h=snowflake-windows&id=0f48dc9e1904fa0643576fc8dcf3db50a4d2f959

It doesn't work yet; after ./mkbundle-windows.sh versions.alpha, it eventually fails with:

+ /home/ubuntu/build/webrtc/src/out_bootstrap/gn gen out/Release '--args= target_os="win" target_cpu="x86" is_debug=false treat_warnings_as_errors=false is_component_build=false is_clang=true use_sysroot=false clang_use_chrome_plugins=false clang_base_path="/home/ubuntu/build/clang" rtc_include_opus=false rtc_include_ilbc=false rtc_include_internal_audio_device=false rtc_include_tests=true'
ERROR at //build/config/BUILDCONFIG.gn:239:3: Assertion failed.
  assert(target_os == host_os, "Win cross-compiles only work on win hosts.")
  ^-----
Win cross-compiles only work on win hosts.

But at this point one can ssh into the build VM and start debugging the build failures.

Note: See TracTickets for help on using tickets.