Windows reproducible build of snowflake

added TorBrowserTeam201909R anti-censorship-roadmap-august component::circumvention/snowflake owner::cohosh parent::19001 priority::high resolution::fixed severity::normal sponsor::28-must status::closed type::project labels

Trac:
Cc: N/A to dcf
Component: - Select a component to Obfuscation/Snowflake

Trac:
Parent: N/A to #19001 (moved)

Trac:
Username: cmm323
Cc: dcf to dcf, hooman@icsi.berkeley.edu

Trac:
Username: cmm323
Cc: dcf, hooman@icsi.berkeley.edu to dcf

Trac:
Cc: dcf to dcf, cmm323

Replying to arlolra:

Breaking this out from https://trac.torproject.org/projects/tor/ticket/19001#comment:36, where dcf wrote,

I pushed some preliminary code for getting started with a windows reproducible build. https://gitweb.torproject.org/user/dcf/tor-browser-bundle.git/log/?h=snowflake-windows&id=0f48dc9e1904fa0643576fc8dcf3db50a4d2f959

It doesn't work yet; after ./mkbundle-windows.sh versions.alpha, it eventually fails with: {{{

/home/ubuntu/build/webrtc/src/out_bootstrap/gn gen out/Release '--args= target_os="win" target_cpu="x86" is_debug=false treat_warnings_as_errors=false is_component_build=false is_clang=true use_sysroot=false clang_use_chrome_plugins=false clang_base_path="/home/ubuntu/build/clang" rtc_include_opus=false rtc_include_ilbc=false rtc_include_internal_audio_device=false rtc_include_tests=true' ERROR at //build/config/BUILDCONFIG.gn:239:3: Assertion failed. assert(target_os == host_os, "Win cross-compiles only work on win hosts.") ^----- Win cross-compiles only work on win hosts. }}} But at this point one can ssh into the build VM and start debugging the build failures.

Some further notes from what we've discovered so far.

https://chromium.googlesource.com/chromium/src/+/master/docs/win_cross.md says that we need to add target_os = ['win'] to our .gclient file before calling gclient sync to get some additional deps.

https://chromium.googlesource.com/chromium/tools/depot_tools.git/+/master/win_toolchain/README.md says that Googlers have access to the necessary toolchain, but we don't.

Ugh. So, you need to use clang to cross-compile the webrtc parts? We want to have this, too, for Firefox as Stylo needs that but we won't go the road Google takes but use a clang/mingw-w64 approach. See: https://bugzilla.mozilla.org/show_bug.cgi?id=1390583 where we basically got this working.

I am not thrilled to have two different clang-based approaches in our build setup for Windows. That said, what are the options for not having to pull in all the Chromium stuff to build the WebRTC parts we need? I can't imagine that everyone wanting to have that lib is always building the whole Chromium thing.

Trac:
Cc: dcf, cmm323 to dcf, cmm323, gk

I'm not sure what the issue is, but you definitely don't need to build the entire Chromium, you just fetch WebRTC with: fetch --nohooks webrtc

See https://webrtc.org/native-code/development/

Also: https://github.com/vsimon/webrtcbuilds

Trac:
Username: cmm323

Maybe this is already known to people, but a future version of Chromium is going to use Clang instead of Microsoft's compiler, for building on Windows.

http://blog.llvm.org/2018/03/clang-is-now-used-to-build-chrome-for.html but: "We still use Microsoft’s headers and libraries to build Chrome, we still use some SDK binaries like midl.exe and mc.exe" https://arstechnica.com/gadgets/2018/03/chrome-on-windows-ditches-microsofts-compiler-now-uses-clang/

Trac:
Type: defect to project

Firefox parent bug for issues with building under MinGW

#139301 --enable-webrtc does not build under MinGW

I am poking around at upgrading go-webrtc to branch-heads/64 of WebRTC, in order to see whether the new Clang support helps with cross compiling. I happened to notice this promising change: https://chromium.googlesource.com/chromium/src/build/+/f021cd07894bad5e01b461872bd341d3fa8634d2

 } else if (target_os == "win") {
   # On Windows we use the same toolchain for host and target by default.
-  assert(target_os == host_os, "Win cross-compiles only work on win hosts.")
+  # Beware, win cross builds mostly don't work yet, see docs/win_cross.md
+  # TODO(thakis): Allow on Mac as well soon.
+  assert(host_os != "mac", "https://crbug.com/774209")

The ticket it's related to has activity three days ago: #495204 Make it possible to build chrome/win on linux, using clang-cl and gn

Here is the win_cross.md file referred to (just five months old): https://chromium.googlesource.com/chromium/src/+/HEAD/docs/win_cross.md (permalink)

Edit: I just realized arlolra basically already said this in the ticket description... Sorry.

FWIW: I am in contact with thakis who managed to get this work done. If you have any questions I should pass to them let me know. See: https://groups.google.com/a/chromium.org/forum/#!msg/chromium-dev/cIA9fBb9vBE/73cOvZu9BQAJ for the official announcement and further useful information.

Trac:
Cc: dcf, cmm323, gk to dcf, cmm323, gk, boklm

steps to reproduce will follow

From https://developer.microsoft.com/en-us/microsoft-edge/tools/vms/ Choose "MSEdge on Win10 (x64) Stable" for "VirtualBox"

Install Visual Studio Community from https://www.visualstudio.com/free-developer-offers/ While installing, from the Workloads, choose "Universal Windows Platform development", for the Windows 10 SDK, and "Desktop development with C++" for DIA SDK

Install git for Windows from https://git-scm.com/downloads for git bash

Install python 2.7 for Windows from https://www.python.org/downloads/windows/

Open Git Bash and put python in the path

$ echo "export PATH=/c/Python27:$PATH" > ~/.bash_profile
$ source ~/.bash_profile

Clone depot_tools with git clone https://chromium.googlesource.com/chromium/tools/depot_tools.git

And then run the packager,

$ cd depot_tools/win_toolchain/
$ python package_from_installed.py -w 10 2017

I've made progress with cross-compiling webrtc for windows-x86_64. It's in two branches. The first branch just upgrades webrtc to branch-heads/64, which has the cross-compiling support. The second branch builds off the first and adds a build script for windows.

The build is currently using the prebuilt clang cross-compiler that Google puts among the dependencies of webrtc. Eventually we will have to make it use our own compiler instead, as we already do on mac and linux. Reliance on the prebuilt compiler is the reason why the build currently only works for windows-x86_64, not windows-i686: there's no 32-bit prebuilt compiler available.

https://github.com/uumaro/tor-browser-build/commits/webrtc-64 (currently c8048d746c):: This branch updates the build scripts to use a webrtc based on branch-heads/64. It depends on having a go-webrtc that also supports the branch-heads/64 API. That code hasn't been merged into go-webrtc yet, but there is a pull request for it. To test the code before the go-webrtc changes have been merged, do this in the tor-browser-build directory before make: {{{ cd git_clones/go-webrtc/ git fetch https://github.com/uumaro/go-webrtc/ branch-heads-64 cd .. }}} This branch can be merged independently of the windows code, once the go-webrtc pull request goes through. https://github.com/uumaro/tor-browser-build/commits/webrtc-win (currently 08613622f6):: This branch lets the webrtc build work for windows-i686. To test it, first git fetch an updated go-webrtc as described above. If have done a build in this directory previously, run {{{ rm gclient/webrtc/.gclient }}} The .gclient will be regenerated with the addition of target_os = ['win'], which will cause gclient sync to pull the prebuilt clang cross-compiler. If you have not yet done a build in this directory, then you will need to do {{{ make submodule-update }}} Then run {{{ rbm/rbm build webrtc --target testbuild --target torbrowser-windows-x86_64 }}} If you want to test that other targets still work, do {{{ make testbuild }}}

The output of the build is a file, out/webrtc/webrtc-88f5d9180eae78a6162cccd78850ff416eb82483-windows-x86_64-ba8356.tar.gz. You can compare it against mine: https://people.torproject.org/~dcf/pt-bundle/snowflake/webrtc-88f5d9180eae78a6162cccd78850ff416eb82483-windows-x86_64-ba8356.tar.gz

A few comments on the second branch:

I compiled my own Windows SDK using the instructions in comment:13 and stashed it in my people.torproject.org space. I have a todo to check if we really need this separate SDK, or if the header files in mingw-w64 are enough.
The upstream cross-compiling instructions say to put the Windows SDK in a case-insensitive ciopfs mount, so that, for example, code can #include <winsock2.h> when the file is actually called WinSock2.h. I tried that initially, but I couldn't get ciopfs to work inside the container. boklm gave me a hint that I haven't tried yet. In the meantime, though, I disabled ciopfs and worked around case sensitivity with a lot of symlinks. I left it in the branch history for now.

I haven't yet tried actually doing anything with the generated libwebrtc-windows-amd64-magic.lib.

I'm not asking for any of this code to be merged right now. But I would appreciate some review of the code (this is my first time writing for rbm), and a check to see if the commands above work for you.

Trac:
Status: new to needs_review

Replying to dcf:

This branch lets the webrtc build work for windows-i686. To test it, first get fetch an updated go-webrtc as described above. If have done a build in this directory previously, run {{{ rm gclient/webrtc/.gclient }}} The .gclient will be regenerated with the addition of target_os = ['win'], which will cause gclient sync to pull the prebuilt clang cross-compiler.

I think we could add something like this, to automatically remove gclient/webrtc/src/testing/{gmock,gtest}: https://gitweb.torproject.org/user/boklm/tor-browser-build.git/commit/?h=bug_25483_v0&id=d5514f7f3b566cf37bfc59ec8105a96130577ff2

And this to remove the .gclient file if it does not contain the target_os line: https://gitweb.torproject.org/user/boklm/tor-browser-build.git/commit/?h=bug_25483_v0&id=60a0b31b389fa2ca47dfbe305f9f9211b58f509e

The output of the build is a file, out/webrtc/webrtc-88f5d9180eae78a6162cccd78850ff416eb82483-windows-x86_64-ba8356.tar.gz. You can compare it against mine: https://people.torproject.org/~dcf/pt-bundle/snowflake/webrtc-88f5d9180eae78a6162cccd78850ff416eb82483-windows-x86_64-ba8356.tar.gz

I have been able to do a build, and I get a tarball containing the same files, except that my version also includes some files in the directory webrtc/include/chromium/: https://people.torproject.org/~boklm/tmp/webrtc-88f5d9180eae78a6162cccd78850ff416eb82483-windows-x86_64-ba8356.tar.gz

Replying to boklm:

I have been able to do a build, and I get a tarball containing the same files, except that my version also includes some files in the directory webrtc/include/chromium/: https://people.torproject.org/~boklm/tmp/webrtc-88f5d9180eae78a6162cccd78850ff416eb82483-windows-x86_64-ba8356.tar.gz

It seems the reason is that I have a chromium directory in gclient/webrtc/src, probably from a previous build.

Replying to boklm:

I have been able to do a build, and I get a tarball containing the same files, except that my version also includes some files in the directory webrtc/include/chromium/: https://people.torproject.org/~boklm/tmp/webrtc-88f5d9180eae78a6162cccd78850ff416eb82483-windows-x86_64-ba8356.tar.gz It seems the reason is that I have a chromium directory in gclient/webrtc/src, probably from a previous build.

Thanks for testing and for the suggestions. I suppose we should more careful in the recursive search for header files. At least the files we had in common are identical, according to diffoscope.

Trac:
Cc: dcf, cmm323, gk, boklm to dcf, cmm323, gk, boklm, t0mmy

go-webrtc merged the update to branch-heads/64. I rebased the webrtc-64 and webrtc-win branches to match. https://github.com/uumaro/tor-browser-build/commits/webrtc-64 (currently 62fe24015a):: https://github.com/uumaro/tor-browser-build/commits/webrtc-win (currently 5a369308bb)::

Trac:
Cc: dcf, cmm323, gk, boklm, t0mmy to dcf, cmm323, gk, boklm, t0mmy, brade, mcs

Trac:
Keywords: N/A deleted, TorbrowserTeam201804 added
Status: needs_review to assigned
Owner: N/A to sukhe

Trac:
Priority: Medium to High

Trac:
Owner: sukhe to sukhbir

As an update to this ticket, I have been trying to get the Windows build to work, the latest progress of which is at https://github.com/azadi/tor-browser-build/commits/go-webrtc ... it's not much and mostly builds on the work of dcf and others to try to fix the error below.

Currently, the status is that we are hitting a linking error when trying to build go-webrtc (mingw-w64) with webrtc (clang). The exact error is:

    /var/tmp/dist/mingw-w64/x86_64-w64-mingw32/include/malloc.h:183:0: note: this is the location of the previous definition
     #define alloca(x) __builtin_alloca((x))
     ^
    # github.com/keroserene/go-webrtc
    /var/tmp/dist/mingw-w64/lib/gcc/x86_64-w64-mingw32/5.4.0/../../../../x86_64-w64-mingw32/lib/../lib/libstdc++.a(cow-stdexcept.o): In function `std::string::_M_data() const':
    /var/tmp/build/gcc/x86_64-w64-mingw32/libstdc++-v3/src/c++11/../../../../gcc-5.4.0/libstdc++-v3/src/c++11/cow-stdexcept.cc:44: multiple definition of `std::logic_error::logic_error(std::logic_error const&)'
    /tmp/go-build868110800/github.com/keroserene/go-webrtc/_obj/peerconnection.cc.o:peerconnection.cc:(.text$_ZNSt11logic_errorC2ERKS_[_ZNSt11logic_errorC2ERKS_]+0x0): first defined here
    /tmp/go-build868110800/github.com/keroserene/go-webrtc/_obj/ctestenums.cc.o:ctestenums.cc:(.text+0x5a9): undefined reference to `cricket::AudioCodec::ToString() const'
    /tmp/go-build868110800/github.com/keroserene/go-webrtc/_obj/ctestenums.cc.o:ctestenums.cc:(.text+0x6de): undefined reference to `webrtc::RtpExtension::ToString() const'
    /tmp/go-build868110800/github.com/keroserene/go-webrtc/_obj/ctestenums.cc.o:ctestenums.cc:(.text+0x813): undefined reference to `cricket::DataCodec::ToString() const'
    /tmp/go-build868110800/github.com/keroserene/go-webrtc/_obj/ctestenums.cc.o:ctestenums.cc:(.text+0x8f5): undefined reference to `rtc::FatalMessage::FatalMessage(char const*, int)'
    /tmp/go-build868110800/github.com/keroserene/go-webrtc/_obj/ctestenums.cc.o:ctestenums.cc:(.text+0x96a): undefined reference to `rtc::FatalMessage::~FatalMessage()'
    /tmp/go-build868110800/github.com/keroserene/go-webrtc/_obj/ctestenums.cc.o:ctestenums.cc:(.text+0xa1b): undefined reference to `rtc::FatalMessage::~FatalMessage()'
    /tmp/go-build868110800/github.com/keroserene/go-webrtc/_obj/ctestenums.cc.o:ctestenums.cc:(.text+0xbf8): undefined reference to `rtc::FatalMessage::FatalMessage(char const*, int)'
    /tmp/go-build868110800/github.com/keroserene/go-webrtc/_obj/ctestenums.cc.o:ctestenums.cc:(.text+0xc6d): undefined reference to `rtc::FatalMessage::~FatalMessage()'
    /tmp/go-build868110800/github.com/keroserene/go-webrtc/_obj/ctestenums.cc.o:ctestenums.cc:(.text+0xd20): undefined reference to `rtc::FatalMessage::~FatalMessage()'
    /tmp/go-build868110800/github.com/keroserene/go-webrtc/_obj/ctestenums.cc.o:ctestenums.cc:(.text+0xdb1): undefined reference to `rtc::FatalMessage::FatalMessage(char const*, int)'
    /tmp/go-build868110800/github.com/keroserene/go-webrtc/_obj/ctestenums.cc.o:ctestenums.cc:(.text+0xe26): undefined reference to `rtc::FatalMessage::~FatalMessage()'
    /tmp/go-build868110800/github.com/keroserene/go-webrtc/_obj/ctestenums.cc.o:ctestenums.cc:(.text+0xed5): undefined reference to `rtc::FatalMessage::~FatalMessage()'
    /tmp/go-build868110800/github.com/keroserene/go-webrtc/_obj/ctestenums.cc.o:ctestenums.cc:(.text+0xf6d): undefined reference to `rtc::FatalMessage::FatalMessage(char const*, int)'
    /tmp/go-build868110800/github.com/keroserene/go-webrtc/_obj/ctestenums.cc.o:ctestenums.cc:(.text+0xfe2): undefined reference to `rtc::FatalMessage::~FatalMessage()'
    /tmp/go-build868110800/github.com/keroserene/go-webrtc/_obj/ctestenums.cc.o:ctestenums.cc:(.text+0x1094): undefined reference to `rtc::FatalMessage::~FatalMessage()'
    /tmp/go-build868110800/github.com/keroserene/go-webrtc/_obj/ctestenums.cc.o:ctestenums.cc:(.text+0x112c): undefined reference to `rtc::FatalMessage::FatalMessage(char const*, int)'
    /tmp/go-build868110800/github.com/keroserene/go-webrtc/_obj/ctestenums.cc.o:ctestenums.cc:(.text+0x11a1): undefined reference to `rtc::FatalMessage::~FatalMessage()'
    /tmp/go-build868110800/github.com/keroserene/go-webrtc/_obj/ctestenums.cc.o:ctestenums.cc:(.text+0x1254): undefined reference to `rtc::FatalMessage::~FatalMessage()'
    /var/tmp/dist/mingw-w64/lib/gcc/x86_64-w64-mingw32/5.4.0/../../../../x86_64-w64-mingw32/bin/ld: $WORK/github.com/keroserene/go-webrtc/_obj/ctestenums.cc.o: bad reloc address 0xc0 in section `.rdata'
    collect2: error: ld returned 1 exit status

I have tried different things but mostly focusing on the webrtc library to start with but it seems like that is not the problem since it has the missing symbols (which other than the usual symbol lookup, we also confirmed by writing a small program that calls them.) It was suggested that this can be due to C++ ABI (in)compatibility between mingw-w64 and clang but at least one mingw-w64 person I talked with disagreed with that assumption.

A post made on the discuss-webrtc Google group was also not helpful and even though there are similar (not same) posts with this problem, there seem to be no solutions that actually work; I have tried incorporating the other suggestions as well in the various threads, but to no effect.

As suggested by Hooman, I have started looking recently into using the asicerik go-webrtc (this wrapper is for native compiling on Windows using Visual Studio and not cross-compiling using Linux) but it doesn't link against webrtc currently.

A post on the mingw-w64 mailing list suggests that these are clang bugs. From https://sourceforge.net/p/mingw-w64/mailman/message/36308258/:

Those are Clang bugs [0].

Multiple definitions error has been fixed in [1] and probably (just
guessing) made it to Clang 6.0.0 release.

Undefined references bug happens when you link code compiled by Clang to
static libstdc++ (and probably other static libs too) compiled by GCC. I
think there is still no solution for this one.

[0] https://lists.llvm.org/pipermail/cfe-dev/2017-April/053530.html
[1] https://reviews.llvm.org/D33620

Move our roadmap tickets to May.

Trac:
Keywords: N/A deleted, TorBrowserTeam201805 added

Trac:
Keywords: TorbrowserTeam201804 deleted, N/A added

So I tried building with clang 6 today just to confirm the theory that one of these bugs was fixed in clang 6.0.0 (and hope the other one was as well :) and ran into the following error. I replaced the precompiled clang webrtc with clang 6.0.0 but I didn't bother to set clang_base_path and just replaced it in the default lookup directory third_party/llvm-build/Release+Asserts.

I will continue to debug this (and try with master instead of release as well) but if you have seen this before, please let me know. A quick search tells me this may be related to cross-compilation but I haven't found any solutions yet.

[188/2538] CC obj/third_party/libvpx/libvpx_intrinsics_avx/quantize_avx.obj
[189/2538] CC obj/third_party/libvpx/libvpx_intrinsics_avx2/sad4d_avx2.obj
FAILED: obj/third_party/libvpx/libvpx_intrinsics_avx2/sad4d_avx2.obj 
../../third_party/llvm-build/Release+Asserts/bin/clang-cl --rsp-quoting=posix /nologo /showIncludes  @obj/third_party/libvpx/libvpx_intrinsics_avx2/sad4d_avx2.obj.rsp /c ../../third_party/libvpx/source/libvpx/vpx_dsp/x86/sad4d_avx2.c /Foobj/third_party/libvpx/libvpx_intrinsics_avx2/sad4d_avx2.obj /Fd"obj/third_party/libvpx/libvpx_intrinsics_avx2_c.pdb"
In file included from ../../third_party/libvpx/source/libvpx/vpx_dsp/x86/sad4d_avx2.c:10:
In file included from /var/tmp/build/webrtc/src/third_party/llvm-build/Release+Asserts/lib/clang/6.0.0/include/immintrin.h:134:
/var/tmp/build/webrtc/src/third_party/llvm-build/Release+Asserts/lib/clang/6.0.0/include/fmaintrin.h(73,10):  error: invalid conversion between vector type '__m128' (vector of 4 'float' values) and integer type 'int' of different size
  return (m128)builtin_ia32_vfmsubss3((v4sf)A, (v4sf)B, (v4sf)C);
		 ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
/var/tmp/build/webrtc/src/third_party/llvm-build/Release+Asserts/lib/clang/6.0.0/include/fmaintrin.h(79,10):  error: invalid conversion between vector type '__m128d' (vector of 2 'double' values) and integer type 'int' of different size
  return (m128d)builtin_ia32_vfmsubsd3((v2df)A, (v2df)B, (v2df)C);
		 ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
/var/tmp/build/webrtc/src/third_party/llvm-build/Release+Asserts/lib/clang/6.0.0/include/fmaintrin.h(97,10):  error: invalid conversion between vector type '__m128' (vector of 4 'float' values) and integer type 'int' of different size
  return (m128)builtin_ia32_vfnmaddss3((v4sf)A, (v4sf)B, (v4sf)C);
		 ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
/var/tmp/build/webrtc/src/third_party/llvm-build/Release+Asserts/lib/clang/6.0.0/include/fmaintrin.h(103,10):  error: invalid conversion between vector type '__m128d' (vector of 2 'double' values) and integer type 'int' of different size
  return (m128d)builtin_ia32_vfnmaddsd3((v2df)A, (v2df)B, (v2df)C);
		 ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
/var/tmp/build/webrtc/src/third_party/llvm-build/Release+Asserts/lib/clang/6.0.0/include/fmaintrin.h(121,10):  error: invalid conversion between vector type '__m128' (vector of 4 'float' values) and integer type 'int' of different size
  return (m128)builtin_ia32_vfnmsubss3((v4sf)A, (v4sf)B, (v4sf)C);
		 ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
/var/tmp/build/webrtc/src/third_party/llvm-build/Release+Asserts/lib/clang/6.0.0/include/fmaintrin.h(127,10):  error: invalid conversion between vector type '__m128d' (vector of 2 'double' values) and integer type 'int' of different size
  return (m128d)builtin_ia32_vfnmsubsd3((v2df)A, (v2df)B, (v2df)C);
		 ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
Note: including file:  /var/tmp/build/webrtc/src/third_party/llvm-build/Release+Asserts/lib/clang/6.0.0/include/fxsrintrin.h
6 errors generated.
[190/2538] CC obj/third_party/libvpx/libvpx_intrinsics_avx2/fwd_txfm_avx2.obj
FAILED: obj/third_party/libvpx/libvpx_intrinsics_avx2/fwd_txfm_avx2.obj 
../../third_party/llvm-build/Release+Asserts/bin/clang-cl --rsp-quoting=posix /nologo /showIncludes  @obj/third_party/libvpx/libvpx_intrinsics_avx2/fwd_txfm_avx2.obj.rsp /c ../../third_party/libvpx/source/libvpx/vpx_dsp/x86/fwd_txfm_avx2.c /Foobj/third_party/libvpx/libvpx_intrinsics_avx2/fwd_txfm_avx2.obj /Fd"obj/third_party/libvpx/libvpx_intrinsics_avx2_c.pdb"
In file included from ../../third_party/libvpx/source/libvpx/vpx_dsp/x86/fwd_txfm_avx2.c:15:
In file included from ../../third_party/libvpx/source/libvpx/vpx_dsp/x86/fwd_dct32x32_impl_avx2.h:11:
In file included from /var/tmp/build/webrtc/src/third_party/llvm-build/Release+Asserts/lib/clang/6.0.0/include/immintrin.h:134:
/var/tmp/build/webrtc/src/third_party/llvm-build/Release+Asserts/lib/clang/6.0.0/include/fmaintrin.h(73,10):  error: invalid conversion between vector type '__m128' (vector of 4 'float' values) and integer type 'int' of different size
  return (m128)builtin_ia32_vfmsubss3((v4sf)A, (v4sf)B, (v4sf)C);
		 ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
/var/tmp/build/webrtc/src/third_party/llvm-build/Release+Asserts/lib/clang/6.0.0/include/fmaintrin.h(79,10):  error: invalid conversion between vector type '__m128d' (vector of 2 'double' values) and integer type 'int' of different size
  return (m128d)builtin_ia32_vfmsubsd3((v2df)A, (v2df)B, (v2df)C);
		 ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
/var/tmp/build/webrtc/src/third_party/llvm-build/Release+Asserts/lib/clang/6.0.0/include/fmaintrin.h(97,10):  error: invalid conversion between vector type '__m128' (vector of 4 'float' values) and integer type 'int' of different size
  return (m128)builtin_ia32_vfnmaddss3((v4sf)A, (v4sf)B, (v4sf)C);
		 ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
/var/tmp/build/webrtc/src/third_party/llvm-build/Release+Asserts/lib/clang/6.0.0/include/fmaintrin.h(103,10):  error: invalid conversion between vector type '__m128d' (vector of 2 'double' values) and integer type 'int' of different size
  return (m128d)builtin_ia32_vfnmaddsd3((v2df)A, (v2df)B, (v2df)C);
		 ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
/var/tmp/build/webrtc/src/third_party/llvm-build/Release+Asserts/lib/clang/6.0.0/include/fmaintrin.h(121,10):  error: invalid conversion between vector type '__m128' (vector of 4 'float' values) and integer type 'int' of different size
  return (m128)builtin_ia32_vfnmsubss3((v4sf)A, (v4sf)B, (v4sf)C);
		 ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
/var/tmp/build/webrtc/src/third_party/llvm-build/Release+Asserts/lib/clang/6.0.0/include/fmaintrin.h(127,10):  error: invalid conversion between vector type '__m128d' (vector of 2 'double' values) and integer type 'int' of different size
  return (m128d)builtin_ia32_vfnmsubsd3((v2df)A, (v2df)B, (v2df)C);
		 ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
Note: including file:   /var/tmp/build/webrtc/src/third_party/llvm-build/Release+Asserts/lib/clang/6.0.0/include/fxsrintrin.h
6 errors generated.
[191/2538] CC obj/third_party/libvpx/libvpx_intrinsics_avx/vp9_diamond_search_sad_avx.obj
ninja: build stopped: subcommand failed.

OK, I managed to resolve comment:28 (by properly linking), and unfortunately, the same error.

/var/tmp/dist/mingw-w64/lib/gcc/x86_64-w64-mingw32/5.4.0/../../../../x86_64-w64-mingw32/lib/../lib/libstdc++.a(cow-stdexcept.o): In function `std::string::_M_data() const':
/var/tmp/build/gcc/x86_64-w64-mingw32/libstdc++-v3/src/c++11/../../../../gcc-5.4.0/libstdc++-v3/src/c++11/cow-stdexcept.cc:44: multiple definition of `std::logic_error::logic_error(std::logic_error const&)'
/tmp/go-build842481917/github.com/keroserene/go-webrtc/_obj/peerconnection.cc.o:peerconnection.cc:(.text$_ZNSt11logic_errorC2ERKS_[_ZNSt11logic_errorC2ERKS_]+0x0): first defined here
/tmp/go-build842481917/github.com/keroserene/go-webrtc/_obj/ctestenums.cc.o:ctestenums.cc:(.text+0x5a9): undefined reference to `cricket::AudioCodec::ToString() const'
/tmp/go-build842481917/github.com/keroserene/go-webrtc/_obj/ctestenums.cc.o:ctestenums.cc:(.text+0x6de): undefined reference to `webrtc::RtpExtension::ToString() const'
/tmp/go-build842481917/github.com/keroserene/go-webrtc/_obj/ctestenums.cc.o:ctestenums.cc:(.text+0x813): undefined reference to `cricket::DataCodec::ToString() const'
/tmp/go-build842481917/github.com/keroserene/go-webrtc/_obj/ctestenums.cc.o:ctestenums.cc:(.text+0x8f5): undefined reference to `rtc::FatalMessage::FatalMessage(char const*, int)'
/tmp/go-build842481917/github.com/keroserene/go-webrtc/_obj/ctestenums.cc.o:ctestenums.cc:(.text+0x96a): undefined reference to `rtc::FatalMessage::~FatalMessage()'
/tmp/go-build842481917/github.com/keroserene/go-webrtc/_obj/ctestenums.cc.o:ctestenums.cc:(.text+0xa1b): undefined reference to `rtc::FatalMessage::~FatalMessage()'
/tmp/go-build842481917/github.com/keroserene/go-webrtc/_obj/ctestenums.cc.o:ctestenums.cc:(.text+0xbf8): undefined reference to `rtc::FatalMessage::FatalMessage(char const*, int)'
/tmp/go-build842481917/github.com/keroserene/go-webrtc/_obj/ctestenums.cc.o:ctestenums.cc:(.text+0xc6d): undefined reference to `rtc::FatalMessage::~FatalMessage()'
/tmp/go-build842481917/github.com/keroserene/go-webrtc/_obj/ctestenums.cc.o:ctestenums.cc:(.text+0xd20): undefined reference to `rtc::FatalMessage::~FatalMessage()'
/tmp/go-build842481917/github.com/keroserene/go-webrtc/_obj/ctestenums.cc.o:ctestenums.cc:(.text+0xdb1): undefined reference to `rtc::FatalMessage::FatalMessage(char const*, int)'
/tmp/go-build842481917/github.com/keroserene/go-webrtc/_obj/ctestenums.cc.o:ctestenums.cc:(.text+0xe26): undefined reference to `rtc::FatalMessage::~FatalMessage()'
/tmp/go-build842481917/github.com/keroserene/go-webrtc/_obj/ctestenums.cc.o:ctestenums.cc:(.text+0xed5): undefined reference to `rtc::FatalMessage::~FatalMessage()'
/tmp/go-build842481917/github.com/keroserene/go-webrtc/_obj/ctestenums.cc.o:ctestenums.cc:(.text+0xf6d): undefined reference to `rtc::FatalMessage::FatalMessage(char const*, int)'
/tmp/go-build842481917/github.com/keroserene/go-webrtc/_obj/ctestenums.cc.o:ctestenums.cc:(.text+0xfe2): undefined reference to `rtc::FatalMessage::~FatalMessage()'
/tmp/go-build842481917/github.com/keroserene/go-webrtc/_obj/ctestenums.cc.o:ctestenums.cc:(.text+0x1094): undefined reference to `rtc::FatalMessage::~FatalMessage()'
/tmp/go-build842481917/github.com/keroserene/go-webrtc/_obj/ctestenums.cc.o:ctestenums.cc:(.text+0x112c): undefined reference to `rtc::FatalMessage::FatalMessage(char const*, int)'
/tmp/go-build842481917/github.com/keroserene/go-webrtc/_obj/ctestenums.cc.o:ctestenums.cc:(.text+0x11a1): undefined reference to `rtc::FatalMessage::~FatalMessage()'
/tmp/go-build842481917/github.com/keroserene/go-webrtc/_obj/ctestenums.cc.o:ctestenums.cc:(.text+0x1254): undefined reference to `rtc::FatalMessage::~FatalMessage()'
/var/tmp/dist/mingw-w64/lib/gcc/x86_64-w64-mingw32/5.4.0/../../../../x86_64-w64-mingw32/bin/ld: $WORK/github.com/keroserene/go-webrtc/_obj/ctestenums.cc.o: bad reloc address 0xc0 in section `.rdata'

This was with clang 6.0.0; I will check with master later.

Trac:
Cc: dcf, cmm323, gk, boklm, t0mmy, brade, mcs to dcf, cmm323, gk, boklm, t0mmy, brade, mcs, tom

Instructions on how to build and reproduce the error: clone https://github.com/azadi/tor-browser-build/tree/go-webrtc. Run make submodule-update followed by rbm/rbm build go-webrtc --target testbuild --target torbrowser-windows-x86_64; this first builds webrtc and then tries to link it with go-webrtc, at which point you will see an error.

As for the clang suggestions above from the mailing list, I couldn't find the changes in the changelog and as mentioned above, building with the latest version didn't help.

Replying to sukhbir:

Instructions on how to build and reproduce the error: clone https://github.com/azadi/tor-browser-build/tree/go-webrtc. Run make submodule-update followed by rbm/rbm build go-webrtc --target testbuild --target torbrowser-windows-x86_64; this first builds webrtc and then tries to link it with go-webrtc, at which point you will see an error.

I did not even get that far. The build is breaking earlier for me following your steps (during the webrtc build). The log ends with:

ln: failed to create symbolic link 'windows.web.http.filters.h': File exists
ln: failed to create symbolic link 'windows.web.http.h': File exists
ln: failed to create symbolic link 'windows.web.http.headers.h': File exists
ln: failed to create symbolic link 'windows.web.syndication.h': File exists
ln: failed to create symbolic link 'winstring.h': File exists
ln: failed to create symbolic link 'wrl.h': File exists
ERROR at //modules/video_capture/BUILD.gn:161:17: Can't load input file.
      deps += [ "//third_party/winsdk_samples" ]
                ^-----------------------------
Unable to load:
  /var/tmp/build/webrtc/src/third_party/winsdk_samples/BUILD.gn
I also checked in the secondary tree for:
  /var/tmp/build/webrtc/src/build/secondary/third_party/winsdk_samples/BUILD.gn

What am I missing?

FWIW: I am at commit c4e3596639091e484410b0622eb4b3a08e83d01e of your branch in case you were wondering.

Actually, that's the second error I get. The first one was

Starting build: Sat May 19 21:29:47 2018
Bootstrapping cipd client for linux-amd64 from https://chrome-infra-packages.appspot.com/client?platform=linux-amd64&version=git_revision:ae28364c740acff97ae118adcb2808b6cb5129c5...

src/testing (ERROR)
----------------------------------------
[0:00:23] Started.
----------------------------------------
Error: Command 'git checkout --quiet 60c665fffe7dc505fdd5d30f9dbcbc50dde1e017' returned non-zero exit status 1 in /home/gk/tor-browser-build/gclient/webrtc/src/testing
error: The following untracked working tree files would be overwritten by checkout:
	gmock/include/gmock/gmock-actions.h
	gmock/include/gmock/gmock-generated-function-mockers.h
	gmock/include/gmock/gmock-matchers.h
	gmock/include/gmock/gmock.h
	gtest/include/gtest/gtest-death-test.h
	gtest/include/gtest/gtest-message.h
	gtest/include/gtest/gtest-param-test.h
	gtest/include/gtest/gtest-spi.h
	gtest/include/gtest/gtest.h
	gtest/include/gtest/gtest_prod.h
Please move or remove them before you switch branches.
Aborting

That's reproducible on two different build machines for me. I worked around both errors on one of the machines my just getting rid of everything webrtc related and starting over in this regard. We should not need to do this, though. I therefore keep the second machine as-is for debugging purposes.

I tried building c4e3596 again and it worked for me. I agree that you should not have to remove everything related to webrtc and I will check that.

The checkout error above suggests that this may have been a network issue; can you try building again? If that doesn't work, I will look why existing builds of webrtc interfere with this one (basing it on the initial state of the tor-browser-build directory to the same as yours).

Replying to gk:

That's reproducible on two different build machines for me. I worked around both errors on one of the machines my just getting rid of everything webrtc related and starting over in this regard. We should not need to do this, though. I therefore keep the second machine as-is for debugging purposes.

Sorry, did you mean the errors you have are reproducible or the error I mentioned in #comment:24?

Replying to sukhbir:

Replying to gk:

That's reproducible on two different build machines for me. I worked around both errors on one of the machines my just getting rid of everything webrtc related and starting over in this regard. We should not need to do this, though. I therefore keep the second machine as-is for debugging purposes.

Sorry, did you mean the errors you have are reproducible or the error I mentioned in #comment:24?

I can reproduce the error in comment:34 on two different machines and that's what I meant.

Over the weekend I tried to look more closely at it. So, updating to mingw-w64 master and GCC 6.4.0 does not help. Then I tried to build clang master and see whether we can verify the claim in comment:25 (and maybe be lucky because our bugs got fixed like last week :) ).

I tried for a while to get the clang build going by setting LLVM_FORCE_HEAD_REVISION to 1. It turns out that breaks the clang plugin compilation and other clang tools' compilation as well. Not compiling those gives a proper clang based on master, unfortunately requires the WebRTC build those missing clang tools.

I then settled to use the latest prebuilt clang binaries (by messing with the update.py script (see: https://chromium.googlesource.com/chromium/src/tools/clang/+/master/scripts/update.py)) using rev 332335 which is just ca. 400 commit behind the latest one. I am still getting the same errors as in comment:24.

I guess it's time to look at the symbols...

Cross compiling WebRTC library for Windows on a Linux machine and linking Snowflake involves multiple steps:

1-Cross compiling WebRTC lib: This step is currently done using Clang, which is Microsoft Visual C++ (MSVC) compatible[1]. 2-Cross compiling Snowflake code: This step is currently done using MingW because Cgo (which integrates Golang with C) does not support Clang for Windows[2]. 3-Linking Snowflake binary against WebRTC lib: This part is problematic. First, Cgo currently doesn't support MSVC object files [3] and MingW and Clang/MSVC object files are not compatible[4]. An attempt to link these two using MingW leads to compiler errors undefined reference to mentioned in[comment:29].

[1]- https://clang.llvm.org/docs/MSVCCompatibility.html [2]- https://github.com/golang/go/issues/17014 [3]- https://github.com/golang/go/issues/20982 [4]- http://www.mingw.org/wiki/MixingCompilers

Trac:
Username: cmm323

The problem in step 3 is that the name mangling schemes used by the two compilers are different and thus the linker cannot find the appropriate function in the object file generated by the other compiler. However, this is a common problem and a popular solution is to export method calls as C functions. Since C doesn't have namespaces there is no name mangling issue. These functions can be exported as a DLL wrapper (see [1] and [2] for more information). Go-Webrtc wrapper by asicerik[3] wraps WebRTC functions called by Snowflake, in C functions and export them as a runtime DLL. I have tested the functionality of this wrapper on Windows. Cross-compiling the wrapper on a Linux machine, however, seems to be harder. The wrapper uses Windows header and some other WebRTC headers. I believe that we can use the Ninja[4] and GYP build systems that are used to build the WebRTC library to cross-compile the wrapper as well. I think the solution is to add a target to Ninja build system to build the wrapper with WebRTC library as a dependency. Unfortunately, I don't know enough about Ninja build system to do that and won't have the time to research it anytime soon.

[1]- http://www.mingw.org/wiki/Visual_Basic_DLL [2]- https://www.transmissionzero.co.uk/computing/building-dlls-with-mingw/ [3]- https://github.com/asicerik/go-webrtc [4]- https://ninja-build.org/manual.html

Trac:
Username: cmm323

Given the roadmap for Tor Browser, switching to mingw-clang sometime in the next 4-ish months - is a reasonable path forward for this to get it built with mingw-clang?

Replying to tom:

Given the roadmap for Tor Browser, switching to mingw-clang sometime in the next 4-ish months - is a reasonable path forward for this to get it built with mingw-clang?

Yes, building webrtc with mingw-clang is totally reasonable.

I think it would be reasonable even if Tor Browser were not switching to it, but as they are, it makes it even better.

Trac:
Sponsor: N/A to Sponsor19

Replying to dcf:

Yes, building webrtc with mingw-clang is totally reasonable.

That's an interesting idea. How easy is this? Building WebRTC with mingw-clang can be tested independently from the Tor Browser part, right? If so, should we give it a try?

Trac:
Username: cmm323

Replying to cmm323:

Replying to dcf:

Yes, building webrtc with mingw-clang is totally reasonable.

That's an interesting idea. How easy is this? Building WebRTC with mingw-clang can be tested independently from the Tor Browser part, right? If so, should we give it a try?

Building the toolchain is complicated; but you can download one below and just give it a shot and see how it goes. I'd expect it to error and not work right out of the box, but that's because nothing really is set up to handle mingw-clang right now. So figure spending at least a full day tinkering with it to fix header guards and whatnot, but hopefully you'll get something working!

Alternately, if I could get a Firefox building with --enable-webrtc with this toolchain: would that indicate that it was at least possible to do what you need to do? Trying that wouldn't be that difficult for me, but trying to work on a new project I am unfamiliar with would be more difficult.

x64: https://index.taskcluster.net/v1/task/gecko.cache.level-3.toolchains.v2.linux64-clang-trunk-mingw-x64.latest/artifacts/public/build/clangmingw.tar.xz

x86: https://index.taskcluster.net/v1/task/gecko.cache.level-3.toolchains.v2.linux64-clang-trunk-mingw-x86.latest/artifacts/public/build/clangmingw.tar.xz

Replying to tom:

Alternately, if I could get a Firefox building with --enable-webrtc with this toolchain: would that indicate that it was at least possible to do what you need to do? Trying that wouldn't be that difficult for me, but trying to work on a new project I am unfamiliar with would be more difficult.

If it's easy for you to do this, please do, especially if it results in a libwebrtc.a as a side-effect build artifact. Then we can see if go-webrtc easily links with it.

Replying to cmm323:

The problem in step 3 is that the name mangling schemes used by the two compilers are different and thus the linker cannot find the appropriate function in the object file generated by the other compiler. However, this is a common problem and a popular solution is to export method calls as C functions. Since C doesn't have namespaces there is no name mangling issue. These functions can be exported as a DLL wrapper (see [1] and [2] for more information). Go-Webrtc wrapper by asicerik[3] wraps WebRTC functions called by Snowflake, in C functions and export them as a runtime DLL. I have tested the functionality of this wrapper on Windows. Cross-compiling the wrapper on a Linux machine, however, seems to be harder. The wrapper uses Windows header and some other WebRTC headers. I believe that we can use the Ninja[4] and GYP build systems that are used to build the WebRTC library to cross-compile the wrapper as well.

So, this means we could start integrating the pre-compiled wrapper and trying to get something going with that one while we think in the mean time about how to fix the cross-compilation of the wrapper? Or did I get that wrong?

Trac:
Owner: sukhbir to cohosh
Status: assigned to accepted

Replying to dcf:

Replying to tom:

Alternately, if I could get a Firefox building with --enable-webrtc with this toolchain: would that indicate that it was at least possible to do what you need to do? Trying that wouldn't be that difficult for me, but trying to work on a new project I am unfamiliar with would be more difficult.

If it's easy for you to do this, please do, especially if it results in a libwebrtc.a as a side-effect build artifact. Then we can see if go-webrtc easily links with it. Just picking this up and looking into how difficult it would be to get go-webrtc to link to the Firefox webrtc API.

I built Firefox in rbm with --enable-webrtc using mingw-w64. Is there a reason to try it now with mingw-clang? (I guess we will eventually move to that with #28238 (moved)?)

Replying to cohosh:

Replying to dcf:

Replying to tom:

Alternately, if I could get a Firefox building with --enable-webrtc with this toolchain: would that indicate that it was at least possible to do what you need to do? Trying that wouldn't be that difficult for me, but trying to work on a new project I am unfamiliar with would be more difficult.

If it's easy for you to do this, please do, especially if it results in a libwebrtc.a as a side-effect build artifact. Then we can see if go-webrtc easily links with it. Just picking this up and looking into how difficult it would be to get go-webrtc to link to the Firefox webrtc API.

I built Firefox in rbm with --enable-webrtc using mingw-w64.

I am 95% sure Firefox will not build with enable-webrtc using mingw-gcc so I think you editing the wrong config file or something.

Is there a reason to try it now with mingw-clang? (I guess we will eventually move to that with #28238 (moved)?)

We will move to mingw-clang 'soon', and it will be required in esr68 so if you want your solution to work beyond Oct 2019, I would pursue that.

Windows reproducible build of snowflake

Child items ...

Activity