Opened 7 months ago

Last modified 5 days ago

#25483 assigned project

Windows reproducible build of snowflake

Reported by: arlolra Owned by: sukhbir
Priority: High Milestone:
Component: Obfuscation/Snowflake Version:
Severity: Normal Keywords: TorBrowserTeam201805
Cc: dcf, cmm323, gk, boklm, t0mmy, brade, mcs, tom Actual Points:
Parent ID: #19001 Points:
Reviewer: Sponsor: Sponsor19

Description

Breaking this out from https://trac.torproject.org/projects/tor/ticket/19001#comment:36, where dcf wrote,

I pushed some preliminary code for getting started with a windows reproducible build.

https://gitweb.torproject.org/user/dcf/tor-browser-bundle.git/log/?h=snowflake-windows&id=0f48dc9e1904fa0643576fc8dcf3db50a4d2f959

It doesn't work yet; after ./mkbundle-windows.sh versions.alpha, it eventually fails with:

+ /home/ubuntu/build/webrtc/src/out_bootstrap/gn gen out/Release '--args= target_os="win" target_cpu="x86" is_debug=false treat_warnings_as_errors=false is_component_build=false is_clang=true use_sysroot=false clang_use_chrome_plugins=false clang_base_path="/home/ubuntu/build/clang" rtc_include_opus=false rtc_include_ilbc=false rtc_include_internal_audio_device=false rtc_include_tests=true'
ERROR at //build/config/BUILDCONFIG.gn:239:3: Assertion failed.
  assert(target_os == host_os, "Win cross-compiles only work on win hosts.")
  ^-----
Win cross-compiles only work on win hosts.

But at this point one can ssh into the build VM and start debugging the build failures.

Some further notes from what we've discovered so far.

Child Tickets

Change History (42)

comment:1 Changed 7 months ago by arlolra

Cc: dcf added
Component: - Select a componentObfuscation/Snowflake

comment:2 Changed 7 months ago by arlolra

Parent ID: #19001

comment:3 Changed 7 months ago by cmm323

Cc: hooman@… added

comment:4 Changed 7 months ago by cmm323

Cc: hooman@… removed

comment:5 Changed 7 months ago by arlolra

Cc: cmm323 added

comment:6 in reply to:  description Changed 7 months ago by gk

Cc: gk added

Replying to arlolra:

Breaking this out from https://trac.torproject.org/projects/tor/ticket/19001#comment:36, where dcf wrote,

I pushed some preliminary code for getting started with a windows reproducible build.

https://gitweb.torproject.org/user/dcf/tor-browser-bundle.git/log/?h=snowflake-windows&id=0f48dc9e1904fa0643576fc8dcf3db50a4d2f959

It doesn't work yet; after ./mkbundle-windows.sh versions.alpha, it eventually fails with:

+ /home/ubuntu/build/webrtc/src/out_bootstrap/gn gen out/Release '--args= target_os="win" target_cpu="x86" is_debug=false treat_warnings_as_errors=false is_component_build=false is_clang=true use_sysroot=false clang_use_chrome_plugins=false clang_base_path="/home/ubuntu/build/clang" rtc_include_opus=false rtc_include_ilbc=false rtc_include_internal_audio_device=false rtc_include_tests=true'
ERROR at //build/config/BUILDCONFIG.gn:239:3: Assertion failed.
  assert(target_os == host_os, "Win cross-compiles only work on win hosts.")
  ^-----
Win cross-compiles only work on win hosts.

But at this point one can ssh into the build VM and start debugging the build failures.

Some further notes from what we've discovered so far.

Ugh. So, you need to use clang to cross-compile the webrtc parts? We want to have this, too, for Firefox as Stylo needs that but we won't go the road Google takes but use a clang/mingw-w64 approach. See: https://bugzilla.mozilla.org/show_bug.cgi?id=1390583 where we basically got this working.

I am not thrilled to have two different clang-based approaches in our build setup for Windows. That said, what are the options for not having to pull in all the Chromium stuff to build the WebRTC parts we need? I can't imagine that everyone wanting to have that lib is always building the whole Chromium thing.

comment:7 Changed 7 months ago by cmm323

I'm not sure what the issue is, but you definitely don't need to build the entire Chromium, you just fetch WebRTC with: fetch --nohooks webrtc

See https://webrtc.org/native-code/development/

Also: https://github.com/vsimon/webrtcbuilds

comment:8 Changed 7 months ago by dcf

Type: defectproject

Maybe this is already known to people, but a future version of Chromium is going to use Clang instead of Microsoft's compiler, for building on Windows.

http://blog.llvm.org/2018/03/clang-is-now-used-to-build-chrome-for.html

but: "We still use Microsoft’s headers and libraries to build Chrome, we still use some SDK binaries like midl.exe and mc.exe"

https://arstechnica.com/gadgets/2018/03/chrome-on-windows-ditches-microsofts-compiler-now-uses-clang/

comment:9 Changed 7 months ago by dcf

Firefox parent bug for issues with building under MinGW

#139301 --enable-webrtc does not build under MinGW

comment:10 Changed 7 months ago by dcf

I am poking around at upgrading go-webrtc to branch-heads/64 of WebRTC, in order to see whether the new Clang support helps with cross compiling. I happened to notice this promising change:
https://chromium.googlesource.com/chromium/src/build/+/f021cd07894bad5e01b461872bd341d3fa8634d2

 } else if (target_os == "win") {
   # On Windows we use the same toolchain for host and target by default.
-  assert(target_os == host_os, "Win cross-compiles only work on win hosts.")
+  # Beware, win cross builds mostly don't work yet, see docs/win_cross.md
+  # TODO(thakis): Allow on Mac as well soon.
+  assert(host_os != "mac", "https://crbug.com/774209")

The ticket it's related to has activity three days ago:
#495204 Make it possible to build chrome/win on linux, using clang-cl and gn

Here is the win_cross.md file referred to (just five months old):
https://chromium.googlesource.com/chromium/src/+/HEAD/docs/win_cross.md (permalink)

Edit: I just realized arlolra basically already said this in the ticket description... Sorry.

Last edited 7 months ago by dcf (previous) (diff)

comment:11 Changed 7 months ago by gk

FWIW: I am in contact with thakis who managed to get this work done. If you have any questions I should pass to them let me know. See: https://groups.google.com/a/chromium.org/forum/#!msg/chromium-dev/cIA9fBb9vBE/73cOvZu9BQAJ for the official announcement and further useful information.

comment:12 Changed 7 months ago by boklm

Cc: boklm added

comment:13 Changed 7 months ago by arlolra

steps to reproduce will follow

From https://developer.microsoft.com/en-us/microsoft-edge/tools/vms/
Choose "MSEdge on Win10 (x64) Stable" for "VirtualBox"

Install Visual Studio Community from https://www.visualstudio.com/free-developer-offers/
While installing, from the Workloads, choose "Universal Windows Platform development", for the Windows 10 SDK, and "Desktop development with C++" for DIA SDK

Install git for Windows from https://git-scm.com/downloads for git bash

Install python 2.7 for Windows from https://www.python.org/downloads/windows/

Open Git Bash and put python in the path

$ echo "export PATH=/c/Python27:$PATH" > ~/.bash_profile
$ source ~/.bash_profile

Clone depot_tools with git clone https://chromium.googlesource.com/chromium/tools/depot_tools.git

And then run the packager,

$ cd depot_tools/win_toolchain/
$ python package_from_installed.py -w 10 2017

comment:14 Changed 7 months ago by dcf

Status: newneeds_review

I've made progress with cross-compiling webrtc for windows-x86_64. It's in two branches. The first branch just upgrades webrtc to branch-heads/64, which has the cross-compiling support. The second branch builds off the first and adds a build script for windows.

The build is currently using the prebuilt clang cross-compiler that Google puts among the dependencies of webrtc. Eventually we will have to make it use our own compiler instead, as we already do on mac and linux. Reliance on the prebuilt compiler is the reason why the build currently only works for windows-x86_64, not windows-i686: there's no 32-bit prebuilt compiler available.

https://github.com/uumaro/tor-browser-build/commits/webrtc-64 (currently c8048d746c)
This branch updates the build scripts to use a webrtc based on branch-heads/64. It depends on having a go-webrtc that also supports the branch-heads/64 API. That code hasn't been merged into go-webrtc yet, but there is a pull request for it. To test the code before the go-webrtc changes have been merged, do this in the tor-browser-build directory before make:
cd git_clones/go-webrtc/
git fetch https://github.com/uumaro/go-webrtc/ branch-heads-64
cd ..
This branch can be merged independently of the windows code, once the go-webrtc pull request goes through.
https://github.com/uumaro/tor-browser-build/commits/webrtc-win (currently 08613622f6)
This branch lets the webrtc build work for windows-i686. To test it, first git fetch an updated go-webrtc as described above. If have done a build in this directory previously, run
rm gclient/webrtc/.gclient
The .gclient will be regenerated with the addition of target_os = ['win'], which will cause gclient sync to pull the prebuilt clang cross-compiler. If you have not yet done a build in this directory, then you will need to do
make submodule-update
Then run
rbm/rbm build webrtc --target testbuild --target torbrowser-windows-x86_64
If you want to test that other targets still work, do
make testbuild

The output of the build is a file, out/webrtc/webrtc-88f5d9180eae78a6162cccd78850ff416eb82483-windows-x86_64-ba8356.tar.gz. You can compare it against mine:

https://people.torproject.org/~dcf/pt-bundle/snowflake/webrtc-88f5d9180eae78a6162cccd78850ff416eb82483-windows-x86_64-ba8356.tar.gz

A few comments on the second branch:

  • I compiled my own Windows SDK using the instructions in comment:13 and stashed it in my people.torproject.org space. I have a todo to check if we really need this separate SDK, or if the header files in mingw-w64 are enough.
  • The upstream cross-compiling instructions say to put the Windows SDK in a case-insensitive ciopfs mount, so that, for example, code can #include <winsock2.h> when the file is actually called WinSock2.h. I tried that initially, but I couldn't get ciopfs to work inside the container. boklm gave me a hint that I haven't tried yet. In the meantime, though, I disabled ciopfs and worked around case sensitivity with a lot of symlinks. I left it in the branch history for now.

I haven't yet tried actually doing anything with the generated libwebrtc-windows-amd64-magic.lib.

I'm not asking for any of this code to be merged right now. But I would appreciate some review of the code (this is my first time writing for rbm), and a check to see if the commands above work for you.

Last edited 7 months ago by dcf (previous) (diff)

comment:15 in reply to:  14 ; Changed 7 months ago by boklm

Replying to dcf:

This branch lets the webrtc build work for windows-i686. To test it, first get fetch an updated go-webrtc as described above. If have done a build in this directory previously, run

rm gclient/webrtc/.gclient

The .gclient will be regenerated with the addition of target_os = ['win'], which will cause gclient sync to pull the prebuilt clang cross-compiler.

I think we could add something like this, to automatically remove gclient/webrtc/src/testing/{gmock,gtest}:
https://gitweb.torproject.org/user/boklm/tor-browser-build.git/commit/?h=bug_25483_v0&id=d5514f7f3b566cf37bfc59ec8105a96130577ff2

And this to remove the .gclient file if it does not contain the target_os line:
https://gitweb.torproject.org/user/boklm/tor-browser-build.git/commit/?h=bug_25483_v0&id=60a0b31b389fa2ca47dfbe305f9f9211b58f509e

The output of the build is a file, out/webrtc/webrtc-88f5d9180eae78a6162cccd78850ff416eb82483-windows-x86_64-ba8356.tar.gz. You can compare it against mine:

https://people.torproject.org/~dcf/pt-bundle/snowflake/webrtc-88f5d9180eae78a6162cccd78850ff416eb82483-windows-x86_64-ba8356.tar.gz

I have been able to do a build, and I get a tarball containing the same files, except that my version also includes some files in the directory webrtc/include/chromium/:
https://people.torproject.org/~boklm/tmp/webrtc-88f5d9180eae78a6162cccd78850ff416eb82483-windows-x86_64-ba8356.tar.gz

comment:16 in reply to:  15 Changed 7 months ago by boklm

Replying to boklm:

I have been able to do a build, and I get a tarball containing the same files, except that my version also includes some files in the directory webrtc/include/chromium/:
https://people.torproject.org/~boklm/tmp/webrtc-88f5d9180eae78a6162cccd78850ff416eb82483-windows-x86_64-ba8356.tar.gz

It seems the reason is that I have a chromium directory in gclient/webrtc/src, probably from a previous build.

comment:17 in reply to:  15 Changed 7 months ago by dcf

Replying to boklm:

I have been able to do a build, and I get a tarball containing the same files, except that my version also includes some files in the directory webrtc/include/chromium/:
https://people.torproject.org/~boklm/tmp/webrtc-88f5d9180eae78a6162cccd78850ff416eb82483-windows-x86_64-ba8356.tar.gz
It seems the reason is that I have a chromium directory in gclient/webrtc/src, probably from a previous build.

Thanks for testing and for the suggestions. I suppose we should more careful in the recursive search for header files. At least the files we had in common are identical, according to diffoscope.

comment:18 Changed 7 months ago by t0mmy

Cc: t0mmy added

comment:19 Changed 7 months ago by dcf

go-webrtc merged the update to branch-heads/64. I rebased the webrtc-64 and webrtc-win branches to match.

https://github.com/uumaro/tor-browser-build/commits/webrtc-64 (currently 62fe24015a)
https://github.com/uumaro/tor-browser-build/commits/webrtc-win (currently 5a369308bb)

comment:20 Changed 7 months ago by mcs

Cc: brade mcs added

comment:21 Changed 7 months ago by gk

Keywords: TorbrowserTeam201804 added
Owner: set to sukhe
Status: needs_reviewassigned

comment:22 Changed 7 months ago by gk

Priority: MediumHigh

comment:23 Changed 7 months ago by sukhbir

Owner: changed from sukhe to sukhbir

comment:24 Changed 6 months ago by sukhbir

As an update to this ticket, I have been trying to get the Windows build to work, the latest progress of which is at https://github.com/azadi/tor-browser-build/commits/go-webrtc ... it's not much and mostly builds on the work of dcf and others to try to fix the error below.

Currently, the status is that we are hitting a linking error when trying to build go-webrtc (mingw-w64) with webrtc (clang). The exact error is:

    /var/tmp/dist/mingw-w64/x86_64-w64-mingw32/include/malloc.h:183:0: note: this is the location of the previous definition
     #define alloca(x) __builtin_alloca((x))
     ^
    # github.com/keroserene/go-webrtc
    /var/tmp/dist/mingw-w64/lib/gcc/x86_64-w64-mingw32/5.4.0/../../../../x86_64-w64-mingw32/lib/../lib/libstdc++.a(cow-stdexcept.o): In function `std::string::_M_data() const':
    /var/tmp/build/gcc/x86_64-w64-mingw32/libstdc++-v3/src/c++11/../../../../gcc-5.4.0/libstdc++-v3/src/c++11/cow-stdexcept.cc:44: multiple definition of `std::logic_error::logic_error(std::logic_error const&)'
    /tmp/go-build868110800/github.com/keroserene/go-webrtc/_obj/peerconnection.cc.o:peerconnection.cc:(.text$_ZNSt11logic_errorC2ERKS_[_ZNSt11logic_errorC2ERKS_]+0x0): first defined here
    /tmp/go-build868110800/github.com/keroserene/go-webrtc/_obj/ctestenums.cc.o:ctestenums.cc:(.text+0x5a9): undefined reference to `cricket::AudioCodec::ToString() const'
    /tmp/go-build868110800/github.com/keroserene/go-webrtc/_obj/ctestenums.cc.o:ctestenums.cc:(.text+0x6de): undefined reference to `webrtc::RtpExtension::ToString() const'
    /tmp/go-build868110800/github.com/keroserene/go-webrtc/_obj/ctestenums.cc.o:ctestenums.cc:(.text+0x813): undefined reference to `cricket::DataCodec::ToString() const'
    /tmp/go-build868110800/github.com/keroserene/go-webrtc/_obj/ctestenums.cc.o:ctestenums.cc:(.text+0x8f5): undefined reference to `rtc::FatalMessage::FatalMessage(char const*, int)'
    /tmp/go-build868110800/github.com/keroserene/go-webrtc/_obj/ctestenums.cc.o:ctestenums.cc:(.text+0x96a): undefined reference to `rtc::FatalMessage::~FatalMessage()'
    /tmp/go-build868110800/github.com/keroserene/go-webrtc/_obj/ctestenums.cc.o:ctestenums.cc:(.text+0xa1b): undefined reference to `rtc::FatalMessage::~FatalMessage()'
    /tmp/go-build868110800/github.com/keroserene/go-webrtc/_obj/ctestenums.cc.o:ctestenums.cc:(.text+0xbf8): undefined reference to `rtc::FatalMessage::FatalMessage(char const*, int)'
    /tmp/go-build868110800/github.com/keroserene/go-webrtc/_obj/ctestenums.cc.o:ctestenums.cc:(.text+0xc6d): undefined reference to `rtc::FatalMessage::~FatalMessage()'
    /tmp/go-build868110800/github.com/keroserene/go-webrtc/_obj/ctestenums.cc.o:ctestenums.cc:(.text+0xd20): undefined reference to `rtc::FatalMessage::~FatalMessage()'
    /tmp/go-build868110800/github.com/keroserene/go-webrtc/_obj/ctestenums.cc.o:ctestenums.cc:(.text+0xdb1): undefined reference to `rtc::FatalMessage::FatalMessage(char const*, int)'
    /tmp/go-build868110800/github.com/keroserene/go-webrtc/_obj/ctestenums.cc.o:ctestenums.cc:(.text+0xe26): undefined reference to `rtc::FatalMessage::~FatalMessage()'
    /tmp/go-build868110800/github.com/keroserene/go-webrtc/_obj/ctestenums.cc.o:ctestenums.cc:(.text+0xed5): undefined reference to `rtc::FatalMessage::~FatalMessage()'
    /tmp/go-build868110800/github.com/keroserene/go-webrtc/_obj/ctestenums.cc.o:ctestenums.cc:(.text+0xf6d): undefined reference to `rtc::FatalMessage::FatalMessage(char const*, int)'
    /tmp/go-build868110800/github.com/keroserene/go-webrtc/_obj/ctestenums.cc.o:ctestenums.cc:(.text+0xfe2): undefined reference to `rtc::FatalMessage::~FatalMessage()'
    /tmp/go-build868110800/github.com/keroserene/go-webrtc/_obj/ctestenums.cc.o:ctestenums.cc:(.text+0x1094): undefined reference to `rtc::FatalMessage::~FatalMessage()'
    /tmp/go-build868110800/github.com/keroserene/go-webrtc/_obj/ctestenums.cc.o:ctestenums.cc:(.text+0x112c): undefined reference to `rtc::FatalMessage::FatalMessage(char const*, int)'
    /tmp/go-build868110800/github.com/keroserene/go-webrtc/_obj/ctestenums.cc.o:ctestenums.cc:(.text+0x11a1): undefined reference to `rtc::FatalMessage::~FatalMessage()'
    /tmp/go-build868110800/github.com/keroserene/go-webrtc/_obj/ctestenums.cc.o:ctestenums.cc:(.text+0x1254): undefined reference to `rtc::FatalMessage::~FatalMessage()'
    /var/tmp/dist/mingw-w64/lib/gcc/x86_64-w64-mingw32/5.4.0/../../../../x86_64-w64-mingw32/bin/ld: $WORK/github.com/keroserene/go-webrtc/_obj/ctestenums.cc.o: bad reloc address 0xc0 in section `.rdata'
    collect2: error: ld returned 1 exit status

I have tried different things but mostly focusing on the webrtc library to start with but it seems like that is not the problem since it has the missing symbols (which other than the usual symbol lookup, we also confirmed by writing a small program that calls them.) It was suggested that this can be due to C++ ABI (in)compatibility between mingw-w64 and clang but at least one mingw-w64 person I talked with disagreed with that assumption.

A post made on the discuss-webrtc Google group was also not helpful and even though there are similar (not same) posts with this problem, there seem to be no solutions that actually work; I have tried incorporating the other suggestions as well in the various threads, but to no effect.

As suggested by Hooman, I have started looking recently into using the asicerik go-webrtc (this wrapper is for native compiling on Windows using Visual Studio and not cross-compiling using Linux) but it doesn't link against webrtc currently.

comment:25 Changed 6 months ago by sukhbir

A post on the mingw-w64 mailing list suggests that these are clang bugs. From https://sourceforge.net/p/mingw-w64/mailman/message/36308258/:

Those are Clang bugs [0].

Multiple definitions error has been fixed in [1] and probably (just
guessing) made it to Clang 6.0.0 release.

Undefined references bug happens when you link code compiled by Clang to
static libstdc++ (and probably other static libs too) compiled by GCC. I
think there is still no solution for this one.

[0] https://lists.llvm.org/pipermail/cfe-dev/2017-April/053530.html
[1] https://reviews.llvm.org/D33620

comment:26 Changed 6 months ago by gk

Keywords: TorBrowserTeam201805 added

Move our roadmap tickets to May.

comment:27 Changed 6 months ago by cypherpunks

Keywords: TorbrowserTeam201804 removed

comment:28 Changed 6 months ago by sukhbir

So I tried building with clang 6 today just to confirm the theory that one of these bugs was fixed in clang 6.0.0 (and hope the other one was as well :) and ran into the following error. I replaced the precompiled clang webrtc with clang 6.0.0 but I didn't bother to set clang_base_path and just replaced it in the default lookup directory third_party/llvm-build/Release+Asserts.

I will continue to debug this (and try with master instead of release as well) but if you have seen this before, please let me know. A quick search tells me this may be related to cross-compilation but I haven't found any solutions yet.

[188/2538] CC obj/third_party/libvpx/libvpx_intrinsics_avx/quantize_avx.obj
[189/2538] CC obj/third_party/libvpx/libvpx_intrinsics_avx2/sad4d_avx2.obj
FAILED: obj/third_party/libvpx/libvpx_intrinsics_avx2/sad4d_avx2.obj 
../../third_party/llvm-build/Release+Asserts/bin/clang-cl --rsp-quoting=posix /nologo /showIncludes  @obj/third_party/libvpx/libvpx_intrinsics_avx2/sad4d_avx2.obj.rsp /c ../../third_party/libvpx/source/libvpx/vpx_dsp/x86/sad4d_avx2.c /Foobj/third_party/libvpx/libvpx_intrinsics_avx2/sad4d_avx2.obj /Fd"obj/third_party/libvpx/libvpx_intrinsics_avx2_c.pdb"
In file included from ../../third_party/libvpx/source/libvpx/vpx_dsp/x86/sad4d_avx2.c:10:
In file included from /var/tmp/build/webrtc/src/third_party/llvm-build/Release+Asserts/lib/clang/6.0.0/include/immintrin.h:134:
/var/tmp/build/webrtc/src/third_party/llvm-build/Release+Asserts/lib/clang/6.0.0/include/fmaintrin.h(73,10):  error: invalid conversion between vector type '__m128' (vector of 4 'float' values) and integer type 'int' of different size
  return (m128)builtin_ia32_vfmsubss3((v4sf)A, (v4sf)B, (v4sf)C);
		 ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
/var/tmp/build/webrtc/src/third_party/llvm-build/Release+Asserts/lib/clang/6.0.0/include/fmaintrin.h(79,10):  error: invalid conversion between vector type '__m128d' (vector of 2 'double' values) and integer type 'int' of different size
  return (m128d)builtin_ia32_vfmsubsd3((v2df)A, (v2df)B, (v2df)C);
		 ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
/var/tmp/build/webrtc/src/third_party/llvm-build/Release+Asserts/lib/clang/6.0.0/include/fmaintrin.h(97,10):  error: invalid conversion between vector type '__m128' (vector of 4 'float' values) and integer type 'int' of different size
  return (m128)builtin_ia32_vfnmaddss3((v4sf)A, (v4sf)B, (v4sf)C);
		 ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
/var/tmp/build/webrtc/src/third_party/llvm-build/Release+Asserts/lib/clang/6.0.0/include/fmaintrin.h(103,10):  error: invalid conversion between vector type '__m128d' (vector of 2 'double' values) and integer type 'int' of different size
  return (m128d)builtin_ia32_vfnmaddsd3((v2df)A, (v2df)B, (v2df)C);
		 ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
/var/tmp/build/webrtc/src/third_party/llvm-build/Release+Asserts/lib/clang/6.0.0/include/fmaintrin.h(121,10):  error: invalid conversion between vector type '__m128' (vector of 4 'float' values) and integer type 'int' of different size
  return (m128)builtin_ia32_vfnmsubss3((v4sf)A, (v4sf)B, (v4sf)C);
		 ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
/var/tmp/build/webrtc/src/third_party/llvm-build/Release+Asserts/lib/clang/6.0.0/include/fmaintrin.h(127,10):  error: invalid conversion between vector type '__m128d' (vector of 2 'double' values) and integer type 'int' of different size
  return (m128d)builtin_ia32_vfnmsubsd3((v2df)A, (v2df)B, (v2df)C);
		 ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
Note: including file:  /var/tmp/build/webrtc/src/third_party/llvm-build/Release+Asserts/lib/clang/6.0.0/include/fxsrintrin.h
6 errors generated.
[190/2538] CC obj/third_party/libvpx/libvpx_intrinsics_avx2/fwd_txfm_avx2.obj
FAILED: obj/third_party/libvpx/libvpx_intrinsics_avx2/fwd_txfm_avx2.obj 
../../third_party/llvm-build/Release+Asserts/bin/clang-cl --rsp-quoting=posix /nologo /showIncludes  @obj/third_party/libvpx/libvpx_intrinsics_avx2/fwd_txfm_avx2.obj.rsp /c ../../third_party/libvpx/source/libvpx/vpx_dsp/x86/fwd_txfm_avx2.c /Foobj/third_party/libvpx/libvpx_intrinsics_avx2/fwd_txfm_avx2.obj /Fd"obj/third_party/libvpx/libvpx_intrinsics_avx2_c.pdb"
In file included from ../../third_party/libvpx/source/libvpx/vpx_dsp/x86/fwd_txfm_avx2.c:15:
In file included from ../../third_party/libvpx/source/libvpx/vpx_dsp/x86/fwd_dct32x32_impl_avx2.h:11:
In file included from /var/tmp/build/webrtc/src/third_party/llvm-build/Release+Asserts/lib/clang/6.0.0/include/immintrin.h:134:
/var/tmp/build/webrtc/src/third_party/llvm-build/Release+Asserts/lib/clang/6.0.0/include/fmaintrin.h(73,10):  error: invalid conversion between vector type '__m128' (vector of 4 'float' values) and integer type 'int' of different size
  return (m128)builtin_ia32_vfmsubss3((v4sf)A, (v4sf)B, (v4sf)C);
		 ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
/var/tmp/build/webrtc/src/third_party/llvm-build/Release+Asserts/lib/clang/6.0.0/include/fmaintrin.h(79,10):  error: invalid conversion between vector type '__m128d' (vector of 2 'double' values) and integer type 'int' of different size
  return (m128d)builtin_ia32_vfmsubsd3((v2df)A, (v2df)B, (v2df)C);
		 ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
/var/tmp/build/webrtc/src/third_party/llvm-build/Release+Asserts/lib/clang/6.0.0/include/fmaintrin.h(97,10):  error: invalid conversion between vector type '__m128' (vector of 4 'float' values) and integer type 'int' of different size
  return (m128)builtin_ia32_vfnmaddss3((v4sf)A, (v4sf)B, (v4sf)C);
		 ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
/var/tmp/build/webrtc/src/third_party/llvm-build/Release+Asserts/lib/clang/6.0.0/include/fmaintrin.h(103,10):  error: invalid conversion between vector type '__m128d' (vector of 2 'double' values) and integer type 'int' of different size
  return (m128d)builtin_ia32_vfnmaddsd3((v2df)A, (v2df)B, (v2df)C);
		 ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
/var/tmp/build/webrtc/src/third_party/llvm-build/Release+Asserts/lib/clang/6.0.0/include/fmaintrin.h(121,10):  error: invalid conversion between vector type '__m128' (vector of 4 'float' values) and integer type 'int' of different size
  return (m128)builtin_ia32_vfnmsubss3((v4sf)A, (v4sf)B, (v4sf)C);
		 ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
/var/tmp/build/webrtc/src/third_party/llvm-build/Release+Asserts/lib/clang/6.0.0/include/fmaintrin.h(127,10):  error: invalid conversion between vector type '__m128d' (vector of 2 'double' values) and integer type 'int' of different size
  return (m128d)builtin_ia32_vfnmsubsd3((v2df)A, (v2df)B, (v2df)C);
		 ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
Note: including file:   /var/tmp/build/webrtc/src/third_party/llvm-build/Release+Asserts/lib/clang/6.0.0/include/fxsrintrin.h
6 errors generated.
[191/2538] CC obj/third_party/libvpx/libvpx_intrinsics_avx/vp9_diamond_search_sad_avx.obj
ninja: build stopped: subcommand failed.

comment:29 Changed 6 months ago by sukhbir

OK, I managed to resolve comment:28 (by properly linking), and unfortunately, the same error.

/var/tmp/dist/mingw-w64/lib/gcc/x86_64-w64-mingw32/5.4.0/../../../../x86_64-w64-mingw32/lib/../lib/libstdc++.a(cow-stdexcept.o): In function `std::string::_M_data() const':
/var/tmp/build/gcc/x86_64-w64-mingw32/libstdc++-v3/src/c++11/../../../../gcc-5.4.0/libstdc++-v3/src/c++11/cow-stdexcept.cc:44: multiple definition of `std::logic_error::logic_error(std::logic_error const&)'
/tmp/go-build842481917/github.com/keroserene/go-webrtc/_obj/peerconnection.cc.o:peerconnection.cc:(.text$_ZNSt11logic_errorC2ERKS_[_ZNSt11logic_errorC2ERKS_]+0x0): first defined here
/tmp/go-build842481917/github.com/keroserene/go-webrtc/_obj/ctestenums.cc.o:ctestenums.cc:(.text+0x5a9): undefined reference to `cricket::AudioCodec::ToString() const'
/tmp/go-build842481917/github.com/keroserene/go-webrtc/_obj/ctestenums.cc.o:ctestenums.cc:(.text+0x6de): undefined reference to `webrtc::RtpExtension::ToString() const'
/tmp/go-build842481917/github.com/keroserene/go-webrtc/_obj/ctestenums.cc.o:ctestenums.cc:(.text+0x813): undefined reference to `cricket::DataCodec::ToString() const'
/tmp/go-build842481917/github.com/keroserene/go-webrtc/_obj/ctestenums.cc.o:ctestenums.cc:(.text+0x8f5): undefined reference to `rtc::FatalMessage::FatalMessage(char const*, int)'
/tmp/go-build842481917/github.com/keroserene/go-webrtc/_obj/ctestenums.cc.o:ctestenums.cc:(.text+0x96a): undefined reference to `rtc::FatalMessage::~FatalMessage()'
/tmp/go-build842481917/github.com/keroserene/go-webrtc/_obj/ctestenums.cc.o:ctestenums.cc:(.text+0xa1b): undefined reference to `rtc::FatalMessage::~FatalMessage()'
/tmp/go-build842481917/github.com/keroserene/go-webrtc/_obj/ctestenums.cc.o:ctestenums.cc:(.text+0xbf8): undefined reference to `rtc::FatalMessage::FatalMessage(char const*, int)'
/tmp/go-build842481917/github.com/keroserene/go-webrtc/_obj/ctestenums.cc.o:ctestenums.cc:(.text+0xc6d): undefined reference to `rtc::FatalMessage::~FatalMessage()'
/tmp/go-build842481917/github.com/keroserene/go-webrtc/_obj/ctestenums.cc.o:ctestenums.cc:(.text+0xd20): undefined reference to `rtc::FatalMessage::~FatalMessage()'
/tmp/go-build842481917/github.com/keroserene/go-webrtc/_obj/ctestenums.cc.o:ctestenums.cc:(.text+0xdb1): undefined reference to `rtc::FatalMessage::FatalMessage(char const*, int)'
/tmp/go-build842481917/github.com/keroserene/go-webrtc/_obj/ctestenums.cc.o:ctestenums.cc:(.text+0xe26): undefined reference to `rtc::FatalMessage::~FatalMessage()'
/tmp/go-build842481917/github.com/keroserene/go-webrtc/_obj/ctestenums.cc.o:ctestenums.cc:(.text+0xed5): undefined reference to `rtc::FatalMessage::~FatalMessage()'
/tmp/go-build842481917/github.com/keroserene/go-webrtc/_obj/ctestenums.cc.o:ctestenums.cc:(.text+0xf6d): undefined reference to `rtc::FatalMessage::FatalMessage(char const*, int)'
/tmp/go-build842481917/github.com/keroserene/go-webrtc/_obj/ctestenums.cc.o:ctestenums.cc:(.text+0xfe2): undefined reference to `rtc::FatalMessage::~FatalMessage()'
/tmp/go-build842481917/github.com/keroserene/go-webrtc/_obj/ctestenums.cc.o:ctestenums.cc:(.text+0x1094): undefined reference to `rtc::FatalMessage::~FatalMessage()'
/tmp/go-build842481917/github.com/keroserene/go-webrtc/_obj/ctestenums.cc.o:ctestenums.cc:(.text+0x112c): undefined reference to `rtc::FatalMessage::FatalMessage(char const*, int)'
/tmp/go-build842481917/github.com/keroserene/go-webrtc/_obj/ctestenums.cc.o:ctestenums.cc:(.text+0x11a1): undefined reference to `rtc::FatalMessage::~FatalMessage()'
/tmp/go-build842481917/github.com/keroserene/go-webrtc/_obj/ctestenums.cc.o:ctestenums.cc:(.text+0x1254): undefined reference to `rtc::FatalMessage::~FatalMessage()'
/var/tmp/dist/mingw-w64/lib/gcc/x86_64-w64-mingw32/5.4.0/../../../../x86_64-w64-mingw32/bin/ld: $WORK/github.com/keroserene/go-webrtc/_obj/ctestenums.cc.o: bad reloc address 0xc0 in section `.rdata'

This was with clang 6.0.0; I will check with master later.

comment:30 Changed 6 months ago by tom

Cc: tom added

comment:31 Changed 5 months ago by sukhbir

Instructions on how to build and reproduce the error: clone https://github.com/azadi/tor-browser-build/tree/go-webrtc. Run make submodule-update followed by rbm/rbm build go-webrtc --target testbuild --target torbrowser-windows-x86_64; this first builds webrtc and then tries to link it with go-webrtc, at which point you will see an error.

As for the clang suggestions above from the mailing list, I couldn't find the changes in the changelog and as mentioned above, building with the latest version didn't help.

comment:32 in reply to:  31 Changed 5 months ago by gk

Replying to sukhbir:

Instructions on how to build and reproduce the error: clone https://github.com/azadi/tor-browser-build/tree/go-webrtc. Run make submodule-update followed by rbm/rbm build go-webrtc --target testbuild --target torbrowser-windows-x86_64; this first builds webrtc and then tries to link it with go-webrtc, at which point you will see an error.

I did not even get that far. The build is breaking earlier for me following your steps (during the webrtc build). The log ends with:

ln: failed to create symbolic link 'windows.web.http.filters.h': File exists
ln: failed to create symbolic link 'windows.web.http.h': File exists
ln: failed to create symbolic link 'windows.web.http.headers.h': File exists
ln: failed to create symbolic link 'windows.web.syndication.h': File exists
ln: failed to create symbolic link 'winstring.h': File exists
ln: failed to create symbolic link 'wrl.h': File exists
ERROR at //modules/video_capture/BUILD.gn:161:17: Can't load input file.
      deps += [ "//third_party/winsdk_samples" ]
                ^-----------------------------
Unable to load:
  /var/tmp/build/webrtc/src/third_party/winsdk_samples/BUILD.gn
I also checked in the secondary tree for:
  /var/tmp/build/webrtc/src/build/secondary/third_party/winsdk_samples/BUILD.gn

What am I missing?

comment:33 Changed 5 months ago by gk

FWIW: I am at commit c4e3596639091e484410b0622eb4b3a08e83d01e of your branch in case you were wondering.

comment:34 Changed 5 months ago by gk

Actually, that's the second error I get. The first one was

Starting build: Sat May 19 21:29:47 2018
Bootstrapping cipd client for linux-amd64 from https://chrome-infra-packages.appspot.com/client?platform=linux-amd64&version=git_revision:ae28364c740acff97ae118adcb2808b6cb5129c5...

src/testing (ERROR)
----------------------------------------
[0:00:23] Started.
----------------------------------------
Error: Command 'git checkout --quiet 60c665fffe7dc505fdd5d30f9dbcbc50dde1e017' returned non-zero exit status 1 in /home/gk/tor-browser-build/gclient/webrtc/src/testing
error: The following untracked working tree files would be overwritten by checkout:
	gmock/include/gmock/gmock-actions.h
	gmock/include/gmock/gmock-generated-function-mockers.h
	gmock/include/gmock/gmock-matchers.h
	gmock/include/gmock/gmock.h
	gtest/include/gtest/gtest-death-test.h
	gtest/include/gtest/gtest-message.h
	gtest/include/gtest/gtest-param-test.h
	gtest/include/gtest/gtest-spi.h
	gtest/include/gtest/gtest.h
	gtest/include/gtest/gtest_prod.h
Please move or remove them before you switch branches.
Aborting

That's reproducible on two different build machines for me. I worked around both errors on one of the machines my just getting rid of everything webrtc related and starting over in this regard. We should not need to do this, though. I therefore keep the second machine as-is for debugging purposes.

comment:35 Changed 5 months ago by sukhbir

I tried building c4e3596 again and it worked for me. I agree that you should not have to remove everything related to webrtc and I will check that.

The checkout error above suggests that this may have been a network issue; can you try building again? If that doesn't work, I will look why existing builds of webrtc interfere with this one (basing it on the initial state of the tor-browser-build directory to the same as yours).

comment:36 in reply to:  34 ; Changed 5 months ago by sukhbir

Replying to gk:

That's reproducible on two different build machines for me. I worked around both errors on one of the machines my just getting rid of everything webrtc related and starting over in this regard. We should not need to do this, though. I therefore keep the second machine as-is for debugging purposes.

Sorry, did you mean the errors you have are reproducible or the error I mentioned in #comment:24?

comment:37 in reply to:  36 Changed 5 months ago by gk

Replying to sukhbir:

Replying to gk:

That's reproducible on two different build machines for me. I worked around both errors on one of the machines my just getting rid of everything webrtc related and starting over in this regard. We should not need to do this, though. I therefore keep the second machine as-is for debugging purposes.

Sorry, did you mean the errors you have are reproducible or the error I mentioned in #comment:24?

I can reproduce the error in comment:34 on two different machines and that's what I meant.

Over the weekend I tried to look more closely at it. So, updating to mingw-w64 master and GCC 6.4.0 does not help. Then I tried to build clang master and see whether we can verify the claim in comment:25 (and maybe be lucky because our bugs got fixed like last week :) ).

I tried for a while to get the clang build going by setting LLVM_FORCE_HEAD_REVISION to 1. It turns out that breaks the clang plugin compilation and other clang tools' compilation as well. Not compiling those gives a proper clang based on master, unfortunately requires the WebRTC build those missing clang tools.

I then settled to use the latest prebuilt clang binaries (by messing with the update.py script (see: https://chromium.googlesource.com/chromium/src/tools/clang/+/master/scripts/update.py)) using rev 332335 which is just ca. 400 commit behind the latest one. I am still getting the same errors as in comment:24.

I guess it's time to look at the symbols...

comment:38 Changed 5 months ago by cmm323

Cross compiling WebRTC library for Windows on a Linux machine and linking Snowflake involves multiple steps:

1-Cross compiling WebRTC lib: This step is currently done using Clang, which is Microsoft Visual C++ (MSVC) compatible[1].
2-Cross compiling Snowflake code: This step is currently done using MingW because Cgo (which integrates Golang with C) does not support Clang for Windows[2].
3-Linking Snowflake binary against WebRTC lib: This part is problematic. First, Cgo currently doesn't support MSVC object files [3] and MingW and Clang/MSVC object files are not compatible[4]. An attempt to link these two using MingW leads to compiler errors undefined reference to mentioned in29.

[1]- https://clang.llvm.org/docs/MSVCCompatibility.html
[2]- https://github.com/golang/go/issues/17014
[3]- https://github.com/golang/go/issues/20982
[4]- http://www.mingw.org/wiki/MixingCompilers

comment:39 Changed 5 months ago by cmm323

The problem in step 3 is that the name mangling schemes used by the two compilers are different and thus the linker cannot find the appropriate function in the object file generated by the other compiler. However, this is a common problem and a popular solution is to export method calls as C functions. Since C doesn't have namespaces there is no name mangling issue. These functions can be exported as a DLL wrapper (see [1] and [2] for more information). Go-Webrtc wrapper by asicerik[3] wraps WebRTC functions called by Snowflake, in C functions and export them as a runtime DLL. I have tested the functionality of this wrapper on Windows. Cross-compiling the wrapper on a Linux machine, however, seems to be harder. The wrapper uses Windows header and some other WebRTC headers. I believe that we can use the Ninja[4] and GYP build systems that are used to build the WebRTC library to cross-compile the wrapper as well. I think the solution is to add a target to Ninja build system to build the wrapper with WebRTC library as a dependency. Unfortunately, I don't know enough about Ninja build system to do that and won't have the time to research it anytime soon.

[1]- http://www.mingw.org/wiki/Visual_Basic_DLL
[2]- https://www.transmissionzero.co.uk/computing/building-dlls-with-mingw/
[3]- https://github.com/asicerik/go-webrtc
[4]- https://ninja-build.org/manual.html

comment:40 Changed 6 days ago by tom

Given the roadmap for Tor Browser, switching to mingw-clang sometime in the next 4-ish months - is a reasonable path forward for this to get it built with mingw-clang?

comment:41 in reply to:  40 Changed 5 days ago by dcf

Replying to tom:

Given the roadmap for Tor Browser, switching to mingw-clang sometime in the next 4-ish months - is a reasonable path forward for this to get it built with mingw-clang?

Yes, building webrtc with mingw-clang is totally reasonable.

I think it would be reasonable even if Tor Browser were not switching to it, but as they are, it makes it even better.

comment:42 Changed 5 days ago by gaba

Sponsor: Sponsor19
Note: See TracTickets for help on using tickets.