Use Debian Stretch for cross-compiling our Windows builds

[gk]

We should move away from Jessie to Stretch for a number of reasons (Jessie is no properly supported Debain distro anymore and it would help with #28238 it seems, just to name two)

[gk]

I've been working on that today while trying to fix #28238 and it's not that hard for x86_64 at least. I got everything cross-compiled for that architecture, just with a small patch needed for the Rust config.

For i686 I got pretty far but am blocked right now by a PyCrypto/Wine issue:

running build_ext
wine: Bad EXE format for Z:\var\tmp\dist\mingw-w64\bin\i686-w64-mingw32-dllwrap..
running build_configure
wine: Bad EXE format for Z:\var\tmp\dist\mingw-w64\bin\i686-w64-mingw32-dllwrap..
building 'Crypto.Random.OSRNG.winrandom' extension
creating build
creating build\temp.win32-2.7
creating build\temp.win32-2.7\Release
creating build\temp.win32-2.7\Release\src
C:\windows\gcc.exe -mno-cygwin -mdll -Wall -std=c99 -O3 -fomit-frame-pointer -Isrc/ -IZ:\var\tmp\dist\winpython\include -IZ:\var\tmp\dist\winpython\PC -c src/winrand.c -o build\temp.win32-2.7\Release\src\winrand.o
/var/tmp/home/.wine/dosdevices/z:/var/tmp/build/pycrypto-2.6.1/src/winrand.c:38:0: warning: "_WIN32_WINNT" redefined
 #define _WIN32_WINNT 0x400
 
In file included from /var/tmp/dist/mingw-w64/i686-w64-mingw32/include/crtdefs.h:10:0,
                 from /var/tmp/dist/mingw-w64/i686-w64-mingw32/include/io.h:9,
                 from /var/tmp/home/.wine/dosdevices/z:/var/tmp/dist/winpython/include/pyconfig.h:68,
                 from /var/tmp/home/.wine/dosdevices/z:/var/tmp/dist/winpython/include/Python.h:8,
                 from /var/tmp/home/.wine/dosdevices/z:/var/tmp/build/pycrypto-2.6.1/src/winrand.c:33:
/var/tmp/dist/mingw-w64/i686-w64-mingw32/include/_mingw.h:232:0: note: this is the location of the previous definition
 #define _WIN32_WINNT 0x502
 
Exception WindowsError: (6, 'Invalid handle') in <bound method Popen.__del__ of <subprocess.Popen object at 0x006970D0>> ignored
writing build\temp.win32-2.7\Release\src\winrandom.def
creating build\lib.win32-2.7
creating build\lib.win32-2.7\Crypto
creating build\lib.win32-2.7\Crypto\Random
creating build\lib.win32-2.7\Crypto\Random\OSRNG
C:\windows\dllwrap.exe -mno-cygwin -mdll -static --output-lib build\temp.win32-2.7\Release\src\libwinrandom.a --def build\temp.win32-2.7\Release\src\winrandom.def -s build\temp.win32-2.7\Release\src\winrand.o -LZ:\var\tmp\dist\winpython\libs -LZ:\var\tmp\dist\winpython\PCbuild -lws2_32 -ladvapi32 -lpython27 -o build\lib.win32-2.7\Crypto\Random\OSRNG\winrandom.pyd
wine: Bad EXE format for Z:\var\tmp\dist\mingw-w64\bin\i686-w64-mingw32-dllwrap..
Traceback (most recent call last):
  File "dllwrap.py", line 32, in <module>
  File "common.pyc", line 27, in popen_faketime
  File "subprocess.pyc", line 711, in __init__
  File "subprocess.pyc", line 948, in _execute_child
WindowsError: [Error 193] Bad EXE format for %1
warning: GMP or MPIR library not found; Not building Crypto.PublicKey._fastmath.
error: command 'dllwrap' failed with exit status 255

[gk]

With wine 3.0.3 things go bad differently:

C:\windows\gcc.exe -mno-cygwin -mdll -Wall -std=c99 -O3 -fomit-frame-pointer -Isrc/ -IZ:\var\tmp\dist\winpython\include -IZ:\var\tmp\dist\winpython\PC -c src/winrand.c -o build\temp.win32-2.7\Release\src\winrand.o
/var/tmp/home/.wine/dosdevices/z:/var/tmp/build/pycrypto-2.6.1/src/
/var/tmp/home/.wine/dosdevices/z:/var/tmp/dist/winpython/include
/var/tmp/home/.wine/dosdevices/z:/var/tmp/dist/winpython/PC
/var/tmp/home/.wine/dosdevices/z:/var/tmp/build/pycrypto-2.6.1/src/winrand.c
/var/tmp/home/.wine/dosdevices/z:/var/tmp/build/pycrypto-2.6.1/build/temp.win32-2.7/Release/src/winrand.o
Exception WindowsError: (6, 'Invalid handle') in <bound method Popen.__del__ of <subprocess.Popen object at 0x006970D0>> ignored
i686-w64-mingw32-gcc: error: : No such file or directory
i686-w64-mingw32-gcc: fatal error: output filename may not be empty

I guess we need to adjust the wrappers accordingly, blech.

[gk]

I think given that FTE is about to go away we'll just rip the remaining Windows support out. This will be done in #29319.

[gk]

bug_29307 (​https://gitweb.torproject.org/user/gk/tor-browser-build.git/commit/?h=bug_29307&id=8b4e03eb5f39e0bb34c8b7cdf00a9dc5102a7d5d) has the commit for review. It's on top of the one from #29319 for easier testing.

That's 9.0a1 material as well.

[gk]

February is gone.

[gk]

For the jessie-backports part:

21:05 <+GeKo> pospeselr: no need to have the pre_pkginst thing for the stretch 
              patch
21:06 <+GeKo> we can just delete thise three lines, good catch

I am not sure what's up with the reported build issue for i686, though.

[pospeselr]

​https://gitweb.torproject.org/user/richard/tor-browser-build.git/log/?h=bug_29307

Rebased gk's branches against master, removed jessie-backports pre-pkg step from tor-browser, and added i386 stretch container.

[gk]

Replying to pospeselr:

​https://gitweb.torproject.org/user/richard/tor-browser-build.git/log/?h=bug_29307

Rebased gk's branches against master, removed jessie-backports pre-pkg step from tor-browser, and added i386 stretch container.

Thanks. There is still a Windows block in the build script for libfte and we can remove the 32bit Jessie container in the debootstrap-image config file. Moreover, I think we should not squash those commits as they are somewhat unrelated.

Unforunately/luckily I worked at that yesterday as well but did forget to update the ticket as my testbuilds took a long time rebuilding everything. But: here is a branch I came up with which I think is ready for review: bug_29307_v5 (​https://gitweb.torproject.org/user/gk/tor-browser-build.git/log/?h=bug_29307_v5).

[pospeselr]

You should be able to remove boklm's fix for the python-future issue (#29868) in tor-browser/config

[gk]

Replying to pospeselr:

You should be able to remove boklm's fix for the python-future issue (#29868) in tor-browser/config

Good catch! Indeed, that landed meanwhile. I've just pushed bug_29307_v6 (​https://gitweb.torproject.org/user/gk/tor-browser-build.git/log/?h=bug_29307_v6) which first reverts the patch for #29868 and does the remaining clean-up later, after the commit for #29319.

[gk]

Moving review tickets to April.

[gk]

Moving my tickets for April

[gk]

I created a rebased version against latest master for review bug_29307_v7 (​https://gitweb.torproject.org/user/gk/tor-browser-build.git/log/?h=bug_29307_v7). It contains commit cfcb23a66e3ebd34b4df298f9e7550f1e0c97702 for this bug and two commits (b8c497a4eb16f8ea80914d6def714115977b6c11 and 0c23af0c8c287a79a195de3df97bb70488e6ab23) for #29319.

[cypherpunks]

for some reason current tor mingw builds are not built with

Liblzma N/A, and Libzstd N/A.

please remind to configure

--enable-lzma   --enable-zstd

them with both libs.

[notice] Tor 0.3.5.7 (git-9beb085c10562a25) running on Windows 8 [or later] with Libevent 2.1.8-stable, OpenSSL 1.0.2q, Zlib 1.2.11, Liblzma N/A, and Libzstd N/A.

they was even available for mingw package very long time...

[gk]

Replying to cypherpunks:

for some reason current tor mingw builds are not built with

Liblzma N/A, and Libzstd N/A.

please remind to configure

--enable-lzma   --enable-zstd

them with both libs.

[notice] Tor 0.3.5.7 (git-9beb085c10562a25) running on Windows 8 [or later] with Libevent 2.1.8-stable, OpenSSL 1.0.2q, Zlib 1.2.11, Liblzma N/A, and Libzstd N/A.

they was even available for mingw package very long time...

#22341 is the ticket you wanted to put this comment on. Yes, we want to have this too, so patches are welcome.

[gk]

No April anymore, moving review tickets to May.

[boklm]

Replying to gk:

I created a rebased version against latest master for review bug_29307_v7 (​https://gitweb.torproject.org/user/gk/tor-browser-build.git/log/?h=bug_29307_v7). It contains commit cfcb23a66e3ebd34b4df298f9e7550f1e0c97702 for this bug and two commits (b8c497a4eb16f8ea80914d6def714115977b6c11 and 0c23af0c8c287a79a195de3df97bb70488e6ab23) for #29319.

This looks good to me. I merged those commits to master with commit b00d30e8a5a91533d8fc8c7786793b72348638cf.