Opened 10 months ago

Closed 7 months ago

#29307 closed enhancement (fixed)

Use Debian Stretch for cross-compiling our Windows builds

Reported by: gk Owned by: tbb-team
Priority: Medium Milestone:
Component: Applications/Tor Browser Version:
Severity: Normal Keywords: tbb-rbm, GeorgKoppen201904, TorBrowserTeam201905R
Cc: pospeselr Actual Points:
Parent ID: #28238 Points:
Reviewer: Sponsor:

Description

We should move away from Jessie to Stretch for a number of reasons (Jessie is no properly supported Debain distro anymore and it would help with #28238 it seems, just to name two)

Child Tickets

TicketTypeStatusOwnerSummary
#29319defectclosedtbb-teamRemove FTE support in Windows bundles

Change History (21)

comment:1 Changed 10 months ago by gk

Keywords: TorBrowserTeam201902 GeorgKoppen201902 added

I've been working on that today while trying to fix #28238 and it's not that hard for x86_64 at least. I got everything cross-compiled for that architecture, just with a small patch needed for the Rust config.

For i686 I got pretty far but am blocked right now by a PyCrypto/Wine issue:

running build_ext
wine: Bad EXE format for Z:\var\tmp\dist\mingw-w64\bin\i686-w64-mingw32-dllwrap..
running build_configure
wine: Bad EXE format for Z:\var\tmp\dist\mingw-w64\bin\i686-w64-mingw32-dllwrap..
building 'Crypto.Random.OSRNG.winrandom' extension
creating build
creating build\temp.win32-2.7
creating build\temp.win32-2.7\Release
creating build\temp.win32-2.7\Release\src
C:\windows\gcc.exe -mno-cygwin -mdll -Wall -std=c99 -O3 -fomit-frame-pointer -Isrc/ -IZ:\var\tmp\dist\winpython\include -IZ:\var\tmp\dist\winpython\PC -c src/winrand.c -o build\temp.win32-2.7\Release\src\winrand.o
/var/tmp/home/.wine/dosdevices/z:/var/tmp/build/pycrypto-2.6.1/src/winrand.c:38:0: warning: "_WIN32_WINNT" redefined
 #define _WIN32_WINNT 0x400
 
In file included from /var/tmp/dist/mingw-w64/i686-w64-mingw32/include/crtdefs.h:10:0,
                 from /var/tmp/dist/mingw-w64/i686-w64-mingw32/include/io.h:9,
                 from /var/tmp/home/.wine/dosdevices/z:/var/tmp/dist/winpython/include/pyconfig.h:68,
                 from /var/tmp/home/.wine/dosdevices/z:/var/tmp/dist/winpython/include/Python.h:8,
                 from /var/tmp/home/.wine/dosdevices/z:/var/tmp/build/pycrypto-2.6.1/src/winrand.c:33:
/var/tmp/dist/mingw-w64/i686-w64-mingw32/include/_mingw.h:232:0: note: this is the location of the previous definition
 #define _WIN32_WINNT 0x502
 
Exception WindowsError: (6, 'Invalid handle') in <bound method Popen.__del__ of <subprocess.Popen object at 0x006970D0>> ignored
writing build\temp.win32-2.7\Release\src\winrandom.def
creating build\lib.win32-2.7
creating build\lib.win32-2.7\Crypto
creating build\lib.win32-2.7\Crypto\Random
creating build\lib.win32-2.7\Crypto\Random\OSRNG
C:\windows\dllwrap.exe -mno-cygwin -mdll -static --output-lib build\temp.win32-2.7\Release\src\libwinrandom.a --def build\temp.win32-2.7\Release\src\winrandom.def -s build\temp.win32-2.7\Release\src\winrand.o -LZ:\var\tmp\dist\winpython\libs -LZ:\var\tmp\dist\winpython\PCbuild -lws2_32 -ladvapi32 -lpython27 -o build\lib.win32-2.7\Crypto\Random\OSRNG\winrandom.pyd
wine: Bad EXE format for Z:\var\tmp\dist\mingw-w64\bin\i686-w64-mingw32-dllwrap..
Traceback (most recent call last):
  File "dllwrap.py", line 32, in <module>
  File "common.pyc", line 27, in popen_faketime
  File "subprocess.pyc", line 711, in __init__
  File "subprocess.pyc", line 948, in _execute_child
WindowsError: [Error 193] Bad EXE format for %1
warning: GMP or MPIR library not found; Not building Crypto.PublicKey._fastmath.
error: command 'dllwrap' failed with exit status 255

comment:2 Changed 10 months ago by gk

Component: ApplicationsApplications/Tor Browser
Owner: set to tbb-team

comment:3 Changed 10 months ago by gk

Summary: Use Debian Stretch for cross-comiling our Windows buildsUse Debian Stretch for cross-compiling our Windows builds

comment:4 Changed 10 months ago by gk

With wine 3.0.3 things go bad differently:

C:\windows\gcc.exe -mno-cygwin -mdll -Wall -std=c99 -O3 -fomit-frame-pointer -Isrc/ -IZ:\var\tmp\dist\winpython\include -IZ:\var\tmp\dist\winpython\PC -c src/winrand.c -o build\temp.win32-2.7\Release\src\winrand.o
/var/tmp/home/.wine/dosdevices/z:/var/tmp/build/pycrypto-2.6.1/src/
/var/tmp/home/.wine/dosdevices/z:/var/tmp/dist/winpython/include
/var/tmp/home/.wine/dosdevices/z:/var/tmp/dist/winpython/PC
/var/tmp/home/.wine/dosdevices/z:/var/tmp/build/pycrypto-2.6.1/src/winrand.c
/var/tmp/home/.wine/dosdevices/z:/var/tmp/build/pycrypto-2.6.1/build/temp.win32-2.7/Release/src/winrand.o
Exception WindowsError: (6, 'Invalid handle') in <bound method Popen.__del__ of <subprocess.Popen object at 0x006970D0>> ignored
i686-w64-mingw32-gcc: error: : No such file or directory
i686-w64-mingw32-gcc: fatal error: output filename may not be empty

I guess we need to adjust the wrappers accordingly, blech.

comment:5 Changed 10 months ago by gk

I think given that FTE is about to go away we'll just rip the remaining Windows support out. This will be done in #29319.

comment:6 Changed 10 months ago by gk

Status: newneeds_review

bug_29307 (https://gitweb.torproject.org/user/gk/tor-browser-build.git/commit/?h=bug_29307&id=8b4e03eb5f39e0bb34c8b7cdf00a9dc5102a7d5d) has the commit for review. It's on top of the one from #29319 for easier testing.

That's 9.0a1 material as well.

comment:7 Changed 10 months ago by gk

Keywords: TorBrowserTeam201902R added; TorBrowserTeam201902 removed

comment:8 Changed 9 months ago by gk

Keywords: TorBrowserTeam201903R added; TorBrowserTeam201902R removed

February is gone.

comment:9 Changed 9 months ago by pospeselr

Cc: pospeselr added
Status: needs_reviewneeds_revision

comment:10 Changed 9 months ago by gk

Keywords: GeorgKoppen201903 TorBrowserTeam201903 added; GeorgKoppen201902 TorBrowserTeam201903R removed

For the jessie-backports part:

21:05 <+GeKo> pospeselr: no need to have the pre_pkginst thing for the stretch 
              patch
21:06 <+GeKo> we can just delete thise three lines, good catch

I am not sure what's up with the reported build issue for i686, though.

comment:11 Changed 8 months ago by pospeselr

https://gitweb.torproject.org/user/richard/tor-browser-build.git/log/?h=bug_29307

Rebased gk's branches against master, removed jessie-backports pre-pkg step from tor-browser, and added i386 stretch container.

comment:12 in reply to:  11 Changed 8 months ago by gk

Keywords: TorBrowserTeam201903R added; TorBrowserTeam201903 removed
Status: needs_revisionneeds_review

Replying to pospeselr:

https://gitweb.torproject.org/user/richard/tor-browser-build.git/log/?h=bug_29307

Rebased gk's branches against master, removed jessie-backports pre-pkg step from tor-browser, and added i386 stretch container.

Thanks. There is still a Windows block in the build script for libfte and we can remove the 32bit Jessie container in the debootstrap-image config file. Moreover, I think we should not squash those commits as they are somewhat unrelated.

Unforunately/luckily I worked at that yesterday as well but did forget to update the ticket as my testbuilds took a long time rebuilding everything. But: here is a branch I came up with which I think is ready for review: bug_29307_v5 (https://gitweb.torproject.org/user/gk/tor-browser-build.git/log/?h=bug_29307_v5).

comment:13 Changed 8 months ago by pospeselr

You should be able to remove boklm's fix for the python-future issue (#29868) in tor-browser/config

comment:14 in reply to:  13 Changed 8 months ago by gk

Replying to pospeselr:

You should be able to remove boklm's fix for the python-future issue (#29868) in tor-browser/config

Good catch! Indeed, that landed meanwhile. I've just pushed bug_29307_v6 (https://gitweb.torproject.org/user/gk/tor-browser-build.git/log/?h=bug_29307_v6) which first reverts the patch for #29868 and does the remaining clean-up later, after the commit for #29319.

comment:15 Changed 8 months ago by gk

Keywords: TorBrowserTeam201904R added; TorBrowserTeam201903R removed

Moving review tickets to April.

comment:16 Changed 8 months ago by gk

Keywords: GeorgKoppen201904 added; GeorgKoppen201903 removed

Moving my tickets for April

comment:17 Changed 7 months ago by gk

I created a rebased version against latest master for review bug_29307_v7 (https://gitweb.torproject.org/user/gk/tor-browser-build.git/log/?h=bug_29307_v7). It contains commit cfcb23a66e3ebd34b4df298f9e7550f1e0c97702 for this bug and two commits (b8c497a4eb16f8ea80914d6def714115977b6c11 and 0c23af0c8c287a79a195de3df97bb70488e6ab23) for #29319.

comment:18 Changed 7 months ago by cypherpunks

for some reason current tor mingw builds are not built with

Liblzma N/A, and Libzstd N/A.

please remind to configure

--enable-lzma   --enable-zstd

them with both libs.

[notice] Tor 0.3.5.7 (git-9beb085c10562a25) running on Windows 8 [or later] with Libevent 2.1.8-stable, OpenSSL 1.0.2q, Zlib 1.2.11, Liblzma N/A, and Libzstd N/A.

they was even available for mingw package very long time...

Last edited 7 months ago by cypherpunks (previous) (diff)

comment:19 in reply to:  18 Changed 7 months ago by gk

Replying to cypherpunks:

for some reason current tor mingw builds are not built with

Liblzma N/A, and Libzstd N/A.

please remind to configure

--enable-lzma   --enable-zstd

them with both libs.

[notice] Tor 0.3.5.7 (git-9beb085c10562a25) running on Windows 8 [or later] with Libevent 2.1.8-stable, OpenSSL 1.0.2q, Zlib 1.2.11, Liblzma N/A, and Libzstd N/A.

they was even available for mingw package very long time...

#22341 is the ticket you wanted to put this comment on. Yes, we want to have this too, so patches are welcome.

comment:20 Changed 7 months ago by gk

Keywords: TorBrowserTeam201905R added; TorBrowserTeam201904R removed

No April anymore, moving review tickets to May.

comment:21 in reply to:  17 Changed 7 months ago by boklm

Resolution: fixed
Status: needs_reviewclosed

Replying to gk:

I created a rebased version against latest master for review bug_29307_v7 (https://gitweb.torproject.org/user/gk/tor-browser-build.git/log/?h=bug_29307_v7). It contains commit cfcb23a66e3ebd34b4df298f9e7550f1e0c97702 for this bug and two commits (b8c497a4eb16f8ea80914d6def714115977b6c11 and 0c23af0c8c287a79a195de3df97bb70488e6ab23) for #29319.

This looks good to me. I merged those commits to master with commit b00d30e8a5a91533d8fc8c7786793b72348638cf.

Note: See TracTickets for help on using tickets.