#24477 closed defect (fixed)

Windows 64 mar files are not generated

Reported by: boklm Owned by: tbb-team
Priority: Medium Milestone:
Component: Applications/Tor Browser Version:
Severity: Normal Keywords: tbb-rbm, TorBrowserTeam201712R
Cc: Actual Points:
Parent ID: Points:
Reviewer: Sponsor: Sponsor4

Description

Currently our Windows 64 build does not produce mar files.

As found by GeKo, it is due to using a 64bit mar tool in a 32bit container. We are building the firefox part in a 64bit container, creating a 64bit mar tools, but the final step is done in a 32bit container to use a 32bit nsis as a workaround for #23561.

Child Tickets

Change History (6)

comment:1 Changed 18 months ago by boklm

We can switch to using a 64bit container, installing the 32bit dependencies for running nsis:

diff --git a/projects/tor-browser/config b/projects/tor-browser/config
index c00af05..564dfa6 100644
--- a/projects/tor-browser/config
+++ b/projects/tor-browser/config
@@ -34,10 +34,10 @@ targets:
   windows-x86_64:
     var:
       mar_osname: win64
-      # We use a 32bit container to be able to use a 32bit nsis and
-      # pe_checksum_fix.py
-      container:
-        arch: i386
+      arch_deps:
+        - python-pefile
+        - libc6-i386
+        - lib32stdc++6
 
 input_files:
   - project: container-image

However we then have an error with pe_checksum_fix.py which doesn't work in 64bit:

Traceback (most recent call last):
  File "/var/tmp/tmp.oYuOVB2u7o/pe_checksum_fix.py", line 55, in <module>
    pef.OPTIONAL_HEADER.CheckSum = pef.generate_checksum()
  File "/usr/lib/python2.7/dist-packages/pefile.py", line 3720, in generate_checksum
    dword = struct.unpack('L', self.__data__[ i*4 : i*4+4 ])[0]
struct.error: unpack requires a string argument of length 8

I think we can fix that by installing a more recent version of python-pefile, as done before commit bb32ec91b538ab17d309b645787149ff376d58e9.

comment:2 Changed 18 months ago by gk

Sponsor: Sponsor4

Sponsor4 stuff.

comment:3 in reply to:  1 Changed 18 months ago by boklm

Replying to boklm:

However we then have an error with pe_checksum_fix.py which doesn't work in 64bit:

Traceback (most recent call last):
  File "/var/tmp/tmp.oYuOVB2u7o/pe_checksum_fix.py", line 55, in <module>
    pef.OPTIONAL_HEADER.CheckSum = pef.generate_checksum()
  File "/usr/lib/python2.7/dist-packages/pefile.py", line 3720, in generate_checksum
    dword = struct.unpack('L', self.__data__[ i*4 : i*4+4 ])[0]
struct.error: unpack requires a string argument of length 8

I think we can fix that by installing a more recent version of python-pefile, as done before commit bb32ec91b538ab17d309b645787149ff376d58e9.

I tried to do that with this patch:
https://gitweb.torproject.org/user/boklm/tor-browser-build.git/commit/?h=bug_24477&id=2bd73ca5c7ff5036dac8f562ba2fae36717519e4

However I am still getting a similar error:

  File "/var/tmp/tmp.f4UwWTvuwY/pe_checksum_fix.py", line 55, in <module>
    pef.OPTIONAL_HEADER.CheckSum = pef.generate_checksum()
  File "/var/tmp/tmp.f4UwWTvuwY/.local/lib/python2.7/site-packages/pefile.py", line 5418, in generate_checksum
    dword = struct.unpack('I', self.__data__[ i*4 : i*4+4 ])[0]
struct.error: unpack requires a string argument of length 4

comment:4 Changed 18 months ago by gk

Can't we just grab the 32bit win mar-tools in that case doing some hard-coding if necessary? This target is already built at that stage, thus those tools should be available. I am fine if that breaks building the 64bit alone (i.e. if one has not built the 32bit variant first). It's just a second order hack for the upcoming release.

comment:5 Changed 18 months ago by boklm

Keywords: TorBrowserTeam201712R added; TorBrowserTeam201711 removed
Status: newneeds_review

In branch bug_24477_v2 I added a workaround, using mar-tools-linux32.zip from a previous build:
https://gitweb.torproject.org/user/boklm/tor-browser-build.git/commit/?h=bug_24477_v2&id=959e6b54ff77a729cc3db976ba3747e4508a4ca9

comment:6 Changed 18 months ago by gk

Keywords: tbb-rbm added
Resolution: fixed
Status: needs_reviewclosed

Works for me although I guess we could have done something like

 # Extract the MAR tools.
-unzip -d $rootdir $rootdir/[% c('input_files_by_name/firefox') %]/mar-tools-*.zip
+[% IF c("var/windows-x86_64") -%]
+  # Workaround for bug 24477
+  unzip -d $rootdir $rootdir/mar-tools-linux32.zip
+[% ELSE %]
+  unzip -d $rootdir $rootdir/[% c('input_files_by_name/firefox') %]/mar-tools-*.zip
+[% END -%]

as well. Anyway, pushed to master as commit 0abccfe4258218612f38ac1c2d762c319781386f.

Note: See TracTickets for help on using tickets.