Opened 6 years ago

Closed 5 years ago

#10104 closed enhancement (fixed)

Rebase our Gitian onto latest upstream

Reported by: mikeperry Owned by: mikeperry
Priority: High Milestone:
Component: Applications/Tor bundles/installation Version:
Severity: Keywords: gitian, MikePerry201402R
Cc: gk Actual Points:
Parent ID: Points:
Reviewer: Sponsor:

Description

Arlo pointed out that some LXC bugs and other issues affecting pluggable transport bundles have been fixed upstream in Gitian. We should rebase

Child Tickets

Change History (9)

comment:1 Changed 6 years ago by mikeperry

Keywords: MikePerry201401 added; MikePerry201311 removed

comment:2 Changed 6 years ago by gk

Rebasing is necessary to fix #10558 (but alas not sufficient).

comment:3 Changed 5 years ago by mikeperry

Keywords: MikePerry201403 added; MikePerry201401 removed

gk - I probably won't get to this this month. If you feel like doing this, don't let me stop you. Otherwise maybe I will pick it up in March.

comment:4 Changed 5 years ago by gk

I already rebased it and am currently testing the new branch (tor-browser-builder-3 in my public repo) and am writing patches for tor-browser-bundle to work with it (there is currently only one patch needed, for LXC mode). Should be ready by the end of next week.

Last edited 5 years ago by gk (previous) (diff)

comment:5 Changed 5 years ago by gk

LXC fails to build libxul on Linux deterministically (not sure about the other OSes yet), it seems:

$ diff -u <(xxd kvm/tor-browser_en-US/Browser/libxul.so) <(xxd lxc/tor-browser_en-US/Browser/libxul.so)
--- /dev/fd/63	2014-02-21 11:13:13.662143877 +0100
+++ /dev/fd/62	2014-02-21 11:13:13.622145008 +0100
@@ -18,8 +18,8 @@
 0000110: 0400 0000 52e5 7464 cc24 d101 cc34 d101  ....R.td.$...4..
 0000120: cc34 d101 34ab 1500 34ab 1500 0400 0000  .4..4...4.......
 0000130: 0100 0000 0400 0000 1400 0000 0300 0000  ................
-0000140: 474e 5500 1c29 877e a8d3 1375 43e3 104f  GNU..).~...uC..O
-0000150: 4664 abbd 5eee 827d 0310 0000 6314 0000  Fd..^..}....c...
+0000140: 474e 5500 c3d3 091b 82b6 e9bc 9bfa 36ce  GNU...........6.
+0000150: 38dc 8635 2a0d 3de9 0310 0000 6314 0000  8..5*.=.....c...
 0000160: 8705 0000 bb10 0000 9206 0000 500f 0000  ............P...
 0000170: 0000 0000 420d 0000 6c02 0000 0000 0000  ....B...l.......
 0000180: 7c01 0000 ba12 0000 6610 0000 5008 0000  |.......f...P...
@@ -2033804,8 +2033804,8 @@
 1f088b0: 0000 0000 0000 0000 0000 0000 4743 433a  ............GCC:
 1f088c0: 2028 5562 756e 7475 2034 2e34 2e33 2d34   (Ubuntu 4.4.3-4
 1f088d0: 7562 756e 7475 352e 3129 2034 2e34 2e33  ubuntu5.1) 4.4.3
-1f088e0: 006c 6962 7875 6c2e 736f 0000 00a7 dc75  .libxul.so.....u
-1f088f0: 8a00 2e73 6873 7472 7461 6200 2e6e 6f74  ...shstrtab..not
+1f088e0: 006c 6962 7875 6c2e 736f 0000 00e6 812c  .libxul.so.....,
+1f088f0: ba00 2e73 6873 7472 7461 6200 2e6e 6f74  ...shstrtab..not
 1f08900: 652e 676e 752e 6275 696c 642d 6964 002e  e.gnu.build-id..
 1f08910: 676e 752e 6861 7368 002e 6479 6e73 796d  gnu.hash..dynsym
 1f08920: 002e 6479 6e73 7472 002e 676e 752e 7665  ..dynstr..gnu.ve

comment:6 Changed 5 years ago by gk

The first difference is due to a build id mismatch: libxul built in KVM has 1c29877ea8d3137543e3104f4664abbd5eee827d and libxul built in LXC has c3d3091b82b6e9bc9bfa36ce38dc86352a0d3de9. This is very likely caused by the second difference.

comment:7 Changed 5 years ago by gk

Actually that is wrong. The second difference is a different CRC32 checksum in the .gnu_debuglink section which is caused by the same build id in the corresponding debug file. So, the questions boil then down to:

1) How could the different build ID get created in the first place, given that the contents of the libxul libs don't differ?
2) What can we do about that problem?

ad 1) I don't know, honestly. The default method to generate a build ID is SHA1 which should *not* emit different hashes given the same library contents.
ad 2) A workaround is disabling the build ID in the libraries. We don't need it if we use the debuglink method (as well) (See: https://sourceware.org/gdb/onlinedocs/gdb/Separate-Debug-Files.html). This will be done in a different bug, though.

comment:8 Changed 5 years ago by gk

Keywords: MikePerry201402R added; MikePerry201403 removed
Status: newneeds_review

I built all bundles successfully with LXC while still being able to build with KVM. Thus, the branch is ready for review. Tagging needs to be done anyway by Mike due to the signature verification...

comment:9 Changed 5 years ago by mikeperry

Resolution: fixed
Status: needs_reviewclosed

This is merged. If anything breaks, we can switch the tag back in the versions file and use the old gitian code easily enough.

I have not yet merged the version file updates though. Right now they are in my branch for #4261.

Note: See TracTickets for help on using tickets.