Skip to content
Snippets Groups Projects
New look: Off

Rebase our Gitian onto latest upstream

    • Closed Issue created by Mike Perry @mikeperry

    Arlo pointed out that some LXC bugs and other issues affecting pluggable transport bundles have been fixed upstream in Gitian. We should rebase

    To upload designs, you'll need to enable LFS and have an admin enable hashed storage. More information

    Child items 0

    • No child items are currently assigned. Use child items to break down this issue into smaller parts.

    Activity

    • Mike Perry added MikePerry201402R component::applications/tor bundles/installation gitian owner::mikeperry priority::high resolution::fixed status::closed type::enhancement labels

      added MikePerry201402R component::applications/tor bundles/installation gitian owner::mikeperry priority::high resolution::fixed status::closed type::enhancement labels

    • Mike Perry
      Mike Perry @mikeperry ·
      Author

      Trac:
      Keywords: MikePerry201311 deleted, MikePerry201401 added

    • Georg Koppen
      Georg Koppen @gk ·

      Rebasing is necessary to fix #10558 (closed) (but alas not sufficient).

    • Mike Perry
      Mike Perry @mikeperry ·
      Author

      gk - I probably won't get to this this month. If you feel like doing this, don't let me stop you. Otherwise maybe I will pick it up in March.

      Trac:
      Keywords: MikePerry201401 deleted, MikePerry201403 added

    • Georg Koppen
      Georg Koppen @gk ·

      I already rebased it and am currently testing the new branch (tor-browser-builder-3 in my public repo) and am writing patches for tor-browser-bundle to work with it (there is currently only one patch needed, for LXC mode). Should be ready by the end of next week.

    • Georg Koppen
      Georg Koppen @gk ·

      LXC fails to build libxul on Linux deterministically (not sure about the other OSes yet), it seems:

      $ diff -u <(xxd kvm/tor-browser_en-US/Browser/libxul.so) <(xxd lxc/tor-browser_en-US/Browser/libxul.so)
--- /dev/fd/63	2014-02-21 11:13:13.662143877 +0100
+++ /dev/fd/62	2014-02-21 11:13:13.622145008 +0100
@@ -18,8 +18,8 @@
 0000110: 0400 0000 52e5 7464 cc24 d101 cc34 d101  ....R.td.$...4..
 0000120: cc34 d101 34ab 1500 34ab 1500 0400 0000  .4..4...4.......
 0000130: 0100 0000 0400 0000 1400 0000 0300 0000  ................
-0000140: 474e 5500 1c29 877e a8d3 1375 43e3 104f  GNU..).~...uC..O
-0000150: 4664 abbd 5eee 827d 0310 0000 6314 0000  Fd..^..}....c...
+0000140: 474e 5500 c3d3 091b 82b6 e9bc 9bfa 36ce  GNU...........6.
+0000150: 38dc 8635 2a0d 3de9 0310 0000 6314 0000  8..5*.=.....c...
 0000160: 8705 0000 bb10 0000 9206 0000 500f 0000  ............P...
 0000170: 0000 0000 420d 0000 6c02 0000 0000 0000  ....B...l.......
 0000180: 7c01 0000 ba12 0000 6610 0000 5008 0000  |.......f...P...
@@ -2033804,8 +2033804,8 @@
 1f088b0: 0000 0000 0000 0000 0000 0000 4743 433a  ............GCC:
 1f088c0: 2028 5562 756e 7475 2034 2e34 2e33 2d34   (Ubuntu 4.4.3-4
 1f088d0: 7562 756e 7475 352e 3129 2034 2e34 2e33  ubuntu5.1) 4.4.3
-1f088e0: 006c 6962 7875 6c2e 736f 0000 00a7 dc75  .libxul.so.....u
-1f088f0: 8a00 2e73 6873 7472 7461 6200 2e6e 6f74  ...shstrtab..not
+1f088e0: 006c 6962 7875 6c2e 736f 0000 00e6 812c  .libxul.so.....,
+1f088f0: ba00 2e73 6873 7472 7461 6200 2e6e 6f74  ...shstrtab..not
 1f08900: 652e 676e 752e 6275 696c 642d 6964 002e  e.gnu.build-id..
 1f08910: 676e 752e 6861 7368 002e 6479 6e73 796d  gnu.hash..dynsym
 1f08920: 002e 6479 6e73 7472 002e 676e 752e 7665  ..dynstr..gnu.ve
    • Georg Koppen
      Georg Koppen @gk ·

      The first difference is due to a build id mismatch: libxul built in KVM has 1c29877ea8d3137543e3104f4664abbd5eee827d and libxul built in LXC has c3d3091b82b6e9bc9bfa36ce38dc86352a0d3de9. This is very likely caused by the second difference.

    • Georg Koppen
      Georg Koppen @gk ·

      Actually that is wrong. The second difference is a different CRC32 checksum in the .gnu_debuglink section which is caused by the same build id in the corresponding debug file. So, the questions boil then down to:

      1. How could the different build ID get created in the first place, given that the contents of the libxul libs don't differ?
      2. What can we do about that problem?

      ad 1) I don't know, honestly. The default method to generate a build ID is SHA1 which should not emit different hashes given the same library contents. ad 2) A workaround is disabling the build ID in the libraries. We don't need it if we use the debuglink method (as well) (See: https://sourceware.org/gdb/onlinedocs/gdb/Separate-Debug-Files.html). This will be done in a different bug, though.

    • Georg Koppen
      Georg Koppen @gk ·

      I built all bundles successfully with LXC while still being able to build with KVM. Thus, the branch is ready for review. Tagging needs to be done anyway by Mike due to the signature verification...

      Trac:
      Status: new to needs_review
      Keywords: MikePerry201403 deleted, MikePerry201402R added

    • Mike Perry
      Mike Perry @mikeperry ·
      Author

      This is merged. If anything breaks, we can switch the tag back in the versions file and use the old gitian code easily enough.

      I have not yet merged the version file updates though. Right now they are in my branch for #4261 (closed).

      Trac:
      Resolution: N/A to fixed
      Status: needs_review to closed

    • Trac closed

      closed

    • Georg Koppen mentioned in issue #10558 (closed)

      mentioned in issue #10558 (closed)

    • Georg Koppen mentioned in issue #11042 (closed)

      mentioned in issue #11042 (closed)

    • Georg Koppen mentioned in issue #11089 (moved)

      mentioned in issue #11089 (moved)

    • Trac mentioned in issue tpo/applications/tor-browser#11089 (closed)

      mentioned in issue tpo/applications/tor-browser#11089 (closed)

    Please register or sign in to reply