Opened 3 years ago

Closed 2 years ago

#20426 closed task (fixed)

Getting rbm-based Tor Browser builds reproducible for all platforms

Reported by: boklm Owned by: tbb-team
Priority: Medium Milestone:
Component: Applications/Tor Browser Version:
Severity: Normal Keywords: tbb-gitian
Cc: Actual Points:
Parent ID: #17379 Points:
Reviewer: Sponsor:

Description

After converting gitian descriptors to rbm in #17380, we should fix any issue that makes the build non-deterministic. This will probably involve adding libfaketime in some parts.

Child Tickets

Attachments (2)

feedconverter.diff (13.8 KB) - added by gk 3 years ago.
libevent.diff (11.9 KB) - added by gk 3 years ago.

Download all attachments as: .zip

Change History (19)

comment:1 Changed 3 years ago by boklm

When building twice on the same host (I did not try building on different hosts yet), the Linux and OSX builds are now reproducible.

For the Windows build, we still have one file in the bundle that is not matching: Browser/TorBrowser/Tor/tor.exe, with the following diff:

--- tmp/1/Browser/TorBrowser/Tor/tor.exe
+++ tmp/2/Browser/TorBrowser/Tor/tor.exe
@@ -2,20 +2,20 @@
│  0000010: b800 0000 0000 0000 4000 0000 0000 0000  ........@.......
│  0000020: 0000 0000 0000 0000 0000 0000 0000 0000  ................
│  0000030: 0000 0000 0000 0000 0000 0000 8000 0000  ................
│  0000040: 0e1f ba0e 00b4 09cd 21b8 014c cd21 5468  ........!..L.!Th
│  0000050: 6973 2070 726f 6772 616d 2063 616e 6e6f  is program canno
│  0000060: 7420 6265 2072 756e 2069 6e20 444f 5320  t be run in DOS 
│  0000070: 6d6f 6465 2e0d 0d0a 2400 0000 0000 0000  mode....$.......
│ -0000080: 5045 0000 4c01 0800 d497 a458 0000 0000  PE..L......X....
│ +0000080: 5045 0000 4c01 0800 e597 a458 0000 0000  PE..L......X....
│  0000090: 0000 0000 e000 0e03 0b01 0216 00c6 2500  ..............%.
│  00000a0: 00fc 3100 004a 0000 c014 0000 0010 0000  ..1..J..........
│  00000b0: 00e0 2500 0000 4000 0010 0000 0002 0000  ..%...@.........
│  00000c0: 0400 0000 0100 0000 0400 0000 0000 0000  ................
│ -00000d0: 00b0 3200 0004 0000 f1d9 3200 0200 4001  ..2.......2...@.
│ +00000d0: 00b0 3200 0004 0000 02da 3200 0200 4001  ..2.......2...@.
│  00000e0: 0000 2000 0010 0000 0000 1000 0010 0000  .. .............
│  00000f0: 0000 0000 1000 0000 0000 0000 0000 0000  ................
│  0000100: 00d0 3000 3833 0000 0000 0000 0000 0000  ..0.83..........
│  0000110: 0000 0000 0000 0000 0000 0000 0000 0000  ................
│  0000120: 0030 3100 a078 0100 0000 0000 0000 0000  .01..x..........
│  0000130: 0000 0000 0000 0000 0000 0000 0000 0000  ................
│  0000140: 0420 3100 1800 0000 0000 0000 0000 0000  . 1.............
│ 

It doesn't look like a timestamp issue as adding faketime there is not fixing the problem.

comment:2 in reply to:  1 Changed 3 years ago by cypherpunks

comment:3 Changed 3 years ago by boklm

To check the status of reproducibility, I did a build using commit bfc288d54a28f00e088ba9ce08bd718156d8ff2f on two machines, and legind did one too.

The OSX bundles matched in all 3 builds. The Windows bundles didn't match because of a small diff in tor.exe (similar to the one above). The linux32 bundles matched in all 3 builds, but the linux64 ones only matched in 2 of the builds, with a diff in libxul.so. I did not yet investigate the reason for the diff in one of the linux64 builds.

comment:4 Changed 3 years ago by cypherpunks

comment:5 in reply to:  4 ; Changed 3 years ago by boklm

Replying to cypherpunks:

Why don't you build tor.exe for Win with libfaketime like in https://gitweb.torproject.org/builders/tor-browser-bundle.git/tree/gitian/descriptors/windows/gitian-tor.yml?

I tried, but it did not fix the problem. Maybe I did not do it correctly, so I will try again to make sure.

comment:6 in reply to:  5 Changed 3 years ago by cypherpunks

Replying to boklm:

Replying to cypherpunks:

Why don't you build tor.exe for Win with libfaketime like in https://gitweb.torproject.org/builders/tor-browser-bundle.git/tree/gitian/descriptors/windows/gitian-tor.yml?

I tried, but it did not fix the problem. Maybe I did not do it correctly, so I will try again to make sure.

With

-  export LD_PRELOAD=/usr/lib/faketime/libfaketime.so.1
+  export LD_PRELOAD=/usr/lib/x86_64-linux-gnu/faketime/libfaketime.so.1

?

comment:7 in reply to:  5 Changed 3 years ago by boklm

Replying to boklm:

I tried, but it did not fix the problem. Maybe I did not do it correctly, so I will try again to make sure.

I probably made an error when I tried to use libfaketime the previous time. After adding libfaketime (in commit 8cc61d29d348e154f43474b47a0b0746301602a9) and setting timestamp on files (commit 2a15e10d49e437dd3637136692a86a9473c78e6d) the Windows builds are now reproductible.

comment:8 Changed 3 years ago by gk

Summary: Getting rbm-based Tor Browser builds reproductible for all platformsGetting rbm-based Tor Browser builds reproducible for all platforms

Okay some preliminary results with commit cd08d1ab0fb638daed52ee5e7452dd78b07591c6. I just looked at the linux* ar files and compared them with the alphas found on http://f4amtbsowhix7rrf.onion/tor-browser-builds/2017-04-10/. The omni.ja files differ it seems (and that is the only difference for the ar bundles). More exactly some of the binary JS modules differ. I attached a binary diff for FeedConverter.js.

Last edited 3 years ago by gk (previous) (diff)

Changed 3 years ago by gk

Attachment: feedconverter.diff added

comment:9 Changed 3 years ago by cypherpunks

Does setting appropriate flags of ticket:10026#comment:21 help?

comment:10 Changed 3 years ago by gk

The Windows ar bundle only hits #21837 which is good.

comment:11 Changed 3 years ago by gk

Okay, for completeness sake the results when checking the ar macOS bundle:

1) TorBrowser.app/Contents/Resources/browser/omni.ja is different from mine. But the contents after extraction are the same.

2) libevent-2.0.5.dylib show differences which I have attached.

Changed 3 years ago by gk

Attachment: libevent.diff added

comment:12 Changed 3 years ago by boklm

The diff in libevent-2.0.5.dylib should be fixed by commit d2c018a26ec61f85a2fa7a0f9a7360025ff0fb24 (using faketime to build it).

comment:13 Changed 3 years ago by cypherpunks

OK. So, direct question: why don't you use https://bugzilla.redhat.com/show_bug.cgi?id=1124342?

comment:14 in reply to:  13 ; Changed 3 years ago by boklm

Replying to cypherpunks:

OK. So, direct question: why don't you use https://bugzilla.redhat.com/show_bug.cgi?id=1124342?

We are building binutils with --enable-deterministic-archives:
https://gitweb.torproject.org/builders/tor-browser-build.git/tree/projects/binutils/config

comment:16 Changed 3 years ago by boklm

After building on 2 machines, with the patch to disable startup cache generation (for #21960), I get matching bundles for linux and windows.

The OSX one is not matching with one file difference: TorBrowser.app/Contents/Resources/TorBrowser/Tor/PluggableTransports/template-profile.meek-http-helper/meek-template-sha256sum.txt. I'm wondering if it could be some race condition as we are adding this file in the directory from which we compute the sha256sum.

comment:17 Changed 2 years ago by gk

Resolution: fixed
Status: newclosed

I think we are done here. Let's open new issues in separate tickets in case they pop up during further testing and the next alpha release building.

Note: See TracTickets for help on using tickets.