Deterministic builds for Instantbird
Deterministic builds for TBB was a major undertaking. This should not be any different for Instantbird but we should leverage the work already done.
Activity
Replying to boklm:
Some progress on this: the Linux builds are now reproducible, at least when built by two users on the same machine. We now need to try the build on a different machine to check that it also produce the same binaries, and look at the Windows and OSX builds.
How is this done? I don't want to end up having different systems we need to maintain given our restricted resources we have. It does not make sense that each sub project reinvents the wheel.
-
Replying to gk:
Replying to boklm:
Some progress on this: the Linux builds are now reproducible, at least when built by two users on the same machine. We now need to try the build on a different machine to check that it also produce the same binaries, and look at the Windows and OSX builds.
How is this done? I don't want to end up having different systems we need to maintain given our restricted resources we have. It does not make sense that each sub project reinvents the wheel.
We now have #17379 (moved) for thinking about having just one tool for our reproducible builds.
-
Replying to boklm:
Some progress on this: the Linux builds are now reproducible, at least when built by two users on the same machine. We now need to try the build on a different machine to check that it also produce the same binaries,
If you could past instructions here I'd be happy trying to match your build on my build infrastructure.
See https://gitweb.torproject.org/tor-messenger-build.git/tree/README for starters.
If you want to try to match a build, using commit 87b47d1d805194b1042b160880f69b12b8237803, running "make tor-messenger-linux-x86_64", I get the following file:
c7d63d5367045cd377f257cbec2d6a90e14fabf2d492f4972838debc3515a8f4 out/tor-messenger/unnamed-messenger-0.1.0b1-linux-x86_64-8b1071.tar.xzI am now building the other architectures.
-
After building on different machines, the linux builds (i686 and x86_64) are matching.
The OSX and Windows builds are not matching.
On OSX, after extracting the dmg files, the only difference is libotr.5.dylib. With differences like this:
--- build1/um/Contents/Resources/extensions/ctypes-otr@tormessenger/chrome/content/libotr.5.dylib +++ build2/um/Contents/Resources/extensions/ctypes-otr@tormessenger/chrome/content/libotr.5.dylib │ @@ -46091,15 +46091,15 @@ │ 000b40a0: 1010 1010 1010 1010 1010 1010 1010 1010 ................ │ 000b40b0: 1010 1010 1010 1010 1010 1010 1010 1010 ................ │ 000b40c0: 1010 1010 1010 1010 1010 1010 1010 1010 ................ │ 000b40d0: 2010 10d0 01d0 01b0 01b0 0110 2010 1010 ........... ... │ 000b40e0: 1010 10b0 0110 d001 30a0 0100 0000 0000 ........0....... │ 000b40f0: 0200 0000 6400 0000 0000 0000 0000 0000 ....d........... │ 000b4100: 2300 0000 6400 0000 0000 0000 0000 0000 #...d........... │ -000b4110: 2d00 0000 6603 0100 7a91 2956 0000 0000 -...f...z.)V.... │ +000b4110: 2d00 0000 6603 0100 ece9 2a56 0000 0000 -...f.....*V.... │ 000b4120: 0100 0000 2e01 0000 000d 0000 0000 0000 ................ │ 000b4130: 5d00 0000 2401 0000 000d 0000 0000 0000 ]...$........... │ 000b4140: 0100 0000 2400 0000 e000 0000 0000 0000 ....$........... │ 000b4150: 0100 0000 4e01 0000 e000 0000 0000 0000 ....N........... │ 000b4160: 0100 0000 2e01 0000 e00d 0000 0000 0000 ................ │ 000b4170: 7900 0000 2401 0000 e00d 0000 0000 0000 y...$........... │ 000b4180: 0100 0000 2400 0000 c000 0000 0000 0000 ....$........... │ @@ -46192,15 +46192,15 @@ │ 000b46f0: 0100 0000 2e01 0000 8029 0000 0000 0000 .........)...... │ 000b4700: 1a03 0000 2401 0000 8029 0000 0000 0000 ....$....)...... │ 000b4710: 0100 0000 2400 0000 9700 0000 0000 0000 ....$........... │ 000b4720: 0100 0000 4e01 0000 9700 0000 0000 0000 ....N........... │ 000b4730: 0100 0000 6401 0000 0000 0000 0000 0000 ....d........... │ 000b4740: 0200 0000 6400 0000 0000 0000 0000 0000 ....d........... │ 000b4750: 2603 0000 6400 0000 0000 0000 0000 0000 &...d........... │ -000b4760: 3003 0000 6603 0100 7a91 2956 0000 0000 0...f...z.)V.... │ +000b4760: 3003 0000 6603 0100 ece9 2a56 0000 0000 0...f.....*V.... │ 000b4770: 0100 0000 2e01 0000 202a 0000 0000 0000 ........ *...... │ 000b4780: 6003 0000 2401 0000 202a 0000 0000 0000 `...$... *...... │ 000b4790: 0100 0000 2400 0000 6000 0000 0000 0000 ....$...`....... │ 000b47a0: 0100 0000 4e01 0000 6000 0000 0000 0000 ....N...`....... │ 000b47b0: 0100 0000 2e01 0000 802a 0000 0000 0000 .........*...... │ 000b47c0: 8303 0000 2401 0000 802a 0000 0000 0000 ....$....*...... │ 000b47d0: 0100 0000 2400 0000 2001 0000 0000 0000 ....$... ....... │ @@ -46244,15 +46244,15 @@ │ 000b4a30: 0100 0000 2e01 0000 9036 0000 0000 0000 .........6...... │ 000b4a40: af04 0000 2401 0000 9036 0000 0000 0000 ....$....6...... │ 000b4a50: 0100 0000 2400 0000 d600 0000 0000 0000 ....$........... │ 000b4a60: 0100 0000 4e01 0000 d600 0000 0000 0000 ....N........... │ 000b4a70: 0100 0000 6401 0000 0000 0000 0000 0000 ....d........... │ 000b4a80: 0200 0000 6400 0000 0000 0000 0000 0000 ....d........... │ 000b4a90: c804 0000 6400 0000 0000 0000 0000 0000 ....d........... │ -000b4aa0: d004 0000 6603 0100 7a91 2956 0000 0000 ....f...z.)V.... │ +000b4aa0: d004 0000 6603 0100 ece9 2a56 0000 0000 ....f.....*V.... │ 000b4ab0: 0100 0000 2e01 0000 7037 0000 0000 0000 ........p7...... │ 000b4ac0: fe04 0000 2401 0000 7037 0000 0000 0000 ....$...p7...... │ 000b4ad0: 0100 0000 2400 0000 b000 0000 0000 0000 ....$........... │ 000b4ae0: 0100 0000 4e01 0000 b000 0000 0000 0000 ....N........... │ 000b4af0: 0100 0000 2e01 0000 2038 0000 0000 0000 ........ 8...... │ 000b4b00: 0905 0000 2401 0000 2038 0000 0000 0000 ....$... 8...... │ 000b4b10: 0100 0000 2400 0000 4000 0000 0000 0000 ....$...@....... │ @@ -46309,15 +46309,15 @@ │ 000b4e40: 6206 0000 2401 0000 7054 0000 0000 0000 b...$...pT...... │ 000b4e50: 0100 0000 2400 0000 5d01 0000 0000 0000 ....$...]....... │ 000b4e60: 0100 0000 4e01 0000 5d01 0000 0000 0000 ....N...]....... │ 000b4e70: 6f06 0000 2000 0000 0000 0000 0000 0000 o... ........... │ 000b4e80: 0100 0000 6401 0000 0000 0000 0000 0000 ....d........... │ 000b4e90: 0200 0000 6400 0000 0000 0000 0000 0000 ....d........... │ 000b4ea0: 8106 0000 6400 0000 0000 0000 0000 0000 ....d........... │ -000b4eb0: 8706 0000 6603 0100 7a91 2956 0000 0000 ....f...z.)V.... │ +000b4eb0: 8706 0000 6603 0100 ece9 2a56 0000 0000 ....f.....*V.... │ 000b4ec0: 0100 0000 2e01 0000 d055 0000 0000 0000 .........U...... │ 000b4ed0: b306 0000 2401 0000 d055 0000 0000 0000 ....$....U...... │ 000b4ee0: 0100 0000 2400 0000 1001 0000 0000 0000 ....$........... │ 000b4ef0: 0100 0000 4e01 0000 1001 0000 0000 0000 ....N........... │ 000b4f00: 0100 0000 2e01 0000 e056 0000 0000 0000 .........V...... │ 000b4f10: c706 0000 2401 0000 e056 0000 0000 0000 ....$....V...... │ 000b4f20: 0100 0000 2400 0000 8001 0000 0000 0000 ....$........... ...On the Windows builds, after extracting the installer files, some instantbird files are different:
Binary files winbuild1/$_OUTDIR/Messenger/components/trayToolkit.dll and winbuild2/$_OUTDIR/Messenger/components/trayToolkit.dll differ Binary files winbuild1/$_OUTDIR/Messenger/instantbird.exe and winbuild2/$_OUTDIR/Messenger/instantbird.exe differ Binary files winbuild1/$_OUTDIR/Messenger/libEGL.dll and winbuild2/$_OUTDIR/Messenger/libEGL.dll differ Binary files winbuild1/$_OUTDIR/Messenger/libGLESv2.dll and winbuild2/$_OUTDIR/Messenger/libGLESv2.dll differ Binary files winbuild1/$_OUTDIR/Messenger/mozglue.dll and winbuild2/$_OUTDIR/Messenger/mozglue.dll differ Binary files winbuild1/$_OUTDIR/Messenger/plugin-container.exe and winbuild2/$_OUTDIR/Messenger/plugin-container.exe differ Binary files winbuild1/$_OUTDIR/Messenger/xul.dll and winbuild2/$_OUTDIR/Messenger/xul.dll differThe files have differences like this (objdump -d diff):
--- o1.txt 2015-10-24 15:37:51.888043607 +0200 +++ o2.txt 2015-10-24 15:37:55.806058575 +0200 @@ -1,5 +1,5 @@ -winbuild1/$_OUTDIR/Messenger/libEGL.dll: file format pei-i386 +winbuild2/$_OUTDIR/Messenger/libEGL.dll: file format pei-i386 Disassembly of section .text: @@ -95,7 +95,7 @@ 64581127: 0f 84 17 01 00 00 je 0x64581244 6458112d: 85 db test %ebx,%ebx 6458112f: 0f 84 cb 00 00 00 je 0x64581200 -64581135: a1 88 fd 63 64 mov 0x6463fd88,%eax +64581135: a1 78 fd 63 64 mov 0x6463fd78,%eax 6458113a: 85 c0 test %eax,%eax 6458113c: 74 1c je 0x6458115a 6458113e: 8b 54 24 28 mov 0x28(%esp),%edx @@ -6618,7 +6618,7 @@ 64586a09: 77 3c ja 0x64586a47 64586a0b: 8b 45 0c mov 0xc(%ebp),%eax 64586a0e: c1 e0 02 shl $0x2,%eax -64586a11: 05 74 e8 63 64 add $0x6463e874,%eax +64586a11: 05 34 e8 63 64 add $0x6463e834,%eax 64586a16: 8b 00 mov (%eax),%eax 64586a18: ff e0 jmp *%eax 64586a1a: 83 7d 10 00 cmpl $0x0,0x10(%ebp) @@ -7034,7 +7034,7 @@ 64586eba: 8b 00 mov (%eax),%eax 64586ebc: 83 f8 33 cmp $0x33,%eax 64586ebf: 77 09 ja 0x64586eca -64586ec1: 8b 04 85 a4 e9 63 64 mov 0x6463e9a4(,%eax,4),%eax +64586ec1: 8b 04 85 64 e9 63 64 mov 0x6463e964(,%eax,4),%eax 64586ec8: ff e0 jmp *%eax 64586eca: b8 00 00 00 00 mov $0x0,%eax 64586ecf: eb 15 jmp 0x64586ee6 @@ -7253,7 +7253,7 @@ 64587154: 89 c3 mov %eax,%ebx 64587156: c7 44 24 08 03 00 00 movl $0x3,0x8(%esp) 6458715d: 00 -6458715e: c7 44 24 04 74 ea 63 movl $0x6463ea74,0x4(%esp) +6458715e: c7 44 24 04 34 ea 63 movl $0x6463ea34,0x4(%esp) 64587165: 64 64587166: 8b 45 08 mov 0x8(%ebp),%eax 64587169: 89 04 24 mov %eax,(%esp) @@ -7590,7 +7590,7 @@ 645875b0: 8b 45 f4 mov -0xc(%ebp),%eax 645875b3: 8b 40 04 mov 0x4(%eax),%eax 645875b6: 8b 00 mov (%eax),%eax -645875b8: c7 44 24 04 78 ea 63 movl $0x6463ea78,0x4(%esp) +645875b8: c7 44 24 04 38 ea 63 movl $0x6463ea38,0x4(%esp) 645875bf: 64 645875c0: 89 04 24 mov %eax,(%esp) 645875c3: e8 c8 3e 01 00 call 0x6459b490 @@ -7817,7 +7817,7 @@ 64587865: 0f 8e 8a 00 00 00 jle 0x645878f5 6458786b: c7 44 24 08 08 00 00 movl $0x8,0x8(%esp) 64587872: 00 -64587873: c7 44 24 04 7b ea 63 movl $0x6463ea7b,0x4(%esp) +64587873: c7 44 24 04 3b ea 63 movl $0x6463ea3b,0x4(%esp) 6458787a: 64 6458787b: 8b 45 f4 mov -0xc(%ebp),%eax 6458787e: 89 04 24 mov %eax,(%esp) @@ -7856,7 +7856,7 @@ 645878d5: 89 50 30 mov %edx,0x30(%eax) 645878d8: c7 44 24 08 15 00 00 movl $0x15,0x8(%esp) 645878df: 00 -645878e0: c7 44 24 04 84 ea 63 movl $0x6463ea84,0x4(%esp) +645878e0: c7 44 24 04 44 ea 63 movl $0x6463ea44,0x4(%esp) 645878e7: 64 645878e8: 8b 45 08 mov 0x8(%ebp),%eax 645878eb: 89 04 24 mov %eax,(%esp) @@ -7965,7 +7965,7 @@ 64587a31: 89 45 e0 mov %eax,-0x20(%ebp) 64587a34: 8b 45 e0 mov -0x20(%ebp),%eax 64587a37: c1 e0 04 shl $0x4,%eax -64587a3a: 05 40 ec 63 64 add $0x6463ec40,%eax +64587a3a: 05 00 ec 63 64 add $0x6463ec00,%eax 64587a3f: 89 45 dc mov %eax,-0x24(%ebp) 64587a42: 8b 45 dc mov -0x24(%ebp),%eax 64587a45: 8b 00 mov (%eax),%eax @@ -8234,7 +8234,7 @@ 64587d85: 83 e8 43 sub $0x43,%eax 64587d88: 83 f8 33 cmp $0x33,%eax 64587d8b: 0f 87 6d 03 00 00 ja 0x645880fe -64587d91: 8b 04 85 20 f0 63 64 mov 0x6463f020(,%eax,4),%eax +64587d91: 8b 04 85 e0 ef 63 64 mov 0x6463efe0(,%eax,4),%eax 64587d98: ff e0 jmp *%eax 64587d9a: 8b 45 08 mov 0x8(%ebp),%eax 64587d9d: 8b 40 30 mov 0x30(%eax),%eax @@ -8502,7 +8502,7 @@ 64588161: 83 e8 41 sub $0x41,%eax 64588164: 83 f8 31 cmp $0x31,%eax 64588167: 0f 87 4e 01 00 00 ja 0x645882bb -6458816d: 8b 04 85 f0 f0 63 64 mov 0x6463f0f0(,%eax,4),%eax +6458816d: 8b 04 85 b0 f0 63 64 mov 0x6463f0b0(,%eax,4),%eax 64588174: ff e0 jmp *%eax 64588176: 8b 45 08 mov 0x8(%ebp),%eax 64588179: 89 04 24 mov %eax,(%esp) As an update, the OS X
libotr.5.dylibdifference was resolved with the commit 412999 by settingfaketime(borrowed from tor-browser-build.git/projects/tor/build). (The difference was because of embedded timestamps in thelibgpg-errorandlibgcrypt)So while there are no files that differ for Windows and OS X on unpacking, the EXE and DMG still differ and it seems like the difference is at the packaging stage.
Trac:
Reviewer: N/A to N/A-
Replying to sukhbir:
So while there are no files that differ for Windows and OS X on unpacking, the EXE and DMG still differ and it seems like the difference is at the packaging stage.
macOS is reproducible as of https://gitweb.torproject.org/tor-messenger-build.git/commit/?id=40a17907e3422785c495e01bfb5c7c0f6b9e7e11
Windows is reproducible with https://gitweb.torproject.org/tor-messenger-build.git/commit/?id=9accac5b740962c85e044ae620789a156e3c17be ... marking this ticket as fixed since all platforms are now reproducible.
Trac:
Status: assigned to closed
Resolution: N/A to fixed<+sukhe> hello. yes, I think it's fine to close the tickets. thanks for doing what we should done earlier :)
sad but true: https://blog.torproject.org/sunsetting-tor-messenger
luckily there are alternatives: https://blog.torproject.org/tor-heart-onion-messaging
.. and maybe someday