Opened 3 months ago

Last modified 2 weeks ago

#26475 new defect

ESR60-based .dmg images are not built reproducibly with Stylo enabled using rustc > 1.25.0

Reported by: gk Owned by: tbb-team
Priority: Very High Milestone:
Component: Applications/Tor Browser Version:
Severity: Normal Keywords: tbb-rbm, GeorgKoppen201809, TorBrowserTeam201809
Cc: boklm, manishearth@…, acrichton@… Actual Points:
Parent ID: Points:
Reviewer: Sponsor:

Description

Contrary to our testing results the .dmg bundles we produce for 8.0a9 are differing on two machines.

Child Tickets

Attachments (1)

disassemble_gkrust-d3a9de07b53ab691.gkrust0.rcgu.o_diff (33.9 KB) - added by gk 2 months ago.

Download all attachments as: .zip

Change History (40)

comment:1 Changed 3 months ago by gk

Only XUL is different in our case:

--- /dev/fd/63	2018-06-24 08:54:28.175516977 +0200
+++ /dev/fd/62	2018-06-24 08:54:28.175516977 +0200
@@ -201,8 +201,8 @@
 00000c80: 0000 0000 0000 0000 0000 0000 0000 0000  ................
 00000c90: 0000 0000 0000 0000 104f 5805 a616 0000  .........OX.....
 00000ca0: 0000 0000 0000 0000 0000 0000 0000 0000  ................
-00000cb0: 1b00 0000 1800 0000 ddc1 d65d f3d8 3a65  ...........]..:e
-00000cc0: 8465 a64e d247 5b4d 2400 0000 1000 0000  .e.N.G[M$.......
+00000cb0: 1b00 0000 1800 0000 18a4 cc7a 95d7 325d  ...........z..2]
+00000cc0: 96c5 6f08 51ec 0b4e 2400 0000 1000 0000  ..o.Q..N$.......
 00000cd0: 0007 0a00 0006 0a00 0c00 0000 5800 0000  ............X...
 00000ce0: 1800 0000 0200 0000 0000 1600 0000 0100  ................
 00000cf0: 2f53 7973 7465 6d2f 4c69 6272 6172 792f  /System/Library/
@@ -3732284,59 +3732284,59 @@
 038f33b0: ff48 8b07 488b 4f08 4889 8d70 ffff ff48  .H..H.O.H..p...H
 038f33c0: 8985 68ff ffff 488d 051e 432a 0048 8985  ..h...H...C*.H..
 038f33d0: 38ff ffff 48c7 8540 ffff ff06 0000 0048  8...H..@.......H
-038f33e0: 8d3d 9adf bf01 ff17 8078 2100 0f85 da01  .=.......x!.....
+038f33e0: 8d3d 9adf bf01 ff17 8078 2100 0f85 d801  .=.......x!.....
 038f33f0: 0000 488d 3d87 dfbf 01ff 1780 7820 0075  ..H.=.......x .u
 038f3400: 1f48 8d3d 78df bf01 ff17 4889 c348 8d3d  .H.=x.....H..H.=
 038f3410: fc95 ffff 4889 dee8 c0c1 0100 c643 2001  ....H........C .
-038f3420: 488d 3d59 dfbf 01ff 1748 8338 0174 6748  H.=Y.....H.8.tgH
+038f3420: 488d 3d59 dfbf 01ff 1748 8338 0174 6548  H.=Y.....H.8.teH
 038f3430: 8d3d 4adf bf01 ff17 660f 6f40 100f 57c9  .=J.....f.o@..W.
-038f3440: 0f29 4810 b901 0000 0066 480f 6ec9 488b  .)H......fH.n.H.
-038f3450: 0866 0f7f 0848 85c9 7429 6648 0f7e c348  .f...H..t)fH.~.H
-038f3460: 85db 741f 660f 70c0 4e66 490f 7ec6 4889  ..t.f.p.NfI.~.H.
-038f3470: df41 ff16 4983 7e08 0074 0848 89df e8c7  .A..I.~..t.H....
-038f3480: b001 0048 8d3d f6de bf01 ff17 4883 3801  ...H.=......H.8.
-038f3490: 0f85 8102 0000 488d 3de3 debf 01ff 1748  ......H.=......H
-038f34a0: 8378 0800 7477 e895 f2ff ff48 89c3 4889  .x..tw.....H..H.
-038f34b0: 9d48 ffff ff48 8b45 9048 8945 d048 8b45  .H...H.E.H.E.H.E
-038f34c0: 8848 8945 c848 8b45 8048 8945 c048 8b85  .H.E.H.E.H.E.H..
-038f34d0: 78ff ffff 4889 45b8 488b 8568 ffff ff48  x...H.E.H..h...H
-038f34e0: 8b8d 70ff ffff 4889 4db0 4889 45a8 488d  ..p...H.M.H.E.H.
-038f34f0: 7dd8 488d b548 ffff ff48 8d55 a8e8 8ef9  }.H..H...H.U....
-038f3500: ffff f048 ff0b 0f85 a100 0000 488d bd48  ...H........H..H
-038f3510: ffff ffe8 58c0 ffff e990 0000 0048 8d3d  ....X........H.=
-038f3520: 5cde bf01 ff17 48c7 4008 ffff ffff 488d  \.....H.@.....H.
-038f3530: 4808 488d 5010 4889 9548 ffff ff48 898d  H.H.P.H..H...H..
-038f3540: 50ff ffff 488b 7010 4885 f60f 84b0 0100  P...H.p.H.......
-038f3550: 0048 8d3d 28de bf01 ff17 4889 c348 8b45  .H.=(.....H..H.E
-038f3560: 9048 8945 d048 8b43 1848 8b4d 8848 894d  .H.E.H.C.H.M.H.M
-038f3570: c848 8b4d 8048 894d c048 8b8d 78ff ffff  .H.M.H.M.H..x...
-038f3580: 4889 4db8 488b 8d68 ffff ff48 8b95 70ff  H.M.H..h...H..p.
-038f3590: ffff 4889 55b0 4889 4da8 488d 7dd8 488d  ..H.U.H.M.H.}.H.
-038f35a0: 55a8 ff50 3048 c743 0800 0000 008b 45d9  U..P0H.C......E.
-038f35b0: 8945 e88a 45d8 0fb7 4ddd 6689 4dec 8a4d  .E..E...M.f.M..M
-038f35c0: df88 4dee 3c04 0f85 8000 0000 e86f f1ff  ..M.<........o..
-038f35d0: ff48 89c3 4889 9d48 ffff ff48 8b45 9048  .H..H..H...H.E.H
-038f35e0: 8945 d048 8b45 8848 8945 c848 8b45 8048  .E.H.E.H.E.H.E.H
-038f35f0: 8945 c048 8b85 78ff ffff 4889 45b8 488b  .E.H..x...H.E.H.
-038f3600: 8568 ffff ff48 8b8d 70ff ffff 4889 4db0  .h...H..p...H.M.
-038f3610: 4889 45a8 488d 7d98 488d b548 ffff ff48  H.E.H.}.H..H...H
-038f3620: 8d55 a8e8 68f8 ffff f048 ff0b 750c 488d  .U..h....H..u.H.
-038f3630: bd48 ffff ffe8 36bf ffff 807d 9803 7531  .H....6....}..u1
-038f3640: 4881 c4c0 0000 005b 415e 5dc3 8845 9848  H......[A^]..E.H
-038f3650: 8b45 e08b 4de8 894d 990f b74d ec66 894d  .E..M..M...M.f.M
-038f3660: 9d8a 4dee 884d 9f48 8945 a080 7d98 0374  ..M..M.H.E..}..t
-038f3670: cf48 8b45 9848 8b4d a048 894d e048 8945  .H.E.H.M.H.M.H.E
-038f3680: d848 8d85 38ff ffff 4889 8548 ffff ff48  .H..8...H..H...H
-038f3690: 8d05 dab6 ffff 4889 8550 ffff ff48 8d45  ......H..P...H.E
-038f36a0: d848 8985 58ff ffff 488d 0501 99ff ff48  .H..X...H......H
-038f36b0: 8985 60ff ffff 488d 052b a7b6 0148 8945  ..`...H..+...H.E
-038f36c0: a848 c745 b002 0000 0048 8d05 f863 2a00  .H.E.....H...c*.
-038f36d0: 4889 45b8 48c7 45c0 0200 0000 488d 8548  H.E.H.E.....H..H
-038f36e0: ffff ff48 8945 c848 c745 d002 0000 0048  ...H.E.H.E.....H
-038f36f0: 8d35 12a7 b601 488d 7da8 e881 e5ff ff0f  .5....H.}.......
-038f3700: 0b48 8d3d 78dc bf01 ff17 48c7 4008 0000  .H.=x.....H.@...
-038f3710: 0000 e98f fdff ff48 8d3d eabb b601 e82d  .......H.=.....-
-038f3720: 8f00 000f 0b66 662e 0f1f 8400 0000 0000  .....ff.........
+038f3440: 0f29 4810 b901 0000 0066 480f 6ec9 4883  .)H......fH.n.H.
+038f3450: 3800 660f 7f08 7429 6648 0f7e c348 85db  8.f...t)fH.~.H..
+038f3460: 741f 660f 70c0 4e66 490f 7ec6 4889 df41  t.f.p.NfI.~.H..A
+038f3470: ff16 4983 7e08 0074 0848 89df e8c9 b001  ..I.~..t.H......
+038f3480: 0048 8d3d f8de bf01 ff17 4883 3801 0f85  .H.=......H.8...
+038f3490: 8102 0000 488d 3de5 debf 01ff 1748 8378  ....H.=......H.x
+038f34a0: 0800 7477 e897 f2ff ff48 89c3 4889 9d48  ..tw.....H..H..H
+038f34b0: ffff ff48 8b45 9048 8945 d048 8b45 8848  ...H.E.H.E.H.E.H
+038f34c0: 8945 c848 8b45 8048 8945 c048 8b85 78ff  .E.H.E.H.E.H..x.
+038f34d0: ffff 4889 45b8 488b 8568 ffff ff48 8b8d  ..H.E.H..h...H..
+038f34e0: 70ff ffff 4889 4db0 4889 45a8 488d 7dd8  p...H.M.H.E.H.}.
+038f34f0: 488d b548 ffff ff48 8d55 a8e8 90f9 ffff  H..H...H.U......
+038f3500: f048 ff0b 0f85 a100 0000 488d bd48 ffff  .H........H..H..
+038f3510: ffe8 5ac0 ffff e990 0000 0048 8d3d 5ede  ..Z........H.=^.
+038f3520: bf01 ff17 48c7 4008 ffff ffff 488d 4808  ....H.@.....H.H.
+038f3530: 488d 5010 4889 9548 ffff ff48 898d 50ff  H.P.H..H...H..P.
+038f3540: ffff 488b 7010 4885 f60f 84b0 0100 0048  ..H.p.H........H
+038f3550: 8d3d 2ade bf01 ff17 4889 c348 8b45 9048  .=*.....H..H.E.H
+038f3560: 8945 d048 8b43 1848 8b4d 8848 894d c848  .E.H.C.H.M.H.M.H
+038f3570: 8b4d 8048 894d c048 8b8d 78ff ffff 4889  .M.H.M.H..x...H.
+038f3580: 4db8 488b 8d68 ffff ff48 8b95 70ff ffff  M.H..h...H..p...
+038f3590: 4889 55b0 4889 4da8 488d 7dd8 488d 55a8  H.U.H.M.H.}.H.U.
+038f35a0: ff50 3048 c743 0800 0000 008b 45d9 8945  .P0H.C......E..E
+038f35b0: e88a 45d8 0fb7 4ddd 6689 4dec 8a4d df88  ..E...M.f.M..M..
+038f35c0: 4dee 3c04 0f85 8000 0000 e871 f1ff ff48  M.<........q...H
+038f35d0: 89c3 4889 9d48 ffff ff48 8b45 9048 8945  ..H..H...H.E.H.E
+038f35e0: d048 8b45 8848 8945 c848 8b45 8048 8945  .H.E.H.E.H.E.H.E
+038f35f0: c048 8b85 78ff ffff 4889 45b8 488b 8568  .H..x...H.E.H..h
+038f3600: ffff ff48 8b8d 70ff ffff 4889 4db0 4889  ...H..p...H.M.H.
+038f3610: 45a8 488d 7d98 488d b548 ffff ff48 8d55  E.H.}.H..H...H.U
+038f3620: a8e8 6af8 ffff f048 ff0b 750c 488d bd48  ..j....H..u.H..H
+038f3630: ffff ffe8 38bf ffff 807d 9803 7531 4881  ....8....}..u1H.
+038f3640: c4c0 0000 005b 415e 5dc3 8845 9848 8b45  .....[A^]..E.H.E
+038f3650: e08b 4de8 894d 990f b74d ec66 894d 9d8a  ..M..M...M.f.M..
+038f3660: 4dee 884d 9f48 8945 a080 7d98 0374 cf48  M..M.H.E..}..t.H
+038f3670: 8b45 9848 8b4d a048 894d e048 8945 d848  .E.H.M.H.M.H.E.H
+038f3680: 8d85 38ff ffff 4889 8548 ffff ff48 8d05  ..8...H..H...H..
+038f3690: dcb6 ffff 4889 8550 ffff ff48 8d45 d848  ....H..P...H.E.H
+038f36a0: 8985 58ff ffff 488d 0503 99ff ff48 8985  ..X...H......H..
+038f36b0: 60ff ffff 488d 052d a7b6 0148 8945 a848  `...H..-...H.E.H
+038f36c0: c745 b002 0000 0048 8d05 fa63 2a00 4889  .E.....H...c*.H.
+038f36d0: 45b8 48c7 45c0 0200 0000 488d 8548 ffff  E.H.E.....H..H..
+038f36e0: ff48 8945 c848 c745 d002 0000 0048 8d35  .H.E.H.E.....H.5
+038f36f0: 14a7 b601 488d 7da8 e883 e5ff ff0f 0b48  ....H.}........H
+038f3700: 8d3d 7adc bf01 ff17 48c7 4008 0000 0000  .=z.....H.@.....
+038f3710: e98f fdff ff48 8d3d ecbb b601 e82f 8f00  .....H.=...../..
+038f3720: 000f 0b66 6666 662e 0f1f 8400 0000 0000  ...ffff.........
 038f3730: 5548 89e5 803f 0074 0e48 8d05 f23f 2a00  UH...?.t.H...?*.
 038f3740: ba2b 0000 00eb 0c48 8d05 c83f 2a00 ba1c  .+.....H...?*...
 038f3750: 0000 0048 89f7 4889 c65d e951 b700 0090  ...H..H..].Q....
@@ -4578681,7 +4578681,7 @@
 045dd780: 2400 0000 fcc0 0200 a85b 31ff ffff ffff  $........[1.....
 045dd790: 4800 0000 0000 0000 0041 0e10 8602 430d  H........A....C.
 045dd7a0: 0643 8304 8e03 0000 2400 0000 24c1 0200  .C......$...$...
-045dd7b0: d05b 31ff ffff ffff a503 0000 0000 0000  .[1.............
+045dd7b0: d05b 31ff ffff ffff a303 0000 0000 0000  .[1.............
 045dd7c0: 0041 0e10 8602 430d 064a 8304 8e03 0000  .A....C..J......
 045dd7d0: 2400 0000 4cc1 0200 585f 31ff ffff ffff  $...L...X_1.....
 045dd7e0: 2f00 0000 0000 0000 0041 0e10 8602 430d  /........A....C.

comment:2 Changed 3 months ago by boklm

Maybe comparing the obj-macos directory from two builds can help debug the issue.

I have started a build with this patch:
https://gitweb.torproject.org/user/boklm/tor-browser-build.git/commit/?h=bug_26475&id=1ceef284b016441783d276ba32542e9fcd9bfd75

with the command:

$ ./rbm/rbm build firefox --target alpha --target torbrowser-osx-x86_64

And will upload the result when it finish.

comment:3 in reply to:  2 Changed 3 months ago by boklm

comment:4 Changed 3 months ago by gk

I compared two obj-macos dirs and gkrust seems to be the culprit (as assumed).

diff -u <(xxd tpo/gkrust-d3a9de07b53ab691.gkrust0.rcgu.o) <(xxd uninett/gkrust-d3a9de07b53ab691.gkrust0.rcgu.o)

gives me among others

 00440e20: ff48 8b07 488b 4f08 4889 8d70 ffff ff48  .H..H.O.H..p...H
 00440e30: 8985 68ff ffff 488d 0500 0000 0048 8985  ..h...H......H..
 00440e40: 38ff ffff 48c7 8540 ffff ff06 0000 0048  8...H..@.......H
-00440e50: 8b3d 0000 0000 ff17 8078 2100 0f85 da01  .=.......x!.....
+00440e50: 8b3d 0000 0000 ff17 8078 2100 0f85 d801  .=.......x!.....
 00440e60: 0000 488b 3d00 0000 00ff 1780 7820 0075  ..H.=.......x .u
 00440e70: 1f48 8b3d 0000 0000 ff17 4889 c348 8d3d  .H.=......H..H.=
 00440e80: 0000 0000 4889 dee8 0000 0000 c643 2001  ....H........C .
-00440e90: 488b 3d00 0000 00ff 1748 8338 0174 6748  H.=......H.8.tgH
+00440e90: 488b 3d00 0000 00ff 1748 8338 0174 6548  H.=......H.8.teH
 00440ea0: 8b3d 0000 0000 ff17 660f 6f40 100f 57c9  .=......f.o@..W.
-00440eb0: 0f29 4810 b901 0000 0066 480f 6ec9 488b  .)H......fH.n.H.
-00440ec0: 0866 0f7f 0848 85c9 7429 6648 0f7e c348  .f...H..t)fH.~.H
-00440ed0: 85db 741f 660f 70c0 4e66 490f 7ec6 4889  ..t.f.p.NfI.~.H.
-00440ee0: df41 ff16 4983 7e08 0074 0848 89df e800  .A..I.~..t.H....
-00440ef0: 0000 0048 8b3d 0000 0000 ff17 4883 3801  ...H.=......H.8.
-00440f00: 0f85 8102 0000 488b 3d00 0000 00ff 1748  ......H.=......H
-00440f10: 8378 0800 7477 e800 0000 0048 89c3 4889  .x..tw.....H..H.
-00440f20: 9d48 ffff ff48 8b45 9048 8945 d048 8b45  .H...H.E.H.E.H.E
-00440f30: 8848 8945 c848 8b45 8048 8945 c048 8b85  .H.E.H.E.H.E.H..
-00440f40: 78ff ffff 4889 45b8 488b 8568 ffff ff48  x...H.E.H..h...H
-00440f50: 8b8d 70ff ffff 4889 4db0 4889 45a8 488d  ..p...H.M.H.E.H.
-00440f60: 7dd8 488d b548 ffff ff48 8d55 a8e8 0000  }.H..H...H.U....
-00440f70: 0000 f048 ff0b 0f85 a100 0000 488d bd48  ...H........H..H
-00440f80: ffff ffe8 0000 0000 e990 0000 0048 8b3d  .............H.=
-00440f90: 0000 0000 ff17 48c7 4008 ffff ffff 488d  ......H.@.....H.
-00440fa0: 4808 488d 5010 4889 9548 ffff ff48 898d  H.H.P.H..H...H..
-00440fb0: 50ff ffff 488b 7010 4885 f60f 84b0 0100  P...H.p.H.......
-00440fc0: 0048 8b3d 0000 0000 ff17 4889 c348 8b45  .H.=......H..H.E
-00440fd0: 9048 8945 d048 8b43 1848 8b4d 8848 894d  .H.E.H.C.H.M.H.M
-00440fe0: c848 8b4d 8048 894d c048 8b8d 78ff ffff  .H.M.H.M.H..x...
-00440ff0: 4889 4db8 488b 8d68 ffff ff48 8b95 70ff  H.M.H..h...H..p.
-00441000: ffff 4889 55b0 4889 4da8 488d 7dd8 488d  ..H.U.H.M.H.}.H.
-00441010: 55a8 ff50 3048 c743 0800 0000 008b 45d9  U..P0H.C......E.
-00441020: 8945 e88a 45d8 0fb7 4ddd 6689 4dec 8a4d  .E..E...M.f.M..M
-00441030: df88 4dee 3c04 0f85 8000 0000 e800 0000  ..M.<...........
-00441040: 0048 89c3 4889 9d48 ffff ff48 8b45 9048  .H..H..H...H.E.H
-00441050: 8945 d048 8b45 8848 8945 c848 8b45 8048  .E.H.E.H.E.H.E.H
-00441060: 8945 c048 8b85 78ff ffff 4889 45b8 488b  .E.H..x...H.E.H.
-00441070: 8568 ffff ff48 8b8d 70ff ffff 4889 4db0  .h...H..p...H.M.
-00441080: 4889 45a8 488d 7d98 488d b548 ffff ff48  H.E.H.}.H..H...H
-00441090: 8d55 a8e8 0000 0000 f048 ff0b 750c 488d  .U.......H..u.H.
-004410a0: bd48 ffff ffe8 0000 0000 807d 9803 7531  .H.........}..u1
-004410b0: 4881 c4c0 0000 005b 415e 5dc3 8845 9848  H......[A^]..E.H
-004410c0: 8b45 e08b 4de8 894d 990f b74d ec66 894d  .E..M..M...M.f.M
-004410d0: 9d8a 4dee 884d 9f48 8945 a080 7d98 0374  ..M..M.H.E..}..t
-004410e0: cf48 8b45 9848 8b4d a048 894d e048 8945  .H.E.H.M.H.M.H.E
-004410f0: d848 8d85 38ff ffff 4889 8548 ffff ff48  .H..8...H..H...H
-00441100: 8d05 0000 0000 4889 8550 ffff ff48 8d45  ......H..P...H.E
-00441110: d848 8985 58ff ffff 488d 0500 0000 0048  .H..X...H......H
-00441120: 8985 60ff ffff 488d 0500 0000 0048 8945  ..`...H......H.E
-00441130: a848 c745 b002 0000 0048 8d05 0000 0000  .H.E.....H......
-00441140: 4889 45b8 48c7 45c0 0200 0000 488d 8548  H.E.H.E.....H..H
-00441150: ffff ff48 8945 c848 c745 d002 0000 0048  ...H.E.H.E.....H
-00441160: 8d35 0000 0000 488d 7da8 e800 0000 000f  .5....H.}.......
-00441170: 0b48 8b3d 0000 0000 ff17 48c7 4008 0000  .H.=......H.@...
-00441180: 0000 e98f fdff ff48 8d3d 0000 0000 e800  .......H.=......
-00441190: 0000 000f 0b66 662e 0f1f 8400 0000 0000  .....ff.........
+00440eb0: 0f29 4810 b901 0000 0066 480f 6ec9 4883  .)H......fH.n.H.
+00440ec0: 3800 660f 7f08 7429 6648 0f7e c348 85db  8.f...t)fH.~.H..
+00440ed0: 741f 660f 70c0 4e66 490f 7ec6 4889 df41  t.f.p.NfI.~.H..A
+00440ee0: ff16 4983 7e08 0074 0848 89df e800 0000  ..I.~..t.H......
+00440ef0: 0048 8b3d 0000 0000 ff17 4883 3801 0f85  .H.=......H.8...
+00440f00: 8102 0000 488b 3d00 0000 00ff 1748 8378  ....H.=......H.x
+00440f10: 0800 7477 e800 0000 0048 89c3 4889 9d48  ..tw.....H..H..H
+00440f20: ffff ff48 8b45 9048 8945 d048 8b45 8848  ...H.E.H.E.H.E.H
+00440f30: 8945 c848 8b45 8048 8945 c048 8b85 78ff  .E.H.E.H.E.H..x.
+00440f40: ffff 4889 45b8 488b 8568 ffff ff48 8b8d  ..H.E.H..h...H..
+00440f50: 70ff ffff 4889 4db0 4889 45a8 488d 7dd8  p...H.M.H.E.H.}.
+00440f60: 488d b548 ffff ff48 8d55 a8e8 0000 0000  H..H...H.U......
+00440f70: f048 ff0b 0f85 a100 0000 488d bd48 ffff  .H........H..H..
+00440f80: ffe8 0000 0000 e990 0000 0048 8b3d 0000  ...........H.=..
+00440f90: 0000 ff17 48c7 4008 ffff ffff 488d 4808  ....H.@.....H.H.
+00440fa0: 488d 5010 4889 9548 ffff ff48 898d 50ff  H.P.H..H...H..P.
+00440fb0: ffff 488b 7010 4885 f60f 84b0 0100 0048  ..H.p.H........H
+00440fc0: 8b3d 0000 0000 ff17 4889 c348 8b45 9048  .=......H..H.E.H
+00440fd0: 8945 d048 8b43 1848 8b4d 8848 894d c848  .E.H.C.H.M.H.M.H
+00440fe0: 8b4d 8048 894d c048 8b8d 78ff ffff 4889  .M.H.M.H..x...H.
+00440ff0: 4db8 488b 8d68 ffff ff48 8b95 70ff ffff  M.H..h...H..p...
+00441000: 4889 55b0 4889 4da8 488d 7dd8 488d 55a8  H.U.H.M.H.}.H.U.
+00441010: ff50 3048 c743 0800 0000 008b 45d9 8945  .P0H.C......E..E
+00441020: e88a 45d8 0fb7 4ddd 6689 4dec 8a4d df88  ..E...M.f.M..M..
+00441030: 4dee 3c04 0f85 8000 0000 e800 0000 0048  M.<............H
+00441040: 89c3 4889 9d48 ffff ff48 8b45 9048 8945  ..H..H...H.E.H.E
+00441050: d048 8b45 8848 8945 c848 8b45 8048 8945  .H.E.H.E.H.E.H.E
+00441060: c048 8b85 78ff ffff 4889 45b8 488b 8568  .H..x...H.E.H..h
+00441070: ffff ff48 8b8d 70ff ffff 4889 4db0 4889  ...H..p...H.M.H.
+00441080: 45a8 488d 7d98 488d b548 ffff ff48 8d55  E.H.}.H..H...H.U
+00441090: a8e8 0000 0000 f048 ff0b 750c 488d bd48  .......H..u.H..H
+004410a0: ffff ffe8 0000 0000 807d 9803 7531 4881  .........}..u1H.
+004410b0: c4c0 0000 005b 415e 5dc3 8845 9848 8b45  .....[A^]..E.H.E
+004410c0: e08b 4de8 894d 990f b74d ec66 894d 9d8a  ..M..M...M.f.M..
+004410d0: 4dee 884d 9f48 8945 a080 7d98 0374 cf48  M..M.H.E..}..t.H
+004410e0: 8b45 9848 8b4d a048 894d e048 8945 d848  .E.H.M.H.M.H.E.H
+004410f0: 8d85 38ff ffff 4889 8548 ffff ff48 8d05  ..8...H..H...H..
+00441100: 0000 0000 4889 8550 ffff ff48 8d45 d848  ....H..P...H.E.H
+00441110: 8985 58ff ffff 488d 0500 0000 0048 8985  ..X...H......H..
+00441120: 60ff ffff 488d 0500 0000 0048 8945 a848  `...H......H.E.H
+00441130: c745 b002 0000 0048 8d05 0000 0000 4889  .E.....H......H.
+00441140: 45b8 48c7 45c0 0200 0000 488d 8548 ffff  E.H.E.....H..H..
+00441150: ff48 8945 c848 c745 d002 0000 0048 8d35  .H.E.H.E.....H.5
+00441160: 0000 0000 488d 7da8 e800 0000 000f 0b48  ....H.}........H
+00441170: 8b3d 0000 0000 ff17 48c7 4008 0000 0000  .=......H.@.....
+00441180: e98f fdff ff48 8d3d 0000 0000 e800 0000  .....H.=........
+00441190: 000f 0b66 6666 662e 0f1f 8400 0000 0000  ...ffff.........

which is basically what we see in comment:1.

comment:5 Changed 3 months ago by gk

Keywords: GeorgKoppen201806 added
Priority: ImmediateVery High

Disabling stylo solves this for now. I'll leave this bug open for further investigation, though.

comment:6 Changed 3 months ago by gk

Summary: ESR60-based .dmg images are not built reproducibleESR60-based .dmg images are not built reproducibly with Stylo enabled

comment:7 Changed 3 months ago by gk

Cc: manishearth@… acrichton@… added
Status: newneeds_information
Summary: ESR60-based .dmg images are not built reproducibly with Stylo enabledESR60-based .dmg images are not built reproducibly with Stylo enabled using rustc > 1.25.0

As I expected this is a problem introduced by us switching to 1.26.1. Using version 1.25.0 of the Rust compiler XUL is still reproducible. So, to sum up:

Starting with the switch to Rust 1.26.1 Tor Browser for macOS (and only for that platform) is not being built reproducibly anymore, once we enable Stylo. There are small differences visible in XUL stemming from gkrust (see: comment:1 and comment:4). Using either Rust 1.25.0 or disabling Stylo (what we are doing right now) is "solving" this problem.

Alex, Manish: Do you know of any commit that could have caused this? If not, I can bisect I guess.

comment:8 Changed 3 months ago by alexcrichton

I know of https://github.com/rust-lang/rust/issues/47086 as an macOS-only deterministic compilation problem, but other than that I unfortunately don't know of any macOS-specific deterministic compilation issues as part of the toolchain itself.

comment:9 in reply to:  8 Changed 3 months ago by gk

Status: needs_informationassigned

Replying to alexcrichton:

I know of https://github.com/rust-lang/rust/issues/47086 as an macOS-only deterministic compilation problem, but other than that I unfortunately don't know of any macOS-specific deterministic compilation issues as part of the toolchain itself.

Okay, than let's try to track the issue down...

comment:10 Changed 3 months ago by gk

Status: assignedneeds_information

So, I tried for a while to get the bisecting going but it seems non-trivial. Alex: What is the recommended way of doing rust compiler bisecting? What I tried was using git bisect with the commit for 1.25.0 and 1.26.1, updating the sub modules and tarring the result up + using it on my build machines, but that breaks for different reasons (first one is no properly vendored crates are included (we compile with --enable-vendor); trying to get crates via crates.io-index fails with an SSL error...).

Maybe easier: Are there somewhere nightly source tarballs available (like they are for the official releases) which I could test? I did not find any so far but that might already narrow down the problem sufficiently.

comment:11 Changed 3 months ago by alexcrichton

If you're ok downloading prebuilt artifacts I'd recommend bisection via nightlies which rustup can download. Otherwise there's nightly source tarballs as well at urls like https://static.rust-lang.org/dist/2018-06-29/rustc-nightly-src.tar.gz

comment:12 Changed 3 months ago by manish.earth

There's also this tool, which is kinda like mozregression

https://github.com/rust-lang-nursery/cargo-bisect-rustc

It works off of the tarballs.

I've never tried it with Firefox though.

comment:13 Changed 3 months ago by gk

Okay, I tried the nightlies from 2018-02-25, 2018-03-28, and 2018-05-08 to find a regression range but to my surprise they all are good, meaning I get the same result on different build machines (which does not happen with stable versions > 1.25.0). So, it seems there is something in the stable code but not in the nightlies what is causing this which is confusing to me. We build with

--enable-local-rust --enable-vendor --enable-extended --release-channel=stable --sysconfdir=etc --target=x86_64-apple-darwin --set=target.x86_64-apple-darwin.cc=x86_64-apple-darwin-clang

but it seems that should not make a difference for nightly vs. stable builds. So, hrm...

comment:14 Changed 3 months ago by alexcrichton

Oh dear that is indeed worrisome! The only difference between nightly and stable builds is --release-channel passed to ./configure so you should already be emulating our stable builds. It looks like you're cross-compiling rustc from Linux though? For us we compile natively on OSX and I wonder if that causes differences?

comment:15 in reply to:  14 Changed 3 months ago by gk

Replying to alexcrichton:

Oh dear that is indeed worrisome! The only difference between nightly and stable builds is --release-channel passed to ./configure so you should already be emulating our stable builds. It looks like you're cross-compiling rustc from Linux though? For us we compile natively on OSX and I wonder if that causes differences?

I avoided that by taking the respective nightly sources and compiled them as we compile the stable compiler.

So, the plan right now is to take the nightly sources that are closest to 1.26.1 and remove parts of the diff until I find the problem.

comment:16 Changed 3 months ago by alexcrichton

Ok sure yeah, you may want to go via git bisection in the repo perhaps? That'll need to download other sources but they're done via lock files and such so you shouldn't be downloading or using any different sources than we used to build the releases

comment:17 Changed 3 months ago by gk

Keywords: TorBrowserTeam201807 added; TorBrowserTeam201806 removed

Moving first batch of tickets to July 2018

comment:18 Changed 3 months ago by gk

Keywords: GeorgKoppen201807 added; GeorgKoppen201806 removed

Moving my tickets.

comment:19 Changed 2 months ago by gk

Okay, some good news here: I double-checked my setup and it seems I made a mistake which led to no bad nightly being found. I fixed that and the last nightly that is good is the one from March 07. Then there are no nightly sources between 03/08 and 03/14 (inclusive) and the nightly from March 15 is the first bad one.

I fought a bit with using the repo for bisecting (I need to vendor all the crates before creating a tarball, making sure that I really have all the submodules initialized and updated, no .git dirs bundled but .gitmodules files available etc.) but it seems to work now. Bisecting...

Last edited 5 weeks ago by gk (previous) (diff)

comment:20 Changed 2 months ago by gk

The first bad commit git bisect gives me is commit 2079a084df08c38eb4dbfc5c8de5c0245170c3d9 which is a merge. So, I need to dig deeper it seems...

comment:21 Changed 2 months ago by gk

Bisecting seems tricky as I get intermittent differences which sometimes go away if one just tries several times. I've attached the disassmebly diff of gkrust-d3a9de07b53ab691.gkrust0.rcgu.o as it might help a bit narrowing things down. I wonder whether we actually got the differences due to an LLVM update...

comment:22 Changed 2 months ago by alexcrichton

Hm interesting! I wonder if this is perhaps related to https://github.com/rust-lang/rust/issues/52044? That claims it was fixed with the most recent LLVM upgrade. Are you able to reproduce the non-determinism on the most recent nightly?

comment:23 in reply to:  22 ; Changed 2 months ago by gk

Replying to alexcrichton:

Hm interesting! I wonder if this is perhaps related to https://github.com/rust-lang/rust/issues/52044? That claims it was fixed with the most recent LLVM upgrade. Are you able to reproduce the non-determinism on the most recent nightly?

Aha! That sounds promising and I certainly feel glandium's "This is driving me crazy", so this should be the issue then, right? ;)

That said, I compiled the nightly from 2018-07-13 which should contain the LLVM upgrade and I can't reproduce the problem anymore. However, I can't either when compiling the one from from 2018-07-11 which should *not* contain the LLVM upgrade (it's based on commit e5f6498d3d5c9dac841009d7b49738923826af75). So, it seem the LLVM uprade (alone) is not enough to explain this bug, or am I missing something?

Trying to figure out where all this started, I am pretty sure that 2018-02-15 is good and 2018-03-07 is bad.

comment:24 Changed 2 months ago by gk

I wonder if building from scratch is important here (which I do and glandium did as well).

comment:25 in reply to:  23 Changed 2 months ago by gk

Replying to gk:

Replying to alexcrichton:

Hm interesting! I wonder if this is perhaps related to https://github.com/rust-lang/rust/issues/52044? That claims it was fixed with the most recent LLVM upgrade. Are you able to reproduce the non-determinism on the most recent nightly?

Aha! That sounds promising and I certainly feel glandium's "This is driving me crazy", so this should be the issue then, right? ;)

That said, I compiled the nightly from 2018-07-13 which should contain the LLVM upgrade and I can't reproduce the problem anymore. However, I can't either when compiling the one from from 2018-07-11 which should *not* contain the LLVM upgrade (it's based on commit e5f6498d3d5c9dac841009d7b49738923826af75). So, it seem the LLVM uprade (alone) is not enough to explain this bug, or am I missing something?

On the other hand testing your script from https://github.com/rust-lang/rust/issues/52044#issuecomment-402349038 shows easily that the one from 2018-07-13 is good while the one from 2018-07-11 is not. I guess I tried the Firefox build just not often enough with 2018-07-11...

So, what's the suggested way to fix that? What until a new stable compiler with a fix for it is out (I guess that's 1.28.0) and use that one?

Althought, tbh, it's scary to hope this is finally fixed without exactly knowing what the issue was. I might try to look closer at it.

comment:26 Changed 2 months ago by alexcrichton

Ah yeah for the LLVM 7 change we're gonna let that ride the trains (aka not backport). If you can temporarily use a nightly that'd work but otherwise this may have to wait until that's released.

Knowing for sure though what was causing the nondeterminism in LLVM would be great!

comment:27 in reply to:  23 Changed 2 months ago by gk

Replying to gk:

Replying to alexcrichton:

Hm interesting! I wonder if this is perhaps related to https://github.com/rust-lang/rust/issues/52044? That claims it was fixed with the most recent LLVM upgrade. Are you able to reproduce the non-determinism on the most recent nightly?

[snip]

Trying to figure out where all this started, I am pretty sure that 2018-02-15 is good and 2018-03-07 is bad.

So, here is another interesting bit: While running the script (https://github.com/rust-lang/rust/issues/52044#issuecomment-402349038) on a Linux box does replicate the issue for a Linux build using the nightly from 2018-07-11, it does not replicate it using the nightly from 2018-03-07 (I ran the script a couple of times). On the other hand, using the nightly from 2018-03-07 to generate a Firefox build for macOS does show differences (while, as mentioned above, I seemingly can't reproduce the problem with the nightly from 2018-07-11 anymore when cross-compiling for macOS).

To test better I adapted your repro script and added a respective --target x86_64-apple-darwin and with that setup it's easy to see the bug with the nightly from 2018-03-07.

So, to sum up, the problem is not just bumping the LLVM version but it seems to be somewhat target dependent, too.

comment:28 Changed 8 weeks ago by gk

Interesting, look what I get after trying to run the repro script for the macOS target after the bump to LLVM 7 (i.e. with the nightly from 2017-07-13):

+ for i in `seq 1 100`
+ rm -rf a b
+ mkdir a b
+ rustc /dev/stdin -O -C lto -C panic=abort -C codegen-units=1 --emit llvm-ir,obj --crate-type staticlib --out-dir a --target x86_64-apple-darwin
+ rustc /dev/stdin -O -C lto -C panic=abort -C codegen-units=1 --emit llvm-ir,obj --crate-type staticlib --out-dir b --target x86_64-apple-darwin
++ md5sum a/stdin.ll
++ awk '{print $1}'
+ a=d3665a14a5cf9bf1c4630c001e8d4dfc
++ md5sum b/stdin.ll
++ awk '{print $1}'
+ b=4d3336f9f9c5bc4b91653c1fc51d0bf6
+ '[' d3665a14a5cf9bf1c4630c001e8d4dfc '!=' 4d3336f9f9c5bc4b91653c1fc51d0bf6 ']'
+ echo IR is different
IR is different
+ exit 1

Linux is fine (which is probably what glandium found).

FWIW: I see different objects for macOS with the repro script when using the nightly from 2018-07-11, so, indeed, I did not do enough Firefox compilations to trigger the problem for macOS there.

comment:29 Changed 8 weeks ago by alexcrichton

Hm fascinating! That's a good data point towards "maybe fixed" in LLVM but only accidentally for one platform rather than across the board?

comment:30 in reply to:  29 Changed 8 weeks ago by gk

Replying to alexcrichton:

Hm fascinating! That's a good data point towards "maybe fixed" in LLVM but only accidentally for one platform rather than across the board?

I actually think the issue glandium had and we have are not the same but probably related: the macOS one is happening with the upgrade to LLVM 6 (I am about to start bisecting those changes) while Linux is unaffected by that. But I'll have clues for the latter as well, but macOS first.

comment:31 Changed 7 weeks ago by gk

Keywords: GeorgKoppen201808 added; GeorgKoppen201807 removed

Moving my tickets.

comment:32 Changed 7 weeks ago by gk

Keywords: TorBrowserTeam201808 added; TorBrowserTeam201807 removed

Move our tickets to August.

comment:33 Changed 7 weeks ago by gk

Alright, so, the problematic commit is 6b7b6b63a928479a29d9fc1282e553e409c66934. I tried to bisect LLVM to find out the bad revision. I compiled just LLVM separately and used --llvm-root to point to it during the rust build. It turns out that doing that, even with just the llvm code copied out of rust/src/llvm being on commit 6b7b6b63a928479a29d9fc1282e553e409c66934 is fine. However, double-checking, building LLVM during the usual rust build (i.e. without --llvm-root) still reproduces the bug.

Thus, I can only think of two reasons causing the reprouducibility issue:

1) The LLVM part is compiled differently within rust than I did. The compiler used is the same, though, not sure what other flags could cause this. I am doing

cmake .. -G "Unix Makefiles" -DCMAKE_INSTALL_PREFIX=$distdir -DCMAKE_BUILD_TYPE:STRING=Release -DLLVM_INSTALL_UTILS=on $LLVM_HOME 

in the --llvm-root build which seems to be not overly fancy.

2) There are additional pieces getting compiled in/used during the LLVM compilation done during the rust build that are causing the problem.

I need to think more about how to proceed but, Alex, if you have any ideas that would be appreciated. :)

comment:34 Changed 7 weeks ago by alexcrichton

Nice! Bisecting to the LLVM upgrade definitely makes sense to me. That was a massive LLVM upgrade though (from LLVM 4.0 to LLVM 6.0), so that would be quite the bisection range for a regression to be introduced in :(

If it works, though, when you build LLVM yourself that's quite curious. The command we use to build LLVM though is pretty huge. Looking at one of our recent builds (https://travis-ci.org/rust-lang/rust/builds/411219658) the command we use on OSX is:

"cmake" "/Users/travis/build/rust-lang/rust/src/llvm" "-DLLVM_ENABLE_ASSERTIONS=OFF" "-DLLVM_TARGETS_TO_BUILD=X86;ARM;AArch64;Mips;PowerPC;SystemZ;MSP430;Sparc;NVPTX;Hexagon" "-DLLVM_EXPERIMENTAL_TARGETS_TO_BUILD=WebAssembly;RISCV" "-DLLVM_INCLUDE_EXAMPLES=OFF" "-DLLVM_INCLUDE_TESTS=OFF" "-DLLVM_INCLUDE_DOCS=OFF" "-DLLVM_ENABLE_ZLIB=OFF" "-DWITH_POLLY=OFF" "-DLLVM_ENABLE_TERMINFO=OFF" "-DLLVM_ENABLE_LIBEDIT=OFF" "-DLLVM_ENABLE_LIBXML2=OFF" "-DLLVM_PARALLEL_COMPILE_JOBS=4" "-DLLVM_TARGET_ARCH=x86_64" "-DLLVM_DEFAULT_TARGET_TRIPLE=x86_64-apple-darwin" "-DLLVM_OCAML_INSTALL_PATH=usr/lib/ocaml" "-DCMAKE_EXE_LINKER_FLAGS=-static-libstdc++" "-DCMAKE_C_COMPILER=sccache" "-DCMAKE_C_COMPILER_ARG1=/Users/travis/build/rust-lang/rust/clang+llvm-6.0.0-x86_64-apple-darwin/bin/clang" "-DCMAKE_CXX_COMPILER=sccache" "-DCMAKE_CXX_COMPILER_ARG1=/Users/travis/build/rust-lang/rust/clang+llvm-6.0.0-x86_64-apple-darwin/bin/clang++" "-DCMAKE_C_FLAGS=-ffunction-sections -fdata-sections -fPIC --target=x86_64-apple-darwin -stdlib=libc++" "-DCMAKE_CXX_FLAGS=-ffunction-sections -fdata-sections -fPIC --target=x86_64-apple-darwin -stdlib=libc++" "-DCMAKE_INSTALL_PREFIX=/Users/travis/build/rust-lang/rust/build/x86_64-apple-darwin/llvm" "-DCMAKE_BUILD_TYPE=Release"

I wonder if perhaps the way we compile LLVM is affecting this? Maybe some flag or maybe our own compiler we use on automation is introducing bugs? Or maybe it has to do with the C++ standard library and which is used?

comment:35 in reply to:  34 Changed 5 weeks ago by gk

Replying to alexcrichton:

Nice! Bisecting to the LLVM upgrade definitely makes sense to me. That was a massive LLVM upgrade though (from LLVM 4.0 to LLVM 6.0), so that would be quite the bisection range for a regression to be introduced in :(

If it works, though, when you build LLVM yourself that's quite curious. The command we use to build LLVM though is pretty huge. Looking at one of our recent builds (https://travis-ci.org/rust-lang/rust/builds/411219658) the command we use on OSX is:

"cmake" "/Users/travis/build/rust-lang/rust/src/llvm" "-DLLVM_ENABLE_ASSERTIONS=OFF" "-DLLVM_TARGETS_TO_BUILD=X86;ARM;AArch64;Mips;PowerPC;SystemZ;MSP430;Sparc;NVPTX;Hexagon" "-DLLVM_EXPERIMENTAL_TARGETS_TO_BUILD=WebAssembly;RISCV" "-DLLVM_INCLUDE_EXAMPLES=OFF" "-DLLVM_INCLUDE_TESTS=OFF" "-DLLVM_INCLUDE_DOCS=OFF" "-DLLVM_ENABLE_ZLIB=OFF" "-DWITH_POLLY=OFF" "-DLLVM_ENABLE_TERMINFO=OFF" "-DLLVM_ENABLE_LIBEDIT=OFF" "-DLLVM_ENABLE_LIBXML2=OFF" "-DLLVM_PARALLEL_COMPILE_JOBS=4" "-DLLVM_TARGET_ARCH=x86_64" "-DLLVM_DEFAULT_TARGET_TRIPLE=x86_64-apple-darwin" "-DLLVM_OCAML_INSTALL_PATH=usr/lib/ocaml" "-DCMAKE_EXE_LINKER_FLAGS=-static-libstdc++" "-DCMAKE_C_COMPILER=sccache" "-DCMAKE_C_COMPILER_ARG1=/Users/travis/build/rust-lang/rust/clang+llvm-6.0.0-x86_64-apple-darwin/bin/clang" "-DCMAKE_CXX_COMPILER=sccache" "-DCMAKE_CXX_COMPILER_ARG1=/Users/travis/build/rust-lang/rust/clang+llvm-6.0.0-x86_64-apple-darwin/bin/clang++" "-DCMAKE_C_FLAGS=-ffunction-sections -fdata-sections -fPIC --target=x86_64-apple-darwin -stdlib=libc++" "-DCMAKE_CXX_FLAGS=-ffunction-sections -fdata-sections -fPIC --target=x86_64-apple-darwin -stdlib=libc++" "-DCMAKE_INSTALL_PREFIX=/Users/travis/build/rust-lang/rust/build/x86_64-apple-darwin/llvm" "-DCMAKE_BUILD_TYPE=Release"

I wonder if perhaps the way we compile LLVM is affecting this? Maybe some flag or maybe our own compiler we use on automation is introducing bugs? Or maybe it has to do with the C++ standard library and which is used?

Well, I dont't know whether the problem we have is happening when using the "official" binaries. This is happening when cross-compiling rust ourselves for macOS. That said, I think I can exclude 1) from comment:33. I compiled LLVM with exactly the same arguments as used during the rust build (using the same runc container, compiler etc.) which are:

cmake .. -G "Unix Makefiles" -DLLVM_ENABLE_ASSERTIONS=OFF -DLLVM_TARGETS_TO_BUILD="X86;ARM;AArch64;Mips;PowerPC;SystemZ;MSP430;Sparc;NVPTX;Hexagon" -DLLVM_EXPERIMENTAL_TARGETS_TO_BUILD=WebAssembly -DLLVM_INCLUDE_EXAMPLES=OFF -DLLVM_INCLUDE_TESTS=OFF -DLLVM_INCLUDE_DOCS=OFF -DLLVM_ENABLE_ZLIB=OFF -DWITH_POLLY=OFF -DLLVM_ENABLE_TERMINFO=OFF -DLLVM_ENABLE_LIBEDIT=OFF -DLLVM_PARALLEL_COMPILE_JOBS=4 -DLLVM_TARGET_ARCH=x86_64 -DLLVM_DEFAULT_TARGET_TRIPLE=x86_64-unknown-linux-gnu -DLLVM_LINK_LLVM_DYLIB=ON -DCMAKE_C_COMPILER=cc -DCMAKE_CXX_COMPILER=c++ -DCMAKE_C_FLAGS="-ffunction-sections -fdata-sections -fPIC -m64" -DCMAKE_CXX_FLAGS="-ffunction-sections -fdata-sections -fPIC -m64" -DCMAKE_INSTALL_PREFIX=$distdir -DCMAKE_BUILD_TYPE:String=Release -DLLVM_INSTALL_UTILS=on $LLVM_HOME

(-DLLVM_INSTALL_UTILS=on is only used when compiling LLVM outside of the rust compilation but I doubt this would make a difference with respect to this bug)

and used it with --llvm-root and the test script is running fine.

Which leaves 2). I reverted the dlmalloc and libcompiler_builtins/compiler-rt submodule updates but compiling LLVM during the rust build still exhibits the failing test script. Thus, I think those updates does not cause this bug. I wonder what else is different between the in-tree LLVM compilation and the one outside of it (I am just tarring up src/llvm, excluding .git, using that tarball for compilation)...

comment:36 Changed 4 weeks ago by alexcrichton

Hm so if LLVM is compiled exactly the same way that is indeed a little worrisome. One other thing we do is that if we're compiling an in-tree LLVM we specify the LLVM_RUSTLLVM define for C++ code (here - https://github.com/rust-lang/rust/blob/bf1e461173e3936e4014cc951dfbdd7d9ec9190b/src/bootstrap/compile.rs#L736-L740). That's used for a few minor features but none that should touch the reproducibility of the build in theory...

I'm running out of ideas unfortunately :(

comment:37 Changed 4 weeks ago by gk

Status: needs_informationnew

Not setting LLVM_RUSTLLVM does not change things as you expected. So, I *could* do the LLVM bisecting with LLVM built during the rust compilation to give us a better hint at what is going wrong. However, I'd like to avoid that as it's probably a tedious pain in the rear and while the LLVM upgrade is triggering the bug it's probably nothing we should fix on the LLVM side. I'll think about something smarter and will go back to this bug after we got Tor Browser 8 out.

We might stick to just building LLVM outside of rust for macOS cross-compilation for the time being to move around this issue.

comment:38 Changed 2 weeks ago by gk

Keywords: GeorgKoppen201809 added; GeorgKoppen201808 removed

Moving my tickets to September.

comment:39 Changed 2 weeks ago by gk

Keywords: TorBrowserTeam201809 added; TorBrowserTeam201808 removed

Moving our tickets to September 2018

Note: See TracTickets for help on using tickets.