Update binutils to a version more recent than 2.26.1

In #16472 (moved) we are updating binutils to 2.26.1. We are not updating it to a more recent version due to an issue with reproducible builds on Windows with later binutils versions.

The issue seems to be caused by binutils commit 13e570f80cbfb299a8858ce6830e91a6cb40ab7b (which we revert in #16472 (moved)).

added TorBrowserTeam201901R boklm201901 component::applications/tor browser owner::tbb-team priority::high resolution::fixed severity::normal status::closed tbb-rbm type::task labels

boklm201805 -> boklm201806

Trac:
Keywords: boklm201805 deleted, boklm201806 added

boklm201806 -> boklm201807

Trac:
Keywords: boklm201806 deleted, boklm201807 added

I filled an upstream binutils ticket for this issue: https://sourceware.org/bugzilla/show_bug.cgi?id=23466

boklm201807 -> boklm201809

Trac:
Keywords: boklm201807 deleted, boklm201809 added

I will try a build using -Wl,-t to see if this can help understand the issue (as suggested in https://sourceware.org/bugzilla/show_bug.cgi?id=23466#c1).

boklm201809 -> boklm201811

Trac:
Keywords: boklm201809 deleted, boklm201811 added

Trac:
Priority: Medium to High
Keywords: TorBrowserTeam201805 deleted, TorBrowserTeam201901 added

I have been building using -Wl,-t two times on one machine, and one time on an other one. The builds from the same machine were identical, but different on the second machine.

Comparing the logs from the non maching builds, the main difference was the order of .o files included inside libstdc++.a. It seems that with previous versions of binutils, the order of objects inside a .a file did not affect the output, but it is not the case anymore with more recent versions of binutils.

The reason why libstdc++.a is including objects in random order is that gcc is using an old version of libtool: https://gcc.gnu.org/ml/gcc-patches/2018-06/msg01115.html

With the libtool patch sorting the output of find, I am now able to get matching builds on different machines.

I am now testing a patch to update binutils to 2.31.1.

Replying to boklm:

with previous versions of binutils, the order of objects inside a .a file did not affect the output, but it is not the case anymore with more recent versions of binutils. Bug or feature for you? I am now testing a patch to update binutils to 2.31.1. Please test with the master branch instead, because of https://sourceware.org/ml/binutils/2019-01/msg00013.html and to avoid issues like https://sourceware.org/bugzilla/show_bug.cgi?id=23600.

Trac:
Username: ld

Replying to ld:

Replying to boklm:

with previous versions of binutils, the order of objects inside a .a file did not affect the output, but it is not the case anymore with more recent versions of binutils. Bug or feature for you? I am now testing a patch to update binutils to 2.31.1. Please test with the master branch instead, because of https://sourceware.org/ml/binutils/2019-01/msg00013.html and to avoid issues like https://sourceware.org/bugzilla/show_bug.cgi?id=23600.

No, 2.31.1 is fine for now I think (if nothing explodes). We'll want to switch to a newer binutils before 2.32 gets released if possible.

In branch bug_26148_v6 we update binutils to 2.31.1: https://gitweb.torproject.org/user/boklm/tor-browser-build.git/commit/?h=bug_26148_v6&id=f804709d68e9c4b3c7b606b89a8f555f21fe3e09

I checked that the linux and windows builds are reproductible on two machines, and the bundles created are running correctly.

Trac:
Keywords: TorBrowserTeam201901, boklm201811 deleted, boklm201901, TorBrowserTeam201901R added
Status: new to needs_review

ticket:16472#comment:60?

Trac:
Username: ld

Replying to boklm:

In branch bug_26148_v6 we update binutils to 2.31.1: https://gitweb.torproject.org/user/boklm/tor-browser-build.git/commit/?h=bug_26148_v6&id=f804709d68e9c4b3c7b606b89a8f555f21fe3e09

I checked that the linux and windows builds are reproductible on two machines, and the bundles created are running correctly.

Thanks! Testing shows this works for me. What we could do is including the landed libtool patch instead of the mail. The patch is at least on GCC master (commit 494c17dc2bf45b8155fb9a14178ab48f01bc12a8) but not in any release yet.

One request: could you add https://sourceware.org/bugzilla/show_bug.cgi?id=17321 to the comment for the relocation patch so that we know this is on file at least?

Trac:
Keywords: TorBrowserTeam201901R deleted, TorBrowserTeam201901 added
Status: needs_review to needs_revision

There is a new version of the patch in branch bug_26148_v7: https://gitweb.torproject.org/user/boklm/tor-browser-build.git/commit/?h=bug_26148_v7&id=5631b327ab3654af5f2b329cf46fd67921004f41

With the following changes:

• we are now using the patch from the GCC commit
• there is now a comment about the relocation patch pointing to https://sourceware.org/bugzilla/show_bug.cgi?id=17321
• we use the tar.xz instead of tar.bz2
• the sed line for #10102 (closed) is removed as it is not needed since https://sourceware.org/git/?p=binutils-gdb.git;a=commit;h=8d5c4b7bfdfa5f2b40fa056132823c8e9497dc72

Trac:
Status: needs_revision to needs_review
Keywords: TorBrowserTeam201901 deleted, TorBrowserTeam201901R added

Looks good, thanks! Merge to master (commit 5631b327ab3654af5f2b329cf46fd67921004f41).

Trac:
Status: needs_review to closed
Resolution: N/A to fixed

closed

mentioned in issue #27503 (moved)

mentioned in issue #27623 (moved)

mentioned in issue #28618 (moved)

moved to tpo/applications/tor-browser#26148 (closed)

mentioned in issue tpo/applications/tor-browser#28618 (closed)