Make our Gitian setup able to build hardened Linux bundles (64bit)

We should think about ways to improve our current Gitian setup (descriptors etc.) to be able to build hardened Linux bundles (64bit) easily.

added TorBrowserTeam201510 component::applications/tor browser owner::gk parent::17304 priority::very high resolution::fixed severity::normal sponsor:: status::closed tbb-gitian type::task labels

We plan to ship the hardened bundles in addition to our regular alpha builds. I wonder what would be the most straightforward way to add a new series given that it will need own updates etc. Having a channel hardened with something like 5.5a3-hardened maybe?

I am still thinking about how to add hardened builds in detail. Does it make sense to have own hardened descriptors given that we would only need two (tor and firefox) and could easily get rid of things we don't want (like the i386 target)? The disadvantage is the need to update different multiple sets of descriptors if we need to change any of them. But that should not happen very often...

Trac:
Cc: N/A to boklm

Trac:
Cc: boklm to boklm, brade, mcs

Do we expect hardened builds to exist side-by-side with non-hardened builds for quite a while? If not, it would be better to avoid creating a new update channel. We could use the ALL "locale" that Kathy and I created in the patches for #12967 (moved) to keep the builds and updates separate for now. But I am not sure what is best.

Replying to mcs:

Do we expect hardened builds to exist side-by-side with non-hardened builds for quite a while? If not,

That is my impression at least.

Trac:
Severity: N/A to Normal
Priority: Medium to Very High
Keywords: N/A deleted, TorBrowserTeam201510 added

After thinking a bit more about it I'll go with boklm's idea of having a dedicated branch for the hardened series. Messing with master might have unintended effects for our alpha series which we want to avoid.

Trac:
Owner: tbb-team to gk
Status: new to assigned

hardened-builds (https://gitweb.torproject.org/user/gk/tor-browser-bundle.git/log/?h=hardened-builds) has the progress I made so far. It seems everything is working, just the updater bits are still missing.

I think what I want is a new channel, say, "hardened" while making sure at the same time that the hardened series is basically the alpha series + special compile time/runtime hardening features.

So, if we are on the hardening branch and are doing something like make alpha there should be a bundle like tor-browser-linux64-5.5a3-hardened_ALL.tar.xz showing up at the end whith an update channel set to "hardened".

The incrementals in turn would be created from tor-browser-linux64-5.5a2-hardened_ALL.mar (if it were existing).

Does that make sense?

Yes, I think it makes sense to do it with a separate hardened branch, in which master is merged.

However I think there will be a problem for the version number which is determined from the git tag: it won't be possible to tag the hardened branch as 5.5a3 as this tag will already be on the master branch.

As an alternative what we can do is:

• use "5.5a3hardened" as the version number and tag the hardened branch using this version
• not modify the names of the .mar and .tar.xz files in gitian/descriptors/linux/gitian-bundle.yml (reverting the changes in this file from commit 37be82da4f2d16766f2b8aceb68ec0bf6783490b)
• update gitian/get-tb-version to recognize 5.5a3hardened as an alpha version (and push this change to master)
• upload the hardened build to https://dist.torproject.org/torbrowser/5.5a3hardened/
• when merging master into the hardened branch before a release, fix conflicts in tools/update-responses/config.yml by taking the master version and adding 'hardened' to all versions

If that sounds good to you, I can make some patches based on your branch to do that.

I tested my code with a local tbb-5.5a3-hardened-build1 to make sure the version gets properly extracted, so, yes, sounds good (I'd prefer a dash between the version number and "hardened" which makes it a bit more readable IMO).

Using an update channel named "hardened" is OK but this will require some code changes (to support "alpha" instead of "aurora" we had to make some changes).

Using "5.5a3-hardened" as a version number (internally, within the browser) won't work well because Mozilla's code expects version numbers to conform to a specific format. For example, the updater code relies on this to be able to compare versions. If this version number is just for git tag purposes, then nevermind. :-)

Replying to brade:

Using an update channel named "hardened" is OK but this will require some code changes (to support "alpha" instead of "aurora" we had to make some changes).

Using "5.5a3-hardened" as a version number (internally, within the browser) won't work well because Mozilla's code expects version numbers to conform to a specific format. For example, the updater code relies on this to be able to compare versions. If this version number is just for git tag purposes, then nevermind. :-)

Would "5.5a3hardened" (without '-') work better or is it the same ?

Replying to brade:

Using an update channel named "hardened" is OK but this will require some code changes (to support "alpha" instead of "aurora" we had to make some changes).

Using "5.5a3-hardened" as a version number (internally, within the browser) won't work well because Mozilla's code expects version numbers to conform to a specific format. For example, the updater code relies on this to be able to compare versions. If this version number is just for git tag purposes, then nevermind. :-)

So, reading on https://developer.mozilla.org/en-US/docs/Mozilla/Tech/XPCOM/Reference/Interface/nsIVersionComparator I see:

Version strings are dot-separated sequences of version-parts. A version-part consists of up to four parts, all of which are optional:

<number-a><string-b><number-c><string-d (everything else)>

A version-part may also consist of a single asterisk "*" which indicates * "infinity". Numbers are base-10, and are zero if left out. Strings are compared bytewise.

would be "5" and would be "a3-hardened". Looking at the examples on that page (e.g. 1.0pre1 < 1.0pre2) there should be no issue for the comparator to estimate that 5.5a3-hardened < 5.5a4-hardened holds especially as this works without "-hardened" very well and "-hardened" is essentially an unchanging string. What am I missing?

Replying to gk:

... would be "5" and would be "a3-hardened". Looking at the examples on that page (e.g. 1.0pre1 < 1.0pre2) there should be no issue for the comparator to estimate that 5.5a3-hardened < 5.5a4-hardened holds especially as this works without "-hardened" very well and "-hardened" is essentially an unchanging string. What am I missing?

I think you are correct in saying that 5.5a3-hardened < 5.5a4-hardened will be true. It is only if we mix together things like ``5.5a3-hardened and < 5.5a3 that it gets confusing. I think for that case, `5.5a3-SOMETHING` is always less than `5.5a3`. Some more details are available here: https://developer.mozilla.org/en-US/docs/Toolkit_version_format#Comparing_versions

So, I think it will work if everything stays within the hardened channel (but it is a little strange to encode the "hardened" concept in the version number).

Replying to mcs:

So, I think it will work if everything stays within the hardened channel (but it is a little strange to encode the "hardened" concept in the version number).

I am fine not doing that as long as the user sees clearly that they are using the hardened version and that it is basically our current alpha. One thing I thought about was using just a version number like "5.5a3" internally which should be fine as we are on a different update channel but amending this to "5.5a3-hardened" in the UI. Not sure how hard that is but it sounded more complex than just using "5.5a3-hardened" everywhere.

Replying to brade:

Using an update channel named "hardened" is OK but this will require some code changes (to support "alpha" instead of "aurora" we had to make some changes).

After looking at the updater patches, I think the code changes to support a "hardened" channel are the following two patches: https://github.com/boklm/gecko-dev/commit/e26ea6a9190c3cf8c8ea02cffebb462c70e50a16.patch https://github.com/boklm/gecko-dev/commit/7c753b9d9fdd09b39d38b6c1d0efbc9c0069edb6.patch

I added a hardened-builds-v2 branch on my tor-browser-bundle.git repo: https://gitweb.torproject.org/user/boklm/tor-browser-bundle.git/log/?h=hardened-builds-v2

I have not tried to build it yet, but I think it includes the changes needed to have a "hardened" channel.

Replying to boklm:

After looking at the updater patches, I think the code changes to support a "hardened" channel are the following two patches: https://github.com/boklm/gecko-dev/commit/e26ea6a9190c3cf8c8ea02cffebb462c70e50a16.patch https://github.com/boklm/gecko-dev/commit/7c753b9d9fdd09b39d38b6c1d0efbc9c0069edb6.patch

Yes, I think these patches are what is needed.

Replying to mcs:

Replying to boklm:

After looking at the updater patches, I think the code changes to support a "hardened" channel are the following two patches: https://github.com/boklm/gecko-dev/commit/e26ea6a9190c3cf8c8ea02cffebb462c70e50a16.patch https://github.com/boklm/gecko-dev/commit/7c753b9d9fdd09b39d38b6c1d0efbc9c0069edb6.patch

Yes, I think these patches are what is needed.

These got merged as commit 517d9fec4b6d464d276410385942323124488255 and f799aa38f56abda676cacd8d435e16286edea772 into tor-browser-38.3.0esr-5.5-2.

Replying to boklm:

I added a hardened-builds-v2 branch on my tor-browser-bundle.git repo: https://gitweb.torproject.org/user/boklm/tor-browser-bundle.git/log/?h=hardened-builds-v2

I have not tried to build it yet, but I think it includes the changes needed to have a "hardened" channel.

Looks good and works so far (although I did not test incremental update generation). One thing I forgot is to comment our the Windows and OS X builds in the Makefile. I'll do that when I create the hardened builds branch.

I pushed the hardened-builds branch and tagged build1. We should be ready to go: the future is hardened!1!!

Trac:
Resolution: N/A to fixed
Status: assigned to closed

closed

moved to tpo/applications/tor-browser#17305 (closed)