Opened 4 years ago

Closed 3 years ago

#17305 closed task (fixed)

Make our Gitian setup able to build hardened Linux bundles (64bit)

Reported by: gk Owned by: gk
Priority: Very High Milestone:
Component: Applications/Tor Browser Version:
Severity: Normal Keywords: tbb-gitian, TorBrowserTeam201510
Cc: boklm, brade, mcs Actual Points:
Parent ID: #17304 Points:
Reviewer: Sponsor: SpnsorU

Description

We should think about ways to improve our current Gitian setup (descriptors etc.) to be able to build hardened Linux bundles (64bit) easily.

Child Tickets

Change History (20)

comment:1 Changed 4 years ago by gk

Cc: boklm added

We plan to ship the hardened bundles in addition to our regular alpha builds. I wonder what would be the most straightforward way to add a new series given that it will need own updates etc. Having a channel hardened with something like 5.5a3-hardened maybe?

I am still thinking about how to add hardened builds in detail. Does it make sense to have own hardened descriptors given that we would only need two (tor and firefox) and could easily get rid of things we don't want (like the i386 target)? The disadvantage is the need to update different multiple sets of descriptors if we need to change any of them. But that should not happen very often...

comment:2 Changed 4 years ago by mcs

Cc: brade mcs added

comment:3 Changed 4 years ago by mcs

Do we expect hardened builds to exist side-by-side with non-hardened builds for quite a while? If not, it would be better to avoid creating a new update channel. We could use the ALL "locale" that Kathy and I created in the patches for #12967 to keep the builds and updates separate for now. But I am not sure what is best.

comment:4 in reply to:  3 Changed 4 years ago by gk

Replying to mcs:

Do we expect hardened builds to exist side-by-side with non-hardened builds for quite a while? If not,

That is my impression at least.

comment:5 Changed 4 years ago by gk

Keywords: TorBrowserTeam201510 added
Priority: MediumVery High
Severity: Normal

comment:6 Changed 4 years ago by gk

Owner: changed from tbb-team to gk
Status: newassigned

After thinking a bit more about it I'll go with boklm's idea of having a dedicated branch for the hardened series. Messing with master might have unintended effects for our alpha series which we want to avoid.

comment:7 Changed 4 years ago by gk

hardened-builds (https://gitweb.torproject.org/user/gk/tor-browser-bundle.git/log/?h=hardened-builds) has the progress I made so far. It seems everything is working, just the updater bits are still missing.

I think what I want is a new channel, say, "hardened" while making sure at the same time that the hardened series is basically the alpha series + special compile time/runtime hardening features.

So, if we are on the hardening branch and are doing something like make alpha there should be a bundle like tor-browser-linux64-5.5a3-hardened_ALL.tar.xz showing up at the end whith an update channel set to "hardened".

The incrementals in turn would be created from tor-browser-linux64-5.5a2-hardened_ALL.mar (if it were existing).

Does that make sense?

comment:8 Changed 4 years ago by boklm

Yes, I think it makes sense to do it with a separate hardened branch, in which master is merged.

However I think there will be a problem for the version number which is determined from the git tag: it won't be possible to tag the hardened branch as 5.5a3 as this tag will already be on the master branch.

As an alternative what we can do is:

  • use "5.5a3hardened" as the version number and tag the hardened branch using this version
  • not modify the names of the .mar and .tar.xz files in gitian/descriptors/linux/gitian-bundle.yml (reverting the changes in this file from commit 37be82da4f2d16766f2b8aceb68ec0bf6783490b)
  • update gitian/get-tb-version to recognize 5.5a3hardened as an alpha version (and push this change to master)
  • upload the hardened build to https://dist.torproject.org/torbrowser/5.5a3hardened/
  • when merging master into the hardened branch before a release, fix conflicts in tools/update-responses/config.yml by taking the master version and adding 'hardened' to all versions

If that sounds good to you, I can make some patches based on your branch to do that.

Last edited 4 years ago by boklm (previous) (diff)

comment:9 Changed 4 years ago by gk

I tested my code with a local tbb-5.5a3-hardened-build1 to make sure the version gets properly extracted, so, yes, sounds good (I'd prefer a dash between the version number and "hardened" which makes it a bit more readable IMO).

comment:10 Changed 4 years ago by brade

Using an update channel named "hardened" is OK but this will require some code changes (to support "alpha" instead of "aurora" we had to make some changes).

Using "5.5a3-hardened" as a version number (internally, within the browser) won't work well because Mozilla's code expects version numbers to conform to a specific format. For example, the updater code relies on this to be able to compare versions. If this version number is just for git tag purposes, then nevermind. :-)

comment:11 in reply to:  10 Changed 4 years ago by boklm

Replying to brade:

Using an update channel named "hardened" is OK but this will require some code changes (to support "alpha" instead of "aurora" we had to make some changes).

Using "5.5a3-hardened" as a version number (internally, within the browser) won't work well because Mozilla's code expects version numbers to conform to a specific format. For example, the updater code relies on this to be able to compare versions. If this version number is just for git tag purposes, then nevermind. :-)

Would "5.5a3hardened" (without '-') work better or is it the same ?

comment:12 in reply to:  10 ; Changed 4 years ago by gk

Replying to brade:

Using an update channel named "hardened" is OK but this will require some code changes (to support "alpha" instead of "aurora" we had to make some changes).

Using "5.5a3-hardened" as a version number (internally, within the browser) won't work well because Mozilla's code expects version numbers to conform to a specific format. For example, the updater code relies on this to be able to compare versions. If this version number is just for git tag purposes, then nevermind. :-)

So, reading on https://developer.mozilla.org/en-US/docs/Mozilla/Tech/XPCOM/Reference/Interface/nsIVersionComparator I see:

Version strings are dot-separated sequences of version-parts. A version-part consists of up to four parts, all of which are optional:

<number-a><string-b><number-c><string-d (everything else)>

A version-part may also consist of a single asterisk "*" which indicates * "infinity". Numbers are base-10, and are zero if left out. Strings are compared bytewise.

<number-a> would be "5" and <string-b> would be "a3-hardened". Looking at the examples on that page (e.g. 1.0pre1 < 1.0pre2) there should be no issue for the comparator to estimate that 5.5a3-hardened < 5.5a4-hardened holds especially as this works without "-hardened" very well and "-hardened" is essentially an unchanging string. What am I missing?

comment:13 in reply to:  12 ; Changed 4 years ago by mcs

Replying to gk:

...
<number-a> would be "5" and <string-b> would be "a3-hardened". Looking at the examples on that page (e.g. 1.0pre1 < 1.0pre2) there should be no issue for the comparator to estimate that 5.5a3-hardened < 5.5a4-hardened holds especially as this works without "-hardened" very well and "-hardened" is essentially an unchanging string. What am I missing?

I think you are correct in saying that 5.5a3-hardened < 5.5a4-hardened will be true. It is only if we mix together things like 5.5a3-hardened and < 5.5a3 that it gets confusing. I think for that case, 5.5a3-SOMETHING is always less than 5.5a3. Some more details are available here:
https://developer.mozilla.org/en-US/docs/Toolkit_version_format#Comparing_versions

So, I think it will work if everything stays within the hardened channel (but it is a little strange to encode the "hardened" concept in the version number).

comment:14 in reply to:  13 Changed 4 years ago by gk

Replying to mcs:

So, I think it will work if everything stays within the hardened channel (but it is a little strange to encode the "hardened" concept in the version number).

I am fine not doing that as long as the user sees clearly that they are using the hardened version and that it is basically our current alpha. One thing I thought about was using just a version number like "5.5a3" internally which should be fine as we are on a different update channel but amending this to "5.5a3-hardened" in the UI. Not sure how hard that is but it sounded more complex than just using "5.5a3-hardened" everywhere.

comment:15 in reply to:  10 ; Changed 3 years ago by boklm

Replying to brade:

Using an update channel named "hardened" is OK but this will require some code changes (to support "alpha" instead of "aurora" we had to make some changes).

After looking at the updater patches, I think the code changes to support a "hardened" channel are the following two patches:
https://github.com/boklm/gecko-dev/commit/e26ea6a9190c3cf8c8ea02cffebb462c70e50a16.patch
https://github.com/boklm/gecko-dev/commit/7c753b9d9fdd09b39d38b6c1d0efbc9c0069edb6.patch

comment:16 Changed 3 years ago by boklm

I added a hardened-builds-v2 branch on my tor-browser-bundle.git repo:
https://gitweb.torproject.org/user/boklm/tor-browser-bundle.git/log/?h=hardened-builds-v2

I have not tried to build it yet, but I think it includes the changes needed to have a "hardened" channel.

comment:17 in reply to:  15 ; Changed 3 years ago by mcs

Replying to boklm:

After looking at the updater patches, I think the code changes to support a "hardened" channel are the following two patches:
https://github.com/boklm/gecko-dev/commit/e26ea6a9190c3cf8c8ea02cffebb462c70e50a16.patch
https://github.com/boklm/gecko-dev/commit/7c753b9d9fdd09b39d38b6c1d0efbc9c0069edb6.patch

Yes, I think these patches are what is needed.

comment:18 in reply to:  17 Changed 3 years ago by gk

Replying to mcs:

Replying to boklm:

After looking at the updater patches, I think the code changes to support a "hardened" channel are the following two patches:
https://github.com/boklm/gecko-dev/commit/e26ea6a9190c3cf8c8ea02cffebb462c70e50a16.patch
https://github.com/boklm/gecko-dev/commit/7c753b9d9fdd09b39d38b6c1d0efbc9c0069edb6.patch

Yes, I think these patches are what is needed.

These got merged as commit 517d9fec4b6d464d276410385942323124488255 and f799aa38f56abda676cacd8d435e16286edea772 into tor-browser-38.3.0esr-5.5-2.

comment:19 in reply to:  16 Changed 3 years ago by gk

Replying to boklm:

I added a hardened-builds-v2 branch on my tor-browser-bundle.git repo:
https://gitweb.torproject.org/user/boklm/tor-browser-bundle.git/log/?h=hardened-builds-v2

I have not tried to build it yet, but I think it includes the changes needed to have a "hardened" channel.

Looks good and works so far (although I did not test incremental update generation). One thing I forgot is to comment our the Windows and OS X builds in the Makefile. I'll do that when I create the hardened builds branch.

comment:20 Changed 3 years ago by gk

Resolution: fixed
Status: assignedclosed

I pushed the hardened-builds branch and tagged build1. We should be ready to go: the future is hardened!1!!

Note: See TracTickets for help on using tickets.