The build id stays the same with every Firefox update
#19400 (moved) was among other things a result of our hard-coded REFERENCE_DATETIME that gets used as the build id. We should change that with every Firefox release.
Activity
Some useful discussion we already had in #11506 (moved):
<boklm> I think there are different solutions: we can use the time of the commit from tor-browser-bundle.git. The drawback is that we can't do partial rebuilds anymore, as any commit will change the timestamp used everywhere we can use the time of the last commit on the gitian descriptor file. However, the descriptor files are not always changed between two releases we can use the time of the last commit from one of the remotes specified in the descriptor (tor-browser.git for the firefox descriptor) we can manually set a value for REFERENCE_DATETIME in gitian/versions and update it while preparing a new release <gk> Partial rebuilds are worth to have. But I actually don't see how any of the four possible solutions you brought up solves this bug while keeping this option. I mean, even if we set REFERENCE_DATETIME in gitian/versions the timestamps would be different (e.g. those embedded in binaries that don't get rebuilt). <boklm> If we set REFERENCE_DATETIME in gitian/versions, we can update its value when preparing a new release, but keep the same value for all the intermediate builds we do for one release. This does not allow partial rebuilds between two releases, but allows it for intermediate builds for one release.-
What about just updating
REFERENCE_DATETIMEin the Firefox descriptor if a new Firefox version is used or using a separate one (likeREFERENCE_DATETIME_FIREFOX) to begin with just for the Firefox part? That way we would solve both problems like #19400 (moved) and we could retain partial builds as we have them now: no need to rebuild all the artifacts if the Firefox version changes or if we only rebundle or are doing a buildN. 
Replying to gk:
What about just updating
REFERENCE_DATETIMEin the Firefox descriptor if a new Firefox version is used or using a separate one (likeREFERENCE_DATETIME_FIREFOX) to begin with just for the Firefox part? That way we would solve both problems like #19400 (moved) and we could retain partial builds as we have them now: no need to rebuild all the artifacts if the Firefox version changes or if we only rebundle or are doing a buildN.This looks like a good idea.
Replying to gk:
We could use the output of e.g.
cat browser/config/version.txtto construct something.Yes, we could probably convert the version number to an increasing number. However, if we release a Tor Browser update based on the same firefox version, then the build id will not change.
-
Replying to boklm:
Replying to gk:
We could use the output of e.g.
cat browser/config/version.txtto construct something.Yes, we could probably convert the version number to an increasing number. However, if we release a Tor Browser update based on the same firefox version, then the build id will not change.
That's perfectly fine as the idea is to prevent things like #19400 (moved) which only happen with Firefox updates.
-
In the branch
bug_19528-v1in my user repo, we are settingMOZ_BUILD_DATE(which is used as the build id) based on the Tor Browser version: https://gitweb.torproject.org/user/boklm/tor-browser-bundle.git/commit/?h=bug_19528-v1I have not yet tried doing a build using this branch. I can also change it to use the version from
browser/config/version.txtrather than the Tor Browser version if that's better.Here is the
MOZ_BUILD_DATEfor some values ofTORBROWSER_VERSION:$ TORBROWSER_VERSION=7.0 ./build-helpers/get-moz-build-date export MOZ_BUILD_DATE=20160208010101 $ TORBROWSER_VERSION=7.0.1 ./build-helpers/get-moz-build-date export MOZ_BUILD_DATE=20160208010201 $ TORBROWSER_VERSION=7.0.2 ./build-helpers/get-moz-build-date export MOZ_BUILD_DATE=20160208010301 $ TORBROWSER_VERSION=7.0.2.1 ./build-helpers/get-moz-build-date export MOZ_BUILD_DATE=20160208010302 $ TORBROWSER_VERSION=7.0a1 ./build-helpers/get-moz-build-date export MOZ_BUILD_DATE=20160108010201 -
Replying to boklm:
I have not yet tried doing a build using this branch. I can also change it to use the version from
browser/config/version.txtrather than the Tor Browser version if that's better.Yes, that would be better because currently this breaks building Tor Browser using already built artifacts in case we don't need to rebuild the browser part: Users building, say 7.0.2, from scratch would get a different result than we which just redid the bundle part and used, say 7.0.1, otherwise (maybe 7.0.2 just needed a newer Torbutton). So, we need something that changes only if a newer Firefox is built to construct
MOZ_BUILD_DATE. -
Replying to gk:
Replying to boklm:
I have not yet tried doing a build using this branch. I can also change it to use the version from
browser/config/version.txtrather than the Tor Browser version if that's better.Yes, that would be better because currently this breaks building Tor Browser using already built artifacts in case we don't need to rebuild the browser part: Users building, say 7.0.2, from scratch would get a different result than we which just redid the bundle part and used, say 7.0.1, otherwise (maybe 7.0.2 just needed a newer Torbutton). So, we need something that changes only if a newer Firefox is built to construct
MOZ_BUILD_DATE.However, we are already using the Tor Browser version as a firefox configure argument
--with-tor-browser-version=${TORBROWSER_VERSION}which is then used in a few places in the updater, so we already can't change the Tor Browser version without rebuilding the firefox part. -
Replying to boklm:
Replying to gk:
Replying to boklm:
I have not yet tried doing a build using this branch. I can also change it to use the version from
browser/config/version.txtrather than the Tor Browser version if that's better.Yes, that would be better because currently this breaks building Tor Browser using already built artifacts in case we don't need to rebuild the browser part: Users building, say 7.0.2, from scratch would get a different result than we which just redid the bundle part and used, say 7.0.1, otherwise (maybe 7.0.2 just needed a newer Torbutton). So, we need something that changes only if a newer Firefox is built to construct
MOZ_BUILD_DATE.However, we are already using the Tor Browser version as a firefox configure argument
--with-tor-browser-version=${TORBROWSER_VERSION}which is then used in a few places in the updater, so we already can't change the Tor Browser version without rebuilding the firefox part.Yes, that's true. But I think the right fix is not to depend on a non-Firefox version number and I still have hope to get #18325 (moved) solved some day. :) Thus, if we can avoid it I'd be happy to not create more dependencies on
TORBROWSER_VERSIONin the Firefox descriptors. -
Ah indeed. I created branch
bug_19528-v2which is using the firefox version: https://gitweb.torproject.org/user/boklm/tor-browser-bundle.git/commit/?h=bug_19528-v2I did not try a build yet with this branch.
A couple of quick comments:
- Can we automate the generation of the year portion? Leaving it as 2016 forever will be a strange thing to do (or if we plan to manually update it there is a good chance we will forget to do so). One approach is to use the same method we use for COPYRIGHT_YEAR in gitian/descriptors/mac/gitian-firefox.yml.
- We should carry the build id through to update_responses so that the update manifests include the correct build ID.
-
Replying to mcs:
A couple of quick comments:
- Can we automate the generation of the year portion? Leaving it as 2016 forever will be a strange thing to do (or if we plan to manually update it there is a good chance we will forget to do so). One approach is to use the same method we use for COPYRIGHT_YEAR in gitian/descriptors/mac/gitian-firefox.yml.
An other part of the script that we need to manually update is the esr branch number that we substract so that it is less than 30. To make sure we have an always increasing build ID when the version increase, I think we should update the esr number we substract and the year at the same time. So I think it is better to have the year manually updated too. I updated the script with a comment and separate variables for the year and esr branch to make updating that easier.
But there is still a good chance we will forget to update this. Maybe we can raise an error if the year used in the script and the year of the Tor Browser commit are different, so we don't forget to update it?
- We should carry the build id through to update_responses so that the update manifests include the correct build ID.
I pushed a new branch
bug_19528-v3doing that: https://gitweb.torproject.org/user/boklm/tor-browser-bundle.git/commit/?h=bug_19528-v3In the bundle step we extract the build id from the
application.inifile and put it in abuildid.txtfile. Theupdate_responsesscript then uses the value from that file as the build id. -
Replying to boklm:
... But there is still a good chance we will forget to update this. Maybe we can raise an error if the year used in the script and the year of the Tor Browser commit are different, so we don't forget to update it?
This seems like a good idea, but if gk and you think it is sufficient to document this as part of our build procedures then just adding a mention there seems OK to me.
- We should carry the build id through to update_responses so that the update manifests include the correct build ID.
I pushed a new branch
bug_19528-v3doing that: https://gitweb.torproject.org/user/boklm/tor-browser-bundle.git/commit/?h=bug_19528-v3r=brade, r=mcs We did not run a test build, but the changes look good.
-
So, hrm. I am not so happy with keeping an extra buildid.txt file around. Can't we extract this info once from one of the full MAR files just before we are creating the update response files?
I am not sure I understand the need for a hard-coded oldest supported ESR version? Could you elaborate?
I am interested in getting rid of hard-coding the year as well but have not looked at how we could do that yet.
Trac:
Status: needs_review to needs_information