Opened 7 months ago

Closed 4 months ago

#28685 closed defect (fixed)

Tor Browser for Android needs a more dynamic Build ID

Reported by: sysrqb Owned by: tbb-team
Priority: High Milestone:
Component: Applications/Tor Browser Version:
Severity: Normal Keywords: tbb-mobile, tbb-rbm, TBA-a3, TorBrowserTeam201902R
Cc: gk, sisbell, boklm Actual Points:
Parent ID: #28708 Points:
Reviewer: Sponsor: Sponsor8

Description

Currently, the build id only changes when the Firefox version changes and when the copyright year changes. Unfortunately, when building the APK, Mozilla expect the build id changes between every build. They use the build id and derive the Android version code (version number) from this. Unfortunately, they only use part of the buildid and they discard lower bits. This doesn't fit well with the current get-moz-build-date script because:

1) if the firefox build date and copyright year remain the same, then the build id remains the same
2) if the patch version increases (such as 60.3.0 to 60.3.1), then the Android version code doesn't change (due to discarded lower bits)

I think we can solve 1. by incorporating the tor browser version number into the calculation. Solving 2. may require multiplying by at least 3600 (therefore offsets the division by 60*60), i think.
https://gitweb.torproject.org/tor-browser.git/tree/python/mozbuild/mozbuild/android_version_code.py?h=tor-browser-60.3.0esr-8.5-1#n33

$ perl get-moz-build-date 2018 60.3.0
export MOZ_BUILD_DATE=20180204040101

$ python python/mozbuild/mozbuild/android_version_code.py --verbose --with-android-cpu-arch armeabi-v7a --with-android-min-sdk-version 16 --with-android-max-sdk-version 26 20180204040101
2015539361


$ perl get-moz-build-date 2018 60.3.1
export MOZ_BUILD_DATE=20180204040201

$ python python/mozbuild/mozbuild/android_version_code.py --verbose --with-android-cpu-arch armeabi-v7a --with-android-min-sdk-version 16 --with-android-max-sdk-version 26 20180204040201
2015539361


$ perl get-moz-build-date 2018 60.4.0
export MOZ_BUILD_DATE=20180204050101

$ python python/mozbuild/mozbuild/android_version_code.py --verbose --with-android-cpu-arch armeabi-v7a --with-android-min-sdk-version 16 --with-android-max-sdk-version 26 20180204050101
2015539369

Notice the android version code is the same for 60.3.0 and 60.3.1.

Currently, the script with the current Tor Browser Build ID is:

import math
import time
                                                                                                                                                                                                                   
fmt = '%Y%m%d%H%M%S'
buildid = "20180204040101"
V1_CUTOFF = 20150801000000

build = time.strptime(str(buildid), fmt)
cutoff = time.strptime(str(V1_CUTOFF), fmt)

base = int(math.floor((time.mktime(build) - time.mktime(cutoff)) / (60.0 * 60.0)))
version = 0b1111000001000000000000000000000
version |= base << 3
version |= 1 << 0
print(version)

Something else I noticed is this current scheme only provides 8 android version codes between 60.3.0 (2015539361) and 60.4.0 (2015539369). We probably want more than that - but this may be accomplished when adjusting the other bits.

Child Tickets

Change History (13)

comment:1 Changed 7 months ago by gk

Cc: boklm added
Component: Applications/rbmApplications/Tor Browser
Keywords: tbb-rbm TorBrowserTeam201812 added
Owner: changed from boklm to tbb-team
Priority: MediumHigh

comment:2 Changed 6 months ago by gk

Parent ID: #28708

comment:3 Changed 6 months ago by gk

Keywords: TBA-a3 added

Setting tag for third Tor Browser for Android alpha milestone.

comment:4 Changed 6 months ago by gk

Sponsor: Sponsor8

Adding Sponsor8 tag.

comment:5 Changed 6 months ago by boklm

#19528 is the ticket where we added the get-moz-build-date script generating a build id based on the firefox version.

To solve the issue of two Tor Browser versions based on the same Firefox having the same build id, I think we could:

  • use the Tor Browser version instead of the firefox version to generate a build id. I think the reason we did not do that was that it would make it harder to do #18325.
  • Use the tor-browser.git commit date as the build id. This should fix the issue when a new release includes a new tor-browser.git commit, however it is still possible to have two releases with the same build id if they use the same tor-browser.git commit.
  • Use the tor-browser.git commit date, and add to it the Tor Browser version.

comment:6 Changed 5 months ago by gk

Keywords: TorBrowserTeam201901 added; TorBrowserTeam201812 removed

Moving tickets to Jan 2019.

comment:7 Changed 4 months ago by gk

Keywords: TorBrowserTeam201902 added; TorBrowserTeam201901 removed

Moving tickets to February.

comment:8 in reply to:  5 Changed 4 months ago by sysrqb

Replying to boklm:

#19528 is the ticket where we added the get-moz-build-date script generating a build id based on the firefox version.

To solve the issue of two Tor Browser versions based on the same Firefox having the same build id, I think we could:

  • use the Tor Browser version instead of the firefox version to generate a build id. I think the reason we did not do that was that it would make it harder to do #18325.
  • Use the tor-browser.git commit date as the build id. This should fix the issue when a new release includes a new tor-browser.git commit, however it is still possible to have two releases with the same build id if they use the same tor-browser.git commit.
  • Use the tor-browser.git commit date, and add to it the Tor Browser version.

Hrm. #18325 is interesting. Android is the only platform where we want exactly what gk wants to avoid. Would it be crazy if we used a different buildid on desktop and Android? I see the benefit of having #18325, but considering on Android we can't install an update if it has the same buildid as the currently-installed version, we must bump the buildid for each release - regardless of whether the release changes any firefox code.

Alternatively, we change how the Android versionCode is created for TBA. This will result in one more patch we carry for tor-browser, but then we can avoid the conflict with #18325 and maybe tor-browser-build can set the Android versionCode independently.

comment:9 in reply to:  5 Changed 4 months ago by gk

Replying to boklm:

#19528 is the ticket where we added the get-moz-build-date script generating a build id based on the firefox version.

To solve the issue of two Tor Browser versions based on the same Firefox having the same build id, I think we could:

  • use the Tor Browser version instead of the firefox version to generate a build id. I think the reason we did not do that was that it would make it harder to do #18325.

I think that's the way to go for now. I am not convinced yet that it would be substantially harder to solve #18325 with that change either. But even if it were so, we should not block on that to get our Android builds fixed right now.

comment:10 Changed 4 months ago by boklm

Keywords: TorBrowserTeam201902R added; TorBrowserTeam201902 removed
Status: newneeds_review

I pushed a patch in branch bug_28685_v3 using the Tor Browser version to compute a MOZ_BUILD_DATE, instead of the firefox version:
https://gitweb.torproject.org/user/boklm/tor-browser-build.git/commit/?h=bug_28685_v3&id=9d6d3f15319cfaf2efdf5677e1e2e9cce022e509

With Tor Browser 8.0.6 and Firefox 60.4.0, the build id was 20190204050101.

With this change the build id would be:

$ ./projects/firefox/get-moz-build-date 2019 8.0.7
export MOZ_BUILD_DATE=20190302080101
$ ./projects/firefox/get-moz-build-date 2019 8.5a9
export MOZ_BUILD_DATE=20190306100101
$ ./projects/firefox/get-moz-build-date 2019 8.5a10
export MOZ_BUILD_DATE=20190306110101
$ ./projects/firefox/get-moz-build-date 2019 8.5
export MOZ_BUILD_DATE=20190307010101
$ ./projects/firefox/get-moz-build-date 2019 8.5.1
export MOZ_BUILD_DATE=20190307020101

comment:11 in reply to:  10 ; Changed 4 months ago by gk

Keywords: TorBrowserTeam201902 added; TorBrowserTeam201902R removed
Status: needs_reviewneeds_revision

Replying to boklm:

I pushed a patch in branch bug_28685_v3 using the Tor Browser version to compute a MOZ_BUILD_DATE, instead of the firefox version:
https://gitweb.torproject.org/user/boklm/tor-browser-build.git/commit/?h=bug_28685_v3&id=9d6d3f15319cfaf2efdf5677e1e2e9cce022e509

With Tor Browser 8.0.6 and Firefox 60.4.0, the build id was 20190204050101.

With this change the build id would be:

$ ./projects/firefox/get-moz-build-date 2019 8.0.7
export MOZ_BUILD_DATE=20190302080101
$ ./projects/firefox/get-moz-build-date 2019 8.5a9
export MOZ_BUILD_DATE=20190306100101
$ ./projects/firefox/get-moz-build-date 2019 8.5a10
export MOZ_BUILD_DATE=20190306110101
$ ./projects/firefox/get-moz-build-date 2019 8.5
export MOZ_BUILD_DATE=20190307010101
$ ./projects/firefox/get-moz-build-date 2019 8.5.1
export MOZ_BUILD_DATE=20190307020101

We shipped 8.0.6 with 60.5.1esr, no? It seems we should adapt the comment in the script accordingly.

The new scheme makes the calculation save until Tor Browser 18, if I see this correctly. That's okay for me.

comment:12 in reply to:  11 Changed 4 months ago by boklm

Keywords: TorBrowserTeam201902R added; TorBrowserTeam201902 removed
Status: needs_revisionneeds_review

Replying to gk:

We shipped 8.0.6 with 60.5.1esr, no? It seems we should adapt the comment in the script accordingly.

I fixed that in branch bug_28685_v4:
https://gitweb.torproject.org/user/boklm/tor-browser-build.git/commit/?h=bug_28685_v4&id=d11f03f7c10c2cf5a942a353d0961249ceee9644

The new scheme makes the calculation save until Tor Browser 18, if I see this correctly. That's okay for me.

We currently subtract 5, so Tor Browser 17 would be month 12. But we can increase the value we subtract when changing year, to allow more versions.

comment:13 Changed 4 months ago by gk

Resolution: fixed
Status: needs_reviewclosed

Looks good. Commit d11f03f7c10c2cf5a942a353d0961249ceee9644 on master has the patch. I've filed #29615 for follow-up work.

Note: See TracTickets for help on using tickets.