#25164 closed enhancement (fixed)

Reproducible Tor Browser for Android builds

Reported by: sysrqb Owned by: tbb-team
Priority: Very High Milestone:
Component: Applications/Tor Browser Version:
Severity: Normal Keywords: tbb-mobile, TorBrowserTeam201812R
Cc: boklm, sisbell, igt0 Actual Points:
Parent ID: Points:
Reviewer: Sponsor: Sponsor8

Description

Make sure the apks builds are reproducable.

Child Tickets

TicketTypeStatusOwnerSummary
#28696defectclosedtbb-teamChanging paths to Gradle dependencies are included in build
#28697defectclosedtbb-teamOur QA and testing .apks are signed with a key per build

Change History (31)

comment:1 Changed 23 months ago by gk

Summary: Reproducable Tor Browser for Android buildsReproducible Tor Browser for Android builds

comment:2 Changed 22 months ago by boklm

Cc: boklm added

comment:3 Changed 22 months ago by sysrqb

Parent ID: #19675#5709

Moving these to #5709, these aren't blockers anymore. We'll merge Orfox patches first, then audit everything as we move to m-c and work on TBA. (We should create more tickets as we find more items that need investigating/fixing)

comment:4 Changed 21 months ago by gk

Keywords: TorBrowserTeam201804 added
Priority: MediumHigh

comment:5 Changed 20 months ago by gk

Keywords: TorBrowserTeam201805 added; TorBrowserTeam201804 removed

Move our roadmap tickets to May.

comment:6 Changed 19 months ago by gk

Keywords: TorBrowserTeam201806 added; TorBrowserTeam201805 removed

Moving our tickets to June 2018

comment:7 Changed 18 months ago by gk

Keywords: TorBrowserTeam201807 added; TorBrowserTeam201806 removed

Moving first batch of tickets to July 2018

comment:8 Changed 18 months ago by gk

Cc: sisbell added

comment:9 Changed 18 months ago by igt0

Cc: igt0 added

comment:10 Changed 17 months ago by boklm

Parent ID: #5709#26693

comment:11 Changed 17 months ago by sisbell

I've implemented building an Android apk with RBM at: https://github.com/sisbell/tor-browser-build/tree/android

I'll open a git project at torproject and import it over for further review and cleanup.

comment:12 in reply to:  11 Changed 17 months ago by boklm

Replying to sisbell:

I've implemented building an Android apk with RBM at: https://github.com/sisbell/tor-browser-build/tree/android

This is good work. Thanks!

Here are a few quick comments:

  • the main ticket for integrating the android build into tor-browser-build is not #25164 but #26693. This one is only for the reproducible part of it (checking that multiple builds on multiple machines create the same build).
  • In the commit subject you should use Bug XXXX: (even if the ticket is of type enhancement).
  • to make it easier to review, you should split the commits in different commits. I think one for adding stretch to debootstrap-image, for adding android-toolchain, for adding the new platform definition to rbm.conf, for fixing the rust build, for fixing the firefox build.
  • should we call the target in rbm.conf torbrowser-android-armv7 instead of just torbrowser-android?
  • var/compiler should be set to android-toolchain instead of gcc.
  • ndk.zip should be included in android-toolchain so that is not needed to download it from projects/firefox/config.

comment:13 Changed 17 months ago by gk

Nice! Do you get identical .apk files when building it several times? If not, this would need to get investigated and fixed as well.

comment:14 Changed 17 months ago by gk

Keywords: TorBrowserTeam201808 added; TorBrowserTeam201807 removed

Move our tickets to August.

comment:15 Changed 16 months ago by sisbell

Apks from two builds are different, even building on same laptop. This is due to some differences in the classes.dex file.

When I do a dexdump of two different dex files, the dexdumps are exactly the same.

When I decompile and recompile the classes.dex file from two different builds, the resulting classes.dex are exactly the same.

This leads me to believe that the differences in the classes.dex file are only cosmetic (padding differences?)

I will need to build a quick program to output exactly what part of the dex file format is different and then go from there.

comment:16 Changed 16 months ago by sisbell

It looks like the annotation offsets in the data area of the dex file are different between the files.

comment:17 Changed 16 months ago by gk

Keywords: TorBrowserTeam201809 added; TorBrowserTeam201808 removed

Moving our tickets to September 2018

comment:18 Changed 15 months ago by sisbell

I've published first version of tool to help identify differences that we are seeing in the Android dex files.

The outputs will give us enough information to start drilling down when we need to. I'm going to pause further work for now and move on to other tickets.

comment:19 Changed 15 months ago by gk

Keywords: TorBrowserTeam201810 added; TorBrowserTeam201809 removed

Moving tickets to October

comment:20 Changed 14 months ago by gk

Parent ID: #26693

That's not necessary for the tor-browser-build integration, unparenting.

comment:21 Changed 14 months ago by pili

Sponsor: Sponsor8

comment:22 Changed 14 months ago by gk

Keywords: TorBrowserTeam201811 added; TorBrowserTeam201810 removed

Moving our tickets to November.

comment:23 Changed 13 months ago by gk

Priority: HighVery High

Now that we have the build integrated this item is the biggest issue left regarding our build process

comment:24 Changed 13 months ago by gk

Keywords: TorBrowserTeam201812 added; TorBrowserTeam201811 removed

I have some good news here. I compared the .apk files boklm and I produced during the 8.5a5 build and there are only two differences which should not be too hard to fix. I mention them here and will open child bugs for actually solving the issues (leaving this one as the parent bug in case we find out more problems while testing).

1) The path to the container gets included via the GRADLE_MAVEN_REPOSITORIES which results in something like "file:///var/tmp/tmp.Flce6AvlTV/gradle-dependencies-3" vs. "file:///var/tmp/tmp.AKrQtlURro/gradle-dependencies-3".

2) There are different keys used in our debug .apks as they are created new per build at the moment.

comment:25 Changed 13 months ago by sisbell

I verified from my side that the classes.dex files are the same between two different apk builds on my linux laptop. The problem I was seeing earlier is no longer an issue for me.

comment:26 Changed 12 months ago by gk

Resolution: fixed
Status: newclosed

We are good here. I added the .apk to our sha265sum step now that it is reproducible in commit 3d7c32b67b25d869d94ec8f23da82e7558fb9993 to get our build going. boklm: let me know whether you are good with that commit.

comment:27 Changed 12 months ago by boklm

Resolution: fixed
Status: closedreopened

(I am temporarily reopening this ticket to set it as parent for #28697)

comment:28 Changed 12 months ago by boklm

Resolution: fixed
Status: reopenedclosed

comment:29 in reply to:  26 Changed 12 months ago by boklm

Replying to gk:

We are good here. I added the .apk to our sha265sum step now that it is reproducible in commit 3d7c32b67b25d869d94ec8f23da82e7558fb9993 to get our build going. boklm: let me know whether you are good with that commit.

This commit looks good to me.

comment:30 Changed 12 months ago by gk

Resolution: fixed
Status: closedreopened

comment:31 Changed 12 months ago by gk

Keywords: TorBrowserTeam201812R added; TorBrowserTeam201812 removed
Resolution: fixed
Status: reopenedclosed
Note: See TracTickets for help on using tickets.