Reproducible Tor Browser for Android builds

Make sure the apks builds are reproducable.

added TorBrowserTeam201812R component::applications/tor browser owner::tbb-team priority::very high resolution::fixed severity::normal sponsor::8 status::closed tbb-mobile type::enhancement labels

Trac:
Summary: Reproducable Tor Browser for Android builds to Reproducible Tor Browser for Android builds

Trac:
Cc: N/A to boklm

Moving these to #5709 (moved), these aren't blockers anymore. We'll merge Orfox patches first, then audit everything as we move to m-c and work on TBA. (We should create more tickets as we find more items that need investigating/fixing)

Trac:
Parent: #19675 (moved) to #5709 (moved)

Trac:
Priority: Medium to High
Keywords: N/A deleted, TorBrowserTeam201804 added

Move our roadmap tickets to May.

Trac:
Keywords: TorBrowserTeam201804 deleted, TorBrowserTeam201805 added

Moving our tickets to June 2018

Trac:
Keywords: TorBrowserTeam201805 deleted, TorBrowserTeam201806 added

Moving first batch of tickets to July 2018

Trac:
Keywords: TorBrowserTeam201806 deleted, TorBrowserTeam201807 added

Trac:
Cc: boklm to boklm, sisbell

Trac:
Cc: boklm, sisbell to boklm, sisbell, igt0

Trac:
Parent: #5709 (moved) to #26693 (moved)

I've implemented building an Android apk with RBM at: https://github.com/sisbell/tor-browser-build/tree/android

I'll open a git project at torproject and import it over for further review and cleanup.

Replying to sisbell:

I've implemented building an Android apk with RBM at: https://github.com/sisbell/tor-browser-build/tree/android

This is good work. Thanks!

Here are a few quick comments:

• the main ticket for integrating the android build into tor-browser-build is not #25164 (moved) but #26693 (moved). This one is only for the reproducible part of it (checking that multiple builds on multiple machines create the same build).
• In the commit subject you should use Bug XXXX: (even if the ticket is of type enhancement).
• to make it easier to review, you should split the commits in different commits. I think one for adding stretch to debootstrap-image, for adding android-toolchain, for adding the new platform definition to rbm.conf, for fixing the rust build, for fixing the firefox build.
• should we call the target in rbm.conf torbrowser-android-armv7 instead of just torbrowser-android?
• var/compiler should be set to android-toolchain instead of gcc.
• ndk.zip should be included in android-toolchain so that is not needed to download it from projects/firefox/config.

Nice! Do you get identical .apk files when building it several times? If not, this would need to get investigated and fixed as well.

Move our tickets to August.

Trac:
Keywords: TorBrowserTeam201807 deleted, TorBrowserTeam201808 added

Apks from two builds are different, even building on same laptop. This is due to some differences in the classes.dex file.

When I do a dexdump of two different dex files, the dexdumps are exactly the same.

When I decompile and recompile the classes.dex file from two different builds, the resulting classes.dex are exactly the same.

This leads me to believe that the differences in the classes.dex file are only cosmetic (padding differences?)

I will need to build a quick program to output exactly what part of the dex file format is different and then go from there.

It looks like the annotation offsets in the data area of the dex file are different between the files.

Moving our tickets to September 2018

Trac:
Keywords: TorBrowserTeam201808 deleted, TorBrowserTeam201809 added

I've published first version of tool to help identify differences that we are seeing in the Android dex files.

• https://github.com/sisbell/dexd
• https://crates.io/crates/dexd

The outputs will give us enough information to start drilling down when we need to. I'm going to pause further work for now and move on to other tickets.

Moving tickets to October

Trac:
Keywords: TorBrowserTeam201809 deleted, TorBrowserTeam201810 added

That's not necessary for the tor-browser-build integration, unparenting.

Trac:
Parent: #26693 (moved) to N/A

Trac:
Sponsor: N/A to Sponsor8

Moving our tickets to November.

Trac:
Keywords: TorBrowserTeam201810 deleted, TorBrowserTeam201811 added

Now that we have the build integrated this item is the biggest issue left regarding our build process

Trac:
Priority: High to Very High

I have some good news here. I compared the .apk files boklm and I produced during the 8.5a5 build and there are only two differences which should not be too hard to fix. I mention them here and will open child bugs for actually solving the issues (leaving this one as the parent bug in case we find out more problems while testing).

1. The path to the container gets included via the GRADLE_MAVEN_REPOSITORIES which results in something like "file:///var/tmp/tmp.Flce6AvlTV/gradle-dependencies-3" vs. "file:///var/tmp/tmp.AKrQtlURro/gradle-dependencies-3".

2. There are different keys used in our debug .apks as they are created new per build at the moment.

Trac:
Keywords: TorBrowserTeam201811 deleted, TorBrowserTeam201812 added

I verified from my side that the classes.dex files are the same between two different apk builds on my linux laptop. The problem I was seeing earlier is no longer an issue for me.

We are good here. I added the .apk to our sha265sum step now that it is reproducible in commit 3d7c32b67b25d869d94ec8f23da82e7558fb9993 to get our build going. boklm: let me know whether you are good with that commit.

Trac:
Resolution: N/A to fixed
Status: new to closed

(I am temporarily reopening this ticket to set it as parent for #28697 (moved))

Trac:
Status: closed to reopened
Resolution: fixed to N/A

Trac:
Resolution: N/A to fixed
Status: reopened to closed

Replying to gk:

We are good here. I added the .apk to our sha265sum step now that it is reproducible in commit 3d7c32b67b25d869d94ec8f23da82e7558fb9993 to get our build going. boklm: let me know whether you are good with that commit.

This commit looks good to me.

Trac:
Resolution: fixed to N/A
Status: closed to reopened

Trac:
Keywords: TorBrowserTeam201812 deleted, TorBrowserTeam201812R added
Resolution: N/A to fixed
Status: reopened to closed

closed

mentioned in issue #26974 (moved)

mentioned in issue #26975 (moved)

mentioned in issue #26976 (moved)

mentioned in issue #26977 (moved)

mentioned in issue #28697 (moved)

mentioned in issue #28754 (moved)

moved to tpo/applications/tor-browser#25164 (closed)

mentioned in issue tpo/applications/tor-browser#26975 (closed)

mentioned in issue tpo/applications/tor-browser-build#28754 (closed)