Opened 4 months ago

Last modified 2 weeks ago

#30665 needs_revision defect

Get Firefox 68 ESR Working with latest android toolchain

Reported by: sisbell Owned by: tbb-team
Priority: Very High Milestone:
Component: Applications/Tor Browser Version:
Severity: Normal Keywords: tbb-rbm, ff68-esr, tbb-9.0-must-alpha, TorBrowserTeam201909
Cc: boklm, sisbell, gk, sysrqb Actual Points:
Parent ID: #30324 Points: 2
Reviewer: Sponsor: Sponsor44-can

Description

Get Firefox 68 ESR working in offline mode with updated android toolchain. This task does not include tor-android-service integration.

It may include patches for Rust 1.34.1.

Child Tickets

Change History (19)

comment:2 Changed 4 months ago by gk

FWIW we need to slightly change the x86 target: https://hg.mozilla.org/mozilla-central/rev/2b2bd723ebc8.

comment:3 Changed 3 months ago by gk

Keywords: TorBrowserTeam201906 added; TorBrowserTeam201905 removed

Moving tickets to June

comment:4 Changed 2 months ago by gk

Keywords: TorBrowserTeam201907 added; TorBrowserTeam201906 removed

Moving tickets to July

comment:5 Changed 2 months ago by gk

Keywords: tbb-9.0-must-nightly added

Starting with 9.0 keywords

comment:6 Changed 6 weeks ago by pili

Sponsor: Sponsor44-can

Adding Sponsor 44 to ESR68 tickets

comment:7 Changed 6 weeks ago by gk

Keywords: TorBrowserTeam201908 added; TorBrowserTeam201907 removed

Moving tickets to August, part 1

comment:8 Changed 3 weeks ago by sisbell

I have branch esr68_0827. This is based off of 9.0-3 branch.

Branch esr68_0827-patched is based off of https://git.torproject.org/user/sysrqb/tor-browser.git : 2dbb7aefeaeb0499ac69c67df470e2ed6df8cb71.

Overall it looks very good. The patched branch builds and I tested the armv7 apk on Android Q. Tor starts and I'm able to access an external site.

Note the following:

  • I provided the android-jsocks.patch which removes the jsocks.aar dependency. This is only used for VPN support, which we don't need. I broke out the tor-android-service VPN code into its own module, which we now exclude from browser builds. This patch can be removed once a similar change is made in firefox code.
  • --disable-tor-browser-update does not work in this build so I removed it from mozconfig until support is added.
  • network is enabled for the container but I added --offline flag to gradle so its can only use our maven cache. I confirmed that nothing gradle-related is downloading in the current build.

comment:9 in reply to:  8 Changed 3 weeks ago by cypherpunks

Replying to sisbell:

Branch esr68_0827-patched is based off of https://git.torproject.org/user/sysrqb/tor-browser.git : 2dbb7aefeaeb0499ac69c67df470e2ed6df8cb71.

Usually, the links to branches are provided. The link to the commit is https://gitweb.torproject.org/user/sysrqb/tor-browser.git/commit/?h=acat30429%2b5_tor-browser_android_68esr_53&id=2dbb7aefeaeb0499ac69c67df470e2ed6df8cb71

Also TorBrowserTeam201908R, if it needs_review.

comment:10 in reply to:  8 ; Changed 3 weeks ago by gk

Replying to sisbell:

I have branch esr68_0827. This is based off of 9.0-3 branch.

Branch esr68_0827-patched is based off of https://git.torproject.org/user/sysrqb/tor-browser.git : 2dbb7aefeaeb0499ac69c67df470e2ed6df8cb71.

Overall it looks very good. The patched branch builds and I tested the armv7 apk on Android Q. Tor starts and I'm able to access an external site.

Compilation does not work for me if I build the x86 target at least. My tor-browser-build repo is at 571d14f36daa6dd1162803882475eae6ff20d3df. And I've pointed my firefox config to 2dbb7aefea for the nightly target and started the build with make nightly-android-x86. What I get is an error early in the browser step:

 0:51.43 FAILURE: Build failed with an exception.
 0:51.43 * What went wrong:
 0:51.43 Could not resolve all files for configuration ':app:withoutGeckoBinariesDebugAndroidTestRuntimeClasspath'.
 0:51.43 > Could not resolve net.freehaven.tor.control:jtorctl:0.2.
 0:51.43   Required by:
 0:51.43       project :app
 0:51.43    > No cached version of net.freehaven.tor.control:jtorctl:0.2 available for offline mode.
 0:51.43    > No cached version of net.freehaven.tor.control:jtorctl:0.2 available for offline mode.
 0:51.43    > No cached version of net.freehaven.tor.control:jtorctl:0.2 available for offline mode.
 0:51.43 > Could not resolve org.slf4j:slf4j-api:1.7.25.
 0:51.43   Required by:
 0:51.43       project :app
 0:51.43    > No cached version of org.slf4j:slf4j-api:1.7.25 available for offline mode.
 0:51.43    > No cached version of org.slf4j:slf4j-api:1.7.25 available for offline mode.
 0:51.43    > No cached version of org.slf4j:slf4j-api:1.7.25 available for offline mode.
 0:51.43 > Could not resolve org.slf4j:slf4j-android:1.7.25.
 0:51.43   Required by:
 0:51.43       project :app
 0:51.43    > No cached version of org.slf4j:slf4j-android:1.7.25 available for offline mode.
 0:51.43    > No cached version of org.slf4j:slf4j-android:1.7.25 available for offline mode.
 0:51.43    > No cached version of org.slf4j:slf4j-android:1.7.25 available for offline mode.

comment:11 in reply to:  10 Changed 3 weeks ago by sisbell

Those missing dependencies are in 0827-patched branch (not in 0827 branch, which doesn't require them). I think you need to clean out your gradle-dependencies-6 cache prior to building.

Replying to gk:

Replying to sisbell:

I have branch esr68_0827. This is based off of 9.0-3 branch.

Branch esr68_0827-patched is based off of https://git.torproject.org/user/sysrqb/tor-browser.git : 2dbb7aefeaeb0499ac69c67df470e2ed6df8cb71.

Overall it looks very good. The patched branch builds and I tested the armv7 apk on Android Q. Tor starts and I'm able to access an external site.

Compilation does not work for me if I build the x86 target at least. My tor-browser-build repo is at 571d14f36daa6dd1162803882475eae6ff20d3df. And I've pointed my firefox config to 2dbb7aefea for the nightly target and started the build with make nightly-android-x86. What I get is an error early in the browser step:

 0:51.43 FAILURE: Build failed with an exception.
 0:51.43 * What went wrong:
 0:51.43 Could not resolve all files for configuration ':app:withoutGeckoBinariesDebugAndroidTestRuntimeClasspath'.
 0:51.43 > Could not resolve net.freehaven.tor.control:jtorctl:0.2.
 0:51.43   Required by:
 0:51.43       project :app
 0:51.43    > No cached version of net.freehaven.tor.control:jtorctl:0.2 available for offline mode.
 0:51.43    > No cached version of net.freehaven.tor.control:jtorctl:0.2 available for offline mode.
 0:51.43    > No cached version of net.freehaven.tor.control:jtorctl:0.2 available for offline mode.
 0:51.43 > Could not resolve org.slf4j:slf4j-api:1.7.25.
 0:51.43   Required by:
 0:51.43       project :app
 0:51.43    > No cached version of org.slf4j:slf4j-api:1.7.25 available for offline mode.
 0:51.43    > No cached version of org.slf4j:slf4j-api:1.7.25 available for offline mode.
 0:51.43    > No cached version of org.slf4j:slf4j-api:1.7.25 available for offline mode.
 0:51.43 > Could not resolve org.slf4j:slf4j-android:1.7.25.
 0:51.43   Required by:
 0:51.43       project :app
 0:51.43    > No cached version of org.slf4j:slf4j-android:1.7.25 available for offline mode.
 0:51.43    > No cached version of org.slf4j:slf4j-android:1.7.25 available for offline mode.
 0:51.43    > No cached version of org.slf4j:slf4j-android:1.7.25 available for offline mode.

comment:12 Changed 2 weeks ago by gk

While skimming over the patch: please don't enable networking during build. What is the error you get?
Additionally, while you remove the patch for 1527534 in the build script you still include it in the config file.

comment:13 in reply to:  12 Changed 2 weeks ago by gk

Replying to gk:

While skimming over the patch: please don't enable networking during build. What is the error you get?

FWIW: I don't get any error when doing a

-      container:
-        # Some issues with gradle daemon running without network access. So enable for now.
-        disable_network:
-          build: 0

on top of your patch. So, compiling still works fine for me without network access.

comment:14 Changed 2 weeks ago by gk

Priority: MediumVery High
Status: newneeds_revision

Additionally, please revert pointing to sysrqb's repo. tor-browser-68.1.0esr-9.0 should be fine. Then you need to change

-  patch -p1 < $rootdir/android-dependencies.patch
+   patch -p1 < $rootdir/android-packages.patch
+   patch -p1 < $rootdir/android-remove-emulator.patch
+   patch -p1 < $rootdir/android-jsocks.patch

Indentation.

Is

+  cp /var/tmp/dist/android-toolchain/android-ndk/arm/lib64/libclang.so.6 /var/tmp/dis
t/android-toolchain/android-ndk/arm/lib64/libclang.so

needed for all architectures? If not, please copy it over only where appropriate. Just doing

+# Android does not support --enable-bundled-fonts option
+ ./mach configure --with-tor-browser-version=[% c("var/torbrowser_version") %] --with-distribution-id=org.torproject --enable-update-channel=[% c("var/torbrowser_update_channel") %] [% IF ! c("var/android") %]--enable-bundled-fonts[% END -%] --with-branding=[% c("var/branding_directory") %]

should be enough.

Please remove the change at +# ac_add_options --disable-tor-browser-update

comment:16 Changed 2 weeks ago by gk

Status: needs_reviewneeds_revision

Looks better, but you still have the patch for 1527534 in the config file and not deleted while removing it from the build script. Additionally, I am wondering why we need the libclang.so.6 at all and then just for arm? Does Mozilla do the same? Could you add a comment on elaborating why this is needed for that architecture?

I moved forward to apply the patches you have so far to have at least some nightly builds before we need to release the alpha. But we should fix up as many loose ends before that as we can.

comment:17 Changed 2 weeks ago by gk

Keywords: tbb-9.0-must-alpha added; tbb-9.0-must-nightly removed

Move must-nightly items to must-alpha ones.

comment:18 Changed 2 weeks ago by gk

Keywords: TorBrowserTeam201909 added; TorBrowserTeam201908 removed

Moving must-alpha tickets to September.

comment:19 Changed 2 weeks ago by pili

Points: 2
Note: See TracTickets for help on using tickets.