Opened 2 years ago

Closed 6 months ago

#28704 closed defect (fixed)

Compile Tor and dependencies on our own for Android

Reported by: gk Owned by: sisbell
Priority: High Milestone:
Component: Applications/Tor Browser Version:
Severity: Normal Keywords: tbb-mobile, tbb-rbm, tbb-parity, TorBrowserTeam202004
Cc: sysrqb, gk, sisbell, eighthave, tbb-team Actual Points:
Parent ID: #33659 Points: 0
Reviewer: Sponsor: Sponsor58

Description

Currently we are building just Orbot in tor-browser-build and fetching the dependencies as we need them. We should at least build Tor and its dependencies on our own, integrating Android specific build logic into our projects we already have (like OpenSSL, Libevent etc.).

This is the parent ticket for that task.

Child Tickets

TicketStatusOwnerSummaryComponent
#28763closedsisbellCreate Tor-Android ProjectApplications/Tor Browser
#28764closedsisbellOpenSSL Build for AndroidApplications/Tor Browser
#28765closedsisbellLibEvent Build for AndroidApplications/Tor Browser
#28766closedsisbellTor Build for AndroidApplications/Tor Browser
#31499closedsisbellUpdate libevent to 2.1.11-stableApplications/Tor Browser
#32991closedtbb-teamTBB Project For ZSTDApplications/Tor Browser
#32992closedtbb-teamTBB Project for LZMAApplications/Tor Browser
#32993closedtbb-teamPackage Tor With Tor Android Service ProjectApplications/Tor Browser
#33215closedtbb-teamAndroid Toolchain: Add NDK bin path to system pathApplications/Tor Browser
#33216closedtbb-teamAdd Android Host and ABI Info to RBM.confApplications/Tor Browser
#33685closedtbb-teamAdd Support for Building zlib for AndroidApplications/Tor Browser
#33877closedtbb-teamDisable Samples and Regression tests For Libevent BuildApplications/Tor Browser

Change History (38)

comment:1 Changed 2 years ago by gk

Cc: sisbell added
Keywords: TorBrowserTeam201812 added
Priority: MediumHigh

comment:2 Changed 22 months ago by gk

Keywords: TorBrowserTeam201901 added; TorBrowserTeam201812 removed

Moving tickets to Jan 2019.

comment:3 Changed 20 months ago by gk

Keywords: tbb-parity added

tbb-parity items.

comment:4 Changed 14 months ago by eighthave

Cc: hans@… added

comment:5 Changed 14 months ago by eighthave

It seems that Android is moving to clang, and clang has more limited options for reproducibility. I think GCC support is still there, but deprecated. I found some specific issues on what is missing from clang here:
https://reproducible-builds.org/docs/build-path/

comment:6 Changed 13 months ago by eighthave

I have a barebones build job for GitLab-CI for this, you can see a test run here:
https://gitlab.com/eighthave/tor/-/jobs/301739923

And the code here:
https://gitlab.com/eighthave/tor/blob/0f2d18708609a52eccd4240409d2c219274e5e55/.gitlab-ci.yml#L56

I'll submit the .gitlab-ci.yml changes for inclusion once its ready.

comment:7 Changed 13 months ago by sysrqb

Points: 0

comment:8 Changed 13 months ago by eighthave

I justed looked through it, you could get rid of tor-android in TBB if you build libevent, openssl, lzma, zstd, and tor. The build configuration for each of those can be taken from https://github.com/guardianproject/tor-android, especially after this merge request, which should simplify things a bunch:

https://github.com/guardianproject/tor-android/pull/21

comment:9 Changed 12 months ago by sysrqb

Keywords: TorBrowserTeam201911 added; TorBrowserTeam201901 removed
Owner: changed from tbb-team to sisbell
Status: newassigned

comment:10 Changed 12 months ago by sysrqb

Cc: tbb-team added

comment:11 Changed 11 months ago by pili

Keywords: TorBrowserTeam201912 added; TorBrowserTeam201911 removed

Moving tickets to December

comment:12 Changed 10 months ago by sysrqb

Keywords: TorBrowserTeam202001 added; TorBrowserTeam201912 removed

comment:13 Changed 9 months ago by sisbell

Cc: sysrqb gk added
Keywords: TorBrowserTeam202001R added; TorBrowserTeam202001 removed
Status: assignedneeds_review

I have 4 commit covering child tickets:

Bug 32993: Package Tor With Tor Android Service Project
    Bug 28766: Tor Build for Android
    Bug 28765: LibEvent Build for Android
    Bug 28764: OpenSSL Build for Android
https://github.com/sisbell/tor-browser-build/commits/bug-28766c

I verified that tor starts up correctly with the armv7 build on a device. The browser loads and displays pages. I have not verified reproducibility yet.

comment:14 in reply to:  13 Changed 9 months ago by boklm

Keywords: TorBrowserTeam202001 added; TorBrowserTeam202001R removed
Status: needs_reviewneeds_revision

Replying to sisbell:

I have 4 commit covering child tickets:

Bug 32993: Package Tor With Tor Android Service Project
    Bug 28766: Tor Build for Android
    Bug 28765: LibEvent Build for Android
    Bug 28764: OpenSSL Build for Android
https://github.com/sisbell/tor-browser-build/commits/bug-28766c

I verified that tor starts up correctly with the armv7 build on a device. The browser loads and displays pages. I have not verified reproducibility yet.

Some comments/questions:

  • in the builds of openssl, libevent, tor, you are setting ANDROID_NDK_HOME and adding $ANDROID_NDK_HOME to PATH. Shouldn't that be done in var/setup in projects/android-toolchain/config instead of in each project?
  • could you explain (for example in the commit message) why libevent needs to be updated from 2.1.8 to 2.1.11?
  • could you explain why the --disable-libevent-regress --disable-samples options are needed to build libevent (and whether it would be useful to use them on other platforms too)?
  • in the libevent build, you are adding CC=clang to the configure for all platforms. I think it should be added to the android builds only as it doesn't seem to be needed for other platforms.
  • it seems we could have a var/configure_opt for android in rbm.conf containing something like CC=clang --host=[% c("var/host") %] [% c("var/configure_opt_project") %], where var/configure_opt_project is defined in each project to define options specific to this project
  • I think the name var/host is a little confusing and could be renamed to var/configure_host to make it more clear where it's used

comment:15 Changed 9 months ago by sisbell

Status: needs_revisionneeds_review

I've made progress to getting libraries running as shared libraries but have hit a blocker that I need help with

The problem occurs when I try to run tor browser. I get a missing symbol where libTor can't find a symbol from openssl

  10744 10744 F linker  : CANNOT LINK EXECUTABLE "/data/app/org.torproject.torbrowser-o2cGxqnhQbWOPhFKKcOFiA==/lib/arm/libTor.so": cannot locate symbol "SSL_CTX_set_security_level" referenced by "/data/app/org.torproject.torbrowser-o2cGxqnhQbWOPhFKKcOFiA==/lib/arm/libTor.so"...

Looking at the libraries, we see the function undefined in libTor. So I'm wondering what flag I need to set to make sure these are included. I'm building with clang.

$ readelf -Ws libTor.so | grep set_security
  3819: 00000000     0 FUNC    GLOBAL DEFAULT  UND SSL_CTX_set_security_level@OPENSSL_1_1_0 (5)
110559: 00000000     0 FUNC    GLOBAL DEFAULT  UND SSL_CTX_set_security_level
$ readelf -Ws libssl.so | grep set_security
   537: 000328bc    12 FUNC    GLOBAL DEFAULT   11 SSL_CTX_set_security_level@@OPENSSL_1_1_0
   543: 000328d4    12 FUNC    GLOBAL DEFAULT   11 SSL_CTX_set_security_callback@@OPENSSL_1_1_0
   679: 0003288c    12 FUNC    GLOBAL DEFAULT   11 SSL_set_security_callback@@OPENSSL_1_1_0
   681: 00032874    12 FUNC    GLOBAL DEFAULT   11 SSL_set_security_level@@OPENSSL_1_1_0
  2743: 000328bc    12 FUNC    GLOBAL DEFAULT   11 SSL_CTX_set_security_level
  2749: 000328d4    12 FUNC    GLOBAL DEFAULT   11 SSL_CTX_set_security_callback
  2885: 0003288c    12 FUNC    GLOBAL DEFAULT   11 SSL_set_security_callback
  2887: 00032874    12 FUNC    GLOBAL DEFAULT   11 SSL_set_security_level

I have the changes here
https://github.com/sisbell/tor-browser-build/commits/bug-28766e

The tor specific changes are here
https://github.com/sisbell/tor-browser-build/commit/b6d9c2edf8cb46e484413554ba76dce9560eef5d

comment:16 Changed 9 months ago by boklm

Does running ldd on libTor.so says that it is linked to libssl.so?

comment:17 in reply to:  16 Changed 9 months ago by sisbell

Replying to boklm:

Does running ldd on libTor.so says that it is linked to libssl.so?

Running ldd I get message

not a dynamic executable

The elf header type

  Type:DYN (Shared object file)

Reading elf dependencies

 0x00000001 (NEEDED)                     Shared library: [libz.so]
 0x00000001 (NEEDED)                     Shared library: [libm.so]
 0x00000001 (NEEDED)                     Shared library: [libssl.so]
 0x00000001 (NEEDED)                     Shared library: [libcrypto.so]
 0x00000001 (NEEDED)                     Shared library: [liblog.so]
 0x00000001 (NEEDED)                     Shared library: [libdl.so]
 0x00000001 (NEEDED)                     Shared library: [libc.so]

comment:18 in reply to:  15 ; Changed 9 months ago by sysrqb

Replying to sisbell:

I've made progress to getting libraries running as shared libraries but have hit a blocker that I need help with

The problem occurs when I try to run tor browser. I get a missing symbol where libTor can't find a symbol from openssl

  10744 10744 F linker  : CANNOT LINK EXECUTABLE "/data/app/org.torproject.torbrowser-o2cGxqnhQbWOPhFKKcOFiA==/lib/arm/libTor.so": cannot locate symbol "SSL_CTX_set_security_level" referenced by "/data/app/org.torproject.torbrowser-o2cGxqnhQbWOPhFKKcOFiA==/lib/arm/libTor.so"...
}}}]

Taking a guess, tor is probably using a version of openssl shipped with Android. I doubt apps look in their lib/ directory by default for libraries.

As an example, Mozilla loads shared libraries manually: https://searchfox.org/mozilla-central/source/mobile/android/geckoview/src/main/java/org/mozilla/gecko/mozglue/GeckoLoader.java#410

You can try setting LD_LIBRARY_PATH on the command line, for testing. If that works, then we should think about how we should solve this problem. There are a few options. One option is we modify TOPL:

https://github.com/thaliproject/Tor_Onion_Proxy_Library/blob/master/universal/src/main/java/com/msopentech/thali/toronionproxy/OnionProxyManager.java#L567

comment:19 Changed 9 months ago by boklm

Status: needs_reviewneeds_revision

comment:20 in reply to:  18 Changed 9 months ago by sisbell

Yes, I think you are right. Android only loads from /system/lib so I'll need to explicitly load openssl and the other shared libs. I'm not sure loading them from the android app will work, however, as tor is a spawned process. I'm looking into this now.

Replying to sysrqb:

Replying to sisbell:

I've made progress to getting libraries running as shared libraries but have hit a blocker that I need help with

The problem occurs when I try to run tor browser. I get a missing symbol where libTor can't find a symbol from openssl

  10744 10744 F linker  : CANNOT LINK EXECUTABLE "/data/app/org.torproject.torbrowser-o2cGxqnhQbWOPhFKKcOFiA==/lib/arm/libTor.so": cannot locate symbol "SSL_CTX_set_security_level" referenced by "/data/app/org.torproject.torbrowser-o2cGxqnhQbWOPhFKKcOFiA==/lib/arm/libTor.so"...
}}}]

Taking a guess, tor is probably using a version of openssl shipped with Android. I doubt apps look in their lib/ directory by default for libraries.

As an example, Mozilla loads shared libraries manually: https://searchfox.org/mozilla-central/source/mobile/android/geckoview/src/main/java/org/mozilla/gecko/mozglue/GeckoLoader.java#410

You can try setting LD_LIBRARY_PATH on the command line, for testing. If that works, then we should think about how we should solve this problem. There are a few options. One option is we modify TOPL:

https://github.com/thaliproject/Tor_Onion_Proxy_Library/blob/master/universal/src/main/java/com/msopentech/thali/toronionproxy/OnionProxyManager.java#L567

comment:21 Changed 9 months ago by eighthave

I think it makes sense to ship PTs as individual shared libraries, but I think that trying to use libssl.so will be painful. We've tried that in the past, and from that experience, committed to always statically linking those kinds of deps in. I guess it could be worth the pain if PTs also need libssl.so or other libs that are linked into libtor.so.

Also, I don't remember the details, but Android does add the app's lib dir to the loading path, and adds it before /system/lib. But that is probably only for libraries that are loaded from Java. "Native code" probably needs to handle that manually.

comment:22 Changed 9 months ago by pili

Keywords: TorBrowserTeam202002 added; TorBrowserTeam202001 removed

Moving tickets to February

comment:23 Changed 9 months ago by sisbell

Cc: eighthave added; hans@… removed
Keywords: TorBrowserTeam202002R added; TorBrowserTeam202002 removed
Status: needs_revisionneeds_review

Latest Set of Commits

https://github.com/sisbell/tor-browser-build/commits/bug-28704a

The following commits specify the info and NDK setup dependent projects need for configuration.

  • Bug 33216: Add Android Host and ABI Info to RBM.conf
  • Bug 33215: Android Toolchain: Add NDK bin path to system path

The above two issues address previous review comments to add ANDROID_NDK_HOME to setup in android-toolchain and another review comment to rename var/host to var/configure_host

Modify the core dependent libraries to build with Android

  • Bug 28764: OpenSSL Build for Android
  • Bug 28765: LibEvent Build for Android

For the reason that LibEvent is upgraded: https://trac.torproject.org/projects/tor/ticket/28765#comment:6 . I also made a change from the previous review commit to only use CC=clang for Android.

Next add compression libraries

  • Bug 32991: TBB Project For ZSTD
  • Bug 32992: TBB Project for LZMA

Compile tor with all dependencies

  • Bug 28766: Tor Build for Android

This last commit packages up everything within the Android library

  • Bug 32993: Package Tor With Tor Android Service Project

Other issues:

  1. There was a suggestion to move some of the fields in configure_opt up to rbm. OpenSSL doesn't use the same configure_host value as other projects so this will require some more discussion if we want to move forward with this suggestion.
  2. Information regarding libevent --disable-libevent-regress--disable-samples. I need to look back through my notes. I'll post in a follow up comment.
  1. Shared Libraries. I was unable to get shared libraries working with LD_LIBRARY_PATH on my device. It also seems support for this can be different across OEMs. So I moved forward with static libraries.

comment:24 Changed 9 months ago by eighthave

One thought that recently struck me is that loading shared libraries works easily in Java space. The LD_LIBRARY_PATH stuff is automatically handled when loading using System.load(), or you can load using the full path using System.loadLibrary(). But that means running all bits that need that shared library via Android/Java methods, e.g. not as daemons. IMHO that's the right direction for the future anyway.

comment:25 in reply to:  24 Changed 9 months ago by sisbell

Replying to eighthave:

One thought that recently struck me is that loading shared libraries works easily in Java space. The LD_LIBRARY_PATH stuff is automatically handled when loading using System.load(), or you can load using the full path using System.loadLibrary(). But that means running all bits that need that shared library via Android/Java methods, e.g. not as daemons. IMHO that's the right direction for the future anyway.

Yes, I agree. After getting this initial version in, I think we should start looking at JNI, which will give us the option of loading the shared libraries through Java.

comment:26 Changed 9 months ago by sisbell

--disable-libevent-regress is used to enabled regression tests

--disable-samples disables sample projects.

We can add these back in but they don't add anything to libraries we use

comment:27 in reply to:  23 Changed 8 months ago by sysrqb

Status: needs_reviewassigned

Replying to sisbell:

Latest Set of Commits

https://github.com/sisbell/tor-browser-build/commits/bug-28704a

In the future, please provide one branch per ticket. If one ticket depends on another, then you can base a ticket's branch on top of another ticket's branch, but separating each ticket into its own branch makes reviews much easier.

I'm setting this ticket as 'assigned' and each child ticket can be worked individually.

comment:28 Changed 8 months ago by pili

Keywords: TorBrowserTeam202003 added

We are no longer in February, moving tickets

comment:29 Changed 8 months ago by pili

Keywords: TorBrowserTeam202003R added; TorBrowserTeam202002R TorBrowserTeam202003 removed

comment:30 Changed 8 months ago by boklm

Keywords: TorBrowserTeam202003 added; TorBrowserTeam202003R removed

comment:31 Changed 7 months ago by pili

Sponsor: Sponsor58

comment:32 Changed 7 months ago by pili

Keywords: TorBrowserTeam202004 added; TorBrowserTeam202003 removed

We are no longer in March

comment:33 Changed 7 months ago by pili

Parent ID: #33659

comment:34 Changed 7 months ago by gk

Resolution: fixed
Status: assignedclosed

Seems we are done here, yay!

comment:35 Changed 7 months ago by gk

Resolution: fixed
Status: closedreopened

comment:36 Changed 6 months ago by gk

Resolution: fixed
Status: reopenedclosed

comment:37 Changed 6 months ago by gk

Resolution: fixed
Status: closedreopened

comment:38 Changed 6 months ago by gk

Resolution: fixed
Status: reopenedclosed
Note: See TracTickets for help on using tickets.