Compile Tor and dependencies on our own for Android
Currently we are building just Orbot in tor-browser-build and fetching the dependencies as we need them. We should at least build Tor and its dependencies on our own, integrating Android specific build logic into our projects we already have (like OpenSSL, Libevent etc.).
This is the parent ticket for that task.
Activity
Trac:
Cc: sisbell to sisbell, hans@guardianproject.info-
It seems that Android is moving to clang, and clang has more limited options for reproducibility. I think GCC support is still there, but deprecated. I found some specific issues on what is missing from clang here: https://reproducible-builds.org/docs/build-path/
-
I have a barebones build job for GitLab-CI for this, you can see a test run here: https://gitlab.com/eighthave/tor/-/jobs/301739923
And the code here: https://gitlab.com/eighthave/tor/blob/0f2d18708609a52eccd4240409d2c219274e5e55/.gitlab-ci.yml#L56
I'll submit the
.gitlab-ci.ymlchanges for inclusion once its ready. -
I justed looked through it, you could get rid of tor-android in TBB if you build libevent, openssl, lzma, zstd, and tor. The build configuration for each of those can be taken from https://github.com/guardianproject/tor-android, especially after this merge request, which should simplify things a bunch:
Trac:
Cc: sisbell, hans@guardianproject.info to sisbell, hans@guardianproject.info, tbb-teamI have 4 commit covering child tickets:
Bug 32993: Package Tor With Tor Android Service Project Bug 28766: Tor Build for Android Bug 28765: LibEvent Build for Android Bug 28764: OpenSSL Build for Android https://github.com/sisbell/tor-browser-build/commits/bug-28766c
I verified that tor starts up correctly with the armv7 build on a device. The browser loads and displays pages. I have not verified reproducibility yet.
Trac:
Keywords: TorBrowserTeam202001 deleted, TorBrowserTeam202001R added
Cc: sisbell, hans@guardianproject.info, tbb-team to sysrqb, gk, sisbell, hans@guardianproject.info, tbb-team
Status: assigned to needs_review
Replying to sisbell:
I have 4 commit covering child tickets:
Bug 32993: Package Tor With Tor Android Service Project Bug 28766: Tor Build for Android Bug 28765: LibEvent Build for Android Bug 28764: OpenSSL Build for Android https://github.com/sisbell/tor-browser-build/commits/bug-28766c
I verified that tor starts up correctly with the armv7 build on a device. The browser loads and displays pages. I have not verified reproducibility yet.
Some comments/questions:
- in the builds of openssl, libevent, tor, you are setting
ANDROID_NDK_HOMEand adding$ANDROID_NDK_HOMEtoPATH. Shouldn't that be done invar/setupinprojects/android-toolchain/configinstead of in each project? - could you explain (for example in the commit message) why libevent needs to be updated from 2.1.8 to 2.1.11?
- could you explain why the
--disable-libevent-regress--disable-samplesoptions are needed to build libevent (and whether it would be useful to use them on other platforms too)? - in the libevent build, you are adding CC=clang to the configure for all platforms. I think it should be added to the android builds only as it doesn't seem to be needed for other platforms.
- it seems we could have a
var/configure_optfor android inrbm.confcontaining something likeCC=clang --host=[% c("var/host") %] [% c("var/configure_opt_project") %], wherevar/configure_opt_projectis defined in each project to define options specific to this project - I think the name
var/hostis a little confusing and could be renamed tovar/configure_hostto make it more clear where it's used
Trac:
Keywords: TorBrowserTeam202001R deleted, TorBrowserTeam202001 added
Status: needs_review to needs_revision- in the builds of openssl, libevent, tor, you are setting
I've made progress to getting libraries running as shared libraries but have hit a blocker that I need help with
The problem occurs when I try to run tor browser. I get a missing symbol where libTor can't find a symbol from openssl
10744 10744 F linker : CANNOT LINK EXECUTABLE "/data/app/org.torproject.torbrowser-o2cGxqnhQbWOPhFKKcOFiA==/lib/arm/libTor.so": cannot locate symbol "SSL_CTX_set_security_level" referenced by "/data/app/org.torproject.torbrowser-o2cGxqnhQbWOPhFKKcOFiA==/lib/arm/libTor.so"...Looking at the libraries, we see the function undefined in libTor. So I'm wondering what flag I need to set to make sure these are included. I'm building with clang.
$ readelf -Ws libTor.so | grep set_security 3819: 00000000 0 FUNC GLOBAL DEFAULT UND SSL_CTX_set_security_level@OPENSSL_1_1_0 (5) 110559: 00000000 0 FUNC GLOBAL DEFAULT UND SSL_CTX_set_security_level$ readelf -Ws libssl.so | grep set_security 537: 000328bc 12 FUNC GLOBAL DEFAULT 11 SSL_CTX_set_security_level@@OPENSSL_1_1_0 543: 000328d4 12 FUNC GLOBAL DEFAULT 11 SSL_CTX_set_security_callback@@OPENSSL_1_1_0 679: 0003288c 12 FUNC GLOBAL DEFAULT 11 SSL_set_security_callback@@OPENSSL_1_1_0 681: 00032874 12 FUNC GLOBAL DEFAULT 11 SSL_set_security_level@@OPENSSL_1_1_0 2743: 000328bc 12 FUNC GLOBAL DEFAULT 11 SSL_CTX_set_security_level 2749: 000328d4 12 FUNC GLOBAL DEFAULT 11 SSL_CTX_set_security_callback 2885: 0003288c 12 FUNC GLOBAL DEFAULT 11 SSL_set_security_callback 2887: 00032874 12 FUNC GLOBAL DEFAULT 11 SSL_set_security_levelI have the changes here https://github.com/sisbell/tor-browser-build/commits/bug-28766e
The tor specific changes are here https://github.com/sisbell/tor-browser-build/commit/b6d9c2edf8cb46e484413554ba76dce9560eef5d
Trac:
Status: needs_revision to needs_reviewReplying to boklm:
Does running ldd on libTor.so says that it is linked to libssl.so?
Running ldd I get message
not a dynamic executableThe elf header type
Type:DYN (Shared object file)Reading elf dependencies
0x00000001 (NEEDED) Shared library: [libz.so] 0x00000001 (NEEDED) Shared library: [libm.so] 0x00000001 (NEEDED) Shared library: [libssl.so] 0x00000001 (NEEDED) Shared library: [libcrypto.so] 0x00000001 (NEEDED) Shared library: [liblog.so] 0x00000001 (NEEDED) Shared library: [libdl.so] 0x00000001 (NEEDED) Shared library: [libc.so]Replying to sisbell:
I've made progress to getting libraries running as shared libraries but have hit a blocker that I need help with
The problem occurs when I try to run tor browser. I get a missing symbol where libTor can't find a symbol from openssl
{{{ 10744 10744 F linker : CANNOT LINK EXECUTABLE "/data/app/org.torproject.torbrowser-o2cGxqnhQbWOPhFKKcOFiA==/lib/arm/libTor.so": cannot locate symbol "SSL_CTX_set_security_level" referenced by "/data/app/org.torproject.torbrowser-o2cGxqnhQbWOPhFKKcOFiA==/lib/arm/libTor.so"... }}}]
Taking a guess, tor is probably using a version of openssl shipped with Android. I doubt apps look in their
lib/directory by default for libraries.As an example, Mozilla loads shared libraries manually: https://searchfox.org/mozilla-central/source/mobile/android/geckoview/src/main/java/org/mozilla/gecko/mozglue/GeckoLoader.java#410
You can try setting LD_LIBRARY_PATH on the command line, for testing. If that works, then we should think about how we should solve this problem. There are a few options. One option is we modify TOPL:
Yes, I think you are right. Android only loads from /system/lib so I'll need to explicitly load openssl and the other shared libs. I'm not sure loading them from the android app will work, however, as tor is a spawned process. I'm looking into this now.
Replying to sysrqb:
Replying to sisbell:
I've made progress to getting libraries running as shared libraries but have hit a blocker that I need help with
The problem occurs when I try to run tor browser. I get a missing symbol where libTor can't find a symbol from openssl
{{{ 10744 10744 F linker : CANNOT LINK EXECUTABLE "/data/app/org.torproject.torbrowser-o2cGxqnhQbWOPhFKKcOFiA==/lib/arm/libTor.so": cannot locate symbol "SSL_CTX_set_security_level" referenced by "/data/app/org.torproject.torbrowser-o2cGxqnhQbWOPhFKKcOFiA==/lib/arm/libTor.so"... }}}]
Taking a guess, tor is probably using a version of openssl shipped with Android. I doubt apps look in their
lib/directory by default for libraries.As an example, Mozilla loads shared libraries manually: https://searchfox.org/mozilla-central/source/mobile/android/geckoview/src/main/java/org/mozilla/gecko/mozglue/GeckoLoader.java#410
You can try setting LD_LIBRARY_PATH on the command line, for testing. If that works, then we should think about how we should solve this problem. There are a few options. One option is we modify TOPL:
-
I think it makes sense to ship PTs as individual shared libraries, but I think that trying to use libssl.so will be painful. We've tried that in the past, and from that experience, committed to always statically linking those kinds of deps in. I guess it could be worth the pain if PTs also need libssl.so or other libs that are linked into libtor.so.
Also, I don't remember the details, but Android does add the app's lib dir to the loading path, and adds it before /system/lib. But that is probably only for libraries that are loaded from Java. "Native code" probably needs to handle that manually.
Latest Set of Commits
https://github.com/sisbell/tor-browser-build/commits/bug-28704a
The following commits specify the info and NDK setup dependent projects need for configuration.
- Bug 33216: Add Android Host and ABI Info to RBM.conf
- Bug 33215: Android Toolchain: Add NDK bin path to system path
The above two issues address previous review comments to add ANDROID_NDK_HOME to setup in android-toolchain and another review comment to rename _var/host _to var/configure_host
Modify the core dependent libraries to build with Android
- Bug 28764: OpenSSL Build for Android
- Bug 28765: LibEvent Build for Android
For the reason that LibEvent is upgraded: https://trac.torproject.org/projects/tor/ticket/28765#comment:6 . I also made a change from the previous review commit to only use CC=clang for Android.
Next add compression libraries
- Bug 32991: TBB Project For ZSTD
- Bug 32992: TBB Project for LZMA
Compile tor with all dependencies
- Bug 28766: Tor Build for Android
This last commit packages up everything within the Android library
- Bug 32993: Package Tor With Tor Android Service Project
Other issues:
-
There was a suggestion to move some of the fields in configure_opt up to rbm. OpenSSL doesn't use the same configure_host value as other projects so this will require some more discussion if we want to move forward with this suggestion.
-
Information regarding libevent --disable-libevent-regress--disable-samples. I need to look back through my notes. I'll post in a follow up comment.
-
Shared Libraries. I was unable to get shared libraries working with LD_LIBRARY_PATH on my device. It also seems support for this can be different across OEMs. So I moved forward with static libraries.
Trac:
Status: needs_revision to needs_review
Cc: sysrqb, gk, sisbell, hans@guardianproject.info, tbb-team to sysrqb, gk, sisbell, eighthave, tbb-team
Keywords: TorBrowserTeam202002 deleted, TorBrowserTeam202002R added-
One thought that recently struck me is that loading shared libraries works easily in Java space. The LD_LIBRARY_PATH stuff is automatically handled when loading using
System.load(), or you can load using the full path usingSystem.loadLibrary(). But that means running all bits that need that shared library via Android/Java methods, e.g. not as daemons. IMHO that's the right direction for the future anyway. Replying to eighthave:
One thought that recently struck me is that loading shared libraries works easily in Java space. The LD_LIBRARY_PATH stuff is automatically handled when loading using
System.load(), or you can load using the full path usingSystem.loadLibrary(). But that means running all bits that need that shared library via Android/Java methods, e.g. not as daemons. IMHO that's the right direction for the future anyway.Yes, I agree. After getting this initial version in, I think we should start looking at JNI, which will give us the option of loading the shared libraries through Java.
Replying to sisbell:
Latest Set of Commits
https://github.com/sisbell/tor-browser-build/commits/bug-28704a
In the future, please provide one branch per ticket. If one ticket depends on another, then you can base a ticket's branch on top of another ticket's branch, but separating each ticket into its own branch makes reviews much easier.
I'm setting this ticket as 'assigned' and each child ticket can be worked individually.
Trac:
Status: needs_review to assigned
Trac:
Parent: N/A to #33659 (moved)mentioned in issue #28763 (moved)
mentioned in issue #31499 (moved)
mentioned in issue #32200 (moved)
mentioned in issue #32476 (moved)
mentioned in issue #32993 (moved)
mentioned in issue #33930 (moved)
mentioned in issue #32992 (moved)
mentioned in issue #32991 (moved)
mentioned in issue #33215 (moved)
mentioned in issue #33216 (moved)
