Opened 7 weeks ago

Last modified 12 days ago

#33291 assigned project

Making the tor library size smaller

Reported by: gaba Owned by: dgoulet
Priority: Medium Milestone:
Component: Core Tor/Tor Version:
Severity: Normal Keywords: tor-size network-team-roadmap-2020Q1
Cc: jnewsome, dgoulet, ahf Actual Points:
Parent ID: Points:
Reviewer: Sponsor:

Description (last modified by gaba)

Goal: To reduce the size of the tor library so that applications that are sensitive to download size can link it easily.

Information from The Guardian Project on data from their "Orbot mini" app:

The Android APK for the arm 32-bit build is about 5MB. An APK is a JAR
file, which is basically a ZIP file. This means everything gets compressed.

Below, then you can see the version of Tor built into this (this is the
old 0.4.0.4 without any of the more recent size optimizations) is almost
7MB. However, it is compressed 67%, ending up somewhere closer to 2.2MB
in actual distribution form.

Archive: 
Orbot-mini-mini-1.0.0-BETA-1-tor-0.4.0.4-rc-mini-armeabi-v7a-release.apk
Zip file size: 4998235 bytes, number of entries: 623

-rw----     2.4 fat  6911768 b- 67% defN 80-000-00 00:00 lib/armeabi-v7a/tor.so

Now, distribution size is thing for sure, the second is amount of space on disk when 
it is unpacked and running. Users of inexpensive phones  with limited storage will often 
scroll through installed apps and look at actual storage size being used, and uninstall 
apps based on that. This means we should be concerned with both distribution size an 
runtime total storage use.

Possible routes

  1. making different parts of tor more optional/modular (like relay mode, dirauth mode) . Did we try this before? Is this possible?
  1. Is there a TLS stack you can link on android? Only in Java
  1. A maybe sketchy possibility is to let google to optimize code in the cloud...
  1. A small java implementation of core onion routing. Would applications be able to run it?
  • java ones, easily
  • other ones with some (considerable?) effort. JNI makes it possible, but not necessarily so easy.
  1. libssl - 600kb is the shared library. The Guardian Project's experiments on making a smaller binary: https://github.com/guardianproject/tor-android/issues/18

Anything else?

Child Tickets

TicketStatusOwnerSummaryComponent
#33371newBuild only with required libevent2 librariesCore Tor/Tor
#33372newAdd support for disabling DNSPortCore Tor/Tor

Change History (11)

comment:1 Changed 7 weeks ago by gaba

Description: modified (diff)

comment:2 Changed 7 weeks ago by gaba

Description: modified (diff)

comment:3 Changed 7 weeks ago by gaba

Keywords: tor-size added

comment:4 Changed 7 weeks ago by gaba

Summary: making the tor library size smallerMaking the tor library size smaller

comment:5 in reply to:  description Changed 7 weeks ago by teor

Replying to gaba:

Now, distribution size is thing for sure, the second is amount of space on disk when 
it is unpacked and running. Users of inexpensive phones  with limited storage will often 
scroll through installed apps and look at actual storage size being used, and uninstall 
apps based on that. This means we should be concerned with both distribution size an 
runtime total storage use.

Tor also downloads directory documents at runtime. Last time I checked, the extra storage was 5-10MB.

Here are some things we could do to reduce that size:

  • store directory documents compressed, and uncompress them when we want to parse them
    • this change uses more CPU, but less disk
    • our directory mmap() changes reduced RAM usage, they may make compressed directory documents a bit harder. Does android use Tor's mmap()?

Possible routes

  1. making different parts of tor more optional/modular (like relay mode, dirauth mode) . Did we try this before? Is this possible?

Yes, we created a relay module as part of Sponsor 31, and did
more work on the existing dirauth module.

But there's still a lot of relay-only code that we could move into
the relay module.

And there is a lot more lower-level code that clients never use.
We could put it in the relay or dirauth modules, or make more
modules as needed.

I'll also do a small amount of relay module work as part of
Sponsor 55, when adding new features, or modifying existing
code.

I suggest that other sponsored work adopts a similar policy.

We can also continue to move config and control code into the
relay module, as a separate project.

And we should continue to delete old, unused code, particularly
code for obsolete options. There's a large amount of IPv6
DirPort code in tor that we could delete right now.

  1. Is there a TLS stack you can link on android? Only in Java

Does Orbot / Tor Browser for Android use NSS?
If so, we can use NSS in tor, rather than OpenSSL.

Do we need to migrate some PTs off OpenSSL as well?

  1. A small java implementation of core onion routing. Would applications be able to run it?
  • java ones, easily
  • other ones with some (considerable?) effort. JNI makes it possible, but not necessarily so easy.

Any re-implementation of tor will probably be
distinguishable on the network. That's ok, as long as
enough users are running it.

But a re-implementation will also come with its own
bugs, privacy issues, and security issues. And its
own costs for testing and release management. We
should think about whether we want to maintain
two parallel implementations, because that's a huge
cost.

As an alternative, we could rewrite tor's client and
common code in Rust. We could use this Rust code
in tor on relays, desktops, and mobile.

comment:6 Changed 7 weeks ago by teor

We could also set tor's CacheDir to an ephemeral location, to reduce the disk usage.

getCacheDir() is the relevant android function:
https://developer.android.com/training/data-storage/app-specific#internal-create-cache

comment:7 Changed 7 weeks ago by nickm

ISTR that one of the breakdowns that I saw linked more libevent libraries than they needed. Tor only requires libevent_core and libevent_extra.

comment:8 Changed 6 weeks ago by jnewsome

Maybe this should be a separate ticket, but another thing to consider might be saving space for users that have both TBB and Orbot installed, as CrazyAstronaut mentioned in #tor.

They propose a "thin" version of the Tor Browser that uses Orbot's Tor client. Perhaps another possibility would be a "fat" Tor Browser that exposes the Tor client for other apps to use (making Orbot unnecessary). Assuming doing that adds significant space requirements though, we'd presumably want to either offer two versions of TBB (confusing), or...:

In general a possibility to consider if there's functionality that some, but not all, users need, is to separate that functionality into modules that can be downloaded and loaded on-demand from within the app itself. This would add *significant* complexity of course. Example: https://github.com/Instagram/ig-lazy-module-loader.

comment:9 Changed 6 weeks ago by jnewsome

Cc: jnewsome added

comment:10 Changed 6 weeks ago by gaba

Cc: dgoulet ahf added
Keywords: network-team-roadmap-2020Q1 added

comment:11 Changed 12 days ago by gaba

Owner: set to dgoulet
Status: newassigned
Note: See TracTickets for help on using tickets.