Opened 5 weeks ago

Last modified 4 hours ago

#32823 new enhancement

support Android SharedPreferences xml as a torrc format

Reported by: eighthave Owned by:
Priority: Medium Milestone: Tor: unspecified
Component: Core Tor/Tor Version:
Severity: Normal Keywords: Android, xml, tbb-mobile, torrc, sharedpreferences
Cc: n8fr8, sisbell Actual Points:
Parent ID: Points:
Reviewer: Sponsor:

Description

With Tor Browser, Orbot, and other apps on Android, some Tor settings are managed via the SharedPreferences/PreferenceFragment Android classes. When used in their standard setup, these handle the whole lifecycle of preferences. Right now, we have to do a lot of tricks to get the settings from the Android classes, convert them to torrc format, then handle reloading Tor at the right time.

Android's SharedPreferences writes out to a standard xml file, e.g /data/data/org.torproject.android/shared_prefs/org.torproject.android_preferences.xml. This XML file has a simple format that has been the same since the beginning. If Tor can read this format as an optional torrc format, and also potentially automatically reload the config when that file changes, then the Android code will be drastically simpler. This will greatly improve the interaction around any kind of user-facing setting, like bridges, exit countries, pluggable transports, etc.

Here's an example of what the XML would look like:

<?xml version='1.0' encoding='utf-8' standalone='yes' ?>
<map>
    <boolean name="RunAsDaemon" value="true" />
    <boolean name="AvoidDiskWrites" value="true" />
    <string name="TransPort">auto</string>
    <int name="DNSPort" value="5400" />
    <string name="TransPort" value="auto" />
    <string name="Bridges">obfs4 77.81.104.251:443 115C90EBD0EB631C177560A872535772215478D9 cert=UsuF7oN4KNKviZP54JOyTCoCphrdM5gwZK4vT8GnCAcmqLUJEJxyw1dpko9a/ii6He4iZg iat-mode=0,obfs4 5.249.146.133:80 FAF3A0073330D6AD92F3B4874B0D945562A633EF cert=TRe8bAODtjcGij7EPQaUayWEOqR99wDh2l3B4hFtCsn1JTJCph03pRZ9tx8wynpLYKWMQg iat-mode=0,meek_lite 0.0.2.0:2 97700DFE9F483596DDA6264C4D7DF7641E1E39CE url=https://meek.azureedge.net/ front=ajax.aspnetcdn.com</string>
</map>

We might want an alternate format for bridge lines, where would be a dedicated XML file for only bridge lines with the name specifying the line number:

<?xml version='1.0' encoding='utf-8' standalone='yes' ?>
<map>
    <string name="0">obfs4 77.81.104.251:443 115C90EBD0EB631C177560A872535772215478D9 cert=UsuF7oN4KNKviZP54JOyTCoCphrdM5gwZK4vT8GnCAcmqLUJEJxyw1dpko9a/ii6He4iZg iat-mode=0</string>
    <string name="1">obfs4 5.249.146.133:80 FAF3A0073330D6AD92F3B4874B0D945562A633EF cert=TRe8bAODtjcGij7EPQaUayWEOqR99wDh2l3B4hFtCsn1JTJCph03pRZ9tx8wynpLYKWMQg iat-mode=0</string>
    <string name="2">meek_lite 0.0.2.0:2 97700DFE9F483596DDA6264C4D7DF7641E1E39CE url=https://meek.azureedge.net/ front=ajax.aspnetcdn.com</string>
</map>

Yxml is a 6k C XML parser in a single file under an MIT license. It has been maintained since 2013. It would fit the bill quite nicely.
https://dev.yorhel.nl/yxml

Yes, XML is not an ideal format. We don't need the most problematic parts of XML for this, e.g. namespaces, custom entities, etc. and Yxml does not support them anyway.

Child Tickets

Change History (5)

comment:1 Changed 11 days ago by dgoulet

Milestone: Tor: unspecified

comment:2 Changed 5 days ago by sisbell

It seems like this might be a more general request in terms of importing/exporting from one data format (in this case XML) into the canonical torrc format. If there is a general config plugin API within tor, then a third-party developer could create this mapper.

As a side note, Android supports the concept of sets in Preferences, which translates to XML as a subnode.

comment:3 Changed 4 days ago by eighthave

What I have in mind is definitely not import/export. I mean Tor directly loading configuration from the Android SharedPreferences xml files when they change (e.g. using Linux inotify). That would make it trivial to make nice, responsive Settings UX on the Android side, like bridge lines, etc. Import/export would not help in this goal much at all, since the Android side would need to have custom code to handle loading the exported XML. That's the thing to avoid. When doing custom things with SharedPreferences, its quite tricky to get everything syncing and loading properly, especially with native code involved. Then there are many bugs with Settings not taking effect immediately, and instead requiring app restarts, etc.

comment:4 in reply to:  3 Changed 4 days ago by sisbell

There should be some canonical format. I'm assuming this would for the flat format specified in torrc. What I was suggesting is a plugin (on the tor side) that would transform to/from a given format to the canonical and then start tor.  Tor handles the monitoring of config file changes and whatever internals it needs. The third-party creates the plugin that handles the transform but it would load and run as part of Tor.

If the config plugin implementation is useful to the general community, then perhaps it could become included within a set of core plugins. In other cases, the plugin would allow more specific, customizable implementations.

So I don't think there is disagreement in the high-level approach. I'm suggesting taking your idea and generalizing it to support any custom format.

Replying to eighthave:

What I have in mind is definitely not import/export. I mean Tor directly loading configuration from the Android SharedPreferences xml files when they change (e.g. using Linux inotify). That would make it trivial to make nice, responsive Settings UX on the Android side, like bridge lines, etc. Import/export would not help in this goal much at all, since the Android side would need to have custom code to handle loading the exported XML. That's the thing to avoid. When doing custom things with SharedPreferences, its quite tricky to get everything syncing and loading properly, especially with native code involved. Then there are many bugs with Settings not taking effect immediately, and instead requiring app restarts, etc.

comment:5 Changed 4 hours ago by eighthave

I agree there should be a canonical config data format, but not necessarily the file format. My guess is that the existing configuration options can already be treated as a data model, then treat the formats as just representations of that data model. This data model is also what already exists in Tor's C code. E.g. like how its easy to convert data files between YAML, JSON, and TOML. Hugo (written in Go) and Jekyll (written in Ruby) can use YAML/JSON/TOML interchangeably for data/config files. Then the data model is the canonical format, and the file formats are just containers for that. At the highest level, there is key/value pairs RunAsDaemon/boolean, DNSPort/int+string, Bridges/string. This will be a little trickier, since we're talking XML and torrc, but unless I've overlooked something in the torrc format, it seems that the data model is already simple enough to map cleanly. SharedPreferences XML already gives us a format with the basic data formats that are needed. As a side note, this would make it easy to also use TOML/YAML/JSON as needed.

Note: See TracTickets for help on using tickets.