Opened 4 years ago

Last modified 18 months ago

#14899 new enhancement

Enable Tor to work without using filesystem for cached files

Reported by: naif Owned by:
Priority: Medium Milestone: Tor: unspecified
Component: Core Tor/Tor Version:
Severity: Normal Keywords: globaleaks-wants needs-design filesystem
Cc: yawning, meejah Actual Points:
Parent ID: Points: 8
Reviewer: Sponsor:

Description

Looking at mid-term improvements for Tor integration within third party application, this ticket is to enable Tor to work without ever touching the filesystem, but keeping basic caching functionalities.

Today Tor need to write the following files:
cached-certs
cached-consensus
cached-microdesc-consensus
cached-microdescs
cached-microdescs.new
cached-descriptors
cached-descriptors.new
state
lock

Those files could be removed/disabled as a first step, but as far as i understood, it would bring some performance impact.

It would be nice if a third party application:

  • before shutdown could download via Tor CP a copy of that data structures
  • immediately after startup could upload via Tor CP of the most recent cached data structures

This ticket, when bundled with #6411 and #13865, would allow a completely filesystem-free use a Tor, being entirely governed by a third party app (e.g,: using TxTorCon or Orbot as controllers).

Child Tickets

Change History (5)

comment:1 Changed 4 years ago by yawning

So I think this would be a good idea, but it's probably a long term project, and hard to do correctly.

Specifically, while the list of files you gave is sufficient (overkill really) for a client, it does not encompass all of the state that needs to be serialized if the tor instance is more than a client.

Cutting things down to the minimal list, a client needs to persist state so the guards are preserved.

A relay/bridge/whatever will additionally need to persist:

  • keys/*
  • pt_state/* (depending on the configured pluggable transports).

Everything else can be regenerated/redownloaded, I think (under the assumption that HSes will all be ephemeral).

Open design questions:

  • How much of the optional state to keep.
  • Should tor instances that are acting as bridges/relays support this?
  • Designing a serialization format.
  • Coming up with good commands.
  • Figuring out exact behavior (Eg: I *assume* loading serialized state should only be allowed as the first authenticated control command given to a tor instance that was launched with DisableNetwork 1. If reloading state can happen at an arbitrary point in time, things get kind of scary.)

comment:2 Changed 4 years ago by nickm

Keywords: needs-proposal added
Milestone: Tor: unspecified

This is pretty interesting, but a little risky. The biggest failure mode here would be if the controller doesn't bother to store the information; these clients would get much worse performance and security, and put more load on the network. We ought to come up with some way to make that a bit less likely.

comment:3 Changed 4 years ago by naif

This feature would be even better when possibly coupled with https://github.com/meejah/txtorcon/issues/102 as it would make possible to run a Tor process completely sandboxed and under third party Tor controller such as TxTorCon

comment:4 Changed 3 years ago by naif

Severity: Normal

Regarding the open design questions i think that:

a) Assume that the Application controlling a Tor initialization that way, will gives to Tor the latests version of the states/versions of those data-structures dumped just before shutting it down
b) This feature should only apply to Tor clients (that need to work, for example, in a mobile application w/out filesystem I/O)
c) A possibility could be to work like a "Blob" that represent exactly the actual filesystem stored files, leveraging the very same validation routines possibly without writing additional validation codes
d) It's nice the idea to enable a "write" feature to be issues only with Tor clients that starts with "DisableNetwork 1", leaving to the third party application logic the duty to "feed" the valid startup files via Tor CP. An application willing to use that feature would first need to startup Tor with "DisableNetwork 1" and something like "DisableStateFilesWrite 1", then somehow tell Tor to "Startup 1st time the Network"
e) If Tor Control Protocol enable to a client to "register" for asynchronous events, it would be really nice to be able to have the third party application to listen for changes in the data structure (example: cached-certs), so it will be able to receiver the latests updated version of that "blobs" from Tor and store it a database

That way the application logic would be minimized, a txtorcon API could provide a set of asynchronous callback to a Twisted application to store the different data-structure that Tor would like to write/update, into the third party managed Tor database.

That kind of constraints and limits would probably still allow filesystem free operations for Tor Clients being integrated into third party application, by limiting the possible risks and the application logic mistake that could happen, having a sort of "simple PIPE" between Tor -> TorCP -> TorCP Client -> TxTorConn -> Application-using-TxTorConn -> Application-Database and vice-versa.

What do you think?

Come to the dark side, we have Italian Red Wine! (not cookies!) :-)

comment:5 Changed 18 months ago by nickm

Keywords: needs-design filesystem added; needs-proposal removed
Points: 8
Note: See TracTickets for help on using tickets.