Test integration of Minion (out-of-order data transmission thing) with Tor relays

Bryan Ford has a small kernel patch, and larger openssl patch, to let TLS 1.2 receive data out-of-order and pass it to the application:

http://arxiv.org/abs/1103.0463

"""Internet applications increasingly employ TCP not as a stream abstraction, but as a substrate for application-level transports, a use that converts TCP's in-order semantics from a convenience blessing to a performance curse. As Internet evolution makes TCP's use as a substrate likely to grow, we offer Minion, an architecture for backward-compatible out-of-order delivery atop TCP and TLS. Small OS API extensions allow applications to manage TCP's send buffer and to receive TCP segments out-of-order. Atop these extensions, Minion builds application-level protocols offering true unordered datagram delivery, within streams preserving strict wire-compatibility with unsecured or TLS-secured TCP connections. Minion's protocols can run on unmodified TCP stacks, but benefit incrementally when either endpoint is upgraded, for a backward-compatible deployment path. Experiments suggest that Minion can noticeably improve the performance of applications such as conferencing, virtual private networking, and web browsing, while incurring minimal CPU or bandwidth costs."""

I know that we've got TLS 1.2 disabled currently, but this sounds like the sort of project that one of our fast relay operators might want to play with to see how easy it is.

changed milestone to %Tor: unspecified

added component::core tor/tor milestone::Tor: unspecified priority::medium severity::normal status::new tor-relay type::project labels

Bryan, can you point us at your current code (ideally with readmes and so on)?

Sorry for the long delay following up on this, due to holiday travels and other stuff. BTW, at least two people should probably be added to the CC list if we continue to use this trac item: Janardhan Iyengar janardhan.iyengar@fandm.edu and MF Nowlan fitz@cs.yale.edu

We don't really have a good official "release" online yet. The (small) kernel portion is quite stable and pretty close to being ready to go, but the userland TLS code still needs some work before it'll be ready for use "in the wild." In particular, because the version of OpenSSL we started with didn't (yet) have full TLS 1.2 support, we hacked some incomplete "quasi-TLS 1.2" support into an earlier OpenSSL version just for experimentation purposes; to deploy it in the wild the code and ensure wire-compatibility will at least require forward-porting it to a more recent version of OpenSSL with full, "real" TLS 1.2 support.

In addition, I understand that the fact that our prototype is based on OpenSSL might itself be a problem, in that Tor is shifting away from OpenSSL and toward preferring NSS so that connections "blend in" better with Firefox-initiated TLS connections and such? How much of an issue is this - is Tor still using OpenSSL in at least some configurations or is it already deprecated?

In any case, if you'd like to look at the current prototype code for preliminary experimentation purposes I'm sure we could put together something pretty quickly, and in the longer term, if there's serious interest in using Minion in Tor, we might be able to put some time into helping get the code into shape.

Trac:
Username: ford

Interesting stuff!

For what it's worth, Tor's still using openssl and isn't currently moving to NSS.

The part I wonder about for us is that it's not actually trivial to handle Tor cells out-of-order: they aren't self-framing and the streams aren't self-synchronizing, so if I decrypt a TLS record out-of-order, I don't see how I can know whether that record's contents starts in the middle of a Tor cell or at the beginning, and so on. Even if I know I'm getting a cell, I need to know how many intervening cells there have been on that circuit, or I won't know how to decrypt the cell.

These aren't insurmountable obstacles, but they do suggest that we'd need to change the Tor link protocol in order to take advantage of Minion.

Trac:
Keywords: N/A deleted, tor-relay added

Hi all,

My name is Fitz and I wrote the uTLS code in the past, and just recently updated it to TLSv1.2, following all the OpenSSL coding style and conventions for enabling uTLS. We have a simple in-house testing program and testbed to verify that records are successfully decrypted out of order (using artificial loss to force drops). My collaborators Jana and David are in charge of the uTCP kernel patch. Together, we are now looking at using uTLS in the Tor source code.

@nickm - Your exactly right. In going through the Tor code, I see that it does not rely on OpenSSL to preserve write boundaries. As I'm sure you know, it reads the data from SSL_read() into a buffer and then reads the first N bytes from it. Two things come to mind: 1) The TLS spec states that an implementation is free to combine records as it sees fit, and 2) The OpenSSL implementation does not combine records (i.e., it preserves record boundaries) as far as I can tell.

Obviously, this is not a robust solution, but for the sake of argument, if Tor writes only on cell boundaries (i.e., one, two, three, etc. cells at a time), then the current SSL_read() with uTLS or regular TLS will produce those complete cells on their boundaries. This would mean the Tor code could assume that the result of an SSL_read() holds complete cells, and they can be processed.

As a note, uTLS "predicts" the sequence number of unordered records so that they can be successfully decrypted. Does Tor use additional encryption for the cells themselves? If so, then you are correct in saying you'd need to predict the sequence number (or know somehow exactly how many cells are in the unreceived "gap"). If not, however, uTLS will give you cleartext cells that can be used and processed right away.

However, even if we kind of cheat and rely on OpenSSL to preserve write boundaries, this incurs a substantial rewrite of the Tor socket code. In my opinion, I think the rewrite might be beneficial because it would allow the Tor code to act like each read produces a datagram. This makes for a cleaner, more abstract processing loop, at the cost of enforcing datagram boundaries somewhere below (if it's not done for you already).

On the other hand, if you're using TCP, you expect to be able to do things as they are: in-order and shift the data down, so it's probably not attractive to have to rewrite all that code. (Of course, this IS uTCP not, TCP after all :) ) The alternative path is to take a look at the fetch_from_buf() function and see how we can force reads along cell boundaries. One way we've done this in the past for another framing mechanism (COBS encoding) is to interpose a data queue between the socket read and the processing code. Each read adds data to the queue, and whenever a complete record resides in the queue, it is processed. This encoding scheme was not encrypted, however, and it was easy to identify the record header (a single marker byte). So, if each cell, including its header, is individually encrypted, then this is not really possible.

Long story long, the uTLS patch (about 650 lines including comments) for OpenSSL is in a good state now, and I can share it with you guys, and the uTCP kernel patch is similarly in a good state, and we should be able to share that with you guys as well.

Trac:
Username: slight

Replying to slight:

Hi all, [...] Long story long, the uTLS patch (about 650 lines including comments) for OpenSSL is in a good state now, and I can share it with you guys, and the uTCP kernel patch is similarly in a good state, and we should be able to share that with you guys as well.

If it's open-source, I'd love to have a look!

Great. The kernel patch is about 600 lines, and the TLS patch is about 600 lines. The easiest way to share the code is probably just for me to generate a diff and email it to you, along with the version names of the kernel and OpenSSL from which we made our changes. What's the best way for me to email it to you? I can't see your email address, so I'm guessing you can't see mine either.

Trac:
Username: slight

You can email nickm at torproject.org, and maybe cc tor-assistants at torproject.org. Or you could just put it online somewhere, really.

Trac:
Username: nikita
Cc: mo, bryan.ford@yale.edu to mo, bryan.ford@yale.edu, nikita@illinois.edu

(did we get a patch here?)

Trac:
Severity: N/A to Normal
Sponsor: N/A to N/A
Summary: Test integration of Minion with Tor relays to Test integration of Minion (out-of-order data transmission thing) with Tor relays
Reviewer: N/A to N/A

It has been five years since Minion has been published, and despite a handful of reports and papers showing it as a good project, we don't even see a single patch! Nor they have the intention to release any code or patches.

Do they really want to cooperate with the community, or simply marketing about their research for their own career, while hide their patches as their "intellectual property"?

Replying to cypherpunks:

Do they really want to cooperate with the community, or simply marketing about their research for their own career, while hide their patches as their "intellectual property"?

It is the norm for research projects to not release their code, unfortunately.

Hi, just noticed a notification about this thread buried deep in my inbox.

My PhD student Fitz Nowland actually submitted his code to the Tor project years ago, but never received a response. After a bit of digging (since neither he nor I are still at Yale), he managed to find a copy of it, and I've made a tarball available here:

http://dedis.cs.yale.edu/2009/tng/code/utcp_for_tor.tar.gz

Just in case it's still of interest. I would still be delighted to see uTCP/uTLS or something like it get used, and I do think it could really help improve general performance and reduce delays.

Trac:
Username: ford

moved to tpo/core/tor#7679 (closed)