Add FTE to Pluggable Transport Tor Browser Bundle
Fully integrate Format-Transforming Encryption [1] into the PTTBB.
Activity
Yesterday, I tagged a candidate release of FTE to included in the PTTBB. This release includes cross-platform support and support for FTE as a managed proxy.
https://github.com/kpdyer/fteproxy/releases/tag/7f5d659
I've also integrated FTE into the bundling process for the PTTBB. I've forked the bundle repo.
https://github.com/kpdyer/bundle
and attached (fte.patch) to this ticket is the necessary delta to include FTE in the PTTBB.
Thanks for this work. I pushed your branch to the torproject.org repo.
https://gitweb.torproject.org/pluggable-transports/bundle.git/shortlog/refs/heads/fte
Here are some notes.
FTEPROXY_TAGis meant to be a git tag. If you're not using git, you might want to call itFTEPROXY_VERSIONor something, and have a separate step for the download. The various targets in Makefile definitely should not be downloading files with curl (and shouldn't be downloading the file fresh every time).The patch changes
MACOSX_BUNDLE_ROOT. This actually changed in master probably since you made your branch. You might have to rearrange your OS X working directory. See https://gitweb.torproject.org/pluggable-transports/bundle.git/commitdiff/7d9fc634cbcb281af775b6e17971777edf6ad1c1, https://gitweb.torproject.org/pluggable-transports/bundle.git/commitdiff/86efa19e1d7fb8f75cd1d9b126d40263658d27bb, and https://gitweb.torproject.org/pluggable-transports/bundle.git/commitdiff/9260e7b591ec7a5ffa7f032c1dd9ff287568f987.Makefile seems to be downloading precompiled binaries of the fteproxy program. Really, it should build its own binaries from source code, like the other transports do. (It's a bit different for flash proxy and obfsproxy because they are written in Python, but for example on Windows we call their py2exe targets.)
Finally, I hate that you got caught in a transition between build systems, but the bundle branch is nearly defunct and probably won't be used to make future bundles. The pt branch in #9444 (closed) is now working well enough to build bundles on all platforms, so you might like to see what is needed to incorporate FTE there. (Grep the history for "Fetch obfsproxy" and "Install obfsproxy" for examples.)
Replying to dcf:
Thanks for this work. I pushed your branch to the torproject.org repo.
https://gitweb.torproject.org/pluggable-transports/bundle.git/shortlog/refs/heads/fte
Here are some notes.
FTEPROXY_TAGis meant to be a git tag. If you're not using git, you might want to call itFTEPROXY_VERSIONor something, and have a separate step for the download. The various targets in Makefile definitely should not be downloading files with curl (and shouldn't be downloading the file fresh every time).Got it. Will make that change.
The patch changes
MACOSX_BUNDLE_ROOT. This actually changed in master probably since you made your branch. You might have to rearrange your OS X working directory. See https://gitweb.torproject.org/pluggable-transports/bundle.git/commitdiff/7d9fc634cbcb281af775b6e17971777edf6ad1c1, https://gitweb.torproject.org/pluggable-transports/bundle.git/commitdiff/86efa19e1d7fb8f75cd1d9b126d40263658d27bb, and https://gitweb.torproject.org/pluggable-transports/bundle.git/commitdiff/9260e7b591ec7a5ffa7f032c1dd9ff287568f987.Will accommodate for this, too.
Makefile seems to be downloading precompiled binaries of the fteproxy program. Really, it should build its own binaries from source code, like the other transports do. (It's a bit different for flash proxy and obfsproxy because they are written in Python, but for example on Windows we call their py2exe targets.)
I chose to rely on precompiled binaries to simplify the interface between the PTTBB build process and FTE. FTE has dependencies such as GMP, boost and OpenFST, which are non-trivial and time-consuming to properly build cross-platform.
The code to build the fteproxy binaries is documented and on github:
https://github.com/kpdyer/fteproxy-builder
The binaries are downloaded over TLS, and I can sign them, if that helps.
My main concern is that I didn't want to substantially increase the time and complexity of an already non-trivial build process.
Finally, I hate that you got caught in a transition between build systems, but the bundle branch is nearly defunct and probably won't be used to make future bundles. The pt branch in #9444 (closed) is now working well enough to build bundles on all platforms, so you might like to see what is needed to incorporate FTE there. (Grep the history for "Fetch obfsproxy" and "Install obfsproxy" for examples.)
Not a problem. I'll work towards integrating FTE into the new build process.
Given that the bundle branch is now (possibly) defunct, is it still worthwhile for me to cleanup my changes to bundle/Makefile?
Not a problem. I'll work towards integrating FTE into the new build process.
Given that the bundle branch is now (possibly) defunct, is it still worthwhile for me to cleanup my changes to bundle/Makefile?
I don't think it's worth spending too much time on bundle/Makefile. (But if it helps you make bundles and test your transport, then go ahead and keep using it, by all means.)
In the Gitian-based build, we're building as many shipping dependencies as possible from source, so we will appreciate it if you include recipes for building GMP, Boost, etc. in the Gitian descriptor.
Getting started with the Gitian build can be daunting, so we should perhaps do some IRC time to help get you started.
