wiki:doc/PluggableTransports

Version 108 (modified by elypter, 4 years ago) (diff)

parameters for pluggable transports

https://www.torproject.org/images/icon-Obfsproxy.jpg

Pluggable Transports

This is the wiki page about pluggable transports. It's directed towards PT researchers, developers and aficionados.

It's a wiki page; anyone can edit it. It's *not* official Tor Project material.

What are Pluggable Transports?

Pluggable transports transform the Tor traffic flow between the client and the bridge. This way, censors who monitor traffic between the client and the bridge will see innocent-looking transformed traffic instead of the actual Tor traffic. External programs can talk to Tor clients and Tor bridges using the pluggable transport API, to make it easier to build interoperable programs.

How to use pluggable transports

First, download Tor Browser.

If it is your first time running the browser, click Configure. Otherwise, click Open Network Settings... under the onion menu. Say Yes to the question Does your Internet Service Provider (ISP) block or otherwise censor connections to the Tor Network? and select the pluggable transport you want to use. If you don't know which one to use, try obfs3 first.

The pluggable transports come with default built-in bridges. If the default bridges are blocked, you need to get your own from https://bridges.torproject.org/.

To use flash proxy, see FlashProxyHowto.

To use meek, see doc/meek#Quickstart.

Pluggable Transports Collaboration Space

https://obfuscation.github.io/ is a web space set up for collaboration on pluggable transports in the wider world outside Tor.

Pluggable Transport IRC meetings

The Pluggable Transports community is conducting biweekly IRC meetings. You can find us in #tor-dev at the OFTC network every second Wednesday at 16:00 UTC.

You are welcome to join the meetings if you are interested in researching or developing pluggable transports or if you are generally curious about pluggable transports.

Traffic Obfuscation Mailing List

The traffic-obf list is for discussion of traffic obfuscation techniques, including pluggable transports.

https://groups.google.com/d/forum/traffic-obf

List of Pluggable Transports

obfs3

  • Description: Look-like-nothing pluggable transport (in obfsproxy)
  • Status: Deployed (Tor Browser)
  • Language: Python
  • Maintainer: asn

Flashproxy

  • Description: Zerg-like browser-based proxies XXX
  • Status: Deployed (Tor Browser)
  • Language: Python, Go, Javascript
  • Maintainer: David Fifield
  • Uses websocket-server on the server side, which can also be used with a standalone websocket-client, without going through a flash proxy.

Format-Transforming Encryption (FTE)

  • Description: Transforms data to arbitrary application-layer traffic.
  • Status: Deployed (Tor Browser)
  • Language: Python/C++
  • Maintainer: Kevin Dyer

ScrambleSuit

  • Description: Sophisticated look-like-nothing pluggable transport (in obfsproxy)
  • Status: Deployed (Tor Browser)
  • Language: Python
  • Maintainer: Philipp Winter

meek

  • Description: Uses HTTP, relays through a third party, TLS for obfuscation.
  • Status: Deployed (Tor Browser)
  • Language: Go.

bananaphone

  • Description: Markov-chains pluggable transport
  • Status: Under development.
  • Language: Python
  • Maintainer: David Stainton

LODP

  • Description: UDP-based pluggable transport.
  • Status: Under active development.
  • Language: C
  • Maintainer: Yawning

sshproxy

  • Description: SSH-based pluggable transport.
  • Status: Developed. Not deployed yet (lack of bundle building tech).
  • Language: Python
  • Notes: Actually uses the ssh binary
  • Maintainer: Yawning

hexchat

  • Description: XMPP-based pluggable transport.
  • Status: Developed. Not deployed yet.
  • Language: Python (SleekXMPP)
  • Notes: Bandwidth issues since most XMPP servers are throttled. Can be solved maybe with multiple hexchat bots.
  • Maintainer: Feynmann

Dust2

  • Description: Transforms traffic to arbitrary formats based on sample traffic.
  • Status: Under development.
  • Language: Haskell
  • Maintainer: Brandon Wiley

obfs2

  • Description: Look-like-nothing pluggable transport (in obfsproxy)
  • Status: Deployed (Tor Browser)
  • Language: Python
  • Notes: Superseded by obfs3
  • Maintainer: asn

Dust

  • Description: Look-like-nothing pluggable transport.
  • Status: Developed.
  • Language: Python
  • Notes: Superseded by Dust2
  • Maintainer: Brandon Wiley

StegoTorus

  • Description: Pluggable transport with modular output formats.
  • Status: Under development.
  • Language: C++
  • Notes: Part of DEFIANCE framework
  • Maintainer: vmon (?)

Code Talker Tunnel (previously: SkypeMorph)

  • Description: Skype-based pluggable transport
  • Status: Developed. Not deployed. (Ship Skype binary to users?!)
  • Language: C/C++
  • Notes: Actually uses the Skype binary
  • Maintainer: Ian Goldberg

git

  • Description: Git-based pluggable transport (in obfsproxy)
  • Status: Developed (?). Not deployed.
  • Language: Python
  • Notes: Git is poll-based. Slow.
  • Maintainer: Björgvin Ragnarsson && Pieter Westein

Castle

  • Description: Encodes messages as commands in online video games.
  • Status: Prototyped.
  • Language: C++.

PT libraries

  • pyptlib is a Python library which makes it easier to turn a proxy into a Tor pluggable transport.
  • goptlib is a pluggable transports library for golang. Check the example client and example server.
  • PLUTO: Pluggable Libraries Using Transport Obfuscation for Android.
  • liballium is a C/C++ library for people that absolutely need to use C or C++ for a pluggable transport.

Visual comparison of different pluggable transports

See a Child's Garden of Pluggable Transports for visual descriptions of how some existing transports appear on the wire.

How to set up a bridge with pluggable transports

Once you set up a transport type, your bridge will automatically advertise support for the transport in its descriptor.

obfs4:

FTE:

ScrambleSuit:

It's not really useful to run a WebSocket (flash proxy) or meek bridge, unless you plan to use it for your own use. These transports have another layer that chooses which bridges get connected to. If you set up a bridge, it won't be used automatically like it would with the other transports.

Circumvention systems to know about

Lantern

Lantern (github) uses proxies that run on your friends' computers. Your Lantern software connects to Lantern software running on a friend's (or a friend of a friend's) computer and uses it as a proxy. Similar to uProxy; differences include: Lantern is a normal program, not a browser extension; and Lantern lets you proxy through friends of friends. Lantern's goal is pure circumvention, not anonymity.

Lantern is capable of using UDT (reliable UDP-based protocol) for communication between instances.

Lantern faces the problem of how to distribute proxy addresses without the censor finding out about all of them. Kaleidoscope is in part a peer-to-peer method of sharing proxy addresses. Lantern has its own implementation of part of Kaleidoscope. Kaleidoscope is discussed in #7520.

Lantern has integrated FTE as a transport.

Psiphon

Psiphon is a circumvention system using SSH and VPN tunnels. Clients discover new servers dynamically.

Psiphon has adapted pluggable transports for their use.

GoAgent

GoAgent: proxy using Google App Engine. You download the software, run it on your own App Engine account, and use it as your personal proxy. See linked article for more.

Brainstorming for new transports

  • Use weaknesses of censoring devices
    • SCTP: a protocol something like TCP and something like UDP, supported natively in current operating systems. Your OS probably supports it: ncat --sctp -l -v starts an SCTP listener. It may be the case that deployed firewall hardware has poor support for filtering SCTP. There is an SCTP sockets API; it is trivial to change a C program that uses TCP sockets into one that uses SCTP sockets in a stream-oriented fashion (without using SCTP's advanced features): just change IPPROTO_TCP to IPPROTO_SCTP. Unfortunately, I did not find support for SCTP in any of: Python, Twisted, Go, libevent; so it may be necessary to write the transport using C sockets. Libs for Python Python Go?.You could prototype this transport using Ncat's SCTP support; for instance the server plugin could be just ncat --sctp -l <bindaddr> --sh-exec 'ncat localhost <orport>'. The Webrtc datachannel uses sctp so that could be a good protocol to mimic.
    • DCCP
    • ipv6. its still not widely enough spread to be worthy to implement it in dpi systems but even when this changes it will still be useful because it is easier to get a big amount of ips for bridges to use.
    • design pluggable transports in a way that they require a lot of resources to detect if it is not possible to avoid detection by a 100%. hide payload in compressed files.
  • Popular services Apart from minicry based attempts, most of these are highly unethical as data transit is being done without the consent, and likely in violation of the terms of service by the provider. what about meek? its also using a service in an unintended way. but thats ok because its not really doing harm? so its actually about not being unfair, not making poweful enemies and public opinion which are legit reasons but i dont think its "highly unethical". its rather a grey area between freedom of speech and bandidth stealing. ethic depends on the case. bandwidth could be throttled to be only usable to find new bridges or do irc chat. or what about a country that enforces whitelisting. then this could become the only information channel. is it really that unethical if a censor cooperating service is being used in that situation?
    • Can computer games be used to tunnel network data? Many people play a lot which means long-lived connections. Also, sometimes games implement a TCP-based control connection and a UDP-based transport connection. From the censor's point of view, messing with computer games might be undesirable as it would upset plenty of people. Done by Castle.
    • Tunnel over Facebook chat/other chat. See facebook-tunnel. It seems that some places give you access to e.g. Facebook but not the whole web.
  • Use private messaging functions and user generated content hosting of popular websites
    • facebook(see above)
    • ebay
    • amazon
    • paypal
    • google services
    • forums
    • soundcloud
    • twitter
  • Obfuscation of data flow(in combination with other pt)
    • cbr: Writes at a constant configurable rate, and embeds payload data in the junk somehow. May be useful as a layer in a transport composition as another way to hide a timing signature, or as a rate limiter. Doesn't have to be obfuscated as that can be in a different layer. Compare FreeWave which had good results with CBR VoIP channels. The rate can be configurable through the command line or SOCKS parameters:
        ClientTransportPlugin cbr exec /usr/bin/cbr-client --rate=40k
        Bridge cbr X.X.X.X:YYYY rate=40k
      
  • Too hot to touch candidates (See the note regarding dead parrot attacks)
    • bgp and other routing protocols: its important for the infrastructure and was even used to enforce censorship.
    • industrial, telemetry and military protocols(scada,lonworks,...). no censor wants to upset their govermnent or investors.
    • important internet infrastucture: ntp sql radius hadoop mongodb ftp
    • DNS pluggable transport. Encode data in recursive DNS queries and responses. Your local recursive resolver sends your packets to the right place. A dns bridge would be an authoritative name server for a particular domain; users would configure a domain rather than an IP address in their Bridge lines. Tools already exist to do DNS tunneling, for example iodine and dnscat. Probably requires a reliability layer and periodic polling by the client. nstx OzymanDNS OzymanDNS Dns2tcp DNScat tuns DaNeSh
    • ICMP(not that important) Ping tunnel Hans itun/simple icmp tunnel icmptx maybe more
    • protocols for IPcams and surveillance devices
  • Provide a long and relativly high bandwidth flow to hide data in (See the note regarding dead parrot attacks)
    • ventrillo, teamspeak and mumble
    • various streaming protocols
    • rdp, rfb(vnc) and teamviewer: possible too hot to touch too. nobody wants shut down tech support.
  • Imitate legit services (See the note regarding dead parrot attacks)
    • Webserver running forum software or cms. Data is transmitted via posting using software that is usually being used to spam
    • emails
  • Data steganography (in combination with other pt)
    • video, image and audio steganography
    • legit message steganography. make connections look like normal conversation by using chatbots.
    • use off the shelf filetype conversion tools to mimic valid filetypes
    • DissidentX: Bram Cohen's steganography scheme. It encodes messages in files on the web. Worth looking at (but might not scale very well for a PT).
  • upload and download files
    • from cloud storages
    • ftp srvers
    • network files(to get arround a local firewall)
    • any mountable network filesystem
  • Machine learning
    • build a network inside the censored country that the censor wants to block(making bittorrent pluggable transport compatible looks like a good option). let the nodes use a maximum flexible connection framework and apply machine learning(a new pt standard that allows parameters and chaining could provide that flexibility). survival of the fittest will lead to resistant configurations. let bridges learn from that.
    • build a test dpi censoring device that also uses machine learning to block tor. feed it with legit and tor data. let both AIs fight each other. take the winning pluggable transport configuration.
    • apply machine learning on stegotorus
  • Not really a pluggable transport
    • Use other working censorship circumvention tools (not sure if its good or worth making a pluggable transport for them but maybe promote them as a last resort if no bridge is working) list
    • mesh netorking. connect to someone with uncensored access or who has sat internet or someone across the border. wlan and other radio or optical connections(directional or undirectional), ethernet, dlan and other cable connections or connections within the same isp are usually not censored.
    • given that pluggable transports are being used by other applications too like filesharing inside the censored country the censors might hesitate to block ips because they could be dynamic or nated. in that case a seperate tor network could be build inside the country. if there are places where no censorship is applied(research facilities, autonous regions, borders) then exit nodes could be deployed there. it had to be a network completely out of bridges though. these exit nodes could then be used to connect to the normal tor network.

Help needed

  • PTs and rate limiting (https://lists.torproject.org/pipermail/tor-dev/2013-September/005419.html)
  • Write collection of Unix one-tool-per-task transports. One transport that simply obfuscates data. Another transport that simply changes data length. Another transport that passes transport through XMPP. Then pipe them together in layers to get combinations.
  • Improve pt-spec.txt.
  • And here are some coding tasks in case you want to write some code:

Tor tickets (in C):

  • #6456 Merge parse_client_transport_line() and parse_server_transport_line()
  • #5304: Obfsproxy should respect OutboundBindAddress in torrc
  • #5148: SIGHUP managed proxies when logs reopened
  • #9957: Tor should consider stderr output of transport proxies
  • #3725: Implement the wildcard "*" protocol in {Client,Server}TransportPlugin lines

obfsproxy tickets (in Python):

  • #12190: pyptlib does not join args
  • #9741: SIGINT handling for obfsproxy
  • #11770: Print a warning if python-gmpy is not found
  • #8001: obfsproxy makes tor warn when one bridge is down
  • #6264: obfsproxy: Add support for dropping privileges and chrooting
  • #9822: obfsproxy: Don't read authcookie file for every new connection
  • #9823: obfsproxy: validate_external_mode_cli() does not abort
  • #9824: Better integration tests for Python-based obfsproxy
  • #10047: PTs could self-shutdown when they detect their stdout is closed

Miscellaneous PT matters

Combining pluggable transports

We've been working on software that can compose transports together. Our original use case was to combine flashproxy with obfsproxy, so that flashproxy streams are obfuscated using obfs3 and harder to DPI for.

However, such software should also be able to compose together arbitrary transports. For example, we can imagine an HTTP transport that needs to remove the redundancy from its input stream, hence it implements a key exchange before it starts sending application data. Using the transport combiner, the author of the HTTP transport doesn't have to implement this, and can instead just combine her transport with obfs3, effectively having http(obfs3(stream)).

We can also imagine another transport, delayer, that delays traffic so that it evades signatures based on interpacket arrival time. And another transport, chopper, that chops up streams to evade signatures based on packet sizes. Now we can combine these new transports with other transports to evade DPI as needed. For example, http(delayer(obfs3(stream))) is a fun one, or even just delayer(chopper(obfs3(stream))). the syntax for this chaining should also allow parameters for each pluggable transport. if an adversary adapts to a set of pluggable transports it might be enough to just tweak some values. eg: delayer could be configured to delay just a bit more so the adversary doesn't block it anymore but lattency is still acceptable without having all tor users to upgrade. this would also be essential for machine learning(see brainstorming). machine learning already is a thing and maybe not in near future but in 5-10 years dpi devices could use it too. if the interfaces of tor do not allow tor to implement countermeasures then we will lose the bridge game.

While the idea is funny, the software is still unfinished and under development. We have an experimental version that is able to combine flashproxy with obfsproxy, and some design notes on how to generalize that for arbitrary transports. The code can be found here: https://gitweb.torproject.org/pluggable-transports/fog.git and some relevant trac tickets are #9744, #10061, #7167 .

Can I use any proxy as a pluggable transport?

Unfortunately, you can't simply use a proxy tool (e.g. GNU httptunnel) as a Tor pluggable transport. Tor and the pluggable transport proxy need to have a way to communicate with each other; for example, Tor needs to orchestrate the rate limiting of its pluggable transport proxies.

For this reason there is a specific interface that pluggable transports must adhere to.

When Tor launches a pluggable transport proxy, it exchanges some information with it (e.g. the directory that the proxy should use if it wants to save files). This information exchange happens using environment variables and the standard output. See the pt-spec for more information. The pyptlib library is able to help with this task.

The client-side of a pluggable transport must implement a SOCKS proxy. When Tor wants to connect to an obfuscated bridge, it connects to the SOCKS proxy, orders a CONNECT towards the bridge host, and then pushes its traffic through the SOCKS proxy. The pluggable transport encodes traffic while it's passing through the SOCKS proxy. (Note: The SOCKS protocol and this architecture is not ideal. See #7153)

The server-side of a pluggable transport must implement a client of the notorious Extended ORPort protocol. A Tor bridge runs an Extended ORPort server and exchanges some metadata with the transport proxy before receiving the actual client data. For example, the pluggable transport proxy tells Tor the name of the pluggable transport used for each connection, so that it can be recorded as a statistic. See the spec and the code of obfsproxy and pt.DialOr in goptlib.

All the above interfaces and protocols are documented in the pt-spec and a pluggable transport proxy is expected to support them.

Things to consider when adding a new pluggable transport

Read some background from developers

Strongly consider using a "safe" language

When getting your pluggable transport deployed, things will be considerably smoother if it is written in a memory safe language. There are exceptions to this rule, however as long term maintenance for pluggable transports is most likely going to end up being undertaken in part by developers/volunteers from the Tor project, languages that are harder to shoot yourself in the foot with are strongly preferred (If you have a reason in using something like C, you probably should talk to asn about it first).

Consider common attack vectors for your proposed protocol

At a minimum, your pluggable transport should be resistant to detection by a casual observer (Eg: DPI middleboxes). Note that it is strongly recommended that new pluggable transports be resistant to active probing: (obfs2 and obfs3 provide the minimum standard for the former. Scramblesuit was written to defeat the latter).

Also note that once, Iran terminated all TCP connections which did not look like HTTP after just 60 seconds.

If you are going to mimic/reuse an existing protocol, make sure that you read The Parrot is Dead: Observing Unobservable Network Communications and Cover Your ACKs: Pitfalls of Covert Channel Censorship Circumvention. countermeasures can be:

  • using the actual software for that protocol and hide the data in the payload on application level.
  • modify the software if it is open source to inject data directly removing gui overhead but keeping netork behavior.
  • switch bridges fast enough before the firewall can detect unusual traffic.
  • machine learning(look at brainstorming section)

Ensure that your protocol works on currently supported platforms

Ideally your protocol should run on the platforms officially supported by the Tor project's official releases. This currently is Linux, OS X (Intel), and Microsoft Windows (>= Windows XP).