wiki:doc/meek

Version 7 (modified by arlolra, 5 years ago) (diff)

--

Meek is a transport that uses HTTP for carrying bytes and TLS for obfuscation. Traffic is relayed through a third-party server (Google App Engine). It uses a trick to talk to the third party so that it looks like it is talking to an unblocked server.

Quick start

git clone https://www.bamsoftware.com/git/meek.git
cd meek/meek-client
export GOPATH=~/go
go get
go build
tor -f torrc

Overview

When meek-client receives a SOCKS request from the Tor client, it generates a random session id string. meek-client makes an HTTP POST to https://meek-reflect.appspot.com/, which is a special web app set up for this transport. The request looks something like this:

GET / HTTP/1.1
Host: meek-reflect.appspot.com
X-Session-Id: x5ej2h96frvLXeqgKNjIyRDRJidU8RMIeRPDzvLVG+E=
Content-Type: application/octet-stream
Content-Length: 100

<data payload follows>

Normally a censor would be able to search for the string meek-reflect.appspot.com and block the connection. But the HTTP request is inside HTTPS, and the IP address and SNI of the HTTPS connection are those of www.google.com.

The web app running at meek-reflect.appspot.com is very simple: it just copies the POST request it receives, and makes an identical request to a meek-server running on a Tor relay. There is a meek-server instance running at http://tor1.bamsoftware.com:7002/. The request from App Engine to meek-server looks like:

GET / HTTP/1.1
Host: tor1.bamsoftware.com:7002
X-Session-Id: x5ej2h96frvLXeqgKNjIyRDRJidU8RMIeRPDzvLVG+E=
Content-Type: application/octet-stream
Content-Length: 100

<data payload follows>

When meek-server receives this request, it reads the session id string x5ej2h96frvLXeqgKNjIyRDRJidU8RMIeRPDzvLVG+E= and checks if it has an existing session (i.e., ORPort connection) by that name. If it does, it copies the data payload to the ORPort. Otherwise, it creates a new ORPort connection, and copies the data payload to it. In any case, it then tries to read from the ORPort, and sends an HTTP response back to App Engine with any data from the Tor relay:

HTTP/1.1 200 OK
Content-Type: application/octet-stream
Content-Length: 200

<data payload follows>

The web app at meek-reflect.appspot.com simply copies the HTTP response and sends it back to the client:

HTTP/1.1 200 OK
Content-Type: application/octet-stream
Content-Length: 200

<data payload follows>

When meek-client finally receives this response, it writes the data payload back to its SOCKS port. The process then repeats: the client tries to read from its SOCKS port, it makes a request to App Engine, App Engine copies request to meek-server, meek-server writes to and reads from the ORPort, meek-server sends a response, App Engine copies the response to meek-client, and meek-client writes the response body to the SOCKS port.

There is no way for the server to push data to the client. It has to wait for a request in order to send its data in the response. For this reason, the client polls the server, making a request periodically even if it has nothing to send. The polling starts out frequent and backs off exponentially while the server has nothing to say. The polling interval resets to the minimum every time the server sends some data.

Other clients operating simultaneously will use different session id strings, so the server can tell them apart. Stale session ids that have not had any traffic in a while are closed and forgotten.

Things to do

Build a PHP middle relay that can be used in the place of App Engine. (Easy and fun.) (#10984)

Make the TLS signature closer to that of a browser; see #Distinguishability. (May need a client rewrite.)

Pin Google's public key like flashproxy-reg-appspot does. Decide what to do if the user asked to use a server other than Google's. (May need a client rewrite.)

General notes about App Engine

Quotas for unpaid apps:

You can pay to get higher quotas:

There is also a higher "premier" level of service that comes with support:

Paying for more bandwidth on a public proxy server would be nice in that users wouldn't have to set up their own instance, and we could hardcode the server inside a browser bundle. (It would be nice to know the cost per gigabyte of running App Engine and, say, an ordinary Tor relay.)

An idea to reduce overhead and eliminate polling is to use HTTP as a long-lived bidirectional channel, sending upstream data in a POST body and receiving data in the response body simultaneously. (That is, you send a POST with no Content-Length, the server reads your header and forwards the request to the relay, the server writes back a header, and after that you use the connection as an ordinary socket, with upstream and downstream data interleaved.) An implementation of this idea is at https://www.bamsoftware.com/git/meeker.git. The idea doesn't work with App Engine, for two reasons. 1) requests must be handled within 60 seconds, and 2) App Engine doesn't support streaming requests of this kind:

"App Engine calls the handler with a Request and a ResponseWriter, then waits for the handler to write to the ResponseWriter and return. When the handler returns, the data in the ResponseWriter's internal buffer is sent to the user. This is practically the same as when writing normal Go programs that use the http package. The one notable difference is that App Engine does not support streaming data in response to a single request."

App Engine doesn't even call your web app code until it has consumed the entire request body, and doesn't start flushing the response body until you close the output stream.

Ideas

The App Engine Channel API provides a way to have long-lived push connections to the client, subject to a restricted interface. (HTTP handlers are otherwise required to finish within 60 seconds.) The client could use HTTP request bodies to send data, and a channel to receive, and remove the need for polling. It would require us to reimplement the client JavaScript channel API in order to make use of the particular Comet-based protocol.

Paid apps can create outbound sockets. I don't think it helps us because then the web app would be responsible for managing the session id mapping.

GoAgent is similar in that it also uses App Engine as a middleman.

Distinguishability

Client Hello of meek-client f748701c (2014-02-08).

Secure Sockets Layer
    SSL Record Layer: Handshake Protocol: Client Hello
        Content Type: Handshake (22)
        Version: TLS 1.0 (0x0301)
        Length: 135
        Handshake Protocol: Client Hello
            Handshake Type: Client Hello (1)
            Length: 131
            Version: TLS 1.2 (0x0303)
            Random
                gmt_unix_time: Feb 12, 2014 02:01:40.000000000 PST
                random_bytes: 38e06b8022577f5b08d7a35b8aae7a8c9813ba6498e86a6e...
            Session ID Length: 0
            Cipher Suites Length: 26
            Cipher Suites (13 suites)
                Cipher Suite: TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256 (0xc02f)
                Cipher Suite: TLS_ECDHE_ECDSA_WITH_AES_128_GCM_SHA256 (0xc02b)
                Cipher Suite: TLS_ECDHE_RSA_WITH_RC4_128_SHA (0xc011)
                Cipher Suite: TLS_ECDHE_ECDSA_WITH_RC4_128_SHA (0xc007)
                Cipher Suite: TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA (0xc013)
                Cipher Suite: TLS_ECDHE_ECDSA_WITH_AES_128_CBC_SHA (0xc009)
                Cipher Suite: TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA (0xc014)
                Cipher Suite: TLS_ECDHE_ECDSA_WITH_AES_256_CBC_SHA (0xc00a)
                Cipher Suite: TLS_RSA_WITH_RC4_128_SHA (0x0005)
                Cipher Suite: TLS_RSA_WITH_AES_128_CBC_SHA (0x002f)
                Cipher Suite: TLS_RSA_WITH_AES_256_CBC_SHA (0x0035)
                Cipher Suite: TLS_ECDHE_RSA_WITH_3DES_EDE_CBC_SHA (0xc012)
                Cipher Suite: TLS_RSA_WITH_3DES_EDE_CBC_SHA (0x000a)
            Compression Methods Length: 1
            Compression Methods (1 method)
                Compression Method: null (0)
            Extensions Length: 64
            Extension: server_name
                Type: server_name (0x0000)
                Length: 19
                Server Name Indication extension
                    Server Name list length: 17
                    Server Name Type: host_name (0)
                    Server Name length: 14
                    Server Name: www.google.com
            Extension: status_request
                Type: status_request (0x0005)
                Length: 5
                Data (5 bytes)
            Extension: elliptic_curves
                Type: elliptic_curves (0x000a)
                Length: 8
                Elliptic Curves Length: 6
                Elliptic curves (3 curves)
                    Elliptic curve: secp256r1 (0x0017)
                    Elliptic curve: secp384r1 (0x0018)
                    Elliptic curve: secp521r1 (0x0019)
            Extension: ec_point_formats
                Type: ec_point_formats (0x000b)
                Length: 2
                EC point formats Length: 1
                Elliptic curves point formats (1)
                    EC point format: uncompressed (0)
            Extension: signature_algorithms
                Type: signature_algorithms (0x000d)
                Length: 10
                Data (10 bytes)

Client Hello of Chromium 31.0.1650.63 Debian jessie/sid (238485).

Secure Sockets Layer
    SSL Record Layer: Handshake Protocol: Client Hello
        Content Type: Handshake (22)
        Version: TLS 1.0 (0x0301)
        Length: 208
        Handshake Protocol: Client Hello
            Handshake Type: Client Hello (1)
            Length: 204
            Version: TLS 1.2 (0x0303)
            Random
                gmt_unix_time: Feb 12, 2014 02:16:29.000000000 PST
                random_bytes: b05b2298a9015323e39cab49d1874c8aa608bcec1fb0a280...
            Session ID Length: 0
            Cipher Suites Length: 36
            Cipher Suites (18 suites)
                Cipher Suite: TLS_ECDHE_ECDSA_WITH_AES_128_GCM_SHA256 (0xc02b)
                Cipher Suite: TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256 (0xc02f)
                Cipher Suite: TLS_DHE_RSA_WITH_AES_128_GCM_SHA256 (0x009e)
                Cipher Suite: TLS_RSA_WITH_AES_128_GCM_SHA256 (0x009c)
                Cipher Suite: TLS_ECDHE_ECDSA_WITH_AES_256_CBC_SHA (0xc00a)
                Cipher Suite: TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA (0xc014)
                Cipher Suite: TLS_DHE_RSA_WITH_AES_256_CBC_SHA (0x0039)
                Cipher Suite: TLS_RSA_WITH_AES_256_CBC_SHA (0x0035)
                Cipher Suite: TLS_ECDHE_ECDSA_WITH_RC4_128_SHA (0xc007)
                Cipher Suite: TLS_ECDHE_ECDSA_WITH_AES_128_CBC_SHA (0xc009)
                Cipher Suite: TLS_ECDHE_RSA_WITH_RC4_128_SHA (0xc011)
                Cipher Suite: TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA (0xc013)
                Cipher Suite: TLS_DHE_RSA_WITH_AES_128_CBC_SHA (0x0033)
                Cipher Suite: TLS_DHE_DSS_WITH_AES_128_CBC_SHA (0x0032)
                Cipher Suite: TLS_RSA_WITH_RC4_128_SHA (0x0005)
                Cipher Suite: TLS_RSA_WITH_RC4_128_MD5 (0x0004)
                Cipher Suite: TLS_RSA_WITH_AES_128_CBC_SHA (0x002f)
                Cipher Suite: TLS_RSA_WITH_3DES_EDE_CBC_SHA (0x000a)
            Compression Methods Length: 1
            Compression Methods (1 method)
                Compression Method: null (0)
            Extensions Length: 127
            Extension: server_name
                Type: server_name (0x0000)
                Length: 19
                Server Name Indication extension
                    Server Name list length: 17
                    Server Name Type: host_name (0)
                    Server Name length: 14
                    Server Name: www.google.com
            Extension: renegotiation_info
                Type: renegotiation_info (0xff01)
                Length: 1
                Renegotiation Info extension
                    Renegotiation info extension length: 0
            Extension: elliptic_curves
                Type: elliptic_curves (0x000a)
                Length: 8
                Elliptic Curves Length: 6
                Elliptic curves (3 curves)
                    Elliptic curve: secp256r1 (0x0017)
                    Elliptic curve: secp384r1 (0x0018)
                    Elliptic curve: secp521r1 (0x0019)
            Extension: ec_point_formats
                Type: ec_point_formats (0x000b)
                Length: 2
                EC point formats Length: 1
                Elliptic curves point formats (1)
                    EC point format: uncompressed (0)
            Extension: SessionTicket TLS
                Type: SessionTicket TLS (0x0023)
                Length: 0
                Data (0 bytes)
            Extension: next_protocol_negotiation
                Type: next_protocol_negotiation (0x3374)
                Length: 0
            Extension: Unknown 16
                Type: Unknown (0x0010)
                Length: 34
                Data (34 bytes)
            Extension: Unknown 30031
                Type: Unknown (0x754f)
                Length: 0
                Data (0 bytes)
            Extension: status_request
                Type: status_request (0x0005)
                Length: 5
                Data (5 bytes)
            Extension: signature_algorithms
                Type: signature_algorithms (0x000d)
                Length: 18
                Data (18 bytes)

Attachments (20)

Download all attachments as: .zip