Opened 4 years ago

Closed 2 years ago

Last modified 2 years ago

#16650 closed enhancement (fixed)

Set up domain fronting for BridgeDB

Reported by: isis Owned by: isis
Priority: Medium Milestone:
Component: Circumvention/BridgeDB Version:
Severity: Normal Keywords: bridgedb-dist, bridgedb-usability, tbb-wants, usability, bridge-distribution, TorCoreTeam201608
Cc: isis, dcf, mikeperry, yawning, mrphs, mcs, gk, brade, iry Actual Points:
Parent ID: Points:
Reviewer: Sponsor:

Description

We've been discussing setting up domain fronting for BridgeDB for a while now.

Benefits include better reachability (to BridgeDB) for clients in censored regions. Solving the problem of clients not being able to reach BridgeDB would allow for Tor Browser to do smarter things w.r.t. helping clients get bridges, helping them get the right kind of bridges, helping clients determine which kind of bridge is the right kind, and helping BridgeDB know more about which (types of) bridges are blocked (in specific regions, possibly). This will also allow Tor Browser to recommend to meek users to obtain a different type of working bridges, which will allow us to hopefully start reducing meek's costs without losing bridge users (and hopefully, without decreasing usability).

This shouldn't be too difficult to set up, however, some open questions include:

  • What changes, if any, will we need to make to meek-server to reuse David's work?
  • What changes will we need to make to BridgeDB?
  • Who will maintain the CDN accounts? Who will pay for them?
  • How can we ensure that the traffic to/from BridgeDB is end-to-end TLS encrypted? Can we do this and yet still get the client's real IP address (which BridgeDB currently uses for some necessary rate-limiting logic)?
  • How many, and which, CDNs do we want to set up?

Child Tickets

TicketTypeStatusOwnerSummary
#21772taskclosedtpaPlease install golang on polyanthum

Change History (17)

comment:1 Changed 4 years ago by isis

Pasting with permission a couple emails with great ideas for moving this forward which dcf sent me:

From: David Fifield
Subject: Fronting BridgeDB
Date: Thu, 25 Jun 2015 23:19:03
To: Isis

I've been thinking about how to make a domain-fronted interface to
BridgeDB. It seems like the best thing to do is tunnel end-to-end HTTPS
to bridges.torproject.org port 443. I think we can do that just by
cajoling meek-server to run as an standalone proxy. I've been thinking
about writing an externalize-pt program to do it, but for a prototype
you can just use a shell script.

export TOR_PT_MANAGED_TRANSPORT_VER=1
export TOR_PT_SERVER_BINDADDR=meek-0.0.0.0:2000
export TOR_PT_SERVER_TRANSPORTS=meek
export TOR_PT_ORPORT=127.0.0.1:443
meek-server --disable-tls

(For actual deployment we'll want a real cert and TLS.) Now you have a
meek-server talking to port 443 instead of an ORPort. If you send it the
right kind of request it will tunnel the body through to port 443.

echo -n $'POST / HTTP/1.0\r\nX-Session-Id: xxxxxxxx\r\nContent-Length: 0\r\n\r\n' | ncat localhost 2000

The next step is to set up the CDN and point it to the meek-server
bindaddr. Like for the appengine one, you would set in reflect.go:

forwardURL = "http://bridges.torproject.org:2000/"

Then on the client side, you have to set up another standalone proxy:

export TOR_PT_MANAGED_TRANSPORT_VER=1
export TOR_PT_CLIENT_TRANSPORTS=meek
export meek-client --url https://bridgedb.appspot.com/ --front www.google.com

(You can also pass the URL and front in the SOCKS request. For testing,
you can skip the CDN and just provide --url http://bridges.torproject.org:2000/.)
At this point the client can make HTTPS requests through the local SOCKS
proxy that meek-client runs (with a normal browser, even) and everything
should work great. The SOCKS proxy will only connect you to
bridges.torproject.org, even if you ask for something else.

It's not totally clear how this is going to work inside Tor Browser. You
have a nice local domain-fronted proxy that Tor Launcher can use, though
process management will be a chore.

One difficulty is communicating the client's IP address to the BridgeDB
HTTPS server. We can assume that meek-server knows the client's address
because it's carried in a header or something. Bur from the web server's
point of view, all the requests will be coming from localhost. We can't
inject a header to the request because it's the client's end-to-end
HTTPS. Tor uses the ExtORPort for this purpose but there's nothing
equivalent in a web server.


and

From: David Fifield
Subject: Re: Fronting BridgeDB
Date: Thu, 25 Jun 2015 23:29:29
To: Isis

On Thu, Jun 25, 2015 at 11:19:03PM, David Fifield wrote:

I've been thinking about how to make a domain-fronted interface to
BridgeDB. It seems like the best thing to do is tunnel end-to-end HTTPS
to bridges.torproject.org port 443. I think we can do that just by
cajoling meek-server to run as an standalone proxy. I've been thinking
about writing an externalize-pt program to do it, but for a prototype
you can just use a shell script.

export TOR_PT_MANAGED_TRANSPORT_VER=1
export TOR_PT_SERVER_BINDADDR=meek-0.0.0.0:2000
export TOR_PT_SERVER_TRANSPORTS=meek
export TOR_PT_ORPORT=127.0.0.1:443
meek-server --disable-tls

(For actual deployment we'll want a real cert and TLS.)

I think I know a better way to handle the TLS. You can use the current
web server as a TLS terminator, have it reverse-proxy to meek-server on
localhost over plain HTTP (carrying tunneled TLS), and then have
meek-server forward the tunneled TLS body straight back to port 443 on
localhost.

CDN --TLS(meek(TLS(GET /bridges)))--> Apache --meek(TLS(GET /bridges))--> meek-server --TLS(GET /bridges)--> Apache

That way you don't need to trust meek-server with your keys and you
don't need to open an external port. The Apache configuration would be
something like

ProxyPass /meek/ http://127.0.0.1:2000/

The setting on the CDN side would be

forwardURL = "https://bridges.torproject.org/meek/"

comment:2 Changed 4 years ago by mrphs

Cc: mrphs added

comment:3 Changed 4 years ago by mcs

Cc: mcs added

comment:4 Changed 4 years ago by gk

Cc: gk added

comment:5 Changed 4 years ago by elypter

this is related #17055

comment:6 Changed 3 years ago by isis

Keywords: TorCoreTeam201608 added

Adding to my august tickets.

comment:7 Changed 3 years ago by isis

Severity: Normal

I've set up a new Google account for the AppEngine instance, the latter of which I'm setting up now. We get $300 free for the first 12 months, and (I believe?) we get 1GB bandwidth (per day? per month? not sure) for free. Beyond that, I've currently attached my corporation's credit card to it. Once we start using it I can get an estimate of how much it will cost (probably nothing) and (hopefully, if Shari says it's okay) switch it over to using a Tor company credit card.

comment:8 in reply to:  7 ; Changed 3 years ago by mrphs

Replying to isis:

I've set up a new Google account for the AppEngine instance

Please don't use Google AppEngine for domain fronting. Google censors Iranian users for no good reason and you also remember they shut down the meek instance without much explanation.

comment:9 in reply to:  8 Changed 3 years ago by dcf

Replying to mrphs:

Replying to isis:

I've set up a new Google account for the AppEngine instance

Please don't use Google AppEngine for domain fronting. Google censors Iranian users for no good reason and you also remember they shut down the meek instance without much explanation.

I hear you, mrphs, but I don't think there's a problem using App Engine while writing the new code necessary for this ticket. App Engine is the easiest one to use for prototyping. With the design I think isis has in mind, it will be easy to replace it with some other service.

comment:10 Changed 3 years ago by isis

I tried to use Debian stable's go, installed in #21772, in the following manner:

#!/usr/bin/env bash

#GOROOT=/home/bridgedb/go
GOPATH=/home/bridgedb/go/
GO_BINARY=`which go`

MEEK_REPO=https://git.torproject.org/pluggable-transports/meek
PTLIB_REPO=https://git.torproject.org/pluggable-transports/goptlib
CRYPTO_REPO=https://go.googlesource.com/crypto
CONTEXT_REPO=https://golang.org/x/net/context

if test -z "$GO_BINARY" ; then
    printf "It seems Go is not installed! Exiting...\n"
    exit 2
fi

PT_PATH=${GO_PATH}/src/git.torproject.org/pluggable-transports
X_PATH=${GO_PATH}/src/golang.org/x

for path in "$PT_PATH" "$X_PATH" ; do
    if ! test -d "$path" ; then
        mkdir -p "$path"
    fi
done

cd "$PT_PATH"
for repo in "$MEEK_REPO" "$PTLIB_REPO" ; do
    git clone $repo
done

# For some reason, it expects the goptlib repo to live in a directory
# called "goptlib.git"
mv goptlib goptlib.git

cd "$X_PATH"
git clone  "$CRYPTO_REPO"

mkdir net
cd net
git clone "$CONTEXT_REPO"

# In  /home/bridgedb/go/src/golang.org/x/crypto/acme/autocert/autocert.go
# and /home/bridgedb/go/src/golang.org/x/crypto/acme/autocert/cache.go
# and /home/bridgedb/go/src/golang.org/x/crypto/acme/autocert/renewal.go
# and /home/bridgedb/go/src/golang.org/x/crypto/acme/acme.go
# you will need to change the line in `import(...)` which says
#
#     "context"
#
# to instead be:
#
#     context "golang.org/x/net/context"
#
# because Debian's go version is extremely old and package which used          
# to be at golang.org/x/net/conext is now top-level in the standard library.
#
# Next, in /home/bridgedb/go/src/golang.org/x/crypto/acme/acme.go
# you will need to add in the `import(...) the following:
#
#     "golang.org/x/crypto/ssh/keys"
#
# and then, in that same file, change all instances of "crypto.Signer" to
# "keys.Signer".

This is all because Debian's go version is 1.3.3. These crypto libraries all expect at least go1.7.

I'm just going to install gotools the proper way, in bridgedb's home directory. I know a lot of Debian people are going to complain about this, but I'm sorry, this is insane and a waste of time. I can build meek on my laptop just fine, with go build, and I should be able to do the same on the server.

comment:11 Changed 3 years ago by isis

Okay, meek-server is installed. Here's what I did:

#!/usr/bin/env bash

GOROOT=/home/bridgedb/go

# Download the official GoTools
curl -O https://storage.googleapis.com/golang/go1.8.1.linux-amd64.tar.gz

# Ensure the checksum is correct
hash=`sha256sum go1.8.1.linux-amd64.tar.gz | cut -d ' ' -f 1`
if [[ "$hash" != "a579ab19d5237e263254f1eac5352efcf1d70b9dacadb6d6bb12b0911ede8994" ]] ; then
    printf "Bad download of GoTools! Hashes did not match! Exiting...\n"
    exit 127
fi

# Untar the archive
tar -C "${GOROOT%%/go}" -xzf go1.8.1.linux-amd64.tar.gz

# Add /home/bridgedb/go/bin to our PATH
PATH="${GOROOT}"/bin:${PATH}

export GOROOT
export PATH

# Go should now be installed
if test -z `which go` ; then
    printf "It seems Go is not installed! Exiting...\n"
    exit 2
fi

# Create the tree for projects and dependencies
PT_PATH=${GOROOT}/src/git.torproject.org/pluggable-transports
X_PATH=${GOROOT}/src/golang.org/x

for path in "$PT_PATH" "$X_PATH" ; do
    if ! test -d "$path" ; then
        mkdir -p "$path"
    fi
done

MEEK_REPO=https://git.torproject.org/pluggable-transports/meek
PTLIB_REPO=https://git.torproject.org/pluggable-transports/goptlib
CRYPTO_REPO=https://go.googlesource.com/crypto

# Clone meek and goptlib
cd "$PT_PATH"
for repo in "$MEEK_REPO" "$PTLIB_REPO" ; do
    git clone $repo
done

# For some reason, it expects the goptlib repo to live in a directory
# called "goptlib.git"
mv goptlib goptlib.git

# Clone golang.org/x/crypto
cd "$X_PATH"
git clone  "$CRYPTO_REPO"

# Build meek-server
cd "${PT_PATH}/meek/meek-server"
go build

comment:12 in reply to:  7 Changed 3 years ago by isis

Replying to isis:

I've set up a new Google account for the AppEngine instance, the latter of which I'm setting up now. We get $300 free for the first 12 months, and (I believe?) we get 1GB bandwidth (per day? per month? not sure) for free. Beyond that, I've currently attached my corporation's credit card to it. Once we start using it I can get an estimate of how much it will cost (probably nothing) and (hopefully, if Shari says it's okay) switch it over to using a Tor company credit card.


I forgot to mention that Google disabled this account I had made sometime during the 2017 Amsterdam meeting. They want "government issued identity documents" for the person known as "Tor Bridges". I could probably get a fake ID, but for now I just created a new account yesterday. I didn't sign up for the $300 free credit, and so far it hasn't been disabled.

comment:13 Changed 3 years ago by isis

Status: newneeds_information

The new Google developer account is configured, and the meek reflector is installed. meek-server is also installed on polyanthum, as mentioned above. Both appear to be working, but they don't want to talk to each other through the Apache reverse proxy. (Which doesn't matter all that much right now, since there's nothing for them to talk to until #7520 is implemented.) Still, some help from someone with Apache wizardry skills would be nice.

Right now the XXXXXXXXXXXXXX.appspot.com domain is forwarding requests to bridges.torproject.org:2000, where Apache appears to be picking it up and then not forwarding to meek.

Last edited 3 years ago by isis (previous) (diff)

comment:14 in reply to:  13 Changed 3 years ago by dcf

Replying to isis:

The new Google developer account is configured, and the meek reflector is installed. meek-server is also installed on polyanthum, as mentioned above. Both appear to be working, but they don't want to talk to each other through the Apache reverse proxy. (Which doesn't matter all that much right now, since there's nothing for them to talk to until #7520 is implemented.) Still, some help from someone with Apache wizardry skills would be nice.

Right now the XXXXXXXXXXXXXX.appspot.com domain is forwarding requests to bridges.torproject.org:2000, where Apache appears to be picking it up and then not forwarding to meek.

The way I pictured it working (might not actually work since I didn't try it):

  • Run meek-server listening on 127.0.0.1:2000 (i.e., not listening externally) with ORPort 127.0.0.1:443
  • XXXXXXXXXXXXXX.appspot.com forwards to https://bridges.torproject.org/meek (i.e., to port 443, not 2000, and with a path that marks it for ProxyPass forwarding)
  • ProxyPass /meek/ ​http://127.0.0.1:2000/ recognizes the forwarded appspot requests through the /meek/ path and sends them to meek-server on localhost
  • meek-server then forwards the tunneled TLS back to the HTTPS port.

The way this would look on the client side is something like:

export TOR_PT_MANAGED_TRANSPORT_VER=1
export TOR_PT_CLIENT_TRANSPORTS=meek
meek-client --url ​https://XXXXXXXXXXXXXX.appspot.com/ --front www.google.com

meek-client will output a line like CMETHOD meek socks5 127.0.0.1:YYYYY telling you it is listening on port YYYY. And then, download a page through the tunnel with

curl --proxy socks4a://127.0.0.1:YYYY https://bridges.torproject.org/

comment:15 Changed 2 years ago by isis

Resolution: fixed
Status: needs_informationclosed

David and I spent some time in April hacking on this, and we've got it set up such that, if you request https://tor-bridges-hyphae-channel.appspot.com/meek it'll go to the Apache reverse proxy on polyanthum, which strips out the authenticated layer of TLS (i.e. using the cert for bridges.torproject.org) and then passes the underlying payload (still TLS encrypted, but with a random cert) to meek which is listening on localhost, which then passes it to the BridgeDB server. (Right now the BridgeDB server doesn't understand the request yet for bridges.torproject.org/meek yet, so it responds with a 404. But the point is that the meek tunnel is set up and works.)

comment:16 Changed 2 years ago by mcs

Cc: brade added

comment:17 Changed 2 years ago by iry

Cc: iry added
Note: See TracTickets for help on using tickets.