Use ESNI via Firefox HTTPS helper

added component::circumvention/meek owner::dcf priority::medium resolution::implemented severity::normal status::closed type::project labels

Trac:
Description: As of 2018-10-18, Firefox Nightly supports encrypted SNI, and Cloudflare supports it on the server side. Because meek supports using Firefox as a channel for issuing HTTPS requests, it ought to be pretty easy to adapt the meek client software to use ESNI rather than domain fronting. The server software doesn't need any change.

These steps are untested:

Download Tor Browser and Firefox Nightly.
Set network.trr.mode=3 and network.security.esni.enabled=true in Firefox Nightly.
Copy the !meek-http-helper@bamsoftware.com.xpi from Tor Browser to Firefox Nightly.
Hack meek-client-torbrowser/{mac,linux,windows}.go to point firefoxPath at the copy of Firefox Nightly and disable the custom profile. (Additional hacks to remove hardcoded Tor Browser assumptions may be required.)
Set up a Cloudflare instance pointing to !https://meek.bamsoftware.com/, call it !https://meek.example.com/.
Set up a [[doc/meek#Howtochangethefrontdomain|custom bridge]] in Tor Browser, using url= without front= (because we're no longer domain fronting).\bridge meek 0.0.2.0:3 url=https://meek.example.com/

Of course, once ESNI support makes it into the version of Firefox used by Tor Browser, this will be even easier, not requiring a separate Firefox Nightly.

to

As of 2018-10-18, Firefox Nightly supports encrypted SNI, and Cloudflare supports it on the server side. Because meek supports using Firefox as a channel for issuing HTTPS requests, it ought to be pretty easy to adapt the meek client software to use ESNI rather than domain fronting. The server software doesn't need any change.

These steps are untested:

Download Tor Browser and Firefox Nightly.
Go to about:config in Firefox Nightly and set
- network.trr.mode=3
- network.trr.uri=!https://1.1.1.1/dns-query
- network.security.esni.enabled=true
Copy the !meek-http-helper@bamsoftware.com.xpi from Tor Browser to Firefox Nightly.
Hack meek-client-torbrowser/{mac,linux,windows}.go to point firefoxPath at the copy of Firefox Nightly and disable the custom profile. (Additional hacks to remove hardcoded Tor Browser assumptions may be required.)
Set up a Cloudflare instance pointing to !https://meek.bamsoftware.com/, call it !https://meek.example.com/.
Set up a [[doc/meek#Howtochangethefrontdomain|custom bridge]] in Tor Browser, using url= without front= (because we're no longer domain fronting).\bridge meek 0.0.2.0:3 url=https://meek.example.com/

Of course, once ESNI support makes it into the version of Firefox used by Tor Browser, this will be even easier, not requiring a separate Firefox Nightly.

Trac:
Username: darkspirit
Cc: N/A to jan@ikenmeyer.eu

Removing the "easy" tag as it seems that the helper will have to be rewritten as a WebExtension first (#29347 (moved)) in order to work with versions of Firefox that are recent enough to support ESNI.

Trac:
Keywords: easy deleted, N/A added

As of e551c05e47, the work in #29347 (moved) is far enough along to start experimenting with this. I didn't yet make a Cloudflare account to actually test ESNI, but I got Tor Browser running with meek-azure and an external Firefox 66 doing DNS-over-HTTPS and all the other necessary configuration for ESNI.

Enter meek/webextension/native and run go build. This produces the native component of the extension.
Enter meek/webextension and run make. This zips up the extension files into an installable extension, !meek-http-helper@bamsoftware.com.xpi.
Download Firefox Developer Edition. You need the developer edition in order to install an unsigned extension. I used firefox-66.0b9.
Run firefox/firefox --ProfileManager and create a new "esni" profile. Go to about:config and set these prefs: {{{ browser.dom.window.dump.enabled network.trr.mode=3 network.trr.uri=https://1.1.1.1/dns-query network.security.esni.enabled=true toolkit.startup.max_resumed_crashes=-1 xpinstall.signatures.required=false }}}
Go to about:addons. Click Extensions. Click ️ and select "Install Add-on From File...". Open meek/webextension/!meek-http-helper@bamsoftware.com.xpi. Say yes to the permissions dialog.
Close Firefox.
Edit meek/webextension/meek.http.helper.json and change the "path" field to be the absolute path to meek/webextension/native/native.
Copy meek/webextension/meek.http.helper.json into the Tor Browser tree. (The path is different on mac.) {{{ mkdir -p tor-browser_en-US/Browser/.mozilla/native-messaging-hosts cp meek/webextension/meek.http.helper.json tor-browser_en-US/Browser/.mozilla/native-messaging-hosts/ }}}
Edit meek/meek-client-torbrowser/linux.go (or mac.go, windows probably doesn't work yet) and set the paths to the Firefox developer edition and the "esni" profile you created: {{{ firefoxPath = "/path/to/firefox/firefox" firefoxProfilePath = "/home/user/.mozilla/firefox/.esni" }}}
In meek/meek-client-torbrowser, run go build.
Copy the resulting meek-client-torbrowser binary to tor-browser_en-US/Browser/TorBrowser/Tor/PluggableTransports/.
Now you're good to go. Start up Tor Browser with meek-azure.

The only step of this that is cheating--the one I don't know how to automate yet--is step 7, where the WebExtension host manifest needs to know the absolute path to the native executable. I suppose meek-client-torbrowser could do a getcwd and write the entire file anew each time.

I set up a Cloudflare account and got this all working: meek with ESNI in place of domain fronting, running in Tor Browser with an external Firefox helper. When Tor Browser starts using a Firefox newer than 60 ESR, it won't need an separate external Firefox.

Cloudflare setup

Register a new domain name. I'll use rinsed-tinsel.site for this example.
Go to https://dash.cloudflare.com/ and log in.
Click "+ Add site", enter the domain name, and choose the free plan.
At the DNS screen, add a new A/AAAA record pointing to the bridge that will host meek-server.
You'll also need a TLS certificate for meek-server. You cannot use meek-server's automatic Let's Encrypt support, because the DNS for meek.rinsed-tinsel.site points to a Cloudflare server (which has its own Cloudflare-issued certificate), not to the meek-server bridge. Go to the Crypto tab and click "Create Certificate" in the "Origin Certificates" panel. Click "I have my own private key and CSR". Generate and upload a CSR:

$ openssl genpkey -algorithm EC -pkeyopt ec_paramgen_curve:P-256 -out meek-cloudflare.key $ openssl req -new -key meek-cloudflare.key -out meek-cloudflare.csr }}} Paste the resulting cert into meek-cloudflare.crt.

Go back to the name registrar and set the nameserver to the two *.ns.cloudflare.com servers that it tells you to set.
I then made the following Cloudflare configuration changes:
- Crypto tab
  - SSL: Full (strict)
  - Always Use HTTPS: On
  - Minimum TLS Version: TLS 1.2
- Firewall tab
  - Security Level: Essentially Off
  - Web Application Firewall
    - Browser Integrity Check: Off
- Caching tab
  - Always Online™: Off
- Scrape Shield tab
  - Email Address Obfuscation: Off
  - Server-side Excludes: Off
  - Hotlink Protection: Off

Bridge setup

Set up meek-server according to the instructions at [[doc/meek#Howtorunameek-serverbridge]].
- Use the command-line options --key meek-cloudflare.key --cert meek-cloudflare.crt using the key and certificate you generated while setting up Cloudflare.

WebExtension build

Start with commit e80c5b3217 or later in the webextension branch.

Enter meek/webextension/native and run go build. This produces the native component of the extension.
Enter meek/webextension/browser and run make. This zips up the extension files into an installable bundle, !meek-http-helper@bamsoftware.com.xpi.

Firefox setup

Download Firefox Developer Edition. You need the developer edition in order to install an unsigned extension.
Run firefox/firefox --ProfileManager and create a new "esni" profile. Go to about:config and set these prefs: {{{ browser.dom.window.dump.enabled=true network.trr.mode=3 network.trr.uri=https://1.1.1.1/dns-query network.security.esni.enabled=true security.OCSP.enabled=0 toolkit.startup.max_resumed_crashes=-1 xpinstall.signatures.required=false }}}
- Explanation: browser.dom.window.dump.enabled=true enables the WebExtension to write to stdout, which is needed as part of the protocol with meek-client-torbrowser. network.trr.mode means Trusted Recursive Resolver and controls DNS-over-HTTPS, which is a prerequisite for ESNI; mode 3 means always use DNS-over-HTTPS. network.trr.uri is the DNS-over-HTTPS server. We use an IP address to avoid a chicken-and-egg name resolution problem. An alternative to using an IP address is to set network.trr.bootstrapAddress. network.security.esni.enabled enables ESNI. security.OCSP.enabled=0 disables OCSP, because OCSP can leak the server name. toolkit.startup.max_resumed_crashes prevents Firefox from uninstalling add-ons when there's a startup error. xpinstall.signatures.required allows the installation of an unsigned add-on.
Go to about:addons. Click Extensions. Click ️ and select "Install Add-on From File...". Open meek/webextension/!meek-http-helper@bamsoftware.com.xpi. Say yes to the permissions dialog.
Close Firefox.

meek-client-torbrowser build

Edit meek/meek-client-torbrowser/{linux,mac,windows}.go (whatever's needed for your platform) and adjust the paths: {{{ firefoxPath = "/path/to/firefox/firefox" firefoxProfilePath = "/home/user/.mozilla/firefox/.esni" helperNativeManifestDir = "/path/to/tor-browser_en-US/Browser/.mozilla/native-messaging-hosts" helperNativeExecutablePath = "/path/to/meek/webextension/native/native" }}}
In meek/meek-client-torbrowser, run go build.
Copy the resulting meek-client-torbrowser binary to tor-browser_en-US/Browser/TorBrowser/Tor/PluggableTransports/.

Tor Browser setup

Click the "Configure" button in Tor Launcher, or "Tor Network Settings..." in the onion toolbar icon.
Click "Tor is censored in my country" and "Provide a bridge I know". Enter the bridge line: {{{ meek 0.0.2.0:3 1922840D0D66CB82EACE4327F5001430227C0127 url=https://meek.rinsed-tinsel.site/ }}}
Because of #12774 (moved), it may not work right away and you'll have to restart.

This is a packet capture of bootstrapping and browsing to example.com: meek-esni.2.pcap. Here's a summary of all the Client Hellos it contains: {{{ No. Time Source Destination Protocol Info 12 2019-03-28 18:04:44 192.168.111.2 1.1.1.1 TLSv1.2 Client Hello 13 2019-03-28 18:04:44 192.168.111.2 1.1.1.1 TLSv1.3 Client Hello 14 2019-03-28 18:04:44 192.168.111.2 1.1.1.1 TLSv1.2 Client Hello 17 2019-03-28 18:04:45 192.168.111.2 1.1.1.1 TLSv1 Client Hello 135 2019-03-28 18:04:45 192.168.111.2 1.1.1.1 TLSv1.2 Client Hello 138 2019-03-28 18:04:45 192.168.111.2 1.1.1.1 TLSv1.2 Client Hello 139 2019-03-28 18:04:45 192.168.111.2 1.1.1.1 TLSv1.2 Client Hello 262 2019-03-28 18:04:47 192.168.111.2 104.27.168.47 TLSv1.3 Client Hello

All the handshakes with 1.1.1.1 are DNS-over-HTTPS name lookup—I'm guessing some of them are Firefox's internal lookups, unrelated to the meek tunnel. 104.27.168.47 is the Cloudflare edge server.

The TLS fingerprints are:
|1.1.1.1       |[8300bf0e26f2a109](https://tlsfingerprint.io/id/8300bf0e26f2a109) ([archive](https://web.archive.org/web/20190227210213/https://tlsfingerprint.io/id/8300bf0e26f2a109)) rank 3620 |[comparison](https://tlsfingerprint.io/compare/bb94e801f7aee52b/8300bf0e26f2a109) ([archive](https://web.archive.org/web/20190227210604/https://tlsfingerprint.io/compare/bb94e801f7aee52b/8300bf0e26f2a109)) with ESR 60 rank 31 |
|--------------|----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|
|104.27.168.47 |[2dcbeba533890640](https://tlsfingerprint.io/id/2dcbeba533890640) ([archive](https://web.archive.org/web/20190227210126/https://tlsfingerprint.io/id/2dcbeba533890640)) rank 6272 |[comparison](https://tlsfingerprint.io/compare/bb94e801f7aee52b/2dcbeba533890640) ([archive](https://web.archive.org/web/20190227210435/https://tlsfingerprint.io/compare/bb94e801f7aee52b/2dcbeba533890640)) with ESR 60 rank 31 |
The differences against the currently ESR 60 fingerprint appear to be partly from the lack of plaintext SNI, and partly from unrelated TLS changes in Firefox.

Trac:
meek-esni.pcap

Calling this done because it works, and the rest is a matter of integration.

Trac:
Resolution: N/A to implemented
Status: new to closed

Trac:
meek-esni.2.pcap

Another packet capture of ESNI bootstrap, this time with security.OCSP.enabled=0.

I revised the instructions in comment:5 to add the pref security.OCSP.enabled=0, because OCSP can leak the server name even with DoH, ESNI, and TLS 1.3 are used. (This was never a problem with meek because any OCSP leaks would have been of the front domain, not the covert domain.)

closed

mentioned in issue #29611 (moved)

mentioned in issue #30095 (moved)

moved to tpo/anti-censorship/pluggable-transports/meek#28168 (closed)

mentioned in issue tpo/applications/tor-browser#29611 (closed)