Opened 8 months ago

Last modified 4 weeks ago

#29347 closed enhancement

Rewrite meek-http-helper as a WebExtension — at Version 6

Reported by: dcf Owned by: dcf
Priority: Medium Milestone:
Component: Circumvention/meek Version:
Severity: Normal Keywords: webextension
Cc: cohosh, brade, mcs, sukhbir, gk Actual Points:
Parent ID: Points:
Reviewer: Sponsor:

Description (last modified by dcf)

Firefox 60 ESR (the current basis of Tor Browser 8) officially doesn't support "legacy" browser extensions using XPCOM/XUL, only the newer WebExtension API.
https://www.mozilla.org/en-US/firefox/60.0esr/releasenotes/#changed
Tor Browser still includes some legacy extensions; apparently what makes them keep working is a extensions.legacy.exceptions pref (#26127; thanks sukhe for knowing that). I don't see where meek-http-helper@bamsoftware.com is being allowed (edit: probably a patch, thanks mcs), but somehow it is still working too.

Assess whether it's possible to rewrite the helper as a WebExtension, and do it if so. Ideally it will be possible to keep 100% compatibility with the current helper interface; but changing meek-client and meek-client-torbrowser is also an option.

Child Tickets

Change History (6)

comment:1 Changed 8 months ago by dcf

Description: modified (diff)

comment:2 Changed 8 months ago by dcf

The basic domain fronting operation seems to be possible. You can't override the Host header in a fetch or XMLHttpRequest because Host is a forbidden header name. I tried it, and any changes I made to Host were silently ignored. However you can set Host in webRequest.onBeforeSendHeaders, at least in Firefox 65, on which I was testing.

The following extension prints out the expected "I’m just a happy little web server." in the --jsconsole.

manifest.json

{
	"manifest_version": 2,
	"name": "Domain fronting demo",
	"version": "1.0",

	"background": {
		"scripts": ["main.js"]
	},

	"permissions": [
		"https://*/*",
		"webRequest",
		"webRequestBlocking"
	]
}

main.js

// https://developer.mozilla.org/en-US/docs/Mozilla/Add-ons/WebExtensions/API/webRequest/onBeforeSendHeaders
browser.webRequest.onBeforeSendHeaders.addListener(
    function(details) {
        let requestHeaders = details.requestHeaders.filter(
            h => h.name.toLowerCase() !== "host"
        );
        requestHeaders.push({name: "Host", value: "meek.azureedge.net"});
        return {requestHeaders: requestHeaders};
    },
    {"urls": ["*://*/*"]},
    ["blocking", "requestHeaders"]
);

// https://developer.mozilla.org/en-US/docs/Web/API/Fetch_API/Using_Fetch
let resp = fetch("https://ajax.aspnetcdn.com/")
    .then(resp => resp.text())
    .then(text => console.log(text));

It's a bit awkward because the listeners like onBeforeSendHeaders are global, not belonging to any single request. Ideally the extension will be able to handle different Host headers for different requests: we need a way to communicate from the outer code where we call fetch to the inner code where we modify the header. One way to do this would be to encode any per-request settings in a magic header or other metadata, which we strip and interpret in the callback. Another way would be to put a lock around the whole fetchonBeforeSendHeaders so that there can only be one in progress at a time, and use a global variable as shared memory. I don't think it would hurt performance because onBeforeSendHeaders is called before anything hits the network; i.e., it should happen almost immediately after fetch and then we can release the lock.

Last edited 8 months ago by dcf (previous) (diff)

comment:3 Changed 8 months ago by mcs

Cc: brade mcs added

comment:4 Changed 8 months ago by mcs

I think it is the following patch that allows meek-http-helper@bamsoftware.com to work in Tor Browser 8.x:
https://gitweb.torproject.org/tor-browser.git/commit/?h=tor-browser-60.5.0esr-8.5-1&id=10e590c79b01b2a30db1f01a2112c8808696a6cf

I might be pleasantly surprised, but I suspect it will be challenging to reimplement it as a Webextension while remaining compatible with the existing external interface. For example, I don't think there is an easy way to write to stdout. The Native Messaging API is another "tool" which you might need:
https://developer.mozilla.org/en-US/docs/Mozilla/Add-ons/WebExtensions/Native_messaging

I was hoping the new direction for the Tor Browser meek client would be to stop using a real browser (but that assumes the TLS fingerprint spoofing works well enough).

comment:5 in reply to:  4 Changed 8 months ago by dcf

Replying to mcs:

I might be pleasantly surprised, but I suspect it will be challenging to reimplement it as a Webextension while remaining compatible with the existing external interface. For example, I don't think there is an easy way to write to stdout.

I haven't tried it all yet, but I've sketched it out mentally and I haven't found any showstoppers yet. (I found and appreciated you catalog of API differences at #17248.) window.dump still works for writing to stdout, as long as the pref browser.dom.window.dump.enabled is set. I'm not hung up on 100% compatibility, but fundamentally we need some way for meek-client to send specifications of HTTP requests into the browser and receive responses (currently we have the browser open a local TCP socket), so I think you're right, we will need the native messaging API. Instead of having the browser open its own socket, it will spawn a subprocess to open the socket and then pass information to/from the subprocess. This is the process tree I envision:

tor
└─meek-client-torbrowser
  ├─firefox --headless
  │ └─socket_shim (opens a socket on 127.0.0.1:XXXX)
  └─meek-client --helper 127.0.0.1:XXXX

meek-client and what I've here called socket_shim communicate over a local TCP socket. The socket_shim can communicate its unpredictable port number XXXX upward through the same native messaging API connection that it uses to exchange HTTP requests/responses.

I was hoping the new direction for the Tor Browser meek client would be to stop using a real browser (but that assumes the TLS fingerprint spoofing works well enough).

That is the plan—there's a near-merge-ready branch at comment:15:ticket:29077 that, if you have time, I'd appreciate you having a look at. Using obfs4proxy meek_lite is also an option, since obfs4proxy also integrated uTLS.

The headless Firefox WebExtension still has value for ESNI. Firefox has an ESNI implementation and Go doesn't. That's what led me to make this ticket—I wanted to try meek-client with ESNI in place of domain fronting, and found that a Firefox that is new enough to support ESNI is also too new to support the current browser extension 😅

comment:6 Changed 8 months ago by dcf

Description: modified (diff)
Note: See TracTickets for help on using tickets.