Implement backend for moat

added actualpoints::5 bridgedb-captcha component::circumvention/bridgedb owner::isis points::3 priority::high resolution::fixed severity::normal sponsor::M status::closed type::enhancement labels

Trac:
Cc: N/A to brade, mcs

A first draft spec of the API is here. Please review! I'm finishing up the implementation of #15967 (moved) and then I'll get starting on implementing this spec.

Trac:
Cc: brade, mcs to brade, mcs, iry

Hi isis!

I am posting the reply in this ticket since it seems to be more related to the topic:

isis:

This API won't be publicly accessible though, it'll be reachable through the API for #22871 (moved), and even then it's only reachable through a special meek reflector as part of #16650 (moved). I love the idea to "Set up domain fronting for BridgeDB:. The benefits are huge as described in #16650 (moved).

However, meek has not been supported neither by Whonix nor by Tails so far. It is very likely because meek has not been packaged in to Debian as a standalone client because of its increasingly high-coupling with TBB: https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=764007

I will also ask Tails about why meek is not available in Tails, given that Tails does ship a Tor Browser (unlike Whonix-gateway).

Is anon-connection-wizard what Tails uses now? I'd be happy to support Tails as well (but I'd strongly prefer the connection to go through the meek reflector).

anon-connection-wizard has not been used by Tails so far. But some quick and dirty test on integrating anon-connection-wizard has been done by anonym from Tails. Some details can be found here: https://mailman.boum.org/pipermail/tails-dev/2017-September/011638.html

Replying to isis:

A first draft spec of the API is here. Please review! I'm finishing up the implementation of #15967 (moved) and then I'll get starting on implementing this spec.

This looks pretty good. Kathy isn't here right now, but I will ask her to take a look later. I have just a couple of comments:

I am not a fan of the verbose type strings, e.g., moat client supported transports. Maybe this is a JSON-API style thing, but I would rather use shorter, easier to compare strings such as 'type': 'client-transports'.
(you asked about this on IRC) The error response seems okay, although if there are distinct reasons for failure that would be helpful for end-users to know, please add enumerated error codes that Tor Launcher can use to pull out a localized string. The way things are now, Tor Launcher will probably not use the title or details field in the UI; maybe just log them (because we don't want to show English text to people who speak other languages).

p.s. I like the 419 status string :)

Replying to mcs:

Replying to isis:

A first draft spec of the API is here. Please review! I'm finishing up the implementation of #15967 (moved) and then I'll get starting on implementing this spec.

This looks pretty good. Kathy isn't here right now, but I will ask her to take a look later. I have just a couple of comments:

I am not a fan of the verbose type strings, e.g., moat client supported transports. Maybe this is a JSON-API style thing, but I would rather use shorter, easier to compare strings such as 'type': 'client-transports'.

Sure, I can go shorter with hyphenations. I honestly didn't know what to put there, so I just made stuff up. I'll update it.

(you asked about this on IRC) The error response seems okay, although if there are distinct reasons for failure that would be helpful for end-users to know, please add enumerated error codes that Tor Launcher can use to pull out a localized string. The way things are now, Tor Launcher will probably not use the title or details field in the UI; maybe just log them (because we don't want to show English text to people who speak other languages).

That makes sense, I'll keep the spec updated and use different error codes if I think of any cases where it might make sense to localise. (As of right now it seems more useful to log these kinds of errors on the BridgeDB server and in the browser console.)

p.s. I like the 419 status string :)

Yeah! It's pretty great. It was nickm's idea.

Replying to iry:

Hi isis!

I am posting the reply in this ticket since it seems to be more related to the topic:

isis:

This API won't be publicly accessible though, it'll be reachable through the API for #22871 (moved), and even then it's only reachable through a special meek reflector as part of #16650 (moved). I love the idea to "Set up domain fronting for BridgeDB:. The benefits are huge as described in #16650 (moved).

However, meek has not been supported neither by Whonix nor by Tails so far. It is very likely because meek has not been packaged in to Debian as a standalone client because of its increasingly high-coupling with TBB: https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=764007

That makes sense, although it's unfortunate. There is a meek-client program included in meek, however, as I understand it, the TLS is more fingerprintable which is why dcf went the route of instrumenting a browser. It would be better to ask dcf about this.

I also just remembered that you actually can't do a POST /meek/* to BridgeDB unless you go through the meek reflector, because of the way the TLS termination is handled. Also FYI, this distributor relies on getting the client's IP address in an X-Forwarded-For header from the meek reflector. We could consider setting up the same moat API as its own separate distributor for clients which can't use meek, but that should be a new ticket. (Also, I'd prefer that they be separate distributors, since there's a possibility that we may need to allocate differently, or treat different automated bridge distribution clients differently, e.g. different rate limiting, in the future.)

I will also ask Tails about why meek is not available in Tails, given that Tails does ship a Tor Browser (unlike Whonix-gateway).

Thanks! I'd be curious to hear why.

Is anon-connection-wizard what Tails uses now? I'd be happy to support Tails as well (but I'd strongly prefer the connection to go through the meek reflector).

anon-connection-wizard has not been used by Tails so far. But some quick and dirty test on integrating anon-connection-wizard has been done by anonym from Tails. Some details can be found here: https://mailman.boum.org/pipermail/tails-dev/2017-September/011638.html

That's great! Is it being considered because Firefox is removing support for extensions? (Wasn't Tails doing something special to run Tor Launcher as a desktop app?)

Done in my fix/22871_r1 branch. I'll merge and deploy it so that the TB team can test it.

Trac:
Actualpoints: N/A to 5
Status: new to merge_ready
Sponsor: N/A to SponsorM
Keywords: SponsorM deleted, N/A added

This was deployed last week but there's some issues with the all the tunneling through meek and apache that still need to be fixed.

Trac:
Resolution: N/A to fixed
Status: merge_ready to closed

Followup tickets are #24432 (moved) #24433 (moved) #24443 (moved)

closed

changed time estimate to 24h

added 40h of time spent

mentioned in issue #23136 (moved)

mentioned in issue tpo/applications/tor-launcher#23136 (closed)

moved to tpo/anti-censorship/bridgedb#22871 (closed)

Implement backend for moat

Child items ...

Activity