Opened 4 years ago

Closed 4 years ago

#8860 closed project (fixed)

Registration over App Engine

Reported by: dcf Owned by: dcf
Priority: High Milestone:
Component: Archived/Flashproxy Version:
Severity: Keywords:
Cc: asn, ioerror Actual Points:
Parent ID: Points:
Reviewer: Sponsor:

Description

It apparently is possible to use Google App Engine apps if you can access https:‍www.google.com/. We can use this for rendezvous.

As an example of doing it manually, you can run flashproxy-reg-url and paste the URL you get into an existing proxy app like https://g-proxy.appspot.com/ or https://bingproxy.appspot.com/, and that is sufficient for rendezvous.

One way of doing it automatically with a custom App Engine app is to have the app figure out the client's IP address from the request, and insert it along with the client's given port number in a new registration to the facilitator. (This is pretty much what flashproxy-reg-http and facilitator.cgi do now, except it's like having facilitator.cgi run on a different host than the facilitator.) The downside of this approach is that the IP:port information becomes known to the app and to Google. (Though we can't hide the IP anyway, because it's part of the HTTP request to the app.)

A better way would be to have the app forward encrypted registration blobs, like Gmail does with the email rendezvous. The problem here is that the client needs to know its own IP address. I propose having the App Engine app interpret requests for /ip as a request for the requestor's IP address. It should return the IP address as a text/plain document in a single line. The other path pattern understood by the app will be /reg/<blob>, which it will simply forward by making a new HTTP request for https:‍fp-facilitator.org/<blob>.

Two parts to this project:

  1. App Engine app handling /ip and `/reg' as above.
  2. A client program flashproxy-reg-appspot. The client program makes a request for /ip to find out its IP, then generates a base64 blob from the IP and port, the same way flashproxy-reg-url does. It then makes a second request to /reg/<blob> to effect the registration. The App Engine app does nothing but a URL fetch of https:‍fp-facilitator.org/reg/<blob>. The client program should have -4 and -6 options.

Child Tickets

Attachments (7)

Change History (26)

comment:1 Changed 4 years ago by dcf

No shortage of existing App Engine–based proxies: http://en.cship.org/wiki/Google_App_Engine.

But we'll use a trick to talk to appspot so we don't need a variety of names nor to keep the name secret.

comment:2 Changed 4 years ago by dcf

Could prototype this without having to deploy to App Engine.

https://agentgatech.appspot.com/ "What is my IP?"

Then flashproxy-reg-url and pass the URL to one of the proxy apps above.

When we deploy our own App, we want it to forward requests only to the facilitator, and not be a general proxy.

comment:3 Changed 4 years ago by arlolra

  • Status changed from new to needs_revision

Attached a prototype as described above. Works with --register-methods=appspot

comment:4 Changed 4 years ago by arlolra

Building on yesterdays prototype, I wrote the App Engine app and deployed it to https://flashproxy-reg.appspot.com/

Please see the attached.

comment:5 Changed 4 years ago by dcf

Hot damn. Thank you for this.

The trick to accessing appspot domains is to make a request for https://www.google.com/, but override the Host header in the request:

Host: flashproxy-reg.appspot.com

That is, you make a connection to port 443 on whatever IP www.google.com resolves to, and start TLS as if you were talking to www.google.com. But in the HTTP request inside the TLS, you set the Host header to the host you want to talk to.

Can you see if it's easy to change the program to work this way? Maybe it's possible to override a header in urllib2.urlopen.

Changed 4 years ago by arlolra

comment:6 Changed 4 years ago by arlolra

  • Status changed from needs_revision to needs_review

No problem. See the attached patch.

comment:7 follow-ups: Changed 4 years ago by asn

Oh there is codez here. Nice!

Some comments on the code:

  • Exception handling in get_external_ip() is kind of crude. Also, its callers are not prepared for it returning None.
  • Can we use flashproxy's normal parse_addr_spec()?

BTW, while this script will work great for now, an adversary might be able to block it by detecting the SSL handshake of urllib2, in the same way as they currently do for Tor. Maybe the next step is to build a Firefox addon that does the registration through https://google.com?

comment:8 in reply to: ↑ 7 Changed 4 years ago by cypherpunks

Replying to asn:

codez

h1pster

comment:9 in reply to: ↑ 7 Changed 4 years ago by arlolra

Replying to asn:

Oh there is codez here. Nice!

Thanks for the comments.

Some comments on the code:

Are you sure? How else would I specify the request path?

  • Exception handling in get_external_ip() is kind of crude. Also, its callers are not prepared for it returning None.

I updated my branch here: https://github.com/arlolra/flashproxy/compare/master...appspot
to return an empty string, instead of None.

  • Can we use flashproxy's normal parse_addr_spec()?

Not sure what you mean. Most of flashproxy-reg-appspot is a copy of what was in flashproxy-reg-url.

BTW, while this script will work great for now, an adversary might be able to block it by detecting the SSL handshake of urllib2, in the same way as they currently do for Tor. Maybe the next step is to build a Firefox addon that does the registration through https://google.com?

Interesting idea.

comment:10 Changed 4 years ago by dcf

  • Status changed from needs_review to needs_information

I'm working on integrating this code. However it doesn't seem to work for me. I get "Thanks." from the flashproxy-reg.appspot.com, but I don't get a connection back, not even a record of a registration in the facilitator log. It also doesn't work if I copy the URL and paste it into a browser (thereby not using the Host header trick). Does it work for you?

Changed 4 years ago by arlolra

comment:11 Changed 4 years ago by arlolra

It works when I test with a local facilitator ... though I needed to apply this attached (0005) patch, to make facilitator.cgi work. Is the url register working for you?

comment:12 Changed 4 years ago by dcf

flashproxy-reg-url works for me when I paste the URL in a browser. On the facilitator, we do make install so flashproxy-reg is in the PATH.

I'm working on setting up my own App Engine account. Do you see any errors from flashproxy-reg.appspot.com, do you see the URLs being loaded?

Try testing with the public facilitator. Registration should be instant and you should see about 10 connection attempts to the address you register.

comment:13 Changed 4 years ago by arlolra

Is the facilitator receiving requests from app engine?

I'm on irc if you want to chat.

Changed 4 years ago by arlolra

comment:14 Changed 4 years ago by arlolra

  • Status changed from needs_information to needs_review

Figured it out. The change to google.com altered the URL.String. Switched to using URL.Path instead. See 0006.

comment:15 Changed 4 years ago by dcf

Excellent, working now. Thanks.

2013-05-18 21:45:39 Remote connection from [scrubbed].
2013-05-18 21:45:39 Data from WebSocket-pending [scrubbed].
2013-05-18 21:45:39 locals  (0): []
2013-05-18 21:45:39 remotes (1): ['[scrubbed]']

comment:16 Changed 4 years ago by dcf

  • Status changed from needs_review to needs_revision

I've merged this now into flashproxy master, and made a lot of changes. I'm working on getting a Google account set up, and then I'll probably make more changes and set up a new app.

Please made the appspot program put brackets around IPv6 addresses. Currently it's not working for IPv6:

2013-05-19 09:17:40 flashproxy-reg-appspot: Error parsing external IP address '2001:db8::1': invalid literal for int() with base 10: 'db8::1'
2013-05-19 09:17:40 flashproxy-reg-appspot exited with status 1.

Changed 4 years ago by arlolra

comment:17 Changed 4 years ago by arlolra

I added you as an administrator on flashproxy-reg.appspot.com. Also, see the attached patch. Looking into IPv6 issues.

comment:18 Changed 4 years ago by arlolra

  • Status changed from needs_revision to needs_review

Patched appspot for IPv6 and attached here.

comment:19 Changed 4 years ago by dcf

  • Resolution set to fixed
  • Status changed from needs_review to closed

Thanks. This was merged and modified and is the new default registration method.

I set up a new fp-reg-a.appspot.com app rather than use flashproxy-reg.appspot.com. I hope that's okay. My reason for that is so that I could use a Google account created only for the purpose of running the app, and nothing else.

Note: See TracTickets for help on using tickets.