Node.js standalone flash proxy

On the tor-talk mailing list the idea was developed to have a standalone flash proxy that does not require a browser (and thus a display) to run in. Useful to run 24/7 on/behind home router and the like, esp. where the address is changing daily.

David proposed using Rhino; there seem to be websocket clients for node.js as well. (How similar to a browser does this need to be?)

added component::archived/flashproxy owner::dcf priority::medium resolution::fixed status::closed type::enhancement labels

Replying to akrey:

On the tor-talk mailing list the idea was developed to have a standalone flash proxy that does not require a browser (and thus a display) to run in. Useful to run 24/7 on/behind home router and the like, esp. where the address is changing daily.

Great idea!

David proposed using Rhino; there seem to be websocket clients for node.js as well. (How similar to a browser does this need to be?)

My first thought is "hopefully not that similar to a browser at all" -- I can see porting the javascript browser stuff at first, but it seems to me that the protocol should be simple enough that a python script by itself should do it. Then we could imagine volunteers doing an apt-get install of the deb, and suddenly they're helping glue Tor users together, even behind nat, without ever having to think about it again. Extra points that these flash proxies would be way more stable than a typical browser tab.

Trac:
Cc: N/A to bastik.public@gmx-topmail.de

This is terrific. Please let it require zero required configuration.

If you can dumb it down to:

# Linux
apt-get install flashbridge

# Windows
download, install, done

I could imagine, that many people would install it to volunteer a brdige, who didn't have the time or skills to set up port forwarding.

Trac:
Cc: bastik.public@gmx-topmail.de to bastik.public@gmx-topmail.de, adrelanos@riseup.net

Replying to arma:

My first thought is "hopefully not that similar to a browser at all" -- I can see porting the javascript browser stuff at first, but it seems to me that the protocol should be simple enough that a python script by itself should do it. Then we could imagine volunteers doing an apt-get install of the deb, and suddenly they're helping glue Tor users together, even behind nat, without ever having to think about it again. Extra points that these flash proxies would be way more stable than a typical browser tab.

Hey speaking of which: in the Flashproxy model, the Tor bridge needs some glue to pretend to be a webserver, since the Flash Proxy is only allowed to talk certain protocols.

This outside-the-browser standalone Flash proxy wouldn't have such a requirement -- does that mean it could (should) just interface with normal Tor bridges directly?

Replying to arma:

Hey speaking of which: in the Flashproxy model, the Tor bridge needs some glue to pretend to be a webserver, since the Flash Proxy is only allowed to talk certain protocols.

Yes, this glue is [websocket-transport https://gitweb.torproject.org/flashproxy.git/tree/HEAD:/websocket-transport] and there is only one bridge I'm aware of running it (mine, tor1.bamsoftware.com). Letting the facilitator cope with more is #7945 (closed).

This outside-the-browser standalone Flash proxy wouldn't have such a requirement -- does that mean it could (should) just interface with normal Tor bridges directly?

Yes, correct, which is potentially very exciting for ideas like #7167 (moved), combining obfsproxy with flash proxy, because it allows us to escape the outer (fingerprintable) layer of WebSocket. There's no reason why a client couldn't speak obfs2, for example, through a standalone flash proxy, to any old obfs2 bridge.

Check comment:2:ticket:5578 regarding WebRTC, where I proposed having client registrations inform the facilitator of the types of connections they want. Currently we assume that all communication is WebSocket, both client–proxy and proxy–relay. It gets more complicated when client–proxy may be WebSocket, WebRTC, or raw TCP; and proxy–relay may be WebSocket, raw TCP, obfs2-in-WebSocket, or obfs2-in-raw TCP. E.g., a standalone proxy doesn't have to care whether it is carrying plain Tor or obfs2 in terms of making a connection, but it has to know which protocol the client plans to tunnel in order to connect to a bridge of the appropriate type.

Replying to dcf: ...

This outside-the-browser standalone Flash proxy wouldn't have such a requirement -- does that mean it could (should) just interface with normal Tor bridges directly?

Yes, correct,

...or have the bridge locally. (I assume running rogue flashproxies is not a valid attack (on the content) as the tor client will verify the bridge and the network behind it.)

Check comment:2:ticket:5578 regarding WebRTC, where I proposed having client registrations inform the facilitator of the types of connections they want.

Good to know. I was thinking - for this standalone proxy the webbrowser-available-communication restriction no longer applies, so we could use other protocols as well (including UDP).

The main advantage of flashproxy that remains then is dynamic nature of discovery via the facilitator (and the reverse connection, o/c), and thus making me able to use my home DSL (which changes IP every day). Bridges need manual work every time that happens on the client side.

Here's an in-progress patch (attached to this bug) that I'm working on to allow node.js to run a flashproxy. It's not working yet; the commit message has more details.

Trac:
0001-flashproxy.js-Begin-support-for-running-under-node.j.patch

cjb, are you making progress? Do you need help or could it be integrated?

I rebased cjb's work onto the HEAD of master and got this working. Also added a package.json file to simplify the npm install and fixed a small bug in the facilitator. This work can be found on the standalone branch of my github repo,

https://github.com/arlolra/flashproxy/tree/standalone

Replying to arlolra:

I rebased cjb's work onto the HEAD of master and got this working. Also added a package.json file to simplify the npm install and fixed a small bug in the facilitator. This work can be found on the standalone branch of my github repo,

https://github.com/arlolra/flashproxy/tree/standalone

Thank you for doing this. For reference below I'm talking about https://github.com/arlolra/flashproxy/commit/3c08ad9c9ea506dfaf158d41128fefa5d6bca39e.

Do you think you could rebase your changes into logical commits? For example, each bug fix should be its own commit, and the addition of package.json needs to be its own commit. Each commit needs to have an informative log message.

Please undo any non-functional changes, like at https://github.com/arlolra/flashproxy/commit/3c08ad9c9ea506dfaf158d41128fefa5d6bca39e#L1L806 where it's hard to tell if you only changed whitespace or if there is something different in the code.

Trac:
Status: new to needs_revision

Is this better? (new branch, standalone2)

https://github.com/arlolra/flashproxy/tree/standalone2

Trac:
Status: needs_revision to needs_review

Cool, thanks for doing this!

I just tried running it, but it looks like the websocket connections aren't succeeding:

Facilitator: got client:{ host: "xx.xx.xx.xx", port: 9002 } relay:{ host: "xx.xx.xx.xx", port: 9901 }. Client: connecting. Relay: connecting. Client: connecting. Relay: connecting.

I'm not seeing anything move to e.g. "Relay: connected.", even though I'm sure (because I can connect to it in another terminal) that the relay host:port is up. Did it work okay for you?

Trac:
Cc: bastik.public@gmx-topmail.de, adrelanos@riseup.net to bastik.public@gmx-topmail.de, adrelanos@riseup.net, cjb

Oh, it's probably because those messages have moved inside if (BROWSER):

    this.onopen_callback = function(event) {
        if (BROWSER) {
            var ws = event.target;
            log(ws.label + ": connected.");
        }
    }.bind(this);

That would be it. See: https://github.com/arlolra/flashproxy/commit/7dab216dce629c0b85f6e72caadbdc5530f8d540

I was able to connect with it through the TBB so I'm confident it does relay traffic.

I created a node.js branch to start tracking and integrating your changes.

https://gitweb.torproject.org/flashproxy.git/shortlog/refs/heads/node.js

About the changes generally, I would prefer that there be fewer if (BROWSER) blocks. There are too many now for this code to be merged. Ideally, all the node.js-specific stuff will be in one block at the top of the file, and other parts of the code will use abstractions set up by that block.

In https://github.com/arlolra/flashproxy/commit/7dab216dce629c0b85f6e72caadbdc5530f8d540, which removes use of the event parameter in callbacks, a better thing to test is if (event) rather than if (!BROWSER). The functions should still print something even if they don't have event details, to prevent confusion as in comment:13.

Even having to check event seems like a bug in the WebSocket library. My preferred solution here is to have the bug reported upstream and not to work around it here. The lack of bufferedAmount in https://github.com/arlolra/flashproxy/commit/4110c0f96273259b162d33ee3a7ed7542d0294c6 also seems like a bug, and ignoring it means we potentially use unbounded memory. The need to set { binary: true } looks like another bug. We could build an abstraction to hide the differences, but I don't want a wrapper function doing something that should be fundamental like send. Better to get these bugs fixed.

In summary: please report what look like bugs to the authors of the WebSocket library, and try to move the if (BROWSER) blocks into one place.

Trac:
Status: needs_review to needs_revision

Definitely agree with removing all the ifdefs.

I'm trying to get this patch landed upstream, https://github.com/einaros/ws/pull/168/files

Trac:
Cc: bastik.public@gmx-topmail.de, adrelanos@riseup.net, cjb to bastik.public@gmx-topmail.de, adrelanos@riseup.net, chris@printf.net

Still waiting on upstream to merge. In the meantime, I've moved most of the workarounds to the top, removing a bunch of ifdefs. Have a look,

https://github.com/arlolra/flashproxy/compare/standalone3

And this (uses the fork of ws below ... npm install to test), https://github.com/arlolra/flashproxy/compare/standalone4

is what it'll look like if and when these changes are pulled upstream, https://github.com/arlolra/ws/compare/workarounds

Ok, ignore my last two posts. The changes were merged upstream. I moved everything on top of the tracking branch you started. Definitely looks cleaner now. Enjoy!

https://github.com/arlolra/flashproxy/compare/node.js

Trac:
Status: needs_revision to needs_review

I rebased and squashed your latest node.js branch and pushed it to https://gitweb.torproject.org/user/dcf/flashproxy.git/shortlog/refs/heads/node.js (note it's under the /user/dcf directory this time).

Please don't merge from master in feature branches like this one, if you can avoid it. I would much rather have you rebase onto master, in the same branch or in a new branch. You should rebase to get rid of commits that do something and then later commits that revert them. In these branches, I'm not interested in history, I want something clean that I can apply to master. See comment:5:ticket:7751 for a example of asn making such a rebased "take2" branch.

Don't make changes like these that only change whitespace: https://github.com/arlolra/flashproxy/compare/node.js#L0L424 https://github.com/arlolra/flashproxy/compare/node.js#L0R445 https://github.com/arlolra/flashproxy/compare/node.js#L0L676 If you notice changes like that in the diff, you need to rebase and get rid of them.

I don't understand this commit: https://github.com/arlolra/flashproxy/commit/dbb8dec3c952983ef903fa3fd4e0c6859855bae6 What is going on there? If it's necessary, you need to find another way to do it. Having this depend on who calls make_websocket is too magical.

Is it still the case that have_websocket_binary_frames should return false for node.js?

Could you add a document under doc that explains how to check out and run the standalone proxy? Then we can ask for testers.

Trac:
Status: needs_review to needs_revision

How about this? https://github.com/arlolra/flashproxy/compare/master...nodejs Should be much cleaner to apply and maintain.

The branch is here: https://github.com/arlolra/flashproxy/tree/nodejs

For some output, you'll want to run:

node flashproxy.js "debug=true"

I still need to write up a doc, but the commit message says it all.

Also, while waiting for this to land, https://github.com/einaros/ws/pull/202

The package.json points to my fork, git://github.com/arlolra/ws#target

Trac:
Status: needs_revision to needs_review

Note that I removed ALL node.js related code from flashproxy.js and moved it to a wrapper flashproxy-node.js.

I also published it to the npm repository to make it easier for testers. All that's required to try it is,

npm install -g flashproxy
flashproxy

The necessary changes to ws landed upstream, https://github.com/einaros/ws/pull/202

Replying to arlolra:

How about this? https://github.com/arlolra/flashproxy/compare/master...nodejs Should be much cleaner to apply and maintain.

It's looking good. I will comment more later.

Can you explain this code? What gets modified otherwise?

// Include flashproxy.js using eval to avoid modifying it.
var file = path.join(__dirname, "flashproxy.js");
var data = fs.readFileSync(file, "utf8");
eval(data);

Can you explain this code?

Node.js doesn't have the equivalent of Rhino's load(), where code gets included in the calling script. It does have require() but that follows the CommonJS pattern of explicitly exporting a module.

Rather than make any changes to flashproxy.js, we just read the file and evaluate it.

What gets modified otherwise?

All the above environment variables in flashproxy-node.js could be conditionally defined in flashproxy.js at the top, but I thought it cleaner to separate the code.

I've merged parts of this and am getting closer to having it all merged.

What do you think about this new patch series? Does it work? I've attached it and it's also the top six commits at https://gitweb.torproject.org/user/dcf/flashproxy.git/shortlog/refs/heads/node.js. The remaining change to flashproxy.js is quite small, just the definition of a new HEADLESS variable.

We still need documentation that is not just part of a commit message. It needs to be something we can update and link people to. Please add a README file stating how to run the standalone proxy. This should include instructions both for running from a source directory (node flashproxy-node.js) and the npm instructions.

Is it possible to mode the Node.js code to a new modules/nodejs directory?

Trac:
Status: needs_review to needs_information

Trac:
nodejs-mkii.patch

Node patch using HEADLESS.

Seems fine to me. A few small problems:

• The version number in package.json needs to follow semver http://semver.org/ ... so I added back the PATCH 0.
• The cookies var was dependent on document.cookies. Small workaround for that.
• Moving to modules/nodejs requires symlinking the flashproxy.js file and a prepublish script for npm
• Can't leave a trailing slash in JSON or it won't parse (the package.json that is)
• I added a README

Here's a patch for the above building off of dcf/flashproxy.git#node.js,

https://github.com/arlolra/flashproxy/commit/a8b22373fe21032eef507ab643e92ab5e16c2cc9

Trac:
Status: needs_information to needs_revision

Same thing, but switched to using v0.4.27 of ws in the package.json and squashed https://github.com/arlolra/flashproxy/commit/d3e90c8c227dc0267887b60378f0119f7d135d14

I've committed what seems to be a working version in 0506689b.

Until we have a way to keep the version number in package.json synchronized, I made it independent.

Trac:
Summary: Standalone flash proxy to Node.js standalone flash proxy
Status: needs_revision to closed
Resolution: N/A to fixed

closed

mentioned in issue #8092 (closed)