Opened 5 years ago

Closed 4 years ago

#7944 closed enhancement (fixed)

Node.js standalone flash proxy

Reported by: akrey Owned by: dcf
Priority: Medium Milestone:
Component: Archived/Flashproxy Version:
Severity: Keywords:
Cc: bastik.public@…, adrelanos@…, chris@… Actual Points:
Parent ID: Points:
Reviewer: Sponsor:

Description

On the tor-talk mailing list the idea was developed to have a standalone flash proxy that does not require a browser (and thus a display) to run in. Useful to run 24/7 on/behind home router and the like, esp. where the address is changing daily.

David proposed using Rhino; there seem to be websocket clients for node.js as well. (How similar to a browser does this need to be?)

Child Tickets

Attachments (2)

0001-flashproxy.js-Begin-support-for-running-under-node.j.patch (3.8 KB) - added by cjb 5 years ago.
nodejs-mkii.patch (7.1 KB) - added by dcf 4 years ago.
Node patch using HEADLESS.

Download all attachments as: .zip

Change History (33)

comment:1 in reply to:  description ; Changed 5 years ago by arma

Replying to akrey:

On the tor-talk mailing list the idea was developed to have a standalone flash proxy that does not require a browser (and thus a display) to run in. Useful to run 24/7 on/behind home router and the like, esp. where the address is changing daily.

Great idea!

David proposed using Rhino; there seem to be websocket clients for node.js as well. (How similar to a browser does this need to be?)

My first thought is "hopefully not that similar to a browser at all" -- I can see porting the javascript browser stuff at first, but it seems to me that the protocol should be simple enough that a python script by itself should do it. Then we could imagine volunteers doing an apt-get install of the deb, and suddenly they're helping glue Tor users together, even behind nat, without ever having to think about it again. Extra points that these flash proxies would be way more stable than a typical browser tab.

comment:2 Changed 5 years ago by bastik

Cc: bastik.public@… added

comment:3 Changed 5 years ago by proper

Cc: adrelanos@… added

This is terrific. Please let it require zero required configuration.

If you can dumb it down to:

# Linux
apt-get install flashbridge

# Windows
download, install, done

I could imagine, that many people would install it to volunteer a brdige, who didn't have the time or skills to set up port forwarding.

comment:4 in reply to:  1 ; Changed 5 years ago by arma

Replying to arma:

My first thought is "hopefully not that similar to a browser at all" -- I can see porting the javascript browser stuff at first, but it seems to me that the protocol should be simple enough that a python script by itself should do it. Then we could imagine volunteers doing an apt-get install of the deb, and suddenly they're helping glue Tor users together, even behind nat, without ever having to think about it again. Extra points that these flash proxies would be way more stable than a typical browser tab.

Hey speaking of which: in the Flashproxy model, the Tor bridge needs some glue to pretend to be a webserver, since the Flash Proxy is only allowed to talk certain protocols.

This outside-the-browser standalone Flash proxy wouldn't have such a requirement -- does that mean it could (should) just interface with normal Tor bridges directly?

comment:5 in reply to:  4 ; Changed 5 years ago by dcf

Replying to arma:

Hey speaking of which: in the Flashproxy model, the Tor bridge needs some glue to pretend to be a webserver, since the Flash Proxy is only allowed to talk certain protocols.

Yes, this glue is [websocket-transport https://gitweb.torproject.org/flashproxy.git/tree/HEAD:/websocket-transport] and there is only one bridge I'm aware of running it (mine, tor1.bamsoftware.com). Letting the facilitator cope with more is #7945.

This outside-the-browser standalone Flash proxy wouldn't have such a requirement -- does that mean it could (should) just interface with normal Tor bridges directly?

Yes, correct, which is potentially very exciting for ideas like #7167, combining obfsproxy with flash proxy, because it allows us to escape the outer (fingerprintable) layer of WebSocket. There's no reason why a client couldn't speak obfs2, for example, through a standalone flash proxy, to any old obfs2 bridge.

Check comment:2:ticket:5578 regarding WebRTC, where I proposed having client registrations inform the facilitator of the types of connections they want. Currently we assume that all communication is WebSocket, both client–proxy and proxy–relay. It gets more complicated when client–proxy may be WebSocket, WebRTC, or raw TCP; and proxy–relay may be WebSocket, raw TCP, obfs2-in-WebSocket, or obfs2-in-raw TCP. E.g., a standalone proxy doesn't have to care whether it is carrying plain Tor or obfs2 in terms of making a connection, but it has to know which protocol the client plans to tunnel in order to connect to a bridge of the appropriate type.

comment:6 in reply to:  5 Changed 5 years ago by akrey

Replying to dcf:
...

This outside-the-browser standalone Flash proxy wouldn't have such a requirement -- does that mean it could (should) just interface with normal Tor bridges directly?

Yes, correct,

...or have the bridge locally. (I assume running rogue flashproxies is not a valid attack (on the content) as the tor client will verify the bridge and the network behind it.)

Check comment:2:ticket:5578 regarding WebRTC, where I proposed having client registrations inform the facilitator of the types of connections they want.

Good to know. I was thinking - for this standalone proxy the webbrowser-available-communication restriction no longer applies, so we could use other protocols as well (including UDP).

The main advantage of flashproxy that remains then is dynamic nature of discovery via the facilitator (and the reverse connection, o/c), and thus making me able to use my home DSL (which changes IP every day). Bridges need manual work every time that happens on the client side.

comment:7 Changed 5 years ago by cjb

Here's an in-progress patch (attached to this bug) that I'm working on to allow node.js to run a flashproxy. It's not working yet; the commit message has more details.

comment:8 Changed 5 years ago by cypherpunks

cjb, are you making progress? Do you need help or could it be integrated?

comment:9 Changed 5 years ago by arlolra

I rebased cjb's work onto the HEAD of master and got this working. Also added a package.json file to simplify the npm install and fixed a small bug in the facilitator. This work can be found on the standalone branch of my github repo,

https://github.com/arlolra/flashproxy/tree/standalone

comment:10 in reply to:  9 Changed 5 years ago by dcf

Status: newneeds_revision

Replying to arlolra:

I rebased cjb's work onto the HEAD of master and got this working. Also added a package.json file to simplify the npm install and fixed a small bug in the facilitator. This work can be found on the standalone branch of my github repo,

https://github.com/arlolra/flashproxy/tree/standalone

Thank you for doing this. For reference below I'm talking about https://github.com/arlolra/flashproxy/commit/3c08ad9c9ea506dfaf158d41128fefa5d6bca39e.

Do you think you could rebase your changes into logical commits? For example, each bug fix should be its own commit, and the addition of package.json needs to be its own commit. Each commit needs to have an informative log message.

Please undo any non-functional changes, like at https://github.com/arlolra/flashproxy/commit/3c08ad9c9ea506dfaf158d41128fefa5d6bca39e#L1L806 where it's hard to tell if you only changed whitespace or if there is something different in the code.

comment:11 Changed 5 years ago by arlolra

Status: needs_revisionneeds_review

Is this better? (new branch, standalone2)

https://github.com/arlolra/flashproxy/tree/standalone2

comment:12 Changed 5 years ago by cjb

Cc: cjb added

Cool, thanks for doing this!

I just tried running it, but it looks like the websocket connections aren't succeeding:

Facilitator: got client:{ host: "xx.xx.xx.xx", port: 9002 } relay:{ host: "xx.xx.xx.xx", port: 9901 }.
Client: connecting.
Relay: connecting.
Client: connecting.
Relay: connecting.

I'm not seeing anything move to e.g. "Relay: connected.", even though I'm sure (because I can connect to it in another terminal) that the relay host:port is up. Did it work okay for you?

comment:13 Changed 5 years ago by cjb

Oh, it's probably because those messages have moved inside if (BROWSER):

    this.onopen_callback = function(event) {
        if (BROWSER) {
            var ws = event.target;
            log(ws.label + ": connected.");
        }
    }.bind(this);

comment:14 Changed 5 years ago by arlolra

That would be it. See: https://github.com/arlolra/flashproxy/commit/7dab216dce629c0b85f6e72caadbdc5530f8d540

I was able to connect with it through the TBB so I'm confident it does relay traffic.

comment:15 Changed 5 years ago by dcf

Status: needs_reviewneeds_revision

I created a node.js branch to start tracking and integrating your changes.

https://gitweb.torproject.org/flashproxy.git/shortlog/refs/heads/node.js

About the changes generally, I would prefer that there be fewer if (BROWSER) blocks. There are too many now for this code to be merged. Ideally, all the node.js-specific stuff will be in one block at the top of the file, and other parts of the code will use abstractions set up by that block.

In https://github.com/arlolra/flashproxy/commit/7dab216dce629c0b85f6e72caadbdc5530f8d540, which removes use of the event parameter in callbacks, a better thing to test is if (event) rather than if (!BROWSER). The functions should still print something even if they don't have event details, to prevent confusion as in comment:13.

Even having to check event seems like a bug in the WebSocket library. My preferred solution here is to have the bug reported upstream and not to work around it here. The lack of bufferedAmount in https://github.com/arlolra/flashproxy/commit/4110c0f96273259b162d33ee3a7ed7542d0294c6 also seems like a bug, and ignoring it means we potentially use unbounded memory. The need to set { binary: true } looks like another bug. We could build an abstraction to hide the differences, but I don't want a wrapper function doing something that should be fundamental like send. Better to get these bugs fixed.

In summary: please report what look like bugs to the authors of the WebSocket library, and try to move the if (BROWSER) blocks into one place.

comment:16 Changed 5 years ago by arlolra

Definitely agree with removing all the ifdefs.

I'm trying to get this patch landed upstream,
https://github.com/einaros/ws/pull/168/files

comment:17 Changed 5 years ago by cjb

Cc: chris@… added; cjb removed

comment:18 Changed 5 years ago by arlolra

Still waiting on upstream to merge. In the meantime, I've moved most of the workarounds to the top, removing a bunch of ifdefs. Have a look,

https://github.com/arlolra/flashproxy/compare/standalone3

comment:19 Changed 5 years ago by arlolra

And this (uses the fork of ws below ... npm install to test),
https://github.com/arlolra/flashproxy/compare/standalone4

is what it'll look like if and when these changes are pulled upstream,
https://github.com/arlolra/ws/compare/workarounds

comment:20 Changed 5 years ago by arlolra

Ok, ignore my last two posts. The changes were merged upstream. I moved everything on top of the tracking branch you started. Definitely looks cleaner now. Enjoy!

https://github.com/arlolra/flashproxy/compare/node.js

comment:21 Changed 5 years ago by arlolra

Status: needs_revisionneeds_review

comment:22 Changed 4 years ago by dcf

Status: needs_reviewneeds_revision

I rebased and squashed your latest node.js branch and pushed it to https://gitweb.torproject.org/user/dcf/flashproxy.git/shortlog/refs/heads/node.js (note it's under the /user/dcf directory this time).

Please don't merge from master in feature branches like this one, if you can avoid it. I would much rather have you rebase onto master, in the same branch or in a new branch. You should rebase to get rid of commits that do something and then later commits that revert them. In these branches, I'm not interested in history, I want something clean that I can apply to master. See comment:5:ticket:7751 for a example of asn making such a rebased "take2" branch.

Don't make changes like these that only change whitespace:
https://github.com/arlolra/flashproxy/compare/node.js#L0L424
https://github.com/arlolra/flashproxy/compare/node.js#L0R445
https://github.com/arlolra/flashproxy/compare/node.js#L0L676
If you notice changes like that in the diff, you need to rebase and get rid of them.

I don't understand this commit:
https://github.com/arlolra/flashproxy/commit/dbb8dec3c952983ef903fa3fd4e0c6859855bae6
What is going on there? If it's necessary, you need to find another way to do it. Having this depend on who calls make_websocket is too magical.

Is it still the case that have_websocket_binary_frames should return false for node.js?

Could you add a document under doc that explains how to check out and run the standalone proxy? Then we can ask for testers.

comment:23 Changed 4 years ago by arlolra

Status: needs_revisionneeds_review

How about this? https://github.com/arlolra/flashproxy/compare/master...nodejs
Should be much cleaner to apply and maintain.

The branch is here:
https://github.com/arlolra/flashproxy/tree/nodejs

For some output, you'll want to run:

node flashproxy.js "debug=true"

I still need to write up a doc, but the commit message says it all.

Also, while waiting for this to land,
https://github.com/einaros/ws/pull/202

The package.json points to my fork,
git://github.com/arlolra/ws#target

comment:24 Changed 4 years ago by arlolra

Note that I removed ALL node.js related code from flashproxy.js and moved it to a wrapper flashproxy-node.js.

I also published it to the npm repository to make it easier for testers. All that's required to try it is,

npm install -g flashproxy
flashproxy

comment:25 Changed 4 years ago by arlolra

The necessary changes to ws landed upstream,
https://github.com/einaros/ws/pull/202

comment:26 in reply to:  23 ; Changed 4 years ago by dcf

Replying to arlolra:

How about this? https://github.com/arlolra/flashproxy/compare/master...nodejs
Should be much cleaner to apply and maintain.

It's looking good. I will comment more later.

Can you explain this code? What gets modified otherwise?

// Include flashproxy.js using eval to avoid modifying it.
var file = path.join(__dirname, "flashproxy.js");
var data = fs.readFileSync(file, "utf8");
eval(data);

comment:27 in reply to:  26 Changed 4 years ago by arlolra

Can you explain this code?

Node.js doesn't have the equivalent of Rhino's load(), where code gets included in the calling script. It does have require() but that follows the CommonJS pattern of explicitly exporting a module.

Rather than make any changes to flashproxy.js, we just read the file and evaluate it.

What gets modified otherwise?

All the above environment variables in flashproxy-node.js could be conditionally defined in flashproxy.js at the top, but I thought it cleaner to separate the code.

comment:28 Changed 4 years ago by dcf

Status: needs_reviewneeds_information

I've merged parts of this and am getting closer to having it all merged.

What do you think about this new patch series? Does it work? I've attached it and it's also the top six commits at https://gitweb.torproject.org/user/dcf/flashproxy.git/shortlog/refs/heads/node.js. The remaining change to flashproxy.js is quite small, just the definition of a new HEADLESS variable.

We still need documentation that is not just part of a commit message. It needs to be something we can update and link people to. Please add a README file stating how to run the standalone proxy. This should include instructions both for running from a source directory (node flashproxy-node.js) and the npm instructions.

Is it possible to mode the Node.js code to a new modules/nodejs directory?

Changed 4 years ago by dcf

Attachment: nodejs-mkii.patch added

Node patch using HEADLESS.

comment:29 Changed 4 years ago by arlolra

Status: needs_informationneeds_revision

Seems fine to me. A few small problems:

  • The version number in package.json needs to follow semver http://semver.org/ ... so I added back the PATCH 0.
  • The cookies var was dependent on document.cookies. Small workaround for that.
  • Moving to modules/nodejs requires symlinking the flashproxy.js file and a prepublish script for npm
  • Can't leave a trailing slash in JSON or it won't parse (the package.json that is)
  • I added a README

Here's a patch for the above building off of dcf/flashproxy.git#node.js,

https://github.com/arlolra/flashproxy/commit/a8b22373fe21032eef507ab643e92ab5e16c2cc9

comment:30 Changed 4 years ago by arlolra

Same thing, but switched to using v0.4.27 of ws in the package.json and squashed
https://github.com/arlolra/flashproxy/commit/d3e90c8c227dc0267887b60378f0119f7d135d14

comment:31 Changed 4 years ago by dcf

Resolution: fixed
Status: needs_revisionclosed
Summary: Standalone flash proxyNode.js standalone flash proxy

I've committed what seems to be a working version in 0506689b.

Until we have a way to keep the version number in package.json synchronized, I made it independent.

Note: See TracTickets for help on using tickets.