websocket-server seems to be (very) slow

added component::circumvention/pluggable transport owner::asn priority::medium resolution::worksforme status::closed type::defect labels

Replying to Aymeric:

After multiple tests and iterations, here is the average behavior with tor1.bamsoftware.com with the following configuration:

Browser (2 Mbps) --> 600kbps --> tor1 300 kbps --> Tor 2 Mbps --> ORDB (400/500 kbps) --> Tor 2 Mbps --> 2 Mbps --> Browser (2Mbps) (real: 300kbps)

Thanks for this report. I don't understand your testing setup? Are you using a browser as a WebSocket client?

For a pure bandwidth measurement, you should try the websocket-client program; otherwise, you may actually be measuring something in a browser.

Also, I recommend setting up a relay on localhost so you can see the log output, and look for these messages in websocket-server.log:

"frame payload length of %d exceeds maximum of %d"

I can imagine that too many of those would cause it too look like a slow connection. Running it locally will also allow you to check if CPU usage becomes too high.

Trac:
Status: new to needs_information

Yes the client is a browser doing the OP (still the same projects see http://www.peersm.com and https://www.github.com/Ayms/node-Tor), Browser1 (left) is sending a file to Browser2 (right) transiting through Tor and relayed by the ORDB.

I have tried above to give the average CPU and bandwidth performances. For the CPU it's computed on iteration on Tor cells (ie how many time to handle x Tor cells), so in the browser this gives for example the performances for javascript crypto which is what impacts most performances (encrypt/decrypt/hash x times for x nodes in the path)

The ORDB shows small performances because it's a very limited server, I must upgrade it, but we don't care about it here.

The target is to have a speed between the two browsers equal to the upload bandwidth (600 kbps), we see that Browser1 is faster than that, for the tests it's sending packets at 800 kbps and I can clearly see that tor1 can not exceed 250/300 kbps, it's very obvious if I monitor at the same time the bufferedAmount for WebSocket and browser2 gets a rate of 250/300 kbps max.

I have replaced tor1 by a node of mine, same server as ORDB handing WebSockets, so not very fast, but can at least handle 1 Mbps and then the final rate I get for Browser2 is 400/500 kbps (then limited by the ORDB)

Now it's possible that I have an upload bandwidth limitation between my browser and tor1 but this seems unlikely, because the limitation is always the same and I have tested tor1 as a normal OR and the download bandwidth is in the average.

That's why I think there is an issue with tor1 websocket-server and/or pipe to the OR port.

If true, then you get this limitation for flash proxys traffic, I suppose you have tested it but probably it would be good to test it again, maybe nobody noticed it because most of the traffic is supposed to be the download way (maybe I should test with my ws node connected to browser1 and tor1 connected to browser2 to see if the limitation is both ways, I did not try).

I don't know if it can be valid for your code but I remember when I did https://github.com/Ayms/websocket that you can easily mess up with the code for ws encode/decode and get really poor performances.

What I recommend you do, is set up websocket-client and websocket-server, both running on localhost in order to reduce the number of things that may be affecting your measurements. I'm afraid I don't have time to investigate this report unless there is clear evidence that websocket-server is the bottleneck.

Today I tried connecting with websocket-client over the Internet to tor1. I got about 600 KB/s, and the CPU usage of websocket-server on the relay was less than 2%. Perhaps the slowness you are seeing is the result of some interaction with the browser or other software. It still may be the case that the browser sends too-big frames and causes "frame payload length of %d exceeds maximum of %d"; that might be a cause of slowness. (In fact it would cause the browser to disconnect and perhaps reconnect.)

Trac:

Could you please test the other way (upload)?

I can not easily setup the websocket client and server on localhost, I could investigate further but right now I am pretty much convinced that websocket-server is the bottleneck for the upload way, as I wrote when I am replacing tor1 by my node the problem disappears.

Trac:
bug_ws.txt

I don't know if you can consider this as more evidences, but please see the attached file, measurements between the browser and tor1 on the upload link.

Same log in bug_ws2.txt but using my WS OR instead of tor1, as you can see the rate is +/- equal to the available upload bandwidth.

It's obvious that tor1 is causing the bottleneck.

Trac:
bug_ws2.txt

Thanks again for following up on this report. But I still really can't do anything with the information you've given me. Please stop testing against tor1 because that is full of confounding factors. You need to run a copy of websocket-server on localhost, and see if it has the same behavior. It's not hard to do—just run a copy of tor with a few lines of configuration. There are instructions for doing it right here. You might also need to set BridgeRelay 1 in your case.

Did you try websocket-client yet, or not?

I'm asking you to do exactly the the same things I would have to do if I were investigating a performance problem; the first thing I would do is try running a copy of the server on localhost. Then I would check CPU usage, profile the execution, and look at packet captures (again on localhost). It's not that I doubt you—it's that doing these things will take me hours and I'm not even sure what I'm looking for. If this issue is important to you, you need to do some of the legwork.

This is important for all flash proxy users.

Can't someone in the Tor team spend 5mn to do this? Probably you have understood that I am not working with the Tor packages, so I really don't know how to set what you are saying is so simple, if not I would have done it already.

What needs to be tested is from the browser to the websocket-server, and not the contrary, a simple test could be to connect as a flash proxy user, download a file and logically observe that the download rate is far lower than the usual Tor one.

Sorry to insist, you should investigate this issue, it's clear that there is a limitation due to the websocket-server on the upload link, see new logs with a 10MB file:

Test1 (first part of the logs):

receiver <-- tor1 <-- Tor network <-- tor1 <-- sender

We see that the effective rate is stable at 300 kbps on sender side and receiver side

Test2 (second part of the logs) :

receiver <-- tor1 <-- Tor network <-- Peersm Bridge <-- sender

The effective rate is stable at 600 kbps

The tests are always providing the same results and the rates are stable, so this can not be the effect of some hazards, tor1 is not able to process more than 300 kbps on the upload link browser --> tor1

Trac:
Tor-bug-receiver.txt

Trac:
Tor-bug-sender.txt

A more relevant proof, with latest release of FF open page1 and page2:

http://peersm.com/peersm

Key: 00112233445566778899aabbccddeeff

Make sure page1 is connected to tor1.bamsoftware.com, if not refresh the page until you see it has chosen tor1 (look at the debug div "start websocket ws://xxx", there are only two ws nodes for now)

Upload a 10 MB file from your disk inside page1 storage.

Use the reference you got to download it from page2, you can try n times you will never get more than 300 kbps

Do the same with page1 connected to 91.121.106.131, you will get a much higher rate.

Replying to Aymeric:

A more relevant proof, with latest release of FF open page1 and page2:

http://peersm.com/peersm

Key: 00112233445566778899aabbccddeeff

Make sure page1 is connected to tor1.bamsoftware.com, if not refresh the page until you see it has chosen tor1 (look at the debug div "start websocket ws://xxx", there are only two ws nodes for now)

Upload a 10 MB file from your disk inside page1 storage.

Use the reference you got to download it from page2, you can try n times you will never get more than 300 kbps

Do the same with page1 connected to 91.121.106.131, you will get a much higher rate.

I tried the page you pointed to, but nothing happens after I select a file to upload, and there are these errors in the Web Console:

[08:32:17.703] InvalidStateError @ https://peersm.com/node-browser.js:45
[08:32:34.507] TypeError: f is undefined @ https://peersm.com/node-browser.js:45

Please go back and read comment:8. I am ignoring this ticket until you take those basic debugging steps. I need you to test websocket-server and websocket-client both on localhost. Do not test against tor1. Do not test with peersm in place of websocket-client, or I will continue to ignore the ticket.

In comment:4 I tested the main flash proxy use case, and it worked as expected.

If you really think there is a problem with the websocket-server, you need to show me evidence that it is the case. You haven't even shown evidence that the problem, if it exists, is with websocket-server and not tor1 itself. This is the simplest thing of all to test: Just set up websocket-server on a different host and see if it behaves the same. I really cannot take any action on this ticket until you do these basic steps.

I am grateful that you are taking the time to make these reports. But please, you need to do some things differently in order for the reports to be more useful. Please read comment:8 again.

I will email you separately for the peersm problem.

Regarding the bug, unless I am not explaining things correctly, it seems clear that what I have written before shows that tor1 does not handle correctly incoming ws traffic.

I will try to install it when I have time but as I mentioned before, I am not familiar with the Tor packages, so you are asking me to do something that takes time while this would take 5mn to someone of the Tor team. And with no certitude to identify the issue, this can be the websocket-server itself, between the websocket-server and the OR port, etc, I don't know how you made this.

You can close it I think, maybe finally it was unfortunate than each time we were using tor1 there was a slow node in the path.

We have released our ws bridges https://github.com/Ayms/node-Tor/tree/master/install

In cas of future doubts about this bug if someone wants to test it's possible to do http://peersm.com/peersm#IP:port , where IP:port are the one of the bridge.

All right. Thanks for taking the time to investigate this issue and helping me understand your point of view.

Trac:
Status: needs_information to closed
Resolution: N/A to worksforme

closed

moved to tpo/anti-censorship/pluggable-transports/trac#10192 (closed)

websocket-server seems to be (very) slow

Child items 0

Activity