Need a virtual host to run OnionPerf

Hi,

The metric team needs a VM to have a test run of OnionPerf (https://github.com/robgjansen/onionperf).

Ideally we would like to keep generated logs for several months (>6) so approximately 500Gigas of minimum disk space are required (based on a rough estimation that 50Mb of logs are generated every few hours).

Please let me know if I can help you with anything regarding this.

added component::internal services/tor sysadmin team owner::tpa priority::medium resolution::fixed severity::normal status::closed type::defect labels

That's a lot of disk space, but not impossible as long as it's ok if it's slow.

How much network traffic do we think this will use?

Network listening things will this host need to run?

Who will maintain the service?

Hi Weasel, we just need to save the logs. For the time being nothing else is required, so I assume it is not a problem if the disk is slow.

We want to run OnionPerf in measure mode. This means a TGen server running on port 8080 and a Twisted web server running on port 8081. Port 8080 must be open we want to do performance measurements with downloads that exit the Tor network. Port 8081 is for the web server, so we need it open only if we want it to be public accessible. Karsten, could you please let me know if we want the web server to allow access to the data?

OnionPerf runs a TGen client/server pair that transfer traffic through Tor and through an ephemeral onion service started by OnionPerf itself. I am not sure exactly how much network traffic the measurement generates, I can find out a rough estimation if you need it, considering that measurements are run every hour.

These are the measurements that it gathers:

Tgen:

• Number of transfer AUTH errors, each client

• Number of transfer PROXY errors, each client

• Number of transfer AUTH errors, all clients over time

• Number of transfer PROXY errors, all clients over time

• Bytes transferred before AUTH error, all downloads

• Bytes transferred before PROXY error, all downloads

• Median bytes transferred before AUTH error, each client

• Median bytes transferred before PROXY error, each client

• Mean bytes transferred before AUTH error, each client

• Mean bytes transferred before PROXY error, each client

Tor

• 60 second moving average throughput, read, all relays

• 1 second throughput, read, all relays

• 1 second throughput, read, each relay

• 60 second moving average throughput, write, all relays

• 1 second throughput, write, all relays

• 1 second throughput, write, each relay

I do not think it will generate lot of network traffic, but, as I said, if you prefer I can run an estimation.

I can maintain the service.

Hi, and thanks for starting this discussion!

I don't quite follow where the 500 GB number comes from. You say there are 50 MB every few hours. If we assume you mean every 4 hours, that's 300 MB per day or ~50 GB per six months. Should we maybe start with 50 GB and consider raising that number if we really need more?

I'm also asking, because 500 GB disk space almost makes this service almost prohibitive for wider deployment, so I hope we'll find a way to require much less.

Regarding ports, I'd hope that we can use port 80 for measurements rather than 8080. Otherwise we'll measure the wrong thing. This probably requires some firewall magic or privilege dropping.

Regarding other ports, I'd say we don't need to make measurement results available for the moment. We might reconsider later, but opening a port when we need it shouldn't be difficult, so let's start with as few ports open as necessary.

Thanks!

Hi Karsten, You are right, sorry I added a 0 in my calculation :(

Weasel: 50Gb would be ok.

Is onionperf the same or a successor to torperf?

Host is papillare, put service related files directly into /srv/onionperf.torproject.org, as user onionperf. port 80 redirects to 10080.

Trac:
Resolution: N/A to fixed
Status: new to closed

Thanks for this.

closed