Opened 5 years ago

Closed 5 years ago

#12655 closed enhancement (implemented)

Add performance statistics to Onionoo's servlet

Reported by: karsten Owned by: karsten
Priority: Medium Milestone:
Component: Metrics/Onionoo Version:
Severity: Keywords:
Cc: n8fr8 Actual Points:
Parent ID: Points:
Reviewer: Sponsor:

Description

n8fr8 plans to release an Orbot version that queries relay information via Onionoo. There are quite a few Orbots deployed, so this might put some load on Onionoo. We should add some simple performance statistics to Onionoo to detect load problems early.

To start with, let's add statistics for the following request, which is the first request that n8fr8 plans to add to Orbot (it's also the standard request that Atlas and Globe make to show relay details):

https://onionoo.torproject.org/details?lookup=$fingerprint

Some simple metrics to start with might be:

  • Number of requests per 1, 5, 15 minutes
  • Time to parse request, time to build response, time to send response

What I'm not sure about yet is where to store and display these statistics. We could start with heartbeat log messages containing these numbers.

Child Tickets

Change History (2)

comment:1 Changed 5 years ago by karsten

Implemented and deployed. Here's some output:

Request statistics (2014-08-12 13:34:42, 3600 s):
  Total processed requests: 165410
  Most frequently requested resource: details (163250), summary (1674), bandwidth (224)
  Most frequently requested parameter combinations: [lookup] (159112), [flag, type, running] (2243), [lookup, fields] (2213)
  Matching relays per request: .500<2, .900<2, .990<16384, .999<16384
  Matching bridges per request: .500<1, .900<1, .990<16384, .999<16384
  Written characters per response: .500<4096, .900<4096, .990<4194304, .999<4194304
  Milliseconds to handle request: .500<8, .900<16, .990<32, .999<128
  Milliseconds to build response: .500<2, .900<4, .990<1024, .999<4096

The last five lines have the following meaning: 90% of requests matched 0 or 1 relays, 99% and even 99.9% of requests matched less than 16384 relays, the remaining 0.01% may have matched even more relays. (These lines only contain powers of two, because that's how these statistics are stored in memory.)

The vast majority of requests is for looking up a single document (159112/165410=96%). These are very likely Orbot users. For comparison, there were only 224 requests for bandwidth documents, which are fetched for Atlas' and Globe's details pages. If we want to reduce the current load caused by Orbot users, we could add a server-side cache, so that most of these requests don't even arrive at Tomcat. That's for a different ticket though.

That concludes this task. We can tweak these statistics as needed. Closing.

comment:2 Changed 5 years ago by karsten

Resolution: implemented
Status: newclosed
Note: See TracTickets for help on using tickets.