Opened 5 years ago

Closed 5 years ago

#12849 closed task (implemented)

Add web cache to Onionoo's setup

Reported by: karsten Owned by:
Priority: Medium Milestone:
Component: Internal Services/Tor Sysadmin Team Version:
Severity: Keywords:
Cc: Actual Points:
Parent ID: Points:
Reviewer: Sponsor:


The Onionoo service has become much more popular since Orbot started asking Onionoo for relay details a few weeks ago. As of today, Onionoo serves around 160,000 requests per hour to Orbot users. These requests are for roughly 1,000 different documents that are less than 4KB in size and updated once per hour. Up to 159,000 of these requests could be served by a cache.

Would it be possible to add something like Squid in front of Apache, or can we add some other caching mechanism to Apache so that it answers certain requests itself rather than forwarding them to Tomcat?

Is it possible to control this cache by making Tomcat add certain headers to its responses, like a suggested cache lifetime? The Tomcat application knows pretty well when a response is unlikely to change for 15 minutes and when it would be better to invalidate the cache entry after 5 minutes.

Child Tickets

Change History (5)

comment:1 Changed 5 years ago by weasel

Can you start by adding proper Cache-Control headers to the application's output?

comment:2 Changed 5 years ago by karsten

Certainly. Added the following header. Does this work?

Cache-Control: public,max-age=300

comment:3 Changed 5 years ago by weasel

I have enabled and configured Apache's mod_mem_cache. Note that this is per apache process, and not shared across workers.

Is that already good enough, or should we investigate something more complex?

comment:4 Changed 5 years ago by karsten

You said on IRC that you switched to disk cache, which I think makes sense. I tweaked the cache-control logic to set max-age to values between 5 and 40 minutes. Right now I'm seeing 80% cache hits, which is pretty good. Let's look again in a day or so, but I'm optimistic.

comment:5 Changed 5 years ago by karsten

Resolution: implemented
Status: newclosed

86% cache hits and down from 160k to 9k requests handled by Tomcat per hour. I'd say this works really well. Thanks, weasel, for implementing this so quickly! Closing.

Note: See TracTickets for help on using tickets.