Opened 11 months ago

Last modified 10 months ago

#26231 new defect

Addressing trac performance issues

Reported by: hiro Owned by: qbi
Priority: Medium Milestone:
Component: Internal Services/Service - trac Version:
Severity: Normal Keywords:
Cc: Actual Points:
Parent ID: Points:
Reviewer: Sponsor:

Description

We have been having issues with trac for a while now. These are mostly caused by a number of crawlers browsing our entire history on trac.

AFAIK we are not sure if these bots are random crawlers or search engines doing their normal duty. We could temporarily look at the log while the issue happen and try to see if we can spot known IP addresses or User Agent tags.

Regardless, there are a few things that we could do:

  1. Implement the crawl delay directive: https://en.wikipedia.org/wiki/Robots_exclusion_standard#Crawl-delay_directive

There is a good chance this is not going to change anything (especially with random bots) but it doesn't affect users and it can be implemented right away.

  1. This could be DB related. I.e. the crawler is performing some complex sql query and the trac process is crashing because of a timeout or something. We know Trac search isn't fast either, so this could be an easy explanation.
  1. Could this be a hardware issue? The host seem happy, but maybe we could easily upgrade the specs of the machine where trac lives?
  1. Try rate limit on iptables / use something like HA-proxy? This will have to be discussed as we would have to change our policy on what we want to log. We wouldn't have to store logs for a long time but probably we would need to keep some state info / addresses / etc for a few minutes. I would consider this a last resort.

Child Tickets

Change History (2)

comment:1 Changed 11 months ago by hiro

Point 1. has been implemented.

comment:2 Changed 10 months ago by hiro

I have been implementing a few configuration options as described in: https://trac.edgewall.org/wiki/TracPerformance

More specifically I have set:

  • max_daysback: 30 (It was set to 90 days before and this is known to cause issues with trac installation with 'some history' like ours)
  • use_chunked_encoding: true

If this doesn't solve our issue maybe it is time to upgrade our hardware or install a loadbalancing solution like HAproxy.

Note: See TracTickets for help on using tickets.