Opened 8 years ago

Closed 7 years ago

#4948 closed enhancement (fixed)

Clean up webstats.tpo

Reported by: runa Owned by: runa
Priority: Medium Milestone:
Component: Metrics/Website Version:
Severity: Keywords:
Cc: Actual Points:
Parent ID: Points:
Reviewer: Sponsor:

Description

I should sit down and clean up webstats.tpo:

  • Webalizer is being high-maintenance and is not happy parsing and appending daily logs to its current output. AWStats is working fine, so maybe we should just drop it.
  • Make sure we rsync tarballs from stenodon to archive.
  • Automatically update the website (if/when possible) when we have new statistics and tarballs.
  • Write the page that explains how and why we sanitize the log files.
  • Make sure we include logs from hosts other than metrics.tpo.

Child Tickets

Change History (2)

comment:1 in reply to:  description Changed 8 years ago by karsten

Replying to runa:

  • Webalizer is being high-maintenance and is not happy parsing and appending daily logs to its current output. AWStats is working fine, so maybe we should just drop it.

Dropping Webalizer sounds fine.

  • Make sure we include logs from hosts other than metrics.tpo.

We'll want to process older logs from added hosts on a different machine than stenodon, or we'll run out of disk space. Please let me know when you're planning to add more hosts.

comment:2 Changed 7 years ago by runa

Resolution: fixed
Status: newclosed

Here's what I have done:

  • Fixed Webalizer to append daily logs to its current out, and dropped AWStats
  • Updated the logarchive script to put tarballs in a to-archive directory
  • Coordinated with weasel to have tarballs moved to archive.tpo/torproject-weblogs/

At the moment, we are only sanitizing and processing logs for metrics.tpo. I will leave it up to Karsten and #4859 to sanitize logs for other hosts. I will also keep an eye on webstats.tpo in the coming weeks to make sure all the scripts are working correctly.

Note: See TracTickets for help on using tickets.