Opened 2 years ago

Closed 2 years ago

#25342 closed task (fixed)

Please sync recent Apache web server logs from torproject.org hosts to colchicifolium

Reported by: karsten Owned by: tpa
Priority: Medium Milestone:
Component: Internal Services/Tor Sysadmin Team Version:
Severity: Normal Keywords:
Cc: metrics-team Actual Points:
Parent ID: Points:
Reviewer: Sponsor:

Description

We're soon going to sanitize and publish web server logs from torproject.org hosts on CollecTor. It's going to take over the job that's currently running on wendelboi, but please don't shut that down yet.

As first step, can you please sync all recent Apache web server logs from torproject.org hosts to colchicifolium? I just created a new directory there, /srv/collector.torproject.org/collector/in/webstats/ where all logs can be synced to, ideally using the same directory structure as on wendelboi.

We'd like to set up everything on CollecTor and run it for a few weeks before switching over. That would be a new ticket then.

Thanks in advance!

Child Tickets

Change History (3)

comment:1 Changed 2 years ago by karsten

When you have a moment, can you please start looking into this or tell me what's needed to get this going? Ideally, I'd want to start deploying things before Rome. Thanks!

comment:2 Changed 2 years ago by weasel

You might want to update /srv/collector.torproject.org/collector/bin/ssh-wrap to support receiving the weblogs.

this is the current script on wendelboi:

/srv/webstats.torproject.org$ cat /srv/webstats.torproject.org/bin/weblog-ssh-wrap                                                        
#!/bin/sh                                           
 
set -e
set -u

if [ "$#" != "1" ] || [ -z "${SSH_ORIGINAL_COMMAND:-}" ]; then
  echo >&2 "Usage: $0 <sourcename>   [via remote ssh]"
  exit 1
fi

target="/srv/webstats.torproject.org/incoming/$1"
mkdir -p "$target"

rsync  --server -logDtpre.iLsfx --delete . "$target/."

Once that is in place I can check how to get that working without breaking the bridgedb sync at the same time.

comment:3 Changed 2 years ago by karsten

Resolution: fixed
Status: newclosed

We talked more about this on IRC yesterday. Everything's set up now. Thanks! Closing.

Note: See TracTickets for help on using tickets.