Opened 3 years ago

Closed 3 years ago

#25047 closed defect (invalid)

Various torproject services down

Reported by: hellais Owned by: tpa
Priority: High Milestone:
Component: Internal Services/Tor Sysadmin Team Version:
Severity: Major Keywords:
Cc: Actual Points:
Parent ID: Points:
Reviewer: Sponsor:

Description (last modified by hellais)

This ticket is possibly related to the issue: https://trac.torproject.org/projects/tor/ticket/25041.

Looking at when some people using the iranicum.torproject.org box went down, it seems like some services went down since ~04:40 UTC.

What's particularly worrying is that eugeni.torproject.org is also down, which is the host running the @torproject.org mail server.

With a quick and dirty shell script (and the data from https://db.torproject.org/machines.cgi), it seems like the affected machines are:

build-arm-01.torproject.org
build-arm-02.torproject.org
build-arm-03.torproject.org
carinatum.torproject.org
chamaemoly.torproject.org
chiwui.torproject.org
colchicifolium.torproject.org
eugeni.torproject.org
feddei.torproject.org
gayi.torproject.org
hedgei.torproject.org
henryi.torproject.org
hyalinum.torproject.org
iranicum.torproject.org
listera.torproject.org
materculae.torproject.org
perdulce.torproject.org
saxatile.torproject.org
staticiforme.torproject.org
textile.torproject.org
togashii.torproject.org
weissii.torproject.org
winklerianum.torproject.org
woronowii.torproject.org

Here the output of the script that includes the resolved address for the affected hosts: http://paste.debian.net/1007443/

Child Tickets

Change History (4)

comment:1 Changed 3 years ago by hellais

Description: modified (diff)
Priority: MediumHigh
Severity: NormalMajor

comment:2 Changed 3 years ago by weasel

Resolution: invalid
Status: newclosed

this ticket is not entirely accurate. only textile with its vms is down, and that's being worked on.

as this tickets lists more, I'll just close it for now.

comment:3 Changed 3 years ago by cypherpunks

Resolution: invalid
Status: closedreopened

Sorry, but where is it documented which VMs are hosted where, and where can this issue be tracked for resolution? I have checked https://db.torproject.org/machines.cgi and https://trac.torproject.org/projects/tor/wiki/org/operations/Infrastructure

I vaguely recall that munin or some other monitoring page used to be world-readable, but this doesn't appear to be the case any more.

Thanks!

comment:4 Changed 3 years ago by cypherpunks

Resolution: invalid
Status: reopenedclosed

I am informed that https://nagios.torproject.org/icinga/ u: tor-guest p: <whatever> is the place to go

Note: See TracTickets for help on using tickets.