Opened 2 months ago

Closed 2 months ago

#31416 closed defect (fixed)

Failure to update ooni.torproject.org due to static-master-update-component failure

Reported by: hellais Owned by: anarcat
Priority: Medium Milestone:
Component: Internal Services/Tor Sysadmin Team Version:
Severity: Normal Keywords:
Cc: Actual Points:
Parent ID: Points:
Reviewer: Sponsor:

Description (last modified by anarcat)

Some minutes ago I tried publishing a new version of ooni.torproject.org running the update-site script here: https://github.com/TheTorProject/ooni-web/blob/master/Makefile#L4 & https://github.com/TheTorProject/ooni-web/blob/master/update-site.sh.

And I get the following error:

+ sudo -u mirroradm /usr/local/bin/static-master-update-component ooni.torproject.org
/usr/local/bin/static-master-update-component: Acquiring lock on /srv/static.torproject.org/master/ooni.torproject.org.lock...
/usr/local/bin/static-master-update-component: Got them.
/usr/local/bin/static-master-update-component: Updating master copy of ooni.torproject.org...
/usr/local/bin/static-master-update-component: Done.  Committing.
/usr/local/bin/static-master-update-component: Triggering mirror runs...
[2019-08-14 16:58:02] Acquiring lock for /srv/static.torproject.org/master/ooni.torproject.org.lock(3).
[2019-08-14 16:58:02] All locks acquired.
[2019-08-14 16:58:02] Serial is 1565801882.
[2019-08-14 16:58:02] Populating /srv/static.torproject.org/master/ooni.torproject.org-live.new-Wr_Zx4.
[2019-08-14 16:58:02] Removing existing /srv/static.torproject.org/master/ooni.torproject.org-current-push.
[2019-08-14 16:58:02] Renaming /srv/static.torproject.org/master/ooni.torproject.org-live.new-Wr_Zx4 to /srv/static.torproject.org/master/ooni.torproject.org-current-push.
[2019-08-14 16:58:02] Calling clients...
[2019-08-14 16:58:02] Stage 1...
[2019-08-14 16:58:03] listera.torproject.org >> mirroradm@listera.torproject.org: Permission denied (publickey).
[2019-08-14 16:58:03] listera.torproject.org: failed with returncode 255
[2019-08-14 16:58:03] hetzner-hel1-03.torproject.org >> mirroradm@hetzner-hel1-03.torproject.org: Permission denied (publickey).
[2019-08-14 16:58:03] hetzner-hel1-03.torproject.org: failed with returncode 255
[2019-08-14 16:58:03] saxatile.torproject.org >> mirroradm@saxatile.torproject.org: Permission denied (publickey).
[2019-08-14 16:58:03] saxatile.torproject.org: failed with returncode 255
[2019-08-14 16:58:03] Stage 1 done.
[2019-08-14 16:58:03] 3 clients failed, aborting...
[2019-08-14 16:58:03] Aborted.
Connection to staticiforme.torproject.org closed.
make: *** [update-site] Error 1

Child Tickets

Change History (4)

comment:1 Changed 2 months ago by anarcat

Owner: changed from tpa to anarcat
Status: newassigned

this is probably my fault, and a regression in a recent puppet refactoring i completed yesterday night. will investigate immediately, sorry for the inconvenience.

comment:2 Changed 2 months ago by anarcat

Description: modified (diff)

comment:3 Changed 2 months ago by anarcat

i've found at least one of the problems and worked around it. permissions on /etc/ssh/userkeys/mirroradm were wrong: it was unreadable by the mirroradm user. i made the file owned by mirrroradm for now and was able to push an update on the ooni website.

i'll investigate the deeper cause for this problem next.

comment:4 Changed 2 months ago by anarcat

Resolution: fixed
Status: assignedclosed

i reported the underlying issue upstream as https://tickets.puppetlabs.com/browse/MODULES-9726

i redid a deployment of the ooni website and things still seem to be working after my new workaround, so I believe we can close this.

let me know if you need anything else.

Note: See TracTickets for help on using tickets.