Opened 5 years ago

Closed 4 years ago

#13493 closed enhancement (invalid)

Automate running ./update-mirrors.pl in the website builder

Reported by: arma Owned by: Sebastian
Priority: Medium Milestone:
Component: Webpages/Website Version:
Severity: Normal Keywords:
Cc: Actual Points:
Parent ID: Points:
Reviewer: Sponsor:

Description

Right now we need to run ./update-mirrors.pl manually to build the list of which mirrors we should list on https://www.torproject.org/getinvolved/mirrors

But actually, this seems like totally the sort of thing that should be run daily in an automated way by our new website auto builder.

weasel points us to these two links:
https://gitweb.torproject.org/project/jenkins/jobs.git/blob/HEAD:/jobs.yaml#l1298
https://gitweb.torproject.org/project/jenkins/tools.git/tree/HEAD:/slaves/linux/website-build-blog-snippets
for how the website auto generates the blog content for the frontpage before building.

We should extend those two scripts to auto generate the mirror content once a day too.

Child Tickets

Change History (7)

comment:1 Changed 5 years ago by arma

We might also want to make update-mirrors not actually fetch a full tor browser from each mirror for each run. That's probably best done as a separate ticket though.

comment:2 Changed 5 years ago by Sebastian

Hrm, I kind of like to do a sanity check of the mirrors by downloading a random package. But if we don't report failures, that's useless. Maybe the real idea is to have something like doctor for mirrors, running independently of the website?

comment:3 Changed 5 years ago by arma

Sounds great -- so the idea would be that the website's update-mirrors script does a daily-ish lightweight "is it obviously broken" check, to build the list of mirrors worth listing, and then doctor or some other external thing does a more heavyweight actual consistency check, and squeals to us if it finds a problem? Where a problem is defined as something that the update-mirrors script would consider "running" but that we wouldn't want users to use?

comment:4 Changed 5 years ago by Sebastian

Yeah, like that

comment:5 Changed 4 years ago by ilv

I'm not familiar with jenkins, but looking at the files, I think adding 'perl update-mirrors.pl' before this line would be enough to automate it, although I'm not sure.

comment:6 Changed 4 years ago by Sebastian

It would, but the script isn't really good at handling error conditions and happily removes all relays for any kind of error.

comment:7 Changed 4 years ago by Sebastian

Resolution: invalid
Severity: Normal
Status: newclosed

We'll likely have a new kind of thing with the new website.

Note: See TracTickets for help on using tickets.