Opened 6 weeks ago

Last modified 10 hours ago

#32711 needs_revision defect

Gitlab repository is full and can't be updated

Reported by: cohosh Owned by: cohosh
Priority: Medium Milestone:
Component: Applications/GetTor Version:
Severity: Normal Keywords: gitlab
Cc: traumschule, hiro, gaba, phw, cohosh Actual Points: 3
Parent ID: Points:
Reviewer: phw Sponsor:

Description

The current script for updating the gitlab gettor repository with new Tor Browser binaries fails because the repository has reached the storage limit size.

We should update this script with something that works for GitLab (note: we can't move to releases as in Github).

Child Tickets

Change History (8)

comment:1 Changed 6 weeks ago by cohosh

Even after we delete a branch, the files stick around in storage due to our usage of LFS.

After uploading about 5GB of files in a new test repository:

$ du -h
4.9G	./.git/lfs

There's no way to remove these files from the remote without deleting the repository. See this github guide to LFS. So if we want to still use gitlab, we should look into to the API which might allow us to easily delete and create a new repository from the command line.

This brings us to another issue: Gitlab repositories are capped at 9.8GB, which is not enough space to store all the binaries we need. There's a few options here:

  • Prioritize locales for places that censor the other GetTor providers (see #32716)
  • Create more than one repository (perhaps one per platform?)
  • Abandon GitLab for another platform (let's make sure the other providers are reachable for people first)

comment:2 Changed 2 weeks ago by cohosh

Owner: set to cohosh
Status: newassigned

comment:3 Changed 11 days ago by cohosh

Status: assignedneeds_review

I've updated the update_github script to update both github and gitlab remotes: https://dip.torproject.org/cohosh/gettor/compare/master...gitlab

This involved a significant refactor to avoid duplicate code.

comment:4 Changed 11 days ago by cohosh

Actual Points: 3

Updating with points so far

comment:5 Changed 9 days ago by cohosh

FWIW, I also tested it on some of my own repositories:
https://gitlab.com/dashboard/projects
https://github.com/cohosh/releases/releases

comment:6 Changed 11 hours ago by cohosh

Reviewer: phw

comment:7 Changed 11 hours ago by cohosh

Yay, figured out how to make merge requests in gitlab: https://dip.torproject.org/cohosh/gettor/merge_requests/2

comment:8 Changed 10 hours ago by phw

Status: needs_reviewneeds_revision

I added a bunch of comments directly to the merge request.

Note: See TracTickets for help on using tickets.