Gitlab repository is full and can't be updated

The current script for updating the gitlab gettor repository with new Tor Browser binaries fails because the repository has reached the storage limit size.

We should update this script with something that works for GitLab (note: we can't move to releases as in Github).

added actualpoints::3 component::applications/gettor gitlab owner::cohosh priority::medium resolution::fixed reviewer::phw severity::normal status::closed type::defect labels

Even after we delete a branch, the files stick around in storage due to our usage of LFS.

After uploading about 5GB of files in a new test repository:

$ du -h
4.9G	./.git/lfs

There's no way to remove these files from the remote without deleting the repository. See this github guide to LFS. So if we want to still use gitlab, we should look into to the API which might allow us to easily delete and create a new repository from the command line.

This brings us to another issue: Gitlab repositories are capped at 9.8GB, which is not enough space to store all the binaries we need. There's a few options here:

• Prioritize locales for places that censor the other GetTor providers (see #32716 (moved))
• Create more than one repository (perhaps one per platform?)
• Abandon GitLab for another platform (let's make sure the other providers are reachable for people first)

Trac:
Owner: N/A to cohosh
Status: new to assigned

I've updated the update_github script to update both github and gitlab remotes: https://dip.torproject.org/cohosh/gettor/compare/master...gitlab

This involved a significant refactor to avoid duplicate code.

Trac:
Status: assigned to needs_review

Updating with points so far

Trac:
Actualpoints: N/A to 3

FWIW, I also tested it on some of my own repositories: https://gitlab.com/dashboard/projects https://github.com/cohosh/releases/releases

Trac:
Reviewer: N/A to phw

Yay, figured out how to make merge requests in gitlab: https://dip.torproject.org/cohosh/gettor/merge_requests/2

I added a bunch of comments directly to the merge request.

Trac:
Status: needs_review to needs_revision

Thanks, made some changes.

Trac:
Status: needs_revision to needs_review

Some minor fixes that came up while deploying: https://dip.torproject.org/cohosh/gettor/merge_requests/3

Replying to cohosh:

Some minor fixes that came up while deploying: https://dip.torproject.org/cohosh/gettor/merge_requests/3

Looks good to me.

Trac:
Status: needs_review to merge_ready

Merged: https://gitweb.torproject.org/gettor.git/commit/?id=4c8dd8568ac7ee9fd01a75d23b9fcfbf507be5c0

These changes have been deployed and tor browser binaries have now been uploaded to gitlab.com: https://gitlab.com/thetorproject

I've also updated the links database and it looks like all links in gettor emails are now working!

You requested Tor Browser for linux.

You will need only one of the links below to download the bundle. If a link does not work for you, try the next one.

None
Tor Browser 9.0.4 for linux-en-US-64 (gitlab): https://gitlab.com/thetorproject/torbrowser-linux/raw/master/tor-browser-linux64-9.0.4_en-US.tar.xz
Signature file: https://gitlab.com/thetorproject/torbrowser-linux/raw/master/tor-browser-linux64-9.0.4_en-US.tar.xz.asc

Tor Browser 9.0.4 for linux-en-US-64 (github): https://github.com/torproject/torbrowser-releases/releases/download/torbrowser-release/tor-browser-linux64-9.0.4_en-US.tar.xz
Signature file: https://github.com/torproject/torbrowser-releases/releases/download/torbrowser-release/tor-browser-linux64-9.0.4_en-US.tar.xz.asc


Should you have issues with any of the links above you can access the following archives:

- Internet Archive: https://archive.org/details/@gettor

- Google Drive folder: https://drive.google.com/open?id=13CADQTsCwrGsIID09YQbNz2DfRMUoxUU

Download the file: tor-browser-linux64-9.0.4_en-US.tar.xz

I've also updated the survival guide with the new script name and description: https://dip.torproject.org/torproject/anti-censorship/gettor-project/gettor/-/wikis/Gettor-Service-Operator-HowTo

Trac:
Status: merge_ready to closed
Resolution: N/A to fixed

closed

added 24h of time spent

moved to tpo/anti-censorship/gettor-project/trac#32711 (closed)