Opened 11 months ago

Closed 9 months ago

#32711 closed defect (fixed)

Gitlab repository is full and can't be updated

Reported by: cohosh Owned by: cohosh
Priority: Medium Milestone:
Component: Applications/GetTor Version:
Severity: Normal Keywords: gitlab
Cc: traumschule, hiro, gaba, phw, cohosh Actual Points: 3
Parent ID: Points:
Reviewer: phw Sponsor:

Description

The current script for updating the gitlab gettor repository with new Tor Browser binaries fails because the repository has reached the storage limit size.

We should update this script with something that works for GitLab (note: we can't move to releases as in Github).

Child Tickets

Change History (12)

comment:1 Changed 11 months ago by cohosh

Even after we delete a branch, the files stick around in storage due to our usage of LFS.

After uploading about 5GB of files in a new test repository:

$ du -h
4.9G	./.git/lfs

There's no way to remove these files from the remote without deleting the repository. See this github guide to LFS. So if we want to still use gitlab, we should look into to the API which might allow us to easily delete and create a new repository from the command line.

This brings us to another issue: Gitlab repositories are capped at 9.8GB, which is not enough space to store all the binaries we need. There's a few options here:

  • Prioritize locales for places that censor the other GetTor providers (see #32716)
  • Create more than one repository (perhaps one per platform?)
  • Abandon GitLab for another platform (let's make sure the other providers are reachable for people first)

comment:2 Changed 10 months ago by cohosh

Owner: set to cohosh
Status: newassigned

comment:3 Changed 10 months ago by cohosh

Status: assignedneeds_review

I've updated the update_github script to update both github and gitlab remotes: https://dip.torproject.org/cohosh/gettor/compare/master...gitlab

This involved a significant refactor to avoid duplicate code.

comment:4 Changed 10 months ago by cohosh

Actual Points: 3

Updating with points so far

comment:5 Changed 10 months ago by cohosh

FWIW, I also tested it on some of my own repositories:
https://gitlab.com/dashboard/projects
https://github.com/cohosh/releases/releases

comment:6 Changed 9 months ago by cohosh

Reviewer: phw

comment:7 Changed 9 months ago by cohosh

Yay, figured out how to make merge requests in gitlab: https://dip.torproject.org/cohosh/gettor/merge_requests/2

comment:8 Changed 9 months ago by phw

Status: needs_reviewneeds_revision

I added a bunch of comments directly to the merge request.

comment:9 Changed 9 months ago by cohosh

Status: needs_revisionneeds_review

Thanks, made some changes.

comment:10 Changed 9 months ago by cohosh

Some minor fixes that came up while deploying: https://dip.torproject.org/cohosh/gettor/merge_requests/3

comment:11 in reply to:  10 Changed 9 months ago by phw

Status: needs_reviewmerge_ready

Replying to cohosh:

Some minor fixes that came up while deploying: https://dip.torproject.org/cohosh/gettor/merge_requests/3


Looks good to me.

comment:12 Changed 9 months ago by cohosh

Resolution: fixed
Status: merge_readyclosed

Merged: https://gitweb.torproject.org/gettor.git/commit/?id=4c8dd8568ac7ee9fd01a75d23b9fcfbf507be5c0

These changes have been deployed and tor browser binaries have now been uploaded to gitlab.com: https://gitlab.com/thetorproject

I've also updated the links database and it looks like all links in gettor emails are now working!

You requested Tor Browser for linux.

You will need only one of the links below to download the bundle. If a link does not work for you, try the next one.

None
Tor Browser 9.0.4 for linux-en-US-64 (gitlab): https://gitlab.com/thetorproject/torbrowser-linux/raw/master/tor-browser-linux64-9.0.4_en-US.tar.xz
Signature file: https://gitlab.com/thetorproject/torbrowser-linux/raw/master/tor-browser-linux64-9.0.4_en-US.tar.xz.asc

Tor Browser 9.0.4 for linux-en-US-64 (github): https://github.com/torproject/torbrowser-releases/releases/download/torbrowser-release/tor-browser-linux64-9.0.4_en-US.tar.xz
Signature file: https://github.com/torproject/torbrowser-releases/releases/download/torbrowser-release/tor-browser-linux64-9.0.4_en-US.tar.xz.asc


Should you have issues with any of the links above you can access the following archives:

- Internet Archive: https://archive.org/details/@gettor

- Google Drive folder: https://drive.google.com/open?id=13CADQTsCwrGsIID09YQbNz2DfRMUoxUU

Download the file: tor-browser-linux64-9.0.4_en-US.tar.xz

I've also updated the survival guide with the new script name and description: https://dip.torproject.org/torproject/anti-censorship/gettor-project/gettor/-/wikis/Gettor-Service-Operator-HowTo

Note: See TracTickets for help on using tickets.