Creating some space to host Tor Browser nightly updates

We need a space to upload and make available the Tor Browser nightly updates, for #18867 (moved).

In the past we had https://nightlies.tbb.torproject.org/, created in #24921 (moved). However it seems it doesn't exist anymore.

We could reuse the same name, https://nightlies.tbb.torproject.org/. An .onion address would be nice too.

Space needed will be ~10G in the beginning (starting with only 5 locales, but we'll maybe want to increase that number in the future).

Trac:
Parent Ticket: #18867 (moved)

added TorBrowserTeam202002 component::internal services/tor sysadmin team owner::tpa parent::18867 priority::medium resolution::fixed severity::normal status::closed tbb-update type::task labels

To upload the files there, I am planning to use a specific ssh key. It would be nice to be able to restrict what this ssh key can do, in authorized_keys, for example using rrsync (/usr/share/doc/rsync/scripts/rrsync.gz in the debian rsync package).

Trac:
Keywords: TorBrowserTeam201912 deleted, TorBrowserTeam202001 added

bumping this one :)

gotcha, pili. i talked about this with weasel quickly today and we might be able to accomodate you on the new static-master infrastructure he deployed a few months ago.

the checklist is:

1. create a role in LDAP, if missing
2. create a new static-source host instead of shoving everyone in the static-master (nightlies-source.tpo?)
3. give the role group access to the role user. create authorized_keys file writable by role user on the source.
4. add a static component, publishing to the web-fsn-* mirrors, with static-fsn-master as ... well, a master. :)

I'm not exactly sure how step 4 works here, but we'll see...

so i checked, and there's already a tbb-nightlies role in LDAP. also, weasel wants to create a separate host for this (step 2) which requires setting up a new virtual machine. i've updated the checklist to reflect that better.

unfortunately, for the next steps i'll have to delay this a bit more. we're in the process of rearchitecturing the virtual hosting infrastructure and we're in this critical stage where we want to empty a old host before creating new machines, to free up some budget.

how urgent is this? can it wait a few weeks more?

sorry for the delays...

We can wait a few more weeks. Thank you for looking into it.

Moving tickets to February

Trac:
Keywords: TorBrowserTeam202001 deleted, TorBrowserTeam202002 added

How user-facing is this service?

I ask because if it's user-facing, we want to put it on somewhat-HA mirroring, whereas if it's just internal we might not want that.

This service should be user facing.

We're running a user research testing program and we'll want end users and user research coordinators (and interested others!) to be able to easily download nightlies.

somewhat-HA sounds good, it doesn't need to be super-HA ;)

https://nightlies.tbb.torproject.org/ is now a thing.

To upload, as tbb-nightlies, put things into tbb-nightlies-master:/srv/tbb-nightlies-master.torproject.org/htdocs and run static-update-component nightlies.tbb.torproject.org

boklm should have sudo access to that user.

There also is an /etc/ssh/userkeys/tbb-nightlies. You can put ssh authorized_keys lines in there. However, tpo policy is that only command-locked keys (i.e. with a command=".." thing) should exist. Also, please use restrict and ideally from= lock the keys also.

Trac:
Resolution: N/A to fixed
Status: new to closed

Replying to weasel:

https://nightlies.tbb.torproject.org/ is now a thing.

Thanks!

To upload, as tbb-nightlies, put things into tbb-nightlies-master:/srv/tbb-nightlies-master.torproject.org/htdocs and run static-update-component nightlies.tbb.torproject.org

boklm should have sudo access to that user.

There also is an /etc/ssh/userkeys/tbb-nightlies. You can put ssh authorized_keys lines in there. However, tpo policy is that only command-locked keys (i.e. with a command=".." thing) should exist. Also, please use restrict and ideally from= lock the keys also.

Should I be using /usr/local/bin/staticsync-ssh-wrap, or something else to restrict rsync access?

I tried with command="/usr/local/bin/staticsync-ssh-wrap nightlies.tbb.torproject.org" in /etc/ssh/userkeys/tbb-nightlies.

Then I tried running rsync like this:

$ rsync --safe-links -lrtHz  /some/directory/. tbb-nightlies-master:/srv/tbb-nightlies-master.torproject.org/htdocs/.

But I get the following error:

This rsync command (nightlies.tbb.torproject.org --server -lHtrze.iLsfxC --safe-links . /srv/tbb-nightlies-master.torproject.org/htdocs/.) not allowed.

So it looks like staticsync-ssh-wrap only allows rsync to read from this directory, but not to write to it.

Should I be using /usr/share/doc/rsync/scripts/rrsync instead, or is there something else?

Replying to boklm:

Should I be using /usr/share/doc/rsync/scripts/rrsync instead, or is there something else?

So I used /usr/share/doc/rsync/scripts/rrsync, by copying it as /home/boklm/tbb-nightlies/rrsync, and it seems to be working.

closed

mentioned in issue #34121 (moved)