Opened 10 months ago

Closed 8 months ago

Last modified 8 months ago

#32800 closed task (fixed)

Creating some space to host Tor Browser nightly updates

Reported by: boklm Owned by: tpa
Priority: Medium Milestone:
Component: Internal Services/Tor Sysadmin Team Version:
Severity: Normal Keywords: tbb-update, TorBrowserTeam202002
Cc: boklm, tbb-team Actual Points:
Parent ID: #18867 Points:
Reviewer: Sponsor:

Description

We need a space to upload and make available the Tor Browser nightly updates, for #18867.

In the past we had https://nightlies.tbb.torproject.org/, created in #24921. However it seems it doesn't exist anymore.

We could reuse the same name, https://nightlies.tbb.torproject.org/. An .onion address would be nice too.

Space needed will be ~10G in the beginning (starting with only 5 locales, but we'll maybe want to increase that number in the future).

Child Tickets

Change History (12)

comment:1 Changed 10 months ago by boklm

To upload the files there, I am planning to use a specific ssh key. It would be nice to be able to restrict what this ssh key can do, in authorized_keys, for example using rrsync (/usr/share/doc/rsync/scripts/rrsync.gz in the debian rsync package).

comment:2 Changed 10 months ago by sysrqb

Keywords: TorBrowserTeam202001 added; TorBrowserTeam201912 removed

comment:3 Changed 9 months ago by pili

bumping this one :)

comment:4 Changed 9 months ago by anarcat

gotcha, pili. i talked about this with weasel quickly today and we might be able to accomodate you on the new static-master infrastructure he deployed a few months ago.

the checklist is:

  1. create a role in LDAP, if missing
  2. create a new static-source host instead of shoving everyone in the static-master (nightlies-source.tpo?)
  3. give the role group access to the role user. create authorized_keys file writable by role user on the source.
  4. add a static component, publishing to the web-fsn-* mirrors, with static-fsn-master as ... well, a master. :)

I'm not exactly sure how step 4 works here, but we'll see...

Last edited 9 months ago by anarcat (previous) (diff)

comment:5 Changed 9 months ago by anarcat

so i checked, and there's already a tbb-nightlies role in LDAP. also, weasel wants to create a separate host for this (step 2) which requires setting up a new virtual machine. i've updated the checklist to reflect that better.

unfortunately, for the next steps i'll have to delay this a bit more. we're in the process of rearchitecturing the virtual hosting infrastructure and we're in this critical stage where we want to empty a old host before creating new machines, to free up some budget.

how urgent is this? can it wait a few weeks more?

sorry for the delays...

comment:6 Changed 9 months ago by pili

We can wait a few more weeks. Thank you for looking into it.

comment:7 Changed 8 months ago by pili

Keywords: TorBrowserTeam202002 added; TorBrowserTeam202001 removed

Moving tickets to February

comment:8 Changed 8 months ago by weasel

How user-facing is this service?

I ask because if it's user-facing, we want to put it on somewhat-HA mirroring, whereas if it's just internal we might not want that.

comment:9 Changed 8 months ago by pili

This service should be user facing.

We're running a user research testing program and we'll want end users and user research coordinators (and interested others!) to be able to easily download nightlies.

somewhat-HA sounds good, it doesn't need to be super-HA ;)

comment:10 Changed 8 months ago by weasel

Resolution: fixed
Status: newclosed

https://nightlies.tbb.torproject.org/ is now a thing.

To upload, as tbb-nightlies, put things into tbb-nightlies-master:/srv/tbb-nightlies-master.torproject.org/htdocs and run static-update-component nightlies.tbb.torproject.org

boklm should have sudo access to that user.

There also is an /etc/ssh/userkeys/tbb-nightlies. You can put ssh authorized_keys lines in there. However, tpo policy is that only command-locked keys (i.e. with a command=".." thing) should exist. Also, please use restrict and ideally from= lock the keys also.

comment:11 in reply to:  10 ; Changed 8 months ago by boklm

Replying to weasel:

https://nightlies.tbb.torproject.org/ is now a thing.

Thanks!

To upload, as tbb-nightlies, put things into tbb-nightlies-master:/srv/tbb-nightlies-master.torproject.org/htdocs and run static-update-component nightlies.tbb.torproject.org

boklm should have sudo access to that user.

There also is an /etc/ssh/userkeys/tbb-nightlies. You can put ssh authorized_keys lines in there. However, tpo policy is that only command-locked keys (i.e. with a command=".." thing) should exist. Also, please use restrict and ideally from= lock the keys also.

Should I be using /usr/local/bin/staticsync-ssh-wrap, or something else to restrict rsync access?

I tried with command="/usr/local/bin/staticsync-ssh-wrap nightlies.tbb.torproject.org" in /etc/ssh/userkeys/tbb-nightlies.

Then I tried running rsync like this:

$ rsync --safe-links -lrtHz  /some/directory/. tbb-nightlies-master:/srv/tbb-nightlies-master.torproject.org/htdocs/.

But I get the following error:

This rsync command (nightlies.tbb.torproject.org --server -lHtrze.iLsfxC --safe-links . /srv/tbb-nightlies-master.torproject.org/htdocs/.) not allowed.

So it looks like staticsync-ssh-wrap only allows rsync to read from this directory, but not to write to it.

Should I be using /usr/share/doc/rsync/scripts/rrsync instead, or is there something else?

comment:12 in reply to:  11 Changed 8 months ago by boklm

Replying to boklm:

Should I be using /usr/share/doc/rsync/scripts/rrsync instead, or is there something else?

So I used /usr/share/doc/rsync/scripts/rrsync, by copying it as /home/boklm/tbb-nightlies/rrsync, and it seems to be working.

Note: See TracTickets for help on using tickets.