Tor Browser Nightly external server support

added component::internal services/tor sysadmin team owner::tpa priority::medium severity::normal status::new type::task labels

Or should TPA be involved in this process and help manage/maintain the OS?

It depends if you want TPA to actually manage the machine. From what I understand from the discussion in #34122 (moved), you absolutely need root to run those builds, and we don't usually give out root to TPA-managed machines. That (and the lack of hardware, of course) is what's making us hesitant in setting up that machine. If there's an horizon after which that requirement could eventually change, we could be more open to changing that constraint, but it doesn't seem like there is an opening there...

In other words, if "run builds as root" is a hard requirement that will never change, I don't think we can help you with a TPA machine. That, of course, can be discussed, and that's what I tried to do in #34122 (moved) but that didn't seem to stick. :)

This does mean that if you setup the machine outside of TPA architecture, we cannot manage it either.

I should warn that previous experiments in that area didn't go so well. From what I understand, the current build box at sunet is severely out of date and not really maintained, so that is something that should be avoided in the future.

Replying to anarcat:

Or should TPA be involved in this process and help manage/maintain the OS?

It depends if you want TPA to actually manage the machine. From what I understand from the discussion in #34122 (moved), you absolutely need root to run those builds, and we don't usually give out root to TPA-managed machines. That (and the lack of hardware, of course) is what's making us hesitant in setting up that machine. If there's an horizon after which that requirement could eventually change, we could be more open to changing that constraint, but it doesn't seem like there is an opening there...

I think there is and we should try to avoid maintaining the machine/OS outside of TPA if we can help it. While we don't have the time to work on this until end of September, would a soft commit for October (this year :)) work for you? (That is a soft commit to work on a fix for #23631 (moved)/#31996 (moved)) We are in maintenance mode for Tor Browser and the you-need-root-to-be-able-to-build-Tor-Browser-issue is high enough on our list of things to fix, I think, to make maintenance easier. So this is fair game. What do you think?

In other words, if "run builds as root" is a hard requirement that will never change, I don't think we can help you with a TPA machine. That, of course, can be discussed, and that's what I tried to do in #34122 (moved) but that didn't seem to stick. :)

This does mean that if you setup the machine outside of TPA architecture, we cannot manage it either.

I should warn that previous experiments in that area didn't go so well. From what I understand, the current build box at sunet is severely out of date and not really maintained, so that is something that should be avoided in the future.

I totally agree. Hence my hope we can get that box under TPA maintenance as well.

Trac:
Cc: N/A to gk

What do you think?

I need to talk with the team about this, we have the change of guard tomorrow, let me see if I can poke at this problem again then.

What, exactly, do you need to run as root? Could we encapsulate only a subset of the build to run as root?

How about if we give you the ability to run Docker containers?

Replying to anarcat:

What do you think?

I need to talk with the team about this, we have the change of guard tomorrow, let me see if I can poke at this problem again then.

To be clear, the current plan is we (Tor Browser devs) get an external server and we maintain it ourselves, in the short-term. I'll take on this responsibility. At some time in the future (but not too far in the future), TPA receive additional budget for adding a machine large enough for building Tor Browser Nightly. We then migrate the nightly build setup onto the new server and cancel the old machine.

What, exactly, do you need to run as root? Could we encapsulate only a subset of the build to run as root?

Not easily in the current architecture of the build system. The system alternates between configuring a clean build environment (and installing dependencies) and then building the component.

Currently, the build system runs the following programs as root (in addition to executing dynamically created build (shell) scripts at run-time):

sudo tar
sudo ip netns add
sudo ip netns exec
sudo runc run
sudo ip netns delete
sudo mkdir
sudo cp
sudo chown
sudo rm
sudo runc --version
sudo id
sudo useradd
...

How about if we give you the ability to run Docker containers?

I've never tried runc-in-docker, but maybe? ticket:23631#comment:2 describes some problems with directly using Docker.

To be clear, the current plan is we (Tor Browser devs) get an external server and we maintain it ourselves, in the short-term. I'll take on this responsibility. At some time in the future (but not too far in the future), TPA receive additional budget for adding a machine large enough for building Tor Browser Nightly. We then migrate the nightly build setup onto the new server and cancel the old machine.

Works for me! :)

Currently, the build system runs the following programs as root (in addition to executing dynamically created build (shell) scripts at run-time):

{{{ sudo tar ... }}}

... game over! might as well just give you root at that point...

I've never tried runc-in-docker, but maybe? ticket:23631#comment:2 describes some problems with directly using Docker.

Thanks for the pointer! I admit I haven't read the entire thread so I hadn't caught that. I'm surprised to hear Docker can't run inside a i386 machine... but as for the container escapes, the point would be that the container would run as as a regular user so that you wouldn't need to do all that pesky runc and netns stuff yourself. ;)

But of course, I have no idea what I'm talking about here... That build seems to do some hairy stuff that maybe I will be happier not knowing about until I need to. :)

Replying to anarcat:

What do you think?

I need to talk with the team about this, we have the change of guard tomorrow, let me see if I can poke at this problem again then.

Thanks, appreciated. Even though it seems we have a plan forward for the nightly server there is still the current build box we'd like to see maintained by TPA.

there is still the current build box we'd like to see maintained by TPA.

which box is that?

Replying to anarcat:

there is still the current build box we'd like to see maintained by TPA.

which box is that?

build-sunet-a.torproject.net

build-sunet-a.torproject.net

do you require root on that box?

Replying to anarcat:

build-sunet-a.torproject.net

do you require root on that box?

Yes. It's the same problem we have as with the proposed nightly machine in this ticket and would be solved, too, by fixing #23631 (moved).

Hi all,

I understood, after our first meeting, that there was no rush in migrating this build machine at this point and that the long term plan was to find a way to have in run under our infra.

Why has that changed?

Replying to hiro:

Hi all,

I understood, after our first meeting, that there was no rush in migrating this build machine at this point and that the long term plan was to find a way to have in run under our infra.

Why has that changed?

I am not sure if something has changed here. This falls into the "time in the future (but not too far in the future)" camp which sysrqb mentioned, that's all. I was mainly responding to anarcat about the unmaintained build box and was asking whether a soft commit to fix the root problem would change the adoption of TPA maintenance timeframe-wise both for the nightly build machine and the one we currently build on as both setups share the same underlying problem.

considering the situation of the sunet box, i would rather retire it completely and build a new one rather than try to fix it.

Tor Browser Nightly external server support

Child items 0

Activity