Opened 6 months ago

Last modified 6 months ago

#34122 assigned project

Create two Tor Browser build machines

Reported by: sysrqb Owned by: hiro
Priority: Medium Milestone:
Component: Internal Services/Tor Sysadmin Team Version:
Severity: Normal Keywords: tap-roadmap-may
Cc: gk Actual Points:
Parent ID: Points:
Reviewer: Sponsor:

Description

Currently, Tor Browser developers have access to one external machine for building Tor Browser. We'd like two new build machines that are maintained by TPA. This will allow us to run parallel builds, and confirm reproducibility of the resulting builds.

The resource requirements for the machines are quite large:

  • For storage: 200GB should be an okay starting point
  • For memory: we'll need at least 16 GB.
  • For CPUs: at least two, but more would be better

The package requirements are documented here:
https://gitweb.torproject.org/builders/tor-browser-build.git/tree/README#n20

apt-get install libyaml-libyaml-perl libtemplate-perl \
                  libio-handle-util-perl libio-all-perl \
                  libio-captureoutput-perl libjson-perl libpath-tiny-perl \
                  libstring-shellquote-perl libsort-versions-perl \
                  libdigest-sha-perl libdata-uuid-perl libdata-dump-perl \
                  libfile-copy-recursive-perl libfile-slurp-perl git runc \
                  mercurial

Currently, the default Tor Browser build system (tor-browser-build) requires the user have (essentially) full sudo permissions (#23631) due to its underlying use of runc for creating deterministic build environments.

Child Tickets

Change History (4)

comment:1 Changed 6 months ago by gk

Cc: gk added

Note that the machine we currently have (which is behind build-sunet-a.torproject.net) could be one of those machines.

I think using the specs from the build-sunet-a.torproject.net is a good starting point. I think we don't want to have less than 4 cores and 8GiB memory is good, too (as a minimum). 200GiB disk might be a bit tight, so 500 GiB would be good.

comment:2 Changed 6 months ago by anarcat

Currently, the default Tor Browser build system (tor-browser-build) requires the user have (essentially) full sudo permissions (#23631) due to its underlying use of runc for creating deterministic build environments.

How open are we to changing how that works? How hard is changing that component, in other words?

I ask because runc and friends have moved quite a bit in recent years, and there is now the possibility of building and running containers (the latter is what runc does, essentially) as regular users (AKA "rootless containers"). In particular, buildah and podman are drop-in Docker replacements that can do that.

Therefore, if "creating deterministic build environments" is the goal, maybe we can look at podman and friends first?

I see some of those ideas were mentioned in #23631 but i figured i would bring them back in scope here first...

comment:3 Changed 6 months ago by hiro

Keywords: tap-roadmap-may added
Owner: changed from tpa to hiro
Status: newassigned

comment:4 in reply to:  2 Changed 6 months ago by gk

Replying to anarcat:

Currently, the default Tor Browser build system (tor-browser-build) requires the user have (essentially) full sudo permissions (#23631) due to its underlying use of runc for creating deterministic build environments.

How open are we to changing how that works? How hard is changing that component, in other words?

I ask because runc and friends have moved quite a bit in recent years, and there is now the possibility of building and running containers (the latter is what runc does, essentially) as regular users (AKA "rootless containers"). In particular, buildah and podman are drop-in Docker replacements that can do that.

Therefore, if "creating deterministic build environments" is the goal, maybe we can look at podman and friends first?

I see some of those ideas were mentioned in #23631 but i figured i would bring them back in scope here first...

Yes, we are totally open for doing that (just to reply here as well as in #34176). We won't have time to do so until October this year, though. But I think we should get that on our agenda for October, in particular if that helps to convince TPA to maintain the machines/OSes.

Note: See TracTickets for help on using tickets.