Installing and Configuring a Virtual Build Environment
Welcome to the Semiofficial Tor Browser Build Virtualization Intro. This page gives useful indications of requirements and recommends virtual machine (VM) configuration most likely to result in a successful build. It covers host and guest environment (VM configuration) as well as a recipe of steps taken to install and configure a sample Ubuntu LTS AMD64 desktop edition OS in a guest VM.
Why use a virtual machine to build TBB?
Even developers coincidentally using the prescribed long term support (LTS) version of Ubuntu on the correct AMD64 architecture benefit from building on a VM guest. When changing to another computer, a virtual machine based build environment can simply be copied to the new computer and reused with no changes. The virtual image can be copied to several machines participating in a distributed cloudcomputer as well.
Obviously, users of non DPKG based Linux distributions and all Unix (BSD and SVR4) distributions cannot natively build TBB using the gitian build scripts and are forced to look for alternatives (like virtual machines) to build TBB on.
Why not use a virtual machine to build TBB?
Building in bare metal development environments benefits from slightly (maybe 15%-20%) better performance and thus decreases build durations.
How about using a chroot(8) instead?
Installing the build environment to a chroot(8) could provide some important advantages like leveraging seamless storage and memory between the host and jailed client to meet the build environment's requirements. Unfortunately this solution requires LXC (just like a virtual machine) but even worse, chrooted jails share the networking configuration of their hosts, requiring extra networking steps (like configuring a lxcbr0 bridge) to be carried out. It's probably possible to fulfill these requirements manually (configuring both from outside of and within the chrooted jail), but this is certainly not as straightforward as native or virtualized building.
tor-browser-bundle/gitian$ USE_LXC=1 make build
Extracting partition for lxc
Must configure lxcbr0 - check the README
i386 lucid VM creation failed
make: *** [build] Error 1Host Requirements
Building TBB on VM guests requires a number of computing resources. This document depends on research done exclusively using the VirtualBox Opensource edition, but any competing virtualization platform will likely produce equally valid results.
Minimum Guest Requirements
Virtualized TBB builds use Linux Containers (LXC) to abstract the different platform bundles, so AMD64 host hardware is required. This means the real brick-and-mortar CPU in use must support the AMD64 instruction set. Respectively, the operating system image installed to the VM guest must be 64-bit as well. Any OS based on the Debian package management system (DPKG) can be used as long as it provides the components specified in TBB build scripts. These include apt-get(1), apt-cacher-ng(1), and lxc(7) among many others. Ubuntu LTS releases since Lucid (12.04) include the required components by default.
| Requirement | Level |
|---|---|
| Memory (RAM) | 3 Go |
| Storage (Disk) | 34 Go |
| Swap | None |
| Network | IP4 |
| CPUs | 1 |
| Instruction set | AMD64 |
| VT-x/AMD-V acceleration | No |
| Graphics system (X11/GDI) | None |
Note that increasing some requirements (like swap) improves build speed or reliability. Other requirements (like graphics) relate to developer preference, and most will enjoy building more by installing a desktop edition (like Ubuntu desktop and X11 or Gnome.) Most important, configuring a graphics system or installing resource hungry applications will have a negative effect on other requirements like RAM. For this reason, it is advisable to increase RAM to 4 Go when installing a graphical desktop environment like Unity, Gnome, or KDE in the VM guest.
Note that a number of relevant services like routing, DNS, and DHCP (or other network configuration) may be technically required but not listed here.
When building using gitian scripts, most of these requirements are automatically discovered with unmet requirements printed to the screen.
Upstream Minimum Requirements
The Tor Browser Bundle (TBB) is composed from binaries built from several bundled projects like Firefox, OpenSSL, Tor, and Torbutton, each requiring different build resources. The Firefox component is by far the most resource intensive of these, and as such it's useful to read the requirements from Mozilla's Firefox build instructions.
Manipulating Minimum Requirements
While unmet requirements surely lead to build failure, some interesting strategies exist. For example, it's conceivable to start building TBB in a VM guest using 1 or 2 Go RAM. Expect a build failure and when it comes restart the VM guest after increasing its RAM to 3 Go (or use dynamic memory ballooning.) A renewed call to make(1) build will succeed. Only a few critical portions in TBB build stages actually depend on the stated memory requirement, so this value can be manipulated if necessary.
Furthermore, VM guest storage can be reduced to the minimum OS requirement (often 6-8 Go) when all TBB build actions (git clone, apt-get install, vm creation, make build, etcetera) are carried out inside a shared folder. This uses storage of the VM host rather than the VM guest and can be used in parallel multibuild (of different TBB versions) strategies, for example.
Sample Configuration 1 (minimal, for advanced users)
A realistic build environment consists of the following VM guest configuration:
- Ubuntu 12.04.4 LTS AMD64 server edition (not using MAAS)
- 36 Go SATA SSD Disk (VDI format static allocated blocks)
- Split in two primary partitions (34 Go root, 2 Go swap)
- 4 Go RAM
- 64-bit
- 2 CPUs
- 1 bridged network adapter
Once this guest boots and installs, carry out typical maintenance like:
- Update packages to current state (apt-get update && apt-get upgrade)
- Install guest additions for shared folder support if desired
- Disable screen blanking and screensaver (useless in a VM)
- Modify sudoers(5) to allow passwordless entry
- For example: sed -i -e 's;(.*) *ALL$;\1 NOPASSWD: ALL;' sudoers
- Start a script(1) file for later examination when debugging
Sample Configuration 2 (graphical, for novice users)
Configuration Tips
Remember that the stated minimum storage TBB build requirement regards the actual filesystem rather than total VM disk size which could be larger due to swap and other partitions.
Manually partition the fresh VM when installing the guest OS. You can get away with less than the recommended swap space in most cases, reducing the resources needed to build TBB and increasing portability (easier to copy) of the VM.
Configure as many virtual CPUs as the host computer has real cores. Ignore hyperthreading, which sometimes deceives and leads to more CPUs on system monitor graphs than there really are.
If installing the Ubuntu desktop edition, consider activating 3D acceleration on the virtual graphics adapter. Then install and enable guest additions in the guest OS to take advantage of it.
If installing the Ubuntu server edition, consider configuring the VM with bridged networking instead of NAT. By doing so, a SSH service (and other services) can be accessed from outside the VM, facilitating interaction with the VM's interfaces. In other words, you can connect to your VM from the host computer (or any other with a route to the VM) in order to do your TBB work.
If installing the Ubuntu server edition, consider configuring for MAAS or use Juju to manage the VM. This paves the way to distributed and elastic computing, which provides the (multiple node) build farm with improved scalability.
What To Do Once the Guest Is Ready
Why, build TBB of course!
Instead of providing guidance here for TBB engineering, please review the hacking wiki for more information.
To get to know the gitian build scripts and how to manipulate and debug individual build stages, please review the gitian wiki for more information.
To understand how (and why) gitian provides for deterministic building, please review deterministic build logic for details.
Troubleshooting the Guest Environment
Developers with newer systems like Ubuntu 14.04 (13.10 too) AMD64 desktop will likely meet with problems relating to hardcoded paths in build scripts. For example, Ubuntu 14.04 distributes a different version of the automake package. To correct this, add a symlink to the guest environment:
cd /usr/share && sudo ln -s automake-1.13 automake-1.11Upstream Gitian Documentation
Additional help can be obtained by appeals to #bitcoin-dev (irc.freenode.net) or a number of online resources:
Next Steps in Virtualization
The intention of this guide is to provide novice VM developers with the tools needed for a quick start in building TBB. More importantly, it paves the way to research of more efficient build environments. Future build farms should be extremely scalable, remotely accessible, and hopefully portable to standards conformant cloud providers. To this end, look for related documents describing virtualization and automated logic in build scripts.