Add LXC support for building with Debian guest VMs
Now that #15578 (moved) is fixed we need to add LXC support for building with Debian guest VMs as well.
Activity
Is there a estimate/roadmap on the release of 6.0a1? I saw in #15578 (moved) that you're aiming to use Debian with Gitian for that release. I am just wondering how soon this is needed.
Trac:
Username: josephbisch
Replying to josephbisch:
Is there a estimate/roadmap on the release of 6.0a1? I saw in #15578 (moved) that you're aiming to use Debian with Gitian for that release. I am just wondering how soon this is needed.
6.0a1 is coming out next week and has the changes in #15578 (moved). But 6.0 will get out end of May.
-
I have something working (with gitian-builder in general, I haven't tried it with TBB): https://github.com/devrandom/gitian-builder/pull/114.
Trac:
Username: josephbisch 
Replying to josephbisch:
I have something working (with gitian-builder in general, I haven't tried it with TBB): https://github.com/devrandom/gitian-builder/pull/114.
I will try it with TBB and let you know how that works. Thanks!
-
The bug_18127 branch on my tor-browser-bundle.git repo has a few fixes: https://gitweb.torproject.org/user/boklm/tor-browser-bundle.git/commit/?h=bug_18127
The bug_18127-v2 branch in my gitian-builder repo contains our patches rebased on master, the patch from PR 114, and a few other fixes: https://github.com/boklm/gitian-builder/commits/bug_18127-v2
Using those changes I have been able to build Tor Browser 6.0a1 with both KVM and LXC, and the binaries produced were identical to the released 6.0a1 builds (except for the Browser/TorBrowser/Docs/sources/bundle.inputs which contains the gitian-builder.git commit hash).
Trac:
Status: new to needs_review -
Could you test your gitian-builder branch with KVM builds as well? In particular, if we merge your branch with our tor-browser-builder-3 I'd like to be sure that both our stable builds and our alpha builds are still working with KVM: from VM creation to building. My reasoning here is that we so far only cherry-picked particular commits from gitian-builder master and I fear there might be commits we now pull in that work fine with LXC but break our KVM builds.
-
However I only tried this branch with a build of the alpha, not the stable release. I don't know if having this in the tor-browser-builder-3 could cause problems in the release builds.
I see that we checkout tor-browser-builder-3 in fetch-inputs.sh. Is there a reason for doing a checkout of this branch and not of $GITIAN_TAG ?
-
Replying to boklm:
However I only tried this branch with a build of the alpha, not the stable release. I don't know if having this in the tor-browser-builder-3 could cause problems in the release builds.
I see that we checkout tor-browser-builder-3 in fetch-inputs.sh. Is there a reason for doing a checkout of this branch and not of $GITIAN_TAG ?
Well, we do in fact check out the
GITIAN_TAGfor the non-nightly builds viaverify-tags.shbut agreed this is a strange construction and, now that I remember, could haven been responsible for ln5's issues with switching to Wheezy for Linux. Could you make a change for this too (in this ticket is fine)? Because I am thinking about taking your rebased branch as a clean, newtor-browser-builder-4we are going to use from now on and it would be good to do the right thing withGITIAN_TAGas well. -
I added a new commit in the branch bug_18127 to use $GITIAN_TAG in fetch-inputs.sh: https://gitweb.torproject.org/user/boklm/tor-browser-bundle.git/commit/?h=bug_18127&id=19570316c67f693d28b8e7dc2cd1edc732a3327b
-
Some things I noted while testing the setup:
I hit:
W: Cannot check Release signature; keyring file not available /usr/share/keyrings/debian-archive-keyring.gpgwhile creating the Linux images which scares me. Can we do something about it? Is that an issue?
Further, upon starting the build I always get a bunch of
base-wheezy-i386 already exists, please remove it first base-wheezy-amd64 already exists, please remove it firstmessages. Do we need to do something to get rid of them?
Finally, I've looked at the LXC tips section in the README.md but could not find a hint why we need sudo privs now to create the images. This is not an issue with the tor-browser-builder-3 branch. Any ideas?
Trac:
Status: needs_review to needs_information -
Replying to gk:
Some things I noted while testing the setup:
I hit: {{{ W: Cannot check Release signature; keyring file not available /usr/share/keyrings/debian-archive-keyring.gpg }}} while creating the Linux images which scares me. Can we do something about it? Is that an issue?
Ah, I see that Ubuntu has a debian-archive-keyring package which might fix this. So we probably want to add it to check-prerequisites.sh.
Further, upon starting the build I always get a bunch of {{{ base-wheezy-i386 already exists, please remove it first base-wheezy-amd64 already exists, please remove it first }}} messages. Do we need to do something to get rid of them?
It seems the image files don't have a .qcow2 extension anymore. So we need to update gitian/make-vms.sh to remove the .qcow2.
Finally, I've looked at the LXC tips section in the README.md but could not find a hint why we need sudo privs now to create the images. This is not an issue with the tor-browser-builder-3 branch. Any ideas?
In tor-browser-builder-3, sudo was used to call vmbuilder. In the new version the same thing is done without vmbuilder, but with different sudo calls to debootstrap, mount, cp, rm. So it is less easy now to allow only specific sudo calls.
If we want to fix that, we can move all the commands from bin/make-base-vm to create an image in the LXC case to a separate file.
-
Replying to boklm:
Replying to gk:
Some things I noted while testing the setup:
I hit: {{{ W: Cannot check Release signature; keyring file not available /usr/share/keyrings/debian-archive-keyring.gpg }}} while creating the Linux images which scares me. Can we do something about it? Is that an issue?
Ah, I see that Ubuntu has a debian-archive-keyring package which might fix this. So we probably want to add it to check-prerequisites.sh.
Maybe we should also patch the debootstrap call in bin/make-base-vm, to add a --force-check-gpg to make sure it fails when the keyring file is missing.
I think we have a similar problem with KVM builds where vmbuilder is used, which calls debootstrap too.
-
The branch bug_18127-v2 has a fix for the first 2 issues: https://gitweb.torproject.org/user/boklm/tor-browser-bundle.git/log/?h=bug_18127-v2
I'm looking at the options for the 3rd one.
-
Replying to boklm:
In tor-browser-builder-3, sudo was used to call vmbuilder. In the new version the same thing is done without vmbuilder, but with different sudo calls to debootstrap, mount, cp, rm. So it is less easy now to allow only specific sudo calls.
The change from vmbuilder to debootstrap was done with this commit: https://github.com/devrandom/gitian-builder/commit/af56f89a6acffd363c845a489ec163f0d85d30be For this ticket: https://github.com/devrandom/gitian-builder/issues/86
I'm not sure what is the best way to fix this problem. The different solutions I can see to fix this are:
- revert the change to move from vmbuilder to debootstrap. But looking at the ticket it looks like we will have problems because of the kernel and grup packages installed by vmbuilder, so this probably implies patching vmbuilder too.
- extract the sudo commands from make-base-vm and put them in a script in a directory such as /usr/local/sbin that we add to sudoers, then patch make-base-vm to use this script with sudo if it exists.
- giving sudoers access to build users to debootstrap, mount, umount, cp, rm commands which is similar to giving full sudoers access
-
Hrm. There is no clear winner for me in the options you collected. Maybe Mike has an idea/preferred solution? I am not concerned with respect to requiring sudo for container creation in general. Rather, our LXC machine usage benefits from not needing that.
Trac:
Status: assigned to needs_information
Cc: josephbisch, boklm to josephbisch, boklm, mikeperry -
Regarding our LXC machine, we currently have the following sudo access:
(root) NOPASSWD: /usr/bin/lxc-execute (root) NOPASSWD: /usr/bin/lxc-start (root) NOPASSWD: /usr/bin/vmbuilder (root) NOPASSWD: /sbin/kpartx (root) NOPASSWD: /bin/cp (root) NOPASSWD: /bin/chownSo what we would need to add is debootstrap, mount, umount, rm.
-
Replying to boklm:
Regarding our LXC machine, we currently have the following sudo access: {{{ (root) NOPASSWD: /usr/bin/lxc-execute (root) NOPASSWD: /usr/bin/lxc-start (root) NOPASSWD: /usr/bin/vmbuilder (root) NOPASSWD: /sbin/kpartx (root) NOPASSWD: /bin/cp (root) NOPASSWD: /bin/chown }}}
So what we would need to add is debootstrap, mount, umount, rm.
I wonder if we would need
lxc-starton the LXC machine anymore. That said do we only need them for VM creation like if we are in KVM mode? I am inclined to merge the patches and find a good way of dealing with sudo problems later when needed. -
Okay, this is
tor-browser-buidler-4in thegitian-builderrepo and commit 4d625212a6689ebd1b291c5450096aeedad94e28, 9e1ff5bf59a07f9a4277d75e2331c47f297a6921 and 1bc9c54d1eb82b3da499d49fe9c54e0675b0001b on tor-browser-bundle master.Trac:
Status: needs_information to closed
Keywords: TorBrowserTeam201603 deleted, TorBrowserTeam201603R added
Resolution: N/A to fixed -
< GeKo> hm.. what was the reasoning for setting GITIAN_HOST_IP=10.0.3.1? < GeKo> it seems setting it to 10.0.2.2 as it is on devrandom's master branch works for my kvm box < GeKo> while setting it to 10.0.3.1 does not < GeKo> aha! see devrandom's make-base-vm: "10.0.2.2 is the host IP as visible from the guest under qemu networking"
Trac:
Status: closed to reopened
Resolution: fixed to N/A mentioned in issue #18181 (moved)
mentioned in issue #18690 (moved)
mentioned in issue #18786 (moved)
mentioned in issue tpo/applications/tor-browser#18690 (closed)
