wiki:doc/TorBOX/Dev/ArchivedDiscussion/SHELLSCRIPTS

Main Article - TorBOX

  1. SHELLSCRIPTS
    1. front page - how to install: download, build/shell script, manual …
    2. Shell Scripts: simplifying testing/updating/expanding [SHELLSCRIPTS]
    3. VM footprint [ANSWERED] [SHELLSCRIPTS]
    4. SocksPort set to 0 in torrc [SHELLSCRIPTS]
    5. fix screen blacks out after grub [SHELLSCRIPT] [REJECTED: UPSTREAM BUG]
    6. Separate security related parts, addons, optimizations [SHELLSCRIPTS] …
    7. How do we keep TBB updated [SHELLSCRIPTS] [DONE]
    8. Application selection [SHELLSCRIPTS] [ANSWERED]
    9. TBB doesn't allow clicking on links in local files (i.e. readme.html) …
    10. I can't figure out how to preconfigure TBB to use socksport 9100 …
    11. The README.html [SHELLSCRIPTS] [CLOSED]
    12. Reduce image size [SHELLSCRIPTS]
    13. Switch to manual network configuration for NAT eth0 [SHELLSCRIPTS] [DONE]
    14. fs.file-max=100000 to in /etc/sysctl.conf on T-G [SHELLSCRIPTS] …
    15. Installation of Tor package done in any case no matter if key …
    16. Script command line options as a modules [SHELLSCRIPTS] [CLOSED?]
    17. Can the script autodetect Virtual Box? [SHELLSCRIPTS] [ANSWERED]
    18. tor-arm by default on T-G [SHELLSCRIPTS] [IMPLEMENTED]
    19. Small script coding style change [SHELLSCRIPTS] [ANSWERED]
    20. Remove hardcoded GPG public keys from T-W script [SHELL SCRIPTS] [DONE]
    21. Do we support bare metal without Virtual Box? [ANSWERED]
    22. Switch from Ubuntu server edition to Ubuntu (minimal console) [REJECTED]
    23. No preinstalled desktop environment / anonymous operating system …
    24. Using virtual box command line for fully automated builds? …
    25. thttpd replacement [SHELL SCRIPTS] [IMPLEMENTED]
    26. t-w.sh todos [SHELL SCRIPTS] [DONE]
    27. /usr/local/bin [SHELLSCRIPTS] [DONE]
    28. torsocks wrapper script / torsocks in T-W [SHELL SCRIPTS] [DONE ?]
    29. [SHELLSCRIPTS] Scroll wheel in VM [FIXED]
    30. [0.2] [SECURITY] TorBrowser two homepages issue / aos Readme issue [DONE]
    31. [0.2] torcheck script discussion [SHELL SCRIPTS] [DONE]
    32. [0.2] T-W kernel update [SHELLSCRIPTS] [DONE]
    33. [SHELLSCRIPTS] Move to file separated source [INFRASTRUCTURE] [DONE]
    34. [BUG] [SHELLSCRIPTS] default color profile for gnome-terminal [FIXED
    35. [SHELLSCRIPTS] Ubuntu unattended (automatic) installation debootstrap …

ArchivedDiscussion

SHELLSCRIPTS

front page - how to install: download, build/shell script, manual configuration [SHELLSCRIPTS]

  • (proper) I'd like to further tweak the front page. Therefore I suggest three ways. 1. download - binaries (for most end users) 2. using the build/shell script - the source code (for developers, advanced linux users, bare metal users, unsupported virtual machines, people who distrust) 3. manual configuration - the technical documentation (for developers, advanced linux users, interest people). Each site must offer all information needed for the chosen method.
  • (proper) Right now the first time user clicks on the front page, clicks to how to install, and then it's a very little mess. Not optimal. As of right now the shell script has also to much discussion inside (gpg keys, which is clarified one page before already...). Your opinion?
  • (anonymous) Sounds great! Shell script is fixed.
  • (anonymous) IMPLEMENTED? Anything missing?
  • (proper) Very well... The only thing which is slightly suboptimal is, that TorBOX/HowToInstall and TorBOX/ManualConfiguration have some redundancy. Until step "sudo apt-get install openssh-server" the pages are equal. First drawback of this is, if you edit one page, you also have to edit the other one. HowToInstall is the easy configuration and should contain as little background information as possible. ManualConfiguration should be fully technically justified.

Solution? Maybe we could delete the redundancy under ManualConfiguration and link to HowToInstall and tell 'do it until "sudo apt-get install openssh-server" and then come back here'. And tell on HowToInstall after "sudo apt-get install openssh-server" 'you can also do it manually, see ManualConfiguration'.
Let's image we get the spoiler plugin, that would allow us to hide the technical comments on HowToInstall and leave it available for interested people. (spoiler plugin is very likely to happen, the author said, it may be possible to use it, even without the plugin through an alternative)

  • (proper) What would maybe also confuse me, if you wouldn't know, that there is also a shell script for the Tor-Workstation. The user isn't informed how to use it. Perhaps it would be more logical to let the user install the Tor-Gateway first and install the Tor-Workstation after that. The EasyConfiguration / Tor-Workstation script could do more useful tasks in the future, such as downloading and configuring TorBrowser for TorBOX.
  • (anonymous) Fixed all suggestions. T-W script is pointless anyway because it's difficult to transfer to the VM (this would only entice users to do unsafe things), I moved it to Dev/ClientVM. I think we should just delete the whole Manual Config. It's pretty pointless IMO. It's only intended for power users and those can modify the shell script (which is pretty well commented, more than necessary I would think). There is the network test step which was missing in the how to but I've fixed that. There isn't much information left that is only available on manual but not on easy config. It's all pretty obvious.
  • (proper) I wouldn't have thought, that the shell script is taking over the project. =) You're right, your solution solves the redundancy quite elegant. There is no need to have ManualConfiguration and a ShellScript at the same time. Let's cannibalise ManualConfiguration, make it a scratch pad. Cut from ManualConfiguration and paste into shell script. IMO the shell script should contain all comments, explanations, reasoning, it's obvious to us, but not necessarily for other new developers who audit our project or want to contribute. Please don't delete anything, like "tor --verify-config" or "usewithtor...", these where all useful findings during development. Do you think we could continue the 'step layout' in the shell script (like "Step 3.5: Firewall Configuration" "Step 3.5a: Create Firewall script"), imo that improves readability and explains.
  • (anonymous) Sure, comments don't hurt. But I don't think we should run something like "tor -verify-config" when we already control the content of torrc and know that it's valid. usewithtor is already in the how to. Generally I'd say things belong into the how to if users should run them (or have to run them if things don't work), the rest can go into the script comments.
  • (proper) All right, agreed. Small clarification: things like "tor -verify-config", don't put it in the script, but also please don't let it get lost in nirvana -> scratch pad, as it was useful in development. Sidenote: thanks for fixing my typos. =) -> (proper) Yep, done very nicely.

Shell Scripts: simplifying testing/updating/expanding [SHELLSCRIPTS]

  • (proper) There are a few things which bug me while editing/testing. While the scripts are running, you don't see which command got executed. You only see the output. Can we do something about it?
    • (anonymous) add tons of echo '"doing this"', add 'sleep 5' if things scroll too fast or pipe to less.
    • (proper) Good idea. The sleep 5 can be replaced with a variable, 0 for productive use, higher volume for debugging. Perhaps we simply replace most of the '# comment' with echo 'comment'?
      • (anonymous) yes
        • (proper) The updates offer now helpful debug output.
  • (proper) Should we allow to re-run the scripts over and over again? Currently re-running results in bogus results (ex: tor repo multiple times in sources.lst, etc.)
    • (anonymous) you could add a clean up routine (in a separate script) like the one in t-g which gets called when the script fails (if you hit ctrl+c while it's still running you can savely rerun it already, but only without the -vm stuff). Obviously you can't auto-run it by default or the script will revert everything on exit and get nothing done and you can't test anything.
    • (proper) It would add an advantage: people who want to update could simply rerun the script, done.
    • (anonymous) but that doesn't allow you to test the full script, only the update part. These two things can't be combined.
    • (proper) If they can re-run the script one way or another, they could also activate/deactivate optional features.
  • (proper) Should we prevent SSH connections from breaking down?
    • (anonymous) if you know how, please. I don't think it's possible, especially not for t-w where the IP gets changed. T-G 'should' work (even with uninstalling openssh-server, it will be kept till you actually log out).
      • (proper) Tested T-G wiki version 96: SSH does not break down during -install or -update. Great!
    • (proper) Perhaps could add at the very end, "Press X to restart the services. Alternatively press Y to exit without restarting. (Note that T-G will not be functional without restarting the services.) To restart the services later, run the script later again with --restart or simply reboot your machine.)?
      • (anonymous) OK. people need to reboot anyway before being able to use it (internal network needs to be renamed...) Try removing ifdown -a and ifup -a and the script should work through ssh. UPDATE: IN T-W, not T-G!
  • (proper) Imho not done yet. I haven't done what you said above. And we haven't looked into T-W yet.
    • (anonymous) but i've done both already ;)
    • (proper) I am impressed, didn't notice yet.
  • (proper) If you run first without any options, a backup is made. Is this wanted? Has this any repercussions for the future script starts?
  • (proper) There is a bug when the script is run first without any options and with -install afterwards. apt_get() manages to add duplicates to /etc/apt/sources.lst.
    • (anonymous) yes, this is wanted, no, no repercussions, I can't reproduce the "bug".
      • (proper) I try to reproduce, describe and post back.
      • (proper) Okay, tested again with T-G wiki version 108. When I run -update twice, I end up with the Tor repos twice in sources.lst.
        • (anonymous) I see, should be fixed now.

VM footprint [ANSWERED] [SHELLSCRIPTS]

"Should we reduce Footprint of Tor-Gateway?"

  • (anonymous): Not really necessary: 34MB RAM usage with Tor active, 18 tasks running. Only tor and dhclient are listening on ports. I've removed dbus (and openssh-server) without obvious adverse effects so far. We probably want to keep a syslog, cron and atd. dhclient3 could be removed if we use static IP for the external interface. I can't think of anything else. We could trim disk footprint but disk space is so cheap. The gateway VM uses less than 700 MB, no swap is needed.
  • (proper): main motivation behind it is not disk space, CPU or RAM, but minimizing the attack surface is. I haven't looked into yet what kind of redundant packages the minimal virtual machine installation has.

SocksPort set to 0 in torrc [SHELLSCRIPTS]

  • (proper) Actually we do need port 9050 on Tor-Gateway (if we decide not to use something like polipo).
  • (proper) SocksPort 0 disables it.
  • (proper) Has to be researched if we need it later to allow hosting hidden services inside Tor-Workstation.
  • (anonymous) We could also think about optionally replacing TransListenAddress with SocksPort if one only uses TBB in Tor-Workstation for example. We'd lose the advantage of automatic application support but we'd still have the increased protection against application level exploits and leaks. Possible advantages: More secure (just one open port vs two, less code base involved? I don't know the internals of Tor. Would it be faster? This also has implications for time syncing(htp/ntp) and software updates (apt-get), see relevant tickets. But I think both applications support socks proxy.
    • (proper) That wouldn't be TorBOX, that would just be a Tor server in LAN. I strongly prefer the transparent proxy method, because of the advantage your mentioned. Dunno if it would be more secure. Theoretically yes, because at the moment SocksPort is much more widely in use, only a very few use TransparentProxy. TorBOX was supposed to push the TransparentProxy forward, as it was a complicated and not fully documented setup. There weren't good instructions on the TransParentproxy wiki site and many things weren't mentioned (start firewall before interface/Tor, TBB inside Tor-Workstation, and so many other things surrounding). So I am not sure, how we can combine all the different methods, like "use SocksPort instead of TransparentProxy" and "only one VM". For the two latter things, I'd rather prefer new wiki pages and new projects if someone really wants to do it. In the end we could also make an overview about all the different projects with it's pros and cons.
  • (anonymous) It wouldn't "just be a tor server on the lan", it would be exactly the same as every TorBOX configuration save for two one liner config changes on Tor-Gateway (torrc and firewall).
  • (proper) No socks support wouldn't be so bad anyway, torify, usewithtor or similar could be used.
  • (proper) Okay, I thought again about it, I got your point. We shouldn't exchange the transparent proxy feature in favor of classical sockify, TorBOX began as a guide how to implement the stuff mentioned on transparent proxy. But if you are adamant about adding it, feel free to add it in a non confusing way. I guess a new wiki site for this kind of approach is to much.
  • (anonymous) No harm in leaving as is: it's local host only. As for alternate setup, we can leave that for the future.

fix screen blacks out after grub [SHELLSCRIPT] [REJECTED: UPSTREAM BUG]

  • (proper) Tor-Gateway and Tor-Workstation will show a black screen after booting for a while. This kind of look and feel is not optimal. Better show all the console messages.
  • (anonymous) Maybe https://bugs.launchpad.net/ubuntu/+source/upstart/+bug/542666 I don't think we can do anything about it.
  • (proper) Outside the scope of this project.

Separate security related parts, addons, optimizations [SHELLSCRIPTS] [DONE]

  • (proper) We clearly mark all the different parts. Security related things (remove ntpdate etc.); addons (install XChat etc.); and optimizations (shutdown without root, remove unnecessary packages to slim down the system).
  • (proper) I personally don't need a slimmed down system. I enjoy command-not-found, fully features desktop environment, etc. Perhaps there are more people like me. I think for now adding command line options like -slimdown or -noslimdown are overkill, for my personal build, I just comment those out.
  • (anonymous) If not all users use exactly the same configuration for their "default" T-W (you can have more for less anonymous usage/make snapshots) they'll lose one of the benefits of TorBOX, we should not encourage that.
  • (proper) Yes, it has some disadvantages. I described those under Software installation on TorBOX's Tor-Workstation. My main motivation behind starting TorBOX was and still is, being able to use also applications, which are not so widespread along most users (developer tools, git, etc.), allowing other stuff than posting. Ex: Anonymous developing in the most secure manner etc. We don't encourage people to do it. We just do not make it harder on purpose. Actually we already 'almost' follow the desing proposal I made here. What I mean in concrete, is for example "remove problematic software, popularity-contest" is next to "remove ntpdate, canonical-census" and should not be in the same list as "remove unnecessary packages to slim down the system, xserver...".
  • (anonymous) We could need a new security section about VM usage: default clean VM for high anonymity stuff, then make a snapshot of it/clone it and use that for the not so common/pseudonymous stuff. I'm still convinced there should be exactly one default VM and it should be small but easy to expand. If you want command-not-found you can still install it anyway, takes a few seconds.
  • (proper) "new security section about VM usage": that makes sense.; "I'm still convinced there should be exactly one default VM and it should be small but easy to expand.": Yes. My proposal is only, to write down what we implicit do with the script already anyway (example above).

How do we keep TBB updated [SHELLSCRIPTS] [DONE]

  • Is there something like a torproject.org/download/latest-tbb.tar.gz?
  • Even with that in place, how to automatically update it AND verify the download?
    • the gpg key could be embedded in the script but I don't like that much either - the script itself isn't signed.
  • https://trac.torproject.org/projects/tor/ticket/5236
  • Thandy/TUF?
  • Again, we want/need delta updates
  • Working on a workaround if no one is willing to support this case. This will not be as reliable as it could be (assumes RecommendedTBBVersions format doesn't change and the bundle will be called *-dev* for example) Still needs testing. Should we add the key of Sebastian Hahn as well? He singed TBB when Erinn was away.
  • (proper) Yes. He also signs the Obfsproxy Tor Browser Bundle.
  • (proper) For the other scripts we are against putting the keys into an automatic script. Be here we break that concept and do it? Also that way the script looks somewhat full blown. Well, what as alternative? Isn't there a package with all the tor developer keys? Or shouldn't we rather educate the users to import all those keys in the first place?
  • (anonymous) I already broke that rule when putting a sha256 checksum in the t-w script for checking TBB. One reason: t-g is absolutely crucial to be trustworthy and secure (see T-W threat model). The next binary build will contain the script in t-w so only I (or whoever packages it) have to verify that the scripts here on the wiki are correct (which I always did). we could maybe replace it with the deb.keyring package but we'd still import at least one gpg key (the deb repository key), which can reasonably done manually same as in t-g.
  • (proper) Sounds good.
  • (anonymous) deb.torproject.org-keyring only contains the deb repository key not the keys we need.
    • (proper) Bad. Shall we an enhancement for such a package?
  • (proper) Now you imported the update script into the TGScript. And we have the same code in two places TorBOX/Update/TBB and /TorBOX/Dev/TGScript. I propose to delete TorBOX/Update/TBB and to add an command line option -tb, --tor-browser to update TB using TGScript.
  • (anonymous) that's an enhancement, more cosmetic than anything (changes can be quickly copy/pasted back as instructed on TB/Update) The real problem is that the script will most likely break in the future because it makes too many assumptions. Also, the script would need to be refactored quite a bit to introduce flags. The default should be "update TB" and the installer should require an --fresh-install flag so people don't accidentally run the wrong thing.

Application selection [SHELLSCRIPTS] [ANSWERED]

If you disagree with anything in this list, please comment!

  • file manager: pcmanfm
  • unarchiver: file-roller, supports zip rar and more
    • used xarchiver before, dropped because no longer maintained upstream
  • terminal: gnome-terminal
    • tried lxterminal before, missing features (warning when closing, can detach tabs)
    • roxterm: there's a segfault when merging the last tab of a window, beware! Switching to gnome-terminal
  • IRC: xchat, for now without plugins, uses it's own SOCKSPort
    • (proper: some issues) (anonymous: ident warning doesn't apply to us) (proper: It's more than that. Perhaps we can supply a pre-configured .conf files.) (anonymous: sure, could be added, do you have one?) (proper: I'll create a new page, post the standard configuration files and my changes versions.)
  • silc (anonymous: preconfigured would also be great but if it's too much work to do in the script we can leave that to the user)
  • image viewer: gpicview
  • thttpd
  • TorBrowser (patched TBB); uses it's own SOCKSPort
  • PDF Reader: evince
    • used epdfview: navigation sucks; apvlv and mupdf would be lighter but are keyboard only - not really suited for target audience.; proper: just use o your favorite one, we are not bound to low traffic anymore
  • Media player: gnome-mplayer (with working sound!)
  • text editor: in addition to terminal apps: leafpad

Missing stuff: (Please add here if you'd like to see certain programs!)

  • HTP/tlsdate [WAIT for upstream]
    • Link - we'll wait for them to become stable and supported upstream
  • IM [WAIT for tpo to fix upstream]
    • Which one to choose?
    • (proper:) cables | TorChat (needs some fix because of Transparent Proxy) | Pidgin + Jabber + OTR encryption (pro: plausible deniability and perfect forward secrecy; con: both must be online) | Miranda + Jabber + GPG encryption (pro: no need to be online, messages will be stored encrypted on the server; cons: no plausible deniability, no perfect forward secrecy
      • (anonymous) I think forward secrecy is worth having and trumps other advantages.
      • (proper) Which type of people will be our targeted users? The top targets can not use a centralized server, because such servers can be forced to block the account too easily. For these cables and TorChat are the only applications I know, unsure about their security, any others? Jabber is good for mode(2), also limited use for mode(1), if the pressure for the Jabber server administrator isn't too big. The decision is yours, as you take the main work with this. I just want to offer the pros and cons of the different possibilities.
      • (anonymous) I'm not really a friend of IM+Tor precisely because it invites combining mode 1 and mode 2 (IM in almost all cases is mode 2) For secure communication real time comms aren't a good choice anyway (most secure option is remailer/mixnet). That's why there still is no IM in TorBOX T-W. If someone else is willing to do the work (research, configs) I'll lend a hand with the scripting part if necessary.
      • (proper) Status is probable "wait for tpo to fix upstream". Tpo has still their IM "temporarily" discontinued; #1676; #5498.
  • VOIP: [REJECTED (for now)]
    • proper: 1. voice is very identifying (automatic speech recognition), you can not whistleblow over voice (no idea how safe voice obfuscaters are) , makes only sense with mode(2); 2. outside the scope of TorBOX, VOIP over Tor has been discussed on the mailing list, there is no suitable VOIP application for Tor, Skype does work quite well, but Skype can not be trusted, it's closed source and the Skype authority has the power over the encryption keys
  • e-mail Client: [REJECTED (for now)]
    • (proper: In my dreams we use Thunderbird + Enigmail (GPG). In reality there are issues we can not fix in a day. ApplicationWarningsAndNotes#E-Mail In meanwhile we recommend webmail.)
    • (anonymous: not really, see Liberte and Tails, using a email client doesn't create new risks apart from attack surface. However I have no interest in setting this up. If someone wants to, patches are welcome!) *(proper: See my reasoning for the web browser, it's quite bulletproof, no one argued about it. Liberte and Tails still use other browsers than Tor Browser, that doesn't mean much. Over time the knowledge and documentation improves. There are still people posting guides to using Bittorrent with Tor using socks4a or sockscap. They are very wrong, but that knowledge hasn't reached mainstream yet.)
    • (anonymous: yeah, but with torbox one can even use torrent safely. If a application in t-w can compromise the identity without user consent, i.e. doing stupid things like posting real name, we failed. fingerprinting, attack surface, clear text passwords - all things to worry about and reason to prefer webmail but these don't automatically result in a compromised ID but depends on how the user acts.)
  • GPG verify files (easy, graphical) [WAIT for upstream?] *Kleopatra
    • If we stick with webmail this is a must have, if we use Enigmail, not so much. For verifying files the command line is enough (all the gpg signed software I know has a documentation on command line usage).
    • Kleopatra has way too many dependencies (wants to pull in the whole KDE). Better suggestions?
    • (proper) Unfortunately I have found none. We'd need at least this features. Write (copy and paste) text, encrypt, decrypt, sign, verify. A keyring manager is not that hard to find.
    • (anonymous) gpa looks nice (it's used by Liberté) but there is a problem with a dependecy no longer being maintained upstream and the last release was in 2009 but gpa itself is not completely dead yet (http://git.gnupg.org/cgi-bin/gitweb.cgi?p=gpa.git). We may see a working release in the future.
  • (anonymous) Open new tickets for apps you want to see in tor-workstation

TBB doesn't allow clicking on links in local files (i.e. readme.html) [SHELLSCRIPTS] [CLOSED]

Using online readme.

I can't figure out how to preconfigure TBB to use socksport 9100 [SHELLSCRIPTS]

[CLOSED] https://trac.torproject.org/projects/tor/ticket/5322

No matter what I try in user.js it always reverts back to the default 9050, or it breaks completely.

Update: found a workaround, a bit complicated. Better options are welcome.

The README.html [SHELLSCRIPTS] [CLOSED]

...isn't too pretty. It does the following:

  • version info/changelog/development disclaimer
  • link to TorBOX Homepage
  • reminder to change password
  • instructions how to change keyboard layout
  • Notice about check.torproject.org falsely claiming that an update is available
  • remind to keep TB updated (and the rest of the system?)
  • tell people to use offline updates if they want large packages like libreoffice or gnome
  • Remind people not to use torrents

  • anything else?
  • That's an awful lot which users need to be aware of :(

Reduce image size [SHELLSCRIPTS]

Currently: T-W.ova: ~410MB T-G.ova: 190MB

  • List of all installed packages: http://pastebin.com/raw.php?i=KnR7rd3B (outdated)
  • Probably the following can be removed without adverse affects
    • aptitude command-not-found* mlocate parted rpm ufw geoip-database telnet sound-theme-freedesktop
  • Other potential targets:
    • fuse (removes ntfs support)
    • manpages man-db (is it necessary? not for me, need input on that)
    • bash-completion (very useful if you use the cli at all)
  • Debian halves the size of T-G and reduces T-W by a third...
    • doesn't seem necessary anymore

Switch to manual network configuration for NAT eth0 [SHELLSCRIPTS] [DONE]

  • (proper) I wonder if it's possible with VirtualBox to use manual network configuration. We could remove dhcp, it would solve known issue for #OptionalFeatureNr.1# and we could deploy a complete /etc/network/interfaces file. Therefore I noted all information from ifconfig and tested with the following: (removed draft with errors) Can't bring up eth0 with that config anymore.
  • (proper) Is VirtualBox dependent from the gateway/DNS form the physical hardware router? Doesn't VirtualBox abstract the network layer and implement their own virtual DNS server and virtual hardware gateway?
  • (anonymous) implemented, see script. can only be enabled for -vm because the IPs will differ on bare metal
  • (proper) What is used as DNS, resolv.conf?
    • (anonymous) the vbox gateway doesn't have it's own dns server, instead it just relays to whatever the host system uses and tells the guest about it. resolv.conf is still used and not redundant. the only way to get that info from the guest seems through dhcp.
  • (proper) Looks like you also made /etc/resolv.conf redundant.
    • (anonymous) no, Linux needs that file and sadly without dhcp it's no longer regenerated, I broke it and I have no idea how to fix this, except to switch back to dhcp.
  • (proper) For now it's working for me. I still wonder how the name and IP of my router could leak into T-G resolv.conf. As soon as I removed +i from resolv.conf, my router name is inside again.
    • (anonymous) you mean it's working with a 127.0.0.1 option? in that case we can only enable static ip for that optionalfeaturenr.1
    • (proper) I am using #OptionalFeatureNr.1#. Currently working without any resolv.conf. Resolv.conf does not get regenerated because resolvconf is uninstalled and all interfaces have a static interface.
  • (proper) Will it work with different router lan ips and also over other connections such as 3g?
  • (proper) Might #OptionalFeatureNr.1# leak DNS now?
    • (anonymous) with fair confidence: no.
    • (proper) Agreed. Test wise removed DNS redirection rule. Result: nslookup and apt-get can no longer resolve. Fine.

fs.file-max=100000 to in /etc/sysctl.conf on T-G [SHELLSCRIPTS] [IMPLEMENTED]

  • (proper) I recently saw the Tor warning "Warning: Your system has very few filedescriptors available in total.", haven't researched that warning yet. Should we do it?
  • (anonymous) never saw that. what did you do to hit that limit?
  • (proper) If I'd knew, that'd be great. Sometimes I Ubuntu setup stops in VMware when I provide only 128 MB RAM for T-G. With 256 MB RAM it always works. Maybe it's about RAM, but another machine of mine has also only 128 MB RAM. I think adding this to sysctl.conf is safe for all cases, as it's recommend by Tor. I'll google that topic soon.
  • (proper) Googeling wasn't very effective. 1; 2 I think, as recommend by Tor, setting it to 100000 is safe. We should do it.
  • (anonymous) agreed.

Installation of Tor package done in any case no matter if key installed or signature verification failed [SHELLSCRIPTS] [CLOSED]

  • (anonymous) It doesn't install unverified code if that's what you are worried about. Worst case scenario: it installs the old version from the ubuntu repo (but then will fail because the torrc is incompatible with 0.2 or if will install the correct version from the cache but fail to update it whenever one runs apt-get upgrade (but it will warn the user. Nothing else we can do about that I'm afraid.
  • (proper) I uninstalled (key removed by script) and installed again. Not sure if it installed from cache or net. My concern is about --yes, I hope it won't install if the key has not been imported.
  • (proper) Hopefully the only way to be safe is not to remove -yes, but for security it would not bug some much either. I know, that the apt will ask you if signature is missing or mismatch and if you press "y" for "yes", it installs anyway. So we have to be careful.
  • (anonymous) Have a look into /var/lib/apt and /var/cache/apt. They contain either full binaries or validated metadata with hash sums. -yes doesn't install unverified packages without the -force-yes option

Script command line options as a modules [SHELLSCRIPTS] [CLOSED?]

  • topic was: Passing Multiple options script
  • (anonymous) You added a todo about that, what's the benefit of that? (Let me just say that while it's possible it's a bit ugly/hackish to do so in the script.
  • (proper) I feared, that I am unable to describe it. I try again with more words. What I mean has nothing to do with multiple command line options. More like to make it even more modular. At the moment a special set of modules "configure_etc, setup_sysctl, etc." are invoked through a specific command line option "-install" or whatever. There are some similarities and redundancies. I mean, -vm does essentially the same like -install does. With the exception, that -vm has some extra modules. What we could do, is put -install in a module itself. And if -vm were run next time, the first thing it would do, were to start the '-install module'. After that's done, it does it's own stuff.
  • (anonymous) Understand. The problem is that the order is not always install functions followed by other functions, sometimes new functions are inserted somewhere in between (take -onvevm). Not really much work/duplications anyway.

Can the script autodetect Virtual Box? [SHELLSCRIPTS] [ANSWERED]

  • (proper) If it's auto-detected, we can enable auto-enable optional_torifytg.
  • (anonymous) I'm sure it could. For example based on the dhcp IP (if command to get local IP = 10.0.0.2 ....)
  • (proper) Thanks, I added a snippet to T-G script.

tor-arm by default on T-G [SHELLSCRIPTS] [IMPLEMENTED]

  • (proper) At the moment TorBOX strips off Vidalia functionality on default configuration. Arm is very good as controller. To see if Tor connection is working, bandwidth usage, switch circuit. Supports virtually all important features like Vidalia. We have arm already as optional configuration. Either we create the control cookie or we make a shortcut and allow 'sudo -u debian-tor arm'.
  • (anonymous) OK, can be done through bash alias.
  • (proper) alias arm="sudo -u debian-tor arm" works, but needs sudo password. Not sure, to we rather add an exception to sudoers or do we prefer the control port on localhost?
  • (anonymous) editing the sudoers files currently is insecure/not reviewed. Doesn't matter for initial set up on a clean VM but in an update it could be problematic (it's more theoretical but I prefer erring on the safe side). localhost opens another attack surface but not really because lolcalhost is supposed to be secure (you already log in with a wheels user. escalating to root is trivial (e.g. alias sudo in .bashrc to record the password...) From a security point of view it doesn't matter. We could ask tor devs for an input here but I think the cleanest option is to run arm as "user" and set torrc accordingly.
  • (proper) Ok, I activated ControlPort. The shortcut 'sudo -u debian-tor arm' in bashrc is still required. With all that together it works well. No questions for password, not running as root, sudoers untouched. Update: Done. Untested, if there is a bug, reopen.
  • (proper) Starting arm still needs root rights. Should we change this? (It was the reason why I wanted to enable control port, didn't seem to help anyway.)
  • (anonymous) either we edit sudoers or we set up ControlPort with authentication and run arm as user. That's the two options, I don't know which is more secure but sudo + entering password as it's now is the most secure option...
  • (proper) I see. Since the controlport is a Tor feature we should go the standard way. Or leave it as it is. We already allow poweroff without password. Protection against someone sitting in front of T-G and typing commands is zero anyway.
    • (anonymous) I think in terms of security control port is a tiny bit less secure because it may make an hypothetical network attack easier. Indeed local user access is already fatal so editing sudoers isn't an issue. we need to add a rule for /usr/bin/arm
    • (anonymous) I though we were using sudo instead of control port but your edit seems to be doing some of both? We should only add a sudo rule and a bash alias and leave control port disabled. The alternative was to start arm without sudo but configure a password in the arm config and torrc. As I said I believe the first is better, more secure and cleaner for our user case (usually I'd say it's exactly the other way round...)
  • (proper) Harder than I thought. The current arm releases require password authentication. Safecookie not supported. See mailing list and the answers. Would require a new release or the git version. Should we use a public password? To enter a password each time would kinda defeat the idea of this (arm without password). And to put that password in a script, I guess we would be told, that's unsafe. My sudoers entry "user ubuntu=NOPASSWD: /usr/bin/arm" doesn't work, no matter. Probable, because users are already allowed to start arm without sudo. To get ride of the password, it needs to be started as another user (root or debian-tor).
    • (anonymous) I fail to see the problem. Does this work for you: clean/unmodified Tor-Gateway 0.2-alpha, sudo -u debian-tor arm? Does that start arm - without the need to set any password?
    • (proper) 1) Entering "sudo -u debian-tor arm" in shell requires password.; 2. Adding "user ubuntu=NOPASSWD: /usr/bin/arm" in /home/user/.bashrc, logout, login do not change this either. 2) Running /usr/bin/arm in shell is telling "Connection refused. Is Control Port enabled?"; 3) Only when I activate ControlPort 9051 in /etc/torrc as well, it works without password.
      • (anonymous) you mean visudo/sudoers, not bashrc. Also you need to type "arm", not the full path or it won't match the alias
      • (proper) Oh, yes, I meant sudoers.
    • (anonymous) because the sudoers line was wrong, correct one is "user ubuntu = (debian-tor) NOPASSWD: /usr/bin/arm" (you need to tell it the "run as user", it defaults to root)
      • (proper) It's working now. (Without control port.) Solved, thanks!

Small script coding style change [SHELLSCRIPTS] [ANSWERED]

  • (proper) Concerning the echos... Makes it better readable for me. Similar to if, else etc. Curly brace, new line, stuff, new line, curly brace. For example, I want to change
    echo "user ubuntu=NOPASSWD: /sbin/shutdown -h now,/sbin/reboot,/sbin/poweroff
    user ubuntu=NOPASSWD: /usr/bin/arm" >> /etc/sudoers
    

to

echo "\
user ubuntu=NOPASSWD: /sbin/shutdown -h now,/sbin/reboot,/sbin/poweroff
user ubuntu = (debian-tor) NOPASSWD: /usr/bin/arm\
" >> /etc/sudoers

or even with three spaces or one tab.

echo "\
   user ubuntu=NOPASSWD: /sbin/shutdown -h now,/sbin/reboot,/sbin/poweroff
   user ubuntu=NOPASSWD: /usr/bin/arm
" >> /etc/sudoers
  • (anonymous) just remember that this will affect how the actual modified files look, indent is not an option because Linux configs don't usually do that. New line is OK.
  • (proper) Yes, script looks better, config files look worse. Newline is an acceptable compromise. Unless one comes up with a suggestion to see spaces inside the script but not inside the configuration files, I second you. A cleaner way could be to put the configuration files into their own files, but the project is not complex enough to justify that.
  • (anonymous) Shell scripts MUST NOT start with an empty line before #!/bin/bash. Escape the New Line to prevent them from showing up in the file, I've edited your sample above.
  • (proper) Thanks for bringing this up. Added your suggestions.

Remove hardcoded GPG public keys from T-W script [SHELL SCRIPTS] [DONE]

  • (proper) I don't like the hardcoded GPG public keys inside the T-W script. It's cumbersome for reviewers to check we haven't tampered with the keys and it also blows the script. Unfortunately we won't see #2617, #5236 or #3994 will probable still take a very long time. What are the alternatives? Can we download the keys from the keyservers and hardcode the fingerprints, which are published on torproject.org? Perhaps like this:
    # with updated short ids / long ids
    gpg --keyserver keys.gnupg.net --recv 886DDD89
    gpg --export A3C4F0F979CAA22CDBA8F512EE8CBC9E886DDD89 | sudo apt-key add -
    
  • (proper) While the short ids are not safe, the long ids are still safe and can not be collided. And those long ids can be reviewed by others more easily.
  • (proper) Done.

Do we support bare metal without Virtual Box? [ANSWERED]

  • (proper) We have to decide, which direction to go. Decide "once and for ever", if we support bare metal without Virtual Box. In any case we can summarize our findings, the pros and cons of this discussion. Pros: smaller attack surface. Cons: less generic setup, no snapshots.
  • (proper) Bare metal without Virtual Box on T-W should be advised against (hardware serials, mac address, revert to snapshot). UPDATE: Very detailed justification included.
  • (proper) Bare metal without Virtual Box on T-G is questionable.
    • (anonymous) What is questionable? VirtualBox based isolation isn't very strong - compared with Fedora 17 with new virtualization KVM/LXC sandbox in addition to SELinux, or microvisors. If we were using one of those instead bare metal wouldn't be very necessary. But as it stands now I absolutely recommend it for security. It works fine behind a ethernet router, what needs improvement is wlan documentation.
      • (proper) I'll added a TODO to the bare metal page, so we don't forget and do it later.
    • (proper) The pros and cons are questionable. And it's questionable if we support it.
      • (anonymous) the cons are self-evident and the baremetal page goes into the hardware requirements. The pros are detailed on the security page. (the "only" pro is security but that's a big one). I don't understand what you mean by "questionable"
      • (proper) Questionable: in means of unknown, needs answers. The T-G is trusted by default. It does not have to hide hardware serials. Adding a virtualizer to T-G expands the used code base a lot. Exploits can directly target Virtual Box on T-G. This is a big cons for security. For T-W the gain in security using virtualizer is clear, for T-G, not so much. One advantage using T-G on bare metal with Virtual Box is, that you can add a an additional VPN on the host with ease.
        • (anonymous) Thanks for clarifying - language barrier, questionable for me means dubious. Obviously (so I thought, it's also explained in the t-w script comments) the default bare metal set up is T-G on bare metal - not on its own Virtual machine. A secure Tor + VPN setup requires (at least) 3 systems: Torified, Tor-GW, VPN-GW. A possible set up is host 1 with t-w and host 2 running VPN with t-g guest. Still a larger TCB but the attack surface is much smaller (there are no untrusted applications on host 2). Otherwise the only option is to really use 3 physical "bare metal" systems (and just one VM if one cares about serial numbers and snapshotting). VPN can run on the router (like dd-wrt). Mobile usage is difficult because only a wired connection offers the required enforcement (tethered phone can be bypassed), but two Raspberry Pis with USB-ethernet could be small enough (btw, they just started shipping the first batch). Also we probably care more about network attacks than client attacks here so wlan/tethering is most likely acceptable. You won't have much fun running VirtualBox on a Raspberry Pi anyway (it probably doesn't even run on ARM).
          • (proper) Okay, I'll integrate this comparison into the bare metal article.
  • (proper) I added to the bare metal article, what I felt was missing. 1) "Using spare hardware + Virtual Machine", 2) "Using spare hardware without Virtual Machine". Should we drop support for 1) or do you want to keep it? If we drop it, we can move that text to some less important site and leave a link for those interested. Imo it was still important to go through that consideration.
    • (anonymous) What would we gain from dropping anything? It's not like we "support" anything at all anyway (as in tech/customer support). We don't even have a product to "ship"... I certainly don't support anything (no warranties, your risk, your fault...)
      • (proper) What would we gain: drop in complexity and time.; Not support defined by: we "do not have" to write documentation, we do not have to answer questions, we do not have to test, we recommend against. "We do not have to" anyway, it's a voluntary project, we still have the freedom to decide, what we want to do. Right now we offer "limited support for T-G in VM". (We documented pros and cons, we provided some tips, we didn't test, we might answer a question.); Of course, no warranties is always valid, as we are not a commercial company.
        • (anonymous) Which one did you want to drop, first you ask "Do we support bare metal without Virtual Box?", then "Should we drop support for Using spare hardware + Virtual Machine"?
          • (proper) Sorry for confusion. I wanted to drop "T-G + bare metal + Virtual Machine".
            • (anonymous) But VPN is entirely optional already anway, in your words "not supported". We need not change anything then...
  • (proper) So if we do not support bare metal without Virtual Box, we can use optional_staticvboxip by default.
    • (anonymous) we could do that anyway and put all the required variables into the top of the script for easy editing or even ask the user to enter the IP and gateway address.
      • (proper) I don't know, how complicated network setups can become and if users are able to do it without DHCP.
        • (anonymous) It will become complicated when ipv6 starts appearing on LANs. By then we should hope Tor supports it and it's secure enough to enable it on the gateway. Till then, static IPv4 networking is trivial and universally works for home networks.
        • (proper) We can ask in the script or more easily to advise to edit the script.

Switch from Ubuntu server edition to Ubuntu (minimal console) [REJECTED]

  • (proper) Either I must have forgotten why we've chosen the server edition and don't find the thread in the archive or we've never discussed it.
  • (proper) The server edition seams to be less supported than the non-server edition. Non-server edition has more users and is less affected by quirks when you install a desktop. Also the bare metal users, why should they install the server edition? What are the advantages/disadvantages of server edition compared to non-server edition?
  • (anonymous) I think there used to be a minimal ubuntu based on alternate desktop but right now the only minimal installation available I know of is through the server iso. Minimal is used for three reasons: Small resulting binary images. More secure (have a look at the amount of default daemons running even in a "small" ubuntu desktop like lubuntu). Ease for TorBOX devs: The scripts needed not a single 12.04 specific edit, what was set up for 11.10 just worked!
  • (anonymous) about support: Canonical used to support LTS Servers for 2 years longer than LTS Desktops. The server certainly has more "serious business" users (with high security and reliability demands) than the desktop editions. Ubuntu Server is an enterprise, cloud focused OS in the same way as RHEL or Windows Server. Community support differs because unlike on the Desktop Ubuntu isn't the unrivaled "number 1" on servers (Debian+Ubuntu is...). Anyway you know that there is a single difference between a minimal server installation and our hypothetical minimal desktop system: the kernel, more info, some is outdated: https://help.ubuntu.com/community/ServerFaq#What.27s_the_difference_between_desktop_and_server.3F http://askubuntu.com/questions/125892/what-is-the-difference-between-12-04-desktop-12-04-server-images https://help.ubuntu.com/8.04/serverguide/preparing-to-install.html
  • (proper) Convincing.== Switch from Ubuntu server edition to Ubuntu (minimal console) [REJECTED] ==
  • (proper) Either I must have forgotten why we've chosen the server edition and don't find the thread in the archive or we've never discussed it.
  • (proper) The server edition seams to be less supported than the non-server edition. Non-server edition has more users and is less affected by quirks when you install a desktop. Also the bare metal users, why should they install the server edition? What are the advantages/disadvantages of server edition compared to non-server edition?
  • (anonymous) I think there used to be a minimal ubuntu based on alternate desktop but right now the only minimal installation available I know of is through the server iso. Minimal is used for three reasons: Small resulting binary images. More secure (have a look at the amount of default daemons running even in a "small" ubuntu desktop like lubuntu). Ease for TorBOX devs: The scripts needed not a single 12.04 specific edit, what was set up for 11.10 just worked!
  • (anonymous) about support: Canonical used to support LTS Servers for 2 years longer than LTS Desktops. The server certainly has more "serious business" users (with high security and reliability demands) than the desktop editions. Ubuntu Server is an enterprise, cloud focused OS in the same way as RHEL or Windows Server. Community support differs because unlike on the Desktop Ubuntu isn't the unrivaled "number 1" on servers (Debian+Ubuntu is...). Anyway you know that there is a single difference between a minimal server installation and our hypothetical minimal desktop system: the kernel, more info, some is outdated: https://help.ubuntu.com/community/ServerFaq#What.27s_the_difference_between_desktop_and_server.3F http://askubuntu.com/questions/125892/what-is-the-difference-between-12-04-desktop-12-04-server-images https://help.ubuntu.com/8.04/serverguide/preparing-to-install.html
  • (proper) Convincing.

No preinstalled desktop environment / anonymous operating system [REJECTED (waiting for user feedback)]

  • (proper) People have different preferences when it comes to choosing the desktop environment. Some prefer the old/new gnome, some kde, unitiy, lxde, openbox or no desktop environment at all. A minority of people enjoys openbox. That also solves the blocking openbox bug, which you added recently to related tickets. We give away loads of users by using openbox as default. There are too many desktop environments with loads of users to decide for one. My proposal: let's try to install only software necessary for anonymity. Install only a minimal console installation. After the default installation with the console is started, educate the users how to download their favorite desktop over Tor (since already torified). Ubuntu offers good metapackages for installing a desktop environment. We could call TorBOX an anonymous operating system.
  • (anonymous) we do not install a desktop environment! Openbox is a window manager, it's not possible to use X.org and applications without one (also note that Openbox can be used as a WM in a GNOME or KDE desktop!) Dropping users to the console surely results in "losing" more users than defaulting to a lightweight preconfigured usable desktop. apt-get install something is not enough, openbox is configured to suite the specific requirements or T-W (like a custom launcher for TorBrowser or a shutdown command that doesn't need an insecure display manager or bloated other solutions. Also, most Linux users really only know Unity, GNOME and/or KDE. All of them are too large (slow, large attack surface, disk footprint) to be considered. The rest of the WMs available are all pretty similar, save for the tiling wms but users of those are more than comfortable to replace openbox manually. Other notes: Liberte is using OpenBOX, Tails uses GNOME and regularely gets heat for being bloated resulting in a large attack surface and poor performance, especially on VMs. I'd be interested in negative feedback by users regarding openbox in TorBOX, maybe the one or three complaints can be fixed without resorting to a more drastic strategy such as the one suggested.
  • (proper) "desktop environment" was the wrong term used by me. Openbox is indeed not as desktop environment. What I wanted to say is "no windows manager preinstalled". The term doesn't matter, since even when I used the wrong term, you made a good reply.
  • (proper) Yes. Right now, it looks like "I" or "we" are the biggest users ourselves. There is barely any feedback. I shouldn't weight my user opinion over others and try to guess what other users think, that may fatally fail. I'll just saw, that many people were interested in an "anonymous operating system" were "everything goes through Tor" (see similar projects and loads of discussions all over the net). Agreed, let's wait for more feedback until we'll decide to make a big change.
  • (TorBox.org) I definitely believe we should keep the window manager preinstalled on the default version of TorBox. I think not having one would put the motivational bar a bit too high for most new users who are just checking it out for the first time. I think a lot of people who would be users of TorBox would not stick with it without a preinstalled GUI. Getting straight to the experience of launching Firefox in a Torified container, without hassles, is an important new user experience, I think. I think the winning project like ours in this new market will be one with a GUI preinstalled by default. That said, I definitely see the expert value in an anonymous OS that doesn't come with a default window manager. So, I wouldn't make that the default, however, I'd probably look to offer it as a secondary TorBox offering at some future point.
  • (TorBox.org) And if a better default GUI than Openbox is discovered, then I'm fine with the notion of switching the default GUI. I just really believe there should be some desktop GUI preinstalled on the default TorBox offering. Primarily for the market success & project growth of TorBox. And other expert focused options can always be offered along side of more user friendly defaults. For example, offering a tutorial or script on removing the default desktop GUI, or just a separate GUI-less build of TorBox. But, definitely include GUI for default TorBox, since I think that's what most people will be initially seeking out and expecting upfront.

Using virtual box command line for fully automated builds? [SHELLSCRIPTS] [DONE]

  • (proper) There is nothing wrong with importing/learning from an open source project. Looking through the code it came to my mind, that we also could use the virtualbox command line, to create the machines.
  • (anonymous) waiting for contributors...
  • (proper) Since no one else taking this one, I am going to do it. Can take a while and help is welcome. I am still convinced, it's a good thing and required for the build machine and frequent updates.
  • (proper) Using VirtualBox command line is done. Added to BuildDocumentation. What's not done is installing Ubuntu unattended, but that's a new ticket.

thttpd replacement [SHELL SCRIPTS] [IMPLEMENTED]

  • (anonymous) Doubt that's coming back: http://bugs.debian.org/cgi-bin/bugreport.cgi?bug=653752 We need a replacement, what is TPO recommending now (they used to recommend it and it's why we chose it in the first place)?
  • (anonymous) Some options: mini and micro-httpd, really small but slow; monkey: small, "basic security features", no mention of ssl, but probably better performance. lighttpd, nginx: these are "proper" web servers, I'd tend towards one of these two, full security features, many devs and eyeballs make their code base more trustworthy, great performance. The main drawback is more complicated to set up and there's more potential for leaks but that doesn't really apply to TorBOX. Some words on web server security: http://security.stackexchange.com/questions/11/which-webserver-is-more-secure-apache-nginx-or-lighttpd
  • (proper) I think tpo recommend thttpd in the past. Tpo is now silent about recommending webservers. lighttpd is used by giants like YouTube. Who knows if they have hardened it with whatever means. Choosing a webserver is a very complex decision. I could probable spend equally as much time configuring a webserver like I spend on TorBOX. If tpo does not recommend one, we probable should not recommend one either. The T-G hidden services part can remain as a little help setting up and the T-W part, well, without server we can't be of any help installing one. Maybe of any help Tails server (still reading).
  • (anonymous) We should ask one or both then.
  • (proper) Asking tpo might not make sense, they're already escaping that question. Asking Tails makes more sense. To post to their list, it's possible to send a mail to tails-dev@…. No subscription required. Answer to that thread come by mail or can read [https://mailman.boum.org/pipermail/tails-dev the Tails archives. We can prepare a message.
  • (anonymous) honestly, I don't think it matters much. We can choose whichever we want, all the mentioned web servers are "secure" if used behind torbox. It comes down to other things like feature set, configuration and preferences. Alternatively we can just drop the "default selection" and let users choose their own but this increases fingerprintability. Another way to go about this is researching what server is the most popular in onion land and just go with that.
  • (anonymous) resources in case we go with lighttpd: http://www.cyberciti.biz/tips/howto-setup-lighttpd-php-mysql-chrooted-jail.html http://redmine.lighttpd.net/issues/1271
  • (proper) Let's go with lighttpd. Compared to nginx they at least have a forum with activity. Forget my overly paranoid comments from above. We can't do the magic and come up with a distro with full grsecurity, apparmor, compile time hardening and so on. I see the webserver as a proof of concept, how easy the hidden service configuration is with TorBOX. Anyone serious about hosting hidden services should think about security again so or so, which we line out on security and hardening page.
  • (anonymous) that comment on the bug by upstream don't exactly instill the greatest confidence in their security, for static content it doesn't really matter but still.
  • (proper) Agreed, that's a silly bug.
  • (proper) Thanks for implementing! Done?

t-w.sh todos [SHELL SCRIPTS] [DONE]

  • (anonymous) "# TODO: NEEDS REVIEW" on gpg, what do you need reviewed?
    • (proper) If the no-emit-version makes sense, is sufficient and/or opens new problems.
      • (anonymous) it's a standard option, I think we can safely use it. Even if there are still leaks, all torbox users will have the same fingerprint, and they have the same fingerprint as the most popular distro. I think this is resolved.
  • (proper) Removed tag [TorBOX 0.2.0]. Imo not blocking TorBOX 0.2.0.
  • (proper) About renaming apt-get to uwt-apt-get etc. "they can't have the same name;)". Well, this was tested. Worked. It's the whole clue, that they have the same name. Otherwise people will 1) not remember to use the uwt version 2) gui applications will still use gpg instant of uwt-gpg. I want to revert that. Update: thanks for reverting.

/usr/local/bin [SHELLSCRIPTS] [DONE]

  • (anonymous) Let's use that instead of /usr/bin?
  • (proper) I am not fit in file system hierarchy; wiki; another but it says /usr/bin isn't the correct. After reading that, I agree with your proposal.
  • (anonymous) if anything comes up we fix it.

torsocks wrapper script / torsocks in T-W [SHELL SCRIPTS] [DONE ?]

  • (proper) uwt has to be added to T-W. That should be the easiest thing. I'll also try to push that functionality upstream, I expect that to take a while, therefore I'll hate doing so, but manually add it to T-W.
  • (proper) Few things nag me about the wrapper... "ip=127.0.0.1 port=9054 uwt /usr/bin/gpg $*" hardcodes the path /usr/bin. Is this okay? Or will this, in the long run, enhance the maintenance effort? All bin's should always be linked to /usr/bin?
    • (anonymous) does it not work with out hardcoding? Anyway, non-system executable are always linked in /usr/bin, I don't forsee any maintenance issues. But you could do something like gpgpath=which gpg and replace the hardcoded path with $gpgpath.
    • (proper) Okay, thanks. I stick with hardcoding if no issues are foreseen.
  • (proper) Is there a more generic solution? Where we could simply make a table of <application name> <proxy-ip> <socks-port>?
  • (proper) The real problem... Looks like the wrapper breaks graphical applications, which execute the cli application.
    • (anonymous) not tested, I don't know how it breaks things, but it could be the >&2, try deleting them.
    • (proper) I removed the "always debug echos output", they'll be shown only when using verbose. Fixed one application but more tests required.
  • (proper) There is one more problem with torsocks. A bug "libtorsocks(2081): The symbol res_query() was not found in any shared library. The error reported was: not found!". It's tracked in Google code and Debian bugtracker. No fix for years. Tails plans to ship a fixed torsocks with their next release, because intrigeri already provided a patch somewhere. I expect their fixed torsocks binary before Tor 0.2.3 so the wrapper is a low priority.
  • (proper) For reference, I mailed Robert Hogan on 06/14/2012 if he is still available for torsocks maintenance. UPDATE: Good news. He is still active and friendly answered the same day.; #6155 Import torsocks from google code to torproject.org trac (assigned defect).
  • (proper) Do you know any gui applications, for testing, which use one of the cli applications, we want to wrap/socksify?
    • (anonymous) maybe links2 -g?, anyway, the most important thing is apt-get, wget, gpg... all cli apps. All networked gui apps in t-w have gui socks options.
  • (proper) I think as long as torsocks bug "The symbol res_query() was not found in any shared library. The error reported was: not found!" does not get fixed, this should not be activated by default in TorBOX. Those messages are really spamming up everything. Do you have an idea how to suppress them?
  • (proper) Tails fixed the bug "The symbol res_query() was not found in any shared library. The error reported was: not found!" or it's fixed in Debian. Dunno. The solution is not simple as getting the torsocks binary from Tails. Probable libtorsocks would have to be extracted. Extracting software from Tails is hard so or so, see "Problem getting software form Tails" above.
  • (proper) I removed the TorBOX 0.2.0 tag. Stream isolation is implemented nowhere yet to that extend and it will take a while until Tor 0.2.3 comes out. When Tor 0.2.3 comes out, we have to release a new version anyway.
    • (anonymous) seems to be debian, not tails. torsocks_1.2-3 in testing is fixed, or appears to be. ubuntu bug report with links to patch: https://bugs.launchpad.net/ubuntu/+source/torsocks/+bug/992068
    • (proper) Should we simply download and use the debian package then? Ubuntu/Debian are not so different after all and I expect that package not to be so complicated.
      • (anonymous) the package works in ubuntu and is simple, but then we need to import (and veriyf and trust) another set of gpg keys, this is not very elegant. Till tor-0.2.3 there won't be a stable release of TorBOX, till then I think we can live with that annoyance. I'd tag WAIT and hope it gets fixed in Ubuntu.
      • (proper) It's not a minor annoyance. Either we ship a fixed torsocks or we can't ship the uwt wrappers. Gui applications, which use uwt wrapped cli applications, will break. Also apt-get, which is used frequently, almost doesn't show any useful output since the bug spams everything. (Extra thread about being stable or not.)
        • (anonymous) For alpha/beta tester it is "cosmetic". Functionality is working, it just looks bad.
  • (anonymous) can we just grab the patch, install the source with apt-get and apply? Though that would require build essentials to be installed as well. Has to be upstreamed to torproject or ubuntu for us to be really useable.
    • (proper) Yes. Working on it...
      sudo apt-get install build-essential
      apt-get source torsocks
      cd torsocks-1.2
      ./configure
      
      # Source: https://bugs.gentoo.org/show_bug.cgi?id=395953#c7
      echo '
      --- torsocks-1.2.orig//src/torsocks.c	2011-10-25 17:49:50.000000000 -0400
      +++ torsocks-1.2/src/torsocks.c	2012-02-21 11:09:20.000000000 -0500
      @@ -124,9 +124,9 @@
       #define LOAD_ERROR(s,l) { \
           const char *error; \
           error = dlerror(); \
      -    show_msg(l, "The symbol %s() was not found in any shared " \
      -                     "library. The error reported was: %s!\n", s, \
      -                     (error)?error:"not found"); \
      +    if (error) \
      +        show_msg(l, "The symbol %s() was not found in any shared " \
      +            "library. The error reported was: %s!\n", s, error); \
           dlerror(); \
           }
           pthread_mutex_lock(&torsocks_init_mutex);
      ' > torsocks_patch
      
      patch -pl < torsocks_patch
      
      make
      sudo make install
      
      cd ..
      
  • (proper) Scripted that and added to T-W script. I still hope Ubuntu and upstream will add the patch. Will be also gone if we switch to Debian. In meanwhile we can use it... Done?

[SHELLSCRIPTS] Scroll wheel in VM [FIXED]

  • (anonymous) scrolling with middle click/wheel doesn't work in firefox
  • (adrelanos) I think that requires guest additions. There are some bug reports in the VBox tracker, but open and unanswered for three years. Also google is full of it. I am afraid, we can do nothing about it. Only VNC could work, but if that is more fun, I doubt that.
    • (anonymous) VNC can't be used because it needs a direct network connection from t-w to host, this is insecure. It should be possible to selectively install only guest drivers for the mouse.
    • (adrelanos) Theoretically we could VNC a hidden service. Or VNC T-G, which would forward to T-W. When I tested VNC directly I wasn't happy with the responsiveness. It's ok for support tasks but not a desktop operating system. Let's forget this theoretical method.
      • (anonymous) let's pretend it never happened :P
    • (adrelanos) Installing only the mouse driver is a good idea.
    • (anonymous) We just have to enable autoscrolling in firefox prefs (general.autoScroll true)
    • (adrelanos) Great! Can be added to the other prefs.
    • (anonymous) middle-click scrolling still works, but scroll wheel doesn't on the latest debootstrapped workstation. Some ubuntu package must be missing.
  • (adrelanos) Removed [0.2]. I don't think we can fix this. Virtual Box can make USB devices visible to VMs but that involves their serial numbers. Otherwise I haven't found any reports mouse wheel was functional without guest additions and any use of guest additions weakens isolation and should not be default.
  • (adrelanos) Or not. Works for me in 0.2.

[0.2] [SECURITY] TorBrowser two homepages issue / aos Readme issue [DONE]

  • (anonymous) "# TODO: We should better enable the Bookmarks Toolbar and add big READ THIS!!! aos README bookmark."
    • (anonymous) not really happy with either the current nor the proposed solution, other solutions: readme in the openbox menu, readme in the thunar home folder, readme "desktop" (like crunchbang), aos.sh script. the goal is to be sure that every user really does read the readme.
    • (adrelanos) One good solution were two homepages (start pages). Working with Firefox. Broken in TorBrowser. #6053 Also dunno how to enable the default bookmarks toolbar. #6025 No documentation how TBB's update check works. #5273 Therefore also not possible to ensure it works well behind transparent proxy.
      • (anonymous) I tried the former too, without success. I don't like the idea of adding the toolbar, the VM desktop is small enough as it is. I'm tending towards a desktop image with relevant information + a custom script that helps with the cli commands + a .bashrc welcome message.
      • (adrelanos) Toolbar to TorBrowser, not desktop. Agreed with the other suggestions. Bonus points if we could keep the readme notification desktop agnostic (gnome, etc.). That's why I wanted to activate TorBrowser bookmark toolbar or two start pages (homepages) in Tor Browser.
        • (anonymous) Sorry, that was confusing. I mean that if we add a toolbar -to the browser- the vertical space becomes even smaller than the already small VM-"Desktop" provides. Cross DE/WM link doesn't work, openbox doesn't auto-add "*.desktop" launchers to its menu. That's the usual way to do desktop agnostic launchers.
  • (adrelanos) Close this one? To solve the issue not having two TorBrowser homepages we would have to fix the TorButton upstream bug. Afaik TorButton is written in javascript. Once you know one script/programming language, you know all script/programming langues, ok not really. You have to learn a new syntax. I can probable do it some day if no other javascript coder would show up, which I'd prefer. For me it's a huge effort to get into js.
  • (anonymous) Should still do something in the meanwhile? Or is the current readme solution OK? A launcher with "tor-browser_en-US/App/Firefox/firefox --profile tor-browser_en-US/Data/profile/ -new-tab http:..." doesn't work either. Update: it does.
  • (adrelanos) Actually the -new-tab idea is great. You can not do it with one command. I assume you have TB open right now. Simply test right now from TB directory "./App/Firefox/firefox -new-tab google.com". We can do it with two commands. In the start script we have after "./App/Firefox/firefox --profile Data/profile -style Cleanlooks" to add "./App/Firefox/firefox -new-tab https://trac.torproject.org/projects/tor/wiki/doc/TorBOX/Readme". The tricks thing is, we the first command is blocking, i.e. sh jumps to the next line after Firefox got terminated. We have to execute the first line non-blocking, i.e. in background. (--profile is not required. Unless -no-remote is used, it connects to the running session. Didn't test if it hurts.) The following works, although not optimal.
    ./App/Firefox/firefox --profile Data/profile -style Cleanlooks &
    echo "Sleeping 20 seconds."
    sleep 20
    echo "Wake up."
    ./App/Firefox/firefox --profile Data/profile -new-tab https://trac.torproject.org/projects/tor/wiki/doc/TorBOX/Readme
    
  • (adrelanos) I am not really proud of the sleep 20. On slow/busy systems it can take a while until TorBrowser starts. If the two commands get executed instantly after reach other, TB will complain being busy and drop the command. The sleep ensures TB is really running, unless on a ultra slow system. A clean solution were to ask if TB is ready, but it don't know if there is such a function. Even more clean it where if we could do it with simply one command.
  • (anonymous) That's no good because it will suddenly launch a site while the user is doing something else. It breaks workflow (UX dead sin) and he might even think there's something wrong with the system. I think it's best to put that command into the menu. The user has to see that menu when he wants to do anything in t-w. It also won't get in the way in the future when he's already read it.
    • (adrelanos) In menu is also not so good. Do you expect them to manually click it every day? They should check the news each time they use aos, in case a critical vulnerability has been found.
      • (anonymous) tbb get's updated all the time, aos once stable, isn't supposed to need any update (apart from apt-get and tbb). So I think it is acceptable to put that into the menu. But it should say "readme/check for aos updates". this could run a script that checks if firefox is running, launches start-tor-browser if not and sleeps, then opens a new tab. All in all I think for now (till multi hp is fixed) this is the way to go.
      • (adrelanos) I am not sure aos won't require updates. Once we get more users, vulnerabilities might be found. Some kind of update notification mechanism is important. New idea for the update mechanism, I create a new topic.
        • (anonymous) You want a full update mechanism just to roll out changes to torrc and iptables? This stuff is supposed to be maintained manually and anything else is not "critical". Remember aos just configures the operating system, it doesn't supply binaries. If it does we'd need a repository and use apt for that.
        • (adrelanos) I wanted a apt (ppa) repository just for notifying about aos news. But such an apt repository simply complicates things and involves a new trust issue. An apt repository can virtually change any files for all users. There must be another solution to inform about news.
          • (anonymous) yeah, a custom program querying rss, this only makes sense when (trademarkdomain user disappeared) is online. And yes, ppa model is brain dead. Like so much in Linux it comes from a security model of the 70s :(
          • (adrelanos) What about installing an rss reader with notification by default? Until (trademarkdomain user disappeared) there might be a "your free rss news hosting" (SSL) service. But there are also risks. Fingerprinting/Exploiting aos users, estimation how many aos users exist, a "hello" each time the feed reader looks for updates. I'd like to ensure, that using aos (at least from source) is nothing more than 1. understanding the source 2. trusting VBox 3. trusting operating system 4. trusting Tor. It shouldn't involve a "trusting aos(.org) devs.
            • (anonymous) good objections, it should query a single static list of all news and determine locally what to show, the real rss protocol leaks too much. However, on second thought, the whole thing is overengineered: people may use roll back snapshot and multiple t-ws, they don't want to repeatedly be prompted with the same news updates. Altogether I like the current readme with important notices there better.
  • (anonymous) and this will require TBB to be running already, no good again.
    • (adrelanos) Yes. I don't think there is a way to ask if TB is no longer busy (starting up).
  • (anonymous) Also is it just me or does the latest torbutton throw a dialog "important information!..." each time it's launched.
    • (adrelanos) You should see it only once. I saw it only once. Try updating TB.
      • (anonymous) Still happens with new t-w build (latest sources)
  • (anonymous) This sucks, I tried using a pdf readme with links, but those don't get opened by TBB, right?
    • (adrelanos) pdf is not suited so or so. Our content is not static. News are changing.
  • (anonymous) Conclusion: TBB is broken as a primary browser and in a transparent proxy setting. There are already bug reports but that has very low priority for TPO. I'm thinking about following Tails...
    • (adrelanos) I don't agree. TB is the most Tor-safe browser against linkability and fingerprinting with so many other security features (experimental defense against website fingerprinting etc.). Using another browser puts aos users into a different anonymity set, which I believe to be a bad thing. Also Tails want in the long run to switch to TB. They are being blocked by things, which do not block aos, i.e. we do not yet have the goal to support loads of non-English people. Getting stuff from Tails is also complicated, see "Problem getting software form Tails" below. I believe some TorButton bugs are trivial to fix, I wonder why the big community doesn't contribute more. See #5611 about how trivial my changes are and how difficult it is to push it forward. But that are other stories.
  • (anonymous) I know and agree with you but right now there is no clean solution that's better than the current situation. We have to wait for TPO to fix the homepage bug or do that ourselves. Is check.tpo doing manually really that bad? A blocker for a stable release? Yet another idea: ship two browsers (e.g. links2 -g) and autostart that on the readme. Not pretty but it would really work (the readme would have to be hosted elsewhere because we need to modify the html to be better suited for a text browser). UPDATE: above
    • (adrelanos) "Is check.tpo doing manually really that bad?" Yes. A weak reason: just in case something got wrong it's and extra check. A strong reason: TBB downloads the list of most recent versions from check.tpo and decides locally if an update is required. (Yes, another bug/stopgap until they finally get Thandy ready.) "Check for TB updates manually." is not a good solution, I don't expect people to remember."
    • (adrelanos) Autostarting another browser... We can't review that browser/make it Tor-safe. I don't want anyone to believe we want to fingerprint/link/exploit people by lurking them on one of our websites with a non-tpo browser.
  • (adrelanos) Another idea: we start two TorBrowser's when the start script is once clicked. One TB can be started with -no-remote. The TB with the readme is supposed to be closed after reading.
    • (anonymous) I don't think that works, we can't use two full copies of everything (that would just be insane) but using the same profile with two instances is bound to fail (firefox probably won't let us, if it does it can result in a corrupted profile.
  • (anonymous) I'm for a custom script in the menu to check for tb updates and keep default settings in tbb. This is more future proof until said bug is fixed (hmm, which ticket?)
    • (adrelanos) Ticket #6053 (listed under related ticket s also).
    • (adrelanos) Sounds good. If we check and notify for updates with a script, we can leave aos/Readme as default homepage.
      • (anonymous) We are talking two different things, my bad. TB=TorBOX, and default settings=leave check.tpo as default homepage. the menu script launches a tbb tab with readme where people manually can check if there's been any important update to aos. If tbb isn't open yet that script could open a terminal window explaining that we need to wait XX seconds (can it really take 20?). What do you think?
        • (adrelanos) It can really take 20 seconds or even longer, if the host is busy or if the VM is busy or on slow CPU's. Waiting and not using events is most times a bad thing.
          • (anonymous) Think I got a solution: there's a way to poll for X windows, run a loop to check if a firefox window exist (pseudocode: xlsclients | grep firefox != "" && launch_readme ) then launch the readme.
          • (anonymous) actual code, save as /usr/local/bin/TorBOX_Readme.sh, add to openbox menu.xml, reset tbb homepage to torcheck. Can we close this ticket if this works reliably?
            #!/bin/bash
            if [ "$(pgrep firefox)" = "" ]; then
               echo "Starting TorBrowser"
               ~/tor-browser_en-US/start-tor-browser &
            else
               echo "TBB is already running"
            fi
            
            while [ "$(xlsclients | grep firefox)" = "" ]
            do
               sleep 3
            done
            
            echo "Opening aos Readme"
            ~/tor-browser_en-US/App/Firefox/firefox --profile Data/profile -new-tab https://trac.torproject.org/projects/tor/wiki/doc/TorBOX/Readme
            
          • (adrelanos) Code works for me, although I am not using Openbox. Can be added, why not. I wouldn't go as far declaring this issue resolved. I don't expect people to click that button on a regular base. Only opening the readme by default ensures, they read it.
        • (anonymous) It could also be added to startup script (just from the while loop...) to open both pages whenever TBB is launched.
        • (adrelanos) Is the ' "$(pgrep firefox)" = "" ' reliable? I imagine there will be cases when Tor Browser is not ready but the window is created. It would result in an hard to reproduce error. "My Tor Browser sometimes says he is busy while starting."
          • (anonymous) It will reliably check if firefox is running = has already been launched. It will only fail if the user juuust closed the firefox window but the firefox process is still running. There is no way around that I'm afraid. Firefox itself sometimes throws that error if you close and reopen immediately (at least in previous version, haven't seen it lately, probably because shutdown got faster).
          • (adrelanos) I think that is going to create confusion.
            • (anonymous) No, it's a fringe case. It shouldn't happen in real life - unless testing indicates otherwise...?
            • (adrelanos) Must not be the case for us. Murphy's law: "Anything can go wrong, will go wrong." Pretty much true in software development. And here we already admitted something can go wrong.
              • (anonymous) There's also nothing we can do about it short term. It's perfectly acceptable as a workaround for alpha and beta versions. And, we don't even know if it really can and does happen in the described case, this should be tested first...
              • (adrelanos) The error message is not so critical after all. Let's do it.
      • (adrelanos) Checking the news date or last modified date shouldn't be so hard and is more error prone than manually checking for news.
      • (adrelanos) It's still arguable, but another question, if the update check should run after boot and every 24 hour or if it should be only manual.
    • (adrelanos) Since aos is more than TorBrowser, couldn't we check after booting if the "you are using Tor" works and notify for being so or not? And also show external IP at this point.
      • (anonymous) yes, but that doesn't replace check.tpo and it isn't all that informative because either tor really works or network fails completely.
      • (adrelanos) Well, it says "not using Tor" if the proxy behind Tor is working or if they messed up their setup. I think it's a useful addition. It should also by the way check if there is a new TBB. It would copy check.tpo's functionality to the operating system, which I believe to be a good idea, since aos is more than TBB.
        • (anonymous) That's all true but let's not get ahead of ourselves here. aos is still only a collection of configs, it doesn't depend on any outside infrastructure but it depends on TBB. It could be made into a full distro like system with its own update mechanism and forked/newly developed features. But as I see it aos doesn't yet have the man power to do any of that. Hey, we can't even get a fixed torsocks, htp, ram wipe or torbutton that allows multiple home pages into aos. I consider that all more important right now than creating new custom code and projects.
        • (adrelanos) I'm sorry to say, but I can't work on issues based on priority. I do it based on mood and skill, and I am still learning all that stuff and while I do easier stuff I learn more stuff, that keeps it fun and it's at least slow progress compared to no progress. On the other hand it was quit easy to phrase check.tpo and recommend versions (you coded that already). Here is the code.
          • (anonymous) Sorry if that came across the wrong way, I'd never tell you what you to work on. I also thought you were talking about making a custom tool to check for aos, not TBB updates.
            • (adrelanos) No problem, no offense taken. The problem with misunderstanding are the terms, I sometimes use TB for TorBrowser because I don't believe aos uses the Bundle.
            • (adrelanos) torcheck script merged into t-w script. Moved discussion to the bottom.
        • (adrelanos) The you are (not) using Tor; IP; Installed TorBrowser version and Latest TorBrowser version should be fade in by some sort of tooltip (like Tails is doing it) or popup (open for suggestions). I imagine the script to run after boot and every 24 hour. I don't think that the programming, maintenance and effort is beyond our skills. When we get this script into aos, we can leave aos/Readme as default homepage. This solution can work as a stopgap until tpo finally fixes Thandy, TorBrowser apt-get, two homepages in TorBrowser etc.
        • (anonymous) Nice job. I don't know how to create an info tray tooltip but we could use zenity though apt-get install zenity --no-install-recommends is still larger than I'd like. Then use it like this:
          zenity  --warning  --text="You need to update your TorBrowser" --title="TorBrowser Update"
          
        • (anonymous) OR: start /usr/lib/notification-daemon/notification-daemon (autostart) then: notify-send "test". options don't work: https://bugzilla.gnome.org/show_bug.cgi?id=665761
          • (adrelanos) I don't like popups (zenity). notify-send is pretty, does not annoy and looks to support all desktops, which is very nice.
            • (anonymous) yes but since notification timout is broken it's a lot easier to miss
            • (adrelanos) Damn it. Well, if it's broken let's rather use zenity.
    • (adrelanos) Does Openbox have some tooltip mechanism or shall we use a popup or how to show the notification?
      • (anonymous) no, don't think so. but tint2 got a notification area. I don't know if that can be done with a bash script or needs a binary/python/perl.
  • (anonymous) what's the TODO here? let's keep open for the torbutton bug, but what about our workaround?
  • (adrelanos) No need to leave it open. The Tor Button bugs are being tracked in Related Ticket s. TODO: add the workaround.

[0.2] torcheck script discussion [SHELL SCRIPTS] [DONE]

  • (adrelanos) Lets use -t 0 again. It is gone too fast. One click is not too much.
    • (anonymous) But it's broken, see bug report below.
    • (adrelanos) Btw -t 0 gives me at least 15 instead of 5 seconds.
    • (anonymous) in tor-workstation? crude workaround would be to repeat with !!
    • (adrelanos) Yes, in T-W. Can you demonstrate the workaround please? Dunno what you suggest.
    • (anonymous) Forget it. doesn't work, history expansion is for interactive shells only
  • (adrelanos) Its trivial to support console users as well. Lets use MSG=" ... " and later echo "$MSG" and notify-send "TorBOX" $msg -t 0
    • (anonymous) good idea
    • (adrelanos) done
  • (anonymous) IP is broken and pretty useless - IMHO should be uncommented for now
    • (adrelanos) Works for me. Strange. In bold.
  • (anonymous) no <b>s in the output of IP=grep IP index.html ??
    • (adrelanos) with the <b>s but thats just a minor annoyance.
    • (adrelanos) Fixed by anonymous.
  • (adrelanos) Zenity or notify-send? As long as notify-send is broken, I tend to zenity.
    • (anonymous) You say that it isn't broken for you. I didn't test on a clean system.
    • (adrelanos) -t 0 gives me 15 seconds which is still too less. It's broken. Should be infinite until click or until next time.
    • (anonymous) Right, I forgot. But if we auto run the script that frequently zenty is no solution and people will notice it, even if it disappears after a short time.
    • (adrelanos) Can't we identify "our" zenity windows and kill it before showing up a new one?
      • (anonymous) I guess so, not sure how. What should be simple to do is: detect if there is _any_ zenity window open at all and then "killall zenity". It'll still nag sers pretty frequently.
      • (adrelanos) I though about that as well, but I don't think that's good. If they install any custom software also other zenity windows could be killed in collateral.
      • (adrelanos) Got it. "zenity --title="TorBOX" --info "test 123" &" and "LASTPID=$(echo $!)"
      • (adrelanos) Nevermind. zenity supports --timeout. Done.
  • (adrelanos) Start this script after boot? Start this script every 24 hours?
    • (anonymous) I'd use start up in addition to cron.daily
    • (adrelanos) Ok.
  • (adrelanos) And lets merge that thing?
    • (adrelanos) Merged.
  • (adrelanos) tor-workstation.sh is currently broken because create_torcheck_script() tries to echo ' within ' ', which won't work. Any idea how to solve that?
    • (adrelanos) One solution might be to Replaced all " with \", all with \ and all $ with \$. It gets messy with $USERNAME. I don't like that solution. Any other?
  • (anonymous) one more separate script isn't too bad, or try echo '\''
  • (adrelanos) \' doesn't work.
    • (anonymous) It does: echo 'IP="`grep IP index.html|sed '\''s/<b>//g'\''|sed '\''s/<\/b><br>//g'\''`"'
    • (adrelanos) That certainly does work. (Wrote that above.) I didn't do, because that makes aos-Workstation script less readable.
    • (anonymous) Sorry, was misunderstanding. I think for now the uglier code is still a good trade off compared to managing and copy and pasting yet another script manually. It's just temporary anyway.
    • (adrelanos) Not sure if it's worth the effort. Depends on how long it takes to move on debbootstrap and file separated source. (We could register a git somewhere right away. If I am not mistaken, it's quite easy to move one git to another (to (trademarkdomain user disappeared))).
    • (adrelanos) Git also makes it easier to obtain the full source.
  • (anonymous) see website ticket for git discussion

[0.2] T-W kernel update [SHELLSCRIPTS] [DONE]

  • (anonymous) We need to run two scripts/the script two times in t-w. First to set up networking and install and upgrade software, then reboot into new kernel and clean up. Otherwise we end up shipping two kernels + headers + initramfs, and those are huge.
  • (adrelanos) Let's either add that to build documentation or let's switch to fully automated builds where we keep care of this before aos 0.2.0.
  • (anonymous) With debootstrap I think this is not an issue, with preseed/network boot we'd still have to split the setup into two parts.
  • (adrelanos) +1 for debootstrap.
  • (anonymous) that's done for t-g. TODO: T-W and we need documentation for aos-phyiso (or whatever we wanna call it)
  • (anonymous) t-w done, we already tell about apt-get updates repeatedly.

[SHELLSCRIPTS] Move to file separated source [INFRASTRUCTURE] [DONE]

  • (adrelanos) I wanted to propose that after Whonix 0.2.0 but I think it's perhaps better to do it earlier. With torcheck and the problem to escape ' within ' ' it demonstrates, were the limits with one big shell script per VM are. It's possible to correctly escape it, but that makes the source less readable. Other than that, there are currently only two people who read and edit the script, proper and anonymous. I won't go as far as saying others can't understand it, but the effort must be high to get into it.
  • (adrelanos) I propose moving the files, i.e. the ones we paste using for example echo '...' > /etc/tor/torrc to separate pages/files. Not sure if that also hits the borders of "wiki development" or if something like git is required.
    # Not complete yet, only for demonstration.
    ~/torbox_source_0.2.0/
    ~/torbox_source_0.2.0/tor-gateway.sh
    ~/torbox_source_0.2.0/tor-workstation.sh
    ~/torbox_source_0.2.0/tor-gateway/etc/torboxfirewall.sh
    ~/torbox_source_0.2.0/tor-gateway/etc/tor/torrc
    ~/torbox_source_0.2.0/tor-gateway/etc/resolv.conf
    ~/torbox_source_0.2.0/tor-gateway/usr/bin/leaktest
    ~/torbox_source_0.2.0/tor-workstation/usr/local/bin/uwt
    ~/torbox_source_0.2.0/tor-workstation/usr/local/bin/apt-get
    ~/torbox_source_0.2.0/tor-workstation/usr/local/bin/gpg
    ~/torbox_source_0.2.0/tor-workstation/usr/local/bin/...
    # HARDCODED!
    ~/torbox_source_0.2.0/tor-workstation/home/user/.gnupg/gpg.conf
    ~/torbox_source_0.2.0/tor-workstation/home/user/.config/openbox/menu.xml
    ~/torbox_source_0.2.0/tor-workstation/usr/bin/arm.sh
    ~/torbox_source_0.2.0/tor-workstation/etc/network/interfaces
    
    # Perhaps there is also some shared code? Like delete_logs.
    ~/torbox_source_0.2.0/shared/delete_logs
    
    • (anonymous) I'd go with a "build_torbox" script in the root which creates tmp build directories and calls the other scripts, then sub-dirs for t-w and t-g and shared. We need separate scripts for building the basic debootstrap chroots and converting to VMI, configuring the chroot and finally to create the Vbox VMs.
      • (adrelanos) I guess VMI=VDI (virtualbox disk image)? Otherwise sounds good. The "build_torbox" script should support "build" and "clean" (delete temp dirs). Perhaps also do only one build step and/or repeat another one to ease debugging.
  • (adrelanos) Perhaps https://trac.torproject.org/projects/tor/wiki/doc/TorBOX/torbox_source_0.2.0/ or https://trac.torproject.org/projects/tor/wiki/doc/TorBOX/Dev/torbox_source_0.2.0/, i.e. https://trac.torproject.org/projects/tor/wiki/doc/TorBOX/torbox_source_0.2.0/tor-gateway/etc/torboxfirewall.sh.
  • (adrelanos) Not sure how to handle the >> files. To get this started we could still leave them in the tor-*.sh scripts.
  • (anonymous) Absolutely. But we need something better than trac to handle them, i.e. git. When (trademarkdomain user disappeared) comes online we could use a self hosted one like Tails does, or we use github (like Liberte).
  • (trademarkdomain user disappeared) Yes, I'm happy to implement an installation of Git on the (trademarkdomain user disappeared) server for our dev needs.
  • (anonymous) Great!
  • (anonymous) As long as we have no source repository we should keep as few scripts as possible for obvious usability reasons. Right now we have 5 scripts (t-g, t-w, t-debootstrap, t-chroot, torcheck). Any more absolutely required?
  • (adrelanos) Agreed and dunno for now.
  • (anonymous) 7 now, collected here: https://trac.torproject.org/projects/tor/wiki/doc/TorBOX/Dev#OurShellscriptsandBuildDocumentation
  • (anonymous) CLOSE? nothing to be done right now
  • (adrelanos) Done in github devel branch.

[BUG] [SHELLSCRIPTS] default color profile for gnome-terminal [FIXED

  • (anonymous) It currently defaults to black fonts on black background, not good.
  • (anonymous) I'm thinking of replacing it with xterm, the other terms just always suck one way or the other. gnome-term is nice, good features, but slow and got a horrendous config system. ~/.Xdefaults for xterm, run "xrdb .Xdefaults" to apply. xterm isn't "user friendly", but anyone who uses the terminal should be knowing what he's doing anyway. But at least copy and paste should be a bit more obvious for windows/mac users... If someone fixes the gnome-term .config that would be appreciated.
    xterm*background:   white
    xterm*foreground:   black
    XTerm*utf8: 1
    XTerm*eightBitInput: false
    XTerm*eightBitControl: false
    XTerm*eightBitOutput: true
    
  • (adrelanos) I am happy with any quick fix for Whonix 0.2.0. xterm lacks features, such as an tabbed interface but it's ok. The geeks are free to install a replacement.
  • (anonymous) so xterm config doesn't really work as I hoped either. It's going to be urxvt, if only temporary (apt-get install -no-install-recommends rxvt-unicode)
  • (anonymous) workaournd implemented but ticket is not solved, urxvt is no replacement for a user friendly terminal that at least supports copy and past with mouse context menu (and tabs would be nice too).
  • (anonymous) I think the roxterm segfault has been fixed in precise. If it's stable we got our replacement!
  • (anonymous) roxterm still crashes, so does terminator. Gnome-terminal is still the best option.
  • (anonymous) right now, it defaults to xterm, even with urxvt installed. UPDATE:or not
  • (adrelanos) Please try:
    USERNAME="user"
    sudo -u $USERNAME gconftool-2 --type string --set /apps/gnome-terminal/profiles/Default/foreground_color "#FFFFFFFFFFFF"
    
  • (adrelanos) Already added to the script. If it works for you, we can close this one.
  • (adrelanos) Unfortunately will not be fixed in [0.2]. When I found the command above it worked, but it didn't work within the script. Maybe gnome terminal has to be started at least once, closed and then the command will have an effect. Needs testing.
  • (adrelanos) Not one gconftool-2 command is required, it's three. Hopefully fixed in latest git devel branch.
  • (adrelanos) Now fixed, because no longer using Openbox.

[SHELLSCRIPTS] Ubuntu unattended (automatic) installation debootstrap vs preseed [DONE]

  • (adrelanos) That's the missing link before we can have fully automated builds. Looks feasible to implement for me.
  • (adrelanos) I found a tutorial, which is pretty straight forward, explaining how to install Ubuntu unattended. There is an updated Ubuntu 12.04 example-preseed.txt. Preseed is the recommend way in the Ubuntu 12.04 Installation Guide.
  • (adrelanos) Alternatively, there is also the FAI (Fully Automatic Installation) project, but the learning curve seams way too flat to me.
  • (adrelanos) Advantages of automatic installation:
    • (adrelanos) When it's done, it will be much easier, faster, secure and motivating to make new binary builds.
    • (adrelanos) More secure, because we eliminate the possibility to setup the wrong settings.
  • (adrelanos) Disadvantages of automatic installation:
    • (adrelanos) Who knows preseed syntax and unattended installation (possible less security reviews).
    • (adrelanos) Requires additional software for building Whonix, such as DHCP server, TFTP server. We have to configure them leak free: they may listen on localhost only. - If that's hard to configure, we simply use iptables as firewall and block all incoming ports. That may be unpleasant for people who want to build themselves and have server ports open. We recommend a dedicated build machine already.
    • (adrelanos) It requires the net installation (netboot.tar.gz).
    • (adrelanos) Optionally, hiding the fact, you are building Whonix could be difficult, when using netboot.tar.gz.
    • (adrelanos) More complex build script. It will be more difficult to manually build Whonix. After adding the command line commands for using VirtualBox by command line, it's already harder to figure out, which VirtualBox gui options to choose to do the same. Should we maintain two versions? Manual creation of VirtualBox VM's and manually choosing all options for Ubuntu installer?
  • (adrelanos) I'd like to hear some opinions, also open for alternatives...
  • (adrelanos) An alternative to preseed could be to manually create the image. Mount the image, chroot into it. I have no experience in that field. Dunno if that makes more sense. Debootstrap
  • (anonymous) I found no helfpful docs on how to do that. I don't know how it could be possible. The problem is we needed to directly access the virtual disk in order to install grub to it. The only way I found is to use a nother VM, mount the drive there and run grub-install. That doesn't really help us... The most promising option I think is to remaster the install CD to include a preseed file and auto start syslinux: http://g0blin.co.uk/programming/ubuntu-preseed-virtualbox-and-hetzner-eq4-virtual-machines/ We could already include the scripts too and copy them somehow to the target systems so we don't need ssh and port forwarding.
  • (adrelanos) Nice link. The most complicated thing with preseed.txt is getting the CD to read it. He remasters the CD instant of using a DHCP and tftp server. Tempting. I did already struggle with tftp package bugs and the DHCP/tftp setup is overly complex.
  • (adrelanos) grub-install should not be a problem. Linux is very flexible. With wubi you can even boot from NTFS (an image gets booted from a file, loop device). Once an image is mounted, there should be no relevant difference from a real hdd. UPDATE: Image generator example, looks easy.
    • (anonymous) I don't see how that helps with building a vdi from a chroot "folder", the OpenWrt-ImageBuilder archive already contains grub stages and squashfs and everything nicely prepared. The script itself may look easy at first glance but it's just a wrapper for the OpenWrt scripts. If we can mount the vdi from the chroot, that would work. But that doesn't seem possible (vdfuse is not in ubuntu); vditool is gone, qemu-ndb doesn't work for me. There most be a working solution... Another way would be to create raw or other image that can be mounted and then convert that to vdi. Update: got it working, nbd software was missing.
  • (adrelanos) We have several options. 1) Remaster the CD, 2) preseed 3) mount, Debootstrap/Multibootstrap or similar. We should compare simplicity, flexibility, leaks, distribution agnostic (easy switching to another distro), and which solution is the least hackish (also supported in a few years). If you could help with any of the methods, that were of course also a strong point.
    • (anonymous) remastering and preseed is distro specific but pretty much every big distro offers similar functionality. preseed with netboot just doesn't sound that secure, chrooting and building that way is the most flexible but definitely more complicated. It's also slow if done over Tor because it fetches everything via network each time you build it as opposed to use packages supplied by the install iso.
    • (anonymous) https://trac.torproject.org/projects/tor/wiki/doc/TorBOX/Dev/Debootstrap I have no further interest in messing around with GRUB, I spent several hours (testing it is a chore, even with VMs) on trying to get it working but to no avail. I think we need to look into transforming a loopback into a vdi but I don't even know if grub2 will handle that. I always just get "/dev/something does not have any corresponding BIOS drive." when running grub-install. Is there no frigging command line anymore? grub-legacy should work but I fear support is going to be dropped by most distros, If it wasn't for the updating issue I'd just go with syslinux and create isos,
      • (adrelanos) VBOX supports VDI, VMDK, VHD, and some HDD (forget about HDD) as disk file images. Perhaps this can be of any help.
  • (adrelanos) Maybe of any help: VMBuilder; Setup Tor DA with KVM; ubuntu-dev-week-vmbuilder
  • (anonymous) Didn't look into it (some links seem dead, vmbuilder only mentions KVM) but from http://ebroder.net/2009/08/04/installing-grub-onto-a-disk-image/ : "For various reasons, I wasn’t able to use the multitude of VM installers already out there, but one thing I noticed is that most of them don’t actually install a bootloader..."
    • (adrelanos) All links up again. Yes KVM and only less important virtualizers only. After switching to Virtual Box command line, I would rejoice trashing that work. But I'd switch to a better supported virtualizer anyway. KVM is build in linux kernel and more supported. We just haven't researched if it's suited for Whonix (hide hardware serials etc.). And also the userinterface virt-manager is not so pretty. Supports only VNC and no "direct output". Looks like KVM is more suited for servers but not as client operating system, browser through VNC is not so much fun.
  • (anonymous) Debootstrap can be used to build Vbox images but we need to investigate host related leaks. Liberte for example can apparently be built behind tor by setting a proxy variable (didn't test it, the whole build takes ages). Does that take care of resolv.conf, dhcp and so on?
    • (adrelanos) Very good questions. How will we find out? Shall we ask the debootstrap devs?
  • (anonymous) I don't like the idea of building vbox in vbox because of performance reasons but also how would one bootstrap such system? The very first t-w one builds needs to be built outside of a t-w...
    • (adrelanos) I understand your concern, those are valid. Performance is indeed poor. It's a real waste of time. Development were faster if they run faster. Bootstrapping is also complicated. The existing devs already have a working Whonix, but new ones? Two solutions: 1. we find a packager who does not require to stay anonymous (low probability) or 2. provide instructions to build a minimal build system manually (as we had in past). Update: Building manually will be easy as soon we move to more file separated source.
  • (anonymous) with debootstrap at least one doesn't have to run nested VMs for the heavy lifting. VBox is only needed for configuring the VM settings and then exporting the already prepared vdi.
    • (adrelanos) Yes. Good. Controlling things inside VBox would have been a mess anyway.
  • (anonymous) basics are done and working. Missing stuff: debconf dialog for certain packages, virtualbox scripting and export. that can all be found in TODOs directly in the script. ticket can be closed?
  • (adrelanos) I can't yet fully understand the new scripts you created. Not your fault! I just have to to read more manuals etc, there are a lot changes in a very short time. Thanks for that work. This is no criticism.
  • (adrelanos) That said, can you update BuildDocumentation as well? Imho now it's not clear how to build. I think the manual installation followed by a T-* script should be deprecated. BuildDocumentation should walk through the automated build method.
  • (adrelanos) I am not sure debootstrap is the right way to go. I spend hours without significant progress. There are too much strange bugs. /var/run not mounted on shutdown/restart (ok, thats maybe minor). dpkg-reconfigure -a hangs on adduser (can always press yes/no, but change will not be recognized). This bug is so seldom, you don't even find it on google. Even if reported, I expect it to be ages until fixed. We're attempting what vmbuilder did when jeos was still an active project. That project is now inactive and full of blocking bugs. (I couldn't get build virtualbox images, perhaps someone else can get it.) When the project was still current, they had way more developers (some with @ubuntu mail address). Debootstrap/chroot is (almost) equally hard as linux from scratch and this is far outside my skill set. ubuntu-vm-builder (python-vm-builder) also supports building kvm, vmware server, vmware 6 or xen images. Some of those formats are probable working better than virtualbox format. I tried building a vmware server image (vdmk) and simply used it for Virtual Box. I couldn't imagine that beside the disk format and the description file, the content of the image differs. That's wasn't a good hack either, also dpkg-reconfigure -a is broken and even apt-get update will show hdd kernel errors. Imho we can't do the "vmdebootstrap thing" our self. There are some alternative projects. https://gitorious.org/vmdebootstrap http://wiki-oar.imag.fr/index.php/Kameleon maybe others. I'd like to carefully choose the best one of them. I am willing to switch to KVM and/or Debian Squeeze (if in mid term update to Wheezy might work) right away if necessary.
  • (adrelanos) And even if we get vm-de-bootstrap or our own debootstrap or anything else running... It's still a custom hack. As soon the system makes any trouble you have some custom hacked up installation. Very very few people go this way = very badly supported, very hard to find solutions. Perhaps we should go back to preseed, that's virtually using the installer without user interaction. The system will be a "standard" one.
  • (adrelanos) I am trying this (with kickstart only, preseed might be broken for Ubuntu 12.04, still investigating). New plan:
    build script will run the following scripts... Easy to enable/disable steps for debugging.
    https://trac.torproject.org/projects/tor/wiki/doc/TorBOX/Dev/Build
    download and verify cd https://trac.torproject.org/projects/tor/wiki/doc/TorBOX/Dev/GetISO
    modify iso (preseed / kickstart) https://trac.torproject.org/projects/tor/wiki/doc/TorBOX/Dev/ModifyISO
    create vm / start vm / install vm (without network card) https://trac.torproject.org/projects/tor/wiki/doc/TorBOX/Dev/CreateVM
    mount virtual hdd / ChangeRoot / leave chroot / unmount virtual hdd
    https://trac.torproject.org/projects/tor/wiki/doc/TorBOX/Dev/ChangeRoot
    T-G script https://trac.torproject.org/projects/tor/wiki/doc/TorBOX/Dev/TGScript
    T-W script https://trac.torproject.org/projects/tor/wiki/doc/TorBOX/Dev/TWScript
    export vm
    cleanup/clean
    
  • (adrelanos) This is probable are more realistic plan and much easier to implement.
  • (anonymous) about dpkg-reconfigure: if you report such bugs on launchpad, I doubt you'd get much feedback. Debian cares a bit more and since it's the upstream for 'deb'ootstrap I think there are good chances it will be fixed. The debian iso as well as Ubuntu server are actually using debootstrap too... It's probably just some missing package. Nothing really broken either, the configuration is still set. /var/run could be a mtab/fstab problem. What other bugs are there? The debootstrapped images work, there can only be minor problems and they can all be fixed. Really, installing via iso and debootstrap is the same thing techincally. Just one of the two is muuuch faster.
  • (adrelanos) Dunno, don't remember very well anymore. The dpkg-reconfigure bug, the "console-utilities console-tools console-data something broken" bug, and 'locale' returned an error, "Scroll wheel in VM" bug. Perhaps that was it. Unfortunately, I really was unable to fix them. That's why I worked on the preseed method, btw I am almost done and can build and release them. Creating images with debootstrap without running something in VMs is certainly more elegant. Using preseed has the advantage that that VM never gets access to /dev/..., therefore less likely to leak information into VM.
  • (adrelanos) Removed [0.2] tag because I am going to release 0.2 using preseed method.
  • (adrelanos) I am currently testing Kameleon. Although it's from 2010, first impression is very good. Looks like it needs to run on Debian to create a Debian VM, but switching to Debian might be a good idea so or so.
  • (adrelanos) Another interesting similar tool for this purpose: Oz is a tool for automatically installing guest OSs with only minimal up-front input from the user.; Playing with oz-install; on github.
  • Done. Switched to grml-debootstrap/chroot. Perhaps in future to kameleon if the adduser bug with "dpgk-reconfigure -a" is gone.
Last modified 5 years ago Last modified on Apr 7, 2013, 5:49:23 PM