Changes between Version 13 and Version 14 of doc/TorBOX/Dev/BuildDocumentation/0.3.0


Ignore:
Timestamp:
Sep 27, 2012, 11:35:44 PM (7 years ago)
Author:
proper
Comment:

--

Legend:

Unmodified
Added
Removed
Modified
  • doc/TorBOX/Dev/BuildDocumentation/0.3.0

    v13 v14  
    1 [[TOC(noheading, depth=0)]]
    2 [https://trac.torproject.org/projects/tor/wiki/doc/TorBOX aos Homepage]
     1TorBOX has been renamed to Whonix.
    32
    4 '''NOT FINISHED!
     3This page has been moved. The History of this page might still be interesting.
    54
    6 {{{
    7 # Copyright:
    8 # adrelanos (aka proper)
    9 # adrelanos (at) riseup (dot) net
    10 #
    11 # License:
    12 # GPL v3 or any later
    13 #
    14 # Any changes you pull into this source will be also licensed
    15 # under GPL v3 or any later. Additionally you grant adrelanos the right to
    16 # re-license your work under a different license. If that is not acceptable,
    17 # you can either fork this source under GPL v3 or any later or contact
    18 # adrelanos. Contact adrelanos, if you require this source code under
    19 # different license.
    20 #
    21 # Authors:
    22 # adrelanos (aka proper)
    23 # Big contributions from anonymous.
    24 # Leaktest and other stuff contributed by smarm.
    25 }}}
    26 
    27 This page documents how the [https://trac.torproject.org/projects/tor/wiki/doc/TorBOX/Download binary distribution images] are built. If you have any questions or need help let us know on [https://trac.torproject.org/projects/tor/wiki/doc/TorBOX/Dev#Questions /aos/Dev#Question].
    28 
    29 Following these instructions will build version aos 0.3.0 based on Tor 0.2.3 and Ubuntu Precise.
    30 
    31 Knowledge assumed: Virtualization and networking basic principles; operation of your platform; Linux knowledge: how to install Ubuntu and basic command line knowledge.
    32 
    33 Only one prerequisites: you need a working internet connection.
    34 
    35 For discussion related to the development and build process of aos images go [https://trac.torproject.org/projects/tor/wiki/doc/TorBOX/Dev/ aos/Dev/].
    36 
    37 = Build Anonymity =
    38 While downloading the required tools for building aos your internet service provider could if he want notice that you want to build aos. This is esspecially interesting, if you want to redistribute aos, but still want to stay anonymous. The full story can be read in the chapter [https://trac.torproject.org/projects/tor/wiki/doc/TorBOX/SecurityAndHardening#BuildAnonymity Build Anonymity].
    39 
    40 = Build Security =
    41  * Build on a dedicated build system, install security updates...
    42  * All install media and all downloaded/used code must be verified (including all software on the host).
    43  * Hashes, fingerprints  in the scripts and the wiki is not to be trusted. Verify everything.
    44  * Read [https://trac.torproject.org/projects/tor/wiki/doc/TorBOX/Trust aos/Trust]
    45 
    46 = Host preparation =
    47 '''Read and apply if necessary [https://trac.torproject.org/projects/tor/wiki/doc/TorBOX/Readme#NetworkTimeSyncing "Network Time Syncing"]!
    48 
    49 We recommend you use a dedicated OS installation just for hosting the aos VMs (See [https://trac.torproject.org/projects/tor/wiki/doc/TorBOX/SecurityAndHardening aos/SecurityAndHardening])
    50 
    51 You need to use Ubuntu. The build scripts could be adapted to run on other *NIX systems as well but currently they assume apt-get to be available. You need about 15 GB of free space.
    52 
    53 Install the latest security updates, install Virtual Box (and qemu-kvm which is required to mount the Virtual Box .vdi images). Reboot to apply kernel updates.
    54 {{{
    55 sudo apt-get update && sudo apt-get dist-upgrade
    56 sudo apt-get install virtualbox qemu genisoimage
    57 sudo reboot
    58 }}}
    59 
    60 If you are going to use Virtual Box inside Virtual Box, be sure to change your host key. Virtual Box -> Preferences -> Input -> Host Key. The "outside" and the "inside" Host Key must differ, otherwise you can not leave the VM "inside" anymore. The "outside" Virtual Box hostkey may NOT be ''ctrl''.^1^ [[BR]]
    61 ,, ^1^ Because the one is used by VBoxSDL "inside".
    62 
    63 Building on Windows is no longer supported. Redistributed aos builds should be build on Linux. If you want to port the aos build scripts to Windows, please contact us. Running aos on a Windows host with Virtual Box installed should is still possible.
    64 
    65 == Using an apt cache to speed up downloading ==
    66 '''OPTIONAL'''
    67 
    68 Does only work with aos-Gateway. [[BR]]
    69 ,, Does not work for aos-Workstation because it never gets direct access to the host by design. We could think about installing apt-cacher-ng on aos-Gateway, if there are any security implications.
    70 
    71 If you want to build multiple times (for debugging etc.), it might make sense to install a local apt proxy on your build machine. That safes download time and traffic. [[BR]]
    72 ,,Thanks to [https://linuxexpresso.wordpress.com/2011/02/13/howto-apt-cacher-ng-on-ubuntu/ source].
    73 
    74 On the host:
    75 {{{
    76 sudo apt-get install apt-cacher-ng
    77 }}}
    78 
    79 {{{
    80 sudo nano /etc/apt/apt.conf
    81 }}}
    82 
    83 {{{
    84 Acquire::http { Proxy "http://127.0.0.1:3142"; };
    85 }}}
    86 
    87 {{{
    88 sudo apt-get update
    89 }}}
    90 
    91 In aos_gateway source code: [[BR]]
    92 Go to /home/user/aos/aos_gateway/usr/local/bin/aos_internal_install_script script and comment in the line ''Acquire::http { Proxy "http://192.168.0.2:3142"; };'', i.e. remove the # (hash) in front of it.
    93 
    94 = Source Code Intro =
    95 '''If you prefer to read and understand the source code just by reading scripts you may skip this optional chapter.''' This chapter is dedicated to give an introduction into the aos source code. It can be quite difficult to get started with hacking existing big complex projects.
    96 
    97 Both Virtual Machines will be started two times while building aos. The first time the operating system will be installed and the second time either the aos_internal_install_script runs.
    98 
    99 '''aos_gateway''' All files within that folder will be copied into aos-Gateway. [[BR]]
    100 '''aos_workstation''' All files within that folder will be copied into aos-Workstation. [[BR]]
    101 '''aos_shared''' All files within that folder will be copied into aos-Gateway and aos-Workstation. [[BR]]
    102 Copying is done with aos_craetevm and implemented in int_copy_gateway, int_copy_workstation and int_copy_shared.
    103 
    104 '''int_xxx''' scripts are internally used by aos_xxx scripts.
    105 
    106 '''aos_build''' is a script, which simply runs all other scripts. Actually it's "optional". It has very little functionality beside running all other scripts. You are free to run all scripts one by one. That is useful for learning and for debugging purposes. In case you want to fix a bug or in case you want to upgrade the distribution or in case you want to switch the operating system or whatever you are better off running the steps manually. You can use the Build script as a reference for which steps have to be run in which order.
    107 
    108 '''aos_getiso''' downloads and verifies the operating system iso image.
    109 
    110 '''aos_modifyiso''' mounts the downloaded iso image as read only, copies it, modifies the copied iso image by adding preseeding (unattended installation), unmounts the iso and finally creates preseed.iso, which contains the operating system installer disc modified not to require any user interaction while installing. This process already sets up lots of important privacy settings and other stuff, such as UTC timezone, hostname ubuntu, US language, username user and so on. (See [https://trac.torproject.org/projects/tor/wiki/doc/TorBOX/SecurityAndHardening aos/SecurityAndHardening] and the modifyiso script itself for a list of all changes and why.) Preseed.iso is configured to power off itself when it's done installing so the aos_build script can continue.
    111 
    112 '''aos_createvm''' creates Virtual Box machines with all settings required for secure networking, devices, security settings. Which those settings are in details can be again read in Security And Hardening or the script itself. aos_createvm is also responsible for starting virtual machines while building. It also features to mount the virtual hdd images for debugging reasons. The aos_internal_install_script will be copied into the VMs by aos_createvm.
    113 
    114 '''aos_internal_install_script''' A aos-Gateway specific version is stored under /home/user/aos/aos_gateway/usr/local/bin/aos_internal_install_script and a aos-Workstation specific version is stored under /home/user/aos/aos_gateway/usr/local/bin/aos_internal_install_script. This script gets copied into the Virtual Machines by aos_createvm. aos_internal_install_script gets automatically started, only once at build time, by rc.local. They will transform the installed operating system into the aos-Gateway or into the aos-Workstation. They update the system, install security relevant software, install useful applications for an anonymous general purpose operating system, set up all relevant privacy and anonymity required settings, desktop manager etc.
    115 
    116 '''rc.local''' (/home/user/aos/rc.local) gets copied to into the VMs to /etc/rc.local and keeps care of automatically starting aos_internal_install_script. ^2^ The final version of rc.local, which the user will see, will be reverted by 'aos_createvm -tX-copyinto-post' and can be found under /home/user/aos/aos_shared/etc/rc.local.
    117 
    118 Thus, given the nature of the build step orientated scripts, you can easily work on the the different aspects of aos. For example, once you have created a clean virtual machine with the operating system only, you can make a clone or snapshot, run either the aos_Gateway or the aos_Workstation script as often as you need to test your changes and if something goes wrong, go back to the clone or snapshot. You don't have to build everything from scratch again. ^3^
    119 
    120 ,, ^3^ For example, we could add the aos_Gateway or aos_Workstation script to the preseed.iso and let it run after installing. If something would go wrong, you would have to reinstall the whole operating system every time again. That's why we use separate steps.
    121 
    122 = Get the aos source code =
    123 {{{
    124 git clone https://github.com/adrelanos/aos
    125 }}}
    126 
    127 = Create the Images =
    128 == Preparations ==
    129 1. Make the build script executable:
    130 {{{
    131 chmod +x ~/aos/aos_*
    132 }}}
    133 
    134 2. Make sure there aren't any VMs in Virtual Box already called "aos-Gateway" or "aos-Workstation" (TODO: automate that)
    135 
    136 3. Check if /home/user/aos/usr/share/version for version number.
    137 
    138 == VM Creation ==
    139 ,, -all not supported yet.
    140 
    141 1. Open a shell and type:
    142 {{{
    143 sudo ~/aos/aos_build -tg
    144 }}}
    145 
    146 2. Check if all went ok.
    147 
    148 3. Power on aos_Gateway
    149 
    150 4. Open a shell and type:
    151 {{{
    152 sudo ~/aos/aos_build -tw
    153 }}}
    154 
    155 The scripts can fail for many reasons, please report back any issues!
    156 
    157 = Debugging =
    158 '''OPTIONAL''' (Only in case something goes wrong or you want to audit or develop aos.)
    159 
    160 == SSH into aos-Gateway ==
    161 Inside aos-Workstation: [[BR]]
    162 {{{
    163 # Run once.
    164 sudo apt-get install opensshd-server
    165 }}}
    166 
    167 On the host: [[BR]]
    168 Add port forwarding from host into Virtual Machine.
    169 {{{
    170 # Run once.
    171 VBoxManage modifyvm "aos-Gateway" --natpf1 "ssh",tcp,127.0.0.1,2222,,22
    172 }}}
    173 
    174 On the host: [[BR]]
    175 Open a ssh session.
    176 {{{
    177 ssh user@127.0.0.1 -p 2222
    178 
    179 # Requires settings a root password beforehand.
    180 ssh root@127.0.0.1 -p 2222
    181 }}}
    182 
    183 On the host: [[BR]]
    184 Or mount aos-Gateway as a folder.
    185 {{{
    186 sshfs user@127.0.0.1:/ -p 2222 /home/user/aos_binary/aos-Gateway_sshfs
    187 
    188 # Requires settings a root password beforehand.
    189 sshfs root@127.0.0.1:/ -p 2222 /home/user/aos_binary/aos-Gateway_sshfs
    190 }}}
    191 
    192 == Mount and inspect images and aosinstalllogs ==
    193 {{{
    194 sudo ./aos_createvm -tg-mount
    195 
    196 cd "/home/user/aos_binary/aos-Gateway_image"
    197 leafpad "/home/user/aos_binary/aos-Gateway_image/home/user/aosinstalllog"
    198 cd ~
    199 
    200 sudo ./aos_createvm -tg-unmount
    201 }}}
    202 
    203 {{{
    204 sudo ./aos_createvm -tw-mount
    205 
    206 cd "/home/user/aos_binary/aos-Workstation_image"
    207 leafpad "/home/user/aos_binary/aos-Workstation_image/home/user/aosinstalllog"
    208 cd ~
    209 
    210 sudo ./aos_createvm -tw-unmount
    211 }}}
    212 
    213 = How to use the ova images =
    214 Reboot both VMs. Please read the [https://trac.torproject.org/projects/tor/wiki/doc/TorBOX/Readme aos/Readme]!
    215 
    216 = Final Steps (Only Required For Redistribution) =
    217  * [https://trac.torproject.org/projects/tor/wiki/doc/TorBOX/LeakTests Leak Testing]!
    218  * [https://trac.torproject.org/projects/tor/wiki/doc/TorBOX/TestAndTroubleshoot Test] the images before release! TODO: Needs big revision with all aos features.
    219  * Update the [https://trac.torproject.org/projects/tor/wiki/doc/TorBOX/Changelog Changelog].
    220  * Create hash sums for verification.
    221 {{{
    222 sha512sum aos-Gateway.ova
    223 sha512sum aos-Workstation.ova
    224 }}}
    225  * Upload the images.
    226  * Post hash sums to build documentation.
    227  * Post download links to build documentation.
    228  * At least a few testers should test before posting a news. Testers may be found by posting a news.
    229  * Finally announce: Post a news.
    230 
    231 = Changelog aos 0.3.0 =
    232 '''UNFINISHED'''
    233 
    234  * aos-Gateway and aos-Workstation
    235   * Added Secure Distributed Network Time Synchronization. Thanks to the Tails developers for their fine, free and Open Source tails_htp!
    236   * Added timesync gui.
    237   * Deactivated Virtual Box time synchronization.
    238   * Deactivating Virtual Box guest additions time synchronization if they are installed.
    239   * Creating a snapshots by default
    240   * Rebranding, the project is now called aos.
    241   * Added BitCoin address for donations.
    242   * Added adrelanos's gpg key.
    243   * Improved GPG verification mechanism.
    244   * torcheck renamed to aoscheck
    245   * Greatly improved aoscheck, now checks aos version, SocksPort, TransPort, stream isolation, Tor Browser version, operating system version and network time synchronization.
    246   * Improved uwt, uwt -t server_type -i ip -p port -c "<command> <options>...".
    247  * aos-Gateway
    248   * No no longer uses transparet proxying. Aos-Workstation can still use transparent proxying. Aos-Gateway now uses uwt for apt-get, gpg, ssh, (tails_)htpdate
    249   * Improved help file: aos.
    250   * Added unsafe user account, which can connect without Tor. Not used, unless user logs in as unsafe user.
    251   * Optional feature for /usr/local/bin/aos_firewall, when commented out (disabled by default), root user can connect without Tor.
    252   * Now using stream isolation and uwt.
    253  * aos-Workstation
    254   * Running every day aoscheck cron job hopefully fixes.
    255   * Partially fixed gnome-terminal black on black bug. Uses ugly colors and users are hopefully motivated to change the colors.
    256   * new commands:
    257    * xchat-reset (Deletes XChat configuration files and recreates the aos original ones, which are tweaked for privacy.)
    258    * hiddenserver-install (Installs and configures lighttpd)
    259  * Source Code
    260   * Now hosted on github, https://github.com/adrelanos/aos/ and therefore safe against malicious edits by random people.
    261   * All files are now inside their own files and no longer in single big scripts.
    262   * Deprecated onevm (no maintainer).
    263   * Deprecated uninstall and uninstall-vm (would require rewrite, was less tested, no users, we don't install the operating system manually anymore).
    264   * Error handling for everything.
    265   * Uwt wrappers share now most code.
    266   * Many fixes, more robustness, step based build system.
     5https://sourceforge.net/p/whonix/wiki/BuildDocumentation/