Opened 2 months ago

Last modified 3 weeks ago

#33733 assigned task

How do home directories work?

Reported by: irl Owned by: irl
Priority: Medium Milestone:
Component: Internal Services/Tor Sysadmin Team Version:
Severity: Normal Keywords: metrics-team-roadmap-2020
Cc: metrics-team Actual Points: 0.1
Parent ID: #33715 Points: 0.5
Reviewer: Sponsor:

Description

There seems to be little consistency here, which isn't what I expect from an orchestrated process, so I'm maybe missing something.

Each service has a directory in /srv/{service}.torproject.org/ and then sometimes there is a home directory, which is sometimes linked in some way to /home/{service}. When there are multiple users for a service, they can share the same /srv directory but then have inconsistent naming of home directories.

Is there some documentation I can read to make sense of this?

Context: I'm putting together our Ansible roles (#33715) that should replicate what TPA will give us when we move things to a TPA host after we're convinced it's ready for deployment and we know what the specs will be, but I'm having trouble generalising even from just the Onionoo and Exit Scanner setups.

I'd like to be able to set some variables, like what usernames exist, what groups exist, and what paths will exist and should be used for stuff, and then let this role set that up. The service specific (e.g. Onionoo or Exit Scanner) roles will then run equally on our AWS dev instances and the production TPA instance.

Child Tickets

Change History (10)

comment:1 Changed 2 months ago by irl

Cc: metrics-team added

comment:2 in reply to:  description Changed 2 months ago by anarcat

Status: newneeds_review

Replying to irl:

There seems to be little consistency here, which isn't what I expect from an orchestrated process, so I'm maybe missing something.

You're right! It's not fully orchestrated yet. Some of those things were created by hands before Puppet, most are managed in LDAP (by hand as well) and some are managed by Puppet *and* LDAP. It's a bit all over the place.

You're dealing with old code here. Sorry for the inconvenience! :)

Each service has a directory in /srv/{service}.torproject.org/ and then sometimes there is a home directory, which is sometimes linked in some way to /home/{service}. When there are multiple users for a service, they can share the same /srv directory but then have inconsistent naming of home directories.

What do you mean "multiple users"? Could you give an example here specifically?

Is there some documentation I can read to make sense of this?

Unfortunately: not much. I created "operational" docs that deal with "how to create a normal or role user", for example, in:

https://help.torproject.org/tsa/howto/create-a-new-user/

Then some user-facing documentation and policy is available in:

https://help.torproject.org/tsa/doc/accounts/

But I understand you might not have found those or that, if you did, they were not satisfactory.

This is not really standardized in Puppet either, as I have just begun refactoring that code. The fact that it's also bound to LDAP complicates things as well because Puppet cannot (currently?) *write* to LDAP so manual operations need to be done on both systems for things to happen.

My hope is that we can eventually replace LDAP for some of the role stuff, and move that into puppet, to have a more uniform experience. But it's a far-ranging project, as it involves touching ud-ldap, which is a huge challenge.

The closest I've got to "standardizing" this is when we setup the new check service. I ended up writing this Puppet profile:

# rewrite of the exit scanner
#
# not to be confused with the old roles::check that is now deprecated.
class profile::check inherits apache2 {
  # disable the default site so the FQDN site alias works
  Apache2::Site['00-default'] {
    ensure => absent,
  }

  include apache2::ssl
  include apache2::proxy_http

  $users = ['tordnsel', 'check']
  $users.each |String $user| {
    file { [
      "/srv/${user}.torproject.org",
      "/srv/${user}.torproject.org/home",
      "/srv/${user}.torproject.org/htdocs",
    ]:
      ensure => 'directory',
      mode   => '0775',
      owner  => $user,
      group  => $user,
    }
    # LDAP expects the home there
    file { "/home/${user}":
      ensure => link,
      target => "/srv/${user}.torproject.org",
    }
    loginctl_user { $user:
      linger => enabled,
    }
    file { "/etc/sudoers.d/${user}":
      mode    => '0440',
      content => @("EOF"),
      # FILE MANAGED BY PUPPET, LOCAL CHANGES WILL BE LOST
      # allow ${user} user sudo access to ${user} group
      %${user}			ALL=(${user})			ALL
      | EOF
    }
  }

  $vhost_name = 'check.torproject.org'
  apache2::site { $vhost_name:
    content => template('profile/check/vhost.erb'),
  }

  ssl::service { 'check.torproject.org': notify  => Exec['service apache2 reload'], key => true, }

  ensure_packages('tor', {ensure => installed})
  # NOTE: tor service is ran by the service admin (through systemd
  # --user), so we mask it here (in systemd --system)
  dsa_systemd::mask { [
    'tor.service',
    'tor@default.service'
  ]:
    notify => Package['tor'],
  }

  # new exit scanner dependencies, see #33362
  ensure_packages(
  # ...
  )
}

That $users.each |String $user| { should really be a new resource that would standardize the way users are created from Puppet's perspective. You can also see the symlink hack to take into account the LDAP home directory there.

Context: I'm putting together our Ansible roles (#33715) that should replicate what TPA will give us when we move things to a TPA host after we're convinced it's ready for deployment and we know what the specs will be, but I'm having trouble generalising even from just the Onionoo and Exit Scanner setups.

Which problem are you having specifically? Hopefully the above code sample should provide you with some guidance, but I understand it's not documentation and I apologize for the lack of standardization in that regard.

I'd like to be able to set some variables, like what usernames exist, what groups exist, and what paths will exist and should be used for stuff, and then let this role set that up. The service specific (e.g. Onionoo or Exit Scanner) roles will then run equally on our AWS dev instances and the production TPA instance.

That shouldn't be a problem: we basically do the same in Puppet, even though the users and groups are actually created in LDAP.

I hope that helps, let me know if you have any further questions!

(PS: one thing that would be useful for me is to know where you look for this documentation, instinctively. Then I could go *there* and add a pointer to the existing documentation. When you're happy with the answers here, I would also go there and add the missing bits. :)

comment:3 Changed 2 months ago by irl

How does the above puppet compare to the Onionoo setup in Puppet?

Can we make that use the same "standardization" as the check stuff before we deploy the new backends (#32268) there?

I think there may be a case not covered by it where we have the onionoo and onionoo-unpriv users, but they both share the same service directory.

I am confused by the difference between the /home/$user and /srv/$service/home directories. For the exit scanner the home directory is linked into /srv/$service and not into /srv/$service/home but for Onionoo:

ssh onionoo-backend-01.torproject.org ls -l '/home/onionoo*' 
lrwxrwxrwx 1 root root 32 Sep  6  2019 /home/onionoo -> /srv/onionoo.torproject.org/home
lrwxrwxrwx 1 root root 39 Sep  6  2019 /home/onionoo-unpriv -> /srv/onionoo.torproject.org/home-unpriv

Is the /home path only there to keep LDAP happy? Perhaps I need to add a key to users for the "real" home directory path as we can't guess it from some pattern, and then always link /home/$user to that specified path.

Regarding where to look for documentation, I had no idea. I guess help.tpo would have been the place to look. For this though I don't so much need a list of steps that you take to do something, or examples of the Puppet usage, but more an understanding of the rationale and intention of doing it in this way so that I'm not making incorrect assumptions when I recreate it for the dev environments.

comment:4 Changed 2 months ago by anarcat

How does the above puppet compare to the Onionoo setup in Puppet?

This is the onionoo_backend class, for example:

# onionoo.torproject.org backend host
class roles::onionoo_backend {
  package { [
    'libcommons-codec-java',
    'libcommons-compress-java',
    'libcommons-lang3-java',
    'libgoogle-gson-java',
    'liblogback-java',
    'libservlet3.1-java',
    'libslf4j-java',
    'libjetty9-java',
    'default-jdk-headless',
  ]:
    ensure => installed,
  }

  file { '/srv/onionoo.torproject.org':
    ensure => directory,
    mode   => '0755',
    owner  => 'onionoo',
    group  => 'onionoo',
  }
  file { '/srv/onionoo.torproject.org/home':
    ensure => directory,
    mode   => '0755',
    owner  => 'onionoo',
    group  => 'onionoo',
  }
  file {'/home/onionoo':
    ensure => link,
    target => '/srv/onionoo.torproject.org/home',
  }
  file { '/srv/onionoo.torproject.org/home-unpriv':
    ensure => directory,
    mode   => '0755',
    owner  => 'onionoo-unpriv',
    group  => 'onionoo-unpriv',
  }
  file {'/home/onionoo-unpriv':
    ensure => link,
    target => '/srv/onionoo.torproject.org/home-unpriv',
  }
  file { '/srv/onionoo.torproject.org/onionoo':
    ensure => directory,
    mode   => '0755',
    owner  => 'onionoo',
    group  => 'onionoo',
  }
  file { '/etc/sudoers.d/onionoo':
    mode   => '0440',
    source => 'puppet:///modules/roles/onionoo_backend/sudoers-onionoo',
  }

  dsa_systemd::linger { 'onionoo': }
  dsa_systemd::linger { 'onionoo-unpriv': }

  # varnish
  class { 'varnish':
    vcl_config  => '/etc/varnish/onionoo.vcl',
    vcl_content => template('roles/onionoo/varnish-onionoo-backend.vcl.erb'),
    memory      => '1g',
  }

  # ipsec tunnels between each frontend and each backend (1:1 tunnels)
  $query = 'nodes[certname] { resources { type = "Class" and title = "Roles::Onionoo_frontend" } }'
  $peer_names = sort(puppetdb_query($query).map |$value| { $value["certname"] })
  $peer_names.each |$peer_name| {
    $network_tag = [$::fqdn, $peer_name].sort().join('::')
    ipsec::network { "ipsec::${network_tag}":
      peer_networks => $base::public_addresses,
    }
  }

  # allow access to varnish from ipsec peers
  ferm::rule { 'ipsec-service-chain':
    domain      => '(ip ip6)',
    description => 'Move incoming ipsec processed packets to a dedicated ipsec chain',
    rule        => 'mod policy dir in pol ipsec goto ipsec',
  }
  ferm::rule::simple { 'ipsec-varnish':
    chain => 'ipsec',
    port  => 6081,
  }
}

It's similar, but not exactly the same, as you can see.

Can we make that use the same "standardization" as the check stuff before we deploy the new backends (#32268) there?

weasel has been taking care of this so far, so I can't speak for him.

i think that would be a great idea.

I think there may be a case not covered by it where we have the onionoo and onionoo-unpriv users, but they both share the same service directory.

I see, yes, that's something that needs to be considered.

I am confused by the difference between the /home/$user and /srv/$service/home directories. For the exit scanner the home directory is linked into /srv/$service and not into /srv/$service/home but for Onionoo:

ssh onionoo-backend-01.torproject.org ls -l '/home/onionoo*' 
lrwxrwxrwx 1 root root 32 Sep  6  2019 /home/onionoo -> /srv/onionoo.torproject.org/home
lrwxrwxrwx 1 root root 39 Sep  6  2019 /home/onionoo-unpriv -> /srv/onionoo.torproject.org/home-unpriv

That could be an oversight on my part, when i setup the check service. Do you want this fixed?

Is the /home path only there to keep LDAP happy?

That might be so yes.

Perhaps I need to add a key to users for the "real" home directory path as we can't guess it from some pattern, and then always link /home/$user to that specified path.

I think you can assume /home/$user is valid, although it can be a symlink pointing somewhere else.

Regarding where to look for documentation, I had no idea. I guess help.tpo would have been the place to look. For this though I don't so much need a list of steps that you take to do something, or examples of the Puppet usage, but more an understanding of the rationale and intention of doing it in this way so that I'm not making incorrect assumptions when I recreate it for the dev environments.

Yeah, that's what's missing I guess. To be honest, I don't quite know what the rationale is here either, I just got here. ;)

comment:5 Changed 2 months ago by anarcat

Owner: changed from tpa to irl
Status: needs_reviewassigned

waiting for answers from irl

comment:6 Changed 8 weeks ago by irl

From today's journal (no action needed as yet, but maybe interesting to see what I'm doing):

16:59 metrics-common looking in good shape

The metrics-common role appears to prepare the system for the oninooo-backend role with no modification required to the onionoo-backend role, so that's a sucess. If the same can be achieved for the exit-scanner role then this ticket can likely be closed. The YAML variables would also make a decent description for giving to TSA when we want a new host from them.

An example looks like this:

https://gitweb.torproject.org/user/irl/metrics-cloud.git/tree/ansible/group_vars/onionoo_backend.yml

It defines the directories we want created, and the users we want created. I've not managed to come up with a scenario where we want a user but we don't want to have it linger, so there is an assumption that all service users will linger.

I was wondering how Onionperf might fit into this. We've worked on the assumption so far that Onionperf is something that we run on systems that we manage, but at some point we might want to have it run as a service on a TSA host. We should try to see if we are able to seperate the parts that require root privileges from the parts that don't, and what each of those parts looks like.

The onionoo-backend role is still running so I'll find out tomorrow if it worked or not, but it's at the stage where it's actually importing data so I figure it's worked.

comment:7 Changed 8 weeks ago by gaba

Keywords: metrics-team-roadmap-2020April added

comment:8 Changed 7 weeks ago by irl

Actual Points: 0.1
Points: 0.5

comment:9 Changed 7 weeks ago by irl

Keywords: irl-roadmap-2020April added

comment:10 Changed 3 weeks ago by gaba

Keywords: metrics-team-roadmap-2020 added; metrics-team-roadmap-2020April irl-roadmap-2020April removed

We need to review all this tickets for metrics roadmap.

Note: See TracTickets for help on using tickets.