Changes between Version 5 and Version 8 of Ticket #30020


Ignore:
Timestamp:
Aug 15, 2019, 8:31:59 PM (5 weeks ago)
Author:
anarcat
Comment:

we now have:

  • 3 has_role references
  • 4 roles left (haproxy, mail_processing, natted, no_hw_clock)
  • 2 localinfo references (in postfix, related to mail_processing)
  • 13 allnodeinfo references
  • 26 nodeinfo references

That's on the stuff that I started working on at all. The hoster.yaml stuff, in particular, is a whole other ball game. It's less work than the larger local.yaml, but still impacts a lot of things, which are mostly visible in the nodeinfo calls:

anarcat@curie:tor-puppet(master)$ git grep -c nodeinfo
manifests/site.pp:2
modules/bind/templates/named.conf.puppet-shared-keys.erb:1
modules/ferm/templates/defs.conf.erb:6
modules/hosts/templates/etc-hosts.erb:2
modules/motd/templates/motd.erb:11
modules/ntp/templates/ntp.conf:1
modules/postfix/templates/main.cf.erb:2
modules/postgres/manifests/backup_server/register_backup_clienthost.pp:1
modules/puppetmaster/lib/puppet/parser/functions/allnodeinfo.rb:2
modules/puppetmaster/lib/puppet/parser/functions/nodeinfo.rb:22
modules/resolv/templates/resolv.conf.erb:3
modules/roles/manifests/onionoo_backend.pp:2
modules/syslog_ng/templates/syslog-ng.conf.erb:1
modules/torproject_org/manifests/init.pp:2
modules/unbound/manifests/init.pp:4
modules/unbound/templates/unbound.conf.erb:4

Similarly, the allnodeinfo construct imports a lot of stuff from LDAP into Puppet, which we might want to move into Hiera. That, however, could be left for a second phase as it would significantly disrupt the current host lifetime workflow.

The status of the YAML file conversion is as follows:

  • ./modules/torproject_org/misc/hoster.yaml: not started
  • ./modules/torproject_org/misc/local.yaml: 53/57 roles done! almost finished, see below for the status of the remaining 4
  • ./modules/ipsec/misc/config.yaml: will be phased out in favor of the new exported resource system built for the new networks on fsn-node-*
  • ./modules/roles/misc/static-components.yaml: maybe easier to keep as such for now, or rewrite the static backend to read the file directly?
  • ./modules/roles/files/spec/spec-redirects.yaml: unsure

The remaining roles are:

  • haproxy: required for syslog-ng configuration, switching to rsyslog would make this easier
  • mail_processing: requires a refactoring of the postfix module
  • natted: small refactoring the hosts module, ignore the nodeinfo stuff, it's not used anywhere according to weasel
  • no_hw_clock: small refactoring of the NTP and torproject_org modules

The bulk of the work will be with mail_processing and, obviously, with the syslog transition if we go that route.

Legend:

Unmodified
Added
Removed
Modified
  • Ticket #30020

    • Property Status changed from assigned to accepted
  • Ticket #30020 – Description

    v5 v8  
    1313 * the `$localinfo` into Hiera (assuming all the data is there)
    1414 * the `$nodeinfo` and `$allnodeinfo` arrays into Hiera (assuming we can switch from LDAP for host inventory)
    15  * basically any other stuff of the kind
     15 * basically any other stuff of the kind, including those files:
     16   {{{
     17./modules/torproject_org/misc/hoster.yaml
     18./modules/torproject_org/misc/local.yaml
     19./modules/ipsec/misc/config.yaml
     20./modules/roles/misc/static-components.yaml
     21./modules/roles/files/spec/spec-redirects.yaml
     22}}}
    1623
    1724Ideally, all YAML data should end up in the hiera/ directory somehow. This is the first step in making our repository public (#29387) but also using Hiera as a more elaborate inventory system (#30273).