Opened 7 months ago

Closed 6 months ago

#29796 closed defect (fixed)

synchronize puppet and LDAP hosts

Reported by: anarcat Owned by: tpa
Priority: Medium Milestone:
Component: Internal Services/Tor Sysadmin Team Version:
Severity: Normal Keywords:
Cc: Actual Points:
Parent ID: Points:
Reviewer: Sponsor:

Description (last modified by anarcat)

We have hosts that are in Puppet and not in LDAP and vice versa. Every host in LDAP should be in Puppet and vice versa.

We have 78 hosts in LDAP and 74 in Puppet, with 73 hosts in common. This is the current diff:

$ diff puppet ldap 
29a30,31
> geyeri.torproject.org
> gillii.torproject.org
36d37
< hyalinum.torproject.org
74a76,78
> weissii.torproject.org
> winklerianum.torproject.org
> woronowii.torproject.org

That is, right now, we have the following hosts in LDAP but not in Puppet:

  • geyeri.torproject.org
  • gillii.torproject.org
  • weissii.torproject.org
  • winklerianum.torproject.org
  • woronowii.torproject.org

The following is in Puppet, but not LDAP:

  • hyalinum.torproject.org

The two lists (puppet and ldap) were obtain using the following commands:

ssh -t pauli.torproject.org 'sudo -u postgres psql puppetdb -P pager=off -A -t -c "SELECT c.certname FROM certnames c WHERE c.deactivated IS NULL"' | tee puppet
tail -n +2 puppet | sort | sponge puppet
ssh alberti.torproject.org 'ldapsearch -h db.torproject.org -x -ZZ -b dc=torproject,dc=org -LLL "hostname=*.torproject.org" hostname | awk "\$1 == \"hostname:\" {print \$2}" | sort' > ldap

... as detailed in the new Puppet docs.

I'm not exactly sure how to resolve this. When weasel saw a previous version of this list, he said:

12:30:00 <weasel> from a quick glance, all but the arm hosts can go.
12:30:06 <weasel> best to double-check with ldap.
12:30:19 <weasel> if they are not in ldap, and they haven't done a puppet run in a while, they should be removed from puppet also.
12:30:45 <weasel> gillii and geyeri are the old CRM hosts.  I think linus wants to kill them soon but maybe keep them around (and offline) for now.

According to nagios, hyalinum has not checked into Puppet since 2018-02-12T08:53:13.339Z, over a month ago. So presumably that should be removed from puppet, and we should double-check the retirement procedure to see if it was completed correctly.

The hosts in LDAP and not in Puppet should probably be added to puppet, carefully (--noop is your friend) to see if it breaks anything.

In the future, we might want to add a Nagios check on the Puppet server to make sure this is synchronized.

Child Tickets

Change History (3)

comment:1 Changed 7 months ago by anarcat

Description: modified (diff)

comment:2 Changed 7 months ago by anarcat

the box not in LDAP should be removed from LDAP:

12:34:11 <weasel> hyalinum is an old arm box.  it's dead.
12:34:24 <anarcat> ack, so i'll do the retirement procedure for it

the other boxes:

  • geyeri.torproject.org
  • gillii.torproject.org
12:36:23 <weasel> geyeri and gillii are on their way out
12:36:32 <weasel> they are the old CRM systems
12:36:40 <weasel> ln is owning that migration
12:36:54 <weasel> maybe they can entirely be removed from everything already, he'd know
  • weissii.torproject.org
  • winklerianum.torproject.org
  • woronowii.torproject.org
12:37:59 <weasel> weissii, winklerianum, woronowii are windows VMs.  I don't know why we would add them to puppet.
12:38:33 <anarcat> hmm...
12:38:39 <anarcat> it would be an interesting experiment i guess :p
12:38:52 <anarcat> but puppet can run on windows, we could add them to puppet for the same reason we add all the other machines
12:39:16 <weasel> and then write manifests that work on both?
12:39:19 <anarcat> but i guess that answers my question (windows stuff, here lies dragons, low prio)
[...]
12:39:55 <weasel> I don't see that happening any time soon, or ever :)

comment:3 Changed 6 months ago by anarcat

Resolution: fixed
Status: newclosed

i don't remember how hyalinum was removed, but geyeri and gillii have both been cleaned up by ln5, and the remaining ones are windows machines so there's nothing much we can do here.

Note: See TracTickets for help on using tickets.