Opened 7 months ago

Closed 6 weeks ago

#29888 closed defect (fixed)

retire nova

Reported by: weasel Owned by: tpa
Priority: Medium Milestone:
Component: Internal Services/Tor Sysadmin Team Version:
Severity: Normal Keywords:
Cc: Actual Points:
Parent ID: Points:
Reviewer: Sponsor:

Description

remove from NSset and delegation,
then zero the disk

Child Tickets

Attachments (1)

screenshot.png (255.7 KB) - added by anarcat 8 weeks ago.

Download all attachments as: .zip

Change History (15)

comment:1 Changed 7 months ago by anarcat

... because it's having hardware issues, right? at least i am getting SMART warnings... and it's just doing DNS on bare metal so it's too "heavy" and could be replaced with a VM swapped with other orga eventually, if we do need *six* DNS servers :)

comment:2 Changed 2 months ago by anarcat

totally forgot about this ticket. do we actually want to retire this host? I can do that next week if it's still relevant.

comment:3 Changed 8 weeks ago by anarcat

i removed ns2.torproject.org from the torproject.net zone, both in git (so on the primary) and in joker (so in the delegation). i don't believe there are glue records there. the TTL is 12h on there, so I'll wait at least that time before making any further changes to see if anything breaks (i doubt so, but still a good precaution).

there are two domains I'm unsure what to do about:

  • onion-router.net - I don't have access to the registrar (Network Solutions?) for this one
  • torsolutionscorp.com - it's in joker, but not in our DNS configuration - ignore? it doesn't resolve anyways

Finally, i'm not sure how to "zero the disk" - i don't, as far as I know, have a remote management interface on this machine, so I don't know how I could do that step. i'll be happy to perform all other steps of course, once the DNS changes are complete.

comment:4 Changed 8 weeks ago by anarcat

i also retired the ns2 record from the torproject.com zone and whois delegation as well.

i'll retire it from torproject.org and glue records tomorrow and start the host retirement procedure on wednesday.

comment:5 in reply to:  3 Changed 8 weeks ago by weasel

Replying to anarcat:

there are two domains I'm unsure what to do about:

  • onion-router.net - I don't have access to the registrar (Network Solutions?) for this one

AIUI, that site is historically interesting. If we want to update its delegation, I suspect we need to ask Paul. For now we could probably just point ns2.tpo to one of our remaining nameservers.

  • torsolutionscorp.com - it's in joker, but not in our DNS configuration - ignore? it doesn't resolve anyways

It's parked.

comment:6 Changed 8 weeks ago by anarcat

i was planning on removing ns2.torproject.org altogether, glue records and all. if we keep it on onion-router.net, it will be more difficult to operate that change...

but then again, the reverse is also true: if we remove ns2 and then later want to add it back, it will be more difficult to do that on onion-router.net as well.

which NS should we point ns2 at anyways?

comment:7 Changed 8 weeks ago by anarcat

just removed ns2 from torproject.org and all reverse DNS zones. i have also removed its glue record, which is hidden in "My Joker -> My Nameservers".

i kept the A, AAAA, and TXT records for ns2.torproject.org so that onion-router.net still works, while we decide how we handle this one.

comment:8 Changed 8 weeks ago by anarcat

i also "renumbered" ns2, ie. i pointed it at ns3's IP addresses, with a note in the zone file about the decision. i picked ns3 because it seems to be the least busy of the nameservers according to Grafana.

the rationale behind keeping a ns2.torproject.org record is that we might still have some glue records (unlikely) or zones (very likely, onion-router.net is one) pointing to it. even though we want to retire the machine, we might not want to deal with changes in that other zone, especially since we might switch DNS services anyways in the long term.

so this will be ready for decommissionning, which i'll finalize tomorrow.

comment:9 Changed 8 weeks ago by anarcat

nova is not receiving traffic anymore, so it can be decommissioned. proof:


So i'm following the retirement procedure:

  1. removed from nagios
  2. N/A
  3. N/A
  4. destroy disk: done
  5. LDAP: removed, no sudoers entry
  6. not present in dns/domains and related. reverse dns: done by weasel
  7. puppet: cleaned and deactivated, Submitted 'deactivate node' for nova.torproject.org with UUID b7ee2e76-9b86-40de-958d-ae98da796f41
  8. puppet: removed nova references in named.conf-puppet-shared-keys.erb and obsolete-packages-ignore.d-hostspecific.erb (commit 555dece7)
  9. removed from tor-passwords: only root password, the "extra" info was basically "ask weasel or dsa for remote mgmt"
  10. removed node from the spreadsheet
  11. backup removal scheduled on bungei for now + 30d
Last edited 6 weeks ago by anarcat (previous) (diff)

Changed 8 weeks ago by anarcat

Attachment: screenshot.png added

comment:10 Changed 8 weeks ago by anarcat

disk wipe procedure in progress, and documented in https://help.torproject.org/tsa/howto/retire-a-host/

comment:11 Changed 8 weeks ago by anarcat

this, for what it's worth, is the old IP addresses of nova:

root@nova:~# ip a
1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000
    link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
    inet 127.0.0.1/8 scope host lo
       valid_lft forever preferred_lft forever
    inet6 ::1/128 scope host 
       valid_lft forever preferred_lft forever
2: eth1: <BROADCAST,MULTICAST> mtu 1500 qdisc noop state DOWN group default qlen 1000
    link/ether 00:0e:0c:4e:9c:03 brd ff:ff:ff:ff:ff:ff
3: eth0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc fq state UP group default qlen 1000
    link/ether 00:0e:0c:4e:9c:02 brd ff:ff:ff:ff:ff:ff
    inet 86.59.30.40/28 brd 86.59.30.47 scope global eth0
       valid_lft forever preferred_lft forever
    inet6 2001:858:2:2:aabb:0:563b:1e28/64 scope global 
       valid_lft forever preferred_lft forever
    inet6 fe80::20e:cff:fe4e:9c02/64 scope link 
       valid_lft forever preferred_lft forever

in case we still need to access it and DNS doesn't give that info for us (as it should)

comment:12 Changed 8 weeks ago by anarcat

nova removed from LDAP

comment:13 Changed 6 weeks ago by anarcat

final destruction of the host was completed. upstream was notified that the machine can be physically taken out of service and has been taken out of reverse DNS.

comment:14 Changed 6 weeks ago by anarcat

Resolution: fixed
Status: newclosed
Note: See TracTickets for help on using tickets.