Opened 7 months ago

Last modified 7 months ago

#33586 new defect

cupani's IP address hardcoded in many places

Reported by: anarcat Owned by: tpa
Priority: Low Milestone:
Component: Internal Services/Tor Sysadmin Team Version:
Severity: Major Keywords:
Cc: Actual Points:
Parent ID: Points:
Reviewer: Sponsor:

Description

just in terms of SSH keys, the IP address of the cupani server is hardcoded in a lot of places:

anarcat@curie:tor-puppet(master)$ cumin-all 'grep -e 78.47.38.228 -e 2a01:4f8:211:6e8:0:823:4:1 /etc/ssh/userkeys/*' 
77 hosts will be targeted:
alberti.torproject.org,archive-01.torproject.org,bacula-director-01.torproject.org,build-arm-10.torproject.org,build-x86-[05-06,08-09].torproject.org,bungei.torproject.org,cache01.torproject.org,cache-02.torproject.org,carinatum.torproject.org,cdn-backend-sunet-01.torproject.org,check-01.torproject.org,chives.torproject.org,chiwui.torproject.org,colchicifolium.torproject.org,corsicum.torproject.org,crm-ext-01.torproject.org,crm-int-01.torproject.org,cupani.torproject.org,eugeni.torproject.org,fallax.torproject.org,forrestii.torproject.org,fsn-node-[01-04].torproject.org,gayi.torproject.org,gettor-01.torproject.org,gitlab-[01-02].torproject.org,henryi.torproject.org,hetzner-hel1-[01-03].torproject.org,hetzner-nbg1-[01-02].torproject.org,kvm[4-5].torproject.org,listera.torproject.org,loghost01.torproject.org,macrum.torproject.org,majus.torproject.org,mandos-01.torproject.org,materculae.torproject.org,meronense.torproject.org,moly.torproject.org,neriniflorum.torproject.org,nevii.torproject.org,nutans.torproject.org,omeiense.torproject.org,onionbalance-01.torproject.org,onionoo-backend-01.torproject.org,onionoo-frontend-01.torproject.org,oo-hetzner-03.torproject.org,orestis.torproject.org,palmeri.torproject.org,pauli.torproject.org,peninsulare.torproject.org,perdulce.torproject.org,polyanthum.torproject.org,rouyi.torproject.org,rude.torproject.org,scw-arm-par-01.torproject.org,static-master-fsn.torproject.org,staticiforme.torproject.org,submit-01.torproject.org,subnotabile.torproject.org,tbb-nightlies-master.torproject.org,troodi.torproject.org,unifolium.torproject.org,vineale.torproject.org,web-cymru-01.torproject.org,web-fsn-[01-02].torproject.org,web-hetzner-01.torproject.org
Confirm to continue [y/n]? y
                                                                                     |█████████████████████████████   |  91% (70/77) [00:58<00:06,  1.12hosts/s]
===== NODE GROUP =====                                                                                                                                               |█████████████████████████████   |  91% (70/77) [00:58<00:06,  1.12hosts/s]
(1) staticiforme.torproject.org                                                                                                                                      |█████████████████████████████   |  91% (70/77) [00:58<00:06,  1.12hosts/s]
----- OUTPUT of 'grep -e 78.47.38...c/ssh/userkeys/*' -----                                                                                                          |█████████████████████████████   |  91% (70/77) [00:58<00:06,  1.12hosts/s]
/etc/ssh/userkeys/torhelp:command="/srv/help-master.torproject.org/bin/update",no-port-forwarding,no-X11-forwarding,no-agent-forwarding,no-user-rc,from="78.47.38.228,2a01:4f8:211:6e8:0:823:4:1" ssh-rsa AAAAB3NzaC1yc2EAAAABIwAAAQEAw9/GOnsO3yhv8RRbIMTUEbFY3og+bzbz4vxaehMxddkVUkJDyDJZnwJRFr/1+q1kkasZoaYN19gfs8XaJnh9cSRNOAWquDqKjpu4z27aycQ3Pc3CtgJROvqTbgIdrq9UoWiFmXPJMCPepCe3T7y5kOmD5spSHTyVunT4fYNwGhILLkHPaDKXYRUIB4MrChuBW/tpzALTbRyWsXa9Ec3DBqSVrlKUXL4cZisa79mqJPmcfRphJFUFLwci1tifNlwOXtTTUqQTGLmt8GXJUEyU/6/QWDvkuq6B/paFnIdAAeH3mdOpXwbY7f+Y5uKl9+c7+i+9BxlJe6XB6S3H18Lbbw== git@cupani
                                                                                     |█████████████████████████████   |  91% (70/77) [00:58<00:06,  1.12hosts/s]
===== NODE GROUP =====                                                                                                                                               |█████████████████████████████   |  91% (70/77) [00:58<00:06,  1.12hosts/s]
(1) vineale.torproject.org                                                                                                                                           |█████████████████████████████   |  91% (70/77) [00:58<00:06,  1.12hosts/s]
----- OUTPUT of 'grep -e 78.47.38...c/ssh/userkeys/*' -----                                                                                                          |█████████████████████████████   |  91% (70/77) [00:58<00:06,  1.12hosts/s]
/etc/ssh/userkeys/gitweb:command="/srv/gitweb.torproject.org/bin/gitweb-ssh-wrap",no-port-forwarding,no-X11-forwarding,no-agent-forwarding,no-pty,from="78.47.38.228,2a01:4f8:211:6e8:0:823:4:1" ssh-rsa AAAAB3NzaC1yc2EAAAABIwAAAQEAw9/GOnsO3yhv8RRbIMTUEbFY3og+bzbz4vxaehMxddkVUkJDyDJZnwJRFr/1+q1kkasZoaYN19gfs8XaJnh9cSRNOAWquDqKjpu4z27aycQ3Pc3CtgJROvqTbgIdrq9UoWiFmXPJMCPepCe3T7y5kOmD5spSHTyVunT4fYNwGhILLkHPaDKXYRUIB4MrChuBW/tpzALTbRyWsXa9Ec3DBqSVrlKUXL4cZisa79mqJPmcfRphJFUFLwci1tifNlwOXtTTUqQTGLmt8GXJUEyU/6/QWDvkuq6B/paFnIdAAeH3mdOpXwbY7f+Y5uKl9+c7+i+9BxlJe6XB6S3H18Lbbw== git@cupani
                                                                                     |█████████████████████████████   |  91% (70/77) [00:58<00:06,  1.12hosts/s]
===== NODE GROUP =====                                                                                                                                               |█████████████████████████████   |  91% (70/77) [00:58<00:06,  1.12hosts/s]
(1) troodi.torproject.org                                                                                                                                            |█████████████████████████████   |  91% (70/77) [00:58<00:06,  1.12hosts/s]
----- OUTPUT of 'grep -e 78.47.38...c/ssh/userkeys/*' -----                                                                                                          |█████████████████████████████   |  91% (70/77) [00:58<00:06,  1.12hosts/s]
/etc/ssh/userkeys/tracweb:command="/srv/trac.torproject.org/bin/trigger-from-githost",no-port-forwarding,no-X11-forwarding,no-agent-forwarding,no-user-rc,from="78.47.38.228,2a01:4f8:211:6e8:0:823:4:1" ssh-rsa AAAAB3NzaC1yc2EAAAABIwAAAQEAw9/GOnsO3yhv8RRbIMTUEbFY3og+bzbz4vxaehMxddkVUkJDyDJZnwJRFr/1+q1kkasZoaYN19gfs8XaJnh9cSRNOAWquDqKjpu4z27aycQ3Pc3CtgJROvqTbgIdrq9UoWiFmXPJMCPepCe3T7y5kOmD5spSHTyVunT4fYNwGhILLkHPaDKXYRUIB4MrChuBW/tpzALTbRyWsXa9Ec3DBqSVrlKUXL4cZisa79mqJPmcfRphJFUFLwci1tifNlwOXtTTUqQTGLmt8GXJUEyU/6/QWDvkuq6B/paFnIdAAeH3mdOpXwbY7f+Y5uKl9+c7+i+9BxlJe6XB6S3H18Lbbw== git@cupani
                                                                                     |█████████████████████████████   |  91% (70/77) [00:58<00:06,  1.12hosts/s]
===== NODE GROUP =====                                                                                                                                               |█████████████████████████████   |  91% (70/77) [00:58<00:06,  1.12hosts/s]
(1) rouyi.torproject.org                                                                                                                                             |█████████████████████████████   |  91% (70/77) [00:58<00:06,  1.12hosts/s]
----- OUTPUT of 'grep -e 78.47.38...c/ssh/userkeys/*' -----                                                                                                          |█████████████████████████████   |  91% (70/77) [00:58<00:06,  1.12hosts/s]
/etc/ssh/userkeys/jenkins:command="/srv/jenkins.torproject.org/bin/update",no-port-forwarding,no-X11-forwarding,no-agent-forwarding,no-user-rc,from="78.47.38.228,2a01:4f8:211:6e8:0:823:4:1" ssh-rsa AAAAB3NzaC1yc2EAAAABIwAAAQEAw9/GOnsO3yhv8RRbIMTUEbFY3og+bzbz4vxaehMxddkVUkJDyDJZnwJRFr/1+q1kkasZoaYN19gfs8XaJnh9cSRNOAWquDqKjpu4z27aycQ3Pc3CtgJROvqTbgIdrq9UoWiFmXPJMCPepCe3T7y5kOmD5spSHTyVunT4fYNwGhILLkHPaDKXYRUIB4MrChuBW/tpzALTbRyWsXa9Ec3DBqSVrlKUXL4cZisa79mqJPmcfRphJFUFLwci1tifNlwOXtTTUqQTGLmt8GXJUEyU/6/QWDvkuq6B/paFnIdAAeH3mdOpXwbY7f+Y5uKl9+c7+i+9BxlJe6XB6S3H18Lbbw== git@cupani
                                                                                     |█████████████████████████████   |  91% (70/77) [00:58<00:06,  1.12hosts/s]
===== NODE GROUP =====                                                                                                                                               |█████████████████████████████   |  91% (70/77) [00:58<00:06,  1.12hosts/s]
(1) nevii.torproject.org                                                                                                                                             |█████████████████████████████   |  91% (70/77) [00:58<00:06,  1.12hosts/s]
----- OUTPUT of 'grep -e 78.47.38...c/ssh/userkeys/*' -----                                                                                                          |█████████████████████████████   |  91% (70/77) [00:58<00:06,  1.12hosts/s]
/etc/ssh/userkeys/dnsadm:command="/srv/dns.torproject.org/bin/from-git-rw",from="78.47.38.228,2a01:4f8:211:6e8:0:823:4:1",no-port-forwarding,no-X11-forwarding,no-agent-forwarding,no-pty,no-user-rc ssh-rsa AAAAB3NzaC1yc2EAAAABIwAAAQEAw9/GOnsO3yhv8RRbIMTUEbFY3og+bzbz4vxaehMxddkVUkJDyDJZnwJRFr/1+q1kkasZoaYN19gfs8XaJnh9cSRNOAWquDqKjpu4z27aycQ3Pc3CtgJROvqTbgIdrq9UoWiFmXPJMCPepCe3T7y5kOmD5spSHTyVunT4fYNwGhILLkHPaDKXYRUIB4MrChuBW/tpzALTbRyWsXa9Ec3DBqSVrlKUXL4cZisa79mqJPmcfRphJFUFLwci1tifNlwOXtTTUqQTGLmt8GXJUEyU/6/QWDvkuq6B/paFnIdAAeH3mdOpXwbY7f+Y5uKl9+c7+i+9BxlJe6XB6S3H18Lbbw== git@cupani
/etc/ssh/userkeys/letsencrypt:command="/srv/letsencrypt.torproject.org/bin/from-githost",from="78.47.38.228,2a01:4f8:211:6e8:0:823:4:1",no-port-forwarding,no-X11-forwarding,no-agent-forwarding,no-pty,no-user-rc ssh-rsa AAAAB3NzaC1yc2EAAAABIwAAAQEAw9/GOnsO3yhv8RRbIMTUEbFY3og+bzbz4vxaehMxddkVUkJDyDJZnwJRFr/1+q1kkasZoaYN19gfs8XaJnh9cSRNOAWquDqKjpu4z27aycQ3Pc3CtgJROvqTbgIdrq9UoWiFmXPJMCPepCe3T7y5kOmD5spSHTyVunT4fYNwGhILLkHPaDKXYRUIB4MrChuBW/tpzALTbRyWsXa9Ec3DBqSVrlKUXL4cZisa79mqJPmcfRphJFUFLwci1tifNlwOXtTTUqQTGLmt8GXJUEyU/6/QWDvkuq6B/paFnIdAAeH3mdOpXwbY7f+Y5uKl9+c7+i+9BxlJe6XB6S3H18Lbbw== git@cupani
                                                                                     |█████████████████████████████   |  91% (70/77) [00:58<00:06,  1.12hosts/s]
===== NODE GROUP =====                                                                                                                                               |█████████████████████████████   |  91% (70/77) [00:58<00:06,  1.12hosts/s]
(1) gitlab-01.torproject.org                                                                                                                                         |█████████████████████████████   |  91% (70/77) [00:58<00:06,  1.12hosts/s]
----- OUTPUT of 'grep -e 78.47.38...c/ssh/userkeys/*' -----                                                                                                          |█████████████████████████████   |  91% (70/77) [00:58<00:06,  1.12hosts/s]
grep: /etc/ssh/userkeys/dip-git: No such file or directory                                                                                                           |█████████████████████████████   |  91% (70/77) [00:58<00:06,  1.12hosts/s]
===== NODE GROUP =====                                                                                                                                               |█████████████████████████████   |  91% (70/77) [00:58<00:06,  1.12hosts/s]
(1) hetzner-hel1-01.torproject.org                                                                                                                                   |█████████████████████████████   |  91% (70/77) [00:58<00:06,  1.12hosts/s]
----- OUTPUT of 'grep -e 78.47.38...c/ssh/userkeys/*' -----                                                                                                          |█████████████████████████████   |  91% (70/77) [00:58<00:06,  1.12hosts/s]
/etc/ssh/userkeys/nagiosadm:command="/home/nagiosadm/bin/from-git-rw",from="78.47.38.228,2a01:4f8:211:6e8:0:823:4:1",no-port-forwarding,no-X11-forwarding,no-agent-forwarding,no-pty,no-user-rc ssh-rsa AAAAB3NzaC1yc2EAAAABIwAAAQEAw9/GOnsO3yhv8RRbIMTUEbFY3og+bzbz4vxaehMxddkVUkJDyDJZnwJRFr/1+q1kkasZoaYN19gfs8XaJnh9cSRNOAWquDqKjpu4z27aycQ3Pc3CtgJROvqTbgIdrq9UoWiFmXPJMCPepCe3T7y5kOmD5spSHTyVunT4fYNwGhILLkHPaDKXYRUIB4MrChuBW/tpzALTbRyWsXa9Ec3DBqSVrlKUXL4cZisa79mqJPmcfRphJFUFLwci1tifNlwOXtTTUqQTGLmt8GXJUEyU/6/QWDvkuq6B/paFnIdAAeH3mdOpXwbY7f+Y5uKl9+c7+i+9BxlJe6XB6S3H18Lbbw== git@cupani
                                                                                     |█████████████████████████████   |  91% (70/77) [00:58<00:06,  1.12hosts/s]
===== NODE GROUP =====                                                                                                                                               |█████████████████████████████   |  91% (70/77) [00:58<00:06,  1.12hosts/s]
(1) alberti.torproject.org                                                                                                                                           |█████████████████████████████   |  91% (70/77) [00:58<00:06,  1.12hosts/s]
----- OUTPUT of 'grep -e 78.47.38...c/ssh/userkeys/*' -----                                                                                                          |█████████████████████████████   |  91% (70/77) [00:58<00:06,  1.12hosts/s]
/etc/ssh/userkeys/sshdist:command="flock -s /var/cache/userdir-ldap/hosts//ud-generate.lock -c 'rsync --server --sender -pr . /var/cache/userdir-ldap/hosts/cupani.torproject.org'",no-port-forwarding,no-X11-forwarding,no-agent-forwarding,from="2a01:4f8:211:6e8:0:823:4:1,78.47.38.228" ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDqKk7DdcughgnjqwLCQBtd5vJueu0xPXONvYFfMAWJYvSLylV7CEAqkCmDN1PUXffH76PGG+X9LrTtQGtG9WrV6Y1lGyYMkR82fkYeXPL3nLdLE+IvSkxKUg3r4qgQ/CsaFKmz8DpfdOqipnKwamncZVemplUDxaC750hCJhacGFtGaM5TbEG+B6Ykx5PXlFPjXJQ8i0tNdwhIq5nfxrUizJzWioTA8LSJ8zb+VrC9/8HaaRnOEIugDC1DJth6pjODmAO+M2aQjbpzBu0CtegIUcW/T76Tt+X3GBFV4uYR+YNA7VKaoI/xxqWku85Tx9G/6E6FUOMhD8QxdIuc968T root@cupani
/etc/ssh/userkeys/sshdist:command="flock -s /var/cache/userdir-ldap/hosts//ud-generate.lock -c 'rsync --server --sender -pr . /var/cache/userdir-ldap/hosts/cupani.torproject.org'",no-port-forwarding,no-X11-forwarding,no-agent-forwarding,from="2a01:4f8:211:6e8:0:823:4:1,78.47.38.228" ssh-ed25519 AAAAC3NzaC1lZDI1NTE5AAAAIIVn+MFJptnxYAGSBSmD06c8Aj2h0zSdde+HK7wHN3Rq root@cupani
                                                                                     |█████████████████████████████   |  91% (70/77) [00:58<00:06,  1.12hosts/s]
================                                                                PASS |███                              |   9% (7/77) [00:58<05:39,  4.85s/hosts]
FAIL |█████████████████████████████   |  91% (70/77) [00:58<00:06,  1.12hosts/s]
90.9% (70/77) of nodes failed to execute command 'grep -e 78.47.38...c/ssh/userkeys/*': archive-01.torproject.org,bacula-director-01.torproject.org,build-arm-10.torproject.org,build-x86-[05-06,08-09].torproject.org,bungei.torproject.org,cache01.torproject.org,cache-02.torproject.org,carinatum.torproject.org,cdn-backend-sunet-01.torproject.org,check-01.torproject.org,chives.torproject.org,chiwui.torproject.org,colchicifolium.torproject.org,corsicum.torproject.org,crm-ext-01.torproject.org,crm-int-01.torproject.org,cupani.torproject.org,eugeni.torproject.org,fallax.torproject.org,forrestii.torproject.org,fsn-node-[01-04].torproject.org,gayi.torproject.org,gettor-01.torproject.org,gitlab-[01-02].torproject.org,henryi.torproject.org,hetzner-hel1-[02-03].torproject.org,hetzner-nbg1-[01-02].torproject.org,kvm[4-5].torproject.org,listera.torproject.org,loghost01.torproject.org,macrum.torproject.org,majus.torproject.org,mandos-01.torproject.org,materculae.torproject.org,meronense.torproject.org,moly.torproject.org,neriniflorum.torproject.org,nutans.torproject.org,omeiense.torproject.org,onionbalance-01.torproject.org,onionoo-backend-01.torproject.org,onionoo-frontend-01.torproject.org,oo-hetzner-03.torproject.org,orestis.torproject.org,palmeri.torproject.org,pauli.torproject.org,peninsulare.torproject.org,perdulce.torproject.org,polyanthum.torproject.org,rude.torproject.org,scw-arm-par-01.torproject.org,static-master-fsn.torproject.org,submit-01.torproject.org,subnotabile.torproject.org,tbb-nightlies-master.torproject.org,unifolium.torproject.org,web-cymru-01.torproject.org,web-fsn-[01-02].torproject.org,web-hetzner-01.torproject.org
9.1% (7/77) success ratio (>= 0.0% threshold) for command: 'grep -e 78.47.38...c/ssh/userkeys/*'.: alberti.torproject.org,hetzner-hel1-01.torproject.org,nevii.torproject.org,rouyi.torproject.org,staticiforme.torproject.org,troodi.torproject.org,vineale.torproject.org
9.1% (7/77) success ratio (>= 0.0% threshold) of nodes successfully executed all commands.: alberti.torproject.org,hetzner-hel1-01.torproject.org,nevii.torproject.org,rouyi.torproject.org,staticiforme.torproject.org,troodi.torproject.org,vineale.torproject.org

those keys should be deployed by Puppet instead. for now they have been renumbered by hand as part of #33446 but it would be important to change those if we ever want to rebuild that service on another host.

Child Tickets

Change History (1)

comment:1 Changed 7 months ago by anarcat

Summary: cupani's IP address is hardcoded all over the placecupani's IP address hardcoded in many places

i did manual changes to make the cupani migration work for /etc/ssh/userkeys/* on the following hosts:

  • staticiforme
  • vineale
  • troodi
  • rouyi
  • nevii
  • hetzner-hel1-01 (actually hardcoded in Puppet)
  • alberti

the old IPs were 78.47.38.228 and 2a01:4f8:211:6e8:0:823:4:1 the new IPs are currently 116.202.120.182 and 2a01:4f8:fff0:4f:266:37ff:fe32:cfb2.

Last edited 7 months ago by anarcat (previous) (diff)
Note: See TracTickets for help on using tickets.