Opened 5 days ago

Last modified 3 hours ago

#31718 assigned defect

Update DNS records for .ooni.torproject.org domains

Reported by: hellais Owned by: anarcat
Priority: Medium Milestone:
Component: Internal Services/Tor Sysadmin Team Version:
Severity: Normal Keywords:
Cc: Actual Points:
Parent ID: Points:
Reviewer: Sponsor:

Description

To make it easier for us to manage where these domains point to it would be great if the records for the domain explorer.ooni.torproject.org were to point to explorer.ooni.io and the record for ooni.torproject.org pointed to ooni.io.

The most high priority is the update of explorer.ooni.torproject.org as we are launching that today and we still have places where we link to explorer.ooni.torproject.org instead of explorer.ooni.io.

Child Tickets

Change History (10)

comment:1 Changed 5 days ago by weasel

I'm not convinced we want to point more of our torproject.org namespace to the outside. I'll bring this up with the rest of the team.

comment:2 Changed 4 days ago by arma

Another option, if we don't like having .torproject.org sites running on non-TPA machines, would be to run a tiny webserver as the .torproject.org site, which sends an http-level redirect to the external site?

I mention it because that http redirect is happening now already, just on the remote site.

comment:3 Changed 4 days ago by anarcat

hellais, do you want an actual CNAME (ie. that the user doesn't know they get redirected to ooni) or a redirect (that the user *does* end up on ooni.io)?

i do agree that it's unconventional to do those things for us. we usually point *.torproject.net at external resources.

is this domain used by non-HTTP clients?

comment:4 Changed 36 hours ago by hellais

Currently this is already happening though, explorer.ooni.torproject.org being a CNAME to explorer.ooni.io, I mistakenly thought this was not currently the case when opening the ticket, but this is already happening and no change is necessary on this front.

It was done this way specifically to make it easier for us to more independently handle how we serve requests to users hitting out website from the various domains that were distributed.

The website ooni.org & ooni.io & ooni.torproject.org is still running on tpo infrastructure, but we would like to change that to reduce the complexity of having something hosted on system where people need LDAP access to administer it.

Our preference would be that we setup a CNAME record for ooni.torproject.org that points to ooni.io or ooni.org so that we are able to on our own setup a redirect, if desirable, or handle the requests directly by keeping the ooni.torproject.org domain (we probably will do this in the beginning).

would be to run a tiny webserver as the .torproject.org site, which sends an http-level redirect to the external site?

hellais, do you want an actual CNAME (ie. that the user doesn't know they get redirected to ooni) or a redirect (that the user *does* end up on ooni.io)?

It would be preferable if we could get a CNAME record, so that we can manage how redirects are handled autonomously.

is this domain used by non-HTTP clients?

It's the domain used for our primary website. We don't make any assumption as to what type of client is going to access it. I suppose most modern browsers will do HTTPS.

comment:5 Changed 36 hours ago by hellais

Another record which is currently setup in a similar fashion is the CNAME for

get.ooni.torproject.org. 3599	IN	CNAME	get.ooni.io.

To be clear it's not a big problem if the policy WRT to setting up CNAME records has changed, I just need to be aware of it and plan according to it.

This is probably also a good opportunity to do some cleanup of other *.ooni.torproject.org domains as we are trying to simplify our infrastructure and reduce our devops cognitive load by simplifying our infrastructure.

comment:6 Changed 32 hours ago by anarcat

i don't exactly know what the policy is regarding CNAMEs, to be honest. :) the best source I know of is this:

https://help.torproject.org/tsa/doc/naming-scheme/

... which outlines the distinction between TPO (torproject.org) and TPN (torproject.net) that weasel was refering to. The problem might not be CNAMEs per se, but pointing to outside stuff.

Another thing is that CNAMEs are not a great way to move stuff around, because they are transparent to clients. An web browser or crawler will not treat a CNAME as "this is now hosted over there", it's just an alias. For those kind of transitions, you want to do a HTTP redirect, that is respond with a 301 (Moved Permanently) or 302 (Found) status code:

https://en.wikipedia.org/wiki/List_of_HTTP_status_codes#3xx_Redirection

Then we can deprecate the *.ooni.tpo namespace and eventually transition to ooni.io cleanly.

This is why I was asking about non-HTTP (and non-HTTPS) clients: those redirections will work only for HTTP clients. If you have people using this over SSH or Git or whatever non-HTTP protocol, those would break of course.

(Sorry if you already know all of this about HTTP status codes vs CNAMEs, but I thought it was useful to get back to the specs to clarify my thoughts.)

comment:7 Changed 29 hours ago by anarcat

Owner: changed from tpa to anarcat
Status: newassigned

we agreed that we'd add a CNAME record and keep the CNAMEs until may 7th 2020, at which point they'd turn into HTTP redirects. we'll do this tomorrow at 1200 EDT (1600 UTC).

comment:8 Changed 8 hours ago by anarcat

seems to me that just adding the CNAME will not be enough, as there are many other things to cleanup. the main procedure should be:

  1. remove ooni.torproject.org from tor-puppet/modules/roles/misc/static-components.yaml
  2. remove ooni from auto-dns, push
  3. add the CNAME, push

Other things to cleanup include:

letsencrypt-domains/domains:46:ooni.torproject.org
tor-nagios/config/nagios-master.cfg:1330:    name: mirror static sync - ooni
tor-nagios/config/nagios-master.cfg:1331:    check: "dsa_check_staticsync!ooni.torproject.org"
tor-puppet/modules/sudo/files/sudoers:63:%ooni			STATICMASTER=(ooni)			ALL
tor-puppet/modules/sudo/files/sudoers:95:%ooni			STATICMASTER=(mirroradm)	NOPASSWD: /usr/local/bin/static-master-update-component ooni.torproject.org, /usr/local/bin/static-update-component ooni.torproject.org
tor-puppet/modules/roles/manifests/static_mirror_web.pp:74:  ssl::service { 'ooni.torproject.org': ensure => 'ifstatic', notify  => Exec['service apache2 reload'], key => true, }
tor-puppet/modules/roles/manifests/static_mirror_onion.pp:37:      'ooni.torproject.org',
tor-puppet/onions/onionbalance-services.yaml:17: [...]

I'm particularly concerned about let's encrypt - wouldn't adding the cname break the X509 cert, as we would now point to another server?

Last edited 7 hours ago by anarcat (previous) (diff)

comment:9 Changed 6 hours ago by hellais

So we looked into this with @anarcat and encountered the following issues:

  • The current setup has both HSTS and certificate pinning enabled for the ooni.torproject.org website
  • It is not straightforward to do custom HTTPS changes on the current ooni hosting service (netlify)

Since the maxage for the certificate pinning is set to 60 days we will need to wait for that amount of time before we are able to migrate over.

In the meantime @anarcat is going to see how to disable the certificate pinning headers from the ooni.torproject.org host config, so that we can begin waiting the 60 days after which we can proceed with the CNAME plan as mentioned above.

comment:10 Changed 3 hours ago by anarcat

i have disabled certificate pinning on ooni.torproject.org around 15 minutes ago. it should therefore expire in 60 days exactly, which is about on saturday november 16th at 19:30UTC. assuming we don't want to do this transition on a saturday, we should probably look into this again on november 18th.

i documented a bit how HPKP works in:

https://help.torproject.org/tsa/howto/letsencrypt/#index3h1

Note: See TracTickets for help on using tickets.