Opened 7 weeks ago

Last modified 5 weeks ago

#32267 assigned project

move to a private nextcloud instance

Reported by: anarcat Owned by: ln5
Priority: Medium Milestone:
Component: Internal Services/Service - nextcloud Version:
Severity: Normal Keywords:
Cc: gaba, micah Actual Points:
Parent ID: Points:
Reviewer: Sponsor:

Description (last modified by anarcat)

in #31540, we agreed to delegate running a dedicated Nextcloud instance to riseup.

this ticket tracks the next steps, so far:

  1. create the private instance (riseup)
  2. point DNS at the private instance (TPA)
  3. make sure HTTPS works (riseup?)
  4. #32332 set up LDAP integration, somehow (service admins + riseup)
  5. move content and users over (gaba + pili + teams)

Child Tickets

TicketStatusOwnerSummaryComponent
#32367closednextcloud-admin@torproject.orgONLYOFFICE not working?Internal Services/Service - nextcloud
#32390newhirodecomission storm / bracteata on February 4, 2020Internal Services/Service - sandstorm
#32391newnextcloud-admin@torproject.orgPurge test accounts and data from riseup in February 4, 2020Internal Services/Service - nextcloud
#32392newnextcloud-admin@torproject.orgEmails are sent from nc@riseup.netInternal Services/Service - nextcloud

Change History (33)

comment:1 Changed 7 weeks ago by anarcat

Owner: set to anarcat
Status: newassigned

riseup asked me to push a few things around here to unblock the deployment. it seems the next step is to generate a HTTPS cert and, if we want it in our namespace, requires us to pick a domain name and point it at their server.

we need to answer the following questions:

  1. CNAME or A/AAAA?
  2. pointing at what?
  3. what name?

I would suggest:

  1. CNAME: so that riseup doesn't have to coordinate with us to move the box around and we clearly see who the hoster is
  2. the box is merlebleu.riseup.net
  3. i would suggest keeping with the nc convention, so nc.torproject.net

I'll do this after quick validation in today's vegas.

comment:2 Changed 7 weeks ago by ln5

Cc: linus@… added

comment:3 Changed 7 weeks ago by hiro

also cloud.tp.n or docs.tp.n could work... just throwing it there...

comment:4 Changed 7 weeks ago by anarcat

vegas approved nc.tpn and suggested we make a redirect from nc.tpo to avoid confusion, will do next.

comment:5 Changed 7 weeks ago by anarcat

nc.torproject.net is now "live" in terms of DNS, next step is to setup the cert and so on from riseup's side.

i've also deployed a nc.torproject.org HTTP redirection to nc.tpn. to quote the DNS commitlog:

commit faf9e63fe52c2efb0771e662f76247b32f946ede
Author: Antoine Beaupré <anarcat@debian.org>
Date:   Thu Oct 24 13:58:11 2019 -0400

    add nc.torproject.org for nextcloud (#32267)
    
    This points to the static mirrors for redirection, using the "vanity
    hosts" macro.
    
    The rationale is that our users might not be familiar with the .net
    and .org distinction. Even if they do, they might not know we don't
    manage this machine directly. Instead, they will probably look for
    nc.tpo and fail and complain to us. So we should help people a little
    here and make an exception for this domain which would be possibly
    heavily used.
    
    This is especially relevant since we *might* eventually use `.org` for
    this purpose and manage the Nextcloud service ourselves.
    
    But we don't do just a CNAME to the end server here: we do a redirect
    so the canonical URL remains the `.net` one, to clearly show this is
    not managed by TPA or the service admin team.

diff --git a/torproject.org b/torproject.org
index 92d68b3..b49dc6f 100644
--- a/torproject.org
+++ b/torproject.org
@@ -149,6 +149,8 @@ bugs				IN	CNAME	static
 censorshipwiki			IN	CNAME	static
 safetyboard			IN	CNAME	static
 wiki				IN	CNAME	static
+; https://trac.torproject.org/projects/tor/ticket/32267
+nc				IN	CNAME	static
 
 ; static-mirrors
 aus1				IN	CNAME	static

comment:6 Changed 7 weeks ago by anarcat

Owner: changed from anarcat to ln5

linus, i think you're it again on this ticket, feel free to ping if you need help.

comment:7 Changed 7 weeks ago by gaba

Cc: gaba added

comment:8 Changed 7 weeks ago by micah

As an update to this issue - the cert has been setup from the riseup side.

What remains now is to plug-in a couple pieces of the setup, and create the admin access for the people who will be managing this. I assume that will be the same people who are doing that already. Then it will be a matter of migration.

comment:9 Changed 7 weeks ago by ln5

Same people, yes.
Would you be able to migrate the admins, including the requirement for MFA and accompanying MFA info?
If not, let's try to add only me in the least insecure way.

comment:10 Changed 7 weeks ago by ln5

Also, micah, do you have experience with migrating NC data from one instance to another?

From the top of my head, we would have to bring

  • files
  • calendars
  • contacts
  • decks
  • circles

over to the new instance.

comment:11 Changed 6 weeks ago by micah

This instance can be reached by gd5tq2xucgv35k7gokt6ts2tslaosjnwferdtqf2r3cbbqv5qvbxnmyd.onion

Besides the base installation, this instance was configured with the U2F and TOTP apps enabled, and forced, Onlyoffice was enabled and configured. Outgoing/incoming email has also been configured.

This new version of nextcloud has these new features that the Nextcloud admins should be aware of:

You can setup two-factor authentication after first login, admins can create one-time login tokens in the web UI and delegate this to group admins/

Remote Wipe: For Nextcloud admins, this is good functionality to know about in order to better handle people departing the organization. https://www.youtube.com/watch?v=oyWXMjb-6ik

Collaborative text: The new nextcloud collaborative text editor comes as default now as well: https://www.youtube.com/watch?v=Nr7cGN6ZJM0

Last edited 6 weeks ago by micah (previous) (diff)

comment:12 Changed 6 weeks ago by micah

ln5: can you tell me which logins for admins I should migrate? I'm unsure which should be done. I'm also unsure how reliable migrating the login/MFA information will be, but we can try it for you. See if you can login. You will need to generate new backup codes

comment:13 Changed 6 weeks ago by micah

Regarding migrating data, I have no experience with migrating data from one instance to another, it does not look trivial and my research has not produced any mechanism to make it easy.

Since you mention circles, I wanted to check in about that. This app is kind of a 'hack' that lets unprivileged users create their own groupings. It was useful on the riseup server, because no tor people were admins and could not create groups. I'd recommend considering *not* using it on the new server, if possible, as its long term viability is questionable.

comment:14 Changed 6 weeks ago by ln5

https://gd5tq2xucgv35k7gokt6ts2tslaosjnwferdtqf2r3cbbqv5qvbxnmyd.onion/login

http://gd5tq2xucgv35k7gokt6ts2tslaosjnwferdtqf2r3cbbqv5qvbxnmyd.onion/ redirects to https://gd5tq2xucgv35k7gokt6ts2tslaosjnwferdtqf2r3cbbqv5qvbxnmyd.onion/login which fails, supposedly bc 443.

Last edited 6 weeks ago by ln5 (previous) (diff)

comment:15 Changed 6 weeks ago by ln5

See if you can login. You will need to generate new backup codes

This, after entering TOTP authn code:

The server was unable to complete your request.

If this happens again, please send the technical details below to the server administrator.

More details can be found in the server log.
Technical details

    Remote Address: <redacted>
    Request ID: PKeWiwlMWjvZrlYItQvF

comment:16 in reply to:  14 ; Changed 6 weeks ago by micah

comment:17 in reply to:  15 Changed 6 weeks ago by micah

Replying to ln5:

See if you can login. You will need to generate new backup codes

This, after entering TOTP authn code:

The server was unable to complete your request.

If this happens again, please send the technical details below to the server administrator.

More details can be found in the server log.
Technical details

    Remote Address: <redacted>
    Request ID: PKeWiwlMWjvZrlYItQvF

I believe that this is because the codes are connected to the domain of the application, so we won't be able to just move those over from the database.

I've removed the TOTP setting for your user, see if you can login without it and then enable it.

comment:18 Changed 6 weeks ago by ln5

I've removed the TOTP setting for your user, see if you can login without it and then enable it.

Worked fine, thanks.

comment:19 Changed 6 weeks ago by ln5

Micah, did you verify outgoing email?
I was hoping to get a signup email when creating a new user but haven't seen one yet. Might be greylisting but I thought I'd ask. I did see some email settings saying From @riseup.net which indicates it might not be configured yet.

comment:20 Changed 6 weeks ago by gaba

Hey! How do I login into the new nextcloud? I'm getting an 'unable to connect' when I try the onion addresses in this ticket.

comment:21 in reply to:  19 ; Changed 6 weeks ago by micah

Replying to ln5:

Micah, did you verify outgoing email?

I did!

But then I put a restrictive firewall in place that denies all outgoing connections that aren't approved, and failed to remember that this connection needed to happen. So I've fixed that now

comment:22 in reply to:  21 ; Changed 6 weeks ago by ln5

Replying to micah:

Replying to ln5:

Micah, did you verify outgoing email?

I did!

But then I put a restrictive firewall in place that denies all outgoing connections that aren't approved, and failed to remember that this connection needed to happen. So I've fixed that now

Works, thanks!

From: is still nc@… though. Is that intentional?

comment:23 in reply to:  20 ; Changed 6 weeks ago by ln5

Replying to gaba:

Hey! How do I login into the new nextcloud? I'm getting an 'unable to connect' when I try the onion addresses in this ticket.

The http:// link is the one you want.

That said, if you did click the https:// link you might need to restart your Tor Browser to not automatically end up on the https:// link even when entering the http:// link. There might be less destructive ways than restarting.

UPDATE (thanks gk): The http:// link is still redirecting to https://

$ curl -x socks4a://127.0.0.1:9050/ -v http://gd5tq2xucgv35k7gokt6ts2tslaosjnwferdtqf2r3cbbqv5qvbxnmyd.onion/
...
< HTTP/1.1 302 Found
...
< Location: https://gd5tq2xucgv35k7gokt6ts2tslaosjnwferdtqf2r3cbbqv5qvbxnmyd.onion/login

Last edited 6 weeks ago by ln5 (previous) (diff)

comment:24 in reply to:  16 Changed 6 weeks ago by ln5

Replying to micah:

Replying to ln5:

http://gd5tq2xucgv35k7gokt6ts2tslaosjnwferdtqf2r3cbbqv5qvbxnmyd.onion/ redirects to https://gd5tq2xucgv35k7gokt6ts2tslaosjnwferdtqf2r3cbbqv5qvbxnmyd.onion/login which fails, supposedly bc 443.

I've resolved this

Is it crazy to redirect 443 to 80 for the onion?

Last edited 6 weeks ago by ln5 (previous) (diff)

comment:25 Changed 6 weeks ago by ln5

Cc: linus@… removed
Component: Internal Services/Services Admin TeamInternal Services/Service - nextcloud

comment:26 Changed 6 weeks ago by ln5

NOTE: Before we start migrating any content, we should get LDAP integration running. This will give us all users and groups (!) from LDAP.

Micah, please comment on #32332 to let us know if any of the requirements for the "LDAP User and group backend" application are problematic.

comment:27 in reply to:  description Changed 6 weeks ago by ln5

Description: modified (diff)

comment:28 Changed 6 weeks ago by hakar11

commit faf9e63fe52c2efb0771e662f76247b32f946ede
Author: Antoine Beaupré <anarcat@…>
Date: Thu Oct 24 13:58:11 2019 -0400

add nc.torproject.org for nextcloud (#32267)


This points to the static mirrors for redirection, using the "vanity
hosts" macro.


The rationale is that our users might not be familiar with the .net
and .org distinction. Even if they do, they might not know we don't
manage this machine directly. Instead, they will probably look for
nc.tpo and fail and complain to us. So we should help people a little
here and make an exception for this domain which would be possibly
heavily used.


This is especially relevant since we *might* eventually use .org for
this purpose and manage the Nextcloud service ourselves.


But we don't do just a CNAME to the end server here: we do a redirect
so the canonical URL remains the .net one, to clearly show this is
not managed by TPA or the service admin team.

diff --git a/torproject.org b/torproject.org
index 92d68b3..b49dc6f 100644
--- a/torproject.org
+++ b/torproject.org
@@ -149,6 +149,8 @@ bugs IN CNAME static

censorshipwiki IN CNAME static
safetyboard IN CNAME static
wiki IN CNAME static

+; https://trac.torproject.org/projects/tor/ticket/32267
+nc IN CNAME static

; static-mirrors
aus1 IN CNAME static

comment:29 Changed 6 weeks ago by micah

Cc: micah added

comment:30 in reply to:  22 Changed 6 weeks ago by micah

Apologies for the delay in response, I was not in CC on this ticket.

Replying to ln5:

From: is still nc@… though. Is that intentional?

It is, but it can be setup to be something else, if you want?

comment:31 in reply to:  23 Changed 6 weeks ago by micah

Apologies for the slow response, I wasn't on CC on this issue.

UPDATE (thanks gk): The http:// link is still redirecting to https://

$ curl -x socks4a://127.0.0.1:9050/ -v http://gd5tq2xucgv35k7gokt6ts2tslaosjnwferdtqf2r3cbbqv5qvbxnmyd.onion/
...
< HTTP/1.1 302 Found
...
< Location: https://gd5tq2xucgv35k7gokt6ts2tslaosjnwferdtqf2r3cbbqv5qvbxnmyd.onion/login

This is working now, again. I had fixed that in the webserver, but it seems like there was a configuration option in nextcloud that also was causing this.

comment:32 Changed 5 weeks ago by gaba

Description: modified (diff)

comment:33 Changed 5 weeks ago by anarcat

Description: modified (diff)

we agreed on postponing the LDAP requirement for now, as we're unsure how to proceed and don't want to block deployment. gaba will manage accounts by hand for now.

Note: See TracTickets for help on using tickets.