Opened 11 months ago

Closed 8 months ago

Last modified 8 months ago

#32532 closed defect (fixed)

Install ZNC on Chives, make pastly admin it

Reported by: pastly Owned by: pastly
Priority: Medium Milestone:
Component: Internal Services/Tor Sysadmin Team Version:
Severity: Normal Keywords:
Cc: gaba, anarcat Actual Points:
Parent ID: Points:
Reviewer: Sponsor:

Description (last modified by pastly)

I think I want to migrate the TPO people who use my bouncer off my server and onto TPO infra. If possible.

Initial discussion with anarcat suggested that chives.tpo would be the box. Okay cool.

Q1: Can it get a valid TLS certificate? Both for the web interface (edit for account management, NOT CHAT) and also for protecting the IRC traffic.

Q2: Can Tor get installed on the box? Right now I also have an onion service pointing to my ZNC and it'd be cool to keep that.

If desired, I can talk more about how I have accomplished Q1 with Let's Encrypt, nginx, and a cron job. Q2 is just because it's easy and cool. No big deal.

Child Tickets

Change History (21)

comment:1 Changed 11 months ago by anarcat

Owner: changed from tpa to anarcat
Status: newaccepted

Q1: Can it get a valid TLS certificate? Both for the web interface and also for protecting the IRC traffic.

Yes. I wonder which domain it should be however?

Q2: Can Tor get installed on the box? Right now I also have an onion service pointing to my ZNC and it'd be cool to keep that.

Sure, that shouldn't be a problem either.

comment:2 in reply to:  1 Changed 11 months ago by pastly

Replying to anarcat:

Q1: Can it get a valid TLS certificate? Both for the web interface and also for protecting the IRC traffic.

Yes. I wonder which domain it should be however?

chives.torproject.org

My users currently use ircbouncer.system33.pw, and I could made DNS for that point to chives.torproject.org, but I'd rather rip the bandaid off and make them updates their host.

If you were thinking the certificate would have to be valid for irc.oftc.net, no. ZNC terminates the TLS and pretends to be a regular ol' IRC server to the clients.

comment:3 Changed 11 months ago by anarcat

Q1: Can it get a valid TLS certificate? Both for the web interface and also for protecting the IRC traffic.

Yes. I wonder which domain it should be however?

chives.torproject.org

I was thinking something more like ircbouncer.torproject.org. :)

comment:4 Changed 11 months ago by pastly

Description: modified (diff)

(Edit description to make explicit that ZNC's web interface is for admin stuff, not chat.)

Sure ircbouncer.torproject.org. Doesn't make much difference to me :)

comment:5 Changed 11 months ago by anarcat

Owner: changed from anarcat to pastly
Status: acceptedassigned

i have created the ircbouncer role (user) and group on chives. the user has the rights to keep persistent user-level services running through systemd, also known as "lingering". the documentation on how to use that to run services is detailed here:

https://help.torproject.org/tsa/doc/services/

it is your responsibility to start the service and keep it running, our systemd things will just run whatever the service file says. :)

so sudo -u ircbouncer to get to the privileged account. i've made you part of the group which should give you that privilege, let me know if that doesn't work.

i've also added the ircbouncer user to the ssl-cert group so it can access the X509 certificates. those certs are the following files:

root@chives:~# ls -al /etc/ssl/private/ircbouncer.torproject.org.* /etc/ssl/torproject/certs/ircbouncer.torproject.org.crt*
-r--r----- 1 root ssl-cert 7178 nov 18 20:42 /etc/ssl/private/ircbouncer.torproject.org.combined
-r--r----- 1 root ssl-cert 3244 nov 18 20:42 /etc/ssl/private/ircbouncer.torproject.org.key
-r--r--r-- 1 root root     2286 nov 18 20:42 /etc/ssl/torproject/certs/ircbouncer.torproject.org.crt
-r--r--r-- 1 root root     1649 nov 18 20:42 /etc/ssl/torproject/certs/ircbouncer.torproject.org.crt-chain
-r--r--r-- 1 root root     3934 nov 18 20:42 /etc/ssl/torproject/certs/ircbouncer.torproject.org.crt-chained

Those are basically:

  • .key: the private key
  • .crt: the public key
  • .crt-chain: the "chain" bits that might be required in some browsers
  • .crt-chained: the above two together
  • .combined: all of the above

Usually, the .key and .crt are enough, but sometimes you need the .crt-chained instead of the .crt.

The onion service is also up and running, under (i believe) eibwzyiqgk6vgugg.onion. It currently points at ircbouncer.torproject.org:80 which of course is not listening. That's the next step: we need to figure our how to give you access to port 80 here. My suggestion would be that you start by setting up the bouncer and its web interface on whatever (stable) port you can, and access it over an SSH tunnel for now. Once you're happy with this (or if you can't use SSH tunnels for some reason), let me know what the port number is, and I'll setup an Nginx forward, reusing those nice little X509 certs as well.

TL;DR: checklist status:

  • [x] znc install (anarcat)
  • [x] ircbouncer role account and group (anarcat)
  • [x] sudo access (anarcat)
  • [x] enable-linger (anarcat)
  • [x] x509 certs (anarcat)
  • [x] hidden service (anarcat)
  • [ ] systemd.service configuration (pastly)
  • [ ] znc configuration (pastly)
  • [ ] web interface configuration (pastly)
  • [ ] nginx proxy (anarcat)

let me know if you have any questions!

comment:6 Changed 11 months ago by pastly

  • [x] systemd.service config (pastly)

Extremely basic service file to run znc. Plus unfortunately fell back to copying the certs out of /etc and putting them in ~/.znc/znc.pem once a week with a cron job due to what seems like a systemd security thing preventing me from reading /etc/ssl/private/ files.

  • [x] znc configuration (pastly)
  • [x] web interface configuration (pastly)

Uhh ... done I think. I have IRC and HTTP on 2000 as well as IRC-over-TLS and HTTPS on 2001. I have an account for myself and can make/migrate additional accounts later without help.

Speaking of the nginx proxy and these ports ...

We can probably skip nginx. Our users can be expected to use Tor Browser in the rare instance they want to access the web interface. Thus HiddenServicePort 80 2000 gets them secure access to the web interface.

For their IRC client, opening 2001 in the firewall gets them IRC over TLS. I guess for completeness we should open 2000 for plaintext IRC. Finally, for the cool kids HiddenServicePort 2000 gets them IRC over Tor.

PS: why not v3 onion service? :p

If what I'm saying sounds reasonable, then in lieu of the "nginx proxy" step, I would request the following lines in the torrc:

HiddenServiceVersion 3
HiddenServicePort 80 2000
HiddenServicePort 2000

And the firewall to allow inbound 2000 and 2001.

And to be notified about what the new onion service is if you actually bump to v3.

Thanks!

comment:7 Changed 11 months ago by anarcat

i started getting a bunch of errors from cron like this:

Subject: Cron <ircbouncer@chives> /home/ircbouncer/bin/znc-ssl-copy.sh
To: ircbouncer@chives.torproject.org
Date: Tue, 19 Nov 2019 02:49:01 +0000

/home/ircbouncer/bin/znc-ssl-copy.sh: line 3: /home/ircbouncer/.znc/znc.pem: Permission denied

not sure what you're doing here, but something is not working. :)

i have changed the role's forward to point to you instead of TPA so we stop receiving those errors, but it's probably something you'd want to fix.

it would also be great if you could document your setup here or somewhere... i have specifically started documenting the irc service here:

https://help.torproject.org/tsa/howto/irc/

the wiki can be edited through this git repository:

https://gitweb.torproject.org/project/help/wiki.git

ask the git admins if you need access...

Last edited 11 months ago by anarcat (previous) (diff)

comment:8 Changed 11 months ago by pastly

Owner: changed from pastly to anarcat

As stated on IRC 24+ hours ago, I fixed the bug generating a ton of email spam. Sorry about that, again.

I've documented everything on the irc branch of my fork of that wiki. https://github.com/pastly/tsa-wiki/commit/dac66a37bef2232ddc234d56918895de23b952c6. Everything expect how to add users. View it rendered here. I'm waiting on adding users and documenting how to do so until the necessary/desired network changes are made.

To distill comment 6 into concrete requests:

  • [ ] allow 2001 inbound to ZNC, TLS-protected web and IRC
  • [ ] configure Tor as follows, or as close to it as willing
Log notice syslog
# to use 3 hops instead of 6. not anonymous
# can't do this if you want a SocksPort
SocksPort 0
HiddenServiceSingleHopMode 1
HiddenServiceNonAnonymousMode 1
# actual interesting config
HiddenServiceDir /var/lib/tor/onion/ircbouncer.torproject.org
HiddenServiceVersion 3
HiddenServicePort 80 2000
HiddenServicePort 2000
  • [ ] share with pastly the onion address if different than eibwzyiqgk6vgugg.onion

I'm assigning the ticket back to you because I think that's how you're keeping track of what's on your plate vs what is on mine. If this was inappropriate, please excuse my ignorance.

comment:9 Changed 11 months ago by anarcat

Status: assignedaccepted

I'm assigning the ticket back to you because I think that's how you're keeping track of what's on your plate vs what is on mine. If this was inappropriate, please excuse my ignorance.

Not at all! That's exactly what I was expecting. :) will followup soon.

Do you want me to merge your github branch it or are you going to push it yourself to git-rw?

thanks for your work!

comment:10 in reply to:  9 Changed 11 months ago by pastly

Replying to anarcat:

Do you want me to merge your github branch it or are you going to push it yourself to git-rw?

I don't think I have access to that repo (haven't tried). Instead of me gaining access, I think it would be easiest and best for you to just grab my commits and push them yourself.

Should you do so now? Nah. Let me add how to add ZNC users to it first.

That would also give you a chance to clean up the document as you see fit. It's serving a lot of purposes right now. Maybe after this is set up then the only thing the document needs is the stuff I wrote. Not my call to make :p

comment:11 Changed 11 months ago by anarcat

Owner: changed from anarcat to pastly
Status: acceptedassigned

i opened port 2001 for ZNC and changed the forward for the hidden service, which stays at eibwzyiqgk6vgugg.onion.

unfortunately, this is all I can give you for now:

HiddenServiceDir /var/lib/tor/onion/ircbouncer.torproject.org
HiddenServiceVersion 2
HiddenServicePort 80 localhost:2000

the Puppet module we use for Tor onion services is limited to v2 addresses right now, and doesn't support single hop and all that jazz. hopefully that can be fixed soon, but in the meantime I hope it will suffice for our purposes here...?

i think if that all works the remaining step is to fix the docs and merge it, reassign to me when that's ready! :)

comment:12 Changed 10 months ago by anarcat

@pastly - is there anything else you need from us here? how is it going?

comment:13 Changed 10 months ago by pastly

I just need to go through the motions of adding a user so I can document it.

How do you feel about the existing contents of tsa/howto/irc.mdwn? Would it be appropriate for me to delete them entirely? Anything I should keep?

comment:14 Changed 10 months ago by gaba

Cc: gaba added

comment:15 Changed 9 months ago by anarcat

I just need to go through the motions of adding a user so I can document it.

Any update on that?

How do you feel about the existing contents of tsa/howto/irc.mdwn? Would it be appropriate for me to delete them entirely? Anything I should keep?

er, you mean https://help.torproject.org/tsa/howto/irc/ here? I definitely want to keep *that*. :) or do you mean your patch? if the latter, then i'm happy to just merge it in, but I would then need to integrate it with the current document structure, as it looks like it's just slapped at the end right now. :)

i would suggest splitting it between the Tutorial (for things that end-users can do easily without prior knowledge) Howto (for things that require a bit more knowledge, maybe only the sysadmin/commandline-level stuff) and Reference (for the "how to configure/install this thing" bits).

makes sense?

comment:16 Changed 9 months ago by anarcat

Cc: anarcat added

comment:17 Changed 9 months ago by anarcat

apparently, i've been told you should be able to push to the wiki yourself. the readonly repository, you probably already know here:

https://gitweb.torproject.org/project/help/wiki.git/

but you should also be able to pull and push from the read-write URL as long as you have an SSH key on the git server:

git@…:project/help/wiki.git

please try that and let me know if works (or doesn't).

also do let me know what the next steps should be here, i'm a bit lost as to what we should do next. :)

comment:18 Changed 8 months ago by anarcat

pastly: chives was rebooted last night and it seems the bouncer didn't come back on. could you take a look?

comment:19 Changed 8 months ago by arma

And: once this is a real service, can somebody add a line to
https://trac.torproject.org/projects/tor/wiki/org/operations/services
along with who is the service maintainer, so it is possible for people to try to report issues with the service without secretly already knowing who runs it? :)

(maybe in the 'internal stuff' section)

comment:20 Changed 8 months ago by pastly

Resolution: fixed
Status: assignedclosed

I finally went through the motions of adding a user so I could document it. I did. I pushed it to project/help/wiki.git on the master branch and the page has automatically updated.

I added the bouncer to the services page like arma suggested.

I am going to open a new ticket about the bouncer going down.

comment:21 Changed 8 months ago by anarcat

thanks for the docs! i linked the service page to the irc docs. i've also split the install and user docs between the reference and howto sections, to respect the existing document structure, but otherwise didn't touch it.

i'll followup with the downtime in that other ticket, thank you for that too! :)

Note: See TracTickets for help on using tickets.