Opened 4 months ago

Last modified 5 days ago

#30608 new enhancement

Have a SMTP out only server

Reported by: dgoulet Owned by: tpa
Priority: Medium Milestone:
Component: Internal Services/Tor Sysadmin Team Version:
Severity: Normal Keywords:
Cc: anarcat Actual Points:
Parent ID: Points:
Reviewer: Sponsor:

Description

I do use my @tpo email address for many communications outside torproject lists or @tpo people.

Lately, I discovered that many of my emails were silent drop by the remote server or put in SPAM. And that was because the person came back to me asking where was my email. For instance, gmail sometimes put it in the SPAM still because we lack DKIM/SPF so it hurts our reputation.

Th reason why is quite simple: I use my own SMTP server to send the emails while forging the From address.

It would honestly be of a great help if we could simply have an authenticated SMTP server that I could use with let say my LDAP account for sending emails with my @tpo and not being worried that it gets dropped...

Child Tickets

TicketTypeStatusOwnerSummary
#31723tasknewggusdesign help documents for the help service and provide support for new users

Change History (12)

comment:1 Changed 4 months ago by gaba

+1

comment:2 Changed 4 months ago by mikeperry

I have the same problem, fwiw. I think anyone who doesn't use gmail with their @torproject.org addr has problems like this.

comment:3 Changed 4 months ago by arma

To clarify the request: are you hoping that we will turn on dkim/spf/etc too? And if so, am I right to imagine that would make it so everybody needs to use this outgoing smtp server when sending as torproject.org?

And if we're not turning on dkim/spf/etc, how much would having this separate server somewhere actually help?

comment:4 Changed 4 months ago by anarcat

@mikeperry - do you use gmail?

I don't, and i don't have this problem, as far as I know. I've had problems with mail delivery before in #29770 but I resolved it by fixing my "enveloppe from". as it turns out, i had a local configuration problem. I suspect a lot of those issues are bound to SPF or DKIM configurations (not on torproject.org!) that are fundamentally incompatible with email forwarding.

It would greatly help if people would report delivery issues they are having with concrete examples, here. as far as I know, the only such documented report is the one i did above, in #29770. I won't deny there are problems, but I think it would be better to have clear documentation of what the issues are if we're going to fix this.

Tthe other thing is that using the all-powerful LDAP password for email means yet another password reuse: then that password, instead of being just in your head or in your password manager, needs to be stored, in cleartext, in some configuration somewhere. this is very bad. So we'd at least need to finish the separation between the "sudo" and "normal" LDAP passwords (#6367) and possibly setup another "service" password specifically for email. This will, naturally, make LDAP management more complex but it's not an impossible task.

Honestly, it is my belief that companies like Google and Facebook are actively trying to destroy the "old" internet standards like email. They have done so with XMPP and SMTP/IMAP are the ones that are left. "Delivery to gmail.com", as a standard, is not simply a matter of delivering emails from torproject.org servers. It's a black box we do not control and that do not provide tech support or answer postmaster@ emails, so it's actually a very hard problem to solve.

comment:5 Changed 4 months ago by anarcat

Cc: anarcat added

comment:6 Changed 4 months ago by dgoulet

Anyone able to impersonate a @tpo is a _serious_ problem imo and we have *no* defenses in place against that. They won't be perfect but it would at least be a start...

The lists + forwarding problem makes it dicy for us to enable SPF/DKIM so I won't tackle that idea thus I'm not asking about that (to answer arma). But for now I would _just_ simply want my email to STOP BEING DROPPED by email servers that check the origin server of the From: address (which honestly I can't blame them to do).

And whatever Gmail/Facebook do, at this point, I honestly do not care, I just want, by all means necessary, to be able to use my @tpo without ending up in SPAM or being silently dropped... This is not happening to only me and it will get worsts as big providers tighten their email rules over time.

Please people, let us use our infrastructure so we can do our work? I mean half-being able to use properly my @tpo address is a big deal for me in my work... For instance, I get emails from people discussing specific commits in the code and I end up in their SPAM when I reply... This is uberly ridiculous considering a known-popular domain that torproject.org is...

comment:7 Changed 4 months ago by ahf

+9000

I would very much appreciate to have a working SMTP server for my @torproject.org email as I've said a lot of times before. I'm fine with using forwards for receiving emails. I do understand that hosting anything related to email is a big burden to the admin team. I do think it is important to follow modern email best practices, including all of SPF, DKIM, DMARC, and their friends. I do think if we ever want to get rid of spam emails in the world we must stop the arcane thought model behind that every SMTP server in the world should be allowed to send email from any domain name in the world - I don't think Tor Project should try to fight that :-)

For LDAP password usage I do agree that I don't think it's good to re-use the same password everywhere, but there must be some solution to that? If I recall correctly, Postfix's authentication system is _very_ flexible and we should be able to find a solution to that problem if we think it's a blocker.

comment:8 Changed 4 months ago by mikeperry

In a fit of desperation, I set up my mail client to use an SSH LocalForward to iranicum as a SMTP server. Preliminary testing indicates that my emails aren't going into SPAM folders of MIT and gmail anymore.

Because I also don't like to connect directly to tor project infrastructure from whatever network I happen to be on, I also do this over my TBB's Tor SOCKS port.

Here's a .ssh/config block that does all this:

host iranicum
   Hostname iranicum.torproject.org
   ProxyCommand /usr/bin/nc -X 5 -x 127.0.0.1:9150 %h %p  # Use TBB's Tor to get to iranicum
   LocalForward 2525 127.0.0.1:25  # Forward my 2525 to iranicum:25

I then tell my mailclient that when I send mail as my @torproject.org address, use 127.0.0.1:2525 with STARTTLS, and no auth. After I run ssh iranicum and store an exception for the cert, it works.

No LDAP! Just netcat, duct tape, and bubblegum.

I'll update this ticket if I notice that any mails fail to go through still.

comment:9 Changed 4 months ago by anarcat

Anyone able to impersonate a @tpo is a _serious_ problem imo and we have *no* defenses in place against that. They won't be perfect but it would at least be a start...

If you're not talking about DKIM/SPF, then I'm not sure this is an argument - anyone can still impersonate @tpo if you enable SMTP-AUTH.

But for now I would _just_ simply want my email to STOP BEING DROPPED by email servers that check the origin server of the From: address (which honestly I can't blame them to do).

I would love to help with that. I would need details about email configuration and the emails delivered.

For now we don't know why/when/who and how do those emails get dropped, because I have no information on when that happens and to who.

I have already asked people here to provide concrete reports of email delivery failure, and I'm still waiting for that proper bug report.

Come on people, you are developers, I'm sure you understand the value of a good bug report! :) I understand this is a feature request, but for now, I need you to help me help you and give me information I can work with.

I have made a short guide to detail the kind of information we need for failure reports:

https://help.torproject.org/tsa/doc/reporting-email-problems/

And whatever Gmail/Facebook do, at this point, I honestly do not care, I just want, by all means necessary, to be able to use my @tpo without ending up in SPAM or being silently dropped... This is not happening to only me and it will get worsts as big providers tighten their email rules over time.

I agree! We all want email to work.

But "just do SMTP-AUTH" is not necessarily a solution here. For example, we had troubles delivering to gmail.com from mailing lists as well, and that, as far as I know, would not necessarily have been solved by SMTP-AUTH.

For LDAP password usage I do agree that I don't think it's good to re-use the same password everywhere, but there must be some solution to that? If I recall correctly, Postfix's authentication system is _very_ flexible and we should be able to find a solution to that problem if we think it's a blocker.

It's definitely a blocker, and there's a solution (#6367). But there's more to SMTP-AUTH than just hooking it into Postfix...

In a fit of desperation, I set up my mail client to use an SSH LocalForward to iranicum as a SMTP server. Preliminary testing indicates that my emails aren't going into SPAM folders of MIT and gmail anymore.

For those wishing to send email over SSH, which is an ... interesting solution, to say the least, you might want to review the notes I have made for my personal use in https://gitlab.com/anarcat/rsendmail It details, among other things, how to setup a passwordless, but restricted, SSH key to ensure unattended delivery works, along with integration with normal, local MTAs.

comment:10 Changed 2 months ago by anarcat

list of valid LDAP users:

ssh alberti.torproject.org 'ldapsearch -ZZ -vLx -h db.torproject.org -b "ou=users,dc=torproject,dc=org" 
"(&(!(|(objectclass=debianRoleAccount)(objectClass=debianGroup)(objectClass=simpleSecurityObject)(shadowExpire=1)))(objectClass=debianAccount))" 2>/dev/null | sed -n "/^uid: /{s/^uid: \([^ ]*\)$/\1/;p}" '

comment:11 Changed 11 days ago by anarcat

we reached a conclusion for this in Stockholm, which was documented here:

https://trac.torproject.org/projects/tor/wiki/org/meetings/2019Stockholm/Notes/EmailNotEmail

the next steps are:

  1. shutdown the Jabber server (#31686)
  2. reset all webRTC passwords
  3. optionally, setup a separate email server to accept submissions and keep mail servers aware that not only eugeni sends email
  4. hook up the webRTC password as authentication in Postfix in that server (or eugeni)
  5. do tests with the users in this ticket, and if this works, propagate to all current LDAP users
  6. create LDAP accounts for more users who want to use the system
Last edited 10 days ago by anarcat (previous) (diff)

comment:12 Changed 5 days ago by anarcat

weasel disagrees we should reuse the webRTC password, which changes the roadmap and frees us from blocking on the jabber server, which is spun off completely separately in #31700 (and not #31686 as previously mentioned).

So our procedure should now be:

  1. create a new field (emailPassword?) in the LDAP schema
  2. update the mail gateway to support changes to the field
  3. update the web interface (to support changing the field as well?)
  4. optionally, setup a separate email server to accept submissions and keep mail servers aware that not only eugeni sends email
  5. hook up the password field as authentication in Postfix in the server (probably through ud-generate?)
  6. do tests with the users in this ticket, and if this works, propagate to all current LDAP users
  7. create LDAP accounts for more users who want to use the system

i'm not very familiar with the first steps here, so that might be more complicated than i thought.

Note: See TracTickets for help on using tickets.