Opened 5 months ago

Last modified 5 months ago

#29770 assigned defect

mails relayed from lists.tpo to gmail.com bounces

Reported by: anarcat Owned by: tpa
Priority: Medium Milestone:
Component: Internal Services/Service - lists Version:
Severity: Normal Keywords:
Cc: Actual Points:
Parent ID: Points:
Reviewer: Sponsor:

Description

It seems we're having trouble relaying mails to gmail.com and possibly other providers.

A similar question was raised by a tor-announce@ participant in Message-ID: <CAHQQdjtuVj68EVw_TsnDGiTRj1bY7iyxuehUDj6bya=KAqE0MQ@mail.gmail.com>

I myself had trouble sending mail to a @tpo account that is forwarded to gmail (bounce is Message-ID: <20190311210933.9AB57E1B16@eugeni.torproject.org>):

<target@gmail.com>: host gmail-smtp-in.l.google.com[64.233.167.26] said:
    550-5.7.1 This message does not have authentication information or fails to
    pass 550-5.7.1 authentication checks. To best protect our users from spam,
    the 550-5.7.1 message has been blocked. Please visit 550-5.7.1
    https://support.google.com/mail/answer/81126#authentication for more 550
    5.7.1 information. f2si4120705wrj.403 - gsmtp (in reply to end of DATA
    command)

I suspect this might have to do with (lack of) SPF records and/or ARC headers.

Child Tickets

Change History (6)

comment:1 Changed 5 months ago by weasel

I don't see where lists is relevant here.

Also, AIUI, this email was sent with an envelope From whose SPF record asks for the mail to be rejected.

comment:2 in reply to:  1 Changed 5 months ago by weasel

Replying to weasel:

I don't see where lists is relevant here.

Also, AIUI, this email was sent with an envelope From whose SPF record asks for the mail to be rejected.

Ok, so the extra information and bounce is probably only btb here.

The original issue is that tor-announce mail is not making it to gmail.

That's probably because the tor-announce list is configured to mess with both subject and body. It should stop doing that.

comment:3 Changed 5 months ago by anarcat

right, i think I have actually fixed my own specific problem, with your help. the problem was indeed that the Envelope-From was incorrectly mapped to my own email address. The fix was to simply change my email client to use the From: header to generate the envelope-from. In my case (notmuch-emacs), that meant the following customize settings:

 '(mail-envelope-from (quote header))
 '(mail-specify-envelope-from t)

That said, I do believe we have delivery problems to gmail in general, and I think the original report from that user should be considered as such, even though we don't have many details.

Sorry for the noise...

comment:4 Changed 5 months ago by anarcat

so we had another report of a problem occurring with gmail.com recipients today on the tor-internal@… mailing list.

the problem was with the email Message-ID: <20190321172741.GB78672@vpn209009.nrl.navy.mil> which gmail refused with the following error:

Mar 21 17:27:53 eugeni/eugeni postfix/smtp[4376]: 6C5ECE0ED2: to=<[REDACTED]@gmail.com>, relay=gmail-smtp-in.l.google.com[2a00:1450:400c:c00::1a]:25, delay=0.9, delays=0.03/0.22/0.15/0.5, dsn=5.7.1, status=bounced (host gmail-smtp-in.l.google.com[2a00:1450:400c:c00::1a] said: 550-5.7.1 Unauthenticated email from nrl.navy.mil is not accepted due to 550-5.7.1 domain's DMARC policy. Please contact the administrator of 550-5.7.1 nrl.navy.mil domain if this was a legitimate mail. Please visit 550-5.7.1  https://support.google.com/mail/answer/2451690 to learn about the 550 5.7.1 DMARC initiative. j5si3735441wmh.102 - gsmtp (in reply to end of DATA command))

The email indeed comes from a domain with the following DMARC policy:

_dmarc.nrl.navy.mil.	1031	IN	TXT	"v=DMARC1; p=reject; rua=mailto:dmarc-reporting@dren.mil; ri=86400"

... an aggressive, but not unusual or invalid DMARC policy.

I'm not sure what the way forward is here, but I believe that a simple fix would be to enable the general/from_is_list setting to "munge" the From header to be the mailing list itself instead of the original sender. this is a controversial change, so a "lesser-evil" alternative might be the privacy/sender/dmarc_moderation_action setting which does that only for messages with a DMARC policy. But then we get inconsistent headers for the mailing list which might confuse filters and/or users.

since this is a little controversial, i'd like to consult with fellow TPAs before going ahead with any change.

for those who want to experiment with those changes, those are direct URLs for the tor-project@ mailing list:

https://lists.torproject.org/cgi-bin/mailman/admin/tor-project/?VARHELP=general/from_is_list
https://lists.torproject.org/cgi-bin/mailman/admin/tor-project/?VARHELP=privacy/sender/dmarc_moderation_action

comment:5 Changed 5 months ago by arma

I'm fine with seeing us experiment with other options. Mail in general is getting
unreliable on the internet these days, but that doesn't mean we can't slow down our eventual demise.

I think it might still be the case that mit.edu silently discards mailman confirmation mails. Now when I want to subscribe arma@mit to a Tor list, I log into the mailman web interface as admin and subscribe it directly. Otherwise I can never confirm my intent to subscribe, because I never get the mail inviting me to confirm.

Once a year or so, gmail decides to start bouncing all of the mails that come from a given list, e.g. tor-relays@…. Our mailman, after a few messages to the list that all bounce from gmail, suspends the subscriptions of all gmail users on the list. Then we go through the horrible mailman web interface and un-suspend each of them.

There has been something going on with state.gov over the last few years, where mails from them to @tpo don't arrive.

And we should remember that @lists.tpo is a different system than @tpo.

So: experimentation good, but we should be sure to keep an eye on what *other* things get broken when we do our various tests.

Thanks!

comment:6 Changed 5 months ago by anarcat

Summary: mails relayed to gmail.com bounce backmails relayed from lists.tpo to gmail.com bounces

thanks for the feedback, @arma. i've marked this ticket specifically about lists.tpo, to distinguish it from email forwarding or other problems.

and also note this is specifically about gmail.com and, so far, only about one specific domain (the .mil one) having a strict DMARC policy. we might want to open separate issues for other specific problems that could have other solutions.

Note: See TracTickets for help on using tickets.