bridges@torproject.org don't respond

[mh828]

I use Gmail to send email to bridges@…, but it doesn't respond.

[arma]

Might be good to use this ticket to figure out how to help people who get into this situation, since there will be more.

[arma]

One possibility is to point out that bridges@… is actually a different mailserver, with different antispam rules, than bridges@… which is where the mail (hopefully) ultimately ends up.

That's the conflict between giving users an easier-to-use email address, vs getting mail reliably to the host running the bridgedb service.

For gmail mails, it shouldn't be a difference. But it is one variable to look at.

[arma]

Another possibility is that bridgedb rate limits responses, since it can't actually authenticate that the mail really comes from you, and it doesn't want to be turned into a mail bombing service by internet jerks. So I wonder if you sent a mail to bridges@ a while ago, and now it won't answer you, and that's why.

[arma]

A third possibility is that gmail has decided that bridgedb is a spammer, and doesn't want to receive its replies. Maybe gmail applies different levels of spam filtering to different gmail accounts, which would make this one particularly tough to track down.

And a fourth possibility is that actually you did get the response, but gmail helpfully hid it from you, in a spam folder, a subfolder, or whatever gmail does to "improve" usability.

[phw]

I could reproduce this issue. Here are BridgeDB's logs after I requested bridges from my gmail address:

22:44:41 DEBUG    L373:server.validateTo()      Validating SMTP 'RCPT TO:' email address...                                                                
22:44:41 INFO     L613:autoresponder.reply()    Got an email; deciding whether to reply.                                                                   
22:44:41 DEBUG    L664:autoresponder.runCheck() Canonicalizing client email domain...                                                                      
22:44:41 DEBUG    L673:autoresponder.runCheck() Canonical email domain: gmail.com                                                                          
22:44:41 ERROR    L682:autoresponder.runCheck() SMTP/Email canonical domain mismatch!                                                                      
22:44:41 DEBUG    L685:autoresponder.runCheck() Canonical domain mismatch: polyanthum != gmail.com                                                         
22:44:41 INFO      L60:dkim.checkDKIM()         Checking DKIM verification results...                                                                      
22:44:41 DEBUG     L61:dkim.checkDKIM()         Domain has rules: ignore_dots, dkim                                                                        
22:44:41 INFO      L72:dkim.checkDKIM()         Rejecting bad DKIM header on incoming email: u'dunno'

Here's the full email:

Received: from polyanthum ([[scrubbed]] helo=polyanthum)                                
  by polyanthum.torproject.org with BridgeDB (0.9.0+0.g681963d.dirty)                                                                                      
  for bridges@bridgedb; Wed, 16 Oct 2019 22:55:49 +0000                                                                                                    
From [scrubbed]  Wed Oct 16 22:55:49 2019                                                                                                                  
X-Original-To: [scrubbed].org                                                                                                                              
Delivered-To: [scrubbed].org                                                                                                                               
Received: from eugeni.torproject.org (eugeni.torproject.org [IPv6:2a01:4f8:10b:239f:0:ab4:202:1])                                                          
  (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits))                                                                                   
  (Client CN "clientcerts/eugeni.torproject.org", Issuer "auto-ca.torproject.org" (verified OK))                                                           
  by polyanthum.torproject.org (Postfix) with ESMTPS id 46B0B1001CF                                                                                        
  for <[scrubbed].org>; Wed, 16 Oct 2019 22:55:49 +0000 (UTC)                                                                                              
Received: from localhost (localhost [[scrubbed]])                                                                                                          
  by eugeni.torproject.org (Postfix) with ESMTP id 24E15E198D                                                                                              
  for <[scrubbed].org>; Wed, 16 Oct 2019 22:55:49 +0000 (UTC)                                                                                              
X-Virus-Scanned: Debian amavisd-new at                                                                                                                     
Received: from eugeni.torproject.org ([[scrubbed]])                                                                                                        
  by localhost (eugeni.torproject.org [[scrubbed]]) (amavisd-new, port 10024)                                                                              
  with ESMTP id QajNQiO4VjCc for <[scrubbed].org>;                                                                                                         
  Wed, 16 Oct 2019 22:55:49 +0000 (UTC)                                                                                                                    
Received: from mail-vs1-xe36.google.com (mail-vs1-xe36.google.com [IPv[scrubbed]])                                                                         
  (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits))                                                                                   
  (Client CN "smtp.gmail.com", Issuer "GTS CA 1O1" (not verified))                                                                                         
  by eugeni.torproject.org (Postfix) with ESMTPS id D10D6E198C                                                                                             
  for <[scrubbed]>; Wed, 16 Oct 2019 22:55:48 +0000 (UTC)                                                                                                  
Received: by mail-vs1-xe36.google.com with SMTP id d3so302626vsr.1                                                                                         
        for <[scrubbed]>; Wed, 16 Oct 2019 15:55:48 -0700 (PDT)                                                                                            
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;                                                                                                      
        d=gmail.com; s=20161025;                                                                                                                           
        h=mime-version:from:date:message-id:subject:to;                                                                                                    
        bh=ngRdnins9CRptXBut2wpGkSCNfH0hUIw7lj/EkgMSE0=;                                                                                                   
        b=clpdZk8NzD3i9cKiuDgzqJSSmCS0hdf0LTyXfSB0+8c5595EdbGEInR9CuZ+MOjzKC                                                                               
         nWQQeZgTt4ZLcIrA1BqKYxMfZc5eRFkyKmyh3Pq8T5aHf2n+NsyAlaX94RSg7YS+B5Rh                                                                              
         6TSXG6DmVZPtarYZriOHEfyXovkdHX9Pr56/wOBOepjNsy0Z1H1PwKMLyIKcCZi4aWRf                                                                              
         K40IZpSr2iHmcM4AUtPQyj85tRk12hbP5hvfYoNmR7j5lBUhXqKcpGofxOQtPusgUqM3                                                                              
         6hZJkFdsLPtizjOWQLWvBwQWdtlDhQOveovsHReZAVL6p8BLPbuJ9wdeuAVQO9Ws4Fey                                                                              
         CEwQ==                                                                                                                                            X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;                                                                                             
        d=1e100.net; s=20161025;                                                                                                                           
        h=x-gm-message-state:mime-version:from:date:message-id:subject:to;                                                                                 
        bh=ngRdnins9CRptXBut2wpGkSCNfH0hUIw7lj/EkgMSE0=;                                                                                                   
        b=la81DfjtXqYcs7zHkDTgdvcTPyJ8hFkLTYU1E6wKpzH8CXdI30YNIvVJ1979loi/20                                                                               
         eYq7bLjp2A2+SdJOio+gYLz0bXr/sFyuOx09uP5/oXuh2Dvj2DAaxejaSpTBOcxvKZn+                                                                              
         ud77fUBBEKE9256gSFeQm0YMz099mjIpSJLwycCG7iHEJXnn2Px/e3lqrIbt9ce+NjNi                                                                              
         L8zyp5qcd2VJFVM5lk940kWNqthsh164ZmVDXDN6QyMPXpi8VkqXw0CgHLL6SBUsAmhU                                                                                       6AivgW7x+zGWJN5MJFtt4mJz/3SgcNGUUmdfGau8ksbRG6y6ETYASc+96MSIaNhNoYb3                                                                              
         x82g==                                                                                                                                            
X-Gm-Message-State: APjAAAWO0amLn5psJwvrlW5Go4jx0NXEhKEOCn2G3ICYtzL2HaCGIzVQ                                                                               
  G0ldr7O2yR/3fvwcnUYoBdJDDzEbLI2Uvw9ioZlC                                                                                                                 X-Google-Smtp-Source: APXvYqzSyJE2SxcmnFJOhPjk/JFjPTf+MJHfpcT/ueP1Jv0/4J67w2iaW9lL+77mWdlZRb+VkhWEK7qw24343s2E2Pk=                                         
X-Received: by [scrubbed] with SMTP id y13mr170018vso.96.1571266545303;                                                                                    
 Wed, 16 Oct 2019 15:55:45 -0700 (PDT)                                                                                                                     
MIME-Version: 1.0                                                                                                                                          
From: Philipp Winter <[scrubbed]>                                                                                                                          
Date: Wed, 16 Oct 2019 15:55:34 -0700                                                                                                                      
Message-ID: <CAF09V-5OYyPxE83Q1SYkBxA=[scrubbed].com>                                                                                                      
Subject:                                                                                                                                                   
To: [scrubbed]                                                                                                                                             
Content-Type: text/plain; charset="UTF-8"                                                                                                                  
X-DKIM-Authentication-Results: dunno

get transport obfs4

The header X-DKIM-Authentication-Results: dunno seems to be the issue here. A tool called dkimproxy is supposed to add this header. But where is it? It doesn't seem to be running or installed on polyanthum.

[phw]

BridgeDB uses the binary dkimverify in ~/.procmailrc. Here's an excerpt:

:0
* ^DKIM-Signature:
{
        :0 HBfw
        * ? dkimverify
                | formail -I "X-DKIM-Authentication-Results: pass"                                        
}

dkimverify used to be provided by the python-dkim package (which is installed on polyanthum) but with the upgrade to Debian Buster, it is now provided by python3-dkim (which is not installed). Here's an excerpt of /usr/share/doc/python-dkim/changelog.Debian.gz:

dkimpy (0.8.0-1) unstable; urgency=medium

  [...]                               
              
  * Provide /usr/bin scripts from python3-dkim rather than python-dkim                                    
    - Update package descriptions
    - Python3-dkim Replaces previous versions of python-dkim                                              

[phw]

Here's a BridgeDB patch that makes this undocumented OS-level dependency explicit: ​https://github.com/NullHypothesis/bridgedb/commit/67b9aad067b00d6e618736ab07b6062cddb04543

[arma]

Good find, phw!

For posterity, the previous times this happened were #10003 and, back when dkimverify was a perl thing, #4930.

This time we've registered, in puppet, the need for the python3-dkim package on the bridgedb server, so next time puppet makes a bridgedb server it should appear.

[arma]

Replying to phw:

Here's a BridgeDB patch that makes this undocumented OS-level dependency explicit: ​https://github.com/NullHypothesis/bridgedb/commit/67b9aad067b00d6e618736ab07b6062cddb04543

Hi phw! That change looks good.

But, we probably want a bigger change too. It looks like bridgedb relies on the .procmailrc contents of the bridgedb user, in order to add the "X-DKIM-Authentication-Results: pass" header, without which bridgedb's email module won't be pleased with the mail it receives. I think that means the mail receiving scripts are part of bridgedb and should be documented, or in git, somewhere too.

[arma]

hiro tells me that she rebuilt the gettor machine last week and added all the needed packages to puppet. so gettor is already all set wrt this bug (and maybe encountered it independently, i'm not sure).

[cohosh]

I left a comment on the commit but otherwise it looks good.

[phw]

Replying to arma:

But, we probably want a bigger change too. It looks like bridgedb relies on the .procmailrc contents of the bridgedb user, in order to add the "X-DKIM-Authentication-Results: pass" header, without which bridgedb's email module won't be pleased with the mail it receives. I think that means the mail receiving scripts are part of bridgedb and should be documented, or in git, somewhere too.

Good point. Here's a patch for bridgedb-admin, which we use for administration-related tooling around BridgeDB: ​https://gitweb.torproject.org/user/phw/bridgedb-admin.git/commit/?h=fix/32105

[cohosh]

Looks good.

I'm not sure whether the point of this repository is to also make it so that other people can set up their own bridgedb instance and easily configure it to work out of the box. If so, it would be nice to factor out some strings like bridges.torproject.org into variables at the top of procmailrc so they are more easily configured.

That being said, I'm sure it wouldn't work out the box as is anyway so maybe this isn't important.

[phw]

Replying to cohosh:

Looks good.

Thanks, merged.

I'm not sure whether the point of this repository is to also make it so that other people can set up their own bridgedb instance and easily configure it to work out of the box. If so, it would be nice to factor out some strings like bridges.torproject.org into variables at the top of procmailrc so they are more easily configured.

That being said, I'm sure it wouldn't work out the box as is anyway so maybe this isn't important.

I think the purpose of bridgedb-admin is to keep track of administrative scripts that we use to manage BridgeDB, e.g., systemd service files, a logrotate config, and our apache config. It's not meant to work out-of-the-box for third parties but it would probably be very helpful when setting up a BridgeDB instance.