Opened 8 weeks ago

Closed 6 weeks ago

#32105 closed defect (fixed)

bridges@torproject.org don't respond

Reported by: mh828 Owned by: phw
Priority: Medium Milestone:
Component: Circumvention/BridgeDB Version:
Severity: Normal Keywords: BugSmashFund
Cc: Actual Points: 0.3
Parent ID: Points:
Reviewer: cohosh Sponsor: Sponsor30

Description

I use Gmail to send email to bridges@…, but it doesn't respond.

Child Tickets

Change History (15)

comment:1 Changed 8 weeks ago by arma

Component: - Select a componentCircumvention/BridgeDB
Priority: Very HighMedium

Might be good to use this ticket to figure out how to help people who get into this situation, since there will be more.

comment:2 Changed 8 weeks ago by arma

One possibility is to point out that bridges@… is actually a different mailserver, with different antispam rules, than bridges@… which is where the mail (hopefully) ultimately ends up.

That's the conflict between giving users an easier-to-use email address, vs getting mail reliably to the host running the bridgedb service.

For gmail mails, it shouldn't be a difference. But it is one variable to look at.

comment:3 Changed 8 weeks ago by arma

Another possibility is that bridgedb rate limits responses, since it can't actually authenticate that the mail really comes from you, and it doesn't want to be turned into a mail bombing service by internet jerks. So I wonder if you sent a mail to bridges@ a while ago, and now it won't answer you, and that's why.

comment:4 Changed 8 weeks ago by arma

A third possibility is that gmail has decided that bridgedb is a spammer, and doesn't want to receive its replies. Maybe gmail applies different levels of spam filtering to different gmail accounts, which would make this one particularly tough to track down.

And a fourth possibility is that actually you did get the response, but gmail helpfully hid it from you, in a spam folder, a subfolder, or whatever gmail does to "improve" usability.

comment:5 Changed 8 weeks ago by gaba

Sponsor: Sponsor30

comment:6 Changed 8 weeks ago by phw

Status: newneeds_information

I could reproduce this issue. Here are BridgeDB's logs after I requested bridges from my gmail address:

22:44:41 DEBUG    L373:server.validateTo()      Validating SMTP 'RCPT TO:' email address...                                                                
22:44:41 INFO     L613:autoresponder.reply()    Got an email; deciding whether to reply.                                                                   
22:44:41 DEBUG    L664:autoresponder.runCheck() Canonicalizing client email domain...                                                                      
22:44:41 DEBUG    L673:autoresponder.runCheck() Canonical email domain: gmail.com                                                                          
22:44:41 ERROR    L682:autoresponder.runCheck() SMTP/Email canonical domain mismatch!                                                                      
22:44:41 DEBUG    L685:autoresponder.runCheck() Canonical domain mismatch: polyanthum != gmail.com                                                         
22:44:41 INFO      L60:dkim.checkDKIM()         Checking DKIM verification results...                                                                      
22:44:41 DEBUG     L61:dkim.checkDKIM()         Domain has rules: ignore_dots, dkim                                                                        
22:44:41 INFO      L72:dkim.checkDKIM()         Rejecting bad DKIM header on incoming email: u'dunno'

Here's the full email:

Received: from polyanthum ([[scrubbed]] helo=polyanthum)                                
  by polyanthum.torproject.org with BridgeDB (0.9.0+0.g681963d.dirty)                                                                                      
  for bridges@bridgedb; Wed, 16 Oct 2019 22:55:49 +0000                                                                                                    
From [scrubbed]  Wed Oct 16 22:55:49 2019                                                                                                                  
X-Original-To: [scrubbed].org                                                                                                                              
Delivered-To: [scrubbed].org                                                                                                                               
Received: from eugeni.torproject.org (eugeni.torproject.org [IPv6:2a01:4f8:10b:239f:0:ab4:202:1])                                                          
  (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits))                                                                                   
  (Client CN "clientcerts/eugeni.torproject.org", Issuer "auto-ca.torproject.org" (verified OK))                                                           
  by polyanthum.torproject.org (Postfix) with ESMTPS id 46B0B1001CF                                                                                        
  for <[scrubbed].org>; Wed, 16 Oct 2019 22:55:49 +0000 (UTC)                                                                                              
Received: from localhost (localhost [[scrubbed]])                                                                                                          
  by eugeni.torproject.org (Postfix) with ESMTP id 24E15E198D                                                                                              
  for <[scrubbed].org>; Wed, 16 Oct 2019 22:55:49 +0000 (UTC)                                                                                              
X-Virus-Scanned: Debian amavisd-new at                                                                                                                     
Received: from eugeni.torproject.org ([[scrubbed]])                                                                                                        
  by localhost (eugeni.torproject.org [[scrubbed]]) (amavisd-new, port 10024)                                                                              
  with ESMTP id QajNQiO4VjCc for <[scrubbed].org>;                                                                                                         
  Wed, 16 Oct 2019 22:55:49 +0000 (UTC)                                                                                                                    
Received: from mail-vs1-xe36.google.com (mail-vs1-xe36.google.com [IPv[scrubbed]])                                                                         
  (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits))                                                                                   
  (Client CN "smtp.gmail.com", Issuer "GTS CA 1O1" (not verified))                                                                                         
  by eugeni.torproject.org (Postfix) with ESMTPS id D10D6E198C                                                                                             
  for <[scrubbed]>; Wed, 16 Oct 2019 22:55:48 +0000 (UTC)                                                                                                  
Received: by mail-vs1-xe36.google.com with SMTP id d3so302626vsr.1                                                                                         
        for <[scrubbed]>; Wed, 16 Oct 2019 15:55:48 -0700 (PDT)                                                                                            
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;                                                                                                      
        d=gmail.com; s=20161025;                                                                                                                           
        h=mime-version:from:date:message-id:subject:to;                                                                                                    
        bh=ngRdnins9CRptXBut2wpGkSCNfH0hUIw7lj/EkgMSE0=;                                                                                                   
        b=clpdZk8NzD3i9cKiuDgzqJSSmCS0hdf0LTyXfSB0+8c5595EdbGEInR9CuZ+MOjzKC                                                                               
         nWQQeZgTt4ZLcIrA1BqKYxMfZc5eRFkyKmyh3Pq8T5aHf2n+NsyAlaX94RSg7YS+B5Rh                                                                              
         6TSXG6DmVZPtarYZriOHEfyXovkdHX9Pr56/wOBOepjNsy0Z1H1PwKMLyIKcCZi4aWRf                                                                              
         K40IZpSr2iHmcM4AUtPQyj85tRk12hbP5hvfYoNmR7j5lBUhXqKcpGofxOQtPusgUqM3                                                                              
         6hZJkFdsLPtizjOWQLWvBwQWdtlDhQOveovsHReZAVL6p8BLPbuJ9wdeuAVQO9Ws4Fey                                                                              
         CEwQ==                                                                                                                                            X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;                                                                                             
        d=1e100.net; s=20161025;                                                                                                                           
        h=x-gm-message-state:mime-version:from:date:message-id:subject:to;                                                                                 
        bh=ngRdnins9CRptXBut2wpGkSCNfH0hUIw7lj/EkgMSE0=;                                                                                                   
        b=la81DfjtXqYcs7zHkDTgdvcTPyJ8hFkLTYU1E6wKpzH8CXdI30YNIvVJ1979loi/20                                                                               
         eYq7bLjp2A2+SdJOio+gYLz0bXr/sFyuOx09uP5/oXuh2Dvj2DAaxejaSpTBOcxvKZn+                                                                              
         ud77fUBBEKE9256gSFeQm0YMz099mjIpSJLwycCG7iHEJXnn2Px/e3lqrIbt9ce+NjNi                                                                              
         L8zyp5qcd2VJFVM5lk940kWNqthsh164ZmVDXDN6QyMPXpi8VkqXw0CgHLL6SBUsAmhU                                                                                       6AivgW7x+zGWJN5MJFtt4mJz/3SgcNGUUmdfGau8ksbRG6y6ETYASc+96MSIaNhNoYb3                                                                              
         x82g==                                                                                                                                            
X-Gm-Message-State: APjAAAWO0amLn5psJwvrlW5Go4jx0NXEhKEOCn2G3ICYtzL2HaCGIzVQ                                                                               
  G0ldr7O2yR/3fvwcnUYoBdJDDzEbLI2Uvw9ioZlC                                                                                                                 X-Google-Smtp-Source: APXvYqzSyJE2SxcmnFJOhPjk/JFjPTf+MJHfpcT/ueP1Jv0/4J67w2iaW9lL+77mWdlZRb+VkhWEK7qw24343s2E2Pk=                                         
X-Received: by [scrubbed] with SMTP id y13mr170018vso.96.1571266545303;                                                                                    
 Wed, 16 Oct 2019 15:55:45 -0700 (PDT)                                                                                                                     
MIME-Version: 1.0                                                                                                                                          
From: Philipp Winter <[scrubbed]>                                                                                                                          
Date: Wed, 16 Oct 2019 15:55:34 -0700                                                                                                                      
Message-ID: <CAF09V-5OYyPxE83Q1SYkBxA=[scrubbed].com>                                                                                                      
Subject:                                                                                                                                                   
To: [scrubbed]                                                                                                                                             
Content-Type: text/plain; charset="UTF-8"                                                                                                                  
X-DKIM-Authentication-Results: dunno

get transport obfs4

The header X-DKIM-Authentication-Results: dunno seems to be the issue here. A tool called dkimproxy is supposed to add this header. But where is it? It doesn't seem to be running or installed on polyanthum.

comment:7 Changed 8 weeks ago by phw

Owner: set to phw
Status: needs_informationaccepted

BridgeDB uses the binary dkimverify in ~/.procmailrc. Here's an excerpt:

:0
* ^DKIM-Signature:
{
        :0 HBfw
        * ? dkimverify
                | formail -I "X-DKIM-Authentication-Results: pass"                                        
}

dkimverify used to be provided by the python-dkim package (which is installed on polyanthum) but with the upgrade to Debian Buster, it is now provided by python3-dkim (which is not installed). Here's an excerpt of /usr/share/doc/python-dkim/changelog.Debian.gz:

dkimpy (0.8.0-1) unstable; urgency=medium

  [...]                               
              
  * Provide /usr/bin scripts from python3-dkim rather than python-dkim                                    
    - Update package descriptions
    - Python3-dkim Replaces previous versions of python-dkim                                              

comment:8 Changed 8 weeks ago by phw

Reviewer: cohosh
Status: acceptedneeds_review

Here's a BridgeDB patch that makes this undocumented OS-level dependency explicit: https://github.com/NullHypothesis/bridgedb/commit/67b9aad067b00d6e618736ab07b6062cddb04543

comment:9 Changed 8 weeks ago by arma

Good find, phw!

For posterity, the previous times this happened were #10003 and, back when dkimverify was a perl thing, #4930.

This time we've registered, in puppet, the need for the python3-dkim package on the bridgedb server, so next time puppet makes a bridgedb server it should appear.

comment:10 in reply to:  8 ; Changed 8 weeks ago by arma

Replying to phw:

Here's a BridgeDB patch that makes this undocumented OS-level dependency explicit: https://github.com/NullHypothesis/bridgedb/commit/67b9aad067b00d6e618736ab07b6062cddb04543

Hi phw! That change looks good.

But, we probably want a bigger change too. It looks like bridgedb relies on the .procmailrc contents of the bridgedb user, in order to add the "X-DKIM-Authentication-Results: pass" header, without which bridgedb's email module won't be pleased with the mail it receives. I think that means the mail receiving scripts are part of bridgedb and should be documented, or in git, somewhere too.

comment:11 Changed 8 weeks ago by arma

hiro tells me that she rebuilt the gettor machine last week and added all the needed packages to puppet. so gettor is already all set wrt this bug (and maybe encountered it independently, i'm not sure).

comment:12 Changed 7 weeks ago by cohosh

I left a comment on the commit but otherwise it looks good.

comment:13 in reply to:  10 Changed 7 weeks ago by phw

Replying to arma:

But, we probably want a bigger change too. It looks like bridgedb relies on the .procmailrc contents of the bridgedb user, in order to add the "X-DKIM-Authentication-Results: pass" header, without which bridgedb's email module won't be pleased with the mail it receives. I think that means the mail receiving scripts are part of bridgedb and should be documented, or in git, somewhere too.


Good point. Here's a patch for bridgedb-admin, which we use for administration-related tooling around BridgeDB: https://gitweb.torproject.org/user/phw/bridgedb-admin.git/commit/?h=fix/32105

comment:14 Changed 6 weeks ago by cohosh

Status: needs_reviewmerge_ready

Looks good.

I'm not sure whether the point of this repository is to also make it so that other people can set up their own bridgedb instance and easily configure it to work out of the box. If so, it would be nice to factor out some strings like bridges.torproject.org into variables at the top of procmailrc so they are more easily configured.

That being said, I'm sure it wouldn't work out the box as is anyway so maybe this isn't important.

comment:15 in reply to:  14 Changed 6 weeks ago by phw

Actual Points: 0.3
Keywords: BugSmashFund added
Resolution: fixed
Status: merge_readyclosed

Replying to cohosh:

Looks good.


Thanks, merged.

I'm not sure whether the point of this repository is to also make it so that other people can set up their own bridgedb instance and easily configure it to work out of the box. If so, it would be nice to factor out some strings like bridges.torproject.org into variables at the top of procmailrc so they are more easily configured.

That being said, I'm sure it wouldn't work out the box as is anyway so maybe this isn't important.


I think the purpose of bridgedb-admin is to keep track of administrative scripts that we use to manage BridgeDB, e.g., systemd service files, a logrotate config, and our apache config. It's not meant to work out-of-the-box for third parties but it would probably be very helpful when setting up a BridgeDB instance.

Note: See TracTickets for help on using tickets.