Opened 4 years ago

Last modified 11 days ago

#10836 new enhancement

Enable mail account autoconfig dialog in TorBirdy

Reported by: ben Owned by: ben
Priority: Medium Milestone:
Component: Applications/TorBirdy Version:
Severity: Normal Keywords:
Cc: tagnaq@… Actual Points:
Parent ID: Points:
Reviewer: Sponsor:

Description

Currently, TorBirdy entirely blocks the mail account autoconfig dialog in Thunderbird. It requires the user to manually configure the mail account servers.


This is suboptimal, because the declared goal of TorBirdy is to reach common users (not geeks), and common users have massive problems with this configuration. This is why they use webmail, and why we write this dialog to help them with Thunderbird - they simply *can't* do it alone.

Furthermore, if they try to find the settings themselves on the web, they

  • expose themselves to similar or worse phishing attempts (if you can serve a bad config XML file, you can serve a bad HTML documentation page)
  • more importantly, the mail configs published by the ISPs are often without encryption.

With the ISPDB, I took great care to find and use the best config that an ISP offers, esp. SSL and encrypted passwords, even if that config is undocumented and not officially supported. In a way, you could compare the ISPDB with HTTPS Everywhere, because it performs a similar function (use SSL where possible, even if not advertized by site) and even similar means (HTTPS Everywhere communicates with some central servers, just like the Mozilla ISPDB).

Thus, I think disabling the autoconfig dialog does users a dis-service not only in convenience and usability (in the literal sense of the word), but more importantly in security, because we know about SSL configs that users might not know or find.


The reason why the autoconfig dialog was disabled were some HTTP (without SSL) calls and direct socket calls.
Thus, in Mozilla bug 669282 [1], I attached a patch to disable them. I wrote this patch specifically for TorBirdy.
[1] https://bugzilla.mozilla.org/show_bug.cgi?id=669282

Child Tickets

Change History (18)

comment:1 Changed 4 years ago by ben

Please take this patch and set the following preference:
mailnews.auto_config.fetchFromISP.emailAddressEnabled = false

If you're concerned about proxy bypass, also set:
mailnews.auto_config.guess.enabled = false
However, all that this would leak outside the proxy is the identity of my ISP, not my own identity.

Note: It is not necessary to set fetchFromISP.enabled = false. Assuming the above pref is set, all the "fetch from ISP" will leak is the identity of my ISP provider *via* the proxy (because fetch ISP is an HTTP that does use the proxy settings). Given that the next step will be to check imap.myisp.com:143, I don't think it's a problem to check autoconfig.myisp.com or myisp.com, either.

comment:2 in reply to:  description Changed 4 years ago by tagnaq

Cc: tagnaq@… added

Hi ben,

thanks for your help to bring autoconfiguration to TorBirdy users.

Replying to ben:

The reason why the autoconfig dialog was disabled were some HTTP (without SSL) calls and direct socket calls.
Thus, in Mozilla bug 669282 [1], I attached a patch to disable them. I wrote this patch specifically for TorBirdy.
[1] https://bugzilla.mozilla.org/show_bug.cgi?id=669282

Autoconfiguration is disabled for mainly two reasons:

(1) it leaks requests and does not adhere to Thunderbird's proxy settings [3]
(This issue is limited to the guessing phase and does not affect the entire autoconfiguration feature. I'm not sure if guessing is something we want at all - even if it wouldn't leak.)
(2) it might uses an insecure channel (HTTP) to fetch the MUA configuration (even if it goes through the proxy this is not something we would want)

Only if both issues are addressed we could probably enable autoconfiguration.

I assume 'mailnews.auto_config.guess.enabled = false' would fix the proxy bypass (1) phase by skipping it.
The insecure HTTP fetches (going to the ISP, not Mozilla/ISPDB) (2) would be fixed by opting-out via 'fetchFromISP.enabled = false'.
So in the end TorBirdy users would only use the ISPDB part (HTTPS fetches) of Thunderbird's autoconfiguration feature with the above settings.

If my understanding is correct, I would test your patch with the above settings to get some real testresults.

Another issue that might result from autoconfiguration:
I generally recommend POP3 over IMAP for TorBirdy users, IIRC autoconfiguration will default to IMAP over POP3.

[2] https://developer.mozilla.org/en-US/docs/Mozilla/Thunderbird/Autoconfiguration
[3] https://bugzilla.mozilla.org/show_bug.cgi?id=669238

comment:3 Changed 4 years ago by tagnaq

Instead of removing the email adress from the GET parameters and still sending a plain HTTP request, a 'forceHTTPS' pref for these ISP fetches would be nice.

comment:4 Changed 4 years ago by ben

Yes, your understanding of the patch is correct.

It should do what you need, because I developed it specifically for you/TorBirdy.

Please note that autoconfig is of high importance for end users. Many users are not capable of configuring Thunderbird manually. I think webmail is popular in part because of that. Thus, please do leave as much of autoconfig intact as possible.

Also note that the result of any autoconfig is presented to the end user for approval. See above for an argument of web search vs. autoconfig.

I generally recommend POP3 over IMAP for TorBirdy users, IIRC autoconfiguration will default to IMAP over POP3.

Yes, I can see why. Nonetheless, I think that depends on the situation. If you can't be sure that your notebook won't be confiscated, it's good to have access to your communication from anywhere. I think a number of your users use webmail, too.

The current UI allows the user the choice (if the ISPs offers both, which many do). Yes, the default normally is IMAP. The current dialog has very little text explanation, but the UI structure is (IIRC) so that the text can be 2-3 lines of dialog text. Thus, I would recommend to change the text label in the UI to educate users more about the tradeoff they make with IMAP and POP3, and present arguments and use cases for both options, and let them make a choice. In fact, I could make you a patch that does that. You could also change the default to POP3, if you're really sure that's better.

Last edited 4 years ago by ben (previous) (diff)

comment:5 Changed 4 years ago by ben

Instead of removing the email adress from the GET parameters and still sending a plain HTTP request, a 'forceHTTPS' pref for these ISP fetches would be nice.

That's not possible. I really wanted to use HTTPS for this. It's that almost no ISP was able to do this. The bigger hosters couldn't, because they need one cert per customer domain (or all customer domains in one cert), and that's not feasible. The small entities can't, because they often don't have a cert and getting one is too complicated - the guy wanting the autoconfig is usually not responsible for the web servers. In fact, I'm having big trouble getting people to use the fetch ISP mechanism at all.

That said, the most important is the ISPDB at Mozilla. That covers about 40% of the accounts. I think fetchISP covers only a very small percentage, if even 1%. Guess config is much more important - in practice - as much as I dislike the idea.

comment:6 Changed 4 years ago by sukhbir

Thanks for your work on this. Just to clarify: the manual configuration wizard sets secure defaults by ensuring that IMAP/POP/SMTP are accessed over SSL/TLS by setting the relevant ports (993, 995, 465). This is of course not as optimal as having a secure automatic configuration of accounts but I just thought I should point this out.

comment:8 Changed 4 years ago by tagnaq

lunar, thanks for pointing that out.

ben I see your point that (almost?) no ISP is serving their xml files via HTTPS (currently there is no point to do so since Thunderbird would not make use of it), but could we at least try HTTPS before falling back to HTTP instead of hard coding HTTP only? (The security benefit is probably questionable since this is vulnerable to downgrade attacks..)

What do you think about merging (some?) of tails modifications to the autoconfig [1] code? (I haven't looked at the code but the design [1] matches the 'forceHTTPS' suggestion.)

I guess
mailnews.auto_config_ssl_only = true
equals more or less to
'fetchFromISP.enabled = false'
since (almost?) no one is serving their xml files via HTTPS.

If you are saying that not even 1% of users is affected by fetchISP, then 'fetchFromISP.enabled = false' wouldn't hurt to many users I guess. bitmessage.ch users would be one of them though.

For simplicity I wanted to have fetchISP do HTTPS requests or none at all ('fetchFromISP.enabled = false'), but I could also imagine insecure HTTP fetches if we impose restrictions on the acceptable hostnames for the mailservers. (TorBirdy - not Thunderbird - would take care of enforcing those restrictions - I hope that's possible.)

Mailserver hostnames for the email address user@… would have to match *.example.com.
If it doesn't match we fall back to manual configuration.
I'm wondering how many email addresses would fail this test.. (all custom domains hosted at gmail?)

If we allow insecure HTTP fetches with that restriction in place we would make use of ben's 'mailnews.auto_config.fetchFromISP.emailAddressEnabled = false' to avoid sending the user's email address in plaintext accross the wire.

A successful attack would require a certificate for a hostname under the domain of the email address (since we only fetch/send emails via SSL/STARTTLS).

I realized that autoconfig xml files can be used for more than just mailserver hostnames and ports/protocols, I'll look at it [2] in more detail to assess if that
opens any new attack vectors. Ben, is [2] up to date?

[2] https://wiki.mozilla.org/Thunderbird:Autoconfiguration:ConfigFileFormat

From tails "Modified autoconfig wizard" design paragraph at:
[1] https://tails.boum.org/blueprint/Return_of_Icedove__63__/#index4h3

When probing a mail provider for an xml config, first try HTTPS, then http (old behaviour: http only).
Introduce a boolean pref called mailnews.auto_config_ssl_only (that has a checkbox in the
autoconfiguration wizard) that does the following when true:

Only allow HTTPS when fetching xml configs from mail provider.
Only allow HTTPS when fetching xml configs from Mozilla's database (luckily the default URL is using HTTPS).
Don't check DNS MX records for mail configurations. This may need some rethinking for DNSSEC.
Only accept fetched xml configs that use safe email protocols (SSL/TLS for SMTP/IMAP/POP).
Only probe the mail server for safe protocols (SSL/TLS for SMTP/IMAP/POP).

comment:9 Changed 4 years ago by tagnaq

Ordering is also crucial.
Ben, is it possible to change the ordering (via pref) within autoconfig?

I would prefer

  1. ISPDB (HTTPS)
  2. fetchISP (HTTPS, HTTP)

over (this is the current setup)

  1. fetchISP (HTTP)
  2. ISPDB (HTTPS)

(This makes me think that HTTPS on ISPDB fetches is actually useless since the insecure HTTP fetch is happening first - even if that domain has an entry in Mozilla's ISPDB.)

Actually this would be my prefered ordering:

  1. fetchISP (HTTPS)
  2. ISPDB (HTTPS)
  3. fetchISP (HTTP) + hostname restrictions


comment:10 Changed 4 years ago by tagnaq

Filed an upstream bug for this (autoconfig ordering).
https://bugzilla.mozilla.org/show_bug.cgi?id=971347

comment:11 Changed 4 years ago by ben

Please remember that
1) The user manually reviews and approves the config
2) We warn about non-SSL (or broken SSL) configs.

So, in order to successfully attack, you not only have to attack the autoconfig algos, but *also* make your user a phishing victim, e.g. either by registering a real domain like googleemailservices.com . Furthermore, you need to register a CA cert for that hostname (which also excludes using 123.234.265.123 as hostname). We put that end user review in there quite deliberately as an additional security measure.


Frankly, I think the easiest way to attack a Thunderbird user is to simply block all his SSL connections. He'll fail with whatever right servers he tries, and all users but the most paranoid people will eventually fall back to using plaintext protocols in order to get to their mail. (People will do everything to see dancing hampsers .) And if you're that much on alert, you'll hoepfully catch the phishing mentioned above. Just to put all this into perspective - not saying your remarks are off-beat.

Last edited 4 years ago by ben (previous) (diff)

comment:12 Changed 4 years ago by ben

Re ordering: Yes, I think that's a good idea.

comment:13 Changed 4 years ago by ben

you could compare the ISPDB with HTTPS Everywhere, because it performs a similar function (use SSL where possible, even if not advertized by site) and even similar means (HTTPS Everywhere communicates with some central servers, just like the Mozilla ISPDB).

Hahahaha. I just found HTTPS Everywhere's database. Compare
https://gitweb.torproject.org/https-everywhere.git/tree/HEAD:/src/chrome/content/rules
https://svn.mozilla.org/mozillamessaging.com/sites/autoconfig.mozillamessaging.com/trunk/
:-)

comment:14 Changed 4 years ago by ben

Mailserver hostnames for the email address user@… would have to match *.example.com. If it doesn't match we fall back to manual configuration. I'm wondering how many email addresses would fail this test..

All hosted customer domains. I register lastname.name, and don't set up a whole server for myself, but use shared hosting. If I also want SSL without hostname/cert mismatch, I will always have a different email server hostname than my email address domain.

Don't check DNS MX records for mail configurations. This may need some rethinking for DNSSEC.

Likewise, this will break all hosted domains, even hosters in the ISPDB, and a number of vanity domains (@iamcool.com etc.). I know it sounds weak security-wise, but the attack surface is reduced significantly by the fact that Mozilla's server makes the MX lookup, and the result comes via HTTPS. (If Mozilla ever implements arbitrary DNS lookups, we could do both and compare the two results.)

I don't hope for DNSSEC anymore. (Very broken concepts in the spec, which hinders deployment.)

A successful attack would require a certificate for a hostname under the domain of the email address (since we only fetch/send emails via SSL/STARTTLS).

Many states, including China, Spain etc., have root CAs. These CAs are not just in the country, they are directly parts of the state. That said, that's a broader discussion and we need to secure one point at a time. Just wanted to point this out.

I realized that autoconfig xml files can be used for more than just mailserver hostnames and ports/protocols, I'll look at it [2] in more detail to assess if that opens any new attack vectors. Ben, is [2] up to date? ​https://wiki.mozilla.org/Thunderbird:Autoconfiguration:ConfigFileFormat

It should be. (If not, please tell me.) In some parts it's even ahead of the implementation, for example: The spec considers LDAP configuration for the future, but Thunderbird currently only configures mail accounts this way.

Last edited 4 years ago by ben (previous) (diff)

comment:15 Changed 4 years ago by ben

(If Mozilla ever implements arbitrary DNS lookups, we could do both and compare the two results.)

Actually, I am not sure that would even improve things. If the user is under attack, you'd have a downgrade attack again, so it could make things worse. And you're trusting ISPDB anyway.
It would be better to make a third party verification by another server in another country, which on mismatch alerts some admin that something's fishy, not the end users, who can't do anything about it anyway.

comment:16 Changed 4 years ago by ben

Only accept fetched xml configs that use safe email protocols (SSL/TLS for SMTP/IMAP/POP). Only probe the mail server for safe protocols (SSL/TLS for SMTP/IMAP/POP).

Thunderbird is essentially doing that anyway, by giving a huge red warning when there's no SSL or the SSL is misconfigured.

comment:17 Changed 4 years ago by ben

The prefs [1] are merged [2] upstream.
https://bugzilla.mozilla.org/show_bug.cgi?id=669282
https://hg.mozilla.org/comm-central/rev/12401af31c63

The new prefs are, with upstream defaults:

// Allow to contact ISP (email address domain)
// This happens via insecure means (HTTP), so the config cannot be trusted,
// and also contains the email address
pref("mailnews.auto_config.fetchFromISP.enabled", true);
// Allow the fetch from ISP via HTTP, but not the email address
pref("mailnews.auto_config.fetchFromISP.sendEmailAddress", true);
pref("mailnews.auto_config.guess.enabled", true);

The better fix will be
https://bugzilla.mozilla.org/show_bug.cgi?id=971347
I'm seeking feedback there.

Last edited 4 years ago by ben (previous) (diff)

comment:18 Changed 11 days ago by teor

Severity: Normal

Set all open tickets without a severity to "Normal"

Note: See TracTickets for help on using tickets.