HTTPS Everywhere's SSL Observatory code doesn't honor domain isolation.

added component::https everywhere/eff-https everywhere owner::legind priority::high resolution::fixed severity::major status::closed tbb-linkability type::defect labels

That's because there is no URL bar domain to isolate the request to. There are a few internal ones which are put on the catch-all circuit which gets rotated every ten minutes. Thus, this seems not to be a bug to me.

Trac:
Keywords: N/A deleted, tbb-torbutton added
Component: Applications/Torbutton to Applications/Tor Browser
Status: new to closed
Resolution: N/A to not a bug

If the request used the catch-all circuit, I wouldn't have opened the bug. The catch-all circuit uses domain isolation (so the rotation works). This does not.

Trac:
Status: closed to reopened
Resolution: not a bug to N/A

Okay, then I am confused. Could you provide a log explaining what happened and what should have happened instead?

Trac:
Keywords: N/A deleted, tbb-linkability added

There is no log. The only reason I caught this was because I was dumping the SOCKS request bodies with my sandbox code.

What happens is, the internal check uses a connection to check.torproject.org to validate that tor is working. That request does not send a SOCKS username/password for isolation. If it were using domain isolation correctly, the catchall circuit (Username: ---unknown---) would be used.

The easiest way to reproduce this would probably be using a system tor instance and wireshark.

Trac:
bug20195.pcapng

Example pcap.

Interestingly enough this only happens with the release channel, and not the latest alpha.

So it's probably ok to eventually close as "fixed" as long as someone knows why the behavior changed, especially considering that it shouldn't affect most people.

Replying to yawning:

There is no log. The only reason I caught this was because I was dumping the SOCKS request bodies with my sandbox code.

You mean setting extensions.torbutton.loglevel to 3 and extensions.torbutton.logmethod to 0 does not show this request in your terminal while all the other ones are visible? Is that fixed in the alphas as well?

Trac:
Status: reopened to assigned
Owner: N/A to tbb-team

Ah there. Looks like the domain isolation code is getting called, but wireshark doesn't lie. FWIW, I took the pcaps without my sandboxing stuff in play, and the behavior is consistent.

[09-22 08:31:02] Torbutton INFO: Component returned failure code: 0x80070057 (NS_ERROR_ILLEGAL_VALUE) [mozIThirdPartyUtil.getFirstPartyURIFromChannel]
[09-22 08:31:02] Torbutton INFO: tor SOCKS isolation catchall: https://check.torproject.org/?TorButton=true#0.99726695529027310.5190771246311907 via --unknown--:0
[09-22 08:31:02] Torbutton WARN: no SOCKS credentials found for current document.

Replying to yawning:

Ah there. Looks like the domain isolation code is getting called, but wireshark doesn't lie. FWIW, I took the pcaps without my sandboxing stuff in play, and the behavior is consistent.
[09-22 08:31:02] Torbutton INFO: Component returned failure code: 0x80070057 (NS_ERROR_ILLEGAL_VALUE) [mozIThirdPartyUtil.getFirstPartyURIFromChannel]
[09-22 08:31:02] Torbutton INFO: tor SOCKS isolation catchall: https://check.torproject.org/?TorButton=true#0.99726695529027310.5190771246311907 via --unknown--:0
[09-22 08:31:02] Torbutton WARN: no SOCKS credentials found for current document.
}}}

Alright, so here is what is going on. First, do you see the weird floating point number thing appended to the # in the check.torproject.org URL? Torbutton does not do such things. It turns out this is part if the HTTPS-Everywhere SSL Observatory code where it checks whether Tor is available and to use (e.g. for submissions). As a sidenode: one does see the issue in the Tor Browser log as well without pcaps. It is visible there that the request does not go over the catch-all circuit but rather is put on one without any username/password isolation at all.

So, even if the request comes from HTTPS-Everywhere why is it not isolated like any other internal request? Looking at Necko logs shows that indeed things are wrong, already at the nsHTTPConnectionManager level: {{{ -957356288[7fd9e2dabb60]: nsHttpConnectionMgr::OnMsgSpeculativeConnect [ci=.S....check.torproject.org:443 (socks:127.0.0.1:9150)[:]]

Before and after the colon in the brackets should be the respective username and password.

Looking closer at the SSL Observatory code shows it is bypassing our proxy filter respoonsible for domain isolation in case the CSRF nonce is found in the path:

if (isSubmission || testingForTor) {
  if (aURI.path.search(this.csrf_nonce+"$") != -1) {

    this.log(INFO, "Got observatory url + nonce: "+aURI.spec);
    var proxy_settings = null;
    var proxy = null;

    // Send it through tor by creating an nsIProxy instance
    // for the torbutton proxy settings.
    try {
      proxy_settings = this.getProxySettings(testingForTor);
      proxy = this.pps.newProxyInfo(
        proxy_settings.type,
        proxy_settings.host,
        proxy_settings.port,
        Ci.nsIProxyInfo.TRANSPARENT_PROXY_RESOLVES_HOST,
        0xFFFFFFFF, null);
    } catch(e) {
      this.log(WARN, "Error specifying proxy for observatory: "+e);
    }

    this.log(INFO, "Specifying proxy: "+proxy);

    // TODO: Use new identity or socks u/p to ensure we get a unique
    // tor circuit for this request
    return proxy;
  }
}


FWIW: the reason you thought this was fixed in the alpha was due to HTTPS-Everywhere 5.2.4 that was the latest version at that time. In that version SSL Observatory code was broken due to #19996. This got fixed in 5.2.5 which makes this issue visible in the alphas again.

**Trac**:  
**Component**: Applications/Tor Browser **to** HTTPS Everywhere/EFF-HTTPS Everywhere   
**Summary**: torbutton-torCheckService doesn't honor domain isolation. **to** HTTPS Everywhere's SSL Observatory code doesn't honor domain isolation.   
**Cc**: *N/A* **to** gk   
**Keywords**: tbb-torbutton **deleted**,  *N/A* **added**  
**Owner**: tbb-team **to** legind   
**Severity**: Normal **to** Major   
**Priority**: Medium **to** High

Replying to gk:

Alright, so here is what is going on. First, do you see the weird floating point number thing appended to the # in the check.torproject.org URL? Torbutton does not do such things. It turns out this is part if the HTTPS-Everywhere SSL Observatory code where it checks whether Tor is available and to use (e.g. for submissions). As a sidenode: one does see the issue in the Tor Browser log as well without pcaps. It is visible there that the request does not go over the catch-all circuit but rather is put on one without any username/password isolation at all.

Nice catch.

Is there a ticket for "SSL Observatory makes at least one network request on startup to check proxy settings, even if it's disabled"? If "Use the Observatory?" isn't checked, this request shouldn't be made at all, but as it stands absolutely everyone (with working SSL-Observatory) is hitting this bug.

Trac:
Cc: gk to gk, brade, mcs

Replying to yawning:

Is there a ticket for "SSL Observatory makes at least one network request on startup to check proxy settings, even if it's disabled"? If "Use the Observatory?" isn't checked, this request shouldn't be made at all, but as it stands absolutely everyone (with working SSL-Observatory) is hitting this bug.

Not yet, but I guess a good solution for this one would solve that problem as well.

So the following things could be done:

If you want to check whether Tor is enabled check for an existing Torbutton component. No request getting sent to check.torproject.org is necessary in this scenario. And if such a component is found let Tor Browser handle the traffic (i.e. don't mess with proxy settings) as Torbutton alone should not be functional anymore (i.e. you can be sure the user has a Tor Browser).

That would be sufficient for us. But what if you don't find an existing Torbutton component? Still, I think, there should not be any check if the SSL Observatory is disabled. Not sure, though, if you want to support Firefox users that have a tor running somewhere but are not using Tor Browser.

Replying to gk:

{{{ [09-22 08:31:02] Torbutton WARN: no SOCKS credentials found for current document. }}} TBB has a lot of places with this warning, e.g. while fetching RecommendedTBBVersions, so what? Alright, so here is what is going on. First, do you see the weird floating point number thing appended to the # in the check.torproject.org URL? FP with two dots? He-he. Torbutton does not do such things. But it looked like yours :) It is visible there that the request does not go over the catch-all circuit but rather is put on one without any username/password isolation at all. If getinfo circuit-status doesn't lie, the request does go over the catch-all circuit, even though without any username/password isolation at all.

This is another one recent crap from HTTPSE: it look like it was developed as a virus or without security audit at all. Is it suitable for TBB?

(Also it is doing 3 requests in a row to check.torproject.org, on NEWNYM too.)

The suggestion in https://trac.torproject.org/projects/tor/ticket/20195#comment:13 has been implemented in https://github.com/EFForg/https-everywhere/pull/7342:

This resolves the issue in https://trac.torproject.org/projects/tor/ticket/20195 where the SSL Observatory proxy checking code and submissions were bypassing domain isolation. That code was a relic from the TorButton days.

Now, check.torproject.org is no longer accessed when we're using Tor Browser, we assume successful Tor access. In this case, we let TB transparently proxy for us, instead of accessing the Tor Browser proxy settings directly.

This can be tested within HTTPS Everywhere by running:

test/tor-browser.sh PATH_TO_TOR_ARCHIVE

I'll close this once the fix is merged on our side.

Trac:
Status: assigned to closed
Resolution: N/A to fixed

And what to do with all other Torbutton WARN: no SOCKS credentials found for current document.?

closed

mentioned in issue #23718 (moved)

mentioned in issue #28075 (moved)

moved to tpo/applications/https-everywhere-eff#20195 (closed)

mentioned in issue tpo/applications/tor-browser#28075 (closed)

HTTPS Everywhere's SSL Observatory code doesn't honor domain isolation.

Child items ...

Activity