Skip to content
Snippets Groups Projects
New look: Off

Disable Speculative Connect and Download

    • Closed (moved) Issue created by Matthew Finkel @sysrqb

    As a follow up to #25770 (moved), it turns out Tor Browser actually begins (and sometimes completes) the download before the user can confirm they actually want the-thing. This manifests in #7449 (moved) and #11254 (moved) as the file being downloaded in /tmp (on Unix, or similar on other platforms) and then it is moved into the correct directory.

    This provides an obvious performance improvement, especially for our use case. Unfortunately, this download begins before the user is even given the "where do you want this file saved" prompt, and I'm worried subtly downloading a file in the background is dangerous is some context, particularly some of our users.

    To upload designs, you'll need to enable LFS and have an admin enable hashed storage. More information

    Child items ...

    • Activity

    • Matthew Finkel added component::applications/tor browser owner::tbb-team priority::medium resolution::not a bug severity::normal status::closed type::defect labels

      added component::applications/tor browser owner::tbb-team priority::medium resolution::not a bug severity::normal status::closed type::defect labels

    • S
      Matthew Finkel @sysrqb ·
      Author

      For some context:

      2018-04-11 02:11:26.262762 UTC - [Main Thread]: V/nsHttp nsHttpConnectionMgr::SpeculativeConnect [ci=.S.P...www.torproject.org:443 (socks:127.0.0.1:9150)[--unknown--:<E3>3'<D4>?^M<BD>L^Y[ET^T'֯5^Y<F0>M^G(<B2>sI
<9D><C1><EA><CA>4]^privateBrowsingId=1]
2018-04-11 02:11:26.262841 UTC - [Main Thread]: D/nsHttp NulHttpTransaction::NullHttpTransaction() mActivityDistributor is active [this=7c2ddc744880, www.torproject.org]
2018-04-11 02:11:26.263023 UTC - [Socket Thread]: V/nsHttp nsHttpConnectionMgr::OnMsgSpeculativeConnect [ci=.S.P...www.torproject.org:443 (socks:127.0.0.1:9150)[--unknown--:<E3>3'<D4>?^M<BD>L^Y[ET^T'֯5^Y<F0>M^G(
<B2>sI<9D><C1><EA><CA>4]^privateBrowsingId=1]
2018-04-11 02:11:26.263036 UTC - [Socket Thread]: V/nsHttp Init nsHttpConnectionInfo @7c2ddb5bf2c0
2018-04-11 02:11:26.263088 UTC - [Socket Thread]: V/nsHttp nsHttpConnectionMgr::AtActiveConnectionLimit [ci=.S.P...www.torproject.org:443 (socks:127.0.0.1:9150)[--unknown--:<E3>3'<D4>?^M<BD>L^Y[ET^T'֯5^Y<F0>M^G(
<B2>sI<9D><C1><EA><CA>4]^privateBrowsingId=1 caps=401]
2018-04-11 02:11:26.263120 UTC - [Socket Thread]: V/nsHttp Creating nsHalfOpenSocket [this=7c2ddcb5bb60 trans=7c2ddc744880 ent=www.torproject.org key=.S.P...www.torproject.org:443 (socks:127.0.0.1:9150)[--unknown--:<E3>3'<D4>?^M<BD>L^Y[ET^T'֯5^Y<F0>M^G(<B2>sI<9D><C1><EA><CA>4]^privateBrowsingId=1]
2018-04-11 02:11:26.263138 UTC - [Socket Thread]: V/nsHttp nsHalfOpenSocket::SetupStreams [this=7c2ddcb5bb60 ent=.S.P...www.torproject.org:443 (socks:127.0.0.1:9150)[--unknown--:<E3>3'<D4>?^M<BD>L^Y[ET^T'֯5^Y
<F0>M^G(<B2>sI<9D><C1><EA><CA>4]^privateBrowsingId=1] setup routed transport to origin www.torproject.org:443 via :443
2018-04-11 02:11:26.263150 UTC - [Socket Thread]: D/nsSocketTransport creating nsSocketTransport @7c2ddd26c000
2018-04-11 02:11:26.263162 UTC - [Socket Thread]: D/nsSocketTransport nsSocketTransport::Init [this=7c2ddd26c000 host=www.torproject.org:443 origin=www.torproject.org:443 proxy=127.0.0.1:9150]
2018-04-11 02:11:26.263184 UTC - [Socket Thread]: D/nsSocketTransport Reset callbacks for secinfo=0 callbacks=7c2df4593dc0
2018-04-11 02:11:26.263196 UTC - [Socket Thread]: D/nsSocketTransport nsSocketTransport::OpenOutputStream [this=7c2ddd26c000 flags=2]
2018-04-11 02:11:26.263206 UTC - [Socket Thread]: D/nsSocketTransport nsSocketTransport::PostEvent [this=7c2ddd26c000 type=0 status=0 param=0]
2018-04-11 02:11:26.263219 UTC - [Socket Thread]: D/nsSocketTransport STS dispatch [7c2df4593ee0]
2018-04-11 02:11:26.263236 UTC - [Socket Thread]: D/nsSocketTransport OnDispatchedEvent Same Thread Skip Signal
2018-04-11 02:11:26.263246 UTC - [Socket Thread]: D/nsSocketTransport nsSocketTransport::OpenInputStream [this=7c2ddd26c000 flags=2]
2018-04-11 02:11:26.263252 UTC - [Main Thread]: D/nsHttp nsHttpChannel::OnCacheEntryAvailable [this=7c2ddc0c1800 entry=7c2ddc60aaa0 new=1 appcache=0 status=0 mAppCache=0 mAppCacheForWrite=0]
2018-04-11 02:11:26.263256 UTC - [Socket Thread]: D/nsSocketTransport nsSocketTransport::PostEvent [this=7c2ddd26c000 type=0 status=0 param=0]
2018-04-11 02:11:26.263311 UTC - [Socket Thread]: D/nsSocketTransport STS dispatch [7c2df4593f70]
2018-04-11 02:11:26.263291 UTC - [Main Thread]: D/nsHttp nsHttpChannel::SetupTransaction [this=7c2ddc0c1800]
2018-04-11 02:11:26.263325 UTC - [Socket Thread]: D/nsSocketTransport OnDispatchedEvent Same Thread Skip Signal
2018-04-11 02:11:26.263360 UTC - [Main Thread]: D/nsHttp Creating nsHttpTransaction @7c2ddd26c400
2018-04-11 02:11:26.263397 UTC - [Main Thread]: D/nsHttp nsHttpChannel 7c2ddc0c1800 created nsHttpTransaction 7c2ddd26c400
2018-04-11 02:11:26.263377 UTC - [Socket Thread]: D/nsSocketTransport nsSocketOutputStream::AsyncWait [this=7c2ddd26c288]
2018-04-11 02:11:26.263563 UTC - [Socket Thread]: D/nsSocketTransport OnDispatchedEvent Same Thread Skip Signal
2018-04-11 02:11:26.263577 UTC - [Socket Thread]: D/nsSocketTransport nsSocketTransport::OnSocketEvent [this=7c2ddd26c000 type=0 status=0 param=0]
2018-04-11 02:11:26.263586 UTC - [Socket Thread]: D/nsSocketTransport   MSG_ENSURE_CONNECT
2018-04-11 02:11:26.263636 UTC - [Socket Thread]: D/nsSocketTransport nsSocketTransport::ResolveHost [this=7c2ddd26c000 www.torproject.org:443]
2018-04-11 02:11:26.263647 UTC - [Socket Thread]: D/nsSocketTransport nsSocketTransport::PostEvent [this=7c2ddd26c000 type=1 status=0 param=0]
2018-04-11 02:11:26.263658 UTC - [Socket Thread]: D/nsSocketTransport STS dispatch [7c2df4593d90]
2018-04-11 02:11:26.263668 UTC - [Socket Thread]: D/nsSocketTransport OnDispatchedEvent Same Thread Skip Signal
2018-04-11 02:11:26.263685 UTC - [Socket Thread]: D/nsSocketTransport nsSocketTransport::OnSocketEvent [this=7c2ddd26c000 type=0 status=0 param=0]
2018-04-11 02:11:26.263694 UTC - [Socket Thread]: D/nsSocketTransport   MSG_ENSURE_CONNECT
2018-04-11 02:11:26.263713 UTC - [Socket Thread]: D/nsSocketTransport   ignoring redundant event
2018-04-11 02:11:26.263695 UTC - [Main Thread]: D/nsHttp nsHttpTransaction::Init [this=7c2ddd26c400 caps=b]
2018-04-11 02:11:26.263744 UTC - [Main Thread]: D/nsHttp nsHttpTransaction::Init() mActivityDistributor is active this=7c2ddd26c400
2018-04-11 02:11:26.263727 UTC - [Socket Thread]: D/nsSocketTransport STS poll iter
2018-04-11 02:11:26.263776 UTC - [Socket Thread]: D/nsSocketTransport   active [2] { handler=7c2dfb170000 condition=0 pollflags=5 }
2018-04-11 02:11:26.263784 UTC - [Main Thread]: I/nsHttp http request [
2018-04-11 02:11:26.263801 UTC - [Main Thread]: I/nsHttp   GET /dist/torbrowser/7.5.3/torbrowser-install-7.5.3_en-US.exe HTTP/1.1
2018-04-11 02:11:26.263819 UTC - [Main Thread]: I/nsHttp   Host: www.torproject.org
2018-04-11 02:11:26.263834 UTC - [Main Thread]: I/nsHttp   User-Agent: Mozilla/5.0 (Windows NT 6.1; rv:52.0) Gecko/20100101 Firefox/52.0
2018-04-11 02:11:26.263849 UTC - [Main Thread]: I/nsHttp   Accept: text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8
2018-04-11 02:11:26.263863 UTC - [Main Thread]: I/nsHttp   Accept-Language: en-US,en;q=0.5
2018-04-11 02:11:26.263878 UTC - [Main Thread]: I/nsHttp   Accept-Encoding: gzip, deflate, br
2018-04-11 02:11:26.263893 UTC - [Main Thread]: I/nsHttp   Referer: https://www.torproject.org/download/download-easy.html.en
2018-04-11 02:11:26.263907 UTC - [Main Thread]: I/nsHttp   Connection: keep-alive
2018-04-11 02:11:26.263921 UTC - [Main Thread]: I/nsHttp   Pragma: no-cache
2018-04-11 02:11:26.263936 UTC - [Main Thread]: I/nsHttp   Cache-Control: no-cache
2018-04-11 02:11:26.263949 UTC - [Main Thread]: I/nsHttp ]

      netwerk/protocol/http/nsHttpChannel.cpp and netwerk/protocol/http/nsHttpConnectionMgr.cpp are interesting here.

    • S
      Matthew Finkel @sysrqb ·
      Author

      I should also mention that because this download occurs speculatively, it is not bound to/associated with a firstPartyURI, so if the server requires a cookie (or the same originating-ip-address, or some other auth) obtained from another context/tab/domain-isolation, then this download will fail in a confusing way for the user.

    • Georg Koppen
      Georg Koppen @gk ·

      Replying to sysrqb:

      I should also mention that because this download occurs speculatively, it is not bound to/associated with a firstPartyURI, so if the server requires a cookie (or the same originating-ip-address, or some other auth) obtained from another context/tab/domain-isolation, then this download will fail in a confusing way for the user.

      I think I don't understand this comment. Do you have an example where I can see this scenario live?

    • Georg Koppen
      Georg Koppen @gk ·

      Replying to sysrqb:

      As a follow up to #25770 (moved), it turns out Tor Browser actually begins (and sometimes completes) the download before the user can confirm they actually want the-thing. This manifests in #7449 (moved) and #11254 (moved) as the file being downloaded in /tmp (on Unix, or similar on other platforms) and then it is moved into the correct directory.

      I think there are two things that we need to keep separate:

      1. The download starting early

      2. Using /tmp for saving parts of the download before it is finally transferred to the location it should be in.

      3. Is a bug and you cited some tickets for it. I am not sure whether 1) is a bug, though: the users clicked on the resource indicating that they want to have it, no?

    • Georg Koppen
      Georg Koppen @gk ·

      Or maybe there is even a third thing involved: 3) The external helper app dialog is implying the download has not started yet while it indeed has (see: comment:2:ticket:18587). So, I agree with that one being confusing and we should come up with a better wording for what is happening. Historically, it has been a pain getting the message on that dialog right but I bet we can improve it further.

    • Georg Koppen
      Georg Koppen @gk ·

      One final remark for now: "speculative connections" you mentioned here have nothing to do with "speculative connections" mentioned in our design doc (see: "20. Speculative and prefetched connections" https://www.torproject.org/projects/torbrowser/design/#identifier-linkability), just to avoid confusion of terminology. For the former, see: https://bugzilla.mozilla.org/show_bug.cgi?id=729133.

    • Georg Koppen
      Georg Koppen @gk ·

      The final, final one now: see https://bugzilla.mozilla.org/show_bug.cgi?id=55690 for arguments why this got implemented the way it is.

    • S
      Matthew Finkel @sysrqb ·
      Author

      Replying to gk:

      Replying to sysrqb:

      As a follow up to #25770 (moved), it turns out Tor Browser actually begins (and sometimes completes) the download before the user can confirm they actually want the-thing. This manifests in #7449 (moved) and #11254 (moved) as the file being downloaded in /tmp (on Unix, or similar on other platforms) and then it is moved into the correct directory.

      I think there are two things that we need to keep separate:

      1. The download starting early

      2. Using /tmp for saving parts of the download before it is finally transferred to the location it should be in.

      3. Is a bug and you cited some tickets for it. I am not sure whether 1) is a bug, though: the users clicked on the resource indicating that they want to have it, no?

      I agree this isn't strictly a bug, but I classify this as a Firefox feature that we don't want. I understand why Firefox begins downloading the file when a user clicks on the link/button. However, if a users opens the context menu of a link and selects "Save Link As...", then I do not believe users expect Firefox begins downloading the file until they select the destination directory/filename and click "Save".

      The issue with saving files in /tmp is a complete mess, and after reading that moz bug I don't know what we should do when a user clicks a link and Firefox cannot handle it internally (opening a pdf using pdfjs, show a plaintext file directly, etc). We should leave this question for the other open /tmp tickets.

      Replying to gk:

      Or maybe there is even a third thing involved: 3) The external helper app dialog is implying the download has not started yet while it indeed has (see: comment:2:ticket:18587). So, I agree with that one being confusing and we should come up with a better wording for what is happening. Historically, it has been a pain getting the message on that dialog right but I bet we can improve it further.

      I originally thought this was worse because the downloading begins before the "External App Blocker" is shown, but after I thought about this more I didn't think the timing makes a difference. That warning/popup is specifically about opening the file, it doesn't say anything about downloading. Considering previous pain related to this, I expect showing the message earlier will not be easy.

      Replying to gk:

      One final remark for now: "speculative connections" you mentioned here have nothing to do with "speculative connections" mentioned in our design doc (see: "20. Speculative and prefetched connections" https://www.torproject.org/projects/torbrowser/design/#identifier-linkability), just to avoid confusion of terminology. For the former, see: https://bugzilla.mozilla.org/show_bug.cgi?id=729133.

      Oh, interesting, I didn't remember this section. Indeed, these are two different features, but maybe they should be treated the same (do not open speculative connections when a proxy is configured). (Thanks for clarifying that difference before anyone else became confused)

      Replying to gk:

      The final, final one now: see https://bugzilla.mozilla.org/show_bug.cgi?id=55690 for arguments why this got implemented the way it is.

      Right, Mozilla have a couple long bugs and reading them is painful - and they're filled with comments about modem lights blinking, T-DSL, and 500MB exceeding available memory.

      The reason I opened this bug is because Tor Browser should not begin downloading a file unless the user explicitly confirms they want the file downloaded and where they want it saved. If clicking "Save Link As..." starts any network transactions before the users clicks "Save" within the file-chooser dialog box, then Tor Browser should make this obvious. I believe Tor Browser should treat the two scenarios differently:

      1. I click on the link and Firefox doesn't know what to do, so it asked me where to save the file
      2. I click on "Save Link As..." and specify where I want the file saved (or I click cancel)

      Within scenario (1), Firefox cannot know what it should do without beginning the download. That's okay. With scenario (2), that is completely against what the user requested. This is almost certainly a Firefox bug, unfortunately it seems Firefox handles (1) and (2) using the same logic.

    • S
      Matthew Finkel @sysrqb ·
      Author

      Replying to sysrqb:

      Within scenario (1), Firefox cannot know what it should do without beginning the download. That's okay. With scenario (2), that is completely against what the user requested. This is almost certainly a Firefox bug, unfortunately it seems Firefox handles (1) and (2) using the same logic.

      Hrm, and this is related to #22649 (moved) (coming from #25672 (moved)). But, to clarify, it may be that #22649 (moved) is resolved if "Save Link As..." does not use a speculative connection. If it uses a connection created within the context of the current first party domain, then maybe the download will use the correct circuit.

    • S
      Matthew Finkel @sysrqb ·
      Author

      Replying to sysrqb:

      Within scenario (1), Firefox cannot know what it should do without beginning the download. That's okay. With scenario (2), that is completely against what the user requested. This is almost certainly a Firefox bug, unfortunately it seems Firefox handles (1) and (2) using the same logic.

      Of course, Firefox wants to handle (2) in a smart way:

          // an object to proxy the data through to
    // nsIExternalHelperAppService.doContent, which will wait for the
    // appropriate MIME-type headers and then prompt the user with a
    // file picker

      https://gitweb.torproject.org/tor-browser.git/tree/browser/base/content/nsContextMenu.js?h=tor-browser-52.7.3esr-7.5-1-build1#n1265

      I wonder if we can simply use the timeout case instead, and skip the async fetch (?):

            onStopRequest: function saveLinkAs_onStopRequest(aRequest, aContext,
                                                       aStatusCode) {
        if (aStatusCode == NS_ERROR_SAVE_LINK_AS_TIMEOUT) {
          // do it the old fashioned way, which will pick the best filename
          // it can without waiting.
          saveURL(linkURL, linkText, dialogTitle, bypassCache, false, docURI,
                  doc);
        }

      https://gitweb.torproject.org/tor-browser.git/tree/browser/base/content/nsContextMenu.js?h=tor-browser-52.7.3esr-7.5-1-build1#n1319

    • Georg Koppen
      Georg Koppen @gk ·

      Replying to sysrqb:

      The reason I opened this bug is because Tor Browser should not begin downloading a file unless the user explicitly confirms they want the file downloaded and where they want it saved. If clicking "Save Link As..." starts any network transactions before the users clicks "Save" within the file-chooser dialog box, then Tor Browser should make this obvious. I believe Tor Browser should treat the two scenarios differently:

      1. I click on the link and Firefox doesn't know what to do, so it asked me where to save the file
      2. I click on "Save Link As..." and specify where I want the file saved (or I click cancel)

      Within scenario (1), Firefox cannot know what it should do without beginning the download. That's okay. With scenario (2), that is completely against what the user requested. This is almost certainly a Firefox bug, unfortunately it seems Firefox handles (1) and (2) using the same logic.

      Why is (2) against what the user requested? The user said: "I want to have that thing AND want to specify where it will be after the download finished." And they get both. Note that the "Save Link" part already means "Yes, I want to have it!1!!" (unrelated to the "As") even if they might not know yet where to put the result or how to name it, or want to select the place themselves with a GUI . Thus, starting the download in the meantime while they are selecting the place where it should land sounds reasonable to me. Sure, if they decided "Nah, I actually don't really want it" (i.e. clicking the Cancel button) it gets discarded, no damage done.

    • S
      Matthew Finkel @sysrqb ·
      Author

      Replying to gk:

      Replying to sysrqb:

      The reason I opened this bug is because Tor Browser should not begin downloading a file unless the user explicitly confirms they want the file downloaded and where they want it saved. If clicking "Save Link As..." starts any network transactions before the users clicks "Save" within the file-chooser dialog box, then Tor Browser should make this obvious. I believe Tor Browser should treat the two scenarios differently:

      1. I click on the link and Firefox doesn't know what to do, so it asked me where to save the file
      2. I click on "Save Link As..." and specify where I want the file saved (or I click cancel)

      Within scenario (1), Firefox cannot know what it should do without beginning the download. That's okay. With scenario (2), that is completely against what the user requested. This is almost certainly a Firefox bug, unfortunately it seems Firefox handles (1) and (2) using the same logic.

      Why is (2) against what the user requested? The user said: "I want to have that thing AND want to specify where it will be after the download finished." And they get both.

      Yes, except as a user, I do not confirm this is the action I want until I click the "Save" button on the file selector. Downloading the file before I click "Save" is preemptive and it is an optimization.

      Note that the "Save Link" part already means "Yes, I want to have it!1!!" (unrelated to the "As") even if they might not know yet where to put the result or how to name it, or want to select the place themselves with a GUI . Thus, starting the download in the meantime while they are selecting the place where it should land sounds reasonable to me. Sure, if they decided "Nah, I actually don't really want it" (i.e. clicking the Cancel button) it gets discarded, no damage done.

      I understand why this design is useful - especially for slow connections (and therefore it is useful for many Tor users). I'm worried this breaks the [of least surprise] and it creates a network call the user may not want - and in some situations it could put a user at risk, I think. I am not saying this is a bad feature, but I did not expect Firefox would begin the downloading process before I clicked "Save". From the Browser's perspective, I agree this behavior makes sense - as you described.

      Trac:
      Resolution: N/A to not a bug
      Status: new to closed

    • Trac closed

      closed

    • teor mentioned in issue #27073 (moved)

      mentioned in issue #27073 (moved)

    • teor mentioned in issue #28096 (moved)

      mentioned in issue #28096 (moved)

    • Mike Perry mentioned in issue #29786 (moved)

      mentioned in issue #29786 (moved)

    • Trac mentioned in issue tpo/core/tor#29786 (closed)

      mentioned in issue tpo/core/tor#29786 (closed)

    • Trac moved to tpo/applications/tor-browser#25773 (closed)

      moved to tpo/applications/tor-browser#25773 (closed)

    Please register or sign in to reply