Opened 4 years ago

Closed 12 months ago

#15933 closed defect (fixed)

Relax domain isolation to use TLD instead of FQDN

Reported by: maxim Owned by: tbb-team
Priority: Medium Milestone:
Component: Applications/Tor Browser Version:
Severity: Normal Keywords: tbb-torbutton, tbb-usability-website, tbb-4.5-regression, TorBrowserTeam201505R
Cc: brade, mcs Actual Points:
Parent ID: Points:
Reviewer: Sponsor:

Description

TorButton 1.9.2.2 in new TorBrowser 4.5, prevents some file host sites from functioning, such as load.to, ziifile.com, and others.

example link: http://www.load.to/StQUxNkHH4/dictionaries.7z

this download link will function in TBB 4.0.8 (with <Forbid Scripts Globally> activated)

in TBB 4.5, the same link will only loop back to the file host home page (with <Forbid Scripts Globally> activated). the download link *will* function, however, if TorButton extension is disabled.

maybe caused by the isolation of requests from same URL domain?? (file hosts frequently redirect user after the Download button is clicked. in this example, from http://www.load.to/ to http://s2.load.to)

thanks for attention

Child Tickets

Change History (21)

comment:1 Changed 4 years ago by gk

Component: TorbuttonTor Browser
Keywords: tbb-torbutton tbb-usability tbb-4.5-regression added
Owner: set to tbb-team

comment:2 in reply to:  description ; Changed 4 years ago by gk

Replying to maxim:

maybe caused by the isolation of requests from same URL domain??

I don't think so. At least it is working fine with JavaScript enabled in 4.5. How do I get the download to work without JS enabled in 4.0.8? Do you have steps for me to follow?

comment:3 in reply to:  2 ; Changed 4 years ago by maxim

Replying to gk:

I don't think so. At least it is working fine with JavaScript enabled in 4.5. How do I get the download to work without JS enabled in 4.0.8? Do you have steps for me to follow?

to download without JS enabled in 4.0.8, one can visit the link in New Private Window / Private Browsing Mode.

comment:4 in reply to:  3 Changed 4 years ago by maxim

a clearer example, with a different file host:
http://ziifile.com/2jlqd5wbfdxm/dictionaries.7z.html

attempting to download from this site in 4.5 (JS disabled) gives the result "File Not Found".

but file can be downloaded with no problems (same settings, JS disabled) in 4.0.8.

there is something in new 4.5 torbutton that blocks file hosts from generating the requisite download links.

comment:5 Changed 4 years ago by mikeperry

Keywords: tbb-usability-website added; tbb-usability removed

comment:6 Changed 4 years ago by gk

Summary: TorButton 1.9.2.2 breaks File Host sitesCircuit Isolation in Tor Browser 4.5 breaks File Host sites

What is happening in this particular case is the following:

1) User clicks on the download button which causes a POST request sent (POST http://ziifile.com/2jlqd5wbfdxm/dictionaries.7z.html) causing a 302.
2) The Location header then has something like http://rebeka.ziifile.com/files/0/vszizsp06hfvlw/dictionaries.7z as value
3) This causes a new request which goes over a different circuit as the FQDN is different.
4) The file hoster gets a different IP address shown (with very high probability) which he can't associate with a former POST request which causes it to fail.

This was no issue in 4.0.8 as we played the 10-minute-circuit game then.

That reminds me at the Referer faking issues back then breaking sites not taking subdomains into account... What exactly would we lose if we take the URL bar base domain instead of the FQDN? I wonder if that would fix all cases at all (looking at https://blog.torproject.org/blog/tor-browser-45-released#comment-92912 I doubt that).

Another strategy which might work better (but is probably way harder to get right) is seeing this behavior as kind of a user-driven redirect which would boil down to #3600. Hrm...

comment:7 Changed 4 years ago by mikeperry

I have been debating if we should use the functionality behind https://developer.mozilla.org/en-US/docs/Mozilla/Preferences/Preference_reference/browser.urlbar.formatting.enabled for all of our isolation. It should be relatively easy to change this. I think all we have to do is modify the getFirstPartyURI API family to use the browser urlbar formatting code to extract a domain for us...

comment:8 Changed 4 years ago by mikeperry

It looks like almost every use of ThirdPartyUtil::GetFirstParty* ultimately converts the URI to a hostname using ThirdPartyUtil::GetFirstPartyHostForIsolation().

I only noticed two cases where we don't do this: In the HTTP auth stripping check, and in the canvas permissions.

This means we should be able to make a simple patch to add TLD isolation in ThirdPartyUtil::GetFirstPartyHostForIsolation() using Mozilla's existing urlbar formatting logic. I will see if I can produce a simple patch for this today.

comment:9 Changed 4 years ago by mikeperry

Keywords: TorBrowserTeam201505R added
Status: newneeds_review

Ah, there already is a routine in ThirdPartyUtil for this. Instead of getting the host from the url in ThirdPartyUtil::GetFirstPartyHostForIsolation(), we just use ThirdPartyUtil::GetBaseDomain() instead. This is a one-line patch.

https://gitweb.torproject.org/user/mikeperry/tor-browser.git/commit/?h=bug15933

Last edited 4 years ago by mikeperry (previous) (diff)

comment:10 Changed 4 years ago by mcs

Cc: brade mcs added

comment:11 Changed 4 years ago by mikeperry

Summary: Circuit Isolation in Tor Browser 4.5 breaks File Host sitesRelax domain isolation to use TLD instead of FQDN

Merged for 4.5.1. (Retitling because this had been annoying me on more than just file hosting sites, and because we fixed this by changing all domain isolation properties).

comment:12 Changed 4 years ago by mikeperry

Resolution: fixed
Status: needs_reviewclosed

comment:13 Changed 4 years ago by maxim

thanks to all the respondents and the tbb-team in general.
you are awesome.

comment:14 Changed 4 years ago by cypherpunks

The fix doesn't help with this hoster:

http://lumload.com/vzth6m53lwa0

Tested with Tor Browser 4.5.1-build1: after solving the captcha, I got a download page http://l2.lumload.com:182/d/zaoqtg3g4dnnofoc5r5djliistfo6e2omqdecblg7j7ghqshhk5ikio7/dictionaries.7z with an error message "Wrong IP". Maybe because of the different port?

... requested new identity, repeated the process - and now it works.

Last edited 4 years ago by cypherpunks (previous) (diff)

comment:15 Changed 4 years ago by cypherpunks63

Find any active link to lumload anywhere. (Problem not restricted to that site.)

Doesn't work on 4.5.2 or 4.5.1. Works on 4.5 sometimes, usually after creating new circuit.

Even if it is because of the port, this should be a toggle.

comment:16 Changed 4 years ago by cypherpunks63

Resolution: fixed
Status: closedreopened

comment:17 Changed 4 years ago by gk

Resolution: fixed
Status: reopenedclosed

Please file a new bug with steps to reproduce. *Relaxing* the domain isolation should not be the issue of your problem.

comment:18 Changed 4 years ago by cypherpunks63

Not isolating ports of a domain would be relaxing it, no?

lumload.com/zi9eiamfbz3x

comment:19 Changed 12 months ago by blagarg

Resolution: fixed
Status: closedreopened

This bug seems to be back as of v8.0.

Example:

  1. Visit https://clicknupload.org/hg5f8qew62hm
  1. Create Download Link - Will create a link to something like https://fs21.clicknupload.net/files/0/-hash-/test.txt
  1. Tor circuit to "fs21.clicknupload.net" will be different than "clicknupload.org" and download will fail because our IP is not the same.

I will try with v7.5.6 next, but I remember that this worked with the release I had before.

comment:20 Changed 12 months ago by blagarg

Getting the same behavior in v7.5.6. The issue might be further back, will investigate.

comment:21 Changed 12 months ago by gk

Resolution: fixed
Severity: Normal
Status: reopenedclosed

That's not this bug as this one is talking about issues between bar.foo.com and foo.com, not about those between foo.com and foo.net. Please open a new bug for your case. However, solving that one will be pretty tricky.

Note: See TracTickets for help on using tickets.