1.3.x: RefSpoofer fails on 5 test cases out of 12.
I conducted a bunch of test on the new refSpoofer feature from version 1.3.0alpha. Here are the result, in 4 situations for each of the 3 modes.
| A - nospoof | B - smartspoof | C - spoofblank | ||
|---|---|---|---|---|
| 1 | one.domain.tld/a -> one.domain.tld/b | OK - sent | OK - sent | OK - not sent |
| 2 | domain.tld -> one.domain.tld | BAD! - not sent | BAD? - sent one.domain.tld | OK - not sent |
| 3 | domain.tld -> www.domain.tld | BAD! - not sent | BAD! - not sent | OK - not sent |
| 4 | google.com -> one.domain.tld | BAD! - not sent | OK - not sent | OK - not sent |
As you can see :
- it is not leaking HTTP Referers when it shouldn't, except in case (B2) but it was not clear from the comments in the source code whether it should send it or not. I would say it should not.
- the smartspoof mode works in the two most obvious cases (1) and (4) but the two cases (2) and (3) have to be better specified.
- the nospoof fails is a non-ambiguous case where the user configure it to send Referers between different domains.
Activity
changed milestone to %Torbutton: 1.3
HI TAILS!!!!!!!!!!!!
Try BeeFREE!!!!!!!! my addon!!!!!!!!! Bee FREE is able also to forge referrers ("softly" keeping the subdomains, or "strictly", keeping only the top domains!!!!), but in its default configuration it works removing the referrers!!!!!!!!!!! You've got to use this key to change the way it works!!!!!!!!!!!!!! http://honeybeenet.altervista.org/beefree/?id=114000#extensions.beefree.websites.default.header.referer.action You can change it from the GUI too!!!!!!!!!!!!!!!!!!!! Ha!!!! A forged referrer being send is always the hostname of the target host you're sending the HTTP request to, so even this way of working won't let leak information!!!!!!!!!!!!!! Filters of beefree can be configured to remove the referrers even when you're surfing between the pages of the same filtered website!!!!!! you've to look at the keys named "*.header.referer.action.boost" to change this!!! this is also a per-filter based configuration and there is no GUI for it!!!!!!! for example you've to create this integer key "extensions.beefree.website.generic.header.referer.action.boost" and set it to "1" if you want to enable this feature everywhere!!!!!!! so all referrers will be removed (or "FORGED" if you want!!!!!!!) every time on all pages even if hosted under the same "domain.name"!!!!!!!!!!! YEAH!!!!!!!!!!
bye!!!!!!!!!!!! ~bee!!!!!!
Trac:
Username: bee
Based on my reading of this and my informal opinion of how this feature should work, I'm going to declare that case B2 and B3 are bugs.
I think it was Kory's intention to implement a superset of the same-origin policy here, not the same-origin policy itself. I think for the goal of making sure sites decide to render properly, case B3 should send a referrer, as should B2.
However, I would also suspect that case B2 should send the proper referrer. If the TAILS developers report correctly, it is sending the destination site in the referrer, and not the origin. Is this correct?
If so, that is also a bug. B2 should send the source site as the referrer.
-
Testing some links here for the referer:
https://blog.torproject.org/blog/tips-running-exit-node-minimal-harassment
Trac:
Actualpoints: N/A to N/A
Actualpointsdone: N/A to N/A
Pointsdone: N/A to N/A Hi Mike,
When I reported the bug I was probably using an older version of Firefox, now I just tried again with version shipped in Debian squeeze, 3.5.16-4 and torbutton 1.3.1-alpha and as far as B column is concerned I get the same results.
In case B3, going from http://domain.tld/something.html to http://www.domain.tld/ I get :
domain.tld:80 xx.xx.xx.xx - - [15/Feb/2011:12:19:33 +0100] "GET / HTTP/1.1" 304 - "-" "Mozilla/5.0 (Windows; U; Windows NT 6.1; en-US; rv:1.9.2.3) Gecko/20100401 Firefox/3.6.3"
So no referrer are sent when the spec says it should send it. And I can agree with that.
In case B2, going from http://domain.tld/something.html to http//host.domain.tld/index.html I get :
host.domain.tld:80 xx.xx.xx.xx - - [15/Feb/2011:12:22:32 +0100] "GET /index.html HTTP/1.1" 304 - "http://host.domain.tld" "Mozilla/5.0 (Windows; U; Windows NT 6.1; en-US; rv:1.9.2.3) Gecko/20100401 Firefox/3.6.3"
So a referrer from the destination host.domain.tld is sent, which is pretty weird indeed.
The spec is unclear but in my opinion it should not send any referrer in this case. I understand the choice made in the spec for base B3 since we can usually assume that domain.tld and www.domain.tld are run by the same entity. But in the more general case of different subdomains we should assume that it is not the case: they can be run by different entities, the DNS can point to different IPs, the admins and owners of the CMSes can be different. Thinking about two different wordpress.com blogs for example one.wordpress.com and two.wordpress.com, I might not want the admin of two.wordpress know that I'm coming from one.wordpress.com to visit her blog.
I would be glad to help you more debugging this. What extra information can I send you ?
-
For your first example here, 'www' is a special case in the code for some reason. I think it should have sent some kind of referrer for this case though.. That may be a bug due to the special case of 'www'. I think that we should not special case 'www'. If we have to, I think we're probably doing something wrong.
'Not sending the referrer' in the case of the smart referrer code means it spoofs to the origin domain of the destination URL. Hence that is why you see the destination url in the second case. This makes intuitive sense to me, but it should be consistent.
I think this means that there are three steps to the solution of this general problem:
- Make the behaviour consistent wrt subdomains, with no special cases.
- Document all this in the design doc.
- Verify for sanity, adjusting the code as needed.
I think that solving this bug definitely means doing #1 here. I am thinking that 2 and 3 might be separate tickets.
- About subdomains :
Ok, first we could recall why in the first place we're spoofing the referrer. For me we're spoofing to hide to the admins of site foo.tld (on the Apache or the CMS) that I was coming from site bar.tld to visit them, right? Hiding that from people sniffing my traffic while going from a private HTTPS site to a public HTTP site would is important too.
So why trying to do smartspoof across subdomains we should first ask ourselves: can we expect the admins of some.thing.tld to be the same as the admins of some.thing.else.tld and if the answer is « no » then spoof it.
Since I fear there are no simple answer to this question we should base our reasoning on what's being done on the Internet, plus take some extra precautions.
And I think it is wrong to assume that the admins of one.domain.tld are the same as the admins of two.domain.tld. Because, as I said before :
- those two sites can be hosted on a different machine, through DNS,
- even if hosted on the same machine, the admin of the CMS or the people able to view the stats of the site usually differ.
So I'm against sending any referrer while moving between subdomains.
I'm ok to keep that reasoning for the case www.domain.tld / domain.tld but we can usually assume they are administered by the same entity. But that might not always be the case so we could also say we don't same the referrer. I would be fine with that.
- About « Not sending the referrer » :
I really thing « Not sending the referrer » should mean « we're not sending a referrer » (like in the example of case B3 in my last comment) and not « we're sending a fake referrer » (like in the example of case B2 in my last comment). The referrer should just not be sent, the browser should behave like if we entered the URL by hand in the location bar or clicked on the link from another app.
So I'm not in favor of sending the « the origin domain of the destination URL » because I don't see the point. Even though I don't see major differences in the privacy implications of both options.
-
Replying to T(A)ILS developers:
So I'm against sending any referrer while moving between subdomains.
Hi TAILS!!!!!!!!!!!!!!!!!!!!!!!! You're very smart!!!!!!!! YEAH!!!!!!!!!!!!!!!!! Indeed we think in the same way!!!!!!!!!! and this the reason why BeeFREE (my addon!!) works in the very same way you're thinking!!!!!!!!!!!! I even wrote everything you wrote, in my blog when i've been asked for the same questions!!!!!!!!!!!! http://honeybeenet.phpbb3now.com/viewtopic.php?f=14&t=237&p=1241#p1241 I knew that copying the code from my addon was a much more simple and very well working solution!!!!!!!! but mikeperry is always jealous of good suggestions!!!!!!!!!!!!
bye!!!!!!!!!!!!! ~bee!!!!!!!
Trac:
Username: bee -
bee - in general, outgoing link clicks aren't something we're interested in blocking right now. It is an arms race that really needs to be planned out if we're even going to attempt it. We would need a way of describing how outgoing link tracking works on sites and providing a rule language that allows people to write rules to undo this tracking, like we have done with HTTPS Everywhere, and like has been done with AdBlock Plus. There are too many sites doing this to expect to make a dent otherwise. Sure, Google. Yahoo, and Mozilla are the big ones, but the reality is many other sites have their own implementations, too.
In this bug, we are only discussing the actual Referer header. You're right, this is only one side of the problem. But it is one that we might be able to solve, or at least improve, with a single, general change to browser behaviour.