Opened 4 months ago

Closed 8 days ago

Last modified 6 days ago

#26146 closed defect (fixed)

Setting `general.useragent.override` does not spoof the platform part anymore in ESR 60 which is confusing

Reported by: gk Owned by: mcs
Priority: High Milestone:
Component: Applications/Tor Browser Version:
Severity: Normal Keywords: ff60-esr, tbb-fingerprinting-os, tbb-8.0-issues, tbb-8.0.1-can, TorBrowserTeam201809R
Cc: viktor_jaegerskuepper@…, dmr, gamma@…, arlolra, aka, tbb-team Actual Points:
Parent ID: Points:
Reviewer: Sponsor:

Description

Despite updating general.useragent.override to match ESR 60 (done according to comment:16:ticket:25543) the platform part is not spoofed to Windows on my Linux box.

Now, that is intentional, see: https://bugzilla.mozilla.org/show_bug.cgi?id=1404608.

So, we probably should not set general.useragent.override at all anymore and just rely on the settings we get with privacy.resistFingerprinting? Because if we explicitly set it to the Windows UA but then don't get that, this is weird.

Child Tickets

Attachments (1)

ua.patch (1.9 KB) - added by mcs 11 days ago.
patch which was used for testing

Download all attachments as: .zip

Change History (70)

comment:1 Changed 4 months ago by cypherpunks

Type 'navigator' in developer console.
Tor Browser is a 'browser' so why not change navigator.* code to match Windows OS?

(make the website owner believe the TorBrowser user is using Windows 10 and Firefox 60(ESR))

comment:2 Changed 4 months ago by cypherpunks

'privacy.resistFingerprinting' is severly broken. They - Mozilla - implemented it wrong.

Firefox with 'privacy.resistFingerprinting:true'
(60 and above)
Mozilla/5.0 (Windows NT 6.1; Win64; x64; rv:60.0) Gecko/20100101 Firefox/60.0
(before 60)
Mozilla/5.0 (Windows NT 6.1; Win64; x64; rv:52.0) Gecko/20100101 Firefox/52.0

Tor Browser
Mozilla/5.0 (Windows NT 6.1; rv:52.0) Gecko/20100101 Firefox/52.0

Expecting
Mozilla/5.0 (Windows NT 6.1; rv:60.0) Gecko/20100101 Firefox/60.0

comment:3 in reply to:  2 ; Changed 4 months ago by tom

Replying to cypherpunks:

Expecting
Mozilla/5.0 (Windows NT 6.1; rv:60.0) Gecko/20100101 Firefox/60.0

If Firefox with RFP or Tor Browser used this format it would not match Firefox's (non-RFP) format and would be easily identifiable.

comment:4 Changed 3 months ago by viktorj

Cc: viktor_jaegerskuepper@… added

This is also bad for anonymity. With Tor Browser 8.0a9 the results on panopticlick.eff.org look as expected, except for "Platform" and "User Agent" which reveal the OS (Linux in my case).

There is also this Mozilla bug which is about not spoofing the platform because TCP/IP fingerprinting reveals the OS either way:
https://bugzilla.mozilla.org/show_bug.cgi?id=1409269

But this should not be relevant for Tor Browser, as I understand it.

I think this has to be fixed before Tor Browser 8.0 is released (hopefully even before that although it only affects the alpha channel at the moment).

comment:5 in reply to:  4 ; Changed 3 months ago by sysrqb

Replying to viktorj:

This is also bad for anonymity. With Tor Browser 8.0a9 the results on panopticlick.eff.org look as expected, except for "Platform" and "User Agent" which reveal the OS (Linux in my case).

The platform/OS on which Tor Browser is running can be detected multiple ways, spoofing the user agent is simply a low-hanging fruit for obscuring it. The usefulness of this is debatable, but we should minimize the differences between platforms when it is possible.

comment:6 in reply to:  5 ; Changed 3 months ago by viktorj

Replying to sysrqb:

The platform/OS on which Tor Browser is running can be detected multiple ways, spoofing the user agent is simply a low-hanging fruit for obscuring it. The usefulness of this is debatable, but we should minimize the differences between platforms when it is possible.

I understand. Ideally the user agent should be the same for all platforms, but I see that the platform can be identified at least via the fonts at the moment.

comment:7 in reply to:  6 Changed 3 months ago by cypherpunks

Replying to viktorj:

Replying to sysrqb:

The platform/OS on which Tor Browser is running can be detected multiple ways, spoofing the user agent is simply a low-hanging fruit for obscuring it. The usefulness of this is debatable, but we should minimize the differences between platforms when it is possible.

I understand. Ideally the user agent should be the same for all platforms, but I see that the platform can be identified at least via the fonts at the moment.

Not everyone does OS detection through installed fonts though everyone collects user-agents. Also with JS disabled OS detection is MUCH harder. Giving away *free* entropy like that is intolerable.

comment:8 Changed 3 months ago by dmr

Cc: dmr added

comment:9 Changed 3 months ago by cypherpunks

Cc: gamma@… added

comment:10 Changed 3 months ago by arlolra

Cc: arlolra added

comment:11 in reply to:  3 Changed 3 months ago by cypherpunks

Replying to cypherpunks:
WTF are you doing?

Replying to tom:

Replying to cypherpunks:

Expecting
Mozilla/5.0 (Windows NT 6.1; rv:60.0) Gecko/20100101 Firefox/60.0

If Firefox with RFP or Tor Browser used this format it would not match Firefox's (non-RFP) format and would be easily identifiable.

Don't mix FF w/RFP with TBB! RTFM of TBB ;)
Actually, Mozilla/5.0 (Unknown; rv:60.0) Gecko/20100101 Firefox/60.0 is everything TBB should expose until we can delete it altogether.

comment:12 Changed 3 months ago by cypherpunks

By the way let's smash this type of thinking once and for all, "X can be found using Y anyway in Z circumstances, so let's expose X directly". Ok, Tor Browser can be identified using exit IP, fingerprint and other stuff, so let's expose that information directly, how about Mozilla/5.0 (Windows NT 6.1; rv:8.0) Gecko/20100101 TorBrowser/8.0 as UA?

comment:13 Changed 4 weeks ago by nez

Tor Browser 8.0a10 reveals Linux, x86_64 and X11.

This is unacceptable.

Mozilla/5.0 (X11; Linux x86_64; rv:60.0) Gecko/20100101 Firefox/60.0

7.47 bits of identifying information
1 in 177.02 browsers

comment:14 in reply to:  13 Changed 4 weeks ago by cypherpunks3

Replying to nez:

This is unacceptable.

I can't stress this enough, as I said on the blog comments the original Mozilla bugzilla report makes no sense since with a proxy (such as Tor) network OS fingerprinting is moot so why fall for it? And it's such a let down to see great privacy people on the Mozilla side (that I won't mention out of respect) fall for such a cheap trap. (PS: As mentioned earlier, the argument ("X can be found using Y anyway in Z circumstances, so let's expose X directly") itself is false)

Anyone willing to make a followup bugzilla report to get this fixed on the Mozilla side as well?

comment:15 in reply to:  13 Changed 4 weeks ago by boklm

Replying to nez:

Tor Browser 8.0a10 reveals Linux, x86_64 and X11.

x86_64 is always x86_64 (even for i686 users), and X11 is always X11 on Linux, so it doesn't really reveal more than Linux.

comment:16 Changed 4 weeks ago by cypherpunks3

Comment 18 on the bugzilla was unaddressed https://bugzilla.mozilla.org/show_bug.cgi?id=1404608#c18 This is just sad.

comment:17 Changed 2 weeks ago by sysrqb

Closed #27495 as a dup.

comment:18 Changed 2 weeks ago by cyberpunks

(I am not the previous cyberpunks. I am just using the public account to comment.)
Does the 'perfect is the enemy of good' argument apply here?

Surely, there are many fingerprinting tactics available if you look hard enough, since most web technology are not designed with privacy in mind. (I would even argue that it is exactly the opposite.) However, this doesn't mean we should give away identifying information ourselves! I consider this being a regression.

By the way, #27520 is a possible duplicate of this.

comment:19 Changed 2 weeks ago by traumschule

Keywords: tbb-fingerprinting-os added

comment:20 Changed 2 weeks ago by cypherpunks3

If there is any possible reason to not spoof the user agent, why not do that only at lower security levels? Assuming we fix the browser so it actually spoofs when we tell it to spoof, we can get the best of both worlds by still preventing those who have JS disabled and do not expect advanced / active fingerprinting attacks from inadvertently leaking more identifying information about their system.

Does that sound like an acceptable idea?

comment:21 Changed 2 weeks ago by traumschule

The actual goal is to not split user groups even more. The current defaults are a tradeoff between usability across platforms and not being fingerprintable.

As i understood it, the fear is that new users open TB, visit their favourite sites, find them broken and switch back to another browser. So essentially the current approach is to make it easy for them and have an UI that invites to change browsing habits by getting more interested in anonymity.

I am not sure if it is possible to not break google's (and other) apps without harming anonymity, but this is the current approach.

Energy is probably best spent finding out how to fix existing fingerprinting issues that allow to guess your OS (see keyword) and to write guides to help others. Trying to convince developers to improve the settings for some by harming may be a waste of time for all.

For the moment interested users have to experiment with UA spoofing addons for themself. Hopefully the current OS specific UAs will be replaced by something better (like a split in two groups: desktop / mobile, or popups offering to lower security for known to be broken websites, etc).

See also https://lists.torproject.org/pipermail/tbb-dev/2017-October/000642.html and discussion in #27495.

comment:22 Changed 2 weeks ago by cypherpunks3

I read both of those. Unfortunately, they really don't have much discussion in them. While I think it's fine to make trade-offs at the lower security levels, doing it at the higher security levels is really a very bad idea. Note that I am only talking about Windows vs OSX vs Linux, not about desktop vs mobile (which is completely new ground).

After all, the high security levels already break a number of sites. It's clearly meant to be used for those who do not mind the occasional site being non-functional in exchange for better anonymity and privacy.

comment:23 Changed 2 weeks ago by gk

Whatever we do with respect to the user agent won't be included in the security slider. If that's an issue then it's a privacy issue and Tor Browser's privacy guarantees are or should be the same for everyone regardless of the security level. The slider is for adjusting the *security* against *browser exploitation*.

comment:24 Changed 2 weeks ago by sysrqb

I closed #27555 as a dup.

comment:25 in reply to:  23 ; Changed 2 weeks ago by temp123

Replying to gk:

The slider is for adjusting the *security* against *browser exploitation*.


https://tb-manual.torproject.org/en-US/security-slider.html

Tor Browser includes a “Security Slider” that lets you increase your security by disabling certain web features that can be used to attack your security and anonymity.


It would make sense to leave general.useragent.override as is and set privacy.resistFingerprinting to false when the security slider is moved to "Safest".

Reasoning:

  • A user-agent which differs from general.useragent.override is an anonymity issue
  • Javascript is disabled when security slider is moved to "Safest"
  • privacy.resistFingerprinting deals with privacy issues which are relevant only when javascript is enabled


This would give those who want to trade a bit of anonymity and security for a better browsing experience the option while not affecting those who want the highest/safest level of anonymity and security.

Also, this would not silently change the user-agent on update of those who have already moved the slider to "Safest", which is what prompted my previous ticket. https://trac.torproject.org/projects/tor/ticket/27495

Last edited 2 weeks ago by temp123 (previous) (diff)

comment:26 in reply to:  23 Changed 2 weeks ago by cypherpunks3

Replying to gk:

Whatever we do with respect to the user agent won't be included in the security slider. If that's an issue then it's a privacy issue and Tor Browser's privacy guarantees are or should be the same for everyone regardless of the security level. The slider is for adjusting the *security* against *browser exploitation*.

This is very precise reasoning, everyone who has a minimal knowledge on the Tor Browser's history and philosophy will come to these same conclusions. The comment above does note presuppose the security slider's function (the tb devs collect all the bugs that affected a Firefox ESR to determine which things to restrict in each slider level). Also what if someone uses a website foo.com with the lowest slider value then changes it to the highest, does he deserve to have his UA leaked then changed back? What kind of treatment is that?

comment:27 Changed 13 days ago by kevinoid

For reference, I just submitted https://bugzilla.mozilla.org/show_bug.cgi?id=1489903 to address privacy.resistFingerprinting=true causing general.useragent.override to be ignored.

Personally, I think TBB should send a Windows User-Agent from desktop browsers and Android from mobile, since it strikes a good balance between feature detection and a minor benefit to fingerprinting resistance. Regardless of the TBB default, I think users should be able to override it by configuring general.useragent.override for themselves, since they may have specific reasons for doing so.

comment:28 in reply to:  25 Changed 12 days ago by gk

Replying to temp123:

Replying to gk:

The slider is for adjusting the *security* against *browser exploitation*.


https://tb-manual.torproject.org/en-US/security-slider.html

Tor Browser includes a “Security Slider” that lets you increase your security by disabling certain web features that can be used to attack your security and anonymity.

Yes, security breaches may easily affect your anonymity (Tor's protections can bypassed that way) which is why both concepts are used in the same sentence. It has nothing to do with fingerprinting resistance here. If you feel that's not clear enough please file a ticket at this bug tracker so we can clarify our manual.

It would make sense to leave general.useragent.override as is and set privacy.resistFingerprinting to false when the security slider is moved to "Safest".

Reasoning:

  • A user-agent which differs from general.useragent.override is an anonymity issue
  • Javascript is disabled when security slider is moved to "Safest"
  • privacy.resistFingerprinting deals with privacy issues which are relevant only when javascript is enabled

privacy.resistFingerprinting deals with way more than the user agent but more importantly even if you have JavaScript disabled then you are *not* safe against fingerprinting risks. That ship has sailed long ago as the web has gotten more powerful over the years.

This would give those who want to trade a bit of anonymity and security for a better browsing experience the option while not affecting those who want the highest/safest level of anonymity and security.

I think you are misunderstanding something here: Tor Browser's privacy guarantees hold for everyone. It's the security side that some of our users need/want to adjust to their specific needs.

comment:29 Changed 12 days ago by gk

Cc: aka added

#27580 is a duplicate.

comment:30 Changed 12 days ago by cypherpunks3

During a brief chat I had with gk on IRC, we were discussing how this impacts Linux as opposed to OSX. While there are clear problems caused by user agent spoofing that affects OSX users, there seem to be none of consequence for Linux users. One possibility that turned up was Google Translate which used to give an impossible captcha on Tails, but I tested it on Windows, Tails, and Debian Linux using both the 8.0 and outdated browsers, and found that that is no longer the case regardless of the status of user agent spoofing. Whatever was causing it, Google fixed it for everyone.

So far, while it may be beneficial to OSX users to disable spoofing (it is supposedly bad enough that some of them are actually leaving), I would like to see spoofing done for Linux users, in part because we are far less common than OSX users and stand out more in access logs with their default log level.

comment:31 in reply to:  30 Changed 12 days ago by gk

Replying to cypherpunks3:

So far, while it may be beneficial to OSX users to disable spoofing (it is supposedly bad enough that some of them are actually leaving), I would like to see spoofing done for Linux users, in part because we are far less common than OSX users and stand out more in access logs with their default log level.

I think that's not true. The point here is not to count the fraction of Linux users per se but rather the fraction of Linux users *that use Tor Browser*. And I think that fraction is actually *larger* than the macOS Tor Browser users, at least if we believe the fractions shown for downloading fresh Tor Browser bundles (AFAICT we don't have statistics somewhere yet for per OS update ping numbers).

comment:32 Changed 12 days ago by cypherpunks3

Good to know. I'd be interested in learning the actual statistics.

My overall point is more that Linux users may not suffer as much as OSX users from using a spoofed user agent. This is partially due to the fact that most websites lack special handling for Linux user agents and only recognize Windows, OSX, and a few mobile agents. Linux user agents are typically unknown and the site falls back to Windows behavior. In addition, Linux users are far more used to websites that do not support them. In fact, there are a number of websites that actually refuse to offer full functionality to non-Windows user agents, which is at least one benefit of keeping user agent spoofing on that platform.

On my regular browser (which is Chromium), I actually spoof my user agent to display Windows since it tends to give me a better browsing experience. While I don't know if the same applies to OSX users, for me as a Linux user, pretending to be Windows results in better compatibility.

Last edited 12 days ago by cypherpunks3 (previous) (diff)

comment:33 Changed 12 days ago by gk

Keywords: tbb-8.0-issues added

Okay, here are the meeting results in condensed form:
1) We keep the mobile UA for Android as the breakage would be too high without it.
2) We investigate whether we can avoid the broken experience on macOS by sending a uniform UA header for all desktop platforms but do not lie about the JS navigator property as the OS detection might happen by JS and not via the HTTP header.

comment:34 in reply to:  33 ; Changed 11 days ago by cypherpunks3

Replying to gk:

Okay, here are the meeting results in condensed form:
1) We keep the mobile UA for Android as the breakage would be too high without it.
2) We investigate whether we can avoid the broken experience on macOS by sending a uniform UA header for all desktop platforms but do not lie about the JS navigator property as the OS detection might happen by JS and not via the HTTP header.

I agree that keeping the mobile UA for Android makes sense.

What was the consensus on the situation for Linux users, or was that not discussed?

comment:35 in reply to:  34 Changed 11 days ago by mcs

Replying to cypherpunks3:

What was the consensus on the situation for Linux users, or was that not discussed?

I think the idea is to make all desktop browsers use the same HTTP User-Agent header. The biggest potential breakage is on macOS due to the command key vs. control key difference. Kathy and I just completed some tests and the news is mixed. In a new minutes, I will attach the patch we applied but the summary is that we changed the User-Agent header and navigator.userAgent in JavaScript to Mozilla/5.0 (Windows NT 6.1; rv:60.0) Gecko/20100101 Firefox/60.0

GitHub: In the source code editor, Cmd+F works as expected. However, in the comment editor Cmd+B does not work to format text as bold (Ctrl+B does).

Google Docs: Command keys do not work; control is recognized as the modifier key for making text bold, italic, etc.

Our conclusion is that if we want to maintain compatibility with these kinds of sites, Tor Browser needs to make the platform available. At this point we do not know if GitHub and Google Docs are looking at the User-Agent header or if they are using JavaScript to test against navigator.userAgent.

Changed 11 days ago by mcs

Attachment: ua.patch added

patch which was used for testing

comment:36 Changed 11 days ago by tom

I'm sorry - I was wrong about navigator.userAgent. We wanted to test with that one reporting the correct platform; and I thought your patch would do that - but I was wrong.

To test with the userAgent DOM property remaining the same, we will also need to edit: https://searchfox.org/mozilla-central/source/dom/base/Navigator.cpp#1617

I think the fastest thing to do for testing purposes would be to strip the '!nsContentUtils::ShouldResistFingerprinting' guarding the 'general.useragent.override' pref and then set 'general.useragent.override' to 'Macintosh; Intel Mac OS X 10.13'

Sorry =(

comment:37 Changed 10 days ago by fufufu

As a Tor Browser user highly concerned with this change, I have two questions based on the dialogue I'm seeing on the comments section of the Tor blog about this subject:

  1. The biggest reason this change seems to be promoted by some (particularly gk) as "not a big deal anyway", even in the context of disabled Javascript where potential OS detection methods are minimized, is because your OS can apparently be detected anyway by what fonts you have (as Tor Browser ships with different fonts depending on the version it seems). My question is how the server communicates this information back to itself after detection without using Javascript. I can find no website, browser uniqueness analyzer, fingerprint analyzer, anonymity analyzer, Panopticlick-style test, etc. that can actually detect anything about my fonts with Javascript disabled in Tor Browser. I can only find a small reference in Whonix documentation to detecting fonts via "CSS introspection". Can gk or somebody else provide more information about how this works?
  1. If this is really all on behalf of fonts, is there a reason not to ship the same fonts with every version of Tor Browser on every platform?

comment:38 in reply to:  36 ; Changed 10 days ago by mcs

Replying to tom:

...
I think the fastest thing to do for testing purposes would be to strip the '!nsContentUtils::ShouldResistFingerprinting' guarding the 'general.useragent.override' pref and then set 'general.useragent.override' to 'Macintosh; Intel Mac OS X 10.13'

Thanks for the guidance. Kathy and I hacked Navigator::GetUserAgent() to respect general.useragent.override and set that pref to Mozilla/5.0 (Macintosh; Intel Mac OS X 10.13; rv:60.0) Gecko/20100101 Firefox/60.0. The result is that our tests with both GitHub and Google Docs were successful: the command key is correctly recognized on macOS.

I am not sure what the next step is; it looks like it will not be trivial to create a shippable patch (since Navigator::GetUserAgent() expects to get the userAgent string from the HTTP protocol handle, but we want HTTP to use a spoofed User-Agent).

comment:39 in reply to:  38 ; Changed 10 days ago by tom

Replying to mcs:

I am not sure what the next step is; it looks like it will not be trivial to create a shippable patch (since Navigator::GetUserAgent() expects to get the userAgent string from the HTTP protocol handle, but we want HTTP to use a spoofed User-Agent).

I'm not sure what OSCPU is supposed to be without fingerprinting; but in RFP mode, it's the same as User Agent. So if RFP is enabled; you could go grab the value from Navigator::GetOscpu and return that instead of querying the HTTP header...

comment:40 in reply to:  37 ; Changed 10 days ago by tom

Replying to fufufu:

As a Tor Browser user highly concerned with this change, I have two questions based on the dialogue I'm seeing on the comments section of the Tor blog about this subject:

  1. The biggest reason this change seems to be promoted by some (particularly gk) as "not a big deal anyway", even in the context of disabled Javascript where potential OS detection methods are minimized, is because your OS can apparently be detected anyway by what fonts you have (as Tor Browser ships with different fonts depending on the version it seems). My question is how the server communicates this information back to itself after detection without using Javascript. I can find no website, browser uniqueness analyzer, fingerprint analyzer, anonymity analyzer, Panopticlick-style test, etc. that can actually detect anything about my fonts with Javascript disabled in Tor Browser. I can only find a small reference in Whonix documentation to detecting fonts via "CSS introspection". Can gk or somebody else provide more information about how this works?

Anything that triggers a conditional load based on the size of other objects could be used to communicate it back. But it's more work and not as fun to program so I'm not surprised it's not common in POCs.

A CSS trick to do this would be something like https://arthuredelstein.github.io/tordemos/media-query-fingerprint.html but I bet you can d the same in canvas and in SVG.

Besides Fonts, another JS-free ways to detect platform could be media support/streaming. But yea, without using JS it definetly gets tougher. (There are a lot more network-level tricks that Tor is immune to but affects Firefox.)

comment:41 in reply to:  37 Changed 10 days ago by traumschule

Replying to fufufu:

Can gk or somebody else provide more information about how this works?

#18097 in the os fingerprinting keyword family has some.

comment:42 in reply to:  39 ; Changed 10 days ago by mcs

Status: newneeds_information

Replying to tom:

I'm not sure what OSCPU is supposed to be without fingerprinting; but in RFP mode, it's the same as User Agent. So if RFP is enabled; you could go grab the value from Navigator::GetOscpu and return that instead of querying the HTTP header...

Maybe I am misunderstanding something, but we need the entire, unspoofed user agent string. As far as I can tell, the only code that knows how to construct such a string is here:
https://searchfox.org/mozilla-central/source/netwerk/protocol/http/nsHttpHandler.cpp#909

Unfortunately, when privacy.resistFingerprinting is true, there is no way for code outside of nsHttpHandler.cpp to access the string. One possible solution would be to add a new attribute to nsIHttpProtocolHandler such as unspoofedUserAgent.

A question for tom: Is Mozilla likely to switch to this approach and accept such a patch?
A question for gk: Do we want to try this change in Tor Browser? Secondarily, should Kathy and I work on a patch this week or someone else or should we wait?

comment:43 in reply to:  42 Changed 10 days ago by gk

Replying to mcs:

A question for gk: Do we want to try this change in Tor Browser? Secondarily, should Kathy and I work on a patch this week or someone else or should we wait?

Yes, this looks promising in the sense that we can avoid the breakage AND the splitting of our desktop users based on User Agent. And, yes, I'd like to have a fix for this in 8.0.1 if possible, so please have a look at it.

comment:44 in reply to:  40 ; Changed 10 days ago by fufufu

Replying to tom:

Anything that triggers a conditional load based on the size of other objects could be used to communicate it back. But it's more work and not as fun to program so I'm not surprised it's not common in POCs.

A CSS trick to do this would be something like https://arthuredelstein.github.io/tordemos/media-query-fingerprint.html but I bet you can d the same in canvas and in SVG.

Besides Fonts, another JS-free ways to detect platform could be media support/streaming. But yea, without using JS it definetly gets tougher. (There are a lot more network-level tricks that Tor is immune to but affects Firefox.)

Well this is probably another dumb question, but is there any reason that all platforms can't ship the same fonts? Or would the differences in rendering them between the various platforms make this pointless anyway?

Also I'm curious about how you use media streaming to detect the OS. Is the way the video is rendered, detection of the audio/video interface names, or what?

comment:45 in reply to:  44 ; Changed 10 days ago by cypherpunks3

Replying to fufufu:

Well this is probably another dumb question, but is there any reason that all platforms can't ship the same fonts? Or would the differences in rendering them between the various platforms make this pointless anyway?

I believe the reason was to preserve the look and feel of the operating system. I think there are also technical issues that make it hard to ship the same fonts, including size constraints (correct me if I'm wrong). It's not too big of a deal though, since the font set is not recorded by the average access log, whereas the user agent is.

comment:46 in reply to:  45 ; Changed 10 days ago by fufufu

Replying to cypherpunks3:

I believe the reason was to preserve the look and feel of the operating system. I think there are also technical issues that make it hard to ship the same fonts, including size constraints (correct me if I'm wrong). It's not too big of a deal though, since the font set is not recorded by the average access log, whereas the user agent is.

But according to some possible font detection is the reason that we're supposed to consider our OSes compromised anyway. This whole issue is starting to seem a lot like an invented problem.

Also it seems obvious to me that fingerprinting defenses should take precedence over aesthetics in any case.

comment:47 in reply to:  40 ; Changed 10 days ago by arthuredelstein

Replying to tom:

Replying to fufufu:

I can only find a small reference in Whonix documentation to detecting fonts via "CSS introspection". Can gk or somebody else provide more information about how this works?

A CSS trick to do this would be something like https://arthuredelstein.github.io/tordemos/media-query-fingerprint.html but I bet you can d the same in canvas and in SVG.

Here is a pure CSS demo for detecting the OS:
https://arthuredelstein.github.io/tordemos/os-detection-font-css.html

comment:48 in reply to:  46 Changed 10 days ago by cypherpunks3

Replying to fufufu:

Replying to cypherpunks3:

I believe the reason was to preserve the look and feel of the operating system. I think there are also technical issues that make it hard to ship the same fonts, including size constraints (correct me if I'm wrong). It's not too big of a deal though, since the font set is not recorded by the average access log, whereas the user agent is.

But according to some possible font detection is the reason that we're supposed to consider our OSes compromised anyway. This whole issue is starting to seem a lot like an invented problem.

The thing is that it requires active detection (whether CSS or JS) to identify the platform, whereas user agent is always sent and is passively recorded by every site out there.

Also it seems obvious to me that fingerprinting defenses should take precedence over aesthetics in any case.

I agree, but I think size constraints are also a bigger problem. I am not sure, though.

comment:49 in reply to:  47 ; Changed 9 days ago by fufufu

Replying to arthuredelstein:

Here is a pure CSS demo for detecting the OS:
https://arthuredelstein.github.io/tordemos/os-detection-font-css.html

I have a few extra addons in my Tor Browser (8.0) install, but this doesn't work for me even with all of them disabled and JS enabled. It simply displays all 3 OS names simultaneously. It makes no network requests beyond the initial page loading either.

Last edited 9 days ago by fufufu (previous) (diff)

comment:50 Changed 9 days ago by traumschule

We collected two more duplicates today (#27671, #27672). Can somebody bump the score, please?

From ​https://arthuredelstein.github.io/tordemos/os-detection-font-css.html:

Your likely operating system is: Linux Mac Windows

I'm fine with that.

Also I'm curious about how you use media streaming to detect the OS.

Me not so much. My focus would be to educate users that all kind of media can harm their anonymity (#13543) and it's best to use the feature filter (aka "Safest"). There are better programs to watch videos than your browser. VLC accepts youtube links and if the terminal is your friend, youtube-dl and mpsyt make your day. libs to decode mp3 and other media are prone to all kinds of exploits to take over your browser. no way around it.

Someone with time could look into https://github.com/pyllyukko/user.js/issues/316
(i hope this won't fall an my feet later)

I suggest (to myself) not to wait for a decision, but to produce something that works. I am confident it will earn positive feedback.

  • #20842 is the place to go for fonts.
  • libm seems unfixable, better keep js turned off (#13018 #15473).
  • i propose to disable power management completely (#23627). This will break things and this is the goal.
  • #23701 confuses me
  • #27128 seems untouched

Did i forget anything?

I vote for: One UA for all Desktop OS with an option to disclose info for selected (by the user) apps.
(Knowing that voting doesn't help much).

comment:51 in reply to:  47 Changed 9 days ago by cypherpunks3

Replying to fufufufufufufu:

Replying to arthuredelstein:

Here is a pure CSS demo for detecting the OS:
https://arthuredelstein.github.io/tordemos/os-detection-font-css.html

I have a few extra addons in my Tor Browser (8.0) install, but this doesn't work for me even with all of them disabled and JS enabled. It simply displays all 3 OS names simultaneously. It makes no network requests beyond the initial page loading either.

This trick is defeated by disabling external fonts, for example enabling noscript.forbidFonts in NoScript 5, which is something Tor Browser should always have done, to hell with the broken sites and their users.

comment:52 Changed 9 days ago by gk

Cc: tbb-team added
Keywords: tbb-8.0.1-can added
Owner: changed from tbb-team to mcs
Priority: MediumHigh
Status: needs_informationassigned

comment:53 in reply to:  49 ; Changed 9 days ago by arthuredelstein

Replying to fufufu:

Replying to arthuredelstein:

Here is a pure CSS demo for detecting the OS:
https://arthuredelstein.github.io/tordemos/os-detection-font-css.html

I have a few extra addons in my Tor Browser (8.0) install, but this doesn't work for me even with all of them disabled and JS enabled. It simply displays all 3 OS names simultaneously. It makes no network requests beyond the initial page loading either.

I guess this doesn't work if NoScript disables remote fonts at the highest security setting. Here's another approach that works in Tor Browser or Firefox without using remote fonts or JavaScript, at least where I was able to test:
https://arthuredelstein.github.io/tordemos/media-query-font-detection.html

comment:54 in reply to:  53 Changed 9 days ago by fufufu

Replying to arthuredelstein:

I guess this doesn't work if NoScript disables remote fonts at the highest security setting. Here's another approach that works in Tor Browser or Firefox without using remote fonts or JavaScript, at least where I was able to test:
https://arthuredelstein.github.io/tordemos/media-query-font-detection.html

This connects to "dummyimage.com" but gives me a result of "None".

comment:55 Changed 9 days ago by gk

Today a user came into #tor and said that some videos are not playing anymore in Tor Browser 8 which played in Tor Browser 7.5.6. After some debugging and back and forth it turned out that the Linux user agent is the reason for that: the spoofed one allows to watch the videos while in the current situation Linux users only get essentially "Sorry, video format can't get played". So, *not* spoofing is actually breaking things for Linux users. I guess that's a further argument for fixing this issue.

comment:56 in reply to:  42 Changed 9 days ago by mcs

Replying to mcs:

Maybe I am misunderstanding something, but we need the entire, unspoofed user agent string. As far as I can tell, the only code that knows how to construct such a string is here:
https://searchfox.org/mozilla-central/source/netwerk/protocol/http/nsHttpHandler.cpp#909

Unfortunately, when privacy.resistFingerprinting is true, there is no way for code outside of nsHttpHandler.cpp to access the string. One possible solution would be to add a new attribute to nsIHttpProtocolHandler such as unspoofedUserAgent.

Never mind; we were not thinking about the problem correctly and now realize we can do things a different way.

comment:57 in reply to:  44 Changed 9 days ago by tom

Replying to fufufu:

Also I'm curious about how you use media streaming to detect the OS. Is the way the video is rendered, detection of the audio/video interface names, or what?

I haven't tried it, but if a site tried to play a streaming video using a media framework built into (e.g.) only Windows, they would know if it was playing based on whether or not the user was streaming the video from the server or not.

comment:58 Changed 9 days ago by mcs

Keywords: TorBrowserTeam201809R added
Status: assignedneeds_review

Here is a patch for review:
https://gitweb.torproject.org/user/brade/tor-browser.git/commit/?h=bug26146-01&id=c8751602f218f40ff243d63de8e8fbcc7e651230
Please read the commit message to learn what it does.

comment:59 in reply to:  58 ; Changed 9 days ago by watt

Replying to mcs:

Please read the commit message to learn what it does.

Do you really think "Windows NT 6.1" is WinXP? lol

comment:60 in reply to:  59 ; Changed 9 days ago by mcs

Replying to watt:

Replying to mcs:

Please read the commit message to learn what it does.

Do you really think "Windows NT 6.1" is WinXP? lol

Not any longer :)
I confused 5.1 and 6.1.

comment:61 in reply to:  60 Changed 9 days ago by watt

Replying to mcs:

Replying to watt:

Replying to mcs:

Please read the commit message to learn what it does.

Do you really think "Windows NT 6.1" is WinXP? lol

Not any longer :)
I confused 5.1 and 6.1.

...and platform https://bugzilla.mozilla.org/show_bug.cgi?id=1472618

comment:62 in reply to:  58 Changed 9 days ago by tom

Replying to mcs:

Here is a patch for review:
https://gitweb.torproject.org/user/brade/tor-browser.git/commit/?h=bug26146-01&id=c8751602f218f40ff243d63de8e8fbcc7e651230
Please read the commit message to learn what it does.

Aside from the exact value of the header, which I did not research the correct value for, LGTM.

comment:63 in reply to:  58 ; Changed 8 days ago by gk

Status: needs_reviewneeds_revision

Replying to mcs:

Here is a patch for review:
https://gitweb.torproject.org/user/brade/tor-browser.git/commit/?h=bug26146-01&id=c8751602f218f40ff243d63de8e8fbcc7e651230
Please read the commit message to learn what it does.

Looks mostly good to me. I guess we could refactor Navigator::GetUserAgent() a bit but we can leave that our for now and tackle that when upstreaming this patch.

Please adjust the WinXP in the commit message and the title given that you delete general.useragent.override from the prefs we ship but have the bug fixed nevertheless. Maybe "Bug 26146: Spoof HTTP User-Agent header for desktop platforms"? (or something like that). Oh, and that's not only JavaScript on/off related. The risk we are writing the patch for is that the OS is collected by any server Tor Browser is talking to, irrespective of your JS settings.

comment:64 in reply to:  63 ; Changed 8 days ago by mcs

Status: needs_revisionneeds_review

Replying to gk:

Looks mostly good to me. I guess we could refactor Navigator::GetUserAgent() a bit but we can leave that our for now and tackle that when upstreaming this patch.

Kathy and I were a little afraid to make big changes there without advice from a Mozilla person and/or more time to test things.

Please adjust the WinXP in the commit message and the title given that you delete general.useragent.override from the prefs we ship but have the bug fixed nevertheless. Maybe "Bug 26146: Spoof HTTP User-Agent header for desktop platforms"? (or something like that). Oh, and that's not only JavaScript on/off related. The risk we are writing the patch for is that the OS is collected by any server Tor Browser is talking to, irrespective of your JS settings.

Here is a revised patch (new commit message):
https://gitweb.torproject.org/user/brade/tor-browser.git/commit/?h=bug26146-02&id=a9b83ea79e2e1751d63fffe62b457008d516145e

comment:65 in reply to:  64 ; Changed 8 days ago by gk

Resolution: fixed
Status: needs_reviewclosed

Replying to mcs:

Replying to gk:

Looks mostly good to me. I guess we could refactor Navigator::GetUserAgent() a bit but we can leave that our for now and tackle that when upstreaming this patch.

Kathy and I were a little afraid to make big changes there without advice from a Mozilla person and/or more time to test things.

Yes, I guessed so and that's been a good idea.


Please adjust the WinXP in the commit message and the title given that you delete general.useragent.override from the prefs we ship but have the bug fixed nevertheless. Maybe "Bug 26146: Spoof HTTP User-Agent header for desktop platforms"? (or something like that). Oh, and that's not only JavaScript on/off related. The risk we are writing the patch for is that the OS is collected by any server Tor Browser is talking to, irrespective of your JS settings.

Here is a revised patch (new commit message):
https://gitweb.torproject.org/user/brade/tor-browser.git/commit/?h=bug26146-02&id=a9b83ea79e2e1751d63fffe62b457008d516145e

Looks good! I cherry-picked the patch to tor-browser-60.2.0esr-8.5-1 (commit 165031359d2db55af85cab32713860bb6a026483) and tor-browser-60.2.0esr-8.0-1 (commit 62d76b2381c1df0b893336f496ae4ffe161d592f). Woo, thanks everyone!

comment:66 in reply to:  65 ; Changed 7 days ago by cypherpunks3

Replying to gk:

Replying to mcs:

Replying to gk:

Looks mostly good to me. I guess we could refactor Navigator::GetUserAgent() a bit but we can leave that our for now and tackle that when upstreaming this patch.

Kathy and I were a little afraid to make big changes there without advice from a Mozilla person and/or more time to test things.

Yes, I guessed so and that's been a good idea.


Please adjust the WinXP in the commit message and the title given that you delete general.useragent.override from the prefs we ship but have the bug fixed nevertheless. Maybe "Bug 26146: Spoof HTTP User-Agent header for desktop platforms"? (or something like that). Oh, and that's not only JavaScript on/off related. The risk we are writing the patch for is that the OS is collected by any server Tor Browser is talking to, irrespective of your JS settings.

Here is a revised patch (new commit message):
https://gitweb.torproject.org/user/brade/tor-browser.git/commit/?h=bug26146-02&id=a9b83ea79e2e1751d63fffe62b457008d516145e

Looks good! I cherry-picked the patch to tor-browser-60.2.0esr-8.5-1 (commit 165031359d2db55af85cab32713860bb6a026483) and tor-browser-60.2.0esr-8.0-1 (commit 62d76b2381c1df0b893336f496ae4ffe161d592f). Woo, thanks everyone!

Can I get a tl;dr as to what this patch does?

comment:67 in reply to:  66 ; Changed 7 days ago by cypherpunks3

Replying to cypherpunks3:

Can I get a tl;dr as to what this patch does?

When RFP is on, http header is spoofed according to a 2-tuple (windows on pc, and android on mobile). Entry in javascript navigator object is spoofed according to the 4-tuple described in the bugzilla ticket above (android, linux, mac, windows).
(That's now another way to pick out tor browser from firefox, not that it makes a difference since there were plenty already)

Last edited 7 days ago by cypherpunks3 (previous) (diff)

comment:68 in reply to:  67 Changed 7 days ago by cypherpunks3

Replying to cypherpunks3:

Replying to cypherpunks3:

Can I get a tl;dr as to what this patch does?

When RFP is on, http header is spoofed according to a 2-tuple (windows on pc, and android on mobile). Entry in javascript navigator object is spoofed according to the 4-tuple described in the bugzilla ticket above (android, linux, mac, windows).
(That's now another way to pick out tor browser from firefox, not that it makes a difference since there were plenty already)

At least it's somewhat ok since with JS fonts leak OS info, I'm satisfied for now with this work but I wish so hard that the great folks here wouldn't fall for such cheap arguments ("muuh changing UA will break this site that can't even be accessed anonymously!").

comment:69 in reply to:  67 Changed 6 days ago by cypherpunks2

Replying to cypherpunks3:

(That's now another way to pick out tor browser from firefox, not that it makes a difference since there were plenty already)

Tor Browser has never intended to hide the fact that it is not vanilla Firefox.

Anyway, now that a fix is in progress, I am happy.

Note: See TracTickets for help on using tickets.