Opened 5 years ago

Last modified 3 weeks ago

#6119 new project

Create our own instance of Panopticlick

Reported by: mikeperry Owned by: cypherpunks
Priority: Very High Milestone:
Component: Applications/Quality Assurance and Testing Version:
Severity: Normal Keywords: tbb-fingerprinting, TorBrowserTeam201704
Cc: pde, runa, gk, adrelanos@…, arthuredelstein@…, tD66BSHWU@…, mcs, boklm Actual Points:
Parent ID: #5292 Points:
Reviewer: Sponsor: Sponsor4

Description

We have issues with how EFF's Panopticlick is run. It has inherent bias against any change from established norms, even if that change is in the direction of uniformity amongst a population.

See for example #4810.

As an alternative, perhaps we should set up our own instance of Panopticlick to measure the variance *among* TBB users?

We could then add our own fingerprinting tests to this site as we see fit.

Child Tickets

TicketSummaryOwner
#18343Implement a panopticlick-like anti-fingerprinting built-in self-test for the browsertbb-team

Change History (36)

comment:1 Changed 5 years ago by mikeperry

pde: Actually, perhaps there should be some kind of per-useragent Panopticlick dropdown query interface? That would also allow the major browser vendors to attempt to instill uniformity amongst their userbase, too. It would also solve the "But we just fixed that fingerprinting bug in our latest major release, and now our users think we're *worse* off?" paradox.

comment:2 Changed 5 years ago by runa

  • Cc runa added

comment:3 Changed 5 years ago by gk

  • Cc g.koppen@… added

comment:4 follow-up: Changed 5 years ago by pde

Open sourcing panopticlick is something I've been thinking about doing for years. While there are concerns with it, the cat was out of the bag before we even did the project.

comment:5 Changed 5 years ago by mikeperry

Yes, at this point it is way more useful to allow web browsers to create their own tests and to evaluate defenses.

We don't need your existing database full of user data (in fact, I think most sane people will tell you not to publish that). We just want the schema + source code dumped at a url somewhere. We're going to have to rely on the community for this one for at least the next few months anyways.

If we're lucky, maybe someone will take the tarball and run with it on their own VM or something so we can start testing against a clean instance populated only with TBB data.

comment:6 Changed 4 years ago by mikeperry

  • Keywords tbb-fingerprinting added

comment:7 follow-up: Changed 4 years ago by cypherpunks

  • Priority changed from major to critical

I think it's important to do it as soon as possible.

#6146

comment:8 in reply to: ↑ 7 Changed 4 years ago by proper

  • Cc adrelanos@… added

Replying to pde:

Open sourcing panopticlick is something I've been thinking about doing for years. While there are concerns with it, the cat was out of the bag before we even did the project.

Any updates on that? Just publish the code. Don't publish the database.

Replying to cypherpunks:

I think it's important to do it as soon as possible.

#6146

Please don't mess with the priority, just as you did in #6146, because you personally care about this issue.

I personally think that #8312 is most dangerous for most Tor users at the moment, #8170 is a simple and powerful attack and #3688 should be done yesterday already.

What I want to say, from all the different very important things to do, it's difficult to get started somewhere and get something done in an acceptable way.

Thanks for reporting bugs, anonymous user. Other things you can do is learning to code and help fixing the issues, paying others to fix it or to learn more about all these things, summarize and publish (papers, news pages) so the awareness for these problems increases. (At the moment still "Tor is fine and just use Tor with normal Firefox" still spreads, while critical issues are still outstanding.)

comment:9 Changed 3 years ago by cypherpunks

There is a project on github called open-panopticlick that seems to do what you are searching for.

comment:10 in reply to: ↑ 4 Changed 3 years ago by gk

Replying to pde:

Open sourcing panopticlick is something I've been thinking about doing for years.

pde: What is the state here? Is this going to happen soon?

comment:11 Changed 3 years ago by arthuredelstein

  • Cc arthuredelstein@… added

comment:12 Changed 2 years ago by feverDream

Hello,

I was wondering what the current status of the project is?

Thanks!

comment:13 Changed 2 years ago by qSKvY

  • Cc tD66BSHWU@… added

comment:14 follow-up: Changed 2 years ago by cypherpunks

browserspy.dk provided the code for Panopticlick, and offers a number of useful tests. Contacting them to discuss a partnership/setting up a new database based on their tests might be a good idea.

comment:15 in reply to: ↑ 14 Changed 2 years ago by feverDream

Replying to cypherpunks:

browserspy.dk provided the code for Panopticlick, and offers a number of useful tests. Contacting them to discuss a partnership/setting up a new database based on their tests might be a good idea.

Thanks.

comment:16 Changed 2 years ago by qSKvY

I wouldn't mind working on a Panopticlick clone as Open Panopticlick seems to be dead.
I'll post a link here if I make any significant progress.

comment:17 Changed 2 years ago by qSKvY

I've started a project called Libre-Panopticlick.
It's available here: https://code.google.com/p/libre-panopticlick/
It's written in Java and is currently in a functional state, however it could use a bit of polish.
If anybody has any suggestions for new fingerprint tests I'd be interested to hear them.

comment:19 follow-up: Changed 23 months ago by qSKvY

I've been working for a while on my fingerprinting website, Libre-Panopticlick (name subject to change when a better one is thought up; suggestions are appreciated).
It's resembles Panopticlick and AmIUnique except that it has a few tests designed specifically for Tor users, based on Tor trac tickets.
Those are:

  1. Whether the client is using Tor. Checked by performing a TorDNSEL request on the client / server combo.
  2. The time difference between the client and server in minutes.
  3. The output of toLocaleString() called on the UNIX epoch. The output of this differs based on browser locale, timezone, and browser, and has been confirmed to differ between instances of the Tor browser running on Linux and Windows.
  4. The output of Math.tan(-1e300), which differs based on operating system and reveals the underlying operating system that the Tor browser is being run on. This leaks the underlying platform that the TBB is being run on. For instance on a 64bit Linux machine it produces the value -1.4214488238747245 and on a Windows machine it produces the value -4.987183803371025.

At the moment it's in workable order and ready for at least a beta test.
I'd like to get an initial instance of it set up and running within the next two weeks.
I can provide hosting but I was wondering whether, when it's set up, I could get the Tor Project to direct some traffic towards it, since it was designed with the Tor project in mind.

comment:20 in reply to: ↑ 19 ; follow-up: Changed 22 months ago by gk

Replying to qSKvY:

I've been working for a while on my fingerprinting website, Libre-Panopticlick (name subject to change when a better one is thought up; suggestions are appreciated).
It's resembles Panopticlick and AmIUnique except that it has a few tests designed specifically for Tor users, based on Tor trac tickets.
Those are:

  1. Whether the client is using Tor. Checked by performing a TorDNSEL request on the client / server combo.
  2. The time difference between the client and server in minutes.
  3. The output of toLocaleString() called on the UNIX epoch. The output of this differs based on browser locale, timezone, and browser, and has been confirmed to differ between instances of the Tor browser running on Linux and Windows.
  4. The output of Math.tan(-1e300), which differs based on operating system and reveals the underlying operating system that the Tor browser is being run on. This leaks the underlying platform that the TBB is being run on. For instance on a 64bit Linux machine it produces the value -1.4214488238747245 and on a Windows machine it produces the value -4.987183803371025.

At the moment it's in workable order and ready for at least a beta test.

Exciting. Do you have a link to the code you are using for it? And one where it is running? You might as well be interested in https://lists.torproject.org/pipermail/tor-dev/2014-March/006486.html where we discussed things around Panopticlick more or less recently.

I'd like to get an initial instance of it set up and running within the next two weeks.
I can provide hosting but I was wondering whether, when it's set up, I could get the Tor Project to direct some traffic towards it, since it was designed with the Tor project in mind.

What do you have in mind if you are saying "direct some traffic towards it"?

comment:21 in reply to: ↑ 20 Changed 22 months ago by qSKvY

Replying to gk:

Replying to qSKvY:

Exciting. Do you have a link to the code you are using for it? And one where it is running?

The code can be found at https://github.com/qqTYXn7/browserprint
The server side stuff is written in Java and it uses MySQL.
When it's up the site will probably be running at http://browserprint.info
Sadly due to issues getting ethics clearance the site won't be up for at least a couple more months and I won't have much time to work on it in the mean time.

People who are interested may also want to check out https://amiunique.org as their code is also open source and they have more tests than Panopticlick (nothing Tor specific though).

You might as well be interested in https://lists.torproject.org/pipermail/tor-dev/2014-March/006486.html where we discussed things around Panopticlick more or less recently.

Thanks.
That email gives me some ideas, such as creating testing for already patched vulnerabilities.

I'd like to get an initial instance of it set up and running within the next two weeks.
I can provide hosting but I was wondering whether, when it's set up, I could get the Tor Project to direct some traffic towards it, since it was designed with the Tor project in mind.

What do you have in mind if you are saying "direct some traffic towards it"?

I mean, for instance, a mention in the Tor blog, or in the Tor Weekly News.

comment:22 follow-up: Changed 20 months ago by elypter

i think it should not only be directed towards tor users. it should have 2 modes. one compares the browsers with all other browsers like panopticlick and the other compares it to how torbrowser should look like with normal settings.
this could help not only users but also developers and researchers because it can be checked if addons or plugins change the fingerprint or if a new version of firefox reintroduces old fingerprints. and finerprints that only occur in unusual circumstances can also be detected with a large userbase.
it can also be useful to compare torbrowser to other anonymizing browsers and private browsing modes of mainstream browsers.

data storage should be optional and the user should be given the option to make a bookmark for his fingerprint for later comparison and the option to download the report as textfile.

comment:23 in reply to: ↑ 22 ; follow-up: Changed 20 months ago by qSKvY

Replying to elypter:

i think it should not only be directed towards tor users. it should have 2 modes. one compares the browsers with all other browsers like panopticlick and the other compares it to how torbrowser should look like with normal settings.

Can you elaborate on what you mean by "compares it to how torbrowser should look like with normal settings"?

data storage should be optional and the user should be given the option to make a bookmark for his fingerprint for later comparison

You mean like a web browser bookmark?

I like this idea, but in order to make it possible fingerprints would have to be publicly accessible; I don't think this is a good thing to do by default so perhaps there should be a checkbox before submitting your fingerprint that says "Make my fingerprint public and allow it to be bookmarked".

and the option to download the report as textfile.

Did you have a particular format in mind?

Do you think a plain text format like the following be fine, or do you think going with something fancier like XML or JSON would be better?

--Platform (JavaScript)
Linux x86_64
--Time Zone-570
--Screen Size and Color Depth
1920x1080x24

Last edited 20 months ago by qSKvY (previous) (diff)

comment:24 in reply to: ↑ 23 Changed 20 months ago by elypter

Replying to qSKvY:

Can you elaborate on what you mean by "compares it to how torbrowser should look like with normal settings"?

it should show which fingerprints are like the one you would have with a clean install of the newest torbrowser(maybe support multiple versions) and which deviate from the norm. it could be like on ip-check.org or with check marks. so an user can easily detect if an addon changed some browser settings or torbrowser has been modified in any way. things that are expected to e different for each torbrowser should be marked differently (like the exit node ip or the referrer)

You mean like a web browser bookmark?

yes or simply a link he can safe

I like this idea, but in order to make it possible fingerprints would have to be publicly accessible; I don't think this is a good thing to do by default so perhaps there should be a checkbox before submitting your fingerprint that says "Make my fingerprint public and allow it to be bookmarked".

i think so too but saving doesnt require it to become public if the link has a random id in it that only the user knows(like with google doc editing). it depends on the user if he wants to trust the server. he could be given the choice to not save it, saving only for personal use and saving it and also allowing to to contribute to statistics.

Did you have a particular format in mind?

Do you think a plain text format like the following be fine, or do you think going with something fancier like XML or JSON would be better?

--Platform (JavaScript)
Linux x86_64
--Time Zone-570
--Screen Size and Color Depth
1920x1080x24

depends on what the information can be used for. possible uses are storing for later review, storing for uploading it to the site again for later comparison or for bug reporting. dont know if its worth making a database file export and upload function if the user could also save it on the server. the user has to trust the server about the servrside fingerprinting anyway on the other hand javascript produces the majority of information. so i think its your choice if its worth the effort.

about the comparing function in general i thought it would be nice to have the site calculate the amount of overlapping fingerprinting information. with the bits of identifying information you could calcuate with what certainty an adversary could link the 2 fingerprints among the group of all users in the database.

Last edited 20 months ago by elypter (previous) (diff)

comment:25 Changed 14 months ago by gk

  • Cc gk added; g.koppen@… removed
  • Severity set to Normal

#18343 is a duplicate of this.

comment:26 Changed 14 months ago by mcs

  • Cc mcs added

comment:27 Changed 14 months ago by boklm

  • Cc boklm added

comment:28 Changed 12 months ago by qSKvY

So on the volunteer page it says that you'd like to have a machine readable interface to the fingerprinting service, such as JSON.
What kind of information would you like to include in that?
I mean obviously you want to have the fingerprint details. A barebones JSON interface would be like:

{

"useragent":"Mozilla/5.0 (Windows NT 6.1; rv:38.0) Gecko/20100101 Firefox/38.0",
"accept-headers":"text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8 gzip, deflate en-AU,en;q=0.7,en-US;q=0.3"
...

}

Do you want to include other data such as how many other browsers have the same fingerprint or the same fingerprint property?

Anything else you want to include?

I assume a CAPTCHA would be out of the question (I've been experimenting with fingerprinting people and browsers through CAPTCHAs lately).

comment:29 follow-up: Changed 11 months ago by qSKvY

After a lot of delays https://browserprint.info is open for business.
It's a fingerprinting suite that has a set of tests specifically to catch the Tor Browser Bundle out, based mostly on Tor Trac tickets.
I'm still working on new tests but let me know what you think.
Nothing would make me happier than my site being useful to the Tor project, and I'm 100% willing to modify it in any way to fit your needs.

comment:30 in reply to: ↑ 29 ; follow-up: Changed 11 months ago by arthuredelstein

Replying to qSKvY:

After a lot of delays https://browserprint.info is open for business.
It's a fingerprinting suite that has a set of tests specifically to catch the Tor Browser Bundle out, based mostly on Tor Trac tickets.
I'm still working on new tests but let me know what you think.

This is great. I noticed a bug in the font detection in fingerprintjs2, which I have reported there: https://github.com/Valve/fingerprintjs2/pull/159

Nothing would make me happier than my site being useful to the Tor project, and I'm 100% willing to modify it in any way to fit your needs.

On thing that might be interesting is to look at CSS-only fingerprinting techniques, because users often disable JS in Tor Browser. Tor Browser protects against quite a lot of CSS attacks, but it's possible more protection is needed. I did one such experiment here: https://arthuredelstein.github.io/tordemos/media-query-fingerprint.html

comment:31 in reply to: ↑ 30 ; follow-up: Changed 11 months ago by qSKvY

Replying to arthuredelstein:

This is great. I noticed a bug in the font detection in fingerprintjs2, which I have reported there: https://github.com/Valve/fingerprintjs2/pull/159

Thanks. I updated the code for that test.

On thing that might be interesting is to look at CSS-only fingerprinting techniques, because users often disable JS in Tor Browser. Tor Browser protects against quite a lot of CSS attacks, but it's possible more protection is needed. I did one such experiment here: https://arthuredelstein.github.io/tordemos/media-query-fingerprint.html

That's a neat test. I'd be interested in modifying it and putting it on my site, if you don't mind.
Do you have a way of reporting the results back to the server?
I think reporting the results back to the server without using JS is a big hurdle, but if it was possible a CSS-only fingerprinting attack would be very powerful.

comment:32 in reply to: ↑ 31 Changed 11 months ago by arthuredelstein

Replying to qSKvY:

Replying to arthuredelstein:

This is great. I noticed a bug in the font detection in fingerprintjs2, which I have reported there: https://github.com/Valve/fingerprintjs2/pull/159

Thanks. I updated the code for that test.

On thing that might be interesting is to look at CSS-only fingerprinting techniques, because users often disable JS in Tor Browser. Tor Browser protects against quite a lot of CSS attacks, but it's possible more protection is needed. I did one such experiment here: https://arthuredelstein.github.io/tordemos/media-query-fingerprint.html

That's a neat test. I'd be interested in modifying it and putting it on my site, if you don't mind.

Yes, feel free to use it.

Do you have a way of reporting the results back to the server?
I think reporting the results back to the server without using JS is a big hurdle, but if it was possible a CSS-only fingerprinting attack would be very powerful.

My demo does report to a server. There's a separate media query that makes a unique HTTP request for each possible width and for each possible height. For example, if the window width is 193px, then the following media query matches:

@media (width: 193px) { #width { background-image: url("https://dummyimage.com/50x30/fff/000&text=193&dim=width"); } }

The image https://dummyimage.com/50x30/fff/000&text=193&dim=width is therefore requested, which results in the number 193 being displayed in the page. But if you wanted to use this to record screen sizes on your own server instead, you could provide a background-image: url(...) that points to your server, with the matched width in a query string.

Here's the script I used to generate the CSS file:
https://raw.githubusercontent.com/arthuredelstein/tordemos/gh-pages/generate-size-query-demo

Last edited 11 months ago by arthuredelstein (previous) (diff)

comment:33 Changed 3 months ago by gk

  • Keywords TorBrowserTeam201702 added

comment:34 Changed 3 months ago by gk

  • Sponsor set to Sponsor4

This is Sponsor4 work

comment:35 Changed 7 weeks ago by gk

  • Keywords TorBrowserTeam201703 added; TorBrowserTeam201702 removed

Moving tickets to March

comment:36 Changed 3 weeks ago by gk

  • Keywords TorBrowserTeam201704 added; TorBrowserTeam201703 removed

Remmove remaining tickets over to April

Note: See TracTickets for help on using tickets.