# Panopticlick: Your Web browsing is less anonymous than you think

Visiting Web sites provides the Web host access to more information than you realize. It may be enough to create a traceable fingerprint.

Visiting Web sites provides the Web host access to more information than you realize. It may be enough to create a traceable fingerprint.

---------------------------------------------------------------------------------------------------------------------------------

After reading articles about user privacy, I noticed that members tend to discuss how they avoid being tracked (identified) while browsing on the Internet. Most recently, in GoogleSharing: A way to prevent tracking by Google, they promoted various add-ons such as NoScript, Adblock Plus, and disallowing cookies.

I was in tentative agreement, but curious. So, I started looking into whether that was actually the case or not. After some searching, I ended up at Electronic Frontier Foundation's (EFF) Web site. The answer I found was somewhat unexpected.

Some history

Last September, I wrote about researchers being able to mine identities from supposedly anonymized electronic databases -- a rather disconcerting capability and why I have not joined Netflix.

To explain, Netflix sponsors contests, offering huge sums of money to entrants that ascertain creative answers to their data-mining issues. On the surface, that sounds harmless. Yet, they use their actual member database. They say the information is anonymized, but to what degree?

According to the EFF, if information like Zip code, date of birth, and gender are part of the sanitized database, individual identities can be figured out. That's because when combined, individual pieces of information work together, reducing entropy.

Entropy

Entropy, in the world of information sharing is the term used to gauge how identifiable an object is. EFF defines entropy as:

"A mathematical quantity which allows us to measure how close a fact comes to revealing somebody's identity uniquely. That quantity is called entropy, and it's often measured in bits."

It took me a while to figure it out, but more entropy means less identifiable. The EFF thankfully has a Web page explaining the mathematical process used to determine how much a piece of information reduces an object's entropy.

Identification

Since there are approximately 7 billion living, breathing people right now, mathematicians have determined that 33 bits (two to the power of 33 is eight billion) of entropy are required for a person to remain anonymous.

Another interesting concept about entropy is identifying information can have different entropy-reducing values. For example, knowing a person's birth day and birth month provides less information (more entropy) than if the birth year is also known.

Web browsers

I'll bet you are wondering where I'm going with this entropy stuff. Well, the EFF feels that every Web browser provides enough unique information to tell one from another. Besides user accounts, IP addresses, and cookies; there is something called a User Agent string that can be used to further reduce the entropy of Web browser applications:

"Our experiment to date has shown that the browser User Agent string usually carries 5-15 bits of identifying information (about 10.5 bits on average). That means on average, only one person in about 1,500 (210.5) will have the same User Agent as you.

On its own, that isn't enough to recreate cookies and track people perfectly, but in combination with another detail like geo-location to a particular ZIP code or having an uncommon browser plug-in installed, the User Agent string becomes a real privacy problem."

If you are curious about your User Agent string, visit the Web site: What's My User Agent. Here is an example of a Firefox User Agent string:

Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.9.1.7) Gecko/20091221 Firefox/3.5.7

Looking at the string, you can pick out the following information:

• Firefox version of 3.5.7
• Windows version NT5.1 which translates to Windows XP
• The user's preferred language of en-US

Three more chunks of information transmitted to the Web host, and it can be used to differentiate Web browsers.

Panopticlick

The EFF developed a way to test the entropy of Web browsers -- an application called Panopticlick. I must commend the EFF for providing this service. It is a great way to learn what information is automatically offered by Web browsers.

I ran some tests, ending up with both expected and unexpected results. The first slide shows what Panopticlick found with my Firefox browser in locked-down mode. No cookies and NoScript disallowing everything:

That configuration contributed 10.55 bits of information. Let's see if allowing cookies changes anything:

The count is less (8.65 bits), meaning this configuration is offering less information to the Web host. I suspect that not allowing cookies made my Web browser more unique. Let me know if you get the same results. Next, let's shut off NoScript and see what happens:

I'm not surprised that this configuration provided the most information (18.33 bits). It is good to know that besides preventing malware from installing, disabling JavaScript increases a Web browser's anonymity.

Increase entropy

The EFF offers several solutions that will help prevent Web browser fingerprinting:

• Use NoScript, as it blocks Web sites from detecting plug-ins, fonts, and cookies.
• Use TorButton when accessing the Tor network. It changes the transmitted information to non-identifying values.
• Switch to a popular Web browser. It decreases the likelihood of being unique among other Web browser fingerprints.

The EFF admits they are finding it near impossible to find a non-unique Web browser. In fact, smart-phone Web browsers are the only ones that come close. That's because for the most part they are not changed from their default condition.

Final thoughts

Before researching this article, I was unaware that it's possible for a Web browser to have a unique and identifiable fingerprint. That said, running Panopticlick proves methods promoting on-line privacy do work. All that's left to do is reduce the amount of identifiable information your Web browser is providing.

As an aside, I was trying to understand why EFF used the term Panopticlick. Thankfully, Selena Frye, my editor at TechRepublic informed me that it probably was derived from Panopticon, a prison design that allows prisoners to be observed without their knowing it. Some how EFF's use of Panopticlick seems appropriate.

Information is my field...Writing is my passion...Coupling the two is my mission.

JCitizen

If it is at all pertinent to your next article, check out the new Avast v. 5 with it's auto-script blocker and Spy Shelter on CNET! I've been evaluating both and they are fantastic, but I'm not ready to review SpyShelter until I get a manual that works. My security system is blocking the HTML help file. I can now feel a locked down as I used to on XP - except for the upcoming patch Tuesday!

ITOdeed

Great article, most informative. Thanks Michael.

sebastijanpelhan

They do not need browser information since mobile provider has all the info :)

Ocie3

MarkGyver

Panopticlick reports that one in 3,110 browsers have the same fingerprint. The most identifying information is the user agent string which is shared by about every 128th browser tested. This is with NoScript and CS Lite installed and set to block everything. Allowing JavaScript makes the fingerprint unique from the 534,996 tested so far. I expect the main uniquifying thing is the "Skype Buttons for Kopete" plugin due to running KDE. The plugins list is shared by one in every 133,749 fingerprints generated, but that's probably because I've tested it multiple times, including with Chrome. Chrome doesn't yet have proper blocking abilities, so it's completely unique, even in an incognito window. I guess I'll have to use Firefox for all my private browsing.

Spitfire_Sysop

I would like to know how to stop Firefox from giving out details on the installed plugins and still enable scripts. Mainly because this data could be used to decide upon an attack vector. If they know your flash is out of date for example, wham! To a lesser extent I would like to remove the font list as well, due to this fingerprint concern. I ran the test with no script and got a good score but then when I clicked allow on the toolbar my score jumped 11 points. I would like to allow a movie to play for example but not give out a list of every damn thing I am running.

JCitizen

Gathers information that one in 530,759 browsers have this value. Comodo blocked the java request that web page made. Firefox 1 in 1502 browsers have this value. Quite a difference; it is funny that even when blocking java, the 64bit browser guarantees you will be unique. So in the U.S. figuring roughly 400 million people, only 753 people in the US would arguably have my same browser. Wow - 753 weirdos?! HA! :^0

mb.techrepublic

Realising that by posting here we, of course, are adding to the body of identifiable data (but then what percentage of TR members have / use credit cards anywhere, not just on the web, and/or are members of one or more loyalty schemes?)..... For the record: - My FF with NoScript gave the expected 1:52,440 (15.68 bits - which is just log(x)/log(2), but we all knew that, eh?) - no surprises there. - Dolphin under Android gave 1:262,400 (18 bits) - Native Android browser under Android gave unique, which I find mildly interesting since I'd expect fewer people using Dolphin than the native browser. If I were feeling paranoid, I might conjecture what's to stop Netflix or ANOther from buying webserver log data, from buying e-mail server log data (even if it were only source IP and source mail address) and combining the two, in return for targetting information, such as e-mail addresses back to the suppliers of the data? The correlation possibilities would be stupendous and horrendous. Paranoid mode off.

Michael Kassner

Have to tell you though, previous versions have been too intrusive and used too an abnormal amount of processor time. Have you looked at Cloud Anti-virus yet? I am a liking it more and more.

Michael Kassner

Stay tuned there is another article in this series. Should be out in a few days.

JCitizen

so in our country, roughly 615,384 have the same data as you. That seems pretty good. I argue that when a criminal is trying to assess your online activities, (s)he may not be able to tie it to you personally, but it helps in analyzing target values. Once just one piece of ID information is gained and cross checked with the unique header data, then it makes the crook's job easier to assess. Mind you these things are not done personally by the attacker, they are smart and have statistical data models they can use to track 100s of thousands of potential victims. But then I'm sure I don't need to tell you that. It is just always my goal to strip the low hanging fruit as high up the tree as possible. Having dynamic addressing is great. It is very unlikely the average crook would have the ISP crosschecking data to track you. Rebooting the router more than once a day would be even better!!

MarkGyver

NoScript lets you allow JavaScript, but block plugins. Unfortunately, it seems that the plugins don't actually have to be usable for them to be reported via JS. Thus, you must completely block scripting if you're worried about privacy. Also, certain plugins (such as Flash) have their own equivalent of cookies, only they can't be cleared by the browser. As for attack vectors, when you have NoScript block plugins, they can't abuse the out-of-date Flash or whatever. You can also selectively allow the plugin, letting you watch the movie on a site you trust, but not allowing anything on other sites. Maybe someone could make an add-on that intercepts whatever JS API is called for the plugin list and return something less useful.

Spitfire_Sysop

I think that is because it defaults to the IE8 32-bit and any power user that would know the difference is using FireFox. In conclusion; yes sir, you are a weirdo.

pgit

I've used user agent switcher for a couple years now. You can go to the developer's web site and download a lot more potential agents than the default mozilla add on gives you. It's more useful than just (attempted) privacy. I run Linux and use firefox. The other day my wife strolled over and asked to see a local furniture maker's web site advertised in a freebie newspaper. When we got there the galleries were java script, but they wouldn't initialize/run on my Linux box. So I reached up to tools--->default user agent and switched it to IE 7 running on Vista, and the galleries loaded and worked. It's primarily a web site compatibility (and testing thereof) tool. Any screwing up of google and other's attempts to gather data is bonus...

Michael Kassner

Do I understand correctly? You could have the add-on supply the User Agent string for an iPhone? That would perfect, exactly what the EFF suggests.

JCitizen

[b][You were right Michael, I had to abandon Spy Shelter as the scope of it wasn't working. It was allowing to many hooks to IE and FF by the time you allowed trusted programs.] [And Prevx oddly enough a cloud based solution, seems to be the answer!!! It supposedly puts a bubble around the browser and keeps any intrusive program from hijacking any keyboard or video sessions.][/b] it uses ZERO (0) CPU time and 6032k of RAM. For what it does this seems light. I wouldn't use it on XP anyway, as Snoopfree is already a proven defender to me. Both of them are the last defense you have for Zues type malware/variants, as they literally block the keyboard and screen hooks the new malware use to rob you of your bank account and/or credit. They need no signatures and are strictly behavior based. I don't see anything intrusive about blocking misbehaving programs like IE, FF, and Skype, especially since they all actually seem to work better after configuration. No browser has the right to watch what you are doing, Internet Messenger especially. I have a lot of legal spies on my machine because of DRM, so I will have to play with that to see how far I can go into denying what they can get their hooks into. So far I'm denying more and more and only seeing better performance, not less! I like the cloud idea for one of my clients anyway, as she can't afford more RAM and McAfee is gobbling too much RAM. She can't even update any longer because of the increasing RAM requirements of SP3 and later! However, Panda doesn't even make the list at AV comparatives, so I hesitate to recommend it to any of my other clients. I really like the script blocking characteristics of Avast v.5 pro and the built in GMER root-kit detection; I will have a hard time switching for now. I am already so confident that I would suggest it is BETTER than NoScript. After all you usually have to give complete control to trusted web pages and Avast only blocks misbehaving scripts on trusted sites. At least I can say this after next Tuesday's (9th Feb) patch cycle. IE has the usual gauntlet of vulnerabilities.

Ocie3

Currently, I "reboot" the DSL router while the computer is powered off (although the Linksys NAT router is not). If I rebooted the DSL router more than once per day, it would most often be while the computer is running. In that case it is best to power-off the Linksys NAT router before powering-off the DSL router. Then restore power to the DSL router and wait until it is functioning before restoring power to the Linksys NAT router. That usually causes the Windows XP Local Area Connection to eventually display a message that it has (re)connected to the network. That is a bit of work, given the current locations of the respective routers, but maybe I could use the exercise. :-)

Michael Kassner

My next article will make you reconsider that statement.

JCitizen

will delete FF flash cookies; however you have to close the browser; which isn't that bad, as FF saves the session.

Ocie3

can be controlled with the Firefox Better Privacy extension. Or you can run the browser in a Sandboxie sandbox. If you don't configure Sandboxie to "recover" any Flash data that is stored in the Windows account Application Data tree, then deleting the contents of the sandbox deletes the cookies. Of course, you can configure Sandboxie to "recover" the Flash data that you want to keep and discard the rest.

JCitizen

What is this 32 bit? I have a 64 bit browser with a 64 bit Java installation, and a one of the first truly 64 bit anti-virus solutions; which is Avast v.5, which has a script blocker that works as well or better actually than NoScript in protecting me against script attacks. My lab tests confirm this so far. Some page controls on webs sites won't work or are [x]'ed out and can't function because they are infected. With NoScript you are still somewhat vulnerable because of the fact that after you give full enough permission on the page, you can still be hit by an infected object. It is true that the granular control will reduce this danger, but is still a fact. I feel blocking the script automatically gives me at least a slight leg up on pages that I would have had to give permissions to anyway, whether I used FF with NoScript or not! (edited)We didn't have much respect for "power users" in the organizations I used to work for. We only had system administrators, 5 of 'em. and users. That's the way to break policy down if you truly want security in an organization. I work for no person or company, I just hate malware to pieces!

pgit

Firefox history search is among the top 10 most useful tools ever, if you have a lot of bookmarks like I do. I found the link to the text (.js?) file that adds more user agnets: http://techpatterns.com/forums/about304.html I searched "user agent switcher" in my library and came up with 17 hits. If I had to hazard a guess I'd say I have around 3,000 bookmarks total. I tried scroogling for this site first but after 5 minutes went to the history, much faster. Just have to remember to bookmark important stuff, and on top of that make sure you name them in some useful fashion. Just started playing with tags the first of this year.

Ocie3

wondering about doing that, because the IE Tab extension is not "compatible" with Firefox 3.5 and sometimes it is better to view a web site in I.E. For what it is worth: according to the Coral IE Tab developer, who picked-up where the original IE Tab developer stopped, that add-on requires use of a library which the Firefox development team banned because of security vulnerabilities.

Michael Kassner

Thanks, pgit. That is very cool stuff to know. I learn so much from comments to my posts.

evil genius tech responsible for developing image for major corporate or agency roll out, say a few thousand systems. As a final step, they use the program to change the user agent string to say the system is a Linux one, and locks it down. Once the roll out is finished, the web sites and system correlating the browsing habits via the use of the user agent will for CRAZY trying to work out a sane choice of ads etc for that unit. Just picture the tech at a major military base, with several thousand people, doing this to about five thousand PCs at once. Suddenly, Google-Analytics is trying to make sense of the browsing habits of five thousand individuals because they think all those people and systems are the one user. Also watch how those Internet Usage stats based on the User Agent information string collection suddenly register a huge change in the operating system market share. oooh, I like it.

Spitfire_Sysop

If you changed FireFox to claim it was an iPhone you would surely be 100% unique because the rest of the information would not match an iPhone browser. You would have to have a similar version of Safari with no changes to any of the preferences. For example, running no-script would take away information that the iPhone normally gives out.

JCitizen

in my estimation! I miss having a dynamic address, where I could toggle the switch on the back of my Netgear router. I can't remember if it was NAT or not. I don't bother now, as I'm static. Linksys routers are/were very touchy to voltage in the past, very scary! I used to put a Linsys hub between each Cat5 connection before and after the router, because of blowing ports. Once my sister blew the whole router, but she always did have trouble in the D town. Wow do I veer off topic! SORRY!

JCitizen

as it blocks misbehaving scripting requests automatically. It's a new feature, and it seems to work better with each kernel update. Between it and MBAM blocking malicious server connections/requests, things are getting tighter!

JCitizen

He is some pretty serious work!! :)

JCitizen

And I shall be back peddling! By the way, I finally found a replacement for Snoopfree Privacy Shield on Vista/Win7!! I am VERY pleased with this, as now I almost feel invulnerable; but I digress until next Tuesday the 9th, when MS closes some SERIOUS holes in the IE 8 browser!

Michael Kassner

I hope you stay tuned for my next article as it goes into that in more detail.

TobiF

Your browser in itself may not reveal too many bits of information. but all bets are off the moment your browser starts running scripts received from the web server. As it turns out, through scripting, a server may enumerate your installed add-ons and system fonts, and maybe poke around among some other stuff. It seems that the bottom line is that if you allow scripting, then ad networks etc may be well positioned to follow you around the net, even if you make sure that cookies bounce off your shields even quicker than they're served to you... And, if any of these places know your identity, then this knowledge could technically be shared among different sites.

Michael Kassner

About Panopticlick is that it is reaffirming what experts are saying. That a vast majority of people do not alter their browser setting to increase security. Those that do then standout and tend to lose entropy.

Ocie3

JCitizen

but ouy I get a headache even thinking about Mozilla's bookmark system! Arrgh! I guess it is how ones brain is wired, mine's spaghetti.

JCitizen

I took a look at Minefield x64 but couldn't find the Windows version, I used in on Windows XP Pro x64 Edition, and it was fine, I can't find it now! I could only find Linux tars. I just figure any vulnerabilities in it would be undiscovered for now.

Ocie3

thank-you. :-)

bookmarks, but have used the keywords in the history section to find a page I'd seen but not bookmarked and couldn't remember who it was from. My understanding of them is that the tags search the 'tag database' as you assign the tags and they can be anything at all. While the keywords looks for the requested word in the pages stored in the cache or the information in the database. Kind of works like this: You visit a page called Frank's Fearsome Security System. You tag it as Uncle's BS. You'll find nothing on the page that has the word uncle, yet a tag search on uncle will find it. Also, a keyword search on uncle will return nothing, will a keyword search on Frank or Fearsome Security will. Get the picture.

Ocie3

Chris Pederick developed and maintains the Firefox User Agent Switcher extension. The following forum message contains a hyperlink to a "huge import list" which, I believe, is the most recent offering of the list creator. http://chrispederick.com/forums/viewtopic.php?id=1772 Edit: BTW, Firefox has a history search [i]and[/i] a bookmark search. :-) You are quite right about the utility of the bookmark search! It seems that my collection is probably about the size of yours, and, like you, I recently started experimenting with "tags". It is an interesting feature. I am not sure what the distinction is between "tags" and the "keyword" feature, unless keywords are limited to one word. I do not recall using the keyword feature in the past.

Fire Fox is set up and used in Linux, the version of Fire Fox for Windows may be slightly different. I know the people who make FF have to do some things different to have it work on Windows. See FF 3.0 for Windows had .NET while FF 3.0 for Linux didn't - just to state the obvious - and that difference meant FF 3.0 for Windows was more vulnerable.

JCitizen

I may figure it out someday, I'm lucky I got any brains left. I keep saying I'm gonna try Minefield x64 someday, but keep forgetting to project it. Think I'll logoff now and check it out!

exactly the same way as I used to do it way back with IE 4. You open the Organise Bookmarks menu item and then use drag and drop to move them around, or right click for the menu to create a new folder. No hassles at all. One of the things I love about Fore Fox is how very easy it is to import and export bookmarks and how simple to find and copy, or replace, the bookmark list - much easier than in IE.

JCitizen

so difficult that I end up using IE to book mark, where it is easier to shuffle them around on the file tree. Maybe it is that easy on FF, but I'm too lazy to take Mozilla 101.

JCitizen

neglecting to bookmark. I always think I can Google it again, but if I don't use the [u]exact[/u] search string as before, I'll NEVER find it!

bookmark folders accessible through the 'Bookmarks' item on the menu. That drops down to show thirteen folders and about fifteen bookmarks I keep right there. Some of the folders have only bookmarks in them, while others have folders and bookmarks in them. I try to keep any one list down to a maximum of twenty-five items in it (bookmarks and folders) - with one special exemption. but some folders have a dozen or more within them, most are a mix. The deepest I go is five levels - that's sub-folders, in sub-folders to the fifth level. It makes it easy to find things - want to check for an update on SimplyMepis, I go Bookmarks - Software - Linux - Mepis - select one of the several bookmarks. The rest work in a similar manner. But, once I get more than twenty-five in a folder (except the special exemption), I review the book marks in that folder and see if any can be logically grouped together in a sub-folder; I can usually find one or two sub-folders will allow me to shift several bookmarks into them. I have no troubles finding anything again. If I want to have a look at the system folders, I've a quick launch item for that - it works the same as the Windows My Computer icon does.

pgit

My folder system is a wreck, but I won't bother straightening it out now that I see the search goes into tags, URLs etc. Let's see here, I have 13 folders and 4 direct links on the bookmarks toolbar. The folder with the least sub folders has 3, the one with the most I can't count atm, 19 in the first sub-level, each of them with 3-4 on average... plus a lot of direct links in each folder at every level. (not just sub folders) And to think I started clean Jan 2008. Prior to that I had counted well over 10,000 bookmarks, but toward the end (late 2007) I began going back to some to find them broken, web sites no longer available etc. Have you tried foxymarks, or whatever they call the on line bookmark sync add-on? I tried it out and it's a tremendous idea, except... like the "cloud," do I trust whomever with my bookmarks? Imagine the data mining that affords someone! It's like it's not just a hit/visit on some site, it's an actual VOTE for the site. The of course imagine what you could know about someone by subtracting thei bookmarks from their history. Boggles the mind. Thank God for search. I'm hoping the KDE desktop search (strigi and nepomuk) will eventually be able to touch the browser data. From left to right my toolbar folders: "open, 8, news, Linux, pgit, lab, ccc, home, search, (a collection of engines) stream, (security now! & etc) civics, hdw and Them." Them is a collection of web cams, mostly railroad oriented. I can get rid of some of those I see, and a few of the direct links as well, as those things were long ago moved to the back burner. news and civics... outta there...

me to track the 1,000 or so I have, and adding more each day.

JCitizen

I think that is one thing we used on our web-servers also. I'm not sure about Exchange though.

means we can now slam it shut and get you both, hehehehe. Damn, I think it's about time I was due for a sanity break. This is starting to get to me.

Michael Kassner

Knowing that I am on the same page as you is a good thing.

The 'G-Man.'

if you find the most used agent string then you could remain anonymous due to the numbers. I'm not sure this alone will do however.

Michael Kassner

Would you have to change the User Agent string often?

The 'G-Man.'

there is a user agent replacement setting on Advanced Proxy for IpCop.

Michael Kassner

Very good point. I did not think that through. When I get a chance, I plan to try the add-on and see what the reduction in entropy is.