Security optimize

The next front in the cookie wars: Fighting the Evercookie

Web-browser cookies, you either love or hate them. There is no in between. Well, get ready to be either more in love or more upset.

Like most technical aspects of the Internet, cookies seemed to make sense when they were first introduced. In some regards, they're still useful. But, there's a dark side. Cookies can be used to track our movement on the Internet and many say, that's not right.

What exactly is a cookie?

Technically, a cookie is a benign piece of text originating at the web server and sent to the web browser, where it is stored in preparation for the user's next visit. Cookies can be used to automate web site authentication, retain web site preferences, shopping choices, or other bits of information intended to facilitate the visitor's experience.

There are two types of HTTP cookies. First-party cookies are sent from the web server listed the address bar. Third-party cookies arrive from different web servers usually serving ads on the displayed web page.

Not being associated with the currently-displayed domain, third-party cookies allow advertisers to compile an online history of users. The ad companies then use behavioral targeting to serve directed ads. This is where it gets complicated. Do you allow your movements on the Internet to be tracked, just to get ads that are better-suited for you?

Removal options

In an on-going struggle, advertisers develop evermore-persistent cookies. Then, security experts devise new ways to prevent cookies from being installed. Each web browser has its own way of handling cookies. Check the web-browser options or preferences tab. Privacy pundits suggest at least disallowing third-party cookies.

Last year, a new type of cookie was quietly introduced. It's officially called the Local Shared Object (LSO), commonly called a Flash cookie. More persistent than HTTP cookies, it requires additional-web browser extensions to remove.

The cookie war continues

For the most part, users control what cookies are installed. That's about to change. While researching an article, I came across Samy Kamkar's (@samykamkar) web site, Evercookie--never forget. The title grabbed my attention. What is an Evercookie? Here is Mr. Kamkar's description:

"Evercookie is a JavaScript API that produces extremely persistent cookies in a browser. Its goal is to identify a client even after they've removed standard cookies, Flash cookies, and others."

Here we go again.

In-depth analysis

If the name Samy Kamkar sounds familiar, it's because he is best known for the Samy worm, the first XSS worm, infecting over one million users on MySpace in less than 24 hours. Currently, he's an independent security researcher and co-founder of Fonality Inc., an IP PBX company.

Not being an expert when it comes to how web-browsers interact with cookies, I thought it best to ask Mr. Kamkar to help explain Evercookie:

TechRepublic: What is an Evercookie and why did you develop it? Samy Kamkar: Evercookie is a Javascript API that allows storing cookie data in a number of different locations when a user visits a web page. Normal sites would typically just store data (such as a session identifier) in something like a cookie.

However, Evercookie not only uses the cookie, but a number of other locations such as Flash cookies, Silverlight isolated storage, and various locations of HTML5 storage. When a user deletes their standard cookies, the other locations remain and are able to rebuild the original cookie.

I built Evercookie as a proof of concept, wanting to show how web sites are able to track users even if they delete standard cookies and LSOs. Evercookie also sheds light on the fact that there are numerous methods for storing cookies locally. Finally, Evercookie acts as a litmus test for users who want to see if they're protected from web sites that track like this. TechRepublic: Several experts have commented that the following two storage methods are brilliantly devious.
  • Storing cookies in RGB values of auto-generated, force-cached PNGs using HTML5 Canvas tag to read pixels (cookies) back out
  • Storing cookies in Web History

Could you explain what they are and why the experts feel that way?

Samy Kamkar: Storing cookies in the PNG image is interesting, since an image is really just data. The data you want to store gets converted to color values. The color values are strung together in an image to produce a PNG file.

Evercookie then tells the browser to store that image for 30 years in its cache. When the user returns to the site, the image is accessed via cache and the page then reads each pixel of the image, extracting the colors from each pixel. The colors are converted back to text which produces the original cookie data.

Storing cookies in web history uses an interesting feature of web browsers.

Let's assume the cookie data we want to store is "bcde". Evercookie then accesses the following URLs in the background:

  • google.com/evercookie/cache/b
  • google.com/evercookie/cache/bc
  • google.com/evercookie/cache/bcd
  • google.com/evercookie/cache/bcde
  • google.com/evercookie/cache/bcde-

These URLs are now stored in the browser's history. When checking for a cookie, Evercookie loops through all the possible characters on google.com/Evercookie/cache/, starting with "a" and moving up, but only for a single character.

Once it sees a URL that was accessed because it's in the browser's history, it attempts to brute force the next letter. This process occurs extremely fast because no requests are made to the server in question. Evercookie knows it has reached the end of the string as soon as it finds a URL that ends in "-".

TechRepublic: Is the installation process automated or does the user have to initiate it? Samy Kamkar: No, the client simply visits the web site. There is no indication that persistent data is being set, exactly like a website with standard HTTP cookies. TechRepublic: Each version of web browser has a method to surf privately. Does that prevent Evercookie from storing a cookie in any of the locations you have chosen to use? Samy Kamkar: Most private-browsing features of web browsers stop almost all features of Evercookie. The problem is it only requires one location to remain for the Evercookie to keep its tab on the user. Hopefully, these features will improve in future versions and prevent all of these storage methods. TechRepublic: Can Evercookie be defeated by disabling JavaScript or using an application like NoScript? Samy Kamkar: Yes, NoScript or turning off JavaScript will prevent the Evercookie from being created. TechRepublic: I use more than one web browser, does the Evercookie work if I switch to a different one after receiving the Evercookie? Samy Kamkar: If a user gets cookied on one browser and switches to another browser, as long as they still have the Local Shared Object cookie, the cookie will reproduce in both browsers. TechRepublic: You mentioned Local Shared Objects (LSO). Do extensions like FlashBlock, CCleaner, or Adobe's Website Storage Settings panel remove Evercookie's version of LSO? Samy Kamkar: While those will stop the LSO, it will not prevent any other methods of storage, and it only takes one storage mechanism to allow full tracking. TechRepublic: I have read a few comments from developers that the Evercookie is exactly what some of their clients want. Do you know if this is still a Proof of Concept or actually being used? Samy Kamkar: I don't know if Evercookie itself is being used, but I know companies have already employed similar, yet less powerful software to do this. TechRepublic: You have an interesting motto, "Think bad, Do good". Could you explain what you mean? Samy Kamkar: I simply believe the best way to protect ourselves is to understand how we can be exploited in the first place. TechRepublic: You also have been quoted as saying (Courtesy of Ars Technica):

"I hope Evercookie simply demonstrates to people what types of methods are being employed to track them and to decide whether or not they want to prevent those methods. Evercookie took less than a day to create for me as a security hobbyist, so I can only imagine the technology that funded developers is producing."

What are your thoughts about the pending lawsuits related to cookies and their ability to track online travels?

Samy Kamkar: I'm not sure it's enough of an issue that lawsuits are necessary, but I do believe users should have the full right to prevent any web site from tracking them. I also believe the web browser should make it extremely easy for a user to prevent this sort of tracking. However, no web browser currently makes it easy to do. I'm hoping Evercookie can spawn some new features that will make it easy to prevent Evercookie-like tracking.

Removing the Evercookie

Mr. Kamkar was correct. I found two researchers who have developed methods to remove Evercookies. Jeremiah Grossman founder and CTO of WhiteHat Security has written a blog showing how to remove the Evercookie from Chrome and Firefox. Dominic White a security consultant working for SensePost has written a tool for removing Evercookies from Safari.

I asked Mr. Kamkar if these were indeed solutions:

"It appears they provide information on how to remove the Evercookie. It's just such a cumbersome and difficult process that the typical user would not make use of them."

Final thoughts

The cookie war is far from over. It appears that standard prevention practices are insufficient, due to the new locations cookies can be hidden. The only for-sure solution is to disable JavaScript to prevent the setting of an Evercookie.

A special thanks to Mr. Kamkar -- I asked a lot of questions.

About

Information is my field...Writing is my passion...Coupling the two is my mission.

174 comments
TobiF
TobiF

I think it could be possible to store identifying information in a css file! A css-file is typically well kept, and can be created in such a way that a unique value is stored and referred back to the site even if scripting and cookies are blocked in the browser! How: Whenever a css-file is requested from the server, a link to a graphical element with a unique url is inserted. The css file shall, of course, be given attributes for stable storage in cache memory. It may be hard to plant a specific value in a css-file, but pro-actively generating unique references and afterwards linking them to an identity shouldn't be that hard.

Who Am I Really
Who Am I Really

when I initially setup Firefox a while back, there were a few [b]about:config[/b] entries that I used to completely disable all history, the only "history" that exists are the 9 most recently closed tabs / windows of the current session all of which vanish when I "clear recent history" or close the browser I also use the XP Guest Browser, which has no write priv. for any location but the Guest Profile folders and the only place I ever go with IE is winupdate or MSDL

bandersnatch42vt
bandersnatch42vt

I've been using the "BetterPrivacy" add-on for Firefox for quite awhile now to remove LSOs both manually (via "Tools/BetterPrivacy") without a browser restart and automatically when closing the browser. Works very well. I use the above add-on in conjunction with the "AskForSanitize" add-on that returns the "Delete browsing history" dialog box when closing Firefox that was taken away by Mozilla a couple of Milestones ago. I'm actually rather surprised that BetterPrivacy isn't more well known by Firefox users since it's hardly some obscure add-on that you have to dig for.

JCitizen
JCitizen

to get rid of javascript/java. I guess I can't bring myself to do it, because I like Open Office, and playing with fire. :D

Michael Kassner
Michael Kassner

I have been looking at the portable version. It does quite a bit. Thank you for bringing it to my attention.

Michael Kassner
Michael Kassner

The topic of Evercookie and where it can be hidden is far from over.

TobiF
TobiF

The referred image file should, of course, be flagged for non-caching (or better, with an expired cache date)

TobiF
TobiF

Before, in order to find the correct URL to change my settings in my flash client, I had to use Google. But I noticed that nowadays, I can reach the same pages through a simple right-click on any flash content in the browser. That't way more convenient. And, again, can't be stressed enough that at least flash (and possibly some other third-party add-ons) have the nasty ability to replicate recognition of a user between different browsers! (I.e. you visit a site with Opera, and when you come back via Internet Explorer or Firefox, you're recognized there, too)

Michael Kassner
Michael Kassner

I am not that confident that it is a permanent solution. In the article I wrote about Flash cookies, members and I were seeing that the settings were getting changed or not working.

TobiF
TobiF

I've told Flash to automatically set storage to 0 for any site (and never ask). If a site doesn't work, then I either go somewhere else, or think whether I'd trust this site.

Michael Kassner
Michael Kassner

I would be curious to learn if what you do prevents the Evercookie from installing.

seanferd
seanferd

This is like the "Bat-Channel" for me. :)

david_heath
david_heath

What if we had a browser add-on that permitted group/shared cookies (and Evercookies). That way, we would ALL present an identical cookie whenever we browsed a site. It would drive the data collectors nuts!

seanferd
seanferd

I don't know enough about CSS to really understand the idea, but is interesting nonetheless!

Michael Kassner
Michael Kassner

I proved that to myself when I was writing my article about Flash cookies. On how the volume control would remain the same regardless of what browser I used.

Who Am I Really
Who Am I Really

my current Firefox config. is: > 3.5.15 the following configured / disabled via options UI and about:config as necessary > No History [0 days] - browser.history_expire_days = 0 - browser.history_expire_days.mirror = 0 - browser.history_expire_days_min = 0 - browser.history_expire_sites = 100 > No location bar history / storage (clicking the ↓ displays nothing) > browser Cache 1MB > browser > No Formfill / AutoComplete > No Password Storage > DOM.storage.enabled = False etc. Delete everything on exit Delete everything on "Clear Recent Histroy" Current Add-ons: > NoScript > FlashBlock > BetterPrivacy > AdBlock Plus > FlagFox > FormFox (doesn't work in 3.6.x)

techrepublic@
techrepublic@

I use Firefox with NoScript so no evercookies by default. But just in case I activate JavaScript, Better Privacy extension takes care of flash cookies, and regular cookies are set to session only (with exceptions for some sites that I would be logging in anyway). Cache is also cleared on exit. Evercookies die on exiting my Firefox.

JCitizen
JCitizen

I will have to look into this "guest browser" also.

JCitizen
JCitizen

That test tool convinced me of it, and also your good arguments, and the links to web sights that pointed to glaring vulnerabilities in the utility. That was enough for me - I'm using Rapport with Key Scrambler now. I'm amazed that Key Scrambler works with my LastPass console as well!!! This really gives me more peace of mind than I've had in a long while. Rapport was a little unstable at first, but seems to be running swimmingly now!? I probably should have used the Prevx cleanup tool, but I don't trust their uninstaller or their cleanup tool. Not surprised Webroot bought out a competitor, it was definitely a better anti-spy tool than Webroot's I had used Key Scrambler way back when, but it was less capable when it first came out. I'm pretty impressed with it now. Prevx flunked every test with that tool your article linked us to. I had to turn AdAware off to install it AND use it(AKLT), but false positives on tools like that are understandable. I can't thank you enough for your patience in this and your guiding hand! :-bd

JCitizen
JCitizen

I've been relying on what coders have speculated on here and other forums. AdWatch really doesn't block cookies, but it does prevent some malware installations. Of course it wouldn't stop Zues - I'm not trying to make it out to be superman, but since I went back to it in 2008, I've had the speediest pageloads in my life! As far as they can figure it clips the extraneous server commo traffic to ad-servers and the like. I still get some ads on IE8, but I think only SpywareBlaster is blocking some ads with the host file it uses. AdBlock Plus overides this on Mozilla of course, but you still get active x protection there. I seem to have full functionality on both browsers, and haven't missed anything that isn't displaying correctly. I haven't resorted to using MVPS on Vista yet. I do use it on XP, and strangely enough, I still need AdAware to get decent page loads. The CPU is 2005 vintage, so I don't need anymore slowdowns on that one!

AnsuGisalas
AnsuGisalas

That's what I assumed. Adwatch apparently manages to cut through some time/resource-consuming ad-load processes. That sounds like it'd be worth your while looking into; if there are ways to stop short some of the back-and-forth involved with targeted ads, it'd have a positive impact on security in other ways too.

Michael Kassner
Michael Kassner

What is communication-clearing effect/affect (I think it's effect, remember I am a writer)?

JCitizen
JCitizen

Just putting SpywareBlaster on board keeps most of the advertisements away in both FF and IE8. It does let some through, unlike AdBlock Plus on Mozilla's FF. It still made a big difference in page loads with IE8 despite the ads. I do notice a lot of dead page controls and images though. Not enough to affect quality. I never know which page controls on IE are being blocked by my security blanket. I've noticed that a good number of them turn out to be infected; this for the last two years, anyway. Even legitimate web sites of course. Avast Pro v.5 has a script blocker that acts like NoScript, only selectively.

AnsuGisalas
AnsuGisalas

If adwatch manages to decapitate the ad loading process without causing the browser to wonder why this process is dumped, it could be a big gain. Especially looking at the method used for pushing those targeted ads; lot's of back-and-forth relaying as I recall. Michael did a piece on that too. I get long hangs especially when an ad image fails to load...

JCitizen
JCitizen

how AdWatch in AdAware works now. My browsing was slowed down tremendously until I went back this old standby. It uses too much RAM, but I've never had such fast browsing! It stops most malware from installing, and apparently blocks some of the server/cookie communication going on with the sites I frequent. Most experts I talk to seem to feel this is they way it works. As far as I know, only SuperAnti-Spyware claims to actually block ordinary cookies. I can't remember if Spybot Search & Destroy could be configured to do this or not. MBAM blocks bad IPs, but it doesn't have the same communication clearing affect as Lavasoft's utility.

TobiF
TobiF

I guess ads in itself aren't such a bad thing. Thanks to ads, we have many sites with interesting and useful content that we can visit/use "free of charge". For several specialized sites, like TR, you have pretty good segmentation simply through the topic of the site itself. If you want to reach IT people, TR is a good choice. The problem comes when someone wants to push ads about database engines and file servers also while you're on more general places, like facebook or checking the weather forecast. Because, in order to do that, they need to track you down. We know that this is happening, that ad networks are fighting for ways to plant their "third-party cookies", and many strange a place on your computer is used to identify you. By the way, the "panopticlick" approach can be used to unite identities from different sites, whereas LSO can be used to unit different browser identities from the same machine. Bit by bit, the networks get fuller profiles. And it becomes harder all the time to "flush" the system for a fresh start - a totally empty profile, but still with a "full web experience" on those sites I choose to trust.

AnsuGisalas
AnsuGisalas

They only look for their own hooks... Of course, one could imagine a "tag-sharing" cloud; where all cookies and tags from all users are shared and proliferated so as to give false positives, but the bad guys can probably find ways to keep up. Better would be to find a weakness in the data-collecting software and then produce a viral cookie to proliferate on the web, one that crashes the ad servers. Ooops... now, that would be bad, wouldn't it?

seanferd
seanferd

Silly me. Considering the general enthusiasm, and the direction MS is taking, I suspect advertisers and bad guys are quite excited. I wouldn't even speculate as to who is more excited. I'm not overly concerned about CSS, but I found the thought interesting.

TobiF
TobiF

All big sites nowadays use css, cascading style sheets, for their design. The recommended approach is that parts of the style sheets that are repeated between different pages should be kept in a separate file. This allows for use of named styles. Just by indicating the name of the style, the css will ensure that the text will be placed and formatted as intended. A named style can even automatically make the browser fetch a specific image that will be used as a bullet in a list etc. Separate css file also makes it very easy to change the style of a complete site, simply by replacing a css-file or pointing to another css-file. In general, css files tend to be quite static, so browsers are happy to cache them. Now, nothing stops a page from referring to two css-pages. Thus, one of these files can be the "real" css, which takes care of the layout and formatting, and a second file could be prepared so that a call will be made to a an image-file, using a unique url. This page, where I'm writing this post, for instance, imports at least 4 css-files, the first of them being http://i.techrepublic.com.com/css/base.css"

Michael Kassner
Michael Kassner

I don't know who is more excited about HTML5, advertisers or the bad guys.

Who Am I Really
Who Am I Really

and wasting valuable backup space, slowing down the system, bloating the MFT, etc. as it is my backup image fits nicely on 1 DVD 4.7GB - add 90 Days of history (The Firefox default) and cached pages etc. and pop! I now need to use DVD DL @ 4x the price of regular DVDs or start splitting the images (not fun) as this system has aged (built Feb. 2007) the only thing that has gotten longer is the boot process, but once windows is loaded, everything is just as snappy as when I first built the system - (except the tardy network places issue, most likely caused by the uninstall / removal of AVG) and no need to prosecute here, I own the business, I don't want or need history, stored pages etc. caching parts of pages was good for the dial-up days with them old 16K-28K modems and static pages that didn't change much, but gives no significant improvement with current connection speeds, even on a slow home connection only heavy media rich pages load slower than if they were cached, and even then, most of those "heavy pages" change so frequently that the cache is out of date the next time you visit

Michael Kassner
Michael Kassner

I was confused about that as well. Can you tell that I am not a DB admin type.

seanferd
seanferd

History, bookmarks, form history, etc., and mail in T-bird or SM, are all stored in sqlite databases. When crap is deleted, it leaves whitespace, as it were between blocks of data. Compacting removes the space. Yes, it probably also affects the index for browser cache as well.

JCitizen
JCitizen

"compact databases" in FireFox; I wonder if they are anything like the index data files of Internet Explorer? I used to prosecute misbehaving employees with those; even if they deleted their history.

Who Am I Really
Who Am I Really

nothing is ever stored beyond the current session, - no addresses - no cookies - no photos - no flash vids etc. the history menu is always blank only "recently closed tabs / windows" will contain anything during the session but nothing is there next time I open Firefox

Michael Kassner
Michael Kassner

If that is activated, the Evercookie does not get installed. But, if you go to a web site and enable it, your defense is weakened. Your other extensions will remove several versions of the Evercookie except the image file and web history ones.

Michael Kassner
Michael Kassner

I had forgotten. The nil password. Thanks for straightening me o ut.

JCitizen
JCitizen

It just goes to show, I should review my old text books more often!

Who Am I Really
Who Am I Really

to read only for Guest on the share docs that would allow retrieving shared docs but not modifying them or writing new docs - I just checked mine again, it's already set that way

Who Am I Really
Who Am I Really

for disabling Guest in the past, (especially win2K) was for the simple reason that it doesn't have a password by default, and that when it's used without a password it retains nothing - the whole Guest profile Folder & sub-folders are deleted at log off - even with a password win2K deletes the profile at log off but XP retains meaning that someone could use the "No Password" Guest account to do all sorts of illegal activity on the net such as surfing Klddle P0rn etc. leaving no trace on the system but in XP you set the password before the first log on and the Guest account retains everything that a normal user account retains edit: additional info

JCitizen
JCitizen

the guest account should be disabled as it is a security risk. I also disable the hidden Administrator account and/or password protect it with an impossible password(to crack) for secure purposes. Also disabling fast user switching is paramount. However this was advice for those that never use the guest account; to use it in this manner is very interesting, if secure. Since it is password protected, I can't see how it would be any more dangerous than another standard account - but then I haven't read up on the guest account features since my MCSE training.

Michael Kassner
Michael Kassner

The shared documents folder would be the downfall. What if an image file version of the Evercookie was placed there?

Who Am I Really
Who Am I Really

to retain settings, there's a feature/bug in XP that allows the Guest Profile to save settings, but only if it's configured in a specific order first, when the Guest account saves settings, you can install add-ons to Firefox and they stick if the account isn't setup that way then Firefox and everything else for that matter is started from scratch every time it's used meaning, the add-ons vanish, and it asks if you want to import anything from IE, etc. If you want the Guest account to save settings, it has to be setup with the following method: - this only works if the Guest account has never been used before, the following instructions work for XP, I don't use Vista or win7 so I can't say if it will work on either of those I know it doesn't work in Win2K a> enable the Guest account from an Administrator account b> Don't Log in to the Guest account yet, remain in the Admin. and do the following: c> open a cmd window d> type net user Guest "password" - where "password" is whatever password you want assigned to the Guest User Account e> exit the command window f> log off the Administrator account g> log into the Guest account with the password just assigned h> configure the browser(s) with whatever history settings, add-ons etc. desired i> log off the Guest account j> log into the user account you normally use and change the shortcut for your browsers, - setting them to "run with different credentials" - right click the shortcut - select properties - on the "Shortcut" tab select advanced - check the box for "run with different credentials" now every time you open the browser from the shortcut, it will ask which user you want to open the program with - check the radio button for "the following user:" and type in the user box Guest and type the password in the password box You are now browsing from the Guest account, which can't install anything or write to any folder except folders in the Guest profile and the shared docs folder