IT Employment optimize

Expert scores and ranks online legalese: The results might surprise you

Michael Kassner interviews an attorney who reads (yes, reads) and ranks online user agreements. Guess who came out on top and where TechRepublic ranked?

Last night, Microsoft began notifying users, of its online services, about changes to the Microsoft Services Agreement. A cursory glance might make it seem like all that's happened is some innocent redrafting. Microsoft says they've "modified the agreement to make it easier to read and understand" by introducing a "question and answer format."

But take care. There are two key changes -- and they are both bad for users.

That's from Andrew Nicol. It gives you an idea as to what he's interested in. From the sound of it, I'm thinking we may want to pay attention.

Pay attention to what?

I've written a lot about online user policies, and a common thread I've observed is everyone believes something needs to be done, but no one seems to know what that something is. Andrew didn't want to wait, so he decided to do what he thought might help. And, you might interested in what that is.

I caught up with Andrew while he was commuting through New York's Union Square, and asked him why he decided to get involved in this thorny issue.

This seemed like a great opportunity to use my knowledge of the law and my interest in technology to try to bring these issues to the attention of more people, and ultimately pressure the sites into being more reasonable.

I started by reading only Terms of Service, but then realized it made more sense to provide a comprehensive report card for each company, taking into account all of their agreements with users as well as their actual practices.

Read every word?

I thought I heard Andrew say he personally reads in entirety the Terms of Service (ToS) and Privacy Policy of the site he is examining. With a healthy dose of skepticism, I asked again to make sure.

That's right. For each site, I read all of the legal agreements that it has with its users (which normally include the terms of service and privacy policy, sometimes an acceptable use policy).

I have a master spreadsheet with the relevant provisions in each category from each site which lets me compare the site to its peers. I also spend some time investigating the site's actual practices (for example, how it responds to government data requests). Whenever possible, I speak to someone from the company about my concerns.

That is ambitious. Not long ago, I wrote an article reporting how two privacy experts determined each of us could easily spend 200 hours a year reading all the legalese that we are presented while on the Internet.

Clickwrapped

Andrew decided to share what he found with the rest of us. The result is a website called Clickwrapped. I wondered about the name Clickwrapped. I was about to ask Andrew, but thought it best to figure this one out myself. It turns out the term clickwrap has significance. This is an excerpt of the term's definition on Wikipedia:

A clickwrap agreement is a common type of agreement often used in connection with software licenses. The name "clickwrap" came from the use of "shrink wrap contracts" commonly used in boxed software purchases, which contain a notice that by tearing open the shrinkwrap, the user assents to the software terms enclosed within.

When you visit Andrew's website, the first thing you see is the following diagram:

Click the image to enlarge.

Wikipedia is worse than Google and Facebook? No way. I settled down after reading that the higher the score, the better.

Each site is scored out of 100, with points allocated equally between four categories: Data Use, Data Disclosure, Amendment & Termination, and Miscellaneous. Although scoring necessarily involves the exercise of some discretion, we try to be as clear as we can about the criteria and we explain on each review page the reasons for the score we have awarded.

If you hover your pointer over each color bar of a specific website, a new window will open with the numerical results. If you click on the same colored bar, you will be sent to Andrew's explanation of why the website received that ranking.

Results table

The following chart is for those fond of numbers:

So, Wikipedia is the best of the bunch, but self-serving Facebook and Google are right up there? I asked Andrew if the high ranking of Google and Facebook was a mistake.

Going into the project, I was aware that both of these sites had been widely criticized for failing to respect their users' privacy. A lot of this is just because of their size: they are the two most popular sites on the internet, and so it is to be expected that they get the most attention in the press.

What I found is that neither of these sites deserve all of this negative press -- at least not when they are compared to their competitors. Facebook was a surprise. It has definitely made some mistakes in the past, but right now, it ranks well on most issues.

Once again, I was skeptical, deciding to investigate how Andrew came up with the rankings. As Andrew mentioned, he looks at four categories asking the following questions.

Data Use:

  • What data does the site collect?
  • What can the site do with content you post?
  • Does the site get more rights to your content than it needs?

Data Disclosure:

  • When can the site disclose your information to others?
  • Does it tell you if the government wants your data?
  • Is it transparent about government requests?

Amendment & Termination:

  • Does the site give you notice when it changes its terms?
  • Under what circumstances can your account be terminated?
  • Does the site let you take your data to another service?

Miscellaneous:

  • What else should you know about the site's legal agreements?

I asked Andrew to take a look at TechRepublic's legal policies (officially CBS Interactive) and see what he thought. Andrew wanted me to mention that this was only a crude first-pass assessment and must be viewed as such. Can you tell he's an attorney?

Here are the results for TechRepublic:

A tie with Microsoft... .

A few more questions

I took advantage of being in contact with someone who actually reads the ToS and Privacy Policies of sites -- and, as a technically inclined attorney, understands them. I was curious as to his threshold, asking him which website policies would be acceptable and which weren't?

Like anyone else, I would find it difficult to run away from some popular sites. For example, even if I had more concerns about Facebook's terms, I would probably still be on there because I use it so much and have invested so much of my time and effort in its platform.

But, since doing this research, there are some services I am avoiding. I tell my friends to use Dropbox rather than Google Drive. And I now avoid PayPal and use other online payment services.

Next, I wanted to find out what it would take to obsolete the Clickwrapped website?

This is a great question. Clickwrapped will no longer be needed when there is a fair balance between company rights and user rights. There are a few simple actions that each company could take to achieve such a balance.

  • They should limit how they use our data.
  • They need to be more reasonable about handing over our personal information to the government.
  • They need to go back and take a look at their agreements and eliminate any terms that are unnecessarily one-sided. For example, they should not be allowed to close our accounts at any time and for no reason.

Final thoughts

As for agreeing with Andrew's opinion about ranking or not, at least there is some digital discussion going on. That has to be better than automatically agreeing. I also find Andrew's honesty about using Facebook regardless of its ranking refreshing.

About

Information is my field...Writing is my passion...Coupling the two is my mission.

23 comments
dimonic
dimonic

Is he avoiding apple through fear of being sued?

bboyd
bboyd

Ok I accept that Facebook(tm) has improved its ToS and other privacy documents but its policy of whole scale exporting of data to "partners" that have no such restrictions leaves me leery that just a baseline score may be misleading. I also accept that they have somewhat limited the government requests for information but they are so open that agencies are importing wholesale great swaths of FB data to mine that it obviates the improved policy. So where does that leave me. I still think the real worst offenders are the credit card companies who sell all our purchase information to their "partners" and undoubtedly reveal just about anything the government agencies desire.

K_Green
K_Green

I can't believe I'm going to write this ... this **attorney** is my hero. I love detail, and the analysis of the same. Looking at his site, the 'About' page succinctly explains what motivated him to do this. He got screwed by a simple disclaimer on a baggage ticket. Plus, it appears he is doing this pro bono. Nowhere on his site does it mention how to contribute to his effort. I wish him every success.

Ambrose3987
Ambrose3987

Nobody will be surprised to see Paypal near the very bottom. And nobody should be using it.

vhurley22
vhurley22

I would love to see an analysis of the terms for Angie's List. It's the first place I have seen them say that if your credit card expires for the auto renewal, you give them the right to contact your bank to get your current credit card information from them! Wow, how brazen. To add to it, you can't sign up without doing the auto renewal! I was stuck on a page trying to purchase the subscription for a short time without doing the renewal. I called them about it and they said you had to sign up but then right after you could contact them to take it off auto renewal. No thanks, Angie's List!

sklauminzer
sklauminzer

How does this compare to http://tos-dr.info/ ? They are attempting a similar ranking, but using school type grading A-F. I'd love to see sites thine the work together to spread the knowledge.

Craig_B
Craig_B

This helps with the comparison and conversation of how these policies are written and I'm thankful for that. My concern is that I'm not sure if this will really make a difference in the big scheme of things. That is will the average user be able to easily understand the policies and make informed choices? Maybe this is just another step on the journey.

Michael Kassner
Michael Kassner

I'll pass your question along to Andrew and let you know what he says.

Deadly Ernest
Deadly Ernest

Mundia.com as they make some of the bad sites listed here look like nice guys.

etafner
etafner

That is quite interesting, it's something that I had never seen. Thanks for the tip.

aknicol
aknicol

I agree that giving user data to third parties when that data won't be subject to the same controls is obviously a major privacy issue. But Facebook's platform actually doesn't quite work that way -- for example, it doesn't actually share user information with advertisers, it just lets advertisers target specific demographics. (The targeting is done by Facebook, not by the advertisers themselves.) Similarly, if you plug a third party app into Facebook, I'm not sure that it makes sense to blame Facebook for sharing your data. It's up to you whether you give the app the necessary permissions or not.

Michael Kassner
Michael Kassner

I think what we need to do is look at our expectations and then at what Andrew determined. The websites may not be close to what we feel is correct, but in the scheme of things, Wikipedia is the best -- followed by the others.

Michael Kassner
Michael Kassner

Andrew is swamped right now, but he wanted me to thank you for your kind words and mentioned that you should stay tuned as he is not done yet.

Michael Kassner
Michael Kassner

I personally did not check out PayPal's policies, but Andrew is thorough.

aknicol
aknicol

I'll definitely take a look at including the Angie's List terms in a future update. Thanks for the comment! -- Andrew Nicol

Michael Kassner
Michael Kassner

Andrew said he would check the comments when he had a chance. I've seen several sites like that. I guess they are hoping people forget to disable auto-renew.

Michael Kassner
Michael Kassner

I was made aware of the site after the article was live. I certainly am going to check it out and see how it compares.

Michael Kassner
Michael Kassner

Andrew wants to force the simplification of wording and elimination of what he calls the asymmetrical contract

Michael Kassner
Michael Kassner

I am curious as to how you ranked the websites? Did you agree with Andrew?

aknicol
aknicol

Thanks for pointing out the Mundia ToS. Their data use provision is certainly very broad (and concerning given that their users could contribute a lot of personal information to their platform). I'll try to cover the terms for genealogy sites like these in the future. -- Andrew Nicol

Deadly Ernest
Deadly Ernest

are really big on getting people tom put personal info up on their site. I do volunteer work at my local Family History Centre and I also have experience in IT security, so I do check up on these things and the Mundia ToS scared the hell out of me.