Project Management

Web 2.0 and contextless data

16 comments
Wayne M.
Wayne M.

The practice of obtaining nearly meaningless numbers and passing them off as statistics or metrics has a long history prior to Web 2.0. Before gathering a number, first determine what the number is to highlight and second determine how the number is to be evaluated. If Monty Python were around today they would have the "Department of Silly Numbers" and this segment would show some of the monthly software development status reports I have had to develop to comply with our processes.

Justin James
Justin James

But the problem seems to be particularly pronouced with "Web 2.0". Or maybe that is my inborn disdain of most "Web 2.0" things kicking in and noticing it more. J.Ja

tc
tc

If you are looking for humor read my other post. If you obviously have to much time to kill read this one. J.Ja the word is "Sin?ah-Cal" (houked on fonix werked fore mee tu) Did we hit a brick wall with "Bigger, Better, Faster and More?" Maybe this time all we got was Faster (or BIGGER if Microsoft gets inv. . . nah, never mind) Maybe we didn't get anything this time ! In realidad, maybe Web 2.0 is nothing more than another buzz word to help me keep earning a paycheck. If your vocabulary lacks enough fuzzy meaningless words to ask for the big money, go check this link and double your perceived value to either your boss or your clients - http://en.wikipedia.org/wiki/Web_2 A quick course in cynical oooops I mean simple economics - Tim O'Reilly coins a term to sell me yet another 50-60 dollar book. I buy the book to gain another corporate client whom I bill $100 ? 200 + per hour. (Hey, I didn't want to look "dumber" than my competitor when I bid the job) My client buys the book to figure out what I just said; he doesn't want to look stupid in front of all the others at the next board meeting. He doesn't have a clue what he read but he lets a few words roll off his tongue and they appoint him the project manager because he appears the best versed on the subject. Nobody else at the table dares to reveal that they are clueless to what he is talking about. Nobody dares state the obvious "Look mommy, the emperor has no clothes." If you didn't recognize that famous line from a children's story, you probably grew up with video games, thought microwaves and VCR's have been around as long as cars. . . Can you say "Hans Christian Andersen? if not, wikipedia can help us yet again http://en.wikipedia.org/wiki/The_Emperors_New_Clothes (And just for the record - Al Gore did not invent the internet, and President Clinton had clothes he just forgot where he left them) Are you laughing yet? I was 24 years old when I first got to wear the "hat and title? 2 days earlier my friend's dad was telling us about touring a plastics museum and the different ways they made plastic as we drove up to a ski resort in Northern Maine. That Monday morning the company's founder was describing a major automation project for a large - are you ready - plastics manufacturer. I was dumb back then, I asked stupid questions like what type of plastic - injection or vacuum molded, shell-sol - Stupid me, and less than 5 years later I was earning more money than I ever dreamed, working in the engineering department at - - - - - - ?the plastics manufacturer?. The older I get, the more I see the quiet wisdom of my granddad - he may have been one of the last; a generation of self sufficient, self reliant, homesteaders. He never went beyond 8th grade yet his wisdom and self sufficiency bordered on genius. When I was young I thought he was foolish. I was a bookworm due to the lack of television and or tv stations that could reach those parts of the Adirondack Mts. back in the early 60's. I would run to tell "grandpa" something fascinating/horrifying/ludicrous/. . . that I had just read and he would just shake his head with a frustrated look. He would tell me that "a blank sheet of paper will lie there and let any uninspired fool write what they will on it." 35 years after his death, the internet proves him correct on a daily basis. Hey gramp - can you hear me? I just found out they still homestead a little south of me here in Mexico. Is 46 to old to become a self sufficient farmer? Just one more corporate project and I can buy a huge ranch with a donkey to plow (Yeah, they got John Deere and Caterpillar here, they make them just 45 minutes from where I sit - I could even fly 65 Victor in and out of one of the pastures, but then I would need to earn money for gas, registration and insurance.) Anybody want to buy a laptop with both english and spanish keyboards ? Wait, my skype is ringing, that must be my prospective client in Florida calling back, got to run . . . . . RIP Arthur Allen C (1900 - 1973) If you could see the world now gramp Tim "speedy" C - coding faster than a Mexican Bullet - (I hope)

Wayne M.
Wayne M.

Though I share much of your disdain of Web 2.0, I have to point out that not long ago almost every web page had a "This page has been viewed XXXXX times" counter. This was coupled with "Most Popular Pages" directories that ensured more and more hits to the pages with more hits; a self reinforcing cycle. I guess I am frustrated because I believe good statistics are valuable, but I repeatedly see people collecting and reporting almost meaningless numbers.

mdhealy
mdhealy

From the Perl FAQ: http://perldoc.perl.org/perlfaq5.html > Here's a much better web-page hit counter: > $hits = int( (time() - 850_000_000) / rand(1_000) ); > If the count doesn't impress your friends, then the code might. :-) Even if you don't know Perl, the "rand()" call should give you a clue what it does...

Justin James
Justin James

I had a page counter on my first Web page. :) Your comment on the "most popular pages" problem is one I 100% agree with... today it is done through link popularity. If someone can't find the site, it will not get linked to, so it will never been found. The same goes for things like those RSS/blog sites, Digg, etc. etc. etc. It is a total Catch 22. The only number worse than "page views" is "hits". I can double my "hits" by splitting every image on a site into two separate files, since each one is a "hit". On that note, beware of outsourced Web apps that measure response time in "hits" vs. "page views", since dinky stle sheets, .JS files, images, and other things that require zero processing go quick... each page could take a full minute to load, meanwhile the average time per "hit" is low. J.Ja

Justin James
Justin James

What do you do to make sure that the data you display is helpful and accurate to your users? J.Ja

mdhealy
mdhealy

Spend some time browsing http://www.edwardtufte.com/tufte/ Buy his books. Read, learn, and inwardly digest them.

Justin James
Justin James

... but never read anything he wrote, to the best of my recollection. I'll try to get a chance to take a look at it. Right now, the "to read" shelf of my bookcase has about 20 items on it already... J.Ja

Justin James
Justin James

Yeah, I saw that chart ages ago, it really is an excellent display of data! I will try to pop that onto the top of the stack, not the end if the queue. ;) J.Ja

mdhealy
mdhealy

I'd start with The Visual Display of Quantitative Information, then go on to other books as time permits. His most stunning example is a nineteenth century chart of what happened to Napoleon's army when it invaded Russia. http://www.edwardtufte.com/tufte/posters

Justin James
Justin James

If I'm going to displace "Noble House" or "Whirlwind" (I recently discovered James Clavell, "Shogun" was amazing and so was "King Rat"), which Tufte book do you suggest? J.Ja

mdhealy
mdhealy

> Right now, the "to read" shelf of my bookcase has about 20 items I know the feeling, but think Tufte's stuff deserves to be near the top of your stack.

mosw
mosw

It is just as easy to over analyze the data. By the time you are finished the data may say whatever the analyst wants it to say. For example I might only be motivated to vote when I disapprove, so all my votes are "thumbs down", that does not make my votes invalid. Others might only be motivated to vote when they approve. So the analysis itself must first be statistically tested not just a "gut feeling" to be valid. Way to much work for me. In most cases I think it is better to show as complete a picture as possible and let the user interpret (misinterpret) as best they can. So show the number of views along with the votes, and don't take it all to literally. I had a teacher that demonstrated how statistics can be manipulated with this hypothetical example: "40% of all accidents are caused by drunk drivers which means 60% are caused by sober drivers. Lets get those sober drivers off the road!". I don't normally vote but I gave you a thumbs up!

Justin James
Justin James

That is a great example of how a lack of context creates bad numbers! A more valuable number would be something like, "1% of drivers are drunk, but they are responsible for 40% of accidents" or rephrased as "drunk drivers are 6,000% more likely to be the cause of an accident". You gave a pretty raw number (simple percentage) and the logical deduction (sober drivers are more dangerous than drunk drivers) is obviously false. J.Ja

tc
tc

After further research of your 40%/60% hypothesis, I just came from the police station where the Mexican "Transito" police are now going to start issuing open beers to every sober driver they stop that lacks one. Further study revealed that 6 out of every 100 fatal accidents involved large busses and tractors towing double trailers. So now, anyone renewing their license that has not yet been killed at least once in a fatal accident will be required to change from a standard license to a commercial one, and will be required to drive one or the other for at least 5 years. This is all a joke of course, but look at the useless "Statistics" we get bombarded with daily, and more frustrating is to look at the miniscule amount of data collection on which we are asked to believe these opinions. It only gets boiled down to the existing opinions of the data collection unit and what was their objective. As an avid photographer that eventually developed enough skill to help pay some of my tuition in engineering school, I learned that even a picture can be made to "lie" merely by where you take the photo. (and that was 15 years before they invented image processing programs) So just give me the raw data and let me prove what I already knew, don't skew my percieved truth with the reality of your percieved facts. Thankyou very much but keep your rapid statistics "2.0" Besides, most people can not remember numbers and for that reason 85% of all statistics are made up on the spot - GRIN. This one immutable statistic you must never forget. It comes from an older gentleman and business mentor. He claimed to have learned it from a psyciatrist during his early, stress filled years when he was starting his now lucrative business. This supposedly comes from real scientific analysis but don't ask me to prove it other than it has been mostly true in my life. 92% of what you worry about will never happen, 6% of what you worry about, you can not change. That leaves only 2% valid for further expenditure of brain cells. Therefore, since you can not tell me which 2% needs further worry, quit worrying and go make or sell something Timo "Speedy" C - able to deliver complex computer code faster than a speeding Mexican bullet - - - I hope (grin)