Leadership

The T-Mobile-Microsoft Sidekick data disaster: Poor IT management going mainstream

ZDNet's Larry Dignan: With cloud computing increasingly reaching the masses, the average consumer will soon be enmeshed in the world of poor IT management. Welcome on board the IT failure express.

If you're a T-Mobile Sidekick user who put data, contacts, and other items in the cloud we have some bad news for you: Your data is gone...probably forever. With cloud computing increasingly reaching the masses, the average consumer will soon be enmeshed in the world of poor IT management. Welcome on board the IT failure express.

T-Mobile depended on Microsoft's Danger unit to provide data services. Danger's servers blew, and apparently there weren't backups. Oops. T-Mobile's forums reveal (Techmeme):

Regrettably, based on Microsoft/Danger's latest recovery assessment of their systems, we must now inform you that personal information stored on your device -- such as contacts, calendar entries, to-do lists or photos -- that is no longer on your Sidekick almost certainly has been lost as a result of a server failure at Microsoft/Danger. That said, our teams continue to work around-the-clock in hopes of discovering some way to recover this information. However, the likelihood of a successful outcome is extremely low.

T-Mobile then sends you to an FAQ so you can rebuild your contacts. In a nutshell, Microsoft/Danger can't recover your data. Granted, it's only photos, contacts, and the rest of your life, but it could have been worse I suppose. Here are a few observations from this debacle:

  • Poor IT management is going mainstream. As we rely on the cloud more there will become a day when everyone will have some basic knowledge of IT management. Rest assured, Sidekick customers will know you're supposed to back up your servers better. Gmail customers may learn a bit about scalability. And TD Bank customers certainly know that you can't merge systems without a fallback plan if things go awry.
  • Cloud is nice. Hybrid may be better. A local backup of data is a necessary backup to the cloud. After all, photos, contacts, and other items could be summed up in two words: your life. T-Mobile's hybrid approach: Don't reset your phone.
  • These IT snafus will become more public. In many cases, IT failures happen behind the scenes. IT failures usually ding a company's financial results or operations, and there's a good bit of finger pointing (the consultant, vendor, customer loop). Customer-facing applications usually don't blow up in such spectacular fashion. Today, all applications are becoming customer facing.
  • Corporate reputations will partially be based on IT management skills. The number of IT failures and a company's reaction to IT screwups will feed into a corporation's reputation.

IT failures make our little world go around-Michael Krigsman (our IT Mr. Sunshine) even has a blog dedicated to them -- and now you're on the IT failure express too. Welcome.

5 comments
markinct
markinct

I really hope T-Mo, the FCC or some enterprising states Attorneys General force MS/Danger to disclose a play-by-play of what went wrong. Maybe it will come out in the inevtitable law suits... I think morally and legally T-Mobile is responsible. They owe their customers a detailed explanation. In this day and age such an outage and data loss is incomprehensible and unforgiveable. It reminds me of something a Veritas/Symantec netbackup genius once said - something like 'backups can cost you some sleep, restores can cost you your job'. (Not his words exactly, but you get the idea.)

psmith
psmith

Why be so negative? Just think how much money was made by cutting corners and being as cheap as possible! Like grandpa always said "Any job that ain't worth doin', ain't worth doin' well!", right?

nittmann
nittmann

awesome quote! that could be seen as an important and very logical corollary to the old "good work ain't cheap, and cheap work ain't good"... love it. added it to my treasure...

mpark71045
mpark71045

Since most of the recruiters and HR people hiring for IT positions are complete idiots about IT then they most likely hire idiots.

nittmann
nittmann

Poor IT management is not only visible in bad configurations, e.g. it took T-MObile over 2 years and a constantly updated BBB complaint to finally correct their SMTP header violation for some phone emails - It also shows in poor judgment in choosing service providers. Danger never had the infrastructure like EC2 (Amazon) or other cloud computing services, I doubt it was a cloud ever. Had they visited Danger's data centers (as to my knowledge they actually do not own a data center....) they could have seen lack in process... I bet it was there, otherwise this disaster would not have happened. Auditing major providers is what most outsourcing projects skimp on: why spend money or time on due diligence... and risk that one's pet recommendation gets shot down (ego! the primary cause of IT failures: the less real knowledge and experience, the more ego and defensiveness! Too many people think that when they can cause a flicker on a screen they are an 'expert') It is not cost, it is poor management not setting the correct priorities to assure budget allocation for the things that are important. ... going with the lowest bidder brings you right there ... Mike