We in IT often find comfort in numbers. However, really big numbers may be fraught with invalid assumptions, or simply measuring something that's largely irrelevant to the core business problem.
If you were around during the last technology boom and bust cycle of the early 2000s you were likely exposed to the concept of really big numbers. Countless business plans for all manner of ill-conceived ideas began with "If just 0.00001% of all internet users visit our site..." and used that as the core foundation of their revenue model. While it all seemed logical at the time, the logic of really big numbers had two fundamental failings.
The first was fairly obvious once internet companies started failing. While internet adoption exceeded even optimistic models, human beings with an internet-connected computer still obeyed normal rules for consumer behavior: they would only buy a product or service they actually wanted or needed, that was effectively marketed. The really big numbers-based business plans assumed that humans would suddenly become some sort of random particle that bounced around the web, arbitrarily disgorging cash every so often-which was the only way a statement like "If X number of internet users visit our site, and Y complete a sale, we'll be zillionaires" made any sense.
The second failing of really big numbers is that they often measured irrelevant metrics, and investors were willing to suspend logic since the internet was somehow new and magical. Billions in venture capital were squandered while companies spent wads of cash to garner clicks, views, and other largely irrelevant measurements that never translated into that most relevant of metrics: money in the bank.
It seems almost quaint to look back on the dot com days, and in hindsight it's easy to laugh (or cry, if you were a poorly-timed investor) at the tech-driven folly that ensued; however, really big numbers are making a return that's just as dangerous, and once again they are being associated with technology.
One of the most obvious resurgences of really big numbers is with social media, where technologists are being tasked with tracking all manner of likes, stumbles, followers, diggs, etc. While there have been all manner of successful social marketing campaigns, there's also been a resurgence of talk that sounds eerily familiar, often starting with "If we can get [a really big number] of followers..."
I often quip that 10 billion followers/likes/etc. and five dollars will get you a cup of coffee at a global chain, and while obviously sarcastic, the sentiment is appropriate. Social media "metrics" are irrelevant if they don't translate to sales, and the newness and ready availability of these metrics cause some to forget that most companies exist to generate revenue rather than pen pals on a social media site.
Even fairly mundane aspects of IT management can fall victim to really big numbers. Many an eight-figure enterprise project has been seemingly going swimmingly, with 95.343498990% of the laundry list of deliverables completed, only to suddenly be thrown into chaos when that really big number of deliverables is found to be delivering something irrelevant or inappropriate.
We in IT often find comfort in numbers. Tracking and reporting a measurable metric lends credence to our work and limits speculation and error. However, authoritative-sounding really big numbers may be fraught with invalid assumptions, or simply measuring something that's largely irrelevant to the core business problem. A great way for IT to gain creditability is to air some of these concerns as peers in other business units investigate and integrate new technologies. IT's familiarity with metrics and measurement, combined with rational skepticism, can move a CIO toward trusted advisor status while helping avoid costly mistakes at the hands of really big numbers.