Big data is most often spoken of in light of trying to find the right metrics to measure for corporate success; trying to recruit or develop the necessary big data skillsets for and within the enterprise; and investing in big data technology that can unearth information from data for ground-breaking insights.
However, just as important are the relatively unsung areas of big data risk that few organizations are giving much thought to as they hurl themselves full speed into a big data "arms race." These unsung risks are embodied in the changes that big data is going to make to the people and the practices within organizations and whether enterprises are really going to end up with better or more clouded insights as a result of their big data forays.
The dilemma was recently touched on in The Wall Street Journal in a big data article by John Jordan (WSJ subscription required):
"In our rush to embrace the possibilities of big data, we may be overlooking the challenges that big data poses — including the way companies interpret the information, manage the politics of data and find the necessary talent to make sense of the flood of new information. Big data, in other words, introduces high stakes to the data-analytics game. There's a greater potential for privacy invasion, greater financial exposure in fast-moving markets, greater potential for mistaking noise for true insight, and a greater risk of spending lots of money and time chasing poorly defined problems or opportunities. Unless we understand, and deal with, these challenges, we risk turning all that data from something that has the potential to enhance our organizations into a diversion, an illusion or a paralyzing turf battle."
The WSJ talked about people who "may try to game the system — to the detriment of the company." It cited the example of a company that goes from a successful sales lead development program achieved through trade shows and conferences to the measurement of sales lead success through a tracking of Twitter mentions, website clicks, and social media traffic reported through high-level dashboards with an ultimate consequence of losing profits.
So if big data isn't the right solution for every situation, where are the risks of not managing where you use big data greatest, and what steps can you take to avoid overexploiting big data to the company's detriment? Here's my take.
1: Big data is not a perfect art or science.
Big data and analytics have been used by universities and research institutions for decades, but the nature of scientific inquiries vs. hard-to-solve business and market questions is decidedly different. Technology providers must also deal with the very different needs of enterprises, along with the shorter times to results that impatient stakeholders expect.
Therefore, it comes as no surprise that tools and methodologies for big data in business are far from perfect, so they should be used judiciously.
2: It's easy to lose the forest in the trees.
There's a lot that has already been said about the difficulties enterprises experience when it comes to knowing how to query big data for the insights they seek. Another equally challenging feat is uncovering the right combinations of data that are capable of producing these answers. There is not an exact art or science that can do this, either.
3: Important data is not necessarily big data.
WSJ's sales lead example is well taken. There is so much anxiety within companies to take advantage of big data that they risk finding themselves prematurely abandoning tried and true practices that have yielded results for new practices captured in dashboards and drilldowns that don't perform nearly as well.
Big data can produce monumental insights, but it's not the answer for every business situation.
4: Companies should proceed thoughtfully in their big data efforts.
Big data and the ability to exploit it will pay big for enterprises that spend the time to determine: where and how it works best; who within their organizations should be working with and using it; and which technologies can get them to their goals. These enterprises will also have an understanding of the areas in which big data should play minor roles — if it plays at all.
Mary E. Shacklett is president of Transworld Data, a technology research and market development firm. Prior to founding the company, Mary was Senior Vice President of Marketing and Technology at TCCU, Inc., a financial services firm; Vice President of Product Research and Software Development for Summit Information Systems, a computer software company; and Vice President of Strategic Planning and Technology at FSI International, a multinational manufacturing company in the semiconductor industry. Mary is a keynote speaker and has more than 1,000 articles, research studies, and technology publications in print.