Big Data

Watch out: Don't go down the wrong big data path

Predictive analytics sometimes lead to project failures. Get faster analytics solutions and be more comfortable with the results by creating a big data ecosystem.

Image: iStock

Michael Schrage, a research fellow at MIT Sloan School's Center for Digital Business, wrote an article in the September 2014 edition of the Harvard Business Review that discussed the value of learning from analytics project failures.

In one case, Schrage described how a major industrial products company made a huge predictive analytics commitment to preventive maintenance to identify and fix key components before the components failed so the firm's limited technical services talent could be optimized. Halfway through this very expensive data collection and analytics review, a couple of the repair people observed that, increasingly, many of the subsystems could be instrumented and remotely monitored in real time as part of a networked system. The insight changed the direction of the entire project, and Schrage observed that "The value emphasis shifted from preventive maintenance to efficiency management with key customers....The predictive focus initially blurred the larger vision of where the real value could be."

Many corporate analytics efforts are like this. Companies begin by asking questions, and those questions in turn direct them into certain directions, but sometimes that's at the expense of other directions that might have yielded greater results.

It is this fear of getting on the wrong track of data analysis that has corporate data scientists and analysts moving carefully and issuing numerous disclaimers about what results might yield along the way. This is also why many of these corporate practitioners look to outside business partners and consultants for help with their analytics, resulting in trusted business partners and consultants becoming key influencers in a company's IT direction, especially in newer fields like analytics.

There is a message here for analytics and big data vendors, too: They can help speed the adoption of their analytics solutions and their clients' comfort levels if they develop an ecosystem of business partners and consultants in key industry verticals. The members of this ecosystem understand the specific information challenges of their industry verticals, and bring in analytics best practices and use cases that build their clients' confidence and abilities in these areas.

A next step for analytics providers is to construct such an ecosystem.

In 2014, the research firm Lavastorm surveyed 495 C-level executives, business analysts, data scientists, and analytics professionals. The survey revealed that 75% of businesses had not yet successfully deployed analytics to the point where they were obtaining insights that were impacting their businesses, although 65% had increased their analytics investments.

"Yes, businesses are more committed to implementing big data analytics than ever before, but, far too many are still struggling with how to maximize the benefit," said Drew Rockwell, Lavastorm's CEO. "These survey results underpin how investing in analytics is just the first step. It's organizations that go the next level by removing complexities from the analytics process and empowering others in the organization, namely business analysts, that are going be able to turn data insights into actionable business enhancements for long-term success."

The good news is that more and more companies are providing ways to abstract the complexities out of big data and analytics with semantic and reporting tools that sit on top of "raw" processing engines like Hadoop and enable end users and data analysts to easily access and manipulate information to gain business results. This effort is further amplified and expedited when expert partner-implementers are called into the process.

Also see

About Mary Shacklett

Mary E. Shacklett is president of Transworld Data, a technology research and market development firm. Prior to founding the company, Mary was Senior Vice President of Marketing and Technology at TCCU, Inc., a financial services firm; Vice President o...

Editor's Picks

Free Newsletters, In your Inbox