There is a lot of agreement out there that Big Data remains msunderstood in some business circles. Last fall, IBM and the University of Oxford cooperated on a survey of 1,061 companies around the world. The survey results shed some light on where Big Data projects stand across multiple industries. Twenty-eight percent of those companies responding to the survey were piloting or implementing Big Data projects.

The IBM/Oxford survey divides corporate Big Data adoption into four phases:

  • Educate
  • Explore
  • Engage
  • Execute

Now that the survey has been out for a while, I thought it was time to get some reactions from industry experts.

Educate

The Educate phase focuses on knowledge gathering and marketing observations. Unfortunately, it’s the stage where many corporations remain, even months after results of the IBM/Oxford survey went public.

Olly Downs, data scientist and senior vice president with Globys, a Big Data firm, didn’t find many surprises in the Oxford Survey and  told me he is seeing more organizations completing the Educate phase.

The key to the Educate phase is developing a crisp vision for your organization’s application of Big Data. While the Educate phase is all about research, any organization wanting to integrate Big Data into their business has to take definite ownership in this phase. Otherwise, they risk never exiting this phase.  Conflicting messages about Big Data – sometimes driven by vendors or self-styled pundits – can make this phase more difficult as companies seek to gain a base level of knowledge in Big Data so they can figure how it will help their business.

Explore

After the Educate phase comes the Explore phase. This is where an organization develops a strategy and Big Data roadmap based on their particular business needs and challenges.

Based on recommendations from Downs of Globys and some of the other experts I spoke with, the Explore phase is the time to do your due diligence on Big Data tools and seek out the right tools for your organization.

I got some interesting insight about the Explore phase from Michael Hay, Vice President of Product Planning and Sara Gardner, Senior Director, Software Product Marketing of Hitachi Data Systems (HDS).  Neither saw anything too earth-shattering in the Oxford Survey but mentioned some points from their own body of Big Data work that companies need to address during the Explore phase:

  • Big data is a much broader discussion than any one particular type of technology or data.
  • The average enterprise company is hoarding data and trying to get more insight from it.
  • Some aspects of big data like volume and diversity (or variety) are not new, but new capabilities like real-time analytics are adding complexity.
  • There is still a lot of uncertainty around Big Data.

The HDS executives point to the reality that a company’s ownership over a Big Data vision is necessary to proceed to a successful Big Data implementation.

Russ Kennedy, VP of Strategy of CleverSafe, a provider of geographically dispersed storage solutions adds that storage strategy should also be a major milestone in this phase. CleverSafe works with customers through all four phases to understand their needs, help them develop a storage strategy, implement the solution, and then work with their clients to refine the strategy.

Insights from Downs of Globys, Gardner and Hay from HDS, and now Kennedy from CleverSafe, point to some enterprises being further along the path to Big Data than others. Tough questions about business problems are going to persist through the phases with many becoming known during the Explore phase.

Engage

Phase 3, the Engage phase, is where organizations pilot Big Data initiatives to validate business requirements and value. This phase is where the developing and testing of Big Data tools, processes, and methodologies meet reality in the form of proof of concepts and pilots.

Based on the feedback I’ve received, just a minority of companies have made it to this phase on their way to a Big Data implementation.

Execute

The Execute phase is where companies have deployed two or more Big Data initiatives and continue to focus on their use of advanced analytics.  Based on his own experience, Downs sees that the execution phase might include perhaps 10% of the companies who’ve embarked on a Big Data path.

The Execute phase seems to be the domain of larger players such as multinational companies. For example, HDS points to multiple Big Data implementations throughout their product and services portfolio. However, their implementations have been in support of data intensive operations such as heavy mining, transportation (think large aircraft and commuter trains), and heavy construction equipment that generate massive amounts of machine data versus the customer based data we commonly associate with Big Data.

Big Data through the phases

Companies need to chart their own course through these phrases as business and technology discoveries dictate and play an active role in the formation of their Big Data strategy to ensure its overall success.

The four stages are a good observation but, at the end of the day, you have to start with the business problem that you decompose iteratively into what algorithms and environment you build for customers.