Gartner published in July 2013 its revised hype cycle on big data. This is a
timeline that predicts the phases that companies will go through as they try to
make their big data work for them.

In the Gartner
model, businesses will first enter a stage where they will innovate and experiment
with big data. This will be followed by a period of optimism and very high expectations
from IT and business management.

Gartner’s next predicted phase for
companies is a period of disillusionment. This is when companies are expected to take a “time out” from big data pursuits. Companies will begin to question whether they are getting all of the competitive-advantage information that they
expected. It is also during this period that companies
will regroup and revisit what they are trying to do with big data, asking whether the business is on the right path. As a consequence of this introspection,
companies will make adjustments to their methodologies and expectations and then
launch into a new path toward big data productivity where they will
begin to see their results of their labor.

The Gartner model
is particularly interesting because it downwardly revises the estimated arrival
of business productivity from big data. The 2012 Gartner big data hype cycle projected
that companies would achieve productivity with their big data within a two to five
year timeframe. In today’s restated hype cycle model, this attainment of
productivity is estimated to be five to ten years away.

Corporate big
data project leaders don’t want to hear this news, but it’s better to
think about it now (and to reflect on what you can do to avoid it) than to continue without considering whether a delayed productivity forecast
could apply to you.

Gartner says the two main reasons for its downward big data productivity revisions for
companies are: (1) Big data tools and techniques are being acquired before
businesses have the expertise and maturity needed to use them, and (2) businesses are struggling with how to ask the right questions and
spot opportunities where they can harness big data for breakthrough insights.

The natural
question is: Do big data champions in organizations have to accept the reality
of extended time to big data productivity? The answer is
no. You can take steps to avoid sand traps that seem to center more around a lack
of organizational know-how about big data than a lack of tools or
resources. Here are six recommendations.

1: Build your business case
first

Nathan Gnanasambandam, Senior Research Analyst at Xerox, recently spoke spoke about “dream sessions” they had with
their clients. In the sessions, clients are asked to come up with questions about
their businesses that they’d always wanted to answer but never could with the
information they had available.

Building your business cases for big data doesn’t
have to be that involved, but no one
should go shopping for a big data tool and system without a clear strategic direction
on what the company expects to learn from the data.

2: Don’t get budget happy

Big data
shopping is a lot like mobile device and cloud shopping was several years ago — everyone wants bragging rights in executive summits that their company “has” big data solutions. For IT, which frequently finds itself embroiled in budget battles,
there is also the challenge to fund big data systems and tools while the budgetary
sun is shining on big data. Try to avoid
this until you have those strategic sessions with business leaders on what
you want to derive from your big data.

3: Don’t “underdesign” pilot
projects

The normal
course in an IT project using new technology is to design a small pilot project
first so you get a quick project success under your belt. The strategy should
be no different with big data.

However, you still need to design that first project
large enough so the methods you use can be applied to subsequent business
cases. If you don’t design a proof of concept that you can leverage for future
work, you could quickly find yourself (and your management) propelled into the
disillusionment phase of big data that Gartner talks about.

4: Seek help if you don’t know how to
ask the right questions

You should start looking for big data consultants and experts with
the help of your big data vendors. If your vendors can’t provide assistance in this
critical developmental area, you should reconsider partnering with them.

5: Perform a proof of concept
before you buy anything

It’s important to have your business cases
in hand before you sign with a big data solutions provider. You should also
plan to execute a proof of concept with one of your business use cases with the
vendor before you buy; this gives you and your executive team greater ability
to see whether the proposed solution is really a good fit.

6: Consider a batch vs. a real-time
big data application first

There is a
lot of buzz about “in-memory” technology and the ability to
get near-real-time analytics reports from big data. The reality is most companies simply want to expand the realm of their business intelligence.

In most cases, big data intelligence can answer key questions just as easily in
a batch reporting mode, and you avoid introducing too much complication into
your processes too soon. The added complication comes from simultaneously having
to manage real-time processing from both systems of record and your big data
analytics in the data center. Unless you work in an industry that must have real-time analytics, such as stock trading or high-volume retail, you can always
add the real-time capability later.