Now that organizations have their big data projects underway, it’s time to review big data strategy and the types of projects that are likely
to deliver the highest business value. Part of this process should include a review of what big data is, and the various approaches that companies can choose from in order to obtain the business results that they want.
Defining point #1
The first clarification that is in order concerns big data.
We have come to define big data as unstructured or semi-structured
data that comes in from man and machine—and most often, from the Internet of Things (IoT), the web, and the world around us. Historically, this data was considered “noise” by traditional systems of records, which processed only fixed length
It is easy to think that today’s big data has no room for
systems of record containing transactions with fixed record lengths, but in
fact, systems of record play an integral role in big data processing and under
the right conditions, might be considered big data if the files
they contain are voluminous and require significant processing resources.
Even when these records are not considered big data, they
play a major role in most enterprise analytics efforts, because they become analytics
indices (e.g., customer records, inventory records, etc.) that are frequently used
to probe bodies of big data in the course of answering analytics questions. The end result is that both unstructured and semi-structured data and traditional systems of records play major roles in most big data initiatives.
Defining point #2
The next defining point for an enterprise’s big data projects
is whether the goal is analytics or business intelligence and process
Most companies start their big data initiatives
with analytics projects that take a body of information (like sales transactions
and web activity of customers over the past six months) and then develop a
series of analytical questions to learn more about these consumers and their
buying patterns. The effort culminates in a series of reports that can tell
marketing and sales what the most optimal time is to launch a promotion for a
In the supply chain, analytics can be run against data to
answer other “what” questions such
as, what areas of the world are most likely to experience a disruption like a
weather event, political unrest, or a strike that could potentially disrupt the
supply chain? Gaining predictive information like this enables supply chain managers
to diversify their supplier bases and to preempt factors that could threaten
their products’ time to market.
If big data analytics is capable of answering many of the “what” questions, there are enterprises that want even more from their big
data. These enterprises want big data probes to answer not only “what” but “how”—and they
want to infuse the “how” results as business intelligence that is capable of
automating their work processes and even making decisions without human
Gartner recently reported that by 2016, 25 percent of large global companies will have adopted at least one big data use case for fraud or security detection.
The fraud detection systems used by banks and payment
processors are “old hat,” and have been in place for many years. But by definition
these systems can be considered big data because they apply business
intelligence and they process enormous volumes of data in real-time, analyzing
this data for usage patterns. These systems “learn” about their card-carrying customers’ buying patterns. If an individual makes nearly all of his purchases within 100
miles of Chicago, with a majority of purchases that are $200 or less, and a
charge for $800 in Moscow shows up, he is likely to get a call to confirm that
he did, indeed, make the charge.
The account representatives who make these calls are part of
an integrated workflow that uses the business intelligence and decisioning of
the fraud detection system. For companies using these systems, the systems are
much harder to build than simple analytics, but they have the ability to
deliver sizable returns. They also guarantee that a given business process that
is especially complicated or sensitive will be executed the same way every day
The bottom line
There are different types of big data and different
types of big data projects. It is important to know where you want to gain improvements from big data so you can determine which approach (or mix of
approaches) is best suited for your organization.