Define these two points before choosing your big data approach

By drilling down on two defining points of your big data strategy, you'll be able to gain the business results you want from your project.


Image: Picasa

Now that organizations have their big data projects underway, it's time to review big data strategy and the types of projects that are likely to deliver the highest business value. Part of this process should include a review of what big data is, and the various approaches that companies can choose from in order to obtain the business results that they want.

Defining point #1

The first clarification that is in order concerns big data.

We have come to define big data as unstructured or semi-structured data that comes in from man and machine—and most often, from the Internet of Things (IoT), the web, and the world around us. Historically, this data was considered "noise" by traditional systems of records, which processed only fixed length transaction records.

It is easy to think that today's big data has no room for systems of record containing transactions with fixed record lengths, but in fact, systems of record play an integral role in big data processing and under the right conditions, might be considered big data if the files they contain are voluminous and require significant processing resources.

Even when these records are not considered big data, they play a major role in most enterprise analytics efforts, because they become analytics indices (e.g., customer records, inventory records, etc.) that are frequently used to probe bodies of big data in the course of answering analytics questions. The end result is that both unstructured and semi-structured data and traditional systems of records play major roles in most big data initiatives.

Defining point #2

The next defining point for an enterprise's big data projects is whether the goal is analytics or business intelligence and process automation/improvement.

Most companies start their big data initiatives with analytics projects that take a body of information (like sales transactions and web activity of customers over the past six months) and then develop a series of analytical questions to learn more about these consumers and their buying patterns. The effort culminates in a series of reports that can tell marketing and sales what the most optimal time is to launch a promotion for a given product.

In the supply chain, analytics can be run against data to answer other "what" questions such as, what areas of the world are most likely to experience a disruption like a weather event, political unrest, or a strike that could potentially disrupt the supply chain? Gaining predictive information like this enables supply chain managers to diversify their supplier bases and to preempt factors that could threaten their products' time to market.

If big data analytics is capable of answering many of the "what" questions, there are enterprises that want even more from their big data. These enterprises want big data probes to answer not only "what" but "how"—and they want to infuse the "how" results as business intelligence that is capable of automating their work processes and even making decisions without human input.

Gartner recently reported that by 2016, 25 percent of large global companies will have adopted at least one big data use case for fraud or security detection.

The fraud detection systems used by banks and payment processors are "old hat," and have been in place for many years. But by definition these systems can be considered big data because they apply business intelligence and they process enormous volumes of data in real-time, analyzing this data for usage patterns. These systems "learn" about their card-carrying customers' buying patterns. If an individual makes nearly all of his purchases within 100 miles of Chicago, with a majority of purchases that are $200 or less, and a charge for $800 in Moscow shows up, he is likely to get a call to confirm that he did, indeed, make the charge.

The account representatives who make these calls are part of an integrated workflow that uses the business intelligence and decisioning of the fraud detection system. For companies using these systems, the systems are much harder to build than simple analytics, but they have the ability to deliver sizable returns. They also guarantee that a given business process that is especially complicated or sensitive will be executed the same way every day by employees.

The bottom line

There are different types of big data and different types of big data projects. It is important to know where you want to gain improvements from big data so you can determine which approach (or mix of approaches) is best suited for your organization.



By Mary Shacklett

Mary E. Shacklett is president of Transworld Data, a technology research and market development firm. Prior to founding the company, Mary was Senior Vice President of Marketing and Technology at TCCU, Inc., a financial services firm; Vice President o...