Farm out big data chores so employees can focus on analytics

Data chores -- gathering, cleaning, and sorting -- can be done by an outside firm, so your employees can turn their attention to analytics.

Image: iStock/gvictoria

Most enterprises are about five years into their big data strategies; they have cleaned and sorted most of their internal data, and they are overwhelmed at the fire hose of data that pours in from the internet and other outside sources. Plus, companies find it a major challenge to stay on top of enterprise data because of the rate of change in businesses and the individuals working in them.

This is where companies such as Avention provide value, because they collect business data from myriad sources and then clean, normalize, categorize, and sort the data to ensure it is accurate and up to date. Avention processes this data in multiple data centers in the cloud, using applications that track hundreds of thousands of companies worldwide.

"We turn the crank daily on this business information, processing matches, updates, and all of the latest incoming business data from sources like Reuters, Associated Press, and Lexus Nexus, as well as from numerous news articles and social streams," said Hank Weghorst, Avention Chief Technology Officer. He added, "We meet with enterprise data management groups to assist them in harnessing their raw business data. We do this by cleaning the data, storing it, and then constantly keeping the data fresh and up to date."

In an enterprise business data collection scenario, business and customer data is frequently stored on in-house systems, with updates coming in from the cloud. The cloud-based data coming in from the outside is then blended with business and customer data that already resides on the enterprise's enterprise resource planning (ERP), customer relationship management (CRM), point of sales, and supply chain systems.

The value proposition for enterprise data analysts and scientists is readily apparent: the quality and confidence levels of business-related data are improved without internal personnel having to do the data sleuthing, cleaning, normalization, categorization, cross-checks, and updates themselves. Consequently, these high-priced specialists can turn their focus to the job of defining the analytics that they want to run against this higher quality data. The data hygiene service can either be fully performed by companies such as Avention, or the enterprise can opt for applications that it can run directly and that can accomplish the same thing.

"What cleaner and more complete data enables companies to do is to better align their services and product offerings with their most important customers and business partners," said Avention's Weghorst. "In many cases, enterprises find that 80 percent of their business is coming from 20 percent of their customers."

A finer layer of analytics can be applied on top of this data that gets into the details of a particular business, such as whether it is minority-owned, or the square footage of its facilities. "On a consultative level, we will go into an enterprise and help it figure out what it wants to target with its business data, and then how to develop an infrastructure that can deliver the data to them," said Weghorst. "The end goal is to refine the fuel that is going to drive the analytics engines, because a sophisticated analytics group in an enterprise wants to analyze the data, not clean and sort it."

Also read