5 ways to demystify big data when talking to users

End users and IT have different vocabularies. Helping users understand the project is important in helping them get the most of their big data.

Big data user

Image: iStock/Prostock-Studio

While 87% of organizations acknowledged in a recent report that data was an asset, only 25% of respondents said they felt prepared to make use of data, only 37% thought their decisions were made better with data, and a full 74% felt completely overwhelmed with data. The Qlik-Accenture survey reported these findings in January.

The initial reaction to this is that so much data is now pouring into enterprises that they just can't keep up. However, I believe a more fundamental problem exists with our inability to communicate clearly within IT and with end business users.

SEE: Report: SMB's unprepared to tackle data privacy (TechRepublic Premium)

One obstacle that complicates communications within IT and between IT and end users is that engineering terms tend to dominate the IT discipline. For years, users spent time trying to understand what default meant. If you worked in finance, default meant that a loan was going bad. In fact, if you look the term default up in the Merriam-Webster online dictionary, definition number one is "failure to do something required by duty or law." It's only when you read through to definition 5b under default that you come to the definition that IT engineers use: "A selection automatically used by a program in the absence of a choice made by the user."

SEE: Your big data reports are not being used as they should: Here's how to change that  (TechRepublic)

This tendency to define IT terms in engineering lingo is confusing, and it's making the path to productive big data use harder. Here are five steps to simplify the process:

1. Explain what big data is to users

Big data is any information that doesn't come in the form of an electronic fixed record in a normal company system like accounts payable, purchasing, sales, manufacturing, etc. If you're talking about a photo, an image, a drawing, a hardcopy of a contract, a video, or a voice recording, that's big data. Because big data can't be processed through normal company transaction systems, it needs to be processed another way.

SEE: Are analytics dashboards dead? Here's how to bring them back to life (TechRepublic)

Most users can understand this, and many already do, but it doesn't hurt for IT to re-emphasize this so everyone knows what is meant by big data when IT talks about it.

2. Explain big data processing

It's up to IT to come up with the methods that process big data, but it's also important for end users to understand the basic steps in the process.

Users know from experience that when they work with IT to develop reports and online transactions for systems like accounting, they first sit down to design an application, and then IT programs it. Then, everyone tests the application until it works as expected, and the application is deployed in production.

SEE: Big data and DevOps: No longer separate silos, and that's a good thing (TechRepublic)

Big data application development is almost like that—but not quite.

3. Develop a data model

When IT sits down with end users to define how a business process works, and what type of information from systems are needed to make the process work, this is normally called requirements and business process definition. But in big data lingo, it's a data model.

SEE: Are you a big data laggard? Here's how to catch up (TechRepublic)

There are two parts to a data model. The first part of the process is where the users describe their business process on a whiteboard or other device. They also list the various kinds of data needed to execute the process. For example, if you need to locate a certain part that you need, you might need part number information, access to the suppliers that provide the part and the locations of the part.

Later, IT can develop an underlying data model that shows how various databases, computing resources, etc., will obtain this information for the user, but the user doesn't need to be involved with this highly technical stage of data model design. The user just needs to confirm that the business process and the information needs are complete.

4. Define the algorithm

The algorithm is the search criterion that the end user uses to query the data. For instance, if the end user wants to find all suppliers of a certain part that are not impacted by a category five hurricane that is raging, transactional data on parts suppliers can be combined with big data weather, with a report generated that indicates parts suppliers that are not in the affected zone.

SEE: Big data: How wide should your lens be? It depends on your use (TechRepublic)

If for some reason, the algorithm doesn't quite meet the business need, users and IT can reconvene so a better algorithm can be found. 

5. Get the data

The data generated from sources processed by the data model must be cleansed for duplicate, incomplete, and inaccurate data so that information returned to end users is of high quality. Data quality is confirmed by the end users in application testing. 

SEE: Big data success: Why desktop integration is key (TechRepublic)

Communication is key

The goal of big data as an IT initiative should be premised on communications that come often and are easy to understand. Take the time to explain a new process to end users in their own terms, and subtract the technical and engineering terms and find substitutes that can be expressed in plain English.

When used in tandem, these techniques can expedite project timelines and results and build trust and teamwork between IT and end users.

Also see