Big Data

Big Data lessons enterprises can learn from the Cloud

Cloud solutions for the supply chain, where the approach to data gathering and data sharing has been nothing short of revolutionary. Here's what those solutions can teach us about Big Data.

Cloud-based solutions are delivering enormous value to enterprises, and nowhere is this more apparent than in cloud's ability to deal with Big Data.

This is well illustrated in cloud solutions for the supply chain, where the approach to data gathering and data sharing has been nothing short of revolutionary.

In the past, companies wrestled with their supply chains of thousands of suppliers by certifying each supplier for access into enterprise ERP (enterprise resource planning) systems through an access method known as EDI (electronic data interchange). With EDI, there was repetitive testing of API (application programming interface) matchups between each supplier and the enterprise until all data transfers and security permissions between the two were achieved. At that point, the supplier was admitted into the enterprise's ERP system so business could be done. The process was laborious and repetitive. It was a real IT resource eater.

Then came cloud solutions for the supply chain that pre-qualified thousands of suppliers and producers around the world for entry into a secure network where data-even Big Data-was shared and secured throughout the network--instead of through sequential and repetitive supplier by supplier certifications. The cloud provider took responsibility for the shared data pool, which not only consisted of transactions but also of shipping and lading documents, order forms, specifications and schematics for products, and any other kind of document deemed vital to the process of manufacturing and shipping goods and services to market. The end result was a single data repository in the cloud that contained a mix of both big and small data that anyone admitted to the network could access at will if he had the right set of security permissions.

The idea of networking every goods producer and supplier into a central network with a single data repository that was a blend of structured, semi-structured and unstructured data was an approach few enterprises had ever considered-but they are seeing the results in their business processes. The process of onboarding a new supplier with the cloud network can be achieved in hours today. In the past days of EDI certification, this process could take months. Confusion in communications is considerably less on the cloud, because every participant is working with the same cloud-resident data repository. The cloud producer and supplier network also makes it possible for many different firms-even the tiny mom and pop operations-to exchange both standard and Big Data-and to do it securely.

How does the cloud accomplish all of this? By tagging each piece of Big Data with a name that can readily be accessed by anyone, and by giving each trading partner on the cloud network a set of business rules that allows each partner to assign security clearances and permissions down to the individual level within every other organization that it exchanges information with.

Enterprises have taken a meaningful step by engaging cloud solutions like this to deal with external business processes that their internal systems are not well positioned to address. But it's also time for enterprises to take a closer look at what cloud has accomplished, and to import some of these "lessons learned" into their own internal systems and how they deal with Big Data. These lessons include:

Go for a "single version of the truth"-Whether you are dealing with structured, semi-structured or unstructured data, the more you can consolidate information into a single set of facts, figures and graphics that everyone across the enterprise works with, the more you can avoid confusion that is caused by disparate systems spewing out disparate data. As you build your Big Data "data marts," there is a golden opportunity to normalize what goes into those marts and to start "doing it right." Use a permissions approach to Big Data security that individual business units can control-Moving security permissions administration to end business units (instead of using IT as the primary administrative agency) creates agility in communications. However, this should be thought out carefully in order to maintain enterprise security standards. It is advisable to get an outside security compliance expert involved on a consultative basis at the start of this process. Assume a more "democratic" approach towards your data, whether large or small-The central data repositories in the cloud work so well because they house both small and Big Data that are germane to a particular business function. Enterprise data marts should be constructed the same way.

About

Mary E. Shacklett is president of Transworld Data, a technology research and market development firm. Prior to founding the company, Mary was Senior Vice President of Marketing and Technology at TCCU, Inc., a financial services firm; Vice President o...

2 comments
kees lucassen
kees lucassen

There is certainly big value in sharing (big) data in the cloud. Besides data exchange the (old fashioned) EDI supports order handling and other transactional activities. How does your cloud solution support this?

dogknees
dogknees

Could you give us a specific example of what you are calling Big Data vs small data? It's not clear in the article what these items are and what distinguishes them. What quantities and types of information are we talking about?