Though IT and its functions and responsibilities have changed over the years, there's one area that remains consistent: IT primarily focuses on major enterprise applications and on large machines—whether they are mainframes or super servers.
When IT deals with big data, the primary arena for it is, once again, large servers that are parallel processing in a Hadoop environment. Thankfully for the company at large, IT also focuses on reliability, security, governance, failover, and performance of data and apps—because if it didn't, there would be nobody else internally to do the job that is required. Within this environment, IT's job is most heavily focused on the structured transactions that come in daily from order, manufacturing, purchasing, service, and administrative systems that keep the enterprise running. In this environment, analytics, unstructured data and smaller servers in end user departments are still secondary.
"When we visit with large organizations, we often find common problems," said Dan Ortega, vice president of marketing for Blazent, which provides big data intelligence solutions. "The fundamental issue is that it is the business that generates the data but IT that delivers it."
Between these functions, data is cleaned, vetted and stored. Since IT performs all of these tasks, it is held responsible if data can't be readily accessed when the business wants it.
"When this happens, the end users in the business tend not to view IT as a strategic partner, and the users in the business adopt an attitude that they will only deal with IT when they absolutely have to," Ortega said.
While communications rifts between IT and end users aren't new, they get further complicated with big data because this data is coming into the enterprise from diverse and multiple sources that IT has never had to handle before—like social media, machines, websites, and mobile devices. Some big data vendors have also worked around IT when they discover that end business units have separate big data and analytics budgets that they can spend.
CEOs and other C-level executives should rightfully be concerned about the effectiveness, efficiency, and accuracy of their big data efforts in this environment—where end users are creating shadow IT organizations of their own and IT isn't making a fast enough transition into understanding what the business wants.
"Ensuring the value of big data and analytics—and that the data that the organization has is a single version of the truth, is an area where IT should really shine," said Ortega.
The question is: Can IT change its outlook and its processes to meet the big data challenges?
A best case scenario has IT business analysts reaching out on a daily basis to users in end business units. These hybrid specialists can glibly talk the lingo of the business as well as the bits and bytes of IT. They're complemented by leaders of end business units that understand the importance of high quality data, analytics—and a single version of the data that everyone in the enterprise is working from—maintained by IT. This isn't necessarily easy, but there are companies out there that are doing it, and doing it well.
"We have found that awareness and education on big data practices is the first step," said Ortega. "But with big data and analytics, you also have to be willing to make big changes in organizational processes, performance expectations and accountability. We have already seen major companies tackle these challenges, and they are seeing the results."
Mary E. Shacklett is president of Transworld Data, a technology research and market development firm. Prior to founding the company, Mary was Senior Vice President of Marketing and Technology at TCCU, Inc., a financial services firm; Vice President of Product Research and Software Development for Summit Information Systems, a computer software company; and Vice President of Strategic Planning and Technology at FSI International, a multinational manufacturing company in the semiconductor industry. Mary is a keynote speaker and has more than 1,000 articles, research studies, and technology publications in print.