The enterprise is awash in big data projects. That doesn't mean, however, that the CIO knows much about them.
In fact, though IT leaders are quick to affirm that big data is alive and well within their firewalls, they are equally fast to display complete ignorance of how this data is being stored and processed, as a new SnapLogic survey suggests.
While ignorance of IT won't keep a CIO's developers from digging into Hadoop, MongoDB, and other big data essentials, it does call into question their ongoing relevance to the business.
While developers increasingly run the enterprise asylum, it would be a mistake to believe that this necessarily means the CIO has lost her relevance. As I've pointed out, today's IT generally pairs savvy CIOs and developers (often employed by the line of business).
This is one reason IT leaders' ignorance of big data technology is disappointing.
There is a rising percentage of enterprises actively experimenting or running big data deployments, with Hadoop often at their heart, according to Gartner data:
Coincidentally, 40% of these same IT leaders expect Hadoop to "play a significant role in most companies analytical infrastructure" within the next four years.
All of this sounds nice until we dig a little deeper.
According to SnapLogic's survey, 78% of respondents weren't sure which Hadoop distribution their company was using or planned to use. Hardly anyone could name a vendor, with Cloudera (6%) leading the list, but barely (MapR tallied 5% and Hortonworks came in third with 3%).
When asked which Hadoop tools were already in use or were planned for use, most (81%) didn't know. The rest rattled off a few of the more likely candidates — MapReduce (9%), Hive (7%), Pig (5%), Spark (6%) — but the dearth of understanding how their enterprises were engaging with big data technologies suggests almost willful ignorance.
This brings us to the question of the biggest barriers to big data ROI. According to these IT leaders, the biggest inhibitor is "Limited skills and resources," accounting for 42% of the big data logjam. Given that the IT leaders can't even identify which technologies they're using, that ignorance seems to be trickling down from the top.
Small wonder, then, that Gartner finds that Hadoop deployments have been "slow to grow so far."
Taking the blinders off
It's time for enterprise IT leaders to quick talking about big data and immerse themselves in it. Their developers already "get it" and are actively embracing big data technologies like Spark, Cassandra, Hive, and more.
Now, it's time for IT leaders to do the same.
After all, as Redmonk analyst Donnie Berkholz notes, even sysadmins need to grok data now:
"As we scale applications and start thinking of servers as cattle rather than pets, coping with the vast amounts of data they generate will require increasingly advanced approaches. That means over time, monitoring will require the integration of statistics and machine learning in a way that's incredibly rare today, on both the tools and people sides of the equation."
What Berkholz says of sysadmins is equally true of the CIOs that manage them: the CIO can no longer afford to be "the last to know" in a world that depends more and more on data. It's time for the CIO to get to know her data... and the technologies used to tame it.
- 10 big data projects that could help save the planet
- How big data is going to help feed nine billion people by 2050
- 10 things you shouldn't expect big data to do
- Has the CIO lost big data?
Matt is currently head of the developer ecosystem at Adobe. The views expressed are his own, not those of his employer.
Matt Asay is a veteran technology columnist who has written for CNET, ReadWrite, and other tech media. Asay has also held a variety of executive roles with leading mobile and big data software companies.