Open source history has a way of repeating itself... and not always in a good way.
This week's "big news" in big data is the banding together of EMC Pivotal, IBM, and Hortonworks in a grand Hadoop alliance called the Open Data Platform (ODP).
The one hitch, however, is that it's neither open, nor grand, nor destined to succeed.
ODP, much like UnitedLinux before it, is an effort by the also-ran vendors in a market to prop themselves up against more successful competitors. It's bad strategy, and bad for Hadoop. Fortunately, like United Linux, ODP will fail.
Winners win, losers partner
I don't blame Hortonworks and Pivotal for trying. Hortonworks is right to welcome lesser competitors to rally around its flag.
Pivotal, for its part, is simply trying to salvage some good from a bad situation. As Gartner analysts Merv Adrian and Nick Huedecker surmised, this could be "Pivotal's way to reduce its investment in a failing effort to build a proprietary way to capture a slice of this trend. Declare victory and retreat."
As stated, this smacks of the same desperation that led the also-rans of Linux (Caldera, TurboLinux, etc.) to team up against the dominant Red Hat. It failed miserably, and rightly so: it wasn't really designed to help customers, but rather the vendors that were losing to Red Hat.
Similarly, ODP is a sign of weakness for the sponsoring members. Analyst Curt Monash described it as "A face-saving way to admit that IBM's and Pivotal's insistence on having their own Hadoop distributions has been silly."
It clearly indicated that they need to partner since they can't win in the market, as classically captured in this Dilbert cartoon:
Because, let's face it: Cloudera is winning.
Cloudera, after all, generates more than $100 million a year in revenue, as it announced this week. That's impressive, but made doubly so when you consider that Cloudera's recurring subscription revenue grows at a torrid 100% pace each year. (MapR, for its part, has also grown strongly, climbing to 700 customers as of December 2014.)
Standards that aren't
So, a partnership was designed to let vendors feel good about themselves and tell the market that they're an industry standard. However, ODP is not, and enterprise buyers should remember what Cloudera's Mike Olson wrote in response to ODP: "code trumps cash."
Unlike the true Apache Hadoop standard, ODP "is clearly for vendors, by vendors," said Adrian and Huedecker. As they continue, "ODP positions Hortonworks as the Hadoop arms dealer for the other players. Basing an open data platform on a single vendor's packaging casts some doubt on 'open.'"
Or, as Olson noted,
"The Pivotal and Hortonworks alliance, notwithstanding the marketing, is antithetical to the open source model and the Apache way.
"While the ASF is open to vendors, the ODP isn't actually open at all. As a vendor-driven consortium, membership is only for enterprises with serious money — it ought to be called the 'Only Dollars Play' alliance. The price of entry is beyond the means of precisely the people who really drive the Hadoop standard — the individual engineers who participate in the Apache projects, and who actually bring the code."
Olson's assessment is spot on. Adrian and Huedecker concluded, "it's not clear who else" besides the paying members of ODP actually want it to exist. Customers certainly don't.
Enough Hadoop to go around?
None of this would matter if the Hadoop market were so big that would-be competitors never bumped into each other... but they do.
As Gartner's Merv Adrian highlighted, the Hadoop market is growing, but "adoption of Hadoop is not rising quite as dramatically as some might believe."
Yes, the surveys were less formal than a standard Gartner survey and, yes, they were only a quarter apart. Gartner's more formal surveys of Big Data adoption do show constant growth in the market, though "big data" doesn't necessarily equal "Hadoop."
Even so, these Gartner webinar participant surveys suggest that the market for Hadoop isn't exploding, leaving less room for multiple competitors to thrive.
Unfortunately for Pivotal, Hortonworks, IBM, and the other ODP member organizations, they just sent a very clear signal to the market that their ODP Hadoop distribution is a distant third to Cloudera and MapR. I doubt that's what they intended, but it's too late to retract this ill-advised attempt at competition through vendor-led and vendor-fed standardization.
- CIOs still don't care about Hadoop data security
- Spark promises to up-end Hadoop, but in a good way
- ZDNet: Oracle updates big data portfolio, aims to be 'visual face of Hadoop'
- ZDNet: Microsoft makes its Hadoop on Azure service available to Linux users
Matt is currently head of the developer ecosystem at Adobe. The views expressed are his own, not those of his employer.
Matt Asay is a veteran technology columnist who has written for CNET, ReadWrite, and other tech media. Asay has also held a variety of executive roles with leading mobile and big data software companies.