It's not hard to see who wins in this Internet of Things (IoT) and big data arms build up: the cloud vendors. According to IDC, IoT will require 750% more data center capacity within the next four years.
That's a lot of servers, networking gear, and more.
But that money won't go to the industry's default vendors. While some vendors may provide differentiated products that the Amazons of the world will continue to buy, most are toast. After all, Amazon, Microsoft, Google, and other cloud leaders generally aren't buying from IBM and traditional data center vendors, because their needs are too specialized and require that they build their own or heavily customize whitebox options.
So, while data-hungry trends like IoT and big data will power a new generation of data center investments, they're going to change the vendor landscape, perhaps forever.
IoT is a force of nature
While sensors in wearables, automobiles, and elsewhere grab the public attention, the reality is that data centers power the growing IoT market. Collecting data is, in many ways, the easy part. Crunching data to figure out what it means and how to react...? That's hard.
And that's a problem that requires an ever-growing mountain of hardware and software to tackle.
Hence, IDC sees data center capacity mushrooming by 750% by 2019 to handle the IoT data workload. As Rick Villars, Vice President, Datacenter and Cloud, IDC, puts it, "Given the number of devices connected and the amount of data generated, businesses must focus on their IoT service platform requirements at the level of the datacenter itself, not just the individual servers or storage devices."
The problem, however, is that few companies are in a position to do this effectively. For every Etsy that manages to pull out of the cloud to run Hadoop and other big data technologies in its own data centers, many more companies go the opposite direction.
As IDC notes, "The agility and scale required in IoT deployments will ensure that much of that datacenter capacity ends up residing in service provider [cloud] datacenters."
The clouds are ready
Not to worry, however, the cloud vendors are gearing up to handle the load.
Indeed, as The Wall Street Journal's Dan Gallagher highlights,
"The cloud-computing market will more than double by 2018 to $127 billion, IDC estimates. That is why companies such as Amazon, Google, and Microsoft are sparing no expense to stay in the game. They can't afford not to."
Even for those Etsys who hope to go it alone, it's hard to see how they can afford to do so.
The reasons, as Redmonk analyst Stephen O'Grady posits, stem from "relentless economies of scale":
"The economies of scale that larger players can bring to bear on the markets they target are, quite frankly, daunting. Their variable costs decrease due to their ability to purchase in larger quantities; their fixed costs are amortized over a higher volume customer base; their relative efficiency can increase as scale drives automation and improved processes; their ability to attract and retain talent increases in proportion to the difficulty of the technical challenges imposed; and so on."
As such, while success has tended to be measured by how many data centers a company builds to handle its scale, IoT may be moving too fast for all but a limited number of companies to handle. Think Facebook, Twitter, Google, and Amazon Web Services (AWS). You probably don't make the cut.
And even for those that think they can beat the odds, do you really want to? As AWS data science chief Matt Wood suggests, it may be too hard to run complicated data science queries against stagnant hardware assets. Elasticity, he argues, is essential to data science, and this is doubly true of IoT.
Your data center vendor is doomed
Regardless of who is building the data centers, one thing is clear: they're not building them out with the enterprise staples of the past.
This is one major reason that HP, IBM, and other data center vendors have been taking a beating for years. The cloud builders have tended to go direct to Taiwanese ODMs for hardware tailored to their particular needs. No self-respecting cloud vendor buys off-the-shelf hardware or software anymore: rather, the Amazons of the world see the engineering of all aspects of their data centers as essential, proprietary knowledge.
In the course of building its own data centers, Amazon has also significantly cut costs and improved performance. Why? Because it has shaved off anything and everything extraneous to its actual needs, as its data center guru James Hamilton says.
So, what does all this mean?
Well, it means that our cloud hardware vendors of tomorrow are actually cloud vendors who, in turn, build or buy their own custom hardware.
This means, in essence, that the bottom is about to drop out of the enterprise data center market, even as the cloud market rockets upward. The data center market, in short, is being completely hollowed out and reshaped: less HP and IBM, more AWS and Microsoft.
- A look at Amazon's world-class data-center ecosystem
- Private cloud's very public failure
- The more a company uses the cloud, the less likely they worry about downtime
- The cloud is a two-horse race between AWS and Microsoft
Matt is currently head of the developer ecosystem at Adobe. The views expressed are his own, not those of his employer.
Matt Asay is a veteran technology columnist who has written for CNET, ReadWrite, and other tech media. Asay has also held a variety of executive roles with leading mobile and big data software companies.