Big Data

Technology success looks like a data center... lots of them

As the pace of innovation accelerates, only those companies that can manage their applications at scale have a chance of sticking around.

Datacenter

Making the Fortune 500 list of the world's biggest companies used to be a big deal. Now, it's kind of a temporary honor, with the average Fortune 500 company life expectancy at 15 years and fading fast, compared to roughly 75 years a half-century ago.

Much of the reason for this acceleration of capitalist destruction is technology, and nowhere is the speed of innovation and expiration more frenetic than with web companies most responsible for undermining yesterday's brick-and-mortar businesses. It's therefore somewhat ironic that the way such companies are finding a quasi-permanence is by building brick-and-mortar data centers.

Hello, Fortune 500, I must be going

"Big" is getting smaller all the time — or, at least, less permanent.

Analyzing the Fortune 500 data since it launched in 1955, Professor Mark Perry uncovers a startling acceleration in creative destruction, the process by which economies reshape themselves by killing off old industries with new ones. As he finds, a mere 12.2% of the Fortune 500 companies in 1955 are still on the list 59 years later in 2014, and nearly 88% of the companies from 1955 have either gone bankrupt, merged, or still exist but have fallen from the top Fortune 500 companies.

Looked at another way, the average company lifespan on the S&P 500 Index is declining all the time, with a company getting bumped from the Index every other week:

Figure A

If we assume that this rate of creative destruction continues, we can expect to see the next 60 years riddled with over 9,000 companies appearing on and disappearing from Fortune's list, as one commentator highlights, compared to just 1,800 companies that have made the list since 1955.

While Perry notes that "The constant turnover in the Fortune 500 is a positive sign of the dynamism and innovation that characterizes a vibrant consumer-oriented market economy," he further stresses that this "dynamic turnover is speeding up in today's hyper-competitive global economy."

Technology is largely to blame — or to credit, depending on your view.

Software is eating the Fortune 500

Entrepreneur and investor Marc Andreessen is in the former camp. In a now famous op-ed piece for The Wall Street Journal, Andreessen declared: "More and more major businesses and industries are being run on software and delivered as online services — from movies to agriculture to national defense. Many of the winners are Silicon Valley-style entrepreneurial technology companies that are invading and overturning established industry structures."

Part of the problem for the old guard is how easy it is to become part of the new guard. Nearly a decade ago, then Excite co-founder Joe Kraus argued that open source and commodity hardware were lowering the bar to starting successful companies.

More recently, Andreessen Horowitz investor Sam Gerstenzang has opined that the level of technical competence necessary to build a billion-dollar software startup has also declined:

"[T]he barriers to becoming a code creator are falling fast. The same software foundation (open source software, development tools like Github, infrastructure as a service provided by the likes of Digital Ocean, and more) that allowed Whatsapp and Imgur to scale, means that experience and skill writing software become less important.

"An individual can now scale a web app to millions of users with Digital Ocean, Heroku and AWS (perhaps coordinated by Mesosphere). It no longer requires a sophisticated understanding of MySQL parameters to scale a database on Google App Engine, just as it no longer requires a knowledge of the CPU chip it's all chugging away on."

All of which makes it highly ironic that the one way web disruptors are achieving some semblance of permanence is by investing heavily in old-school, capital-intensive infrastructure.

Data centers as a lagging indicator of success

Asymco analyst and founder Horace Dediu makes this point in a recent post, showcasing just how much Google and its posse invest in data centers:

Figure B

These aren't small investments. When Apple planned for growth in its cloud operations, it dumped a billion dollars into a data center. Google, for its part, spent $7.3 billion in 2013 to build out data centers — over $21 billion total.

As Kraus and Gerstenzang argue, it truly is (relatively) cheap to start a company. It even can be cheap to create a billion-dollar company, as Instagram, Whatsapp and others have demonstrated.

However, to become a company that doesn't merely get bought for a billion dollars, but can regularly buy other companies for a billion dollars, you need a data center. Some, like Server Density's David Mytton, argue that the cost advantages of building on others' infrastructure goes out the window once you hit a certain scale.

Regardless of cost, to achieve any semblance of staying power, leading tech companies have all pursued the same course: bigger and better data centers. Only Netflix has successfully eschewed this path.

In other words, the new scale-out architecture for startups may have nothing to do with NoSQL or AWS. Instead, their strategy may consider scaling out of public cloud infrastructure into their own data centers.

About

Matt Asay is a veteran technology columnist who has written for CNET, ReadWrite, and other tech media. Asay has also held a variety of executive roles with leading mobile and big data software companies.

Editor's Picks