Why VCs look to tech giants like Google and Facebook to see the future of data infrastructure

Look at what Google, Twitter, and Facebook are doing today, and you'll see where tech is going in the future. Right now, it's all about microservices.

Image: iStockphoto/HAKINMHAN

If you want to see the future, the best place to look might be Google, Facebook, or one of the small handful of companies whose data infrastructure is so advanced that mainstream enterprises won't catch up for a decade or more. If you wanted to get a good read on the importance of containers 10 years ago, for example, you wouldn't have been able to look up a Gartner profile of Docker. Instead, you would have had to get access to Google's engineering elite.

Most of us don't have that luxury, but a few select investors do it on a somewhat regular basis. Take Benchmark managing partner Peter Fenton, for example. I've known Fenton for years, because he was one of the earliest VCs to crack the code on open source's importance, investing first in JBoss and then Zimbra, both of which were acquired for outsized returns. Since that time he has invested in Docker, Hortonworks, New Relic, Yelp, Zendesk, and more.

Along the way, Fenton has kept close tabs on the uber geeks who make the world's most sophisticated infrastructure hum, which most recently led him to invest $10.5M in Buoyant's Series A round. I first spoke to Buoyant CEO William Morgan a few months ago about his time at Twitter conquering the fail whale and his subsequent inspiration to create a "service mesh" abstraction layer for this new era of microservices. Fenton, of course, spoke to him well before I did.

Seeing the future

The entire software ecosystem is undergoing a radical platform shift from monoliths to microservices and onto the cloud native stack. As Benchmark uncovered in its conversations with Morgan, companies on the leading edge of these trends quickly discover the major need for what we now call a service mesh, a dedicated layer for managing microservice communication. The service mesh enables companies running microservices in production to have the peace of mind that past architectures promised for monolithic apps.

SEE: How Twitter's Fail Whale could save your company (TechRepublic)

In this new complex world of cloud-native applications, you now have hundreds of microservices running on things like Docker and Kubernetes. Buoyant's open source technology addresses the rest of the puzzle--the runtime reliability of the microservices themselves. Discovering this need, however, required Benchmark to dig deep with the elite of Silicon Valley's engineering corps, as Morgan told me:

Companies like Twitter, Google, and Facebook are living in the future. Their internal software infrastructure is far ahead of anyone else. At Twitter, we had major performance issues for a long time, but now no one reads about service failures at Twitter anymore. We conquered that problem through a tremendous investment in infrastructure. And that's where Oliver and I draw inspiration for the service mesh. We knew that most companies, if they're moving onto the cloud, will end up needing a product like the Twitter engineering team built.

The best products are built to solve a need that somebody feels themselves acutely, Morgan continued. Having built out Twitter's microservices architecture, Morgan and Buoyant CTO Oliver Gould kicked off Buoyant as the best way to take these hard lessons they already learned to the rest of the world.

However, being open source, solving complex problems doesn't mean Buoyant will get paid for its efforts.

Opening up, getting paid

Benchmark, and Peter Fenton in particular, enjoys a reputation as a strong backer of open source companies. Some of those companies (like XenSource) got bought long before they had to solve the monetization riddle. When I asked Morgan about Buoyant's business model, he was clear that adoption comes first: "Our priorities are clear right now, which is focus purely on adoption and on making Linkerd the best open source product it can be, and the Linkerd community the most amazing community possible. But in the long run, I think we will have a lot of options."

SEE: Why microservices are about to have their "cloud" moment (TechRepublic)

Of course, there are examples of open source commercial success. Red Hat is on fire with a $15 billion market cap. Sourcefire was acquired for billions. And,as Morgan reminded me, "Other than cloud and SaaS, it's hard to find companies with market caps that cleared $10 billion recently that are not based on open source one way or another."

It's a good point, though, as Cloudera co-founder Mike Olson has stressed, "Pure-play open source companies never survive. That's a law of nature." It will be interesting to see how Buoyant fares. Undoubtedly it's on to a great technical solution to a problem enterprises are just beginning to realize they have. Whether that will turn into lots of money for its founders and Benchmark, however, remains to be seen. Benchmark's track record would suggest it's worth giving them the benefit of the doubt.

Also see