Benchmarks are usually the way vendors lie to us. But there are plenty of ways. Sometimes we call it “marketing,” but really it’s just fraud, as Dilbert concisely captures:

Given the importance of technology, what with software eating the world, it’s never been more important to make smart technology decisions. Unfortunately, we also live in a world that has become ever smarter about using big data to drive us to think and do things that may not be best for us or our companies.

This is why open source has never been more important. By making truth just a download away, open source dramatically changes how technology decisions are made.

Of course it doesn’t scale

I left MongoDB nearly a year ago, but the scars are still fresh. Those scars came from the constant drumbeat of “MongoDB doesn’t scale” messaging that its competitors trumpeted. Over time, that message sunk in:

The only downside to the message is that it’s not true.

China’s largest car-sharing service, serving over 100 million users, scales happily with MongoDB. Nor are they alone. I helped to write a page while still at MongoDB that walks through a range of companies that have over 100 billion documents running millions of operations per second in 1,000-plus node clusters.

That’s serious scale. But it doesn’t stop the constant benchmark fiction from competitors.

MongoDB isn’t exempt from such un-fan fiction. While I was there, I often would talk with customers about Cassandra scales that requires a PhD to understand. It was my own little myth that comforted me and (no surprise) happened to play to MongoDB’s strengths of being easy and intuitive to use.

In other words, we all do this. We all paint our competitors in the blackest hues while we array ourselves in brilliant light, as Microsoft does in its Azure advertisements:

Separating truth from fiction

The reality at MongoDB was that while it was always possible to scale MongoDB, it wasn’t always easy. That has changed after recent releases, but a more truthful slam of MongoDB would have stated that.

But marketing–or benchmarketing, as Facebook’s database guru Mark Callaghan calls it–often isn’t concerned with truth. It’s concerned with sales.

This is why we need open source more than ever, particularly in the underlying data infrastructure that undergirds the modern enterprise. You don’t need to take my word for it. You can download it. You can trust the code and your own experience.

While the cardinal virtue of open source may be that anyone is free to modify/fork the code, the reality is that few actually do. But the first virtue–free and unfettered access to code–is powerfully important, too, and it’s the right that most people associate with open source.

Public cloud computing perfects open source in some ways, as I’ve argued before–but with cloud, you’re always subject to someone else’s rules, even if it’s merely a matter of where the code runs. However, in open source, everything is free… including the freedom to be underwhelmed by code, or confused by it, or impressed.

We live in a world that increasingly delivers software as a service, whether it’s infrastructure or applications. This is a positive trend that makes running complex software less complex, thereby democratizing access to ever-increasing quantities of great software.

But in this rush to cloud, let’s not forget the truth–real truth–that software is not a cloud thing. It’s an open source thing–and it’s downloaded (not bought) by developers, the least credulous crowd in the enterprise, and therefore perhaps the best positioned to make technology decisions.