AWS has quietly and steadily been improving with open source. Sure, Corey Quinn might have been right when he said that, in the past, AWS “consistently and, in my opinion, incorrectly [tried] to shape a narrative where they’re contributing to the open-source ecosystem at a level that’s on par with its big tech company peers.” Perhaps this used to be true; it’s arguably not now.
Instead, AWS has discovered that one key to delivering on its first Leadership Principle (Customer Obsession) is to show up and contribute in meaningful ways to the open-source projects its customers care about. Apache Kafka is just the latest example of this.
A switch flipped
Divij Vaidya’s tweet surprised me. Here’s an AWS engineer that has become one of the most active contributors to Apache Kafka. Granted, that’s just over the last month. Looking at the Kafka PMC committee, it’s filled with individuals from Confluent and other companies that have contributed to Kafka for years, not months.
Even so, it’s telling that Vaidya, upon joining the Amazon Managed Service for Kafka (MSK) team a few months back, immediately started to contribute code to Kafka and is hiring a team that will be dedicated to contributing code to Kafka.
This is exactly what critics have been saying AWS doesn’t do. And, for years, they were mostly correct.
SEE: 40+ open source and Linux terms you need to know (TechRepublic Premium)
AWS was, and is, far more concerned with taking care of customers than being popular with open-source audiences. So, the company has focused on being “the best place for customers to build and run open-source software in the cloud.”
Historically, that tended to not involve or require contributing to the open-source projects it kept building managed services around. Many felt that was a mistake—that a company so dependent on open source for its business was putting its supply chain at risk by not sustaining the projects upon which it depended. There were plenty of good reasons for all this, but there were also more compelling reasons to change and do more.
And so it has; though, generally not with trumpets and fanfare.
PostgreSQL contributor (and sometime AWS open-source critic) Paul Ramsey has noticed. As he told me recently, it “[f]eels like a switch flipped at AWS a year or two ago. The strategic value of being a real stakeholder in the software they spin is now recognized as being worth the dollars spent to make it happen.”
Taking care of customers
Years ago Tim Bray, then an engineering executive at AWS, argued that operating open source software was at least as important as building it.
“The qualities that make people great at carving high-value software out of nothingness aren’t necessarily the ones that make them good at operations,” Bray added.
AWS might not be contributing much code, the implication ran, but making that code easy for customers to actually use was a big contribution in itself. All true.
SEE: Master Linux and Docker before the next Linux adoption boom (TechRepublic Academy)
But, what seems to be happening at AWS, if quietly and usually behind the scenes, is a shift toward AWS service teams taking greater ownership in the open-source projects they operationalize for customers. This allows them to more effectively deliver results because they can help shape the roadmap for customers, and it ensures AWS customers get the full open-source experience, rather than a forked repo with patches that pile up as technical debt.
Vaidya and the MSK team is an example along with Madelyn Olson, an engineer with AWS’s ElastiCache team and one of five core maintainers for Redis. And then there are the AWS employees contributing to Kubernetes, etcd and more.
No, AWS is still not the primary contributor to most of these. Not yet. Google, Microsoft and Red Hat tend to top many of the charts, to Quinn’s point above. This also isn’t somehow morally wrong, as Quinn also argued: “Amazon (and any company) is there to make money, not be your friend.”
But slowly and surely, AWS product teams are discovering that a key element of obsessing over customers is taking care of the open-source projects upon which those customers depend. In other words, part of the “undifferentiated heavy lifting” that AWS takes on for customers needs to be stewardship for the open-source projects those same customers demand.
Disclosure: I work for MongoDB, but the views expressed herein are mine.