In tech, everyone wants to be a platform, the center of an ever-growing universe of applications. Achieving that dream, however, is a combination of dumb luck and smart strategy.
In the case of Redis, its decision to make the popular database extensible through Redis Modules is an incredibly smart strategy and a “big deal,” as 451 Research analyst Donnie Berkholz said. Now, Redis just needs some dumb luck.
As data volumes boom and data variety proliferates, NoSQL databases like MongoDB, Cassandra, and Redis have challenged the established relational database hierarchy, as DB-Engines database popularity index shows:
Despite its popularity, Redis has a somewhat narrow range of applicability. It’s fantastic as a message queue, for session caching or leaderboards, and other top use cases. But, a developer can’t use it for the hardier stuff that MongoDB (personalization, single view of the customer, content management, etc.) or Cassandra (fraud detection, recommendation engines, product catalogues, etc.) nail with aplomb.
SEE NoSQL is a “complete game changer,” declares database expert (TechRepublic)
Up until now, that seemed to be baked into the Redis DNA. “Single-core awesomeness,” as Crowdstrike engineer Jeff Jirsa told me, and nothing more. As an open source project, Redis was open to outside code contributions, but not in a way that fundamentally changed the scope of the database. There was, in other words, no real “architecture of participation,” as Tim O’Reilly called it, which would facilitate deep participation in the future of Redis.
That is, until Redis Modules was announced.
The promise and peril of extensibility
As I stated, most technologies at some level aspire to be a platform. Aspiration and ability, however, are often worlds apart. Such was the case with Redis.
As Redis creator Salvatore Sanfilippo noted, modularizing something as complicated as a database is fraught with peril: “Modules can be the most interesting feature of a system and the most problematic one at the same time: API incompatibilities between versions, low quality modules crashing the system, a lack of identity of a system that is extendible are possible problems.”
Part of the problem, it turns out, is that Redis is as much a programming language as a database. This matters because, he continued, “To extend Redis properly, the module needs to have access to the internal API of the system.” But what happens if that internal API changes, which it necessarily will, as Redis evolves?
Directly exporting the Redis core functions to modules creates huge problems: The module starts to depend on the internal details of Redis. If the Redis core evolves, the module needs to be rewritten. This creates either a fragile modules ecosystem or stops the evolution of the Redis core.
As such, Sanfilippo avoided diving into loadable modules for years. And Redis, despite growing in popularity, has remained a distant follower to the more popular (and more generally applicable) MongoDB and Cassandra.
SEE NoSQL keeps rising, but relational databases still dominate big data (TechRepublic)
With the introduction of Redis Modules, however, Redis just gave developers power to go “faster” than the project itself, in using Redis to model their problems,” Sanfilippo warrants. The promise is bigger than this, though. Redis Modules also gives developers a way to embrace and extend the hitherto simple database they love.
Where to from here?
Already Redis has added modules that turn Redis into an image processing engine and a full-text search engine. This is just the start.
Part of “architecting participation” is not only making it technically feasible to extend a platform, but also making it somewhat simple to do so. As O’Reilly’s Rachel Roumeliotis highlights, “Because Redis uses commands and arguments like a programming language, extending it through modules written in C is familiar and natural.” That means you don’t have to be a rocket scientist to help build Redis’ future.
Maybe this doesn’t mean that Redis will magically be transformed into the next MongoDB or Cassandra. Arguably, that’s not the point.
Yet, Redis Modules should help to make Redis into a far richer, more broadly applicable database, thereby making the popular database even more popular.