I have been amazed at the explosive growth that we are seeing on top of relational databases in terms of the layers that developers add to them. For at least the last 10 years, it has been the norm for applications to use direct database calls, possibly parameterized, to access the database. Advanced developers might store the application's SQL statements externally from the source code, perhaps in a text file or even a configuration table in the database, to give some separation between the two. Then we saw the rise of the data layers; then the use of Web services to loosely couple the data layers; and now many developers add an object-role modeling (ORM) system or similar abstraction on top of the database just to make the data layer.
For at least the last 20 years, developers have been trying to figure out how to make the relational database work better for our needs. When do we start trying something entirely different?
I understand the history associated with relational database management systems (RDBMS); these systems are fast, ACID-compliant, standardized (for the most part), and well understood. A developer can code an application against a vendor's RDBMS and switch to another one without major heartache in most cases. Indeed, there are piles of systems out there that abstract away the vendor differences to the point where the developer is not aware of them.
Another consideration is the RDBMS alternatives (such as object-oriented database management system or OODBMS) have had a lot of issues in the past. The alternatives were slow, completely proprietary, vendor specific, lacked ACID-compliance, and were a total mystery to most developers. This is not a formula for success. Folks who experimented with OODBMSs tended to get burned. The closest that OODBMSs got to mainstream was Smalltalk, which effectively was an OODBMS, amongst other things.
My frustration level with the RDBMS model has been steadily increasing; RDBMS simply does not represent the business model of most applications very well. The industry likes to call this "impedance mismatch," but I call it a pain in my neck.
Why am I so annoyed with RDBMS?
The applications that I work on that have data storage requirements (I'm tasked with programming and supporting these apps) have lately had very irregular data needs. Here's a good example: Take a look at the database for any application that allows users to create their own types of content (not just their own content). (SharePoint and Team Foundation Server are good instances of this, and they illustrate the pattern well.) At the database level, they are architected as best as they can be, given their requirements. All the same, you see a real mess in the storage systems; there are piles of columns whose purpose to the application is not determined by the database structure itself but rather a cross reference of each row to a "content type" table somewhere. This means that a developer looking to extend, maintain, or otherwise use the database is forced to recreate significant amounts of logic. And woe to the system administrator who is trying to solve a problem with the application.
Developer tool creators started to really notice these issues in the last few years, and they have released a flurry of products and components designed to provide some relief. Most of these ideas build on the existing RDBMS infrastructure, which makes sense. Customers have an existing investment in those systems; the systems are understood, tested, and known entities; and the systems are already in place.
In addition, there is the bizarrely illogical (yet amusingly "human") tendency for people to chase "sunk costs" in an effort to recover them (also known as "throwing good money after bad"). In this case, programmers would rather add layers of complexity to their applications (often at a significant cost in time and sometimes tools), which makes maintenance and debugging more difficult, to compensate for the "impedance mismatch" between their applications and their databases. Systems in the ADO.NET Entities Framework Hibernate and so on are good examples of what I mean. I've looked at the ADO.NET Entities Framework, and I have no plan of touching it unless my application's database diagram can cover my kitchen table; it's just too complex and heavy to bother with unless my application is already complex and heavy. Some systems I've seen (such as OutSystems' Agile Platform) hide the complexity very well, but it is still there.
Meanwhile, the OODBMS market has made a lot of strides. At the suggestion of a friend, I checked out db4O. While I haven't used it yet, my friend (whose opinion I value and trust) said some really nice things about it. I plan to take a look at db40 in the near future. Many OODBMSs are now ACID-compliant. Looking through the documentation and examples of a few of them, not only are they less work to interface with than an RDBMS, they are less effort than dealing with the ORM tools and similar systems. And while the raw data access speeds (particularly reads) are probably slower than an RDBMS, once you account for the various data access layers in a typical application plus the cost of converting database data to language-native objects, the OODBMS will be faster. (Again, I haven't had the chance to use an OODBMS yet.)
I plan to provide more coverage about this topic in the future, though I suggest that you start to investigate the market. Most developers I know recognize that we've been bandaging this issue for a long time, but they do not realize that there are options. There are choices, and I think it's time that we started exercising them. Our applications are far too complex to keep trying to force them into the inherent limits of the RDBMS model.
J.JaDisclosure of Justin's industry affiliations: Justin James has a working arrangement with Microsoft to write an article for MSDN Magazine. He also has a contract with Spiceworks to write product buying guides.
---------------------------------------------------------------------------------------Get weekly development tips in your inbox Keep your developer skills sharp by signing up for TechRepublic's free Web Developer newsletter, delivered each Tuesday. Automatically subscribe today!
Justin James is the Lead Architect for Conigent.